Proxmox VE Shared Storage: LVM Thin, MeshXFS, CetraFS

Proxmox clustering storages has been a very changing topic in 2026, with several new developments and improvements finally bringing more possibilities to the table. However, when looking at the bigger picture, there are still some gaps compared to what we have been used to in the VMware world. One of the golden rules in VMware environments has always been VMFS, providing a shared clustered filesystem that allows multiple ESXi hosts to access the same storage and run virtual machines on top of it. In Proxmox VE, we still lack a similarly straightforward solution that covers all the typical use cases.

Of course, this does not mean that we have no alternatives. Linux has offered clustered filesystems such as GFS2 and OCFS2 for years. Both are established technologies, but they are not really the first choice for most Proxmox environments with virtualization based workloads. We could also look at distributed filesystems like Lustre or BeeGFS, but these are often far too complex for smaller clusters. They introduce additional components, operational overhead, and requirements that can quickly become excessive when all we really want is shared storage for virtual machines.

Another option is Ceph, which is well integrated into Proxmox VE and provides distributed storage with features such as replication and self-healing. However, Ceph also requires a certain amount of infrastructure, resources, and operational knowledge. Not every environment needs a distributed storage system, and many administrators already have dedicated SAN infrastructure available that they want to continue using.

For a long time, the typical approach for shared block storage has been to use technologies such as iSCSI, Fibre Channel (FC), or NVMe over Fabrics (NVMe-oF), with a thick-provisioned LVM on top. This is a well-established setup and works reliably for many environments. However, compared to what VMware administrators are used to, there have been some limitations, especially when it comes to snapshots and thin provisioning. With Proxmox VE 9, things started to change. The introduction of snapshot support for shared LVM storage through snapshot volume chains was an important step forward. It finally became possible to use snapshots with shared, thick-provisioned LVM without having to switch to a completely different storage backend. Still, one important piece was missing: thin provisioning at the LVM layer.

This is where I wanted to take things a step further. I have developed a shared LVM thin-provisioning storage plugin for Proxmox VE, bringing thin provisioning to shared block storage setups. The idea is to make better use of existing SAN infrastructure while keeping the familiar LVM approach and avoiding the need to deploy an entirely new distributed storage stack. Combined with the recent improvements in Proxmox VE 9, this opens up new possibilities for environments that need shared storage, snapshots, and more efficient capacity usage.

Looking back at 2026, the Proxmox storage ecosystem is clearly moving forward. There is still no universal replacement for VMFS that fits every scenario, but we now have more options than before. For me, the next step is making these solutions easier to deploy, maintain, and integrate into everyday Proxmox clusters. Not every environment needs Ceph, and not every administrator wants to introduce a complex clustered filesystem just to get the features expected from a modern virtualization platform. So, let's have a deeper look at the current market situation where we will focus at the following solutions:

• Shared LVM Thin
• rbdmirror
• MeshXFS
• CetraFS
• Linbit SDS

Shared LVM Thin

Shared block storage has been a recurring topic when discussing Proxmox VE as an alternative to VMware. Many enterprise environments already rely on existing storage infrastructures based on iSCSI, Fibre Channel (FC), or NVMe over Fabrics (NVMe-oF). Replacing the virtualization platform should not necessarily mean replacing the entire storage architecture as well.

For a long time, administrators had to choose between shared LVM without thin provisioning and local LVM-Thin with support for thin provisioning, snapshots, and linked clones. Neither option covered all the requirements of environments that wanted to use their existing shared SAN infrastructure. Thick provisioning can waste a significant amount of capacity because the full configured size of a virtual disk must be allocated upfront, even when the guest only uses a fraction of it.

With Proxmox VE 9 and the introduction of snapshot volume chains, snapshot support for shared, thick-provisioned LVM became available. This was an important improvement, but thin provisioning remained a missing piece for many deployments.

This is where my Shared LVM Thin storage plugin for Proxmox comes into play. Instead of trying to activate a single LVM thin pool simultaneously on multiple cluster nodes, the plugin creates a dedicated thin pool for each guest inside the shared LVM volume group. Since a virtual machine or container normally runs on only one node at a time, its thin pool can be activated exclusively on that node. When the guest migrates, the pool is deactivated on the source node before being activated on the destination node.

This approach allows virtual disks to use thin provisioning while keeping the underlying block storage accessible across the cluster. The plugin also integrates with Proxmox VE's storage operations, including snapshots, rollbacks, disk resizing, templates, linked clones, and other common disk lifecycle operations. Thin pools can grow as required, with cluster-wide locking used to protect operations that modify shared LVM metadata.

Another important aspect is usability. The plugin is implemented as a custom Proxmox VE storage backend and can be managed through the command line and API. An additional web interface integration makes it possible to configure the storage directly through the familiar Datacenter and Storage menus.

The plugin is distributed as a Debian package, so it can be installed using the standard package management tools instead of requiring administrators to manually copy files and configure the integration. The Shared LVM Thin plugin is also available through the gyptazy Debian repository, making installation and future updates easier.

The main goal is to provide another option for environments that want to keep their existing SAN infrastructure while benefiting from thin provisioning and snapshots. This does not remove the need to monitor physical capacity, validate multipath behavior, and test migration and recovery scenarios, but it makes shared block storage more flexible for Proxmox VE clusters.

rbdmirror (for geo-redundency)

This solutions hits a bit different and is not directly but still worth to mention where Ceph RBD Mirroring is another interesting option for Proxmox VE environments, particularly when it comes to disaster recovery (DR) and geo-redundant virtualization. Unlike the shared storage approaches discussed above, RBD Mirroring replicates virtual machine disks asynchronously between two independent Ceph clusters. This allows administrators to protect workloads against a complete datacenter failure without having to stretch a single Ceph cluster across multiple locations.

At our Proxday 2026, my colleague Carsten Feuls from credativ GmbH recently gave a talk about geo-redundant virtualization with Proxmox VE, Ceph, and RBD Mirroring. His presentation covered the technical background, replication modes, network requirements, and practical considerations for implementing disaster recovery across multiple locations. For further details, check out Carsten's Proxday 2026 slides.

One of the main advantages of this approach is that Ceph and RBD are already well integrated into Proxmox VE. Ceph provides distributed block storage, while RBD Mirroring replicates virtual disk images to a remote cluster. This makes it possible to maintain a secondary copy of virtual machine disks at another location and recover workloads if the primary site becomes unavailable. Since replication is asynchronous, the achievable recovery point objective (RPO) depends on the replication mode, workload, network capacity, and replication lag.

Ceph offers two different approaches to RBD Mirroring: journal-based and snapshot-based replication. Journal-based mirroring continuously transfers recorded write operations to the remote cluster, providing low replication latency at the cost of additional write activity on the source storage. Snapshot-based mirroring transfers changes between mirror snapshots at configured intervals. It can reduce the continuous write overhead and work well across higher-latency WAN connections, although the recovery point depends on the snapshot schedule and how quickly changes can be replicated.

However, RBD Mirroring does not replicate the complete Proxmox VE configuration. It mirrors the contents of virtual disk images, but VM and container configuration files, firewall rules, and other cluster-specific settings must be handled separately. Proxmox VE's internal cluster filesystem, pmxcfs, does not automatically synchronize these configurations between independent Proxmox clusters. A complete disaster recovery solution therefore needs additional automation to synchronize the relevant configuration and ensure that workloads can be started correctly at the recovery site.

Failover and failback also require careful planning. Before activating a replicated disk at the recovery site, administrators must ensure that the original workload is no longer writing to the primary copy. Otherwise, both sites could attempt to operate on different versions of the same disk, resulting in a split-brain situation. Network connectivity, replication health, image promotion, VM configuration, and recovery procedures should all be monitored and tested regularly.

One particularly interesting aspect of RBD Mirroring is that it allows organizations to build disaster recovery environments without introducing an entirely separate storage platform. Existing Proxmox VE and Ceph deployments can be extended with a second cluster, making the approach attractive for organizations that want geographic redundancy while retaining their existing technology stack. The architecture also allows the two sites to be designed independently, without requiring a single stretched Ceph cluster across a high-latency connection.

And here it comes, my mates Alexander Wirt (formorer) and Carsten Feuls (both from credativ Gmbh) are currently developing a dedicated Proxmox VE storage plugin integration for RBD Mirroring. The goal is to bring the management and control of the replication workflow directly into the Proxmox VE web interface, rather than requiring administrators to perform every operation manually through the command line. This integration aims to make configuring replication, managing mirrored images, and handling day-to-day operations more convenient for Proxmox VE administrators.

It is important to distinguish this approach from shared SAN storage and clustered filesystems such as MeshXFS or CetraFS. RBD Mirroring focuses on replicating virtual disk data between independent Ceph clusters for disaster recovery; it does not provide concurrent filesystem access to the same block device across multiple hosts. For environments that already use Ceph and need a robust recovery strategy for an entire site, however, it is a valuable option. For more information, take a look at the Proxday 2026 presentation by Carsten Feuls and the official Ceph RBD Mirroring documentation.

MeshXFS

MeshXFS is a new open-source, POSIX-compatible clustered filesystem built on top of XFS. It aims to solve a common challenge in virtualization environments: allowing multiple nodes to access the same shared block device concurrently in read-write mode. In one of my recent blog posts, I also covered more details and the installation steps of MeshXFS on Proxmox VE 9 clusters.

In the VMware world, VMFS has been the established solution for this use case for years. On Linux, there are alternatives, but there is still no single, straightforward solution that fits every Proxmox VE environment. GFS2 and OCFS2 have been around for a long time, but they are not necessarily the first choice for new deployments. Distributed filesystems such as BeeGFS and Lustre offer powerful capabilities, but their architecture and operational complexity can be excessive for smaller clusters.

For many Proxmox VE environments, shared LVM has therefore been the typical alternative when working with external block storage. However, this approach has its own limitations, which is why I also developed the Shared LVM Thin plugin. MeshXFS explores a different direction: instead of allocating individual virtual disks through LVM, it provides a clustered filesystem that multiple hosts can mount on the same shared block device.

The project builds on the upstream Linux kernel XFS implementation and adds the clustering mechanisms required for concurrent access. These include distributed locking, fencing, coordination, and journal recovery. MeshXFS supports shared iSCSI SAN storage and dual-primary DRBD configurations, with network-based locking or on-disk coordination using SCSI COMPARE AND WRITE.

According to the project documentation, shared iSCSI configurations have been validated with 2, 4, 8, and 16 nodes. This makes the project particularly interesting for environments that already operate shared block storage and want to explore concurrent filesystem access without introducing a complete distributed storage system.

One particularly interesting aspect of MeshXFS is its development approach. The clustering, coordination, distributed locking, fencing, and recovery code was fully vibe-coded using AI agents, while the underlying XFS implementation originates from the upstream Linux kernel. This makes MeshXFS an interesting example of how AI-assisted development can be used to build infrastructure software and explore solutions to long-standing problems.

MeshXFS also provides DKMS-based Debian packages for supported systems, including Proxmox VE 9, Debian 12 and 13, RHEL 9, and Ubuntu 24.04 and 26.04. The DKMS approach allows the kernel module to be built for the installed kernel, provided the matching headers and supported kernel version are available. An additional Debian package provides a dedicated Proxmox VE storage plugin, making the filesystem easier to integrate into the web interface.

This means that MeshXFS can be installed directly using the shipped Debian packages rather than requiring administrators to compile and install every component manually. Once the packages are installed on the participating nodes, the shared device can be initialized, mounted, and registered as shared storage in Proxmox VE.

However, MeshXFS is still a young project. Its performance, recovery behavior, fencing mechanisms, and suitability for production workloads need to be evaluated carefully. As with any clustered filesystem, correct storage configuration and reliable fencing are essential. For now, it is an interesting candidate for lab environments, testing, and further development. For more information, visit MeshXFS on GitHub.

CetraFS

CetraFS is another approach to bringing shared-disk clustered filesystem capabilities to Linux and Proxmox VE. Like MeshXFS, it aims to allow multiple nodes to mount the same shared block device concurrently, with virtual machine disk images stored directly on the filesystem. The overall idea is similar to what administrators have been using with VMFS in VMware environments for years.

However, CetraFS has a different technical background and development approach. Rather than extending an existing filesystem such as XFS, CetraFS is developed as an independent filesystem with its own on-disk format, kernel module, and userspace tools. Its design focuses on shared block storage, coordinated allocation, crash consistency, and the requirements of virtualization workloads.

One of the central components is its use of atomic SCSI COMPARE AND WRITE operations for coordination between nodes. This allows competing operations to be coordinated through the storage device itself, rather than depending on a separate network-based locking service. The approach is conceptually similar to the atomic storage operations used by VMFS for coordinating access to shared storage.

CetraFS includes per-node write-ahead journals, checksums for on-disk structures, sparse-file support, and tools for filesystem checking and maintenance. It also uses per-node allocation groups to reduce contention when multiple nodes allocate space concurrently. For virtual machine workloads, it aims to support shared disk images, online migration workflows, and concurrent access where the appropriate locking and migration options are configured.

Unlike MeshXFS, CetraFS does not build on the upstream XFS filesystem implementation. It is an independent implementation with its own filesystem structures and recovery logic. This means the two projects explore different technical approaches to the same broader problem: making shared block storage usable as a concurrent filesystem in Linux virtualization clusters.

I have also been involved in the CetraFS project, particularly in bringing it closer to Proxmox VE users. One part of this work has been the Proxmox VE storage plugin and web interface integration. The aim is to make CetraFS manageable through the familiar Proxmox VE interface, instead of requiring administrators to perform every step manually through the command line.

The project provides Debian packages for the filesystem's DKMS kernel module and userspace tools. The packages can be installed through the gyptazy Debian repository, which also makes updates and installation on multiple cluster nodes more convenient. Additional packages provide the Proxmox VE storage plugin and its web interface integration. This allows administrators to install the components using standard Debian package management and then configure the storage within Proxmox VE. I recently contributed to the project a native Proxmox Web UI integration for CetraFS via PR#1.

There are important limitations to consider. CetraFS is a young, pre-1.0 project and is not yet as production-mature as established filesystems such as VMFS or ext4. Concurrent mounting also depends on suitable storage hardware supporting atomic COMPARE AND WRITE operations. Production deployments require appropriate host fencing, and multipath configurations must be checked carefully to ensure that path failures cannot lead to unsafe recovery behavior.

As with MeshXFS, CetraFS should be tested thoroughly before being used for critical production workloads. Crash recovery, concurrent access, migration, hardware failures, and fencing need to be validated in the target environment. Independent backups remain essential. For more information, visit CetraFS but Massimo Cetra is also quickly responding to geedback via Reddit.

Linbit SDS

LINBIT SDS takes a different approach to shared storage for Proxmox VE. Instead of providing a clustered filesystem on top of a shared SAN, it combines DRBD and LINSTOR to build a software-defined storage solution based on replicated block devices. This makes it particularly interesting for environments that want to run highly available virtual machines without depending on a centralized storage array. Although LINBIT has been an established provider of storage solutions for Proxmox VE for many years, its solutions remain highly relevant and deserve to be mentioned when discussing the current shared storage landscape.

At the heart of the solution is DRBD, a Linux kernel driver that replicates block devices between nodes. Depending on the configuration, data can be synchronously replicated to multiple nodes, ensuring that writes are replicated before they are acknowledged. On top of that, LINSTOR provides the management and orchestration layer, handling storage provisioning, resource placement, and replica management across the cluster.

One of the main advantages of LINBIT SDS is its integration with Proxmox VE. The LINSTOR Proxmox storage plugin allows virtual machine disks and container volumes to be provisioned and managed directly through the familiar Proxmox interface. Replicas can be distributed across multiple nodes, and the number of replicas can be configured according to the requirements of the environment. This also enables fast live migrations because the destination node already has access to a replicated copy of the virtual disk, avoiding the need to transfer the entire disk during migration.

Unlike Shared LVM Thin, MeshXFS, or CetraFS, LINBIT SDS does not require a shared SAN or a clustered filesystem mounted concurrently by multiple hosts. Instead, each replica is maintained on its respective node, with DRBD handling replication at the block layer. This makes it a particularly interesting option for hyperconverged infrastructures, where compute and storage resources run on the same servers. LINBIT also supports disaggregated storage architectures, allowing storage and compute capacity to be scaled independently.

Another important aspect is high availability. Combined with Proxmox VE's HA functionality and an appropriately configured LINSTOR and DRBD deployment, virtual machines can be restarted on surviving nodes when a host fails. However, this requires a correctly designed cluster, sufficient surviving replicas, appropriate quorum and fencing configuration, and tested recovery procedures. Replication alone does not automatically guarantee application availability or protect against every failure scenario.

Installation is also relatively straightforward compared to building a storage solution entirely from individual components. LINBIT provides the linstor-proxmox plugin and packages for the underlying DRBD and LINSTOR components through its software repositories. Once the required components are installed and the LINSTOR cluster is configured, administrators can register the storage backend in Proxmox VE and provision virtual disks through the web interface. The solution can be managed using the LINSTOR command-line tools and API, with additional management options available through LINBIT's software offerings.

However, LINBIT SDS solves a somewhat different problem from the clustered filesystems discussed above. It provides replicated block storage rather than a general-purpose filesystem that multiple nodes mount concurrently. Its architecture introduces replication traffic and requires careful consideration of network capacity, replica placement, quorum, recovery, and the performance impact of synchronous writes. In return, it provides a mature approach to replicated storage and high availability without requiring a separate shared SAN.

For Proxmox VE environments that prioritize redundancy, fast migrations, and automated storage provisioning, LINBIT SDS is a compelling option. For administrators who already have shared SAN infrastructure and specifically want to use that storage directly across multiple nodes, Shared LVM Thin, MeshXFS, and CetraFS address different requirements. The right choice ultimately depends on whether the goal is replicated hyperconverged storage or concurrent access to existing shared block devices. For more information, visit LINBIT's Proxmox VE solutions page.

Installing the Projects Through Debian Packages

An important aspect of these projects is that administrators do not have to manually compile every component or integrate the storage backends themselves. Shared LVM Thin, MeshXFS, and CetraFS provide Debian packages that simplify installation on supported systems. The packages can be installed using standard Debian tools, making it easier to deploy the components consistently across multiple Proxmox VE nodes.

Shared LVM Thin and CetraFS are also available through the gyptazy Debian repository. This provides a convenient way to install the packages and receive updates through the regular APT workflow. For clustered environments, the required packages must be installed on every node that is expected to use the respective storage backend.

MeshXFS provides its own Debian packages through its project releases. These include the filesystem kernel module and, optionally, the Proxmox VE storage plugin. Administrators should always check the release documentation for supported kernel versions and installation requirements before deploying the packages.

Packaging and integration are important parts of making these projects practical. A storage backend is not just about the filesystem or the block device itself. Administrators also need a reliable installation process, compatibility with kernel updates, consistent configuration across cluster nodes, and a way to manage the storage through the tools they already use.

Benchmark Overview

This benchmark compares LVM thin, LVM thick, and CetraFS using FIO workloads executed inside a virtual machine. The tests cover sequential throughput, random read and write performance, mixed random workloads, and low-queue-depth read latency. Unless otherwise noted, tests ran for 120 seconds with a queue depth of 32. The mixed workload ran for 180 seconds.

The overview below summarizes the primary performance metrics. Higher IOPS and bandwidth indicate better performance, while lower latency indicates better performance. The detailed results include average latency, median completion latency (p50), 99th-percentile completion latency (p99), and maximum latency.

Sequential Read

The sequential read test used 1 MiB blocks, a queue depth of 16, and a runtime of 120 seconds. LVM thin achieved the highest throughput, reaching approximately 1.5 GiB/s and substantially outperforming both LVM thick and CetraFS.

Sequential Write

The sequential write test used 1 MiB blocks, a queue depth of 16, and a runtime of 120 seconds. LVM thick achieved the highest throughput and the lowest average and p99 latency. CetraFS experienced a pronounced latency spike, with a maximum latency exceeding 13 seconds.

Random Read

The random read test used 4 KiB blocks, a queue depth of 32, and a runtime of 120 seconds. LVM thick clearly led in IOPS and bandwidth, while also achieving the lowest average and p99 completion latency. Its maximum latency was higher than that of LVM thin, however.

Random Write

The random write test used 4 KiB blocks, a queue depth of 32, and a runtime of 120 seconds. LVM thick achieved the highest IOPS and bandwidth, together with the lowest average latency. However, its p99 latency was notably higher than that of the other configurations, indicating a less consistent high-latency tail during the test.

Mixed Random Read/Write

The mixed workload used 4 KiB blocks, a 70/30 read/write ratio, a queue depth of 32, and a runtime of 180 seconds. LVM thick led in both read and write throughput. It also achieved the lowest average latency and p50 latency for both operations. LVM thin and CetraFS delivered lower read and write IOPS, with CetraFS showing the highest maximum read and write latencies in this test.

Random Read Latency

The latency-focused random read test used 4 KiB blocks, a queue depth of 1, and a runtime of 120 seconds. With only one outstanding I/O operation, the test emphasizes latency rather than parallel throughput. Average latency was similar across all three configurations, with LVM thick recording the lowest average and median latency. CetraFS achieved the lowest p99 latency.

Performance... and now?

The results show that LVM thick delivers the strongest overall performance across most of the tested workloads, particularly random I/O and mixed read/write operations. It achieves the highest random IOPS and generally lower median latency. However, its high p99 latency during random writes and the pronounced maximum-latency spike in the queue-depth-one test deserve further investigation.

LVM thin stands out in sequential reads, reaching 1536 MiB/s, approximately 1.76 times the throughput of LVM thick. This difference is large enough to warrant a repeat test before drawing firm conclusions. Caching, the state of the underlying storage, and reads from unallocated thin-provisioned blocks are possible factors to investigate.

CetraFS performs relatively close to LVM thin in several workloads but trails LVM thick in random and mixed I/O performance. Its most notable outlier is the 13.8-second maximum latency during sequential writes, which should be investigated to determine whether it represents a transient stall or a repeatable filesystem behavior.

These results should be treated as an initial comparison rather than a definitive ranking. Repeating the tests under controlled conditions, verifying that the test data is fully allocated and that caches do not distort sequential-read results, and checking the underlying device and filesystem state would help establish whether the observed differences are consistent.

Note: Average latency refers to FIO's total latency metric. The p50 and p99 values refer to completion latency (clat) percentiles. Bold values indicate the best result for each metric, with lower values preferred for latency.

Conclusion

Looking at Proxmox cluster storage in late 2026, it is clear that there is a lot happening in this area. Shared LVM Thin, MeshXFS, CetraFS, and rbdmirror are all approaches to addressing different gaps in the Proxmox storage ecosystem. Especially for people moving from VMware to Proxmox VE, the lack of a straightforward equivalent to VMFS as a clustered filesystem for VM workloads has been a recurring topic. Many administrators want to keep using their existing shared SAN infrastructure without having to introduce a completely different storage architecture.

These projects address different requirements and come with different architectural trade-offs. Shared LVM Thin focuses on thin provisioning and snapshots on existing shared block storage, while MeshXFS and CetraFS explore different approaches to concurrent filesystem access across multiple nodes. They are not direct replacements for one another, and the right choice depends on the existing infrastructure, operational requirements, and the level of complexity an environment is prepared to manage.

It is important to emphasize that this article reflects the current stage of development. There is a lot of excitement around filling some of the gaps in the Proxmox ecosystem, and I think this is a good thing. At the same time, we should remain realistic about what it takes to build reliable clustered storage. Projects such as GFS2 and OCFS2 have been worked on by many people for years. Developing a new clustered filesystem is not just about getting multiple nodes to mount the same device. Locking, concurrent access, fencing, crash recovery, data integrity, and countless failure scenarios all need to be handled correctly.

Therefore, a new clustered filesystem developed by a small group of individuals is not going to become an immediate drop-in replacement for VMFS. Reaching that level of maturity can take a considerable amount of time, testing, and community involvement. The same applies to the other solutions discussed here: each needs to prove itself against real-world workloads and failure scenarios before it can be considered for production use. New projects should be seen as promising developments, not as a guarantee that every challenge has already been solved.

Nevertheless, I am excited to see people working on these problems instead of simply accepting the existing limitations. With Debian packages, easier installation through repositories, and integration into the Proxmox VE web interface, these projects are also becoming more accessible to administrators who want to experiment with them. Community feedback, testing, contributions, and shared experience will be important in determining how far these approaches can go.

I hope to showcase some of these projects in a webinar or live demo at our next Virtualization Gathering at credativ GmbH. It will be interesting to see how the projects evolve and what further improvements we can bring to shared storage in Proxmox VE. There is still plenty of work ahead, but late 2026 is certainly an exciting time to follow Proxmox cluster storage!