Many options exist for setting up clustered and highly available storage, but figuring out what each does will take a bit of research. Your choice of storage architecture as well as file system is crucial, as most have severe limitations that require careful design workarounds.
In this article we will cover a few common physical storage configurations, as well as clustered and distributed file system options. Hopefully, this is a good starting point to begin looking into the technology that will work best for your high availability storage needs.
Underlying Architectures
Some readers may wish to configure a cluster of servers that simply have concurrent access to the same file system, while others want to replicate storage and provide both concurrent access and redundancy. There are two ways to go about providing multiple servers access to the same disks: let them both see it, or via replication.
Shared-disk configurations are most common in the Fibre Channel SAN and iSCSI worlds. It is quite simple to configure storage systems such that multiple servers can see the same logical block device, or LUN, but without a clustered file system, chaos will ensue if both try to use it at the same time. This problem is dealt with by using clustered file systems, which we will cover in a moment.
Generally speaking, shared-disk setups have a single point of failure: the storage system. This is not always true, however, as “shared-disk” is a confusing term with today’s technology. SANs, NAS appliances, and commodity hardware running Linux can all replicate the underlying disks in real-time to another storage node, which provides a simulated shared-disk environment. Since the underlying block devices are replicated, the nodes have access to the same data and both run a clustered file system, but this replication breaks the traditional shared-disk definition.
Shared-nothing, in contrast, was the original answer to shared-disk single points of failure. Nodes with distinct storage would notify a master server with changes, as each block was written. Nowadays, shared-nothing architectures still exist in file systems like Hadoop, which purposely creates multiple copies of data across many nodes for both performance and redundancy. Also, clusters that employ replication between storage devices or nodes with their own storage, are said to be share-nothing as well.
Read further about design choices and clustered file systems, incl
- GFS: Global File System
- OCFS: Oracle’s Oracle Clustered File System.
- VMFS: VMware’s Virtual Machine File System.
- Lustre: Sun’s clustered, distributed file system.
- Hadoop: a distributed file system, like Google uses.
