Category Archives: File Systems

KQ ZFS no longer actively being worked on – ZFS on Linux

The ZFS file system didn’t get ported to Linux for a long time as its source-code is distributed under the CDDL license, which is incompatible with the GNU GPL, therefore making it impossible to integrate it into the mainline Linux kernel.

However, KQ Infotech, and Indian tech company, ported the ZFS file-system to Linux as an out-of-tree kernel module. It performed relatively well and didn’t depend upon FUSE, (file system in user space), though performances hadn’t matched ‘native ZFS’.

Now it seems that KQ Infotech has abandoned the project, and it is not clear for what reasons. Did they receive a cease and desist letter from Oracle, and choose rather to halt development instead of going through courts? What are your thoughts?

Since the code was available online,  it was copied over and is now actively maintained at and ZFS on Linux.

High-availability storage with GlusterFS on Ubuntu

This tutorial shows how to set up a high-availability storage with two storage servers (Ubuntu 9.10) that use GlusterFS.

Each storage server will be a mirror of the other storage server, and files will be replicated automatically across both storage servers. The client system (Ubuntu 9.10 as well) will be able to access the storage as if it was a local filesystem.

GlusterFS is a clustered file-system capable of scaling to several peta-bytes. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. Storage bricks can be made of any commodity hardware such as x86_64 servers with SATA-II RAID and Infiniband HBA.

About Glusterfs: Gluster Storage Platform is an open source clustered storage solution. The software is a powerful and flexible solution that simplifies the task of managing unstructured file data whether you have a few terabytes of storage or multiple petabytes. Gluster Storage Platform integrates the file system, an operating system layer, and a web-based management interface and installer.

Google to switch to EXT4

Google is apparently in the process of migrating their current EXT2 file-systems over to the more current EXT4 file-system (Ubuntu 9.10 uses it by default)

Phoronix reports

This was brought up in a JFS benchmarking discussion. Google’s Michael Rubin shared that they chose EXT4 after benchmarking it as well as XFS and JFS (possibly with our Phoronix Test Suite carrying out some of the testing, which they have used in other areas). Their results showed EXT4 and XFS performing close to one another, but with it being easier to upgrade from EXT2 to EXT4 rather than EXT2 to XFS, they went with the easier path. Btrfs is still too experimental for Google to even consider that an option at this point.

Also they have now hired the main developer behind EXT4: Ted Ts’o

Will Google become even faster with processing large quantities of data, and displaying them to the end-user?

Gluster enhances open-source clustered NAS

Gluster adds data storage management, virtual server support to open-source clustered NAS

Gluster is joining a recent wave of emerging vendors adding enterprise storage management features to clustered network-attached storage (NAS) systems based on commodity hardware with this week’s release of the Gluster Storage Platform.

Gluster came out of stealth in 2007 with GlusterFS, a scale-out file system for clustered NAS based on open-source code but reengineered “from the ground up,” according to senior director of marketing Jack O’Brien. Version 2 of GlusterFS came out last May with striping, data replication and management tools.

The Gluster Storage Platform, which became available this week, continues to build on those management features with a new software delivery model, an updated Web-based management GUI, and new support for virtual servers, including the ability to self-heal data errors in virtual server environments.

With the Gluster Storage Management Platform, users can now get GlusterFS, the Linux operating system kernel layer and management tools in one package loaded on a thumb drive. Gluster calls this “clustered storage on a stick.” This package also includes a two-step installation process with the goal of making the open-source software accessible to customers who aren’t used to working with code. “We call this release rated ‘E’ for everyone,” O’Brien said.

Similarly, the Gluster Storage Platform uses a Web-based GUI that’s added support for more of Gluster’s management features, such as event logging, which used to require a command line interface.

Finally, though not officially certified with any major server virtualization vendor yet, Gluster is offering support for running virtual machines (VMs) on its clustered NAS. Customers who choose this option can use the cluster’s internal replication to provide high availability (HA) failover for VMs running on the cluster, which is set up using a checkbox at the time of installation. From there, the file system automatically handles the replication using the underlying object-based storage system.


New FreeBSD Foundation Project: HAST

FreeBSD foundation logoThe FreeBSD Foundation has announced that is funding a new funded project: HAST

“Pawel Jakub Dawidek has been awarded a grant to implement storage replication software that will enable users to use the FreeBSD operating system for highly available configurations where data has to be shared across the cluster nodes. The project is partly being funded by OMCnet Internet Service and TransIP BV.

The software will allow for synchronous block-level replication of any storage media (GEOM providers, using FreeBSD nomenclature) over the TCP/IP network and for fast failure recovery. HAST will provide storage
using GEOM infrastructure, which means it will be file system and application independent and could be combined with any existing GEOM class. In case of a master node failure, the cluster will be able to
switch to the slave node, check and mount UFS file system or import ZFS pool and continue to work without missing a single bit of data.

High-availability is the number one requirement for any serious use of any operating system,

Pawel Jakub Dawideksaid Pawel Jakub Dawidek, FreeBSD Developer.

Highly available storage is one of the key components in such environments. I strongly believe there are many FreeBSD users that have been waiting a long time for this functionality. I’ll do my best to deliver software that matches FreeBSD quality and that will satisfy the needs of our users.

Pawel has been an active FreeBSD committer since 2003. During this period, he has touched almost every part of the kernel. But, his main interest in FreeBSD is storage and security related topics. Pawel is the author of various GEOM classes (eli, mirror, gate, label, journal, hsec, etc.), geom(8) utility, various opencrypto improvements as well as port of the ZFS file system from OpenSolaris to FreeBSD.

The project will complete by February 2010.”

If you want, you can support this project too.