ZFS on Linux 0.7.7 Disappearing File Bug

ZFS is known to be a very stable storage platform. While it may not be the fastest storage platform around, it has great checksumming and other features that make it reliable which is why it is popular. The rise in ZFS popularity, originally of Sun Microsystems origins, has made it the go-to for users of FreeBSD OSes like FreeNAS. More recently, Ubuntu incorporated ZFS on Linux which opened the floodgates to ZFS usage on Linux. Many users trust ZFS for their storage and that is what makes a bug in the current ZFS on Linux 0.7.7 so concerning.

ZFS on Linux 0.7.7 Disappearing File Bug

The latest bug impacts users with large numbers of files on ZoL version 0.7.7. Here is the GitHub bug. This has been reported in file systems with thousands of files and seems to mostly impact RHEL, CentOS, and Scientific Linux based installations. It seems that users of Ubuntu among others may be safe from this one. Edit: It now appears that Ubuntu and other OSes are having issues as well. Stay safe and avoid 0.7.7.

Reading through the bug, you can see that the bug has been confirmed. The mitigations are twofold. According to STH forum members, they mitigation plan and path forward looks something like this:

1. Downgrade to ZFS on Linux 0.7.6
2. ZFS on Linux 0.7.7 rpms are going to be pulled
3. ZFS on Linux 0.7.8 is expected soon which will fix this bug reverting the change that caused it.

Final Words

In the world of storage, reliability is king. Speed is good, but if you are using RHEL in your organization with ZFS on Linux for storage, this is a very bad bug. We suggest reverting the transaction that upgraded from 0.7.6 to 0.7.7 if you have a production ZFS system with the current version. Then wait for 0.7.8 to become available. For the ZFS on Linux team, this is the scariest type of bug as it has the potential to cause data loss in storage servers thought to be safe.