Question about File Systems

[AndreL]

They say that reiserfs is better with small files and that xfs is better with large files... I believe that this must be compared to ext3. But what is considered a "small" file and a "large" file?

[danieldk]

Today that mantra only holds true for a bit. A part of this was caused by the fact that ext2/ext3 was very slow at handling large directories. But this has been corrected by the addition of directory index hashing (which normally is O(log n) for looking up files, rather than O(n)). On GNU/Linux, I can only recommend ext3 any day for various reasons:

- No one really supports reiserfs anymore. SUSE, which was the last big sponsor, has indicated that they intend to move to ext3 as their primary/default filesystem. ext3 is the filesystem that everyone uses to run their critical systems (of course, besides clustering filesystems), and gets the most reliability checking by far.
- reiserfs (and JFS) miss critical features that filesystems should have, like good quota, extended attribute, and access control lists support. reiserfs can not be used with SELinux, which seems to be the access control system that will rule the Linux world.
- ext3 provides different levels of journaling, including data journalling.
- In practice, reiserfs and XFS are harder to recover after a crash (for different reasons). For instance, reiserfs will try to find anything that resembles a B+-tree, and attach it to the filesystem being recovered, and it has very bad bad block handling. XFS on the other hand was designed for high-end servers, and has assumptions about how memory and disks will act in the case of e.g. a power surge or loss (basically, on cheap hardware memory goes bezerk when the disk is still rolling, potentially causing all kinds of havoc). Besides that, XFS uses delayed writes, meaning that blocks are kept in memory for a long time before they are written away. This allows for better ordering, potentially leading to less fragmentation. Though, again, not safe on average off-the-shelf hardware, because if a system crashes, you'll lose all the blocks that were delayed before being written.

The only filesystem that looks to be a potential contender is Solaris ZFS (and, of course, FFS2 on BSD, but FFS is fairly identical to ext2/ext3). Maybe we will see a port of that if the Solaris sources will become available under the GPL. For the time being it is only available for Solaris and FreeBSD (and Linux with FUSE, but it is a lot slower than an in-kernel implementation).

[AndreL]

Thanks Daniel. Very useful comment.
By the way, I've read somewhere that they are working on ext4.

[danieldk]

Yeah, like ext2 -> ext3 the step will be evolutionary. I think the first bits are already in some test kernels. ext4 will support larger volumes, and extends (a technique to give file the some opportunity to grow, which keeps down fragmentation). IIRC ext4 filesystems will be backwards compatible, unless extends are used. Some more information can be found on LWN (as usual ;)):

http://lwn.net/Articles/203915/
http://lwn.net/Articles/190169/

[AndreL]

Thanks for the links. I've collected quite a few Linux url but, somehow, I had missed that one... Lots of interesting info, there!