Chapter 1. Overview of File Systems in Linux

Contents

1.1. Terminology
1.2. Major File Systems in Linux
1.3. Other Supported File Systems
1.4. Large File Support in Linux
1.5. Additional Information

SUSE® Linux Enterprise Server ships with a number of different file systems from which to choose, including Ext3, Ext2, ReiserFS, and XFS. Each file system has its own advantages and disadvantages. To meet the requirements of high-performance clustering scenarios, SUSE Linux Enterprise Server includes OCFS2 (Oracle Cluster File System 2) in the High-Availability Storage Infrastructure (HASI) release.

1.1. Terminology

metadata

A data structure that is internal to the file system. It assures that all of the on-disk data is properly organized and accessible. Essentially, it is data about the data. Almost every file system has its own structure of metadata, which is on reason that the file systems show different performance characteristics. It is extremely important to maintain metadata intact, because otherwise all data on the file system could become inaccessible.

inode

A data structure on a file system that contains various information about a file, including size, number of links, pointers to the disk blocks where the file contents are actually stored, and date and time of creation, modification, and access.

journal

In the context of a file system, a journal is an on-disk structure containing a type of log in which the file system stores what it is about to change in the file system’s metadata. Journaling greatly reduces the recovery time of a file system because it has no need for the lengthy search process that checks the entire file system at system startup. Instead, only the journal is replayed.

1.2. Major File Systems in Linux

SUSE Linux Enterprise Server offers a variety of file systems from which to choose. This section contains an overview of how these file systems work and which advantages they offer.

It is very important to remember that no file system best suits all kinds of applications. Each file system has its particular strengths and weaknesses, which must be taken into account. In addition, even the most sophisticated file system cannot replace a reasonable backup strategy.

The terms data integrity and data consistency, when used in this section, do not refer to the consistency of the user space data (the data your application writes to its files). Whether this data is consistent must be controlled by the application itself.

[Important]

Unless stated otherwise in this section, all the steps required to set up or change partitions and file systems can be performed by using YaST.

1.2.1. Ext2

The origins of Ext2 go back to the early days of Linux history. Its predecessor, the Extended File System, was implemented in April 1992 and integrated in Linux 0.96c. The Extended File System underwent a number of modifications and, as Ext2, became the most popular Linux file system for years. With the creation of journaling file systems and their short recovery times, Ext2 became less important.

A brief summary of Ext2’s strengths might help understand why it was—and in some areas still is—the favorite Linux file system of many Linux users.

1.2.1.1. Solidity and Speed

Being quite an old-timer, Ext2 underwent many improvements and was heavily tested. This might be the reason why people often refer to it as rock-solid. After a system outage when the file system could not be cleanly unmounted, e2fsck starts to analyze the file system data. Metadata is brought into a consistent state and pending files or data blocks are written to a designated directory (called lost+found). In contrast to journaling file systems, e2fsck analyzes the entire file system and not just the recently modified bits of metadata. This takes significantly longer than checking the log data of a journaling file system. Depending on file system size, this procedure can take half an hour or more. Therefore, it is not desirable to choose Ext2 for any server that needs high availability. However, because Ext2 does not maintain a journal and uses significantly less memory, it is sometimes faster than other file systems.

1.2.1.2. Easy Upgradability

Because Ext3 is based on the Ext2 code and shares its on-disk format as well as its metadata format, upgrades from Ext2 to Ext3 are very easy.

1.2.2. Ext3

Ext3 was designed by Stephen Tweedie. Unlike all other next-generation file systems, Ext3 does not follow a completely new design principle. It is based on Ext2. These two file systems are very closely related to each other. An Ext3 file system can be easily built on top of an Ext2 file system. The most important difference between Ext2 and Ext3 is that Ext3 supports journaling. In summary, Ext3 has three major advantages to offer:

1.2.2.1. Easy and Highly Reliable Upgrades from Ext2

The code for Ext2 is the strong foundation on which Ext3 could become a highly-acclaimed next-generation file system. Its reliability and solidity are elegantly combined in Ext3 with the advantages of a journaling file system. Unlike transitions to other journaling file systems, such as ReiserFS or XFS, which can be quite tedious (making backups of the entire file system and recreating it from scratch), a transition to Ext3 is a matter of minutes. It is also very safe, because re-creating an entire file system from scratch might not work flawlessly. Considering the number of existing Ext2 systems that await an upgrade to a journaling file system, you can easily see why Ext3 might be of some importance to many system administrators. Downgrading from Ext3 to Ext2 is as easy as the upgrade. Just perform a clean unmount of the Ext3 file system and remount it as an Ext2 file system.

1.2.2.2. Reliability and Performance

Some other journaling file systems follow the metadata-only journaling approach. This means your metadata is always kept in a consistent state, but this cannot be automatically guaranteed for the file system data itself. Ext3 is designed to take care of both metadata and data. The degree of care can be customized. Enabling Ext3 in the data=journal mode offers maximum security (data integrity), but can slow down the system because both metadata and data are journaled. A relatively new approach is to use the data=ordered mode, which ensures both data and metadata integrity, but uses journaling only for metadata. The file system driver collects all data blocks that correspond to one metadata update. These data blocks are written to disk before the metadata is updated. As a result, consistency is achieved for metadata and data without sacrificing performance. A third option to use is data=writeback, which allows data to be written into the main file system after its metadata has been committed to the journal. This option is often considered the best in performance. It can, however, allow old data to reappear in files after crash and recovery while internal file system integrity is maintained. Ext3 uses the data=ordered option as the default.

1.2.2.3. Converting an Ext2 File System into Ext3

To convert an Ext2 file system to Ext3:

  1. Create an Ext3 journal by running tune2fs -j as the root user.

    This creates an Ext3 journal with the default parameters.

    To specify how large the journal should be and on which device it should reside, run tune2fs -J instead together with the desired journal options size= and device=. More information about the tune2fs program is available in the tune2fs man page.

  2. Edit the file /etc/fstab as the root user to change the file system type specified for the corresponding partition from ext2 to ext3, then save the changes.

    This ensures that the Ext3 file system is recognized as such. The change takes effect after the next reboot.

  3. To boot a root file system that is set up as an Ext3 partition, include the modules ext3 and jbd in the initrd.

    1. Edit /etc/sysconfig/kernel as root, adding ext3 and jbd to the INITRD_MODULES variable, then save the changes.

    2. Run the mkinitrd command.

      This builds a new initrd and prepares it for use.

  4. Reboot the system.

1.2.3. Oracle Cluster File System 2

OCFS2 is a journaling file system that has been tailor-made for clustering setups. In contrast to a standard single-node file system like Ext3, OCFS2 is capable of managing several nodes. OCFS2 allows you to spread a file system across shared storage, such as a SAN or multipath setup.

Every node in an OCFS2 setup has concurrent read and write access to all data. This requires OCFS2 to be cluster-aware, meaning that OCFS2 must include a means to determine which nodes are in the cluster and whether these nodes are actually alive and available. To compute a cluster’s membership, OCFS2 includes a node manager. To monitor the availability of the nodes in a cluster, OCFS2 includes a simple heartbeat implementation. To avoid problems arising from various nodes directly accessing the file system, OCFS2 also contains a distributed lock manager. Communication between the nodes is handled via a TCP-based messaging system.

Major features and benefits of OCFS2 include:

  • Metadata caching and journaling

  • Asynchronous and direct I/O support for database files for improved database performance

  • Support for multiple block sizes (where each volume can have a different block size) up to 4 KB, for a maximum volume size of 16 TB

  • Cross-node file data consistency

  • Support for up to 255 cluster nodes

For more in-depth information about OCFS2, refer to the High Availability Storage Infrastructure Administration Guide.

1.2.4. ReiserFS

Officially one of the key features of the 2.4 kernel release, ReiserFS has been available as a kernel patch for 2.2.x SUSE kernels since version 6.4. ReiserFS was designed by Hans Reiser and the Namesys development team. It has proven itself to be a powerful alternative to Ext2. Its key assets are better disk space utilization, better disk access performance, faster crash recovery, and reliability through data journaling.

1.2.4.1. Better Disk Space Utilization

In ReiserFS, all data is organized in a structure called a B*-balanced tree. The tree structure contributes to better disk space utilization because small files can be stored directly in the B* tree leaf nodes instead of being stored elsewhere and just maintaining a pointer to the actual disk location. In addition to that, storage is not allocated in chunks of 1 or 4 KB, but in portions of the exact size needed. Another benefit lies in the dynamic allocation of inodes. This keeps the file system more flexible than traditional file systems, like Ext2, where the inode density must be specified at file system creation time.

1.2.4.2. Better Disk Access Performance

For small files, file data and stat_data (inode) information are often stored next to each other. They can be read with a single disk I/O operation, meaning that only one access to disk is required to retrieve all the information needed.

1.2.4.3. Fast Crash Recovery

Using a journal to keep track of recent metadata changes makes a file system check a matter of seconds, even for huge file systems.

1.2.4.4. Reliability through Data Journaling

ReiserFS also supports data journaling and ordered data modes similar to the concepts outlined in Section 1.2.2, “Ext3”. The default mode is data=ordered, which ensures both data and metadata integrity, but uses journaling only for metadata.

1.2.5. XFS

Originally intended as the file system for their IRIX OS, SGI started XFS development in the early 1990s. The idea behind XFS was to create a high-performance 64-bit journaling file system to meet extreme computing challenges. XFS is very good at manipulating large files and performs well on high-end hardware. However, even XFS has a drawback. Like ReiserFS, XFS takes great care of metadata integrity, but less care of data integrity.

A quick review of XFS’s key features explains why it might prove to be a strong competitor for other journaling file systems in high-end computing.

1.2.5.1. High Scalability through the Use of Allocation Groups

At the creation time of an XFS file system, the block device underlying the file system is divided into eight or more linear regions of equal size. Those are referred to as allocation groups. Each allocation group manages its own inodes and free disk space. Practically, allocation groups can be seen as file systems in a file system. Because allocation groups are rather independent of each other, more than one of them can be addressed by the kernel simultaneously. This feature is the key to XFS’s great scalability. Naturally, the concept of independent allocation groups suits the needs of multiprocessor systems.

1.2.5.2. High Performance through Efficient Management of Disk Space

Free space and inodes are handled by B+ trees inside the allocation groups. The use of B+ trees greatly contributes to XFS’s performance and scalability. XFS uses delayed allocation, which handles allocation by breaking the process into two pieces. A pending transaction is stored in RAM and the appropriate amount of space is reserved. XFS still does not decide where exactly (in file system blocks) the data should be stored. This decision is delayed until the last possible moment. Some short-lived temporary data might never make its way to disk, because it is obsolete by the time XFS decides where actually to save it. In this way, XFS increases write performance and reduces file system fragmentation. Because delayed allocation results in less frequent write events than in other file systems, it is likely that data loss after a crash during a write is more severe.

1.2.5.3. Preallocation to Avoid File System Fragmentation

Before writing the data to the file system, XFS reserves (preallocates) the free space needed for a file. Thus, file system fragmentation is greatly reduced. Performance is increased because the contents of a file are not distributed all over the file system.

1.3. Other Supported File Systems

Table 1.1, “File System Types in Linux” summarizes some other file systems supported by Linux. They are supported mainly to ensure compatibility and interchange of data with different kinds of media or foreign operating systems.

Table 1.1. File System Types in Linux

File System Type

Description

cramfs

Compressed ROM file system: A compressed read-only file system for ROMs.

hpfs

High Performance File System: The IBM* OS/2* standard file system. Only supported in read-only mode.

iso9660

Standard file system on CD-ROMs.

minix

This file system originated from academic projects on operating systems and was the first file system used in Linux. Today, it is used as a file system for floppy disks.

msdos

fat, the file system originally used by DOS, is today used by various operating systems.

ncpfs

File system for mounting Novell® volumes over networks.

nfs

Network File System: Here, data can be stored on any machine in a network and access might be granted via a network.

smbfs

Server Message Block is used by products such as Windows* to enable file access over a network.

sysv

Used on SCO UNIX*, Xenix, and Coherent (commercial UNIX systems for PCs).

ufs

Used by BSD*, SunOS*, and NextStep*. Only supported in read-only mode.

umsdos

UNIX on MS-DOS*: Applied on top of a standard fat file system, achieves UNIX functionality (permissions, links, long filenames) by creating special files.

vfat

Virtual FAT: Extension of the fat file system (supports long filenames).

ntfs

Windows NT file system; read-only.


1.4. Large File Support in Linux

Originally, Linux supported a maximum file size of 2 GB. This was enough before the explosion of multimedia and as long as no one tried to manipulate huge databases on Linux. Becoming more and more important for server computing, the kernel and C library were modified to support file sizes larger than 2 GB when using a new set of interfaces that applications must use. Today, almost all major file systems offer LFS support, allowing you to perform high-end computing. Table 1.2, “Maximum Sizes of File Systems (On-Disk Format)” offers an overview of the current limitations of Linux files and file systems.

Table 1.2. Maximum Sizes of File Systems (On-Disk Format)

File System

File Size (Bytes)

File System Size (Bytes)

Ext2 or Ext3 (1 KB block size)

234 (16 GB)

241 (2 TB)

Ext2 or Ext3 (2 KB block size)

238 (256 GB)

243 (8 TB)

Ext2 or Ext3 (4 KB block size)

241 (2 TB)

244 -4096 (16 TB-4096 Bytes)

Ext2 or Ext3 (8 KB block size) (systems with 8 KB pages, like Alpha)

246 (64 TB)

245 (32 TB)

ReiserFS v3

246 (64 TB)

245 (32 TB)

XFS

263 (8 EB)

263 (8 EB)

NFSv2 (client side)

231 (2 GB)

263 (8 EB)

NFSv3 (client side)

263 (8 EB)

263 (8 EB)


[Important]

Table 1.2, “Maximum Sizes of File Systems (On-Disk Format)” describes the limitations regarding the on-disk format. The 2.6 Linux kernel imposes its own limits on the size of files and file systems handled by it. These are as follows:

File Size

On 32-bit systems, files cannot exceed 2 TB (241 bytes).

File System Size

File systems can be up to 273 bytes in size. However, this limit is still out of reach for the currently available hardware.

1.5. Additional Information

The File System Primer on the Novell Web site describes a variety of file systems for Linux. It discusses the file systems, why there are so many, and which ones are the best to use for which workloads and data.

Each of the file system projects described above maintains its own home page on which to find mailing list information, further documentation, and FAQs:

A comprehensive multipart tutorial about Linux file systems can be found at IBM developerWorks in the Advanced Filesystem Implementor’s Guide. An in-depth comparison of file systems (not only Linux file systems) is available from the Wikipedia project in Comparison of File Systems.