EXT3 and EXT4 File Systems for a Journaling File System

Summary

The following report discusses the topic “Journaling File Systems” with the focus on the functionality and implementation of the EXT3 and EXT4 file systems. After providing an overview over the purpose and general functionality of file systems, as well as the difference that journaling makes to the security and functionality of a file system, the EXT3 and EXT4 file systems are compared in the context of system performance and security measures and different journaling modes are discussed. Journaling file systems is then discussed in context of two significant areas of application; forensics and data recovery. Further on, the report highlights significant advantages and disadvantages of both file systems as well as advantages arising from the development of EXT3 to EXT4, and the effect on journaling from the development. Conclusively, the core findings of the report are summarised, and after comparing both file systems and journaling modes, the ultimate choice of a file system or the need to change journaling modes is entirely up to the requirements and preferences of the end user.

Background Information

File system journaling was originally introduced to reduce the required amount of time that a file system would need to recover from events such as a system crash or power-out. A journaling file system works by keeping a record of any changes made to the file system before it reflects the new changes on the system. Instead of needing to scan the complete file system again, the journaling file system can process any recorded changes to the system files again (Lim, 2013). This eliminates lengthy file system checks for example after a sudden system shutdown such as a crash or power-out. There are different versions of file system journals available and depending on the operating system, there may be different ones installed. This research focuses mainly on the EXT and EXT4 file system journals that are most commonly found in Linux operating systems. There is three different levels of journaling in Linux: Journal mode, Ordered mode and Writeback mode, which will be discussed in more details in the discussion part (Kerrisk, 2010).

The EXT3 file journaling system had been developed as an extension of the EXT2 file system, which in its core is of a very similar structure as the EXT3 file system, however a new feature was implemented in the EXT3 file system: file journaling.

A file system journal records any data before it is going to be written onto the actual file system. This prevents data loss in the case of a malfunctioning system and is an important feature that ensures the file system’s integrity. All of the commonly used operating systems such as Linux, Windows and Mac OS X use a file system, however the type of journal may differ depending on the operating system. While the EXT3 and EXT4 are used as per default in Linux, Windows uses a different journal called NTFS. If deleted files or overwritten versions of a file need to be retrieved, file system journaling is essential, as it can complete the interrupted writes to a file by re-playing the journal once power is restored in the case of a power-out and recover recently overwritten or even deleted files.  This is particularly important for forensics.

A file system also contains system files. In the case of EXT3, information about these files are stored in a number of entries, called “inodes”. Every inode is allocated a number, which then stores file information such as the file size or permissions of the file. While the inode does not contain the name of a file, it does control the file which it is pointed to. The file name, however, does contain the inode, which allows to retrieve or access the file through the inode it is allocated to. To ensure that a file system does not reach capacity, there is more entries available for more than all the files that could ever be created on the file system. A file record then stores the combined file name and inode number of the file, instead of having to store a duplicate copy of the file data itself (Wells, 2005).

Throughout a file system, the block size of the block structure being used remains the same. The default size of a block is either 1024 or 4096 bytes, whereas the first block of bytes is reserved. A superblock then stores general information about the file system, such as its name and the block size. In the case of EXT3, it has one primary superblock available. Another type of block, an inode table, then stores the inodes which have a length of 256 bytes. Every single inode can contain a maximum of 12 direct pointers associated with the address of the file system blocks and the location that holds the first 12 blocks of the file. Pointers can be maintained to double and triple indirect pointer block if a file does not fit into 12 blocks. An indirect pointer is defined as the address of a file system block, which stores the direct pointers. The EXT3 blocks are further divided into block groups, which are described in a set of blocks: the group descriptor table.  Depending on the size of the file system and on the block size, the number of blocks per group may vary between file systems. However, the number of blocks per group is always eight times the number of the bytes per block. Across the block groups, the inodes are distributed evenly. Two bitmap blocks are contained for each block group- where one is stored for the blocks and the other one for inodes. The status of the blocks and inodes per group are reflected by bits as either allocated or unallocated. Those blocks and inodes associated with versions of files whose content has been overwritten or deleted, as well as those not allocated, are represented by 0. Represented with 1 are those blocks and inodes that are containing file contents and metadata, as well as those that have been reserved by the file system. The size of a journal in the EXT3 file system is depending on the overall size of the file system. For purposes of saving space, the journal in the EXT file system only records metadata of the file unless specified otherwise. Information that is specific to the journal is always stored in the first block of the journal. Every individual journal entry, which is kept in a circular queue, has a one descriptor block at the minimum, as well as a commit block which appears only at the end of the transaction (Craiger, 2005). A transaction is a group of related metadata in which any updates to the file system are logged. It is also where writes are stored before they are reflected on the file system, which allows to redo an incomplete update to a file if the system crashes before the transaction was completed (Kerrisk, 2010). The descriptor block provides the sequence number of a transaction as well as the file system block that is being stored. In the case that a transaction involves more than one descriptor block, another one can be created in order to accommodate the remaining blocks. Through these mechanisms, new versions of data are only being written to the disk after having been recorded in the journal when the file system is in journaling mode. This way, the journal can provide potentially vital information about contents of old, deleted and modified files (Swenson, 2007)

While the number of subdirectories is limited to 32000 in EXT3 and EXT3 does not checksum its journals, the development of EXT4, the successor of EXT3, does provide those functionalities and provides an unlimited number of subdirectories (Kerrisk, 2010). EXT4, commonly used by Linux and Android, uses 48-bit internal addressing, which in theory makes it possible to allocate files up to 15 TiB on a filesystem of up to 1 EiB, as opposed to EXT3 which uses 32-bit internal addressing. The reading and writing performance in EXT4 is improved through an improvement of the storage blocks. EXT4 introduced “extents”, ranges of contiguous blocks that can be both reserved and addressed at the same time. This reduces the number of inodes that are required file a file, as well as reducing fragmentation and improving performance significantly, particularly for writing large files. While EXT3 has to call on the block allocator for each new block that needs to be allocated, potentially leading to large fragmentations, EXT4 uses delayed allocation which allows for better block allocations for writes that are not yet committed. While EXT3 allocates blocks immediately, delayed allocation in EXT4 allows the file system to wait until it allocates the blocks the data is going to be written onto, until it is ready to also commit the data to the disk. While this increases the risk for potential data loss especially in programs where the data is committed to the disk manually, it improves the system performance significantly by reducing fragmentation. Furthermore, EXT4 provides nanosecond parts of timestamps, which could used to build a system of stenographic strength, meaning the system would not leave any evidence of information exchange. While this could be misused by attackers and cyber criminals, the knowledge about its functionality can be crucial to develop countermeasures against attacks (Goebe, 2018).

In the EXT3 file system in Linux there is three different levels of file system journaling available with different characteristic in terms of system performance and security. While a journal mode writes both a files data and metadata to a journal before it gets reflected on the file system, making it the lowest risk mode, it can potentially slow down a systems performance. The default journaling mode in Linux is ordered mode, in which the metadata is written to the journal, but the data is committed to the file system directly. Firstly, the metadata is recorded on the journal. Secondly, the data is written to the file system, before as a last step the associated metadata in the journal is send to the filesystem itself (Chen, 2017). While this might be beneficial in terms of system performance since the data is written to the system directly and does not get stored in the journal along with the metadata, it does provide a lower level of safety since the data is committed to the file system directly. Writeback mode is considered to be the least safest journaling mode, as the metadata is recorded in the journal, but the actual data is not. While it may increase system performance since the data does not get journaled, it provides less safety measures as the metadata and the actual data are recorded independently from each other (Kerrisk, 2010).

In EXT3, only changes made to metadata are recorded in the journal per default. However, mounting the file system through a series of commands can enable the journaling of all data and allowing to analyse journals for possible information about older versions of a file, as well as deleted and edited files (Swenson, 2007).

In terms of data recovery, file journaling can be essential. A journal contains blocks that are going to be written to the hard disk; these blocks could either be unformatted user data or blocks in the internal structure of the file system tree. Once changes are made to a file on the drive, the blocks are rewritten together with the metadata. The block map at the beginning of a journal is helpful in order to determine which blocks are associated with which files (Swenson, 2007).

A virtual file system layer that is available in Linux makes it possible to associate inode numbers to file names, making it much more efficient to determine a specific file name when journal data is analysed. Especially when working with honeypots, this functionality is particularly useful, however it needs to be ensured that the logging mechanism cannot be deactivated by the root user. If attackers gain access to a system, they would otherwise be able to change the privilege level of the root user and deactivate the logging mechanism. Therefore, it is recommended to not assign the root user the privilege of activating or deactivating logging mechanisms. A Virtual File System provides an interface through which processes can run input and output operations while simultaneously hiding information of the underlying file system (Fairbanks, 2009).

While a file system specifies how files and metadata are stored and a file system journal allows the oporeating system to determine a location of free space to store new files more efficiently, it is crucial to be aware of the risks and dangers on the other side of improved performance. Each superblock has a minimum of 384 bytes reserved for any updates to file system specifications, however since this particular group of bytes does not have a specific purpose, it can be misused to hide data. While a superblock is desgined to fit into 1k space, it may requier an entire block of the file system, up to a maximum of 4k, as the space cant be shared. This means that any leftover space, 3k at the maximum, are unused and could be used to hide data. Though this is a vulnerability that could technically be exploited, knowing about it is an opportunity to develop countermeasures (Piper, 2005).

Conclusively, while there is different file systems for every operating system available as well as different journaling modes, it always depends on the context that a file system is used in and the requirements of the task that the journaling mode is needed for to decide which file system and journaling mode to choose. While Linux file systems provide the highest safety level, other file systems might prioritise performance over security. It is entirely up to the preferences of the end user to weigh up the advantages and disadvantages of each file system and then decide which one, including their choice of journaling mode, suits their needs the most. While a file system is not something that a user could change, the journaling mode can be altered, though usually the journaling mode set as default for the file system of the operating system will be capable of fulfilling its task, unless any specific operations are required to be performed.

References 

• Chen, C. Y., Jun; Wei, Qinsong; Wang, Chudong; Xue, Mingdi (2017). “Optimizing File Systems with Fine-grained Metadata Journaling on Byte-addressable NVM.” https://doi.org/10.1145/3060147
• Craiger, P. (2005). Recovering Digital Evidence from Linux Systems. Paper presented at the IFIP International Conference on Digital Forensics, Boston, MA. https://doi.org /10.1007/0-387-31163-7_19
• Fairbanks, K. D. X., Ying H.; Own, Henry L. (2009). A Method for Historial Ext3 Inode o Filename Translation on Honeypots. Paper presented at the 2009 33rd Annnual IEEE International Computer Software and Applications Conference, Seattle, WA.
• For Reliabe And High-Performance Storage Systems. Personal and Ubiquitous Computing, 17(8), 1761-1774. doi:https://doi-org.ezproxy.ecu.edu.au/10.1007/s00779-012-0603-5
• Goebel, T. B., Harald. (2018). Anti-Forensics in ext3: On secrecy and usability of timestamp-based data hiding. Proceedings of the Fith Annual DFRWA Europe, 24, 111-120. doi:https://doi.org/10.1016/j.diin.2018.01.014
• Lim, S.-H. C., Hyun Jin; Park, Doo-Soon. (2013). Efficient Journaling Writeback Schemes
• Kerrisk, M. (2010). The Linux Programming Interface. San Francisco, CA: No Starch Press.
• Piper, S. D., M.; Manes, G.; Shenoi, S. . (2005). Detecting Hidden Data in Ext2/Ext3 File Systems. Paper presented at the IFIP Internatiol Conference on Digital Forensics, Boston, MA. https://doi.org/0.1007/0-387-31163-7_20
• Swenson, C. P., Rachel;  Shenoi, Sujeet. (2007). File System Journal Forensics. Paper presented at the IFIP International Conference on Digital Forensics, Orlando, Florida. https://doi.org/chapter/10.1007/978-0-387-73742-3_16#enumeration
• Wells, N. (2005). The Complete Guide to Linux System Administration. Boston, MA: Cengage.