File Systems: From Path to Blocks

Whatever the on-disk format, a file system has to answer four questions: which object does this name refer to (directories), what are its attributes and where are its bytes (metadata), which blocks hold byte range 8192–12287 of this object (block mapping), and which blocks are free (allocation). Storage is addressed in fixed-size blocks — 4 KiB on nearly every modern file system, matching the memory page — so every file is a sequence of block addresses, and a 5-byte file occupies one whole block.

The block mapping is where designs diverge most. Early Unix file systems and ext2/ext3 stored an array of block pointers per object: 12 direct pointers, then single, double and triple indirect blocks — a tree whose depth grows with file size. Modern designs (ext4, XFS, btrfs, APFS, NTFS) use extents: (start block, length) runs, so a contiguous 1 GB file is described by a single entry instead of 262,144 pointers. Extents make sequential I/O cheap and fragmentation visible: filefrag shows how many runs a file has.

Free space is usually a bitmap — one bit per block, so 32 GB of 4 KiB blocks needs 1 MB of bitmap — grouped into regions (ext4 block groups, XFS allocation groups) so allocation can be local to a region and so several CPUs can allocate concurrently. XFS and btrfs additionally keep free space in B-trees indexed by size, which lets them find a large contiguous run without scanning. df reads the summary counters; a full disk is a bitmap with no zero bits left in any group.

Be precise about what is universal: the four questions are; the structures are not. NTFS keeps everything in the Master File Table, where each record holds attributes and, for small files, the data itself. FAT/exFAT has no separate metadata object — the directory entry is the metadata, and the block chain lives in the File Allocation Table. "All file systems have inodes" is false; Inodes covers the Unix-style design specifically.

Nothing in the ladder above is read from storage twice if it can be helped. Linux keeps three caches, all sized dynamically from otherwise free RAM and reclaimed under Memory Pressure, Swap and the OOM Killer. The page cache holds file data, 4 KiB pages indexed by (object, page number); a read() that hits it is a memory copy costing about a microsecond. The dentry cache holds resolved (directory, name) → object lookups, including negative entries ("no such file"), which is why a missing-file probe in a hot directory is nearly free. The inode cache holds metadata objects so stat on a recently used file never touches storage.

Writes go to the page cache too. write() copies your bytes into a page, marks it dirty, and returns — in a microsecond — long before anything reaches the device. Kernel writeback threads flush dirty pages after about 30 seconds (dirty_expire_centisecs) or when dirty memory exceeds a threshold (dirty_ratio), coalescing small writes into large sequential ones. This is why copying a large file looks fast and then sync takes ten seconds, and why a machine with 64 GB of RAM can absorb gigabytes of writes and lose them all on power failure.

The consequence for anyone building a durable system is that `write` returning does not mean anything is durable. It means the kernel has a copy. Free memory that shows as "buff/cache" in free -m is these caches; it is available to applications, but evicting it costs re-reads, so a database whose working set fits in page cache and then gets squeezed by a memory-hungry neighbour goes from RAM-speed to disk-speed with no code change.

Creating /var/log/app.log writes: the free-object bitmap (allocate a metadata object), the object itself (initialise it), the directory’s data block (add the entry), and the directory’s metadata (new size and mtime). Four blocks, four separate device writes, no atomicity across them. Crash after the bitmap but before the object: a leaked object. Crash after the directory entry but before the object: a name pointing at garbage. The old answer was fsck at boot, walking the whole disk to reconcile bitmaps with reality — hours on a large volume.

Journaling fixes this by writing the intended changes first to a sequential log, then applying them in place, then marking the journal entry complete. After a crash, replay complete entries, discard incomplete ones; the on-disk structures are always either entirely before or entirely after an operation. What goes in the journal is the key design choice. Metadata journaling (ext4 default data=ordered, XFS, NTFS $LogFile) journals only the structures above, and writes file data blocks before the metadata that points at them, so you never see a file that points at stale blocks — but the data itself may be lost or partial. Full data journaling (ext4 data=journal) writes every byte twice and is rarely worth it. ext4 data=writeback relaxes the ordering and can expose old block contents in a new file after a crash.

Copy-on-write file systems — btrfs, APFS, ZFS — take a different route: they never overwrite a live block. A change writes new blocks, then atomically flips a root pointer (a superblock or checkpoint) to the new tree. The old tree is intact until that flip, so consistency needs no journal, and snapshots and clones fall out for free because an old root is just a tree that has not been freed. The cost is fragmentation of frequently rewritten files — database files and VM images on btrfs are the canonical case, which is why chattr +C (disable CoW) exists.

fsync(fd) asks the kernel to write the file’s dirty pages and metadata to the device and not return until the device reports them stable. On a journaling file system that means: flush the data pages, commit the journal transaction, and issue a cache flush (or a Force Unit Access write) to the drive so that bytes sitting in the drive’s volatile write cache reach the medium. Cost: tens of microseconds on an enterprise NVMe drive with power-loss protection, hundreds of microseconds to a few milliseconds on a consumer SSD, 10–30 ms on a spinning disk. That per-call cost is the reason a database commits in groups and the reason pg_test_fsync exists.

Two details bite. First, on Linux, fsync of a newly created file does not guarantee the directory entry is durable; you must also fsync the directory descriptor, or the file can be complete but nameless after a crash. Second, on macOS fsync only pushes data to the drive and does not flush the drive cache; fcntl(fd, F_FULLFSYNC) is needed for the real guarantee, and it is dramatically slower — SQLite and PostgreSQL both know this and switch accordingly. fdatasync skips non-essential metadata (mtime) and is what write-ahead logs use.

This is the exact boundary between the OS and the database domains. A database commit is durable when its write-ahead log record has been fsynced (Transactions and ACID); everything else — data pages, indexes — can be rebuilt from that log after a crash. The database is layering its own journal on top of the file system’s journal because the file system only promises consistency of its own structures, not of the database’s pages. Some engines open their files with O_DIRECT to skip the page cache entirely and manage their own buffer of Pages: The Unit of Everything; others, PostgreSQL among them, rely on the page cache and pay the double-buffering cost for the simplicity.

The names you will meet: ext4 is the Linux default on most distributions; XFS is the default on RHEL and the usual choice for large parallel workloads; btrfs is the copy-on-write Linux option with checksums and snapshots; APFS is what every Apple device runs; NTFS is Windows. They agree on blocks, directories, and some form of crash consistency; they disagree on nearly everything else, so treat any statement about "the file system" as a statement about one of them until proven otherwise.

Two practical differences show up constantly. ext4 fixes the number of metadata objects at format time; XFS and btrfs allocate them dynamically (Inodes covers the failure). And only btrfs, ZFS and (for metadata) APFS checksum what they store; on ext4, XFS and NTFS a silently corrupted block is returned to you as data, which is why databases and object stores add their own page checksums.