Memory Mapping, the Page Cache and Network I/O

The kernel keeps recently used file pages in RAM — the page cache — and every read(), write(), mmap and sendfile goes through it (see Follow a File Read and Memory Mapping). A process that writes to a file is writing into page-cache pages that the kernel flushes to disk later; a process that maps the file sees those same pages in its address space; a sendfile to a socket hands those same pages to the NIC. There is one copy of the data in memory and three ways to reach it. Systems that exploit this let the page cache be their buffer, their cache and their transfer mechanism at once, and skip building all three.

The price is that the page cache is managed by the kernel’s policy, not the application’s: eviction is approximately LRU across every file on the box (see Memory Pressure, Swap and the OOM Killer); a write is durable only after fsync, which the application must call; and a page fault on a mapped page that was evicted costs a disk read at a moment the application did not choose. Systems split on whether that trade is acceptable, and the split is the most interesting design decision in this lesson.

A Kafka broker stores each partition as a sequence of append-only segment files. Producers’ batches are written with ordinary file writes into the page cache — sequential, so the disk sees large writes when the kernel flushes, and the write call itself is a memory copy. The broker does not maintain its own cache of messages: recent data is in the page cache because it was just written, and the operating system keeps it there as long as memory allows. On a machine with 64 GB and a 32 GB heap deliberately kept small, 30 GB of page cache holds the last minutes of every partition.

Consumers fetch by offset; the broker answers with FileChannel.transferTo — sendfile — from the segment file to the consumer’s socket. With a scatter-gather NIC the data goes from page cache to wire with no CPU copy and no visit to the JVM heap (see Zero-Copy: Serving a File Without Touching It). Ten consumers of the same partition read the same page-cache pages ten times; the disk is read once, if at all. This is why a Kafka broker’s throughput is bounded by network and disk bandwidth rather than by CPU, and why its documentation tells you to leave memory to the OS.

Durability is the other half: Kafka does not fsync on every write by default — it relies on replication across brokers for durability and lets the kernel flush in the background. A system that needed each write on disk before acknowledging (a database’s write-ahead log — see A Transaction, Inside the Engine) would call fsync or fdatasync per commit, at a cost of tens of microseconds to milliseconds each, and would batch commits to amortise it. The same page cache serves both; the fsync policy is the durability decision.

A relational database usually refuses the page cache’s help and manages its own buffer pool (see How Is Database Data Physically Stored? and Pages: The Unit of Everything): it knows which pages are index roots and which are a one-off scan, it needs dirty pages to be written in a specific order relative to the WAL, and it needs eviction decisions to respect transactions. PostgreSQL reads through the page cache into shared_buffers (so hot pages are cached twice — the "double buffering" cost, accepted for the control it gives); InnoDB and most commercial engines open files with O_DIRECT and bypass the page cache entirely, so their buffer pool is the only cache and its size is a first-class tuning parameter.

The consequence for the network path is that a database cannot use `sendfile`: the bytes to send are in its private buffers (and are usually a computed result, not a file region anyway), so query results go out through ordinary send() — user space to socket buffer, one CPU copy — and the engine batches rows into large writes to amortise the syscall. mmap as a buffer-pool strategy (LMDB, early MongoDB, some LSM engines for reads) is the middle road: the page cache is the buffer pool, eviction is the kernel’s, durability still requires msync/fsync, and the well-known paper "Are You Sure You Want to Use MMAP in Your DBMS?" catalogues why most engines that tried it moved away — page-fault stalls, TLB shootdowns, no control over write ordering.

The rule of thumb: let the OS own the cache when data is append-only or immutable, accessed sequentially, and served unchanged (logs, object storage, static files, search-index segments). Own the cache yourself when you need write ordering, transactional eviction, or knowledge the kernel does not have about which pages matter. Both are correct; the mistake is choosing without knowing which regime you are in.

A syscall costs on the order of 100–300 ns to enter and leave on modern hardware, more with mitigations for speculative-execution bugs, plus the cache and TLB disturbance of running kernel code (see System Calls and User Mode vs Kernel Mode). At a million operations per second, one syscall each is a substantial fraction of a core doing nothing but crossing. Every high-throughput system therefore does fewer, larger crossings. The oldest tools are writev/readv and sendmsg/recvmsg: a vector of buffers in one call, so a header and a body, or a hundred small records, cost one crossing and no concatenation copy. sendmmsg/recvmmsg (Linux) go further and move many datagrams per call, which is how QUIC stacks recover UDP’s per-packet cost.

io_uring (Linux ≥ 5.1) is the generalisation: two ring buffers shared between the process and the kernel — a submission queue and a completion queue. The process writes operation descriptors (read this file at this offset, send this buffer on this socket, accept on this listener) into the submission ring and, in the extreme configuration, never makes a syscall at all: a kernel thread polls the ring. Completions appear in the other ring. It is a completion model, like Windows IOCP, and it works for regular files — which The Event-Driven Server noted epoll cannot handle. One crossing can submit hundreds of operations; a database or a proxy can queue a batch of disk reads and socket writes together and wait once.

Batching has a cost the throughput numbers hide: latency for the first item. A record that waits for its batch to fill waits by the batch interval (Kafka’s linger.ms, a database’s group-commit window, Nagle’s algorithm in TCP — see TCP: A Reliable Ordered Byte Stream over an Unreliable Network). Every batching system has a knob that trades those two, and the right setting is workload-specific. Keep the implementation details labelled: io_uring is Linux; IOCP is Windows; kqueue supports aio differently; the concept — amortise the crossing — is what transfers.