Memory Mapping

mmap(addr, len, PROT_READ, MAP_SHARED, fd, offset) creates a region in the address space whose pages are backed by the file: virtual page N of the region is byte range offset + N × 4096 of the file. No data is read at that moment; the kernel records the region (a VMA with the file and offset, The Virtual Address Space) and returns a pointer. The first access to each page faults (Page Faults); the handler finds the page in the page cache — reading it from disk if it is not there — and installs a PTE pointing at that page-cache frame directly. There is no copy: the process’s virtual page *is* the page cache page.

From then on the file is memory. p[1_000_000] is a load; memcpy over it is a sequential read with readahead; a struct overlaid on it is a parser with no parsing. munmap removes the region; msync forces dirty pages to disk; madvise tells the kernel the access pattern (MADV_SEQUENTIAL, MADV_RANDOM, MADV_WILLNEED to prefetch, MADV_DONTNEED to drop).

Everything that makes mmap efficient is the page cache’s doing. Two processes mapping the same file map the same frames, so a shared library’s code, a database’s data files or a dataset used by ten workers exist in RAM once. read() on the same file copies out of the same frames, so mapped and unmapped access are coherent. Reclaim (Memory Pressure, Swap and the OOM Killer) treats mapped file pages like any file pages: clean ones can be dropped and refetched, which means a mapped 10 GB file on an 8 GB machine works — the kernel keeps the hot part resident and the process never manages a cache.

Writes distinguish the two mapping modes. With MAP_SHARED a store dirties the page-cache page; the kernel writes it back on its own schedule (writeback threads, dirty_expire_centisecs, or when dirty pages exceed dirty_ratio), and msync(MS_SYNC) forces it now — the mapped-file equivalent of fsync. Every other mapper sees the store immediately, because it is the same frame. With MAP_PRIVATE the first store to a page triggers copy-on-write (Copy-on-Write): the process gets a private anonymous copy, the file is never modified, and the copy consumes anonymous memory that only swap can evict. Executables and libraries are mapped private: code shared, data copied on first write.

The appeal is concrete. Zero read syscalls: after the first fault, accessing a page is a load, not a trap — a hash index over a mapped file does lookups at memory speed. Zero copies: read() moves bytes page cache → user buffer; mmap has no user buffer. Free caching: the kernel’s reclaim is the buffer manager, sized automatically to whatever RAM is not otherwise used, shared across processes. Simplicity: the storage engine is pointer arithmetic over one array. LMDB (behind OpenLDAP and many embedded uses) is built entirely on a read-only shared mapping with copy-on-write B-tree pages; SQLite offers it as an option; Kafka and many log-structured stores rely on the page cache and sendfile in the same spirit; MongoDB’s original MMAPv1 engine and early LevelDB used it for reads.

The same properties apply beyond databases: language runtimes map executables and shared objects (that *is* exec and dlopen), JITs map code caches, Go and Rust allocators obtain arenas by anonymous mmap, and the Zero-Copy: Serving a File Without Touching It and Memory Mapping, the Page Cache and Network I/O lessons show mapped buffers handed to the NIC without a copy.

The costs are exactly the mirror of the benefits. Faults at unpredictable times: a "memory access" can become a 10 ms disk read, inside a critical section, while holding a lock, in a thread that thought it was doing arithmetic — and there is no error path, no timeout, no way to cancel; the thread is in D state. read() at least tells you where the I/O happens. No control over writeback: with MAP_SHARED the kernel may write a dirty page at any moment and in any order, so a database that needs "the log page before the data page" cannot rely on mapped writes and must msync in the right order or use write + fsync. Transactional safety therefore usually means the engine uses mmap for *reads* and explicit I/O for writes — the 2022 paper "Are You Sure You Want to Use MMAP in Your DBMS?" catalogues why several engines moved away from it, and PostgreSQL never used it for data.

Further sharp edges: SIGBUS if the file is truncated under the mapping; I/O errors surface as signals rather than return codes; a 32-bit process cannot map more than 2–3 GB at once and must window through a large file; every mmap/munmap is a VMA change and a TLB shootdown across all threads (The TLB); and the page-cache "buffer manager" has no idea which of your pages matter — the backup job that flushes it takes your working set with it. Huge mapped regions also carry page-table costs (Page Tables) that explicit buffers do not.

Windows exposes the same mechanism as two calls: CreateFileMapping makes a section object from a file (or the page file for anonymous shared memory), and MapViewOfFile maps a window of it into the address space; FlushViewOfFile is msync, and UnmapViewOfFile is munmap. The semantics — page cache backing, copy-on-write private views, coherence with ReadFile — match Unix closely, which is why cross-platform engines can offer mmap on both.