Huge Pages: More Coverage per Entry, and What It Costs
If the TLB can only hold so many entries, make each entry cover more memory. That is the whole idea, and it can transform a translation-bound workload — but it costs memory, complicates allocation, and the transparent variety can make things worse.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
The coverage arithmetic
The mechanism is arithmetic rather than magic. A TLB with a fixed number of entries covers entries times page size. Multiply the page size and you multiply the coverage, without adding a single entry or making the TLB any bigger or faster.
A second benefit comes free: because a huge page is described by an entry higher up the page-table tree, the walk terminates one or more levels early. Fewer levels means fewer dependent loads on the miss path, so the misses that do happen are cheaper as well as rarer.
The catch, and the reason this is not simply switched on everywhere, is that the region must be *contiguous and aligned* in physical memory. That is easy on a freshly booted machine and can be very hard on one that has been allocating and freeing for a week.
| Property | Small pages | Huge pages |
|---|---|---|
| Coverage per TLB entry | One small page | Larger by the size ratio — often hundreds of times |
| Walk depth on a miss | Full depth to the leaf | Terminates early at a higher level |
| Physical contiguity required | One small frame | A large aligned contiguous region |
| Memory wasted on a partial mapping | At most part of a small page | Potentially most of a huge page |
| Allocation under fragmentation | Almost always succeeds | May fail, or stall while memory is compacted |
| Granularity of protection and paging | Fine | Coarse — the whole region shares one entry's bits |
What it costs
The costs are real and they are not only about wasted bytes. Internal fragmentation is the obvious one: a mapping that needs slightly more than one huge page consumes two, and a sparsely-touched huge page occupies its full size in physical memory regardless of how little of it you use.
Allocation is the subtler cost. Finding a large aligned contiguous physical region on a fragmented machine may require compaction, and compaction is work that happens at an unpredictable moment — often as a latency spike in the middle of something that was previously smooth. A system that was fast on average can become erratic.
And granularity coarsens. Protection bits, dirty tracking and paging decisions all apply to the whole huge page. An OS that wants to page out a cold portion, or mark a small range read-only, cannot do so without splitting the mapping back down. For workloads that rely on fine-grained memory management, that is a genuine loss.
| Situation | Likely outcome | Why |
|---|---|---|
| Large, dense, long-lived working set | Clear improvement | Coverage was the constraint and the mapping is fully used |
| Large but sparsely touched region | Memory wasted, little gain | Full pages are resident for a fraction of use |
| Small working set already inside coverage | No change, some memory cost | Translation was never the bottleneck |
| Latency-sensitive service on a fragmented host | Possible regression | Allocation stalls and compaction arrive unpredictably |
| Many short-lived processes | Usually a regression | Setup cost is paid repeatedly and never amortised |
Transparent huge pages cut both ways
Transparent mechanisms promote regions to huge pages automatically, without the application asking. When it works, you get the benefit with no code change at all — which is exactly why it is enabled by default in many places.
When it does not work, the failure is unpleasant precisely because it is invisible. Promotion may trigger compaction at an arbitrary moment, producing latency spikes with no corresponding event in the application. Sparse regions get promoted and then sit mostly untouched, inflating resident memory. And because nothing in the application asked for any of it, the cause is genuinely hard to find.
The honest guidance is narrow. Measure first with counters, decide whether coverage is actually your constraint, and then choose deliberately between explicit huge pages for the specific mappings that need them and disabling the transparent mechanism for latency-sensitive services. "Turn it on and see" is a reasonable experiment; "turn it on because it is faster" is not a claim anyone should make without the measurement.
1// nothing in the program mentions pages at all2buffer = allocate(2 GiB) // sparsely touched3// ... system promotes regions to huge pages opportunistically4 5// possible results, none of them visible from here:6// - resident memory far exceeds what was touched7// - an allocation stalls while memory is compacted8// - a latency spike with no application-level cause1// applied only to the region that counters showed was translation-bound2hot = allocate_huge_backed(256 MiB) // dense, long-lived, fully used3cold = allocate(2 GiB) // ordinary pages; sparse, no benefit4 5// coverage improves where it was the constraint6// no promotion machinery runs against the sparse regionSame total memory, same program logic. The difference is that the second version applies the mechanism where a measurement said it would pay, and leaves the sparse region alone — so it gets the coverage benefit without the fragmentation cost or the unpredictable compaction.
Key points
- Coverage is entries times page size; larger pages multiply coverage without needing a larger TLB.
- A huge-page mapping also shortens the walk, because it terminates at a higher level of the tree.
- The cost is physical contiguity: internal fragmentation, allocation failure and compaction stalls.
- Granularity coarsens — protection, dirty tracking and paging all apply to the whole region.
- Transparent promotion can help invisibly and hurt invisibly; measure before enabling it for latency-sensitive work.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Allocator → OS: a large aligned contiguous physical region is requested, explicitly or through transparent promotion.
- 2OS → physical memory: a suitable region is found, possibly requiring compaction if memory is fragmented.
- 3OS → page table: the mapping is installed as a single entry at a higher level, rather than many leaf entries.
- 4MMU → TLB: one entry now covers the whole region, and a walk that reaches this level terminates early.
- 5Program → memory: accesses anywhere in the region reuse that single translation, so the miss rate falls sharply.
- • Treating huge pages as a general speedup rather than a coverage fix for a specific, measurable constraint.
- • Judging the change on mean latency alone; compaction stalls appear in the tail, which is where they matter.
- • Assuming the memory cost is negligible because the region is large — sparse use is exactly when the waste is worst.
- • Concluding transparent promotion is harmless because it is on by default. Defaults are chosen for a general case that may not be yours.
Consequences, controls and cost
- • A translation-bound workload can improve dramatically with no change to its data-cache behaviour whatsoever.
- • Resident memory can rise substantially for sparsely-touched regions, because a partially-used huge page is fully resident.
- • Latency can become less predictable if promotion triggers compaction during a request.
- • Short-lived processes usually lose, because setup cost recurs and never amortises.
- • Fine-grained `mprotect`-style operations may force the mapping to be split back down, undoing the benefit silently.
- • Confirm with TLB counters that coverage is the constraint before changing anything — this is the step most often skipped.
- • Prefer explicit huge-page backing for the specific mappings that measurement implicated, over blanket transparent promotion.
- • For latency-sensitive services on long-running hosts, consider disabling transparent promotion and measuring the tail rather than the mean.
- • Keep huge-page-backed regions dense and long-lived; sparse or short-lived ones pay the costs and collect none of the benefit.
- • Re-measure resident memory as well as latency — the improvement and the regression show up in different numbers.
- • Diff `dTLB-load-misses` and walk-cycle counters with promotion enabled and disabled — that delta is the benefit, stated directly.
- • Track resident set size alongside latency; a large RSS increase for a small latency gain is a bad trade you would otherwise miss.
- • Watch tail latency, not the mean — compaction stalls are rare and large, exactly the shape a mean hides.
- • Check how much of each promoted region is actually touched; sparse promotion is the main source of wasted memory.
- • Coverage for memory: a partially-used huge page is fully resident, so sparse mappings waste substantially.
- • Throughput for predictability: compaction to satisfy an allocation can produce latency spikes with no application-level cause.
- • Simplicity for granularity: one entry covering a large region means protection and paging decisions can no longer be fine-grained.
Scope
§224 — what these claims are specific to.
- PLATFORM-SPECIFICPage sizes, request interfaces, and whether a transparent mechanism exists are properties of the OS and ISA together. Linux transparent huge pages, explicit hugetlbfs and other systems' equivalents behave differently enough that advice does not port between them.
- MICROARCH-SPECIFICWhether huge-page entries come from a separate and smaller TLB pool varies by design, which changes how much coverage you actually gain from a given page-size increase.