Garbage Collection: Pause, Throughput, Footprint — Pick Two

A garbage collector must find unreachable objects and reclaim them. Every design decision about *how* trades three quantities against each other: how long the application is stopped (pause time), how much total CPU is spent collecting rather than serving (throughput), and how much memory the heap occupies above the live set (footprint).

The trades are concrete. Collecting concurrently with the application shrinks pauses but costs throughput, because the collector must coordinate with a mutating heap and do extra bookkeeping on writes. Collecting less often improves throughput but raises footprint and makes each collection longer. Giving the heap more headroom reduces collection frequency and raises memory cost. There is no configuration that improves all three, which is why "GC tuning" that produces a dramatic win in one dimension usually paid for it in another that nobody measured.

Not every runtime plays the same game. CPython uses reference counting for the common case with a cycle collector on top, so it has no large stop-the-world pause but pays a continuous per-operation cost and struggles with reference cycles — see CPython Performance: The Interpreter Tax and the GIL. C++ has no collector at all, moving the cost to deterministic destruction and allocator behaviour (C++ Memory Performance: Allocation, Copies and Locality). Claims about "GC pauses" are claims about a particular runtime, and should be labelled as such.

Collector flags get the attention, but the input to the whole system is allocation rate. A service allocating 50 MB/s and one allocating 500 MB/s on the same collector and heap configuration have completely different GC behaviour: the second collects ten times as often, spends roughly ten times the CPU on collection, and produces ten times as many opportunities to pause during a request.

This makes allocation profiling the highest-leverage measurement here, not GC log analysis. GC logs tell you the collector is working hard; an allocation profile tells you *which code path* is making it work hard, which is the actionable version (Allocation Rate Is a Cost Even Without a Leak). The usual culprits are boring: string concatenation in loops, JSON serialization of large payloads, defensive copying of collections, boxing in hot paths, and per-request object graphs that could be reused.

The connection to Payload Size: 20KB, 200KB, 5MB is worth naming: an endpoint returning a 5 MB response allocates that payload at least once and often several times through serialization. Trimming the response is simultaneously a network fix, a serialization-CPU fix and a GC-pressure fix. The same change shows up in three different dashboards, which is a good sign you have found a real cause rather than a symptom.

GC pauses are invisible to application instrumentation by construction: the application is stopped, so it cannot record anything. What you see in a trace is a *gap* — a parent span whose duration exceeds the sum of its children with nothing accounting for the difference. That unexplained gap is the fingerprint, and it is the reason a waterfall is more useful here than a metric.

Confirming it takes one correlation: overlay GC pause events on the latency timeline and check whether the slow requests coincide with collections. Coincidence is not proof — a periodic batch job could produce the same rhythm — but combined with an allocation profile pointing at a hot path, it is usually enough to act on. Keep Correlation Is Not the Root Cause in mind: the honest statement is "requests overlapping a collection pause are slow", and the test is whether reducing allocation reduces the spikes.

One practical note on tail sampling: if your tracing samples traces at a fixed low rate, you will rarely capture the affected requests, because GC-affected requests are by definition a small fraction. Tail-based sampling that keeps slow traces is what makes this diagnosable at all (Sampling Without Throwing Away the Evidence).