Allocation Rate Is a Cost Even Without a Leak
Memory that is allocated and immediately freed never shows up as growth, so leak hunting finds nothing. It still costs: every megabyte allocated is a megabyte the collector must eventually walk, and at 500 MB/s that is where your latency went.
Frame the diagnosis
Performance work starts from a symptom and a signal — never from a resource dashboard.
Churn is invisible to every memory-growth signal
A memory leak is retained memory: allocated, still reachable, never freed, and visible as an upward slope in resident set size. Churn is the opposite shape — allocate, use briefly, drop the reference, collect — and it produces a perfectly flat memory graph. Every dashboard designed to catch leaks reports health while the process allocates hundreds of megabytes a second.
The cost is not the memory; it is the collector's work. Most collectors do work proportional to allocation volume and to the number of surviving objects, so a high allocation rate means frequent collections, and frequent collections mean either CPU spent collecting instead of serving, or pause time, or both, depending on the collector's design. A service at 500 MB/s allocation can easily spend 20–30% of its CPU in GC without a single byte leaking.
The relationship to Garbage Collection: Pause, Throughput, Footprint — Pick Two matters: tuning heap sizes and collector parameters treats the symptom, and is sometimes the right call under time pressure. Allocating less treats the cause and is usually a smaller change than people expect, because allocation is dominated by a handful of hot paths — the same concentration that makes CPU profiling effective.
| Signal | Value | What it tells you | Verdict |
|---|---|---|---|
| resident set size | 2.1 GB, flat for 6 days | No leak. This is what fools people. | normal |
| heap after collection | stable, ~400 MB | Live set is small and constant | normal |
| allocation rate | 480 MB/s | The process churns its entire live set every second | smoking gun |
| GC CPU share | 24% | A quarter of the CPU is collecting, not serving | smoking gun |
| GC frequency | 38/s (was 3/s) | Collections triggered constantly by allocation volume | smoking gun |
| p99 latency | 340 ms (p50 22 ms) | Tail dominated by pauses, median untouched (Tail Latency: Why p50 Being Fine Does Not Help) | suspect |
Reading an allocation profile
An allocation profile is structurally the same as a CPU profile — stacks, aggregated — but weighted by bytes (or object count) at the allocation site rather than by samples on a timer. It answers "which code path is producing the garbage", and the answer is usually concentrated: a serializer, a string-building loop, a per-request buffer, or a defensive copy inside something hot.
Bytes and object count are different rankings and both matter. A path allocating a few enormous buffers is a bytes problem, usually fixable with reuse or pooling. A path allocating millions of tiny objects is a count problem — the collector's cost is often driven more by object count and pointer-chasing than by raw bytes, so a profile sorted only by bytes can miss the more expensive path.
The most common findings are unglamorous. Defensive copies of collections that nobody mutates. String concatenation in a loop. Boxing values to put them in a generic container. Parsing a request body into an intermediate representation before converting it again. Each is invisible in a CPU profile — the allocation itself is fast — and each shows up immediately when the profile is weighted by bytes.
BYTES/s OBJECTS/s SITE
310 MB 2.1 M json.deserialize -> makeIntermediateMap
(parsed body copied into a map, then into a struct)
94 MB 8.4 M scoring.normalizeWeights -> box(float)
(millions of tiny boxed values -- the object-count problem)
48 MB 0.1 M response.buffer.allocate
(fresh 512 KB buffer per request; poolable)
18 MB 0.9 M log.format -> stringConcat in loop
10 MB 0.4 M (everything else)
---
480 MB/s 11.9 M/s totalAllocating less, without rewriting everything
The fixes rank consistently. Stop creating the intermediate representation — parsing straight into the final shape removes the largest allocator in most profiles and is usually a local change. Reuse buffers via pooling for large, fixed-size, per-request allocations, with the caveat that pools introduce lifetime bugs if a buffer escapes its request. Avoid boxing and defensive copying on hot paths, which is language-specific and often a one-line change with an outsized effect on object count.
Then the ones to be careful with. Increasing heap size reduces collection frequency and can be the correct emergency mitigation, but it trades memory for GC frequency without reducing the work per collection, and larger heaps can mean longer pauses depending on the collector. Changing collectors is a real lever with real trade-offs — throughput versus pause time versus memory overhead — and belongs in Garbage Collection: Pause, Throughput, Footprint — Pick Two rather than being applied as a reflex.
Whatever you change, the validation is allocation rate per request rather than total allocation rate, because traffic moves and total rate moves with it. Bytes per request is the efficiency number that stays honest across a traffic change, and it is the one to put on a dashboard (Capacity or Efficiency: Which Problem Are You Solving?).
1function handle(req: Request) {2 const raw = JSON.parse(req.body) // 1: full intermediate object3 const map = new Map(Object.entries(raw)) // 2: copy into a map4 const order = mapToOrder(map) // 3: copy into the real shape5 6 const buf = Buffer.alloc(512 * 1024) // fresh 512 KB, every request7 const weights = items.map((i) => i.score) // boxed floats, one array per call8 return serialize(order, buf, weights)9}1const bufPool = new BufferPool(512 * 1024)2 3function handle(req: Request) {4 const order = parseOrder(req.body) // straight to the final shape5 const buf = bufPool.acquire() // reused; released in finally6 try {7 // reuse a preallocated typed array instead of boxing per call8 fillWeights(scratchWeights, items)9 return serialize(order, buf, scratchWeights)10 } finally {11 bufPool.release(buf)12 }13}The allocation profile drops from ~480 MB/s to a fraction of it without any algorithm changing. The risk moves too: pooled buffers must not escape the request, so this trades a GC problem for a lifetime-discipline problem that needs a test.
Key points
- Churn allocates and frees continuously, so resident memory stays flat and every leak-detection signal reports health while the collector burns CPU.
- The cost is collector work proportional to allocation volume and surviving objects — 500 MB/s can mean 20–30% of CPU spent collecting.
- Rank allocation profiles by bytes *and* by object count: millions of tiny objects can cost more than a few large buffers.
- The usual culprits are intermediate representations, defensive copies, boxing and string building in loops — all fast individually and invisible in a CPU profile.
- Validate on bytes allocated per request, not total allocation rate, which moves with traffic and hides efficiency changes.
Follow the diagnosis
The causal chain, hop by hop — and the readings that invite the wrong conclusion.
- 1Request → handler: each request parses the body into an intermediate map before converting to the final struct.
- 2Handler → heap: three copies plus a fresh 512 KB buffer per request produce ~480 MB/s of short-lived garbage.
- 3Heap → collector: allocation volume triggers collections 38 times a second instead of 3.
- 4Collector → latency: each collection steals CPU and, depending on the collector, pauses the process, adding a second bump at p99.
- 5Dashboards → engineer: resident memory is flat, so a leak hunt finds nothing and the investigation stalls.
- • "Memory is flat, so memory is not the problem." Flat memory rules out a leak and says nothing about churn.
- • "GC is high, so we need a bigger heap." A bigger heap reduces collection frequency without reducing the work per byte allocated; it buys time and can lengthen individual pauses.
- • "The CPU profile does not show allocation as hot." Individual allocations are cheap. The cost is deferred to the collector, which is a different set of frames or none at all.
- • "Allocation rate went up, so we have a leak." Rate and retention are independent — high rate with a flat live set is churn, not a leak (Leak or Unbounded Cache? The Question That Picks the Fix).
Measure, fix, validate
An optimization is not finished until the metric that motivated it has moved.
- • Record allocation rate (bytes/s and objects/s) and divide by request rate to get the per-request efficiency number.
- • Track GC CPU share and collection frequency alongside it — the mechanism by which allocation becomes latency.
- • Take an allocation profile weighted by bytes, then a second sorted by object count, and compare the top entries.
- • Correlate GC pause timestamps against the latency histogram to confirm the tail bump is collection rather than something else ([[tail-latency]]).
- • Eliminate intermediate representations: parse directly into the shape you need, which is usually the single largest entry in the profile.
- • Pool large, fixed-size, per-request buffers — with a test that catches a buffer escaping its request, because that bug is nastier than the one you are fixing.
- • Remove boxing and defensive copies on hot paths; these are usually small changes with disproportionate effect on object count.
- • Tune heap or collector only as mitigation or after the allocation work is done, and state explicitly which trade you are making ([[garbage-collection]]).
- • Bytes allocated per request should fall, and stay fallen when traffic changes — the honest efficiency measure.
- • GC frequency and GC CPU share should drop proportionally; if they do not, the remaining allocation is elsewhere in the profile.
- • The p99 bump attributable to pauses should shrink while p50 stays roughly unchanged — that asymmetry confirms the mechanism.
- • Re-take the allocation profile and confirm the top site changed rather than merely shrinking a little.
- • Buffer pooling introduces lifetime and aliasing bugs, which are harder to debug than the GC pressure they remove.
- • Parsing directly into the target shape couples the parser to the domain model and can hurt readability and reuse.
- • Avoiding boxing often means less idiomatic, more type-specialized code that the next engineer may revert.
- • Larger heaps trade memory cost — and potentially longer individual pauses — for lower collection frequency.
- • Put bytes-allocated-per-request on the service dashboard and alert on step changes after deploys.
- • Add a CI check for allocation on the hot path where the runtime supports it — allocation regressions are easy to introduce with an innocuous refactor.
- • Alert on GC CPU share, which catches this class of problem generically regardless of which code path caused it.
- • Include the allocation profile in the incident review so the shape is recognizable next time (Debugging an Incident in Progress).
Accuracy
Performance numbers are conditional. These are the conditions.
- ILLUSTRATIVEThe 480 MB/s profile and the 24% GC share are constructed to show the shape and the arithmetic.
- RUNTIME-SPECIFICHow allocation rate translates into GC cost and pause time depends entirely on the runtime and collector — generational, concurrent and reference-counted designs behave very differently under the same churn.