Memory Leaks in Backend Services
Distinguishing a leak from ordinary heap growth, finding the reference that retains, and doing it on a live process.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.
Memory climbs steadily until the process restarts. Is that a leak, and how do you find what is holding the references?
Instances are being killed every few hours by the platform for exceeding their memory limit. Requests in flight are lost each time. Restarting on a schedule is currently keeping the product alive.
It grows and then gets killed, so it is a leak. Increase the memory limit and add a nightly restart.
Managed runtimes grow their heap on purpose: the collector defers work while there is headroom, so rising memory is the *expected* shape even with no leak.
- Managed runtimes grow their heap on purpose: the collector defers work while there is headroom, so rising memory is the *expected* shape even with no leak.
- Raising the limit changes the interval between restarts and nothing else, and it delays the point at which anyone investigates.
- Scheduled restarts hide the cost until the growth rate increases — then the restarts arrive during peak traffic instead of at 04:00.
- A real leak eventually degrades before it kills: collection runs more often and takes longer, so latency rises well before the process dies.
- A bounded cache that fills to its limit looks exactly like a leak on a memory graph and is entirely correct behaviour (Local vs Distributed Cache).
What is actually happening
- In a garbage-collected runtime a "leak" is not unfreed memory — it is memory that is still reachable and will never be used again. The collector is working correctly; something is holding a reference.
- The distinguishing measurement is live heap after a full collection. Total heap sawtooths and tells you nothing; a live-heap floor that rises monotonically across collections is a leak.
- Growth that plateaus is not a leak. A cache filling to its bound, a connection pool reaching steady state, or a runtime settling into its working set all plateau.
- The usual retainers are long-lived containers: an unbounded map or cache, a growing array of listeners or subscriptions, an interval or timer never cleared, a closure captured by a registered callback, request-scoped data stored in a module-level structure (Request Context Propagation).
- Not all memory is heap. Native buffers, memory-mapped files, thread stacks and allocator fragmentation can grow while the heap looks flat — which is why a container limit can be exceeded with a healthy-looking heap graph.
- The container limit is enforced on the whole process, not on the heap. A runtime configured with a heap ceiling above the container limit will be killed before it ever decides to collect aggressively.
Growth, caches and leaks look the same on the graph
The first job is not to find the leak. It is to establish that there is one. Three very different situations produce a rising memory line, and only one of them is a bug.
The measurement that separates them is the live heap immediately after a full collection. Ordinary growth sawtooths back to a stable floor. A filling cache raises the floor and then stops. A leak raises the floor and does not stop.
heap_total
512M | ....--'''
384M | ..--'''
256M | .-'''
+--------------------
"memory is going up, we are leaking"heap_live_after_gc
cache filling leak
120M | __-------- | /
80M | / | /
40M | / | /
+---------- +----------
rises, then flat rises without boundThe floor is the only line that distinguishes "the collector has not needed to work yet" from "something is retained forever". Acting on the total heap graph leads teams to bound caches that were fine and to raise limits that were not the problem.
Where the reference usually lives
Leaks in backend services are not exotic. They are almost always one of a handful of long-lived containers holding on to something request-scoped, and the fix is usually a bound or a matching removal rather than a redesign.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
A Map or dictionary keyed by user, session or request id with no eviction | Growth proportional to unique users, not to time | Unbounded cache used as memoization | Bound it with a size limit and an eviction policy, or move it out of process (Local vs Distributed Cache) |
| Event listeners or subscriptions added per request | Listener count climbs; growth proportional to request count | Registration without a matching removal | Count registrations as a gauge; every on needs an off on every path including errors |
| A timer or interval created per request or per connection | Steady growth proportional to time and traffic, unaffected by load shedding | Timer holds its closure, which holds the request context | Clear timers in a finally; audit anything scheduled inside a handler |
| Request context stored in a module-level structure | Growth proportional to requests; snapshots full of request objects | Per-request data promoted to process scope (Stateless Services) | Use the runtime's request-scoped context mechanism instead of a global |
| A queue or buffer that grows when consumers are slow | Growth correlates with downstream latency, not with traffic | Missing backpressure (Backpressure) | Bound the buffer and reject or block producers when it is full |
| Accumulated error objects, log buffers or metric labels | Growth tracks error rate or endpoint variety | High-cardinality labels retained indefinitely (The Metrics a Backend Must Emit) | Template the labels; never put ids in metric label values |
| Native buffers or file handles never released | RSS grows, heap flat, file descriptor count climbs | Streams or handles not closed on error paths | Watch the descriptor count as its own gauge; close in finally |
Diagnosing a live process without killing it
The instinct to reproduce a leak locally usually fails, because the leak is proportional to production traffic patterns, data shapes and uptime. The answer is to take the measurement in production, carefully.
The safe procedure is to remove one instance from rotation, let it drain, then take snapshots on an instance that no longer serves users. This costs one instance of capacity and removes essentially all of the risk.
- 1Confirm with the floor
Plot live heap after collection over several hours
fails by Skipped, and the team spends a day bounding a cache that was healthy
- 2Correlate the growth
Compare growth rate against request rate and against elapsed time
fails by Per-request and per-time leaks have different suspects; conflating them widens the search
- 3Drain one instance
Remove from the load balancer, let in-flight requests finish (Graceful Shutdown)
fails by Snapshotting a serving instance pauses it and can trigger health-check removal anyway — uncontrolled
- 4Snapshot twice
One now, one after a controlled interval or synthetic load
fails by Snapshots under different workloads diff to noise
- 5Diff by type and retainer
Find the object types that grew and walk the retaining path to a root
fails by Stopping at "many strings"; the retainer path is the answer, not the type
- 6Fix the pair or the bound
Add the missing removal, or bound the container
fails by Bounding too aggressively and moving the load onto a dependency
- 7Verify over a full cycle
Watch the floor for at least as long as the previous growth took
fails by Declaring victory right after a deploy, when memory is reset for unrelated reasons
How to build it
Most important first.
- Measure live heap after collection first. Everything else is guessing until that graph exists, and it settles leak-versus-growth in one look.
- Bound every long-lived collection. A cache without a maximum size and an eviction policy is a leak that has not happened yet (TTL and Expiry).
- Pair every registration with a removal: event listeners, subscriptions, timers, intervals, abort handlers. The leak is almost always the missing half of a pair.
- Take two heap snapshots at different times under similar traffic and diff them. The types that grew between them name the retainer far faster than reading code.
- Set the runtime heap ceiling below the container limit, so the collector gets a chance to act and you get an in-process error instead of an opaque kill (Containerizing a Backend).
- Keep the scheduled restart as a mitigation while you investigate — it is a legitimate way to buy time — but treat it as an open incident, not a solution (Graceful Shutdown).
What can go wrong
- Taking a heap snapshot on a live instance and stalling it long enough to fail health checks and be removed from rotation.
- Diffing snapshots taken under different traffic, so the difference is workload rather than leak.
- Finding a large object and assuming it is the leak, when it is a correctly-sized cache and the actual leak is many small objects.
- Fixing one retainer while a second remains, so growth slows and the graph looks convincingly better for a day.
- A "fix" that bounds a cache so aggressively that hit rate collapses and the database absorbs the difference.
- Concurrent requests each adding to the same unbounded structure make growth proportional to concurrency, so a leak can appear suddenly when concurrency rises rather than when code changes (Backend Races).
- A snapshot taken mid-request captures partial state, which can make short-lived allocations look like retained ones.
- Heap snapshots contain everything the process holds: tokens, session identifiers, request bodies, personal data. They are credential-bearing artefacts and need restricted storage and short retention.
- A leak that retains request-scoped data is also a data-retention problem — personal data lives far longer than intended, and outlives deletion requests (The Trust Boundary).
- Memory exhaustion is a denial-of-service vector when an unauthenticated request can allocate proportionally to input size; bound request bodies and result sets (Request Bodies and Streaming).
- Exposing a heap-dump or profiling endpoint on the application port makes it reachable by anyone who can reach the service. Bind diagnostics to a separate interface with its own authorization.
- "Memory is growing, so we have a leak." Growth is the default in a managed runtime. The leak signal is the live-heap floor, not the total.
- "We fixed the leak, memory is flat now" — after a deploy. Every deploy resets memory. Give it a full growth cycle before believing it — memory returns to a low floor after every restart regardless of the leak.
- "Garbage collection means leaks are impossible." Collection removes *unreachable* objects. Every leak in a managed runtime is a reachable object nobody will use.
- "RSS is what matters." RSS includes shared pages and allocator-retained free memory; a rising RSS with a flat live heap is a different problem with a different fix.
- "Restarting is a hack." A supervised restart is a legitimate mitigation with a real cost. The mistake is closing the ticket, not the restart.
Operating it
- Live heap after collection, as a floor line. Rising floor across hours is the diagnosis.
- Collection frequency and total pause time. Both rise before the process dies, and both show up as latency first (Why Is My API Slow?).
- Resident set size (RSS) alongside heap. RSS growing while heap is flat points at native memory, buffers or fragmentation rather than object retention.
- Container OOM kill events and restart counts, correlated with deploys (Deploys Are the First Suspect).
- Object counts by type from two snapshots — the diff, not the absolute numbers.
- Growth rate versus request rate. A leak proportional to requests points at a per-request retainer; a leak proportional to time points at a timer or subscription.
- Higher traffic makes a per-request leak reach the limit sooner, which is often why a leak that existed for months appears the week traffic grew.
- More instances spread the same leak across more processes, so the fleet looks fine on average while individual instances cycle — visible only in per-instance graphs.
- At small scale, a nightly restart genuinely is an acceptable engineering answer for a low-stakes internal service. Saying so is more honest than pretending otherwise.
- Bounding caches costs hit rate, and the lost hits become load on the database or dependency.
- Heap snapshots in production are the only reliable source of truth and carry a real risk of pausing the process. Drain the instance first.
- Continuous memory profiling costs a few percent of CPU permanently in exchange for having the answer before the next incident.
Where this applies
Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.
- RUNTIME-SPECIFICWhat a leak *is* differs by runtime. In V8 (Node) it is a reachable object in a long-lived scope, and the tool is a heap snapshot diff. On the JVM it is the same idea with different tooling and generational behaviour that can mask it for longer. In CPython, reference counting frees most garbage promptly, so leaks concentrate in reference cycles containing objects with finalizers, and in C-extension allocations the interpreter cannot see. In C++ there is no collector at all and a leak is genuinely unfreed memory (C++ Backend Services).
- CLOUD-SPECIFICContainer platforms enforce a memory limit on the whole process and kill it without warning; a VM with swap degrades gradually instead. The same leak therefore presents as an abrupt restart in one environment and as creeping slowness in the other.
- SIMPLIFIEDTreated here as one heap. Real runtimes have generations, arenas and native allocations that behave differently and account for much of the confusion between RSS and heap.
Where the depth lives
This domain teaches the application-side mechanism and hands the rest off.
- — Programming Languages & Runtime Internals — how each collector decides what is reachable, and why generational collection can hide a leak for hours.