The question this answers
I added exponential backoff and the load spikes are still there, just further apart. Why?
Bounds the retry rate of a *single* client and decorrelates attempt times across clients, so that N clients recovering from a common fault present roughly uniform load over the backoff window rather than a spike of N. It does not bound aggregate retry volume (that is Cap Retries as a Fraction of Traffic, Not as a Count per Request) and it does not make a retry safe (that is idempotence).
Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.
A retrying client knows when its own attempt failed and how many attempts it has made. It does not know how many other clients failed at the same moment, and this is the whole problem: the correlation that creates the spike is invisible from every participant’s vantage point. Jitter works precisely because it needs no knowledge of the correlation — each client independently randomises and the aggregate smooths out.
A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.
The failure event is a synchronisation event
Ten thousand clients are talking to a service. The service restarts. Every in-flight request fails within the same few milliseconds — not because the clients coordinated, but because they share a cause. That shared cause has just given them all the same clock start.
Now each one backs off one second and retries. At T+1s the service, which is still cold, receives ten thousand simultaneous requests. It fails again. Each backs off two seconds. At T+3s: ten thousand simultaneous requests. The intervals are growing, and the *peak* is not shrinking at all. Backoff has spread each client’s attempts over time while leaving the clients in phase with each other.
The dangerous property is that a service which needs a quiet minute to warm caches, refill pools and establish connections never gets one. It gets a full-amplitude spike at 1s, 3s, 7s, 15s — the exact load that killed it, delivered repeatedly, and it fails each time in a way that re-synchronises everyone for the next round.
Jitter breaks the phase lock. If each client waits random(0, 1s) instead of exactly 1s, those ten thousand attempts spread across the whole second and the peak instantaneous rate drops by orders of magnitude for the same total volume. Nothing was coordinated. Each client made an independent random choice, and the aggregate became smooth — one of the few places in distributed systems where randomness substitutes for agreement outright.
Three jitter strategies, and why full jitter usually wins
Full jitter — sleep = random(0, min(cap, base · 2^n)) — draws uniformly from zero up to the backoff window. It produces the flattest arrival distribution, which is why it is the usual recommendation. Its cost is variance for the individual client: a draw near zero means retrying almost immediately, and an unlucky client can retry several times in quick succession.
Equal jitter — half + random(0, half) where half = min(cap, base · 2^n) / 2 — guarantees a minimum wait while still spreading. Slightly worse smoothing, more predictable per-client behaviour. Reasonable when you genuinely need a floor on the interval, for example to let a downstream failover complete.
Decorrelated jitter — sleep = min(cap, random(base, previous · 3)) — makes each interval a random function of the previous one rather than of the attempt number. It avoids the shared attempt-counter correlation entirely and tends to recover faster than full jitter at similar smoothing. It is the choice when clients may have started their retry sequences at slightly different attempt numbers.
All three matter far less than the binary question of whether you jitter at all. Going from no jitter to any jitter is the large effect; the choice between them is tuning.
1const BASE_MS = 1002const CAP_MS = 20_0003 4function fullJitter(attempt: number): number {5 return Math.random() * Math.min(CAP_MS, BASE_MS * 2 ** attempt)6}7 8function equalJitter(attempt: number): number {9 const half = Math.min(CAP_MS, BASE_MS * 2 ** attempt) / 210 return half + Math.random() * half11}12 13// Each interval derives from the previous one, not from the attempt number,14// so clients that began at different attempt counts do not re-synchronise.15function decorrelatedJitter(previousMs: number): number {16 return Math.min(CAP_MS, BASE_MS + Math.random() * (previousMs * 3 - BASE_MS))17}18 19// The sleep is spent from the SAME budget as the attempts. A backoff that20// outlives the caller's deadline is not patience, it is wasted capacity:21// see [[timeout-budgets]].22function shouldWait(sleepMs: number, remainingBudgetMs: number, worstCaseAttemptMs: number): boolean {23 return sleepMs + worstCaseAttemptMs < remainingBudgetMs24}Retries are not the only thing that synchronises
Once you see the pattern, correlated timing turns up everywhere, and each instance needs the same treatment. Cron on the hour: every instance runs its refresh job at 00:00:00, so a job that is trivial at any other moment saturates the database sixty seconds a day. TTL expiry: keys written together expire together, which is the stampede in One Key Expires and Five Hundred Instances Miss at the Same Millisecond. Deploys: a rolling restart brings instances up in lockstep, and they all warm their caches and open their connection pools simultaneously. Partition healing: when a network partition ends, every isolated node reconnects at the same instant — the fault’s end is as synchronising as its beginning.
The remedy is identical in each case, and it is cheap: add a random offset the same size as the natural interval. Spread cron start times across the window. Jitter TTLs by ±10%. Stagger rolling restarts. Randomise reconnect delays. None of these need coordination between the participants, which is exactly why they scale.
There is one more source worth naming because it is the least expected: client libraries with identical defaults. A thousand services using the same SDK with the same 1-second reconnect interval are a synchronised population that no single team created. Defaults are a distributed-systems decision, and jitter belongs in the default rather than in the documentation.
| Source of synchronisation | What lines up | Fix |
|---|---|---|
| A shared failuretypical | All clients start their backoff at the same instant | Jitter the retry interval |
| Cron at a round timetypical | All instances run the job at the same second | Random offset within the period |
| TTLs written togethertypical | All keys expire in the same millisecond | Jitter the TTL by ±10% |
| Rolling restarttypical | Cold caches and pool warm-up in lockstep | Stagger, and readiness-gate on warm |
| Partition healstypical | Every isolated node reconnects at once | Jittered reconnect and connection rate limit |
Key points
- A shared failure is a synchronisation event: it gives every affected client the same clock start.
- Exponential backoff without jitter spaces one client’s attempts while leaving all clients in phase — the peak load is unchanged.
- Jitter achieves aggregate smoothing through independent local randomness, with no coordination between clients at all.
- Full jitter smooths best; decorrelated jitter recovers faster; any jitter beats none by far more than they differ from each other.
- Retries are one of several synchronisation sources — cron, TTL expiry, rolling restarts and partition healing all need the same treatment.
The chain, answered
Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.
- • An attempt fails and the client computes a nominal backoff window that grows exponentially with the attempt number, up to a cap.
- • Instead of sleeping for the window, the client draws a random sleep from within it.
- • Before sleeping, it checks the remaining deadline: if the sleep plus a worst-case attempt exceeds the budget, it gives up now rather than waking to a request nobody wants.
- • The retry is also checked against the dependency’s retry budget, so volume is bounded independently of timing.
- • The cap prevents unbounded growth, and the randomness prevents the population from re-synchronising at the cap.
- • Jitter is applied to the backoff but not to the first retry, so attempt #1 is still a synchronised spike.
- • The random source is seeded identically across instances — from a fixed seed or a startup timestamp — and the "random" delays are the same everywhere.
- • Backoff sleeps outlive the caller’s deadline, so the retry is issued for a request nobody is waiting on.
- • The cap is reached by all clients, which re-synchronises them at the cap interval unless jitter is applied there too.
- • Backoff is implemented at one layer while another layer retries immediately, and the immediate one dominates.
- • Periodic spikes at the backoff intervals: request-rate graph shows sharp peaks at 1s, 3s, 7s, 15s after an incident begins, with near-zero troughs between. This is the unmistakable no-jitter fingerprint.
- • A service that cannot complete a restart: it comes up, is immediately saturated by a synchronised retry wave before caches warm, fails readiness, and is restarted — a crash loop caused entirely by client timing.
- • Load spike exactly on the hour, every hour, with a flat baseline otherwise. Nobody deployed anything; a cron entry is synchronised across the fleet.
- • Connection storm on partition heal: the network recovers and the database immediately hits its connection limit, so the recovery event itself causes a second outage.
- • None — and that is the point worth internalising. Jitter is the canonical example of solving a distributed timing problem with local randomness instead of agreement.
- • A coordinated alternative (a scheduler assigning each client a slot) achieves better smoothing and requires a coordinator that must be available during the incident. Almost never worth it.
- • The only shared thing needed is a decent independent random source per process, which is why a shared seed silently defeats the entire mechanism.
- • Individual request latency gets worse and more variable: jitter trades per-request predictability for aggregate stability.
- • Total retry volume is unchanged — jitter reshapes arrivals, it does not reduce them. Pair it with a budget.
- • The recovering service gets quiet windows in which to warm up, which is usually the difference between recovering and crash-looping.
- • Detect: look at request rate at sub-second resolution. Synchronisation is invisible at 1-minute granularity, which is why it is so often missed.
- • Contain: during an incident, the fastest lever is a connection or request rate limit at the edge, which flattens the wave even if clients will not.
- • Recover: allow the service to come up behind a readiness gate that only opens when caches and pools are warm, so a synchronised wave meets a warm instance.
- • Reconcile: nothing to reconcile — this is purely a timing pathology with no correctness consequence.
- • Verify: fail a dependency in a load test with a thousand simulated clients and plot arrival rate at 10ms resolution. Peaks mean somewhere in the stack is not jittering.
- • Request rate at sub-second resolution during the first minute of an incident — the only way to see the spike shape at all.
- • The coefficient of variation of inter-arrival times: near zero means clients are in phase, near one means well-jittered.
- • Time between attempts as a distribution rather than an average; a healthy schedule looks broad, a synchronised one looks like spikes.
- • Connection establishment rate, separately from request rate — reconnect storms show up here first.
- • Any retry path with more than a handful of clients, which is essentially all of them.
- • Anything periodic across a fleet: scheduled jobs, cache refreshes, token renewals, heartbeat intervals.
- • Recovery paths specifically — reconnects, failovers, restarts — where the population is maximally synchronised and the target is maximally fragile.
- • Latency-critical single-client paths where the extra variance matters more than the aggregate smoothing, and there is no population to desynchronise.
- • When jitter is used as a substitute for a volume cap: a smooth 30× load is still 30×, and the smoothness can make it harder to spot.
- • When the backoff window grows past the caller’s deadline, in which case backing off at all is just a delayed way of wasting the attempt.
- • A retry budget, which bounds volume rather than reshaping timing — orthogonal, and you want both. See Cap Retries as a Fraction of Traffic, Not as a Count per Request.
- • A circuit breaker: stop retrying entirely for a while, which is the extreme form of backoff and also the most effective smoothing of all.
- • Server-driven pacing: return
Retry-Afterwith a value the server jitters itself, moving the decision to the party that knows its own recovery state. - • Client-side rate limiting on connection establishment, which handles reconnect storms that retry backoff never sees.
Backoff without jitter reproduces the spike one second later
full sleep = random(0, min(cap, base · 2^n)) equal sleep = half + random(0, half), half = min(cap, base · 2^n) / 2 decorrelated sleep = min(cap, base + random(0, previous · 3 − base)) none sleep = min(cap, base · 2^n) <- every client, the same value
What people believe, and what is true
We have exponential backoff, so we are protected from retry spikes.
Exponential backoff without jitter keeps the whole client population in phase. The spikes get further apart and stay exactly as tall.
Jitter reduces retry load.
It redistributes retry load in time. The total is unchanged — bound that separately with a budget.
Randomising by a few milliseconds is enough.
The jitter must be comparable to the interval you are spreading. Jittering a one-second backoff by ±5ms leaves the spike essentially intact.
Backoff is free, so back off for as long as you like.
The sleep comes out of the caller’s deadline. Waking up after the caller has gone means you spend a request on work nobody will read.
Go deeper
Only the levels this lesson can honestly fill — a missing level is a claim nobody had.
Overview
Everyone failed at the same moment, so everyone will retry at the same moment. Add randomness to each wait so the retries spread out instead of arriving as one wall.
Practical
Use full jitter — random(0, min(cap, base·2^n)) — on every retry including the first, cap the growth, and check the remaining deadline before sleeping. Then apply the same idea to cron offsets, TTLs, reconnects and rolling restarts, and confirm at 10ms resolution that no spikes remain.
Advanced
The population is a point process, and a common fault makes it a delta function. Convolving with a uniform kernel of the same width as the backoff window flattens the peak by roughly the ratio of window to original burst width. Backoff without jitter convolves with a shifted delta — it translates the peak and preserves its height, which is precisely why the load graph shows the same amplitude at 1s, 3s and 7s.
Apply it
- 💬 You added exponential backoff and the load graph still shows spikes, now at 1s, 3s and 7s. Diagnose it.
- 💬 Why does jitter work without any coordination between clients?
- 💬 Name three sources of synchronised load that are not retries.