Latencytail latencyp99fan-outamplificationscale

Tail Latency: Why p50 Being Fine Does Not Help

A one-in-a-hundred slow response sounds harmless until a page makes 40 calls, a user makes 30 page views, and every dependency has its own one-in-a-hundred. Rare events compose, and at scale the tail becomes the typical experience.

Follow the diagnosis

Frame the diagnosis

Performance work starts from a symptom and a signal — never from a resource dashboard.

Diagnostic question
Why do users report constant slowness when p50 is healthy — and how does a rare slow response become a common bad experience?
Symptom
p50 is 40 ms and stable. Users describe the product as "randomly slow", and the complaints do not correlate with any single endpoint or customer.
Signal
The **p99 of the dependency spans**, combined with the **fan-out count** per user-visible operation, confirms it. Per-service p50 is the signal that misleads: every service looks healthy in isolation while the composed request is not.
SymptomSignalMeasurementHypothesisEvidenceRoot CauseChangeValidationRegression Check

Rare events compose

Take a service whose p99 is 1 second: one call in a hundred is slow. Now build a page that needs data from five such services and cannot render until all five return. The probability that a page avoids every slow call is roughly 0.99^5 ≈ 0.951 — about one page in twenty is slow. Extend it to a search result that fans out to 100 shards and waits for all of them, and 0.99^100 ≈ 0.366: nearly two out of three requests hit at least one slow leaf.

This is the tail-at-scale argument, and it has a counter-intuitive consequence: the p99 of your dependencies becomes the p50 of your user experience once fan-out is wide enough. Improving a dependency's median does nothing for this; only the tail matters. That reframes what "healthy" means for a service with many consumers — its p99 is not a vanity metric, it is other people's median.

The arithmetic assumes independence, which is the honest caveat. In real systems slow calls are often correlated — the same GC pause, the same hot shard, the same overloaded node serves several of your calls — and correlation can make the picture better (one bad node affects one call) or much worse (one bad node affects every call in the fan-out). Use the model to understand the *shape* of the risk; measure to get the number.

Fraction of composed requests that hit at least one slow leaf, if each leaf independently has p99 = 1 s. ESTIMATED from `1 − 0.99ⁿ`; assumes independence, which real systems violate.
Leaf calls per requestRequests hitting ≥1 slow leafWhat that means for the user
11.0%The dependency p99 is genuinely rare
54.9%One page view in twenty is slow
109.6%Roughly the p90 of the page is now the leaf p99
2018.2%Slowness is a routine part of the product experience
10063.4%The leaf tail has become the typical case

One slow dependency owns the request

When calls run in parallel and the response needs all of them, the request takes the maximum, not the average. The waterfall below shows four parallel dependency calls: three finish in under 60 ms and one takes 840 ms. The request takes 840 ms. Optimising the three fast calls to zero would save nothing at all — a point worth making concrete before someone spends a sprint on it (The Critical Path Is the Only Path That Pays).

This changes what you do about it. Since the max dominates, the highest-leverage moves are ones that attack the slow *instance* rather than the average cost: hedged requests (issue a duplicate to a second replica after a short delay and take whichever answers first), tighter per-dependency timeouts with a partial-response contract, and making the slow path optional so the page can render without it. Each of these trades extra load or completeness for a shorter maximum.

The same effect reappears in the network stack itself, where one delayed packet can hold up everything behind it on the same connection (Head-of-Line Blocking), and in any shared resource with a single queue. The pattern to recognise is: *a shared or awaited slowest element sets the pace for everything that depends on it*.

Four parallel dependency calls. Three are fast. The request is as slow as the slowest.
critical pathILLUSTRATIVE
0215430645860
GET /dashboard860 ms
profile-service32 ms
billing-service51 ms
notification-service44 ms
usage-service840 ms
profile-serviceFast — optimising this saves nothing
billing-serviceFast
notification-serviceFast
usage-serviceThis one call is the entire user-visible latency

Where the tail comes from

Tails are not random noise; they have a small number of recurring sources, and knowing the list turns "why is p99 bad" from a mystery into a checklist. The common causes are shared-resource contention, periodic runtime work, occasional expensive requests, retries, and cold state — each with a distinct fingerprint in the signals.

The diagnostic that separates them fastest is: *are the slow requests the same requests every time, or different ones?* If specific inputs are always slow, you have an expensive-request problem — a query that occasionally hits a bad plan, a customer with 500× the normal data volume. If any request can be slow and slowness clusters in time, you have a contention or pause problem — GC, a noisy neighbour, a lock, a saturated pool.

Both are fixable, but the fixes have nothing in common: expensive requests need bounds, pagination and query work (Unbounded Collections: The Anti-Pattern With a Fuse, The Slow Query Workflow); contention needs capacity, isolation or scheduling changes (Queueing: Why Systems Get Slow Before They Get Broken, Low CPU, High Latency: Lock Contention). Guessing wrong costs a sprint, which is why this question comes before any fix.

Key points

  • Rare slow responses compose: with fan-out of n, roughly 1 − (1−p)ⁿ of composed requests hit at least one slow leaf.
  • A dependency's p99 becomes its consumers' median once fan-out is wide enough — your tail is someone else's typical experience.
  • Parallel calls that must all complete cost the maximum, so optimising the fast ones changes nothing.
  • The first diagnostic question is whether the same requests are always slow (expensive work) or different ones are (contention or pauses).
  • Hedging, per-dependency timeouts and optional degradation attack the maximum; average-cost optimisations do not.

Follow the diagnosis

The causal chain, hop by hop — and the readings that invite the wrong conclusion.

  1. 1
    Dependency → service: each of five backends serves p99 = 1 s, which each owning team considers acceptable.
  2. 2
    Service → page: the page awaits all five, so ~5% of page loads inherit at least one slow call.
  3. 3
    Page → user: a user performing 30 actions in a session encounters slowness with near certainty, and describes the product as "randomly slow".
  4. 4
    Trace → responder: on slow requests, one span is 840 ms while its siblings are under 60 ms — the maximum, not the sum, is the request.
  5. 5
    Root cause → team: the composed experience is governed by dependency tails that every individual dashboard reports as healthy.
What this evidence makes people conclude — wrongly
  • "Every service is within SLO, so the user experience is within SLO" — composed experiences have their own SLI; per-service health does not compose (SLIs: Measuring What the User Actually Feels).
  • "p99 is only 1% of requests" — 1% of requests is not 1% of users. A user making 30 requests has a ~26% chance of hitting at least one.
  • "We should optimise the average dependency call" — with parallel fan-out the maximum sets the pace; average improvements are invisible.
  • "The tail is noise, so we should smooth the chart" — smoothing the chart removes the signal, not the problem.

Measure, fix, validate

An optimization is not finished until the metric that motivated it has moved.

How to measure it
  • • Record p99 (and p99.9 where volume supports it) per dependency call, not just for the whole request.
  • • Count fan-out per user-visible operation: the number of downstream spans in a trace, distributed as a histogram rather than an average.
  • • Compare the request's own p99 against the maximum of its children's p99s — if they track, one dependency owns your tail.
  • • Check whether slow requests cluster by input (same customer, same query shape) or by time (same 200 ms window across unrelated requests).
  • • Look at p99 as a function of load; a tail that grows with traffic is queueing, a tail that is flat under load is expensive work or pauses.
What actually fixes it
  • • Set an explicit per-dependency timeout with a defined partial or degraded response, so one slow leaf cannot own the whole request ([[timeouts-and-latency]]).
  • • Make non-essential calls optional: render without the slow panel and fill it in asynchronously rather than blocking the whole page.
  • • Reduce fan-out — batch, aggregate at a backend-for-frontend, or denormalise — so fewer independent chances to be unlucky exist ([[sequential-vs-parallel]]).
  • • Hedge latency-critical reads: send a second request to another replica after a delay near p95 and take the first answer. Costs extra load; buys a shorter maximum.
  • • Attack the specific tail source once identified — pauses, contention or expensive inputs each have their own fix.
How you know it worked
  • • Measure the p99 of the **composed** user-visible operation, before and after; per-service tails may not move at all while the user experience does.
  • • Confirm fan-out actually fell if that was the fix — a batching change that produces the same number of calls has not been applied.
  • • For hedging, verify both the tail improvement and the added load: extra requests are a real cost that must be accounted for.
  • • Check that timeouts produce the intended degraded response rather than an error, by inspecting what users actually receive on timeout.
What it costs
  • • Hedged requests spend extra capacity — typically a few percent of additional load — to shorten the maximum; under saturation they make things worse, so they need a circuit ([[circuit-breaker]]).
  • • Aggressive per-dependency timeouts convert slowness into incompleteness; someone must decide what a partial answer means to the user.
  • • Reducing fan-out by denormalising or aggregating trades write complexity and staleness for read latency ([[performance-tradeoffs]]).
Stop it coming back

Accuracy

Performance numbers are conditional. These are the conditions.

What these numbers depend on
  • ESTIMATEDThe amplification table is computed from 1 − (1−p)ⁿ with p = 0.01, assuming independent slow events. Real slow calls are correlated — shared hosts, shared caches, shared GC — so treat the table as the shape of the risk rather than a prediction.
  • ILLUSTRATIVEThe waterfall is constructed to show max-dominance in parallel fan-out. The millisecond values are invented.

Misconceptions

Claim
“p99 affects 1% of users.”
Reality
It affects 1% of *requests*. Users make many requests: at 30 requests per session, the chance of encountering at least one p99 event is about 26%. Tail latency is a user-level problem measured with a request-level metric.
Claim
“Fan-out is fine as long as the calls are parallel.”
Reality
Parallelism removes the *sum*, not the *maximum*, and it multiplies the number of chances to be unlucky. Ten parallel calls to a p99 = 1 s service produce a slow request roughly 10% of the time.
Claim
“Tail latency is inherent randomness you cannot control.”
Reality
Tails have a short list of recurring causes — queueing, pauses, contention, expensive inputs, retries, cold state — each with a distinguishable fingerprint and a specific fix.

Apply it

Where the depth lives

Probability
Order statistics and the maximum of independent samples

A parallel fan-out that awaits every call has a latency equal to the maximum of its children. The distribution of a maximum shifts right as you add samples, which is the formal statement of why wide fan-out is slow even when every child is fast.