Packet Loss Buys You a Timeout, Not a Retransmit

The intuition that loss costs "the time to send one more packet" is wrong by two orders of magnitude. When a packet is lost, the sender does not know immediately. Recovery happens either through duplicate acknowledgements triggering a fast retransmit — relatively quick, roughly an RTT — or, when there is not enough subsequent traffic to generate those duplicates, through a retransmission timeout, which is conservative by design and typically hundreds of milliseconds.

Small requests are the worst case. A large transfer has plenty of packets in flight to trigger fast retransmit; a small request-response may have so few that a single loss can only be recovered by the timeout. So the requests that suffer most from packet loss are frequently the smallest and most latency-sensitive ones, which is precisely the opposite of what most people expect.

The result is a distribution with a small, distant cluster: nearly all requests normal, a few hundreds of milliseconds out. Averages absorb this completely, which is why loss-induced tail latency is one of the clearest illustrations of the argument in The Average Was Fine and Users Were Not.

A function call within one process has essentially no variance. The moment a call crosses a network, its latency becomes a distribution shaped by queueing at every switch and router on the path, by competing traffic from systems you do not control, by scheduling on the receiving host, and by loss and recovery. That variance — jitter — is not noise around a true value; it is the actual behaviour.

Jitter compounds badly with fan-out. If each of seven parallel calls is usually fast but occasionally jittery, the request takes the maximum, so it inherits the worst jitter of the seven on every request. This is the amplification from Fan-Out: Waiting for the Slowest of Seven, now driven by network variance rather than by dependency slowness, and it is why distributed systems have tails that look qualitatively different from monolithic ones.

The diagnostic value of jitter is that it discriminates. Application-level slowness usually correlates with something — an endpoint, a tenant, a payload size, a time of day. Loss-induced tails correlate with a *path*: a particular availability zone pair, a particular provider link, a particular client network. If the slow requests have no application-level pattern but cluster by network path, that is the signal.

Loss on a link you own or pay for is an operational problem with an operational fix: identify the segment, escalate to whoever runs it, and route around it while they work. Loss on the public internet between your edge and a user's device is not yours to fix at all, and pretending otherwise wastes time. The first useful question is therefore always "which segment", not "how do we fix it".

What you can always do is reduce your exposure. Fewer round trips on the critical path means fewer chances to lose a packet at a moment that costs a timeout — connection reuse, avoiding handshakes on hot paths, and collapsing chatty exchanges all shrink the target. Protocol choice matters too: transports with better loss recovery semantics can avoid head-of-line blocking that makes one lost packet stall unrelated concurrent work.

And you can stop letting a small tail become a large one. A per-call timeout with a fallback bounds the damage from a lost packet to the timeout value rather than to whatever the retransmission timer decides. Retrying is legitimate for idempotent calls, and must be bounded and jittered — a retry on a congested path adds to the congestion that caused the loss, which is the feedback loop in Retry Storms: The Load You Generated Yourself.