Packet Loss: Duplicate ACKs, Fast Retransmit and the RTO
When a segment is lost, TCP learns it either from three duplicate ACKs (fast, one round trip) or from a retransmission timer (slow, at least 200 ms on Linux and doubling), retransmits, and — because loss is also read as congestion — cuts its sending rate; the application sees only a stall, and a loss rate of 1% can cost most of a link’s throughput.
The problem
Segment 2 is lost
Nothing in the network tells the sender. Routers drop packets silently when a queue overflows; radios lose frames to interference; nobody sends a notice. The sender’s only information channel is the ACK stream from the receiver. Segment 1 produces ACK 1000. Segment 3 arrives out of order; the receiver holds it (Sequence Numbers, ACKs and Reassembly) and sends ACK 1000 again — a duplicate ACK. Segments 4 and 5 produce two more. The sender now has three duplicates of ACK 1000, which is the signal.
Three, not one, because the network reorders packets, and a single duplicate ACK can mean segment 2 is merely late. Three duplicates mean at least three segments sent after the missing one have arrived, which makes "lost" far more likely than "delayed". This threshold (tcp_reordering = 3 on Linux, adaptive upward when reordering is detected) is the compromise between reacting fast and reacting to phantoms.
t=0 seg1 [0..999] -> t=0 seg2 [1000..1999] -x dropped at a congested router t=0 seg3 [2000..2999] -> t=0 seg4 [3000..3999] -> t=0 seg5 [4000..4999] -> t=100 <- ACK 1000 (seg1) t=100 <- ACK 1000 SACK 2000-3000 dup #1 t=100 <- ACK 1000 SACK 2000-4000 dup #2 t=100 <- ACK 1000 SACK 2000-5000 dup #3 => fast retransmit seg2; cwnd reduced t=100 seg2' [1000..1999] -> t=200 <- ACK 5000 hole filled; app receives 4000 bytes at once stall as seen by the application: ~1 RTT beyond normal delivery
Two recovery paths: fast retransmit and the RTO
Fast retransmit fires on the third duplicate ACK: the sender resends the segment at the ACK number immediately, without waiting for any timer, and enters fast recovery — it keeps sending new data at a reduced rate while the retransmission is in flight. Total cost: roughly one round trip of delay for the bytes behind the hole. This is the path you want every loss to take.
When there are no duplicate ACKs — because the lost segment was the *last* one (tail loss), or because the whole window was lost, or the ACKs themselves are lost — the only signal is silence, and silence is detected by the retransmission timeout (RTO). The RTO is computed from the smoothed RTT and its variance (SRTT + 4·RTTVAR, RFC 6298), with a floor of 200 ms on Linux (TCP_RTO_MIN) and an initial value of 1 second before any RTT sample exists. On a 1 ms data-centre path an RTO is therefore 200× the RTT. Each unanswered RTO doubles (exponential backoff) up to a cap of 120 s, and after tcp_retries2 (15) consecutive backoffs — around 15 minutes — the connection is abandoned with ETIMEDOUT. An RTO also resets the congestion window to one segment and restarts slow start (Congestion Control: Protecting the Network), so it is not only slower to detect, it is far more expensive afterwards.
Because tail loss is common (the last packets of every request are tail packets) and RTOs are so costly, Linux adds Tail Loss Probe: about two RTTs after the last transmission with no ACK, it sends one more segment to provoke a SACK, which usually triggers fast recovery instead of an RTO. Together with RACK (loss inferred from the time a SACKed segment was sent relative to unSACKed earlier ones), most losses on modern Linux are recovered without an RTO ever firing.
| Signal | Detection delay | Then | Typical trigger |
|---|---|---|---|
| 3 duplicate ACKs / SACK | ~1 RTT | fast retransmit; cwnd roughly halved (CUBIC: ×0.7) | a single loss in the middle of a burst |
| RTO timer | ≥ 200 ms on Linux, ≥ 1 s initially, doubling | retransmit; cwnd = 1 segment; slow start again | tail loss, whole-window loss, lost ACKs |
| Tail Loss Probe (Linux) | ~2 RTT | one probe segment; usually converts to fast recovery | the last segment of a request |
What loss looks like to the application
Nothing. No error, no callback, no flag. write() succeeded long ago — it copied into the send buffer. read() on the other side simply does not return for a while, then returns a large chunk. A request that usually takes 30 ms takes 130 ms, or 230 ms, or, after an RTO with backoff, 1.2 s. The p50 barely moves; the p99 grows a long tail. Server CPU is idle during the stall; the server’s own latency metric may not include it at all if it measures from the first byte of the request to the last byte written.
This is the most important property of TCP loss for anyone operating software: loss is latency, not failure. A connection to a dead peer stalls for the full backoff sequence — minutes — before the kernel reports anything, which is why every client needs its own read timeout (TCP Debugging: Reading the Handshake on the Wire) and why a "hung" request is far more often a lost packet or a dead peer than a slow server. It also means that retrying at the application layer on a timeout is often a retry over the same lossy path; the packet-loss-slow-transfer challenge walks through diagnosing one.
Why 1% loss can halve your throughput — or worse
Loss does two things: it delays the bytes behind it, and it tells congestion control to slow down. The second dominates. A loss-based sender halves (Reno) or reduces by 30% (CUBIC) its window on every loss event and then climbs back slowly, so the *average* window — and therefore throughput — is bounded by how often losses interrupt the climb. The classic model (Mathis et al., 1997) gives, for a Reno-style sender with loss probability *p*: throughput ≈ (MSS / RTT) × 1.22 / √p.
Put numbers in. MSS 1460 bytes, RTT 100 ms: at p = 0.01% (1 in 10,000) the bound is ~14 Mbit/s; at 0.1%, ~4.5 Mbit/s; at 1%, ~1.4 Mbit/s. On a gigabit link. The link speed does not appear in the formula because it is not the limit; loss and round-trip time are. Halve the RTT and throughput doubles; the same loss rate on a 10 ms data-centre path allows ten times what it allows across a continent. This is why a "1% packet loss" ticket is an emergency and not a rounding error, and why the fix for slow transfers across a lossy path is often a CDN or a closer endpoint (shorter RTT) rather than more bandwidth.
The model is a model: CUBIC recovers faster than Reno; BBR does not use loss as its primary signal at all and tolerates a few percent of random loss with far less penalty (Congestion Control: Protecting the Network); and real loss is bursty, not uniform. But the shape — throughput falling with the square root of loss and inversely with RTT — holds well enough that it should be the first thing you compute when someone says "the link is 1 Gbit/s, why am I getting 5 Mbit/s?".
loss p RTT 10 ms RTT 100 ms RTT 250 ms 0.001% 450 Mbit/s 45 Mbit/s 18 Mbit/s 0.01% 142 Mbit/s 14 Mbit/s 5.7 Mbit/s 0.1% 45 Mbit/s 4.5 Mbit/s 1.8 Mbit/s 1% 14 Mbit/s 1.4 Mbit/s 0.57 Mbit/s throughput ≈ MSS/RTT × 1.22/√p — link speed does not appear
Diagnosing loss
On Linux, ss -ti prints per-connection counters: retrans (current/total), lost, sacked, reordering, the current rto, rtt with its variance, and cwnd. A connection with a climbing retransmit count and a cwnd stuck small is loss-limited; one with a huge cwnd and a small rwnd is receiver-limited (Flow Control: The Receive Window). System-wide, nstat -az | grep -i retrans shows totals, and a TcpRetransSegs rate above a fraction of a percent of OutSegs is worth a look. mtr to the destination shows *where* along the path loss appears — with the caveat that loss at an intermediate hop that does not continue to later hops is ICMP rate limiting, not real loss (traceroute: Discovering the Path Hop by Hop).
A packet capture shows the truth: duplicate ACKs, SACK blocks, retransmissions and — the diagnostic that separates loss from reordering — whether the "lost" segment ever arrives late. Wi-Fi and cellular lose packets at the link and usually recover them there (link-layer retransmission), which shows up as RTT variance rather than TCP loss; wired loss is almost always a full queue somewhere, which is the subject of Congestion Control: Protecting the Network and Throughput: Requests, Packets and Bytes per Second.
$ ss -ti dst 203.0.113.10
ESTAB 0 1448000 192.168.1.42:51234 203.0.113.10:443
cubic wscale:7,7 rto:412 rtt:104.2/18.5 mss:1448 cwnd:9 ssthresh:7
bytes_sent:52M bytes_retrans:1.1M segs_out:36200 retrans:0/812 lost:0 sacked:0
reordering:3 delivery_rate:1.0Mbps busy:51s rwnd_limited:0 sndbuf_limited:0
# cwnd 9 segments on a 104 ms path, 812 retransmits: loss-limited at ~1 Mbit/sKey points
- The network never reports a drop; the sender infers loss from duplicate ACKs (fast, ~1 RTT) or from a timeout (slow, ≥ 200 ms on Linux, doubling).
- Three duplicate ACKs → fast retransmit and a moderate cwnd reduction; an RTO → cwnd to one segment and slow start from scratch.
- Tail loss has no duplicate ACKs; Tail Loss Probe and RACK exist so it does not always cost an RTO.
- To the application, loss is a stall in
read(), not an error; a dead peer takes ~15 minutes to surface without an application timeout. - Throughput on a lossy path scales as MSS/RTT × 1/√p: 1% loss on a 100 ms path caps a Reno-style flow near 1.4 Mbit/s regardless of link speed.
ss -tishows retransmits, rto, rtt and cwnd per connection;mtrshows where loss begins; a capture distinguishes loss from reordering.
Why does this exist?
Mechanisms are answers to constraints. Open each question before reading the answer.
▸Why wait for three duplicate ACKs instead of one?
One duplicate can be reordering; retransmitting on it would waste bandwidth and, worse, halve the window for a loss that never happened. Three is the point where reordering becomes unlikely enough to act.
▸Why is the minimum RTO so large?
A spurious timeout is expensive — it resets the window to one segment — and delayed ACKs alone can hold a reply for 40 ms; the 200 ms floor was chosen to make spurious RTOs rare on the internet of the 1990s. Data centres suffer from it, which is why fast-retransmit paths and TLP matter so much there.
▸Why does loss cut the rate instead of just retransmitting?
Because on wired networks loss almost always means a queue overflowed, and a sender that retransmitted at the same rate would keep it overflowing. Reading loss as "slow down" is what prevents congestion collapse; it is also what makes random wireless loss so costly.
Fast retransmit vs RTO
How it fails
What the failure looks like from inside real software.
- p99 latency spikes to exactly RTT + 200 ms multiples with no server-side cause: tail loss hitting the RTO floor.
- A bulk transfer over a link with 0.5% loss running at 3% of link speed: loss-limited throughput; more bandwidth will not help, a shorter RTT or a different congestion controller might.
- A request that hangs for 15 minutes before failing: the peer died and nothing at the application layer had a timeout; TCP retried through its full backoff.
- Retransmit rate climbing every day at 14:00: a shared uplink saturates at that hour; the loss is queue overflow at the bottleneck, visible in
mtrat one hop and all hops after it. - A Wi-Fi client with high RTT variance but few TCP retransmits: link-layer retries are hiding the loss as jitter; the fix is radio, not TCP.
- Application retries on a 1-second timeout over a lossy path: each retry starts a new connection with a fresh handshake and slow start and lands on the same path, multiplying load exactly when the path is struggling.