TransportAdvanced

TIME_WAIT

“Why does a TCP connection sit in TIME_WAIT after it closes? Which side ends up in that state, and when does it become a production problem?”

What this tests

  • The two reasons TIME_WAIT exists, not just its name
  • The active-closer rule and its consequence for clients that open many short connections
  • Ephemeral port exhaustion as arithmetic rather than folklore
  • Which mitigations are safe, which are dangerous, and which are the real fix

Answers by level

Read the beginner answer first and notice what is missing.

TCP closes with a four-way exchange: each side sends FIN and acknowledges the other’s. The side that sends the first FIN — the active closer — must, after receiving the peer’s FIN and sending the final ACK, stay in TIME_WAIT for 2×MSL (on Linux a fixed 60 s; tcp_fin_timeout does *not* change it — it governs FIN_WAIT_2). The passive closer goes straight to CLOSED. See The Connection Lifecycle: Close, Reset, TIME_WAIT, CLOSE_WAIT.

Two reasons. First, the final ACK may be lost; the peer then retransmits its FIN, and someone must still be there to ACK it — otherwise the peer gets a RST and reports an error on a connection that closed cleanly. Second, old duplicates: a segment from this connection delayed somewhere in the network could arrive after a *new* connection with the same four-tuple (src IP, src port, dst IP, dst port) is established, and be accepted as part of it because sequence numbers can overlap. Holding the tuple reserved for 2×MSL lets those stragglers expire. TIME_WAIT is correctness, not laziness.

It becomes a problem because of who closes. A client that opens a fresh connection per request to the same server — an HTTP client without keep-alive, a proxy without an upstream pool, a service hammering one database — is usually the active closer, so every request leaves a TIME_WAIT socket holding one ephemeral port for 60 s. Linux’s default range ip_local_port_range is 32768–60999, about 28 000 ports, so against a single destination the sustainable rate is 28 000 / 60 ≈ 470 connections per second. Beyond that connect() fails with EADDRNOTAVAIL ("Cannot assign requested address") before a packet is sent, which looks like a network outage and is not one. See Ports: Addressing a Process, Not a Machine.

The real fix is to stop opening connections: keep-alive and a bounded pool (Connection Pooling) turn 470 connections per second into a handful of long-lived ones. Secondary knobs: widen the port range (to ~64 000), use several source IPs, or on the client net.ipv4.tcp_tw_reuse=1, which lets a TIME_WAIT tuple be reused for an *outgoing* connection after 1 s when TCP timestamps prove the new segments are newer — safe. tcp_tw_recycle was removed in Linux 4.12 because it dropped legitimate SYNs from clients behind NAT whose timestamps were not monotonic; recommending it is a red flag. SO_LINGER with a zero timeout sends RST instead of FIN and skips TIME_WAIT entirely — at the cost of possibly discarding unsent data and every guarantee above.

Green flags · Red flags

Strong green flag · Points out that TIME_WAIT lands on whichever side closes first and that moving it (server-close vs client-close) is a design decision with a cost on each side.
Green flags
  • Names the active closer as the side that enters TIME_WAIT and explains the two purposes
  • Does the arithmetic: port range / 60 s ≈ hundreds of connections per second per destination
  • Says the real fix is connection reuse, not a sysctl
  • Knows tcp_tw_reuse is safe for outgoing connections and tcp_tw_recycle is gone
  • Reads EADDRNOTAVAIL as port exhaustion rather than a network error
Red flags
  • Recommends tcp_tw_recycle
  • Thinks TIME_WAIT is a bug or a leak
  • Claims tcp_fin_timeout shortens TIME_WAIT
  • Cannot say which side enters the state

Follow-up questions

F1
A proxy makes 2 000 upstream requests per second to one backend IP and starts failing with "Cannot assign requested address". Why, and what is the one-line fix?
F2
Why was tcp_tw_recycle removed from Linux?
F3
What does SO_LINGER with timeout 0 do, and why is it dangerous?

Scenario

A load-test tool on one machine cannot push past ~450 requests per second against a server that handles 20 000 rps from real clients, and every failure is a connect error on the tool’s side. Explain what limits the tool, prove it with one command, and describe two fixes.

Learn this topic