Keep-alive and connection reuse

Without reuse every request pays a TCP handshake (1 RTT), a TLS handshake (1–2 RTT) and a fresh slow start in which the first response is throttled to ~14 kB per RTT. With persistent connections — the HTTP/1.1 default, Connection: keep-alive in 1.0 — the second request pays none of that: the socket is open, the keys exist, and the congestion window is already grown. On a 100 ms path that is 200–300 ms saved per request; for an API client making thousands of calls it is the difference between a busy pool of 20 connections and a stream of TIME_WAIT sockets and port exhaustion (Keep-Alive and Connection Reuse, Connection Pooling).

The cost is state that outlives the request. Each idle connection holds a file descriptor, a kernel socket with its buffers, and on a TLS server tens of kilobytes of session state; a server holding 50 000 idle keep-alive connections must be sized for it (fd limits, memory) and its event loop or worker model must not tie a thread to an idle socket (The Event-Driven Server). So both ends impose an idle timeout: nginx defaults to 75 s toward clients, Node’s http.Server.keepAliveTimeout to 5 s, AWS ALB to 60 s, Go’s http.Server.IdleTimeout to whatever you set. HTTP/1.1’s Keep-Alive: timeout=5, max=100 header is only a hint.

The race is the classic failure. The server decides an idle connection is dead at exactly the moment the client picks it from the pool and writes a request. The client’s bytes meet a closed socket: it gets ECONNRESET, EPIPE, "socket hang up" (Node) or "Connection reset by peer" for a request that was never processed — or, worse, was fully written and *may* have been processed before the close raced. A proxy between them turns the same event into a 502. The rule that avoids it: the client’s idle timeout must be shorter than the server’s, at every hop. The textbook incident is Node behind an ALB: ALB idle 60 s, Node default 5 s → intermittent 502s until Node is set to keepAliveTimeout = 65 000 (and headersTimeout above it).

This is unrelated to TCP keepalive (SO_KEEPALIVE): kernel probes on an idle connection, by default only after 2 hours (tcp_keepalive_time=7200), whose purpose is to detect a dead peer and to keep NAT and firewall entries from expiring. It does nothing for HTTP request semantics, and its default timing is far too long to matter for the race above.