Keep-Alive and Connection Reuse
Reusing a connection skips the TCP and TLS handshakes and starts with a grown congestion window — but both ends, and every proxy between them, close idle connections on their own timers, and the race between a server closing and a client reusing produces the sporadic ECONNRESET every production system eventually meets.
The problem
What reuse saves
Without reuse, each request is connect → request → close: a The Three-Way Handshake (1 RTT), a The TLS Handshake (1 RTT with 1.3, 2 with 1.2), the request/response (1 RTT), then a FIN exchange and a socket parked in TIME_WAIT for a minute (The Connection Lifecycle: Close, Reset, TIME_WAIT, CLOSE_WAIT). Three requests cost nine round trips and three TIME_WAIT sockets. With reuse it is connect → request → request → request → close: two handshake RTTs paid once, then one RTT per request — five round trips for the same work, and one socket.
The second saving is invisible in the RTT count. A new TCP connection starts with an initial congestion window of about 10 segments (~14 kB) and grows it per round trip (Congestion Control: Protecting the Network); a reused connection has already grown its cwnd to whatever the path allows and can send a 200 kB response in one burst. On a 100 ms link that is the difference between one round trip and four for the same response. Reused connections are warm in two senses: no handshake, and a window sized to the path.
HTTP/1.1 makes persistence the default (HTTP/1.1: Persistent Connections and Their Limits); HTTP/2 and HTTP/3 connections are designed to live for the whole session and carry all requests as streams. The mechanism therefore matters most for 1.1 clients — every backend service calling another backend service, every SDK, every script — where the client library’s defaults decide whether reuse happens at all.
WITHOUT REUSE WITH REUSE TCP handshake 100 ms TCP handshake 100 ms TLS handshake 100 ms TLS handshake 100 ms request 1 100 ms request 1 100 ms close (FIN/ACK) — request 2 100 ms (cwnd already grown) TCP handshake 100 ms request 3 100 ms TLS handshake 100 ms close (later, idle) request 2 100 ms ───────────────────────────── TCP handshake 100 ms total ≈ 500 ms, 1 socket TLS handshake 100 ms request 3 100 ms ───────────────────────────── total ≈ 900 ms, 3 sockets in TIME_WAIT
Idle timeouts and the reuse race
An open connection costs a file descriptor, kernel buffers and — on a thread-per-connection server — a thread, so servers close connections that have been idle for a while: Node’s http.Server after keepAliveTimeout (5 s by default), nginx after keepalive_timeout (75 s), Apache after KeepAliveTimeout (5 s), Go’s net/http never by default. Clients keep their own idle timers. When those timers differ, a race appears: the server’s timer fires and it sends FIN; at the same instant the client, whose timer has not fired, writes the next request onto the socket. The request arrives at a socket the server has closed; the kernel answers with RST; the client sees ECONNRESET, socket hang up, Connection reset by peer, or a 502 from an intermediate proxy that hit the same race against its upstream.
The symptom is maddening because it is rare and unreproducible: it happens only when a request lands in the window between the server deciding to close and the client learning about it — one RTT wide, and only after an idle period of exactly the server’s timeout. Dashboards show a 0.1% error rate at a steady trickle, always on the first request after a quiet moment. The classic production instance: Node’s 5-second keepAliveTimeout behind an AWS ALB whose idle timeout was 60 s — the ALB reused connections that Node had closed and returned 502s, until Node’s timeout was raised above the balancer’s (and headersTimeout above that).
The rule is: the client’s idle timeout must be shorter than the server’s, at every hop. Then the client always closes first, and never writes to a connection the other side has given up on. Where you cannot control the client, make the server’s timeout longer than any intermediary’s. And because a race can still be lost (a server closing for other reasons: deploy, overload), clients should retry *idempotent* requests that fail with a reset before any response bytes were received — which is why the method semantics in HTTP: Requests, Responses, Headers and Status Codes matter.
- Node:
server.keepAliveTimeout(default 5000 ms); set it above your load balancer’s idle timeout; setheadersTimeouthigher still. - AWS ALB idle timeout default 60 s; nginx
keepalive_timeout75 s downstream andkeepalivepool +keepalive_timeoutupstream; Envoy/Istio have their own. - Every proxy in the chain has *two* timeouts — one facing the client, one facing the upstream — and the race exists at each hop.
- A
FINfrom the server closes only its write side; the kernel answers later writes withRST. The client cannot detect the close before writing unless it reads first.
Two different keep-alives
The word names two unrelated mechanisms. The HTTP `Keep-Alive` header (Keep-Alive: timeout=5, max=1000) is an HTTP/1.x, hop-by-hop hint accompanying Connection: keep-alive that tells the peer how long the connection may stay idle and how many requests it will accept; it is advisory, dropped by proxies, and forbidden in HTTP/2. It is about connection reuse at the HTTP layer.
TCP keepalive is a socket option (SO_KEEPALIVE) that makes the kernel send empty probe segments on an idle connection to detect that the peer has vanished — a crashed host, a NAT that dropped the mapping, a cable unplugged. Linux defaults are 7200 s before the first probe (tcp_keepalive_time), then 9 probes 75 s apart, so a dead peer is noticed after about two hours unless the application lowers the values. It is about liveness detection at the transport layer and has nothing to do with HTTP persistence. Enabling one does not enable the other; a Node http.Agent({ keepAlive: true }) enables HTTP reuse *and* sets SO_KEEPALIVE on the socket, which is a source of the confusion.
For long-lived connections through NATs and load balancers, TCP keepalive (or an application-level ping — HTTP/2 PING, WebSocket ping frames) is what keeps the intermediary’s mapping alive and detects half-open connections; HTTP keep-alive just decides whether the next request may use the same socket.
| HTTP `Keep-Alive` header | TCP keepalive (`SO_KEEPALIVE`) | |
|---|---|---|
| Layer | HTTP/1.x (hop-by-hop header) | TCP, in the kernel |
| Purpose | Allow the next request on the same connection; hint idle timeout and max requests | Detect a dead or half-open peer by sending empty probes |
| Who acts | HTTP client/server code | The kernel, after the application sets the socket option |
| Default timing | Server-specific: 5 s (Node, Apache), 75 s (nginx) | Linux: first probe after 7200 s, 9 probes every 75 s |
| In HTTP/2 and /3 | Forbidden / not applicable; connections are persistent by design | Still useful; also PING frames at the protocol level |
Key points
- Reuse turns connect/request/close × N into connect/request × N/close: two handshake RTTs paid once, and the congestion window stays grown.
- Both ends and every proxy close idle connections on their own timers; a server closing while the client reuses yields sporadic
ECONNRESET/socket hang up/ 502. - Rule: the client-side idle timeout must be shorter than the server-side one at every hop, and idempotent requests that fail with a reset before any response should be retried.
- Node’s 5 s
keepAliveTimeoutbehind a 60 s ALB is the canonical instance of the race. - HTTP
Keep-Alive(connection reuse hint) and TCP keepalive (SO_KEEPALIVEliveness probes) are different mechanisms at different layers. - Long-lived connections through NATs need transport- or protocol-level pings to stay mapped and to detect half-open peers.
Why does this exist?
Mechanisms are answers to constraints. Open each question before reading the answer.
▸Why do servers close idle connections at all?
Each open connection holds a descriptor, kernel buffers and possibly a thread; ten thousand idle clients would pin resources doing nothing. Closing after a few seconds of silence reclaims them at the cost of a handshake for the occasional returning client.
▸Why can the client not check the connection before writing?
A FIN sits in the client’s receive buffer until the client reads; a write does not look there first. Only a read (or a failed write answered by RST) reveals the close, and by then the request is gone. Ordering the timeouts is the only reliable prevention.
▸Why does a reused connection deliver a large response faster even with the same RTT?
TCP’s congestion window grows with successful round trips and persists for the connection; a new connection starts at ~14 kB and needs several RTTs to reach the path’s capacity, a warm one already has it.
Keep-alive
How it fails
What the failure looks like from inside real software.
- 0.1% of requests fail with
ECONNRESET/socket hang up, always after an idle gap: server idle timeout shorter than client’s (or than the load balancer’s). - ALB returns intermittent 502s to a Node backend:
keepAliveTimeout(5 s) below the ALB idle timeout (60 s). Raise the server’s above the balancer’s. - Client library with pooling disabled (
Connection: close,keepAlive: false, one-shotrequests.get): p50 latency two handshakes higher than necessary andTIME_WAITsockets pile up until ephemeral ports run out. - Long-lived connection through a NAT dies silently after 5 minutes idle; the next write hangs for the full TCP retransmission timeout because TCP keepalive is at its 2-hour default.
- Non-idempotent
POSTretried after a reset by a client that could not tell whether the server processed it; duplicate side effects. - A proxy honours
Keep-Alive: timeout=5from the origin but the client behind it has a 30 s idle timer; the race moves to the proxy’s upstream side.