The Three-Way Handshake

The client sends a segment with the SYN flag and its initial sequence number (ISN), say 1,000,000 — "I want to talk; my bytes start here." The server replies with SYN and ACK set: its own ISN, say 5,000,000, and an acknowledgment number of 1,000,001 — "I heard you; your next byte is 1,000,001; mine start here." The client replies with ACK 5,000,001. The connection is ESTABLISHED on both sides, and the client’s first data segment can travel in the same packet as that final ACK.

The SYN segments also carry options that only exist at this moment: the MSS each side will accept, window scale (Flow Control: The Receive Window — it cannot be negotiated later, which is why an old middlebox stripping it caps a connection at 64 kB windows forever), SACK-permitted (Sequence Numbers, ACKs and Reassembly) and timestamps. The handshake is where the connection’s capabilities are fixed.

Notice that a SYN consumes one sequence number even though it carries no data (hence the ACK of ISN+1). So does a FIN. That bookkeeping is what lets the SYN itself be acknowledged and retransmitted like any other byte.

Two messages are not enough. After SYN and SYN-ACK the client knows the server is alive and knows both ISNs; the server knows only that *someone* sent a SYN — it has not heard its own ISN acknowledged, so it cannot know the client received the SYN-ACK or that the client is who the source address says. The third message closes that gap: each side has now sent its ISN and had it acknowledged. Four messages (SYN, ACK, SYN, ACK) would be correct too; TCP merges the middle two.

The three-way exchange also defends against old duplicate SYNs. A delayed SYN from a previous connection can arrive at a server; the server replies SYN-ACK; the client, which has no such connection, sends RST instead of ACK, and the half-open entry is discarded. Without the third step the server would create a connection nobody wanted.

ISNs are randomised (RFC 6528 — a hash of the 4-tuple and a secret, plus a clock) for two reasons: so that segments from an old incarnation of the same 4-tuple cannot be mistaken for the new one, and so that an attacker who cannot see the traffic cannot guess the sequence numbers and inject data or forge a connection from a spoofed address. The offset in every Wireshark trace where the relative sequence number starts at 0 is hiding a 32-bit random value.

A server calls listen(fd, backlog) and then accept() in a loop. Between the SYN arriving and accept() returning, the kernel keeps the connection in one of two queues. The SYN queue (half-open connections, state SYN_RECV) holds entries that have been sent a SYN-ACK and are awaiting the final ACK. When the ACK arrives the entry moves to the accept queue (fully established, waiting for the application), and accept() pops from there.

On Linux the accept queue length is min(backlog, net.core.somaxconn) — somaxconn defaults to 4096 since kernel 5.4, and was 128 before. If the accept queue is full when the final ACK arrives, the default behaviour is to *drop the ACK* (tcp_abort_on_overflow = 0): the client believes it is connected and sends data, the server’s SYN-ACK is retransmitted, and the client sees a stall of one to several seconds rather than an error. A server that is too slow to call accept() — its event loop blocked, its thread pool exhausted — produces exactly this signature: ss -ltn shows Recv-Q at or above Send-Q on the listening socket, and nstat -az TcpExtListenOverflows climbs.

The SYN queue is the attack surface. An attacker sends SYNs from spoofed source addresses and never completes the handshake; every one costs the server an entry and a SYN-ACK retransmission schedule (five retries over about a minute by default on Linux). Fill the queue and legitimate SYNs are dropped: the service is down while the server is idle. That is a SYN flood, and it is thirty years old.

The defence is SYN cookies: when the queue is under pressure, the server stops storing state and instead encodes the essentials (a timestamp, the MSS, a hash of the 4-tuple and a secret) into its ISN. If a genuine ACK ever returns, the server can reconstruct the entry from the acknowledged number and create the connection; if not, nothing was stored. The cost is that options that do not fit in the cookie — window scale and SACK, unless timestamps are on — are lost for that connection. Linux enables cookies automatically on overflow (net.ipv4.tcp_syncookies = 1) and records it in nstat as TcpExtSyncookiesSent.

One round trip before the first byte. On a LAN that is 0.2 ms and irrelevant; across a continent it is 60–80 ms; across an ocean, 150 ms or more — and then The TLS Handshake adds another round trip on top (TLS 1.3; two for 1.2), and Congestion Control: Protecting the Network starts the connection at ten segments, so a 100 kB response needs several more. A request that transfers 300 bytes can spend 300 ms in setup and 1 ms in transfer. Latency, not bandwidth, is what short connections pay for — Bandwidth vs Latency.

The remedies are all forms of *not doing it again*. Keep-Alive and Connection Reuse reuses one connection for many HTTP requests; Connection Pooling keeps a set of established connections to a database or a service ready. TCP Fast Open (RFC 7413) lets a client that has previously connected send data in the SYN itself, using a cookie the server issued last time — it works, is supported by Linux on both ends, and is rarely enabled on the public internet because middleboxes drop unfamiliar SYNs. QUIC (HTTP/3 and QUIC) folds the transport and TLS handshakes into one round trip and offers 0-RTT resumption, which is the same idea done in user space where middleboxes cannot see it.

Two handshake failures look different and mean different things. If the SYN reaches a host where *nothing is listening* on that port, the kernel answers with RST, and connect() fails at once with ECONNREFUSED. That is good news: the network is fine, the host is up, and the process is not running or is bound to a different port or address — the connection-refused-nothing-listening challenge. If the SYN (or the SYN-ACK) is *dropped* — by a firewall, a wrong route, a dead host — nothing comes back, the client retransmits the SYN with doubling delays (1, 2, 4, 8, 16, 32 s on Linux, tcp_syn_retries = 6) and gives up after about two minutes with ETIMEDOUT.

So: instant refusal → look at the server process; a long hang → look at the path (Why Can’t I Connect?, Firewalls). A firewall that *rejects* rather than *drops* will also produce an instant failure, with an ICMP unreachable rather than an RST; tcpdump distinguishes them.