TogetherOS + Networkingblockingacceptbackloghead-of-linerecv

The Blocking Server

accept → read → process → write → next: the simplest correct server, and the clearest demonstration that a blocking call parks the whole program on one client’s behaviour.

ConceptualLinux
Interview question
Progress

The problem

One thread, one loop: accept a connection, read the request, compute, write the response, close, repeat. Where exactly does time go when a client is slow — and why does a server that is doing nothing look, from outside, like a server that is overloaded?

The loop and its three blocking points

Conceptual

A blocking server has three places where the thread sleeps in the kernel. accept() blocks until a completed connection is in the listen socket’s queue. recv() blocks until at least one byte is in that connection’s receive buffer. send() blocks until the bytes fit into the send buffer — which with a healthy client and a small response is instant, but with a client that has stopped reading is not. See Blocking, Non-blocking, Multiplexed, Asynchronous for what "blocks" means: the thread is moved off the run queue and woken by the socket.

The loop is sequential by construction. Between accept() returning for client A and close() for A, no other client is touched. Every other client is somewhere in the kernel’s queues, and the kernel — not your program — decides what they experience. This is fine as long as each iteration is short and bounded. It is catastrophic the moment one iteration depends on a client.

One iteration of the blocking loop
  1. accept()sleeps until a handshake completes; returns a new fd
    ↓
  2. recv()sleeps until the client’s bytes arrive — the client controls this
    ↓
  3. processCPU; bounded by you
    ↓
  4. send()sleeps if the send buffer is full — the client controls this too
    ↓
  5. close()FIN; TIME_WAIT on the side that closes first

One slow client

Conceptual

Client A completes the handshake and sends nothing. The server is asleep in recv() on A. Clients B, C and D complete their handshakes — the kernel does that, without the server’s participation, and places them in the accept queue. They believe they are connected, because they are. They send their requests; the bytes land in their receive buffers. Then they wait, and see a server that accepted their connection and never answers. Their timeout fires; they retry; the new connection also lands in the queue.

This is head-of-line blocking at the application layer: the same shape as Head-of-Line Blocking in TCP, one level up. A queue with a single consumer is only as fast as the slowest item at its head. The slow client does not have to be malicious — a mobile client on a bad link, a client that opened the connection early to save a handshake, a health checker that only opens a TCP connection and closes it, all produce the same effect.

The defence within the model is a receive timeout (SO_RCVTIMEO, or settimeout() in Python), which turns an infinite wait into a bounded one. It does not remove the problem; it caps it at the timeout per slow client. That is enough for tools, and not enough for anything on the internet.

The accept backlog fills

Linux

While the server is stuck, completed connections accumulate in the listen socket’s accept queue. Its capacity is the backlog argument to listen(), capped by net.core.somaxconn (4096 by default since Linux 5.4; 128 before). Linux keeps a second queue for half-open connections (SYN received, ACK not yet) bounded by tcp_max_syn_backlog. When the accept queue is full, Linux by default drops the final ACK of new handshakes (tcp_abort_on_overflow = 0), so the client thinks it is connected while the server has no record of it; the client’s first data is retransmitted with backoff, and the request that "connected fine" times out after 30–120 s.

ss -ltn shows the state directly: for a listening socket, Recv-Q is the number of connections waiting in the accept queue and Send-Q is the backlog limit. A listening socket with Recv-Q at its Send-Q value is a server that is not calling accept() fast enough — either stuck (this lesson) or genuinely overloaded (the difference is the CPU column in top). nstat -az TcpExtListenOverflows counts the drops.

A blocking server stuck on one client, seen from the kernel
$ ss -ltn 'sport = :8080'
State   Recv-Q  Send-Q  Local Address:Port  Peer Address:Port
LISTEN  128     128     0.0.0.0:8080        0.0.0.0:*        <- accept queue full

$ nstat -az TcpExtListenOverflows TcpExtListenDrops
TcpExtListenOverflows   1873    0.0
TcpExtListenDrops       1873    0.0

$ top -p $(pidof server)
  PID USER  %CPU  %MEM  S  COMMAND
 4127 app    0.0   0.1  S  server        <- S: sleeping in recv(), not overloaded

When blocking is the right design

Conceptual

The blocking loop is not a beginner’s mistake; it is the correct architecture whenever the number of concurrent clients is one, or the clients are trusted and fast. A CLI that talks to a local daemon over a Unix socket. A batch job that pulls one stream from a queue and processes it. A single-purpose control port that an operator uses one session at a time. An internal tool whose only client is another process on the same host. In each case the sequential model is easier to read, impossible to race, and has no scheduler overhead at all.

It is also the model inside every worker of a thread pool and inside every green thread: a goroutine handling a connection is a blocking server whose "thread" costs 2 kB and whose blocking is intercepted by the runtime. Understanding the blocking loop precisely is what lets you understand what those runtimes are hiding. See Thread per Connection for the first step away from it.

  • Use blocking when concurrency is ≤ 1, clients are trusted, or a runtime with cheap threads is doing the multiplexing for you.
  • Always set a receive timeout on any socket a remote party controls.
  • Set the listen backlog deliberately, and monitor ListenOverflows — it is the earliest signal that a server has stopped accepting.

Key points

  • A blocking server sleeps in accept(), recv() or send(); two of those three are controlled by the client, not by the server.
  • One client that connects and sends nothing parks the entire server: application-layer head-of-line blocking.
  • The kernel completes handshakes without the server; waiting clients believe they are connected and sit in the accept queue, bounded by listen() backlog and somaxconn (Linux).
  • A full accept queue on Linux silently drops the final ACK by default; the client sees a successful connect followed by a long timeout.
  • ss -ltn shows the accept queue (Recv-Q) against its limit (Send-Q); ListenOverflows counts drops. A sleeping process with a full queue is stuck, not overloaded.
  • Blocking is correct for single-client tools, batch consumers, and as the model inside each worker or green thread.

Why does this exist?

Mechanisms are answers to constraints. Open each question before reading the answer.

▸Why does recv() block at all instead of returning what is there?

Because "nothing is there yet" is the common case and the caller almost always wants to wait for it. Blocking lets the kernel put the thread to sleep and use the CPU elsewhere; polling would waste it. Non-blocking mode exists for callers who have something better to do, and readiness APIs exist so they can find out when to come back.

▸Why does the kernel accept connections the server has not asked for?

The handshake must complete promptly or the client retransmits its SYN with exponential backoff; the kernel finishes it on the server’s behalf and queues the result. The backlog is the contract: "I will hold this many completed connections for you". Overflow policy — drop the ACK or send RST — is a sysctl on Linux precisely because both choices are defensible.

▸Why is a timeout not a fix?

A timeout bounds the damage per slow client to the timeout length. With ten slow clients and a 30-second timeout the server delivers nothing for five minutes. The structural fix is to stop letting one connection own the only thread — every later version in Build a Tiny Server: V0 to V5.

How it fails

What the failure looks like from inside real software.

  • A load balancer’s TCP health check (connect, then close) is handled fine, but a real client that connects first and sends slowly stalls every subsequent check: the instance flaps between healthy and dead.
  • A client library that opens the connection at startup and sends its first request minutes later holds the server for those minutes; the symptom is "the server hangs whenever service X restarts".
  • Sending a large response to a client that stopped reading blocks send() indefinitely; a SO_SNDTIMEO was never set, and the process cannot even be shut down cleanly.
  • Backlog left at Python’s default of a few connections: a burst of 50 clients produces 45 silent ACK drops and a wave of 1-second retransmit delays that nobody can reproduce in testing.
← Build a Tiny Server: V0 to V5Thread per Connection →