TogetherOS + Networkingserveracceptblockingthreadsthread pool

Build a Tiny Server: V0 to V5

Six versions of the same server, each one born from the specific failure of the previous one: a single request, a blocking loop, a thread per client, a pool, non-blocking sockets, and finally an event loop.

ConceptualEducational model
▶ InteractiveInterview question
Progress

The problem

You have a socket and you want to serve more than one client. Every server architecture in production — Apache prefork, Java’s thread pools, nginx, Node.js, Go’s runtime — is an answer to "what broke when we tried the simpler thing?" This lesson builds that sequence so each design is a consequence, not a choice from a menu.

V0 → V1: one request, then a loop

Educational model

V0 is the smallest program that speaks TCP: socket(), bind(), listen(), accept() once, recv(), send(), exit. It works, and it teaches the four calls every later version keeps. Its failure is trivial: it serves one client and dies. See The Socket: A Descriptor With Two Kernel Buffers Behind It for what each call creates in the kernel.

V1 wraps V0 in while True. Now the server lives forever and serves clients one after another. Its failure is the one that motivates everything else: `recv()` blocks. While the server waits for client A to send its request, client B’s connection sits in the accept backlog, and clients C through Z queue behind it. A single client that connects and never sends stalls the whole server. The The Blocking Server lesson dissects this.

V1 — blocking, sequential
1import socket
2srv = socket.socket(); srv.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
3srv.bind(("0.0.0.0", 8080)); srv.listen(128)
4while True:
5 conn, addr = srv.accept() # blocks until a client arrives
6 data = conn.recv(4096) # blocks until THIS client sends
7 conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok")
8 conn.close() # next client only after this

V2: a thread per connection

Educational model

The fix for "one slow client blocks everyone" is to let each client block its own thread. accept() hands the new socket to a fresh thread; the main thread returns to accept() immediately. The kernel scheduler now does the interleaving: when thread A blocks in recv(), thread B runs. This is why the design is so natural — the code stays sequential and the OS supplies concurrency. See Threads: Several Instruction Streams in One Process and Process versus Thread.

Its failure is arithmetic. Each thread costs a kernel structure, a stack (8 MB *reserved* by default on Linux, of which only touched pages are real memory — see Thread per Connection for the honest accounting), and a scheduler entry. Ten thousand idle connections is ten thousand threads the scheduler must consider; a burst of new connections means ten thousand clone() calls. And an attacker or a buggy client can open connections until the server cannot create threads at all.

V2 — thread per connection
1import socket, threading
2def handle(conn):
3 data = conn.recv(4096) # blocks only this thread
4 conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok")
5 conn.close()
6srv = socket.socket(); srv.bind(("0.0.0.0", 8080)); srv.listen(128)
7while True:
8 conn, _ = srv.accept()
9 threading.Thread(target=handle, args=(conn,), daemon=True).start()

V3: a fixed pool and a queue

Educational model

If unbounded threads are the problem, bound them. V3 creates N worker threads once and puts accepted connections on a queue; workers pull the next connection when they finish the previous one. Thread creation cost disappears, memory is capped, and the queue length becomes a visible measure of overload. This is the shape of Apache’s worker MPM, Java servlet containers, Python’s ThreadPoolExecutor servers, and most database servers’ connection handling. See The Thread Pool Server.

Its failure is the return of V1’s problem at a smaller scale: a slow client occupies a worker for its whole lifetime. With 32 workers, 32 slow clients — deliberately or by accident — stall the server exactly as one client stalled V1. The queue grows without bound unless you cap it, and if you cap it you must decide what to do with the overflow: reject, or make the client wait in the accept backlog. See Semaphores and Condition Variables for the counting primitive behind a bounded pool.

V3 — thread pool with a work queue
1import socket, threading, queue
2work: "queue.Queue[socket.socket]" = queue.Queue(maxsize=1024)
3def worker():
4 while True:
5 conn = work.get() # blocks until a connection is queued
6 conn.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok" if conn.recv(4096) else b"")
7 conn.close()
8for _ in range(32): threading.Thread(target=worker, daemon=True).start()
9srv = socket.socket(); srv.bind(("0.0.0.0", 8080)); srv.listen(1024)
10while True:
11 work.put(srv.accept()[0]) # blocks when the queue is full: backpressure

V4: stop blocking at all

Educational model

The deepest cause of every failure so far is that recv() blocks a thread, and threads are the expensive unit. V4 marks every socket non-blocking: recv() returns immediately with data or with EAGAIN. One thread can now hold every connection in a list and try each one in turn. No thread is ever parked on a slow client. See Blocking, Non-blocking, Multiplexed, Asynchronous.

Its failure is that "try each one in turn" is a busy loop. With 10,000 mostly idle sockets the thread performs 10,000 system calls per pass and burns a core to discover that nothing happened. The server is correct and unusable. What it needs is a way to ask the kernel "which of these have something to do?" — and that question is exactly what I/O Multiplexing: select, poll, epoll, kqueue, IOCP answers.

V4 — non-blocking polling loop (correct, and CPU-bound while idle)
1import socket, errno
2srv = socket.socket(); srv.bind(("0.0.0.0", 8080)); srv.listen(1024); srv.setblocking(False)
3conns: list[socket.socket] = []
4while True:
5 try: c, _ = srv.accept(); c.setblocking(False); conns.append(c)
6 except BlockingIOError: pass # EAGAIN: nobody is connecting right now
7 for c in list(conns):
8 try: data = c.recv(4096) # EAGAIN if nothing has arrived
9 except BlockingIOError: continue
10 c.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok"); c.close(); conns.remove(c)

V5: the event loop

Educational model

V5 registers every socket with the kernel’s readiness mechanism — epoll on Linux, kqueue on BSD/macOS, IOCP (a completion model, not readiness) on Windows — and blocks in one call, epoll_wait(), that returns only the sockets with work. The thread sleeps while nothing happens and handles exactly the ready sockets when something does. Ten thousand idle connections cost one sleeping thread and a few kilobytes of kernel state each. This is nginx, Node.js, Redis, HAProxy, Envoy, and the core of every asyncio/Tokio/Netty runtime. See The Event-Driven Server and The Event Loop.

Its failure is the one the other models never had: since one thread serves everyone, any blocking or CPU-heavy work in a handler stalls every connection at once. The discipline "never block the loop" is the price of this design, and the standard remedy — hand CPU work to a thread pool and return to the loop — brings V3 back as a component rather than as the architecture. Production servers are almost always this hybrid.

The matrix compares the six versions. The right column is the point: none of these is "correct"; each is the answer to a specific constraint, and green threads (How C++, JavaScript and Python Map onto the OS such as Go’s goroutines or Java’s virtual threads) let a runtime give you V2’s code with V5’s cost, which is why the choice is a runtime question as much as an OS one.

V5 — event-driven with the kernel’s readiness API (Linux/BSD `selectors` picks epoll/kqueue)
1import socket, selectors
2sel = selectors.DefaultSelector()
3srv = socket.socket(); srv.bind(("0.0.0.0", 8080)); srv.listen(1024); srv.setblocking(False)
4sel.register(srv, selectors.EVENT_READ)
5while True:
6 for key, _ in sel.select(): # sleeps until at least one socket is ready
7 if key.fileobj is srv:
8 c, _ = srv.accept(); c.setblocking(False); sel.register(c, selectors.EVENT_READ)
9 else:
10 c = key.fileobj; data = c.recv(4096)
11 c.sendall(b"HTTP/1.1 200 OK\r\nContent-Length: 2\r\n\r\nok"); sel.unregister(c); c.close()
Six servers, one question each
VersionConnections handledCPU while waitingMemory per connectionComplexityWhat breaks
V0 one request1, everidle (blocked)trivialexits after one client
V1 blocking loop1 at a timeidle (blocked)one sockettrivialone slow client stalls all; backlog fills
V2 thread/connectionthousandsidle (each blocked)thread stack (touched pages) + kernel tasklowthread count; creation storms; scheduler load
V3 thread poolN active, rest queuedidlebounded: N stacks + queue entriesmoderateN slow clients stall the pool; queue growth
V4 non-blocking pollthousands100% (busy loop)socket + app statemoderateburns a core doing nothing
V5 event looptens of thousands+idle (epoll_wait)socket + app statehigh (inversion of control)any blocking handler stalls everyone

Key points

  • V1 fails because recv() blocks the only thread; V2 fixes it with threads and fails on thread cost; V3 bounds threads and fails on slow clients occupying workers; V4 removes blocking and fails on busy-polling; V5 asks the kernel what is ready.
  • The expensive unit is a blocked thread, not a connection. Every architecture is a strategy for not parking a thread on a socket.
  • Thread pools turn overload into a visible queue; event loops turn it into latency. Both need a limit somewhere.
  • The event loop’s discipline — never block the loop — is the cost of its efficiency; production servers pair it with a worker pool for CPU work.
  • Green threads (Go, Java virtual threads, Erlang) give V2’s programming model at close to V5’s cost, which is why "threads vs events" is a runtime question.
  • No version is universally right: a batch tool is best as V1, an interactive high-fan-out proxy as V5, most business services as V3 or a runtime that hides the choice.

Why does this exist?

Mechanisms are answers to constraints. Open each question before reading the answer.

Why not start at the event loop, since it handles the most connections?

Because its programming model — callbacks or coroutines, and the rule that nothing may block — is harder to write and harder to debug, and for a service with 200 connections and CPU-bound handlers a thread pool is simpler and just as fast. The event loop is the answer to a specific constraint: many mostly-idle connections.

Why is a thread pool so common if it has the slow-client problem?

Because the problem is bounded and visible (queue length, active workers), the code is sequential, every language has one, and the fix — timeouts plus a front proxy that absorbs slow clients — is well understood. Most production Java, Python and Ruby services are V3 behind an nginx that is V5.

Why does non-blocking I/O alone (V4) not solve anything?

Non-blocking changes what a call does when it cannot proceed — return instead of sleep — but not how you learn when it can proceed. Without a readiness or completion mechanism the only way to find out is to ask repeatedly, which is the busy loop. Readiness APIs are the actual invention.

Server evolution V0–V5

Build a tiny server, six times
Each version fixes the failure of the one before and introduces its own. Pick a version and a load, then run the ticks.
Version
Load (10% slow clients)
listen(fd)
for (;;) {
  c = accept(fd)        // blocks
  req = read(c)         // blocks on a slow client
  write(c, handle(req))
  close(c)
}
Served
1
Waiting
999
Threads
1
Memory (simulated)
1.0 MB
Connections served1
Threads in use1
Why this version is not enough: One slow client holds read() and everyone behind it waits. Throughput is bounded by the slowest peer, not by the CPU. Motivates concurrency.
1/20 · tick 1Simulated

How it fails

What the failure looks like from inside real software.

  • V1 in production behind a load balancer: one client that opens a connection and sends nothing makes the health check time out; the balancer marks the instance dead.
  • V2 with unbounded threads: a connection flood leads to pthread_create: Resource temporarily unavailable (EAGAIN) once ulimit -u or threads-max is hit, and the accept loop dies with it.
  • V3 with an unbounded queue: clients get accepted and then wait minutes in the queue; they time out and retry, which adds to the queue. Latency grows until restart.
  • V5 with one synchronous file read or a JSON.parse of a 50 MB body in a handler: every connection’s latency spikes to the duration of that call. See Timers fire late, health checks fail, one core at 100% in the challenges.
  • Any version with listen() backlog left at a tiny value: under a connection burst the kernel drops SYNs or refuses, and clients see connect timeouts while the server appears idle.