OS + Networking Together

Between `send()` returning in one process and `recv()` returning in another there are two kernels, two NICs, three copies, a congestion gate, a routing decision and at least one context switch — and every one of them is a place where bytes wait.

Six versions of the same server, each one born from the specific failure of the previous one: a single request, a blocking loop, a thread per client, a pool, non-blocking sockets, and finally an event loop.

accept → read → process → write → next: the simplest correct server, and the clearest demonstration that a blocking call parks the whole program on one client’s behaviour.

Give every client its own thread and let the scheduler interleave them: the code stays sequential and the OS supplies the concurrency — until the number of threads becomes the workload.

A fixed set of workers pulling connections from a bounded queue: thread cost becomes a constant, overload becomes a queue length you can see, and the slow client returns as "one slow request occupies a worker".

One thread, many non-blocking sockets, and a kernel API that says which ones are ready: the server sleeps until something happens and then does exactly the work that is possible — as long as nothing in it ever blocks.

The limits are concrete and countable: threads, descriptors, kernel socket memory, wake-up cost, ephemeral ports, middlebox state — and each has a mechanism that moved it, which is why the number went from 10K to 10M without the laws of physics changing.

Application buffer → socket send buffer → device queue → wire → NIC ring → socket receive buffer → application: a chain of bounded queues in which every full buffer pushes back on the one above, sized by bandwidth × delay and dangerous when oversized.

A fast sender and a slow reader: the receive buffer fills, the window closes, the send buffer fills, and the sender’s `write()` blocks, returns EAGAIN, returns false, or awaits — depending only on which I/O model it chose. Buffer in user space instead and it fails by running out of memory.

Serving a file the naive way copies it four times and crosses the user/kernel boundary four times; `sendfile`, `splice`, scatter-gather DMA and, at the extreme, kernel bypass remove the copies the CPU does not need to make — until TLS puts one back.

High-throughput systems are built by letting the page cache be the shared buffer between disk, process and NIC, and by batching every crossing of the user/kernel boundary: Kafka’s log, a database’s buffer pool, `writev`, and io_uring.

Application → kernel → network → remote: exhaust descriptors, fill a buffer, block a thread, drop packets, add latency, kill the process, restart the server — and follow each failure across the layers to the symptom the user sees and the tool that proves it.

A user in Warsaw opens your application hosted in another region and the page takes three seconds: enumerate every layer where the time could be, assign each to the domain that explains it, put a typical cost and a measurement next to each, and bisect.