Threads versus Async versus Processes

The problem is always the same: you have more logical tasks than cores, and most tasks spend most of their time waiting. Thread per task hands each task a kernel thread; when it blocks, the kernel parks it and runs another. Async (an event loop) keeps one thread per core, never blocks it, and expresses each task as a chain of callbacks or an async function that the runtime resumes when its I/O completes. Process per task is thread-per-task with isolation: each task gets an address space, so it can crash, leak or be killed alone.

They are not exclusive. A modern server typically runs *processes* for isolation (one per core, or per tenant), each running an *event loop* for the I/O-bound majority, offloading the CPU-bound minority to a *thread pool*. Knowing which layer does what is the point of this lesson; The Event Loop and Async I/O: What `await readFile()` Actually Does cover the async mechanism in detail, and The Thread Pool Server and The Event-Driven Server build the server versions.

Concurrency is structure: many tasks in progress, interleaved. Parallelism is execution: many tasks running at the same instant on different cores. An event loop on one thread is fully concurrent (10,000 open connections, each mid-request) and has zero parallelism. Eight threads on eight cores are parallel. Threads on a single core are concurrent but not parallel — Concurrency versus Parallelism spells it out.

The distinction decides the strategy. I/O-bound work (waiting on sockets, disks, databases, other services) needs concurrency, not parallelism: the CPU is idle anyway, and what matters is how cheaply you can keep a task suspended. CPU-bound work (compression, image transforms, JSON parsing at scale, cryptography) needs parallelism: a task that never waits gains nothing from an event loop and everything from another core. An async runtime that receives a CPU-bound task stalls every other task on the loop for its duration — the single most common async performance bug.

The table gives typical figures. Per-thread memory is dominated by the stack reservation — 8 MB virtual on glibc Linux, 1 MB on Windows, 512 kB for Rust’s default, with only the touched pages committed — plus the kernel’s task state. An async task is a heap object: a Promise and its closure in JavaScript (hundreds of bytes to a few kB), a coroutine frame in Python (~1–2 kB), a Future state machine in Rust (as small as tens of bytes). The 100× to 1,000× gap in memory and switching cost is why 10,000 kernel threads is a serious number and 1,000,000 async tasks is routine.

Threads win when tasks are CPU-heavy or call blocking libraries you do not control (a legacy C client, a driver with no async API), when the count is modest (a few hundred), and when sequential code is worth more than peak connection count. Thread pools bound the count and reuse the expensive creation. Async wins when the work is dominated by waiting and the count is large — proxies, chat servers, API gateways, anything holding thousands of idle sockets — and when a runtime already does it well (Node, asyncio, Tokio, Go’s scheduler, which is async underneath sequential-looking code). Processes win when isolation matters more than sharing: untrusted code, crash containment, per-tenant limits, or a runtime that cannot use cores otherwise (CPython in the default build).

The failure mode of picking wrong is asymmetric. Threads where async was needed gives a server that runs out of memory or scheduler capacity at a few thousand connections — the C10K problem, see C10K: Ten Thousand Connections, Then a Million. Async where threads were needed gives a server whose p99 latency is dominated by the longest CPU-bound task, because everything else waits behind it. Processes where threads were needed gives a system that spends its time copying data through pipes.

The production shape is layered. Node runs its event loop on one thread and delegates file I/O, DNS and crypto to a libuv thread pool (4 threads by default); heavy CPU work goes to worker_threads or a separate process. Python’s asyncio offers loop.run_in_executor to push a blocking call to a thread pool or a ProcessPoolExecutor for CPU work. C++ servers pair an io_uring/epoll reactor per core with a work-stealing pool. Go hides the whole arrangement: goroutines are async tasks, the runtime multiplexes them over GOMAXPROCS kernel threads, and blocking syscalls are detected and moved off the loop.

Two rules keep the mix honest. Never call a blocking function on a loop thread — a synchronous file read, a sleep, a lock held across a slow section — because it stalls every task on that loop, and the symptom is latency with idle CPU (The Event Loop shows how to see it). And size the pools from the resource they consume: CPU pools at the core count, I/O pools at whatever the downstream can absorb, not "as many as possible".