The question this answers
If every real server mixes models, what is actually being decided and where do the bugs live?
An API server that accepts on an event loop, runs handlers as async tasks, offloads image resizing to a CPU worker pool, and talks to a database through a blocking driver wrapped in a second, separate pool.
The response cache, touched from loop handlers and from worker threads; the two pools' queues; and the connection pool. The cache is the dangerous one — it stops being loop-exclusive the moment a worker touches it.
Any state mutated on the loop is never mutated concurrently by a worker, and every unit of work runs on the executor appropriate to its blocking behaviour.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Three layers, always
Underneath everything there are OS threads, because that is what a kernel schedules — Threads: Several Instruction Streams in One Process. On top of some of them sits an event loop, which multiplexes I/O readiness. On top of the loop sit tasks, which are suspendable units of work. And beside the loop sits at least one pool of threads for work the loop must not do. That is four layers in the common case, and the number of servers that use exactly one of them is very small.
This is not a compromise or a transitional state. It is the correct architecture, because the models solve different problems: readiness multiplexing makes idle connections cheap, and OS threads make blocking and computing safe. A design that insists on one everywhere either stalls on compute (pure loop) or wastes memory on idleness (pure threads).
The framing to reject is "threads versus async". The real questions are: which executor does this unit of work belong on, how does work cross between them, and how many of each do you have. Those are answerable, and the answers are where performance and correctness both live.
The boundaries are where the bugs are
Inside the loop, handlers do not overlap and shared state is safe. Inside a worker pool, threads run genuinely in parallel and shared state is not. The bug appears when a structure that was loop-exclusive — and therefore written with no synchronization, correctly — gains a second accessor on a worker thread. Nothing in the code changed; the concurrency assumption did, silently.
The rule that prevents it is a *transfer* discipline: work crossing a boundary hands over ownership of data rather than sharing it. Serialize, copy, or move; do not pass a reference into a pool and keep using it. That is Copy or Share? and Message Passing applied at a runtime boundary, and it is the same discipline that makes Web Workers and Worker Threads safe.
The second class of boundary bug is sizing. The two pools have different correct sizes for different reasons: the CPU pool is bounded by cores because more threads than cores does not create more compute, while the blocking-I/O pool is bounded by how many concurrent blocking calls the downstream can absorb, which is usually the connection limit. Sharing one pool for both is the common mistake — a burst of CPU work then starves the I/O offloads, or vice versa. There is no universal formula for either number; Sizing a Thread Pool is the reasoning and measurement is the answer.
| Work | Executor | Why | Cost of crossing |
|---|---|---|---|
| Socket read/write, protocol parsing | Event loop | Pure I/O readiness; microseconds of CPU | None — it is already there |
| Handler orchestration, awaiting downstreams | Async task on the loop | Mostly waiting; a task is far cheaper than a thread | None |
| Image resize, compression, hashing, large JSON | CPU worker pool (size ~ cores) | Would stall the loop for everyone; needs real parallelism | Serialize or transfer the buffer; a queue hop; result posted back |
| Blocking driver call, synchronous file I/O | Blocking-I/O pool (size ~ downstream limit) | Blocks a thread by construction; must not be a loop thread | A queue hop each way, plus a thread held for the duration |
| Long-running jobs (minutes) | Neither — a job queue in another process | Holding any in-process executor for minutes breaks every bound | Durable handoff; the request no longer owns the work |
| Shared cache access | Pick ONE side and keep it there | Loop-exclusive needs no lock; worker-accessible needs one | If both sides touch it, every access needs synchronization — including the loop's |
What the handoff actually costs
Offloading is not free and the cost is worth quantifying before reaching for it. The round trip is: enqueue onto the pool's queue, wait for a worker, serialize or transfer the payload, run, post the result back to the loop, wait for the loop to pick it up. Each hop is a queue and a potential wait, and the serialization can dominate for large payloads.
The practical consequence is a threshold: below some duration, offloading a task costs more than running it on the loop. Compressing 2KB inline is correct; compressing 40MB inline is an outage. Where the line falls depends on your loop's latency budget and the handoff cost, and it is a measurement, not a rule — but the existence of the threshold is what people miss when they either offload everything or nothing.
The timeline below shows both hops for one offloaded resize, with the loop free in between. Note the two queue waits: they are the reason a saturated CPU pool shows up as latency on requests that never touch the CPU pool at all, because the loop is fine and the *result* is stuck behind other workers' results.
Key points
- Every real server has OS threads, at least one event loop, tasks, and at least one worker pool. "Threads or async" asks about one layer of a stack that has four.
- The models solve different problems — readiness multiplexing makes idle connections cheap; OS threads make blocking and computing safe — so a hybrid is the correct design, not a compromise.
- The bugs live at the boundaries: state that was loop-exclusive and correctly unsynchronized becomes shared the moment a worker touches it.
- The discipline is transfer, not sharing: serialize, copy or move ownership when work crosses an executor boundary.
- The CPU pool and the blocking-I/O pool have different correct sizes for different reasons, and sharing one pool for both makes each starve the other.
- Offloading costs two queue hops plus serialization, so there is a duration threshold below which inline is correct. Measure it; do not assume it.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • The event loop owns the sockets and dispatches readiness events to tasks, none of which may block.
- • When a task encounters CPU-bound work, it serializes or transfers the payload onto a CPU pool's queue and suspends.
- • A pool worker picks it up, runs it on a real OS thread in genuine parallel with the loop, and posts the result back onto the loop's completion queue.
- • The loop picks up the completion on a subsequent iteration and resumes the suspended task where it left off.
- • Blocking driver calls follow the same path through a separate pool, sized to the downstream's concurrency limit rather than to the core count.
- • The kernel schedules the loop threads and both pools onto cores, which is why the total thread count across all executors — not any one pool — is what determines oversubscription.
- • A loop handler writes
cache.set(k, v)with no lock, correct for years. A CPU worker is then given cache access for a warmup path; worker and loop now write the same map concurrently, and the map's internal structure is corrupted in a way that surfaces as an unrelated crash hours later. - • CPU pool of 8 shared with blocking I/O: a burst of 40 image resizes fills the queue; the next database call waits behind them for 300ms even though the database is idle. Two workloads, one queue, mutual starvation.
- • Total threads across loop (8), CPU pool (8) and blocking pool (64) is 80 on an 8-core machine. Each pool is sized sensibly in isolation; together they oversubscribe by 10x and every executor gets slower. See Oversubscription.
- • A 2KB payload is offloaded to the CPU pool: 40µs of serialization plus two queue hops to save 15µs of inline work. Throughput drops and the change was made in the name of "not blocking the loop".
- • A pure event loop guarantees no data races between handlers. A hybrid guarantees that only for state no worker can reach — which is a guarantee you must maintain, not one you are given.
- • A worker pool guarantees genuine parallelism for CPU work. It does not guarantee latency, because the queue in front of it is unbounded unless you bound it.
- • Offloading guarantees the loop is not held. It does not guarantee the request is faster — two queue hops can exceed the work saved.
- • Sizing each pool correctly in isolation guarantees nothing about the whole: oversubscription is a property of the sum across all executors.
- • Nothing in the runtime enforces the transfer discipline. A reference passed into a pool compiles and runs.
- • Both pools' queues are contention points, and their queue age is the signal that matters more than their depth.
- • The loop's completion queue is a shared handoff point: every offload result crosses it, so a busy loop delays every worker's output.
- • Any structure reachable from both sides is contended across two very different access patterns — high-frequency small accesses from the loop, bursty large ones from workers.
- • Total thread count across executors contends for cores, and each pool's owner usually cannot see the others' sizes.
- • Silent loss of the loop-exclusivity assumption, producing corruption in structures that were correct when only the loop touched them.
- • Mutual starvation between CPU and blocking work sharing a single pool.
- • Aggregate oversubscription from independently-sized pools, presenting as uniform slowness with no single culprit.
- • Unbounded offload queues, converting a slow worker into unbounded memory growth — Unbounded Concurrency.
- • Over-offloading small work, where handoff cost exceeds the work and throughput drops.
- • Cancellation gaps: the request is cancelled, but the work already sitting in a pool queue runs anyway and its side effects land. See Cancellation Propagation.
- • Any server with both high connection counts and some genuinely CPU-bound endpoints — which is most servers once image processing, compression or large serialization appears.
- • Integrating a blocking library into an async server without abandoning the async model for everything else.
- • Getting parallelism out of a runtime whose main loop is single-threaded, by moving compute to a pool that can use other cores.
- • When it happens by accident rather than by design — pools appearing one at a time from different libraries, with nobody owning the total thread count.
- • On uniformly small, I/O-bound workloads, where offloading adds two queue hops to work that never needed them.
- • When the boundary discipline is not enforced, at which point you have the correctness hazards of threads *and* the debugging difficulty of async.
- • Queue age for each pool separately, which is where offload latency actually accumulates.
- • Total OS thread count across every executor against core count — the number nobody owns and everybody affects.
- • Loop lag alongside pool utilization: high pool utilization with low loop lag means offloading is working exactly as intended.
- • Handoff cost measured directly — enqueue-to-start and finish-to-resume — against the duration of the offloaded work itself.
- • Whether any shared structure is reachable from both loop and worker code. This is a code-review measurement and it is the important one.
- • Two or three executors to size, monitor, bound and reason about, each with its own queue and saturation signature.
- • A transfer discipline that must be maintained by convention in most languages, with no enforcement.
- • Debugging spans two worlds: async task dumps for the loop side, thread dumps for the pool side, and the correlation between them is manual — Task Dumps: When the Threads Look Idle and Nothing Is Moving, Reading a Thread Dump.
- • Cancellation must propagate across boundaries, including into queued-but-not-started work, which is easy to omit and hard to notice.
- • A pure thread pool server, when connection counts are moderate — one executor, one mental model, far less to get wrong. See Thread per Request: The Model That Reads Like Ordinary Code.
- • One process per core with a single loop each, which recovers parallelism without a shared-memory boundary at all — Worker Threads.
- • Moving CPU work out of the process entirely into a job queue, which removes the boundary rather than managing it — Background Jobs and Workers.
- • Choosing a runtime whose scheduler handles blocking calls natively, so the offload boundary is the runtime's problem rather than yours.
What people believe, and what is true
You pick threads or async.
Every server of any size runs both. The decision is which executor each unit of work belongs on, and how ownership transfers between them.
Offloading to a worker pool makes the request faster.
It makes the loop free. The request usually gets slightly slower, by two queue hops plus serialization — which is the correct trade for anything long enough to matter.
Each pool is sized correctly, so the process is sized correctly.
Oversubscription is a property of the total across all executors. Three sensibly-sized pools can add up to ten times the core count.