The question this answers
For this workload, does an event loop or a thread-per-task model actually cost less — in latency, memory, and engineer-hours?
An API service handling 5,000 concurrent connections, each of which makes two downstream HTTP calls and one database query, plus a 12% minority of requests that render a PDF.
A rate-limit counter per API key and a warm in-process cache. On a loop they are plain objects with no lock and interleaving hazards at every await; with threads they are the same objects behind a mutex, safe across a whole function body and contended under load.
Every accepted request eventually gets a response or an explicit error within its deadline, and the rate-limit counter equals the number of requests actually admitted for that key.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
The two columns, with the costs kept in
The comparison is usually presented as "async scales, threads are heavy", which is half of one row. The full picture has trade-offs pointing in both directions, and the direction that decides most real projects is not performance at all — it is what the ecosystem's libraries do when they wait.
The single most decisive question: does anything on the request path block? One synchronous database driver, one native module without an async path, one fs.readFileSync in a config reload, and the loop model collapses for every concurrent request. Threads absorb a blocking call as a matter of course; that is what they are for. OS-side mechanics live in Threads versus Async versus Processes and Blocking, Non-blocking, Multiplexed, Asynchronous.
The second most decisive question is what your team can reason about. Threads let you hold a lock across an entire function and know nothing interleaves. Loops let you skip locking on your own heap but demand that you audit every await point (Await Is a Yield Point). Both are learnable; neither is free.
| Dimension | Event loop | Thread per task | Which wins |
|---|---|---|---|
| 10,000 mostly-idle connections | A closure and a promise each — a few hundred bytes | A stack each — commonly 512 KB to 8 MB of reserved address space | Loop, decisively |
| A blocking library on the path | Stalls every concurrent request in the process | Stalls one thread; the rest keep serving | Threads, decisively |
| Parallel CPU work | None from async; needs workers or processes | Native — the OS schedules threads across cores | Threads |
| Shared mutable state | No data races on the loop's heap; race conditions across every await | Data races are real; a mutex covers a whole function body | Loop for data races, threads for reasoning span |
| Debuggability | Stack traces cut at await boundaries; async task dumps are immature | Thread dumps, breakpoints and traces that show the whole call path | Threads |
| Tail latency under a slow handler | One slow synchronous task delays every pending task | One slow thread delays only its own request | Threads |
| Cost per additional concurrent task | An allocation | A thread creation, or a slot in a pool that must be sized | Loop |
| Context-switch overhead at high concurrency | None between tasks — the loop just calls the next callback | Kernel switches, scheduler pressure, cache pollution | Loop |
| Backpressure | Must be built: the loop happily accepts more than it can serve | Emergent: the pool is a natural bound, and the queue is visible | Threads |
| Cancellation | A promise cannot be cancelled; needs AbortController plumbing | Also awkward — interruption is cooperative in every sane design | Neither |
Where the memory and the switching actually go
The loop's structural advantage is that an idle task costs almost nothing. Five thousand connections waiting on a downstream response are five thousand heap objects; the process is blocked in one readiness syscall and using no CPU. The equivalent thread-per-connection design reserves five thousand stacks and puts five thousand threads on the scheduler's runqueues, which is why that model is usually replaced by a bounded pool (Thread per Connection, The Thread Pool Server).
But the bounded pool is where the story turns. With a pool of N threads and blocking calls, in-flight requests are capped at N — request 201 waits for a thread even though the machine is idle, because all 200 threads are parked in recv(). The loop has no such cap, which is simultaneously its advantage and the reason it needs explicit Bounding Concurrency: it will happily accept 20,000 in-flight requests and turn a capacity problem into a memory problem.
The timeline makes the difference concrete for the interesting case — three requests where the work is mostly waiting, on a pool of two threads.
The failure that decides it: one blocking call
Everything above is a performance argument, and performance arguments can be argued with. The blocking-call failure cannot. It converts a per-request problem into a per-process one, and it does so silently — the code looks correct, the tests pass, and the symptom in production is that unrelated endpoints got slow.
The schedule below is the whole case. Note what is *not* happening: no lock, no shared data, no race condition. Three independent requests, and one of them takes the only executor there is.
The practical answer when this is unavoidable is the hybrid: keep the loop for I/O and hand blocking or CPU-bound work to a real thread pool — worker_threads in Node (Worker Threads), loop.run_in_executor in asyncio, a dedicated pool in a C++ reactor. That is what Hybrid Runtimes: It Was Never Threads Versus Async is about, and it is where most mature services land.
| # | Request 1 — /report (sync PDF lib) | Request 2 — /health | Request 3 — /users/42 | Event loop | State |
|---|---|---|---|---|---|
| 1 | · | · | · | accepts R1, R2, R3; all three handlers are ready | pending=3 elapsed=0 ms |
| 2 | enters renderPdfSync() — a synchronous native call | · | · | · | pending=3 elapsed=0 ms loop=held by R1 |
| 3 | · | ready to run, cannot: the loop is inside R1 | · | · | pending=3 elapsed=40 ms loop=held by R1 |
| 4 | · | · | database response arrives on the socket; the continuation is enqueued and waits | · | pending=3 elapsed=80 ms loop=held by R1 |
| 5 | renderPdfSync() returns after 900 ms; R1 responds | · | · | · | pending=2 elapsed=900 ms loop=free |
| 6 | · | health check finally runs and responds | · | · | pending=1 elapsed=901 ms ✕ A 1 ms health check took 901 ms. The load balancer marked the instance unhealthy and removed it, so the remaining instances now carry its traffic and start doing the same thing. |
| 7 | · | · | responds with data that was ready 820 ms ago | · | pending=0 elapsed=902 ms ✕ The 100 ms deadline was missed by work R3 had nothing to do with. The trace blames the database, which answered in 8 ms. |
Key points
- The loop wins on many concurrent waiting tasks, per-task memory and the absence of context switching between tasks.
- Threads win on blocking libraries, parallel CPU work, debuggability, and tail latency isolation between requests.
- The decisive question is usually not throughput but whether anything on the path blocks — one synchronous call stalls every concurrent request on a loop.
- A bounded blocking pool caps in-flight requests at the pool size; a loop caps them at nothing, which is why it needs explicit concurrency limits.
- Threads have data races and need locks; loops have race conditions across every await and need discipline. Neither model removes concurrency bugs.
- Hybrid — loop for I/O, real thread pool for blocking and CPU work — is where most mature systems land, and it is a design, not a compromise.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • Event loop: register interest, return to the scheduler, the kernel reports readiness, the continuation is enqueued and runs on the one executor.
- • Thread per task: the task owns an executor for its whole life, blocks inside syscalls, and the kernel scheduler switches it out while it waits.
- • The loop's per-task cost is an allocation; the thread's per-task cost is a stack plus a scheduler entity.
- • The loop has no preemption between tasks; the OS preempts threads on a quantum, so a runaway thread cannot starve its peers the way a runaway callback can.
- • A blocking call on the loop occupies the single executor; a blocking call on a thread occupies one of many.
- • Hybrid runtimes keep a loop for readiness-driven I/O and dispatch blocking or CPU work onto a bounded pool, paying a handoff to keep the loop free.
- • Loop: R1 enters a 900 ms synchronous call; R2 (a 1 ms health check) and R3 (whose data arrived at 80 ms) both respond at ~900 ms; the instance is pulled from the load balancer.
- • Threads: the same R1 occupies one pool thread; R2 and R3 respond in single-digit milliseconds on other threads. The identical code has a completely different blast radius.
- • Bounded pool of 200 with blocking I/O: request 201 waits for a thread while every CPU is idle — latency created purely by the concurrency bound.
- • Loop with no limit: 20,000 requests accepted, all in flight, heap climbing, the downstream database at 20,000 concurrent queries. The absence of a bound became the incident (Parallelism Moves the Load Downstream).
- • Threads: two requests read the rate-limit counter simultaneously on two cores and both write the same value — a genuine data race that a mutex fixes and a loop never has.
- • Loop: two requests read the same counter around an
awaitand both write the same value — a race condition with no data race, that a mutex would not have helped with anyway.
- • Event loop guarantees: no data race on that loop's heap; no context switch between tasks; very low per-task memory.
- • Event loop does NOT guarantee: isolation between requests, fairness, parallel CPU execution, or any protection across an await.
- • Threads guarantee: one slow or blocked task cannot occupy the whole process; preemption, so no task starves its peers; real parallelism across cores.
- • Threads do NOT guarantee: safety of shared state, a useful pool size (there is no universal formula — see Sizing a Thread Pool), or that the memory cost is acceptable at high concurrency.
- • Neither guarantees: freedom from concurrency bugs, working cancellation, or backpressure. All three are design work in both models.
- • Loop: the executor is the contended resource, and the queue delay is the contention metric — event-loop lag.
- • Threads: locks are the contended resource, plus the scheduler itself once runnable threads exceed cores.
- • Pool: the pool queue is the contention point, and pool saturation is the signal that in-flight work exceeds the bound.
- • Hybrid: the handoff between loop and pool is a queue too, and it is the one nobody instruments until it backs up.
- • Loop: one synchronous handler stalls every pending task, and the metrics blame whichever dependency was in flight at the time.
- • Loop: unbounded acceptance turns a traffic spike into memory exhaustion and downstream overload.
- • Threads: data races on shared state, deadlock from lock ordering, and convoying under contention.
- • Threads: pool exhaustion, where every thread is parked in a blocking call and new work queues behind them.
- • Threads: oversubscription, where more runnable threads than cores makes every request slower without raising throughput.
- • Hybrid: work dispatched to the pool while holding a loop-side claim, so a saturated pool wedges the loop-side state as well.
- • Choose the loop when the work is dominated by waiting, concurrency is high, the whole dependency stack is genuinely async, and per-connection memory matters.
- • Choose threads when libraries block, when CPU work is on the request path, when requests must be isolated from each other's latency, or when the team needs ordinary stack traces to operate the system.
- • Choose hybrid when both are true, which is most of the time: loop at the edge for connection handling, pool behind it for anything that blocks or computes.
- • Choose neither when the concurrency is low: a simple sequential process is cheaper to run and far cheaper to debug (Concurrency Is Always Bought With Complexity).
- • The loop hurts the moment one dependency blocks, and you will not always control when that becomes true — a dependency upgrade can introduce it.
- • The loop hurts teams that read "single-threaded" as "no concurrency bugs" and stop auditing await points.
- • Threads hurt at very high connection counts, where stacks and scheduler pressure dominate before any real work does.
- • Threads hurt when shared state is pervasive and the locking discipline is not, which produces bugs that vanish under a debugger (Heisenbugs: The Bug That Leaves When You Look at It).
- • Hybrid hurts by doubling the operational surface: two saturation points, two queues, two sets of metrics.
- • Loop: event-loop lag percentiles and the count of synchronous tasks over a threshold. If lag rises with load, the model is the bottleneck.
- • Threads: pool queue depth, threads blocked versus runnable, and lock wait time at p99. See
worker-pool-saturationandlock-contention. - • Both: memory per concurrent request, measured under a load test at the concurrency you actually expect, not at your laptop's.
- • Both: tail latency of a trivial endpoint under load. A health check that degrades with unrelated traffic is the loop-blocking signature.
- • Run the decision as an experiment where you can: the same handler behind a loop and behind a pool, under
load-test-shapesthat match production arrival patterns.
- • The loop adds await-point auditing, bespoke mutual exclusion, and stack traces that omit the interesting frames.
- • Threads add locking discipline, lock ordering, a pool size to justify, and the whole memory-model conversation.
- • Hybrid adds both, plus the handoff — and the handoff is where the subtle failures live, because it is a queue that nobody drew on the architecture diagram.
- • Whichever you pick, the choice propagates into every library selection you make afterwards; it is one of the least reversible decisions in a service.
- • One process per core with an event loop in each (the Node cluster / nginx worker shape): parallelism without shared memory, at the cost of per-process caches and connection pools.
- • Separate processes by workload: a loop-based API service and a thread-based rendering service. The blocking work cannot hurt the fast path because it is not in it.
- • A runtime that hides the choice — Go's goroutines, Java's virtual threads, Rust's async runtimes — where blocking code is written plainly and the scheduler multiplexes it. Genuinely the best of both, if you can choose the language.
- • Do less concurrently: a queue plus a single consumer removes the question and is correct by construction when throughput allows it.
Server model lab
Each model is the same simulator given a different worker shape: a thread per in-flight request, a fixed pool, or one task per core where waiting does not occupy a worker. Memory is a per-thread stack estimate. Real servers differ by orders of magnitude in all of these, and every runtime has its own hybrids. Concurrency in flight is the knob; offered load is derived from it as concurrency ÷ service time.
Thread pool: utilization and queue
capacity = workers / service = 8 / 50 ms = 160.0 req/s ρ = arrivals / capacity = 120 / 160.0 = 0.750 Little L = λ × W → 0.120/ms × 59.8 ms = 7.2 in flight engine status = healthy
Concurrency lab
They come from a queueing and contention model inside Engineer Atlas. What is faithful is the behaviour: work that waits benefits from more workers, work that computes does not, a wide critical section pins parallelism near 1 no matter how many cores you buy, and arrivals past capacity produce an unbounded queue rather than a large latency. Real arrivals are burstier than this model assumes, so real systems reach every one of these walls earlier than the sliders suggest. Do not quote a millisecond from this page.
What people believe, and what is true
Async is faster than threads.
Async is cheaper *per waiting task*. For CPU work it is slower — it adds a state machine and gives no parallelism. For blocking libraries it is catastrophic.
Threads do not scale.
Thread *per connection* does not scale to tens of thousands. A bounded pool scales fine and is the default for most of the industry; the real cost is stack memory at high concurrency.
Picking the loop means I never need a thread pool.
You will need one the first time you meet a blocking library, a CPU-bound handler, or a native module. Plan the hybrid boundary rather than discovering it.
Go deeper
Overview
Loop: many waiting tasks share one executor cheaply. Threads: each task owns an executor and can block or compute freely.
Practical
Ask whether anything on the path blocks. If yes, threads or a hybrid. If no, and concurrency is high and mostly waiting, the loop. If concurrency is low, neither — write it sequentially.
Advanced
The loop trades isolation for density: no per-task executor means no per-task protection either. Threads trade density for isolation. Hybrid buys both and pays with a queue at the boundary that must be bounded and instrumented.
Internals
Both models sit on the same kernel facilities. The loop uses readiness notification (epoll/kqueue) plus non-blocking sockets; the thread model uses blocking syscalls and the scheduler's runqueues. Modern completion-based interfaces such as io_uring blur the line by making the "loop" a completion queue rather than a readiness one.