RuntimesRUNTIME-SPECIFICGENERALSIMPLIFIED

The Node Event Loop

One thread runs your JavaScript, a small pool runs some of the I/O, and knowing which is which explains most Node production behaviour.

What actually happens How to build it

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.

The question

If Node is single-threaded, how does it serve thousands of concurrent requests — and what is it actually single-threaded about?

The requirement

A Node service handles many concurrent requests, most of them waiting on a database and an external API. We need to know what its limits are before we find them.

The obvious build

Node is asynchronous, so it handles concurrency automatically. Use async/await everywhere and it scales.

Why it breaks

await on a CPU-bound function yields nothing: await only yields at a real asynchronous boundary, so await computeHugeThing() blocks exactly as hard as calling it directly (Blocking the Event Loop).

How it breaks in production

await on a CPU-bound function yields nothing: await only yields at a real asynchronous boundary, so await computeHugeThing() blocks exactly as hard as calling it directly (Blocking the Event Loop).
A service on a 16-core machine uses roughly one core, because one process runs application code on one thread (Worker Processes).
Latency spikes appear on endpoints that do nothing, at the same moments as an expensive endpoint runs — everything shares the loop.
File or DNS-heavy work becomes a bottleneck at a surprisingly low concurrency, because those go through a small thread pool whose default size is 4.
A single unhandled synchronous operation — a large JSON.parse, a synchronous readFileSync in a hot path — makes the whole process look like a network problem.

Requirement→API Contract→Application Logic→Data Access→External Deps→Concurrency→Failure→Security→Observability→Deployment→Scale

What is actually happening

One thread executes your JavaScript. It runs a callback to completion, then picks the next one. There is no preemption: nothing takes the thread away from a running function.
The loop runs in phases, repeatedly: timers (setTimeout, setInterval), pending callbacks, poll (where I/O completions arrive and where the loop waits when it has nothing to do), check (setImmediate), and close callbacks.
Between callbacks, the microtask queue is drained: process.nextTick callbacks first, then resolved promise continuations. A promise chain that never awaits real I/O can therefore starve the loop without ever entering a phase.
Network I/O is not done on threads. libuv registers sockets with the OS readiness mechanism — epoll on Linux, kqueue on macOS — so ten thousand idle connections cost memory and file descriptors, not threads (Accepting Connections).
Some work does use a thread pool: file system operations, dns.lookup, zlib, and several crypto functions. Its default size is 4, configurable through UV_THREADPOOL_SIZE at process start. This pool is invisible in application code and is a real bottleneck for file-heavy or DNS-heavy services.
worker_threads gives you additional JavaScript threads with separate heaps, communicating by message passing (and SharedArrayBuffer where you want shared memory). This is the supported way to run CPU work without blocking the loop.
cluster and process managers give you N processes, each with its own loop, sharing a listening socket — the way a Node service uses more than one core (Worker Processes).
Event loop lag — the delay between when a timer should have fired and when it did — is the direct measure of whether the loop is keeping up. It is the single most valuable Node-specific metric.

One thread for your code, several for everything else

The picture worth carrying: your JavaScript runs on one thread; sockets are watched by the kernel and reported to that thread; and a small, separate pool handles the work that has no non-blocking equivalent.

That last group is the one people are surprised by. File system calls, dns.lookup and several crypto operations are dispatched to a pool whose default size is 4 — so a service doing many concurrent file reads has a concurrency limit of 4 for those, no matter how asynchronous the code looks.

UserLLMAgentToolDataDecisionHumanGuardrail

What `await` does and does not do

RUNTIME-SPECIFICNode-specific in its API names, and the underlying rule holds for Python asyncio too: awaiting a synchronous computation blocks that loop identically, which is why asyncio.to_thread exists (Python Runtime Models).

The most expensive misunderstanding in Node is that await makes something non-blocking. It does the opposite of what people assume: it is a *suspension point* for the current function, and it only actually yields the thread if what you awaited is genuinely asynchronous.

The second thing to see here is that suspension is interleaving. Between the two halves of an await, another request runs. That is where single-threaded read-modify-write races come from, and no lock exists to fix them because there is no thread to block.

Two functions that both look asynchronous

Awaiting CPU work

async function handler(req, res) {
  // hashPasswordSync burns 200ms of CPU.
  // `await` does NOT make this yield: there is no I/O
  // to wait for, so the loop is held for the full 200ms.
  const hash = await hashPasswordSync(req.body.password)
  res.json({ hash })
}

// Also blocking, for the same reason:
//   JSON.parse(hugeString)
//   readFileSync(path)
//   crypto.pbkdf2Sync(...)
//   /(a+)+$/.test(attackerControlledInput)

Awaiting something that actually suspends

import { pbkdf2 } from 'node:crypto'
import { promisify } from 'node:util'

const pbkdf2Async = promisify(pbkdf2)

async function handler(req, res) {
  // The async form dispatches to the libuv thread pool,
  // so the loop is free while it runs -- but note the pool
  // has 4 slots by default, so this is bounded concurrency,
  // not unlimited.
  const hash = await pbkdf2Async(req.body.password, salt, 210_000, 32, 'sha512')
  res.json({ hash: hash.toString('hex') })
}

The async variants of crypto, zlib and file APIs hand work to the libuv thread pool, which is a different thread from the one running your handlers. The Sync variants run on the loop thread. await is not what makes the difference — which function you called is.

Reading the loop's health

Event-loop lag deserves to be on the first dashboard anyone opens. It answers a question that request metrics cannot: is the runtime itself keeping up?

The measurement is simple enough to write by hand, which is worth doing once because it makes the meaning concrete: schedule a timer for a known interval and measure how late it actually fires.

Measuring event-loop lag

1import { monitorEventLoopDelay } from 'node:perf_hooks'
2 
3// Built in, low overhead, and gives you a histogram rather
4// than a single number -- the p99 is the part that matters.
5const h = monitorEventLoopDelay({ resolution: 20 })
6h.enable()
7 
8setInterval(() => {
9  metrics.gauge('event_loop_delay_p50_ms', h.percentile(50) / 1e6)
10  metrics.gauge('event_loop_delay_p99_ms', h.percentile(99) / 1e6)
11  metrics.gauge('event_loop_delay_max_ms', h.max / 1e6)
12  h.reset()
13}, 10_000).unref()
14 
15// The hand-rolled version, to make the meaning obvious:
16//   const start = process.hrtime.bigint()
17//   setTimeout(() => {
18//     const actual = Number(process.hrtime.bigint() - start) / 1e6
19//     const lag = actual - 100        // we asked for 100ms
20//   }, 100)
21// If a 100ms timer fires after 900ms, something held the loop
22// for ~800ms and every other request waited exactly as long.

Rising p99 lag with flat CPU means one long synchronous operation. Rising lag with saturated CPU means the process is genuinely out of capacity and needs more processes rather than better code.

How to build it

Most important first.

Measure event-loop lag continuously and alert on it. It is the health signal that explains correlated latency, and it has no equivalent in request-level metrics (The Metrics a Backend Must Emit).
Run one process per core (or per CPU allocation) via a process manager, container orchestrator or cluster. One Node process is one core for application code (Containerizing a Backend).
Move CPU work off the loop: a worker thread for latency-sensitive work, a job queue for anything that can be asynchronous (Background Jobs).
Never call the synchronous file, crypto or zlib APIs on a request path. Their names all end in Sync and that suffix is a production incident waiting for enough traffic.
Raise UV_THREADPOOL_SIZE deliberately if the service is file-, DNS- or compression-heavy — and know that it is fixed at process start and shared by all of those users.
Bound concurrency explicitly for fan-out. await Promise.all(items.map(fetchOne)) over ten thousand items issues ten thousand simultaneous requests, because nothing in the runtime limits it (Unbounded Concurrency).
Chunk unavoidable in-process CPU work so it yields — process a slice, then setImmediate, then continue — so other requests interleave.

What can go wrong

Failure modes

Loop starvation from microtasks: a recursive promise chain or process.nextTick loop runs forever between phases, so timers never fire and I/O completions are never processed.
Thread-pool starvation: four concurrent file reads occupy the pool and every subsequent fs and dns.lookup call queues behind them, presenting as I/O latency with no I/O saturation on the host.
Health checks failing during CPU work, so an orchestrator restarts a process that was making progress (Health Checks: Startup, Readiness, Liveness).
Memory growth from unbounded queueing: work accepted faster than the loop can process it accumulates as pending callbacks and closures (Backpressure).
An unhandled promise rejection terminating the process on modern Node defaults — a whole worker lost to one missing catch (Error Boundaries: Three Translations, Not One).
The mitigation failing: moving work to worker_threads and then passing large objects, where structured-clone serialization costs enough to reintroduce the problem on the sending side.

What can race

await is a yield point. Between a read and a write separated by an await, another request can run and modify the same in-memory value — a read-modify-write race on a single thread, with no locks available and no data race in the memory-model sense (Backend Races).
Two requests arriving concurrently can both find a cache empty and both populate it — a stampede that a single-threaded runtime does nothing to prevent (Cache Stampede, Request Coalescing).
Cluster workers race to accept on a shared listening socket; the distribution across workers is the operating system's or the cluster module's to decide, not yours (Worker Processes).

Security

Any request that can make the loop do superlinear work is an availability attack from one client: catastrophic regular-expression backtracking, deeply nested JSON, or an unbounded array parameter (Transport Validation).
Size and complexity limits are concurrency controls here, not just validation. One expensive request degrades every concurrent user in the process (Request Bodies and Streaming).
Module-level mutable state is shared by every request in the process. Caching request-scoped data — the current user, a tenant id — in a module variable is a cross-request data leak, and await makes the interleaving easy to reach (Stateless Services).
Worker threads share the process and its file descriptors; they are a performance boundary, not a security sandbox (Dependency Security).

Misreads

"Node is single-threaded." The JavaScript execution is. The process has a thread pool for some I/O, garbage collection threads, and any worker threads you start.
"await makes it non-blocking." await suspends at a real asynchronous boundary. Awaiting a synchronous computation blocks the loop for its full duration.
"async functions run in parallel." They interleave on one thread. Parallelism requires worker threads or more processes.
"The event loop is a queue." It is several queues visited in a fixed phase order, plus a microtask queue drained between callbacks. The ordering surprises — setTimeout(fn, 0) versus setImmediate versus process.nextTick — come directly from that structure.
"More CPU will fix it." Only with more processes. A single loop uses one core for your code no matter how many are available.

Operating it

How you see it in production

Event-loop lag as a histogram, not a mean. The p99 is what your users feel and the mean will look fine throughout an incident (Percentiles: Which One, and How Many Users Is That?).
Active handles and active requests distinguish "waiting on many things" from "computing".
A CPU profile taken during a spike names the blocking function directly — this is the fastest path from symptom to line number in Node (Why Is My API Slow?).
Heap used versus heap total, plus garbage-collection pause counts: major collections also stop the loop, and their pauses show up in the same lag metric.
Requests in flight per process, so you can see whether the ceiling you are hitting is the loop or a downstream pool (Connection Pools).

What changes at 10x and 100x

At 10x, the loop is fine as long as it never blocks; the ceiling you meet first is usually a downstream pool or an external dependency, not the loop.
At 100x, per-connection memory and the process count become the plan: N processes times one loop each, times per-process pool sizes, is what the database sees.
Idle connections stay nearly free, which is why Node suits websocket and SSE workloads well — the cost of a waiting connection is memory, not a thread (Keep-Alive and Connection Reuse).

What this costs

One thread means no in-process data races on ordinary objects and no locks — a genuine simplification — at the price that any CPU work is a service-wide problem.
Worker threads recover parallelism and add message-passing costs and a second execution context to reason about.
One process per core recovers the machine and multiplies every per-process resource: memory, connection pools, in-memory caches, warm-up cost (Worker Processes).
Non-blocking everything makes waiting cheap and makes backpressure your responsibility: nothing stops you accepting more work than you can finish.

Where this applies

Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.

RUNTIME-SPECIFICNode and its libuv event loop specifically. Browsers have an event loop with a different task model and no libuv thread pool; Python asyncio has a loop with the same yield-point semantics but a completely different phase structure and a configurable executor rather than a fixed-size pool; Go has no event loop exposed to the programmer at all.
GENERALThe transferable part: any single-loop runtime makes waiting cheap, makes non-yielding work catastrophic, and turns await into an interleaving point that can produce read-modify-write races without any threads.
SIMPLIFIEDThe phase list is the teaching version. The precise ordering of timers, setImmediate and I/O callbacks has edge cases that depend on where in the loop you started, and Node's own documentation is the authority on the exact rules.

Where the depth lives

This domain teaches the application-side mechanism and hands the rest off.

Performanceevent-loop-lag javascript-runtime-performance cpu-profiling garbage-collection

Concurrencyevent-loops javascript-concurrency async-await async-is-not-parallel worker-threads sequential-await-trap

OS & Networkingevent-loop io-multiplexing async-io file-descriptors