Concurrency Fundamentals

Classifying the Work: Computing or Waiting?

Before you can choose an execution model you have to know what the work is doing with the wall clock. Compression, resizing, encryption, parsing and numerics compute. Network calls, database queries, disk reads and third-party APIs wait. Most real handlers do both, in phases, and the phases want different models.

▶ Run the lab

The question this answers

The question

Is this unit of work spending its wall clock on a core or in a wait queue, and what does that dictate about how to run it?

The work

One "generate export" handler: fetch 200 000 rows from Postgres (waiting), transform them to CSV (computing), gzip the result (computing), upload to object storage (waiting), write an audit row (waiting).

What is shared

The row buffer as it passes between phases, the connection pool used by phases 1 and 5, and — if the transform is parallelised — the output ordering. Within a phase, nothing else.

The invariant — what must stay true under every interleaving

Every fetched row appears exactly once in the uploaded CSV, in query order, and the audit row is written if and only if the upload succeeded — whatever model each phase runs under.

Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.

WorkCan it overlap?Can it parallelise?What is shared?What ordering?What synchronization?Where is contention?What can deadlock?What can race?What is gained?What complexity?

The two ends of the axis, and the honest middle

Performance & Observability owns diagnosing this from live signals — [[cpu-bound-vs-io-bound]] is the lesson that teaches you to read the graphs. This lesson takes the classification as an input and asks the design question: given that answer, what execution model does this work get?

The mapping is unglamorously direct. Waiting-bound work wants a model where a waiting unit releases its execution context: async, coroutines, an event loop, or simply more threads if the counts are small. Compute-bound work wants a model that can occupy more cores: threads in a runtime that permits it, processes in one that does not, or a data-parallel primitive.

The trap is the third row of the matrix. Almost nothing real is purely one or the other, and the mistake is classifying the *handler* rather than its *phases*. This export handler is 88% waiting by wall clock, so a superficial reading says "async, done" — and then gzip on the event loop stalls every other request on the process for 900 ms. Phases get classified, not handlers.

ClassExamplesWhat the wall clock is spent onModel that fitsFailure when you pick the other one
CPU-boundgzip, image resize, AES, JSON/CSV parsing, matrix maths, template rendering, PDF layout, hashinga core retiring instructionsparallelism: threads on distinct cores, a small pool, processes where threads cannot, or SIMDOn an event loop it serialises AND blocks every neighbour. Async adds suspension points and returns nothing.
I/O-boundHTTP calls, SQL queries, disk reads, object storage, third-party APIs, queue consumption, DNSa wait queue, with the core freeconcurrency: async/await, coroutines, an event loop, or a bounded thread poolCores are useless here — the CPU was already idle. Thread-per-wait works but costs stacks and switches for nothing.
Mixed (most real handlers)fetch → transform → upload; receive → decode → persist; query → render → returnboth, in distinct phasesboth, split: loop or async for the waiting phases, a bounded worker pool for the computing phase, a queue betweenClassifying the handler by its dominant phase puts CPU work on the I/O model. The 12% of the time that computes stalls the 88% that does not.
Memory-bandwidth-boundlarge array scans, big joins, streaming reductions over multi-GB datawaiting on memory while the core is nominally "busy"fewer bytes per operation before more cores — layout, smaller types, fused passesLooks CPU-bound on every graph. Adding cores contends for the resource that was already saturated.
Classification → execution model. The third row is where most real work lives.

The classification procedure

The single number that decides it is CPU time over wall-clock time for the phase. If a phase takes 900 ms of wall clock and 890 ms of CPU, it computes. If it takes 500 ms of wall clock and 2 ms of CPU, it waits. There is no ambiguity in either case and the measurement is cheap in every runtime.

The follow-up question — the one that separates "add cores" from "you cannot add cores" — is whether the compute-bound phase is partitionable. gzip over one stream is not: it is inherently sequential over its window. gzip over 200 independent chunks is. Compression that cannot be split is a phase you move off the loop but cannot make faster with more workers, and knowing that before you build a pool saves a week.

The last branch matters for cost rather than latency: if a phase is compute-bound *and* not on the critical path of a user-visible request, the right answer is usually neither concurrency nor parallelism but relocation — enqueue it, answer the user, and let a worker fleet do it. See [[background-jobs]].

Classifying one phase of work
< 0.2> 0.80.2 – 0.8reclassify each sub-phasenoyesyesno — e.g. single-stream gzipif speedup < 2× at 4 workersOne phase of workCPU time ÷ wall time?Below ~0.2: I/O-boundAbove ~0.8: CPU-boundIn between: split it furtherConcurrency: async / event loop / bounded poolOn a user-visible path?Partitionable into independent chunks?Relocate: enqueue and answer immediatelyParallelism: worker pool near core countOff the loop, but one worker — no speedup availableCheck achieved memory bandwidth first
ClientGateway / LBServiceWorkerDatabaseCacheQueue / LogObject storageCDNExternal system

The export handler, phase by phase

Laid out over time, the handler is obviously two different animals wearing one function. The fetch, upload and audit phases hold no core at all. The transform and gzip phases hold one core solidly and hold it for 1.4 seconds combined.

On a single-threaded event loop, that 1.4 seconds is not "12% of a slow handler". It is 1.4 seconds during which every other request on the process makes zero progress, including the health check. Ten concurrent exports on one loop is fourteen seconds of total stall spread across every unrelated user of that process. This is the failure that [[blocking-the-event-loop]] is named after, and phase classification is what prevents it.

The fix follows directly from the picture: keep phases 1, 4 and 5 on the loop where their waiting overlaps with everything else, and move phases 2 and 3 to a bounded worker pool. The queue between the two is not overhead — it is the place you get to see and limit how much CPU work is outstanding, which is a property the naive version does not have at any price.

One export request on one event loop. The two running segments are the entire problem.SIMULATED
Export handler
fetch 200k rows
transform to CSV
gzip
upload to storage
Every other request on this loop
interleaving normally
stalled — nothing else can run
interleaving normally
↑ loop frozen↑ loop released
runningreadywaitingblockedidle1 tick ≈ 100 ms

Key points

  • Classify phases, not handlers. A handler that is 88% waiting can still hold a core for 1.4 seconds and stall everything around it.
  • CPU time divided by wall time is the measurement. Below roughly 0.2 is waiting; above roughly 0.8 is computing.
  • Waiting-bound work wants concurrency. Compute-bound work wants parallelism. Neither substitutes for the other.
  • Compute-bound is not automatically parallelisable: single-stream compression is compute-bound and inherently sequential.
  • Memory-bandwidth-bound work looks CPU-bound on every graph and does not respond to more cores. Check achieved bandwidth before adding workers.
  • For compute-bound work that is not on a user-visible path, relocation beats both models: enqueue it and answer immediately.
  • The queue between an I/O phase and a CPU phase is where you get visibility and a limit — that is a feature, not overhead.

The loop, answered

Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.

How it works
  • Break the unit of work into phases at every boundary where it starts or stops waiting on something external.
  • For each phase, measure CPU time and wall-clock time over the same interval and take the ratio.
  • Classify: below ~0.2 waiting-bound, above ~0.8 compute-bound, in between means the phase is still two phases.
  • For compute-bound phases, ask whether the input partitions into independent chunks; if not, parallelism is unavailable and offloading is the only move.
  • For compute-bound phases, check achieved memory bandwidth against the machine's peak before assuming more cores will help.
  • Assign a model per phase and put an explicit, bounded hand-off between phases with different models.
Interleavings that matter
  • Ten exports arrive on one loop: their fetch phases overlap perfectly, then their gzip phases run strictly one after another, and total latency is roughly ten times gzip regardless of how much concurrency the fetch enjoyed.
  • Export A is mid-gzip when B's fetch completes; B is ready but not running for 900 ms. B's trace attributes that 900 ms to nothing at all, because no span was open for it.
  • Transform parallelised across four workers: chunks complete in the order 2, 0, 3, 1, and if the CSV is written in completion order the export contains every row exactly once in the wrong order — the invariant's "in query order" clause is what catches it.
  • The gzip worker pool has two slots and ten exports arrive: eight queue. The queue is bounded and visible, which is the entire reason to have created it, and the alternative was ten stalls on the loop.
What it guarantees — and does not
  • Classification tells you which model can possibly help. It does not tell you that it will: a partitionable compute phase can still be bandwidth-bound and gain nothing.
  • Moving CPU work off the event loop guarantees the loop stays responsive. It guarantees nothing about the CPU work getting faster — one worker is still one core.
  • A bounded hand-off queue guarantees the amount of outstanding CPU work is knowable and capped. It does not guarantee callers get served; past the cap they wait or are rejected, and that must be a decision. See [[concurrent-backpressure]].
  • The CPU-over-wall ratio is a property of the phase on this hardware with this data, not a universal label. A phase that waits on a warm cache computes on a cold one.
Where contention appears
  • Waiting phases contend for the connection pool and for whatever the upstream limits — not for cores.
  • Computing phases contend for cores, for last-level cache and for memory bandwidth, and those three are hard to distinguish from application-level metrics.
  • A mixed handler on one loop makes the computing phase contend with every unrelated waiting phase, which is the pathological case: work that needs no core is blocked by work that needs one.
  • The hand-off queue is contention made explicit and measurable, which is why introducing it usually improves diagnosis even before it improves latency.
How it fails
  • Event-loop starvation from a CPU phase inside a nominally I/O-bound handler — the single most common concurrency incident in server-side JavaScript and Python.
  • Thread pool sized for I/O (say 200) used for CPU work: 200 runnable threads on 8 cores, and throughput falls while CPU rises. See [[oversubscription]].
  • Thread pool sized for CPU (say 8) used for I/O work: 8 concurrent requests and a 500-deep queue, with the box 96% idle.
  • Order loss when a compute phase is parallelised and results are collected in completion order.
  • Misclassification via averages: a handler whose mean says "I/O-bound" while p99 is dominated by a compute path that only large inputs take.
When it helps
  • Before choosing any library. The classification is fifteen minutes of measurement and it eliminates entire branches of the design space.
  • When a service has been "optimised" twice with no result — usually the model matches the dominant phase and the incident comes from the other one.
  • When sizing pools: the class of work is the only principled input to the sizing conversation, and it still does not give you a formula. See [[thread-pool-sizing]].
When it hurts
  • When the phases are so short that splitting them costs more than they take. A 2 ms handler does not need a phase model.
  • When it becomes an excuse to build a pipeline: two phases, two pools and a queue is real operational surface, and for a low-traffic endpoint the sequential version is better engineering.
  • When classification is done once and treated as permanent. Adding a cache turns a waiting phase into a computing one, and the model quietly stops matching.
How you would know
  • Per-phase CPU time versus wall-clock time — process.cpuUsage() deltas, resource.getrusage, or CPU-seconds from the profiler over the span.
  • Event-loop lag or scheduler run-queue delay during the handler, which converts "the loop was blocked" from a theory into a number.
  • Per-core utilisation during the compute phase: one core pinned and the rest idle means parallelism is available and unused.
  • Achieved memory bandwidth against peak for large-array phases — this is the check that stops you buying cores you cannot use.
  • The p50/p99 split by input size. Classification derived from the mean misses the compute-heavy tail entirely.
Complexity it introduces
  • Splitting a handler into phases with different models turns one function into a small pipeline with a queue, a pool and two failure surfaces.
  • The hand-off must serialise data across a worker boundary, and for large payloads the copy can consume the gain.
  • Cancellation and timeouts have to cross the boundary too, or a cancelled request leaves a worker computing a result nobody wants. See [[cancellation-propagation]].
  • Two pools mean two capacity decisions and two saturation modes, and the interaction between them is not obvious from either one.
Simpler alternatives
  • Make the phase disappear. The cheapest export is one that streams rows to storage and never materialises a 200 000-row buffer to compress.
  • Move the compute to where the data is: COPY ... TO PROGRAM 'gzip' or a database-side aggregate avoids both the transfer and the phase.
  • Use a library that already crosses the boundary correctly — a streaming gzip that yields between chunks keeps the loop alive with no pool at all.
  • Relocate the entire handler to a job queue and give the user a status endpoint. See [[async-job-pattern]].

Concurrency lab

Concurrency lab
Six knobs, one model. Ask it the only question that matters: does more concurrency help this workload, and what stops it?
SIMULATEDThese numbers describe no real system.

They come from a queueing and contention model inside Engineer Atlas. What is faithful is the behaviour: work that waits benefits from more workers, work that computes does not, a wide critical section pins parallelism near 1 no matter how many cores you buy, and arrivals past capacity produce an unbounded queue rather than a large latency. Real arrivals are burstier than this model assumes, so real systems reach every one of these walls earlier than the sliders suggest. Do not quote a millisecond from this page.

Controls
Cores the process may actually run on. This is the parallelism ceiling.
Threads or tasks in flight. Not the same quantity as cores, and rarely the same number.
Time actually holding a core. This is the only part cores can parallelise.
Waiting while holding no core. This is the part concurrency can hide.
The slice of the CPU work only one task may execute at a time. Clamped to the CPU time.
Offered load. Past capacity the queue has no steady state at all.
Snapshot the current settings, then change one thing. The model is pure, so the “before” column costs nothing to keep.
throughput
150/s
offered 150/s
latency
31 ms
service 30 ms
effective parallelism
2.67
of 4 cores
lock wait
0.0 ms
no critical section
core wait
0.5 ms
queued for a core
switch overhead
0.3 ms
5 switches/task
CPU utilisation38%
Lock utilisation (no critical section)0%
healthy
Retiring 150/s at 38% CPU. Headroom remains; the next constraint appears at about 267/s.
Change one thing · each preset snapshots the current settings first
healthystatus comes from the model’s discriminated result, not from reading the sentence belowSIMULATED

CPU parallelism simulator

Scaling 100 CPU tasks
100 independent tasks of 20 ms each. The tasks do not share anything — the job around them does.
SIMULATEDA composed model, not a benchmark.

Amdahl’s term, a synchronisation term, an oversubscription term and a bandwidth ceiling, each one a knob you can switch off. Real curves have more causes than four and are rarely this smooth. There is no ideal core count to read off this chart.

Cores
The serial part is the split and the merge, not the tasks. The sync term is what each worker pays to coordinate with the others. The ceiling is where the memory system stops feeding cores, whatever the core count says.
1 workerdashed = linear speedup16 workers · max 16.0×
ideal
4.0× · 500 ms
Amdahl only
3.48×
modelled
3.28× · 610 ms
efficiency
82%
Where the 4× went
delivered3.3×
lost to the serial part0.5×
lost to sync, switching and bandwidth0.2×
At 4 cores the model delivers 3.28× of a possible 4×, so 109 ms of the run is overhead rather than work. The serial part dominates. Splitting the input, merging the results and the one section that cannot overlap now cost more than the cores save — and no core count fixes that term.
One hundred tasks that share nothing still do not scale linearly, because the job that owns them is not the tasks. Read the gap between the dashed line and the curve as the price of coordination — and note it is charged even when every task is independent.
limited by: serialSIMULATED

Three I/O calls, one thread

Three I/O calls, one thread
Each request costs 2 ms to dispatch, waits on the network, then costs 5 ms to parse. Nothing here is parallel — there is exactly one thread in both runs.
Event loop
idle — nothing else to run
idle — nothing else to run
idle — nothing else to run
GET /profile
awaiting I/O 120 ms
GET /flags
awaiting I/O 40 ms
GET /orders
awaiting I/O 80 ms
↑ done 261 ms
runningreadywaitingblockedidlems
wall clock
261 ms
CPU actually used
21 ms
thread-time spent waiting
0 ms
threads used
1
sequential   const a = await getProfile(); const b = await getFlags(); const c = await getOrders()
concurrent   const [a, b, c] = await Promise.all([getProfile(), getFlags(), getOrders()])

wall clock   261 ms  →  127 ms       (2.06× less waiting)
CPU used     21 ms  →  21 ms       (identical — no extra core was touched)
261 ms of wall clock to do 21 ms of work. 92.0% of the run is the event loop sitting idle with nothing to do, because each `await` suspends the whole chain before the next request has even been issued. The three calls are independent — nothing in `getFlags()` needs the profile. Flip the toggle and the same thread, the same code path and the same CPU budget finish in 127 ms.
SIMULATEDRUNTIME-SPECIFIC

What people believe, and what is true

Claim

This handler is I/O-bound, so async is enough.

Reality

The handler is 88% waiting and still holds a core for 1.4 seconds. The 12% is what stalls the loop, and the dominant phase tells you nothing about it.

Claim

CPU-bound means it will parallelise.

Reality

Single-stream gzip is entirely CPU-bound and entirely sequential. Compute-bound says "not concurrency"; only partitionability says "parallelism".

Claim

The box is at 100% CPU, so we need more cores.

Reality

It may be memory-bandwidth-bound, in which case the cores are stalled on loads and look busy. Check achieved bandwidth before buying anything.

Go deeper

Overview

Ask what the work does with its wall clock: sit on a core, or sit in a queue. Computing wants cores; waiting wants overlap.

Practical

Measure CPU time over wall time per phase. Below 0.2 is waiting, above 0.8 is computing, in between means you have not split finely enough yet.

Advanced

The classification is a property of a phase on specific hardware with specific data, and it moves. Adding a cache converts waiting into computing; growing the dataset converts computing into memory-bandwidth-bound. Re-derive it when the system changes rather than inheriting a label.

Apply it