The question this answers
Does this workload need overlapping progress, simultaneous execution, both, or neither?
Two candidate services on identical four-core boxes: a gateway that fans a request out to five upstream APIs, and a report renderer that turns 40 MB of rows into a PDF.
The gateway shares a connection pool and a response accumulator across in-flight requests. The renderer shares only the input rows, which are read-only after load — the single most useful property either of them has.
Whichever model is chosen, every request produces a response derived from all five upstream calls, and every report contains every input row exactly once, in the same order it would have had if a single thread had done the whole thing.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Four quadrants, and all four exist
The definitions live in [[concurrency-vs-parallelism]] over in Operating Systems, and they are worth reading once. What matters here is that the two axes are genuinely independent, which is why all four boxes have real inhabitants and why "is it concurrent or parallel?" is a badly formed question.
The one people trip over is *parallel but not concurrent*. A SIMD loop adding two million-element arrays does four lanes at once with no interleaving, no tasks, no scheduler and no shared mutable state — nothing about it is concurrent in the structural sense, and it needs no synchronisation whatsoever. See [[simd]].
The one people ship by accident is *concurrent but not parallel*: an event loop handling 2000 connections on one thread. Structurally there are 2000 things in progress. Physically one core is executing, and if any handler computes for 40 ms, the other 1999 wait 40 ms.
| Not parallel (one core executing) | Parallel (several cores executing) | |
|---|---|---|
| Not concurrent (one thing in progress) | A plain script. One order, fully deterministic, no coordination. The correct default. | A SIMD or GPU kernel: one logical operation, many lanes, no tasks and no synchronisation. |
| Concurrent (many things in progress) | An event loop or a coroutine runtime: thousands of tasks, one executing thread. Overlaps waiting, not computing. | A thread pool serving requests: many tasks, several cores. Both ceilings apply, and so do both failure classes. |
The two workloads, side by side on the same box
The gateway spends 96% of each request waiting on sockets. Give it four threads and you get four concurrent requests; give it an event loop and you get as many as the connection pool and the file-descriptor limit allow. Adding cores does nothing measurable, because no core was busy in the first place. This workload wants concurrency.
The renderer spends 98% of its time in layout and compression, both of which are pure computation over data already in memory. Give it an event loop and you get exactly one report at a time, rendered no faster, with the entire loop frozen for the duration. Give it four cores and four page ranges and you get most of a 4× reduction in wall clock. This workload wants parallelism.
Note the asymmetry in the failure mode. Under-using concurrency on the gateway costs throughput: requests queue, latency climbs, nothing breaks. Putting the renderer on the event loop costs *everything*: one report stalls every unrelated request on the process for the whole render. See [[blocking-the-event-loop]].
Applying the wrong one
The cheapest way to internalise this is to watch each model applied to the wrong workload. Both snippets below look like a reasonable engineer trying to make something faster. Neither does.
The failure in the first is quiet: Promise.all over CPU-bound work runs the work sequentially on one thread and *also* removes the ordering you had. The failure in the second is loud but misattributed: a thread per socket at 5000 connections spends most of its scheduling budget on context switches, and the graph that moves is CPU, so somebody concludes the box is too small. See [[oversubscription]] and [[context-switching-cost]].
The correct pairing is unglamorous: async or an event loop for the waiting-bound service, a small worker pool sized near the core count for the compute-bound one, and — when the workload is genuinely mixed — both, with the CPU work moved off the loop onto workers. [[hybrid-runtimes]] is that shape.
1// 400 pages of layout. Nothing here waits on anything.2const pages = await Promise.all(3 ranges.map(async (r) => renderRange(r)) // renderRange is pure CPU4)5// Same one thread. Same total CPU. Same wall clock.6// What changed: the event loop is now frozen for the whole render,7// every unrelated request on this process is stalled behind it,8// and a throw in range 3 discards ranges 1, 2 and 4 mid-flight.1// One worker per core, page ranges handed over as messages.2const pool = new WorkerPool(os.availableParallelism())3const pages = await Promise.all(4 ranges.map((r, i) => pool.run({ range: r, index: i }))5)6pages.sort((a, b) => a.index - b.index) // completion order is not page order7// Four cores execute simultaneously; the event loop stays responsive8// because no layout code ever runs on it.The first snippet is concurrent and not parallel, applied to work that needed the opposite. Promise.all schedules; it does not add execution units. The second moves the computation to threads that can occupy other cores and pays the explicit price: serialisation across the worker boundary, and a sort because completion order is not page order.
Key points
- Concurrency and parallelism are independent axes: a system can be either, both or neither, and all four combinations ship.
- Waiting-bound work wants concurrency. Compute-bound work wants parallelism. Getting this backwards produces effort with no result.
- Adding threads to a waiting problem changes memory and context-switch cost, not wall clock — the threads block in exactly the place the async version awaited.
- Adding an event loop to a computing problem is worse than doing nothing: it serialises the work *and* stalls everything else sharing the loop.
Promise.allandgatherare concurrency constructs. They introduce no execution units and make nothing parallel on their own.- Real servers are usually mixed, and the answer is usually both: a loop for the I/O, a bounded pool for the CPU.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • Classify the work first: measure where wall-clock time goes — in a syscall waiting, or on a core computing. See
[[classifying-the-work]]. - • If waiting dominates, choose a model that lets a waiting task release its execution context: async/await, coroutines or an event loop.
- • If computing dominates, choose a model that can occupy more than one core: threads in a language without a global interpreter lock, processes in one that has, or a parallel algorithm.
- • If both dominate different phases, split the phases and give each the right model, with an explicit hand-off queue between them.
- • If neither dominates because the work is small, stay sequential and stop.
- • Gateway, async: request 1 dispatches 5 calls and suspends; request 2 runs on the same thread; call 3 of request 1 returns and resumes it — one thread, both requests advancing.
- • Gateway, thread-per-call: 5 threads each block in
recv(); the scheduler runs none of them; wall clock is identical to async and memory is 5 stacks higher. - • Renderer on the loop: request 47 begins layout; requests 48 through 300 arrive and sit in the accept queue for 4 seconds; a health check times out and the orchestrator restarts a perfectly healthy process.
- • Renderer on 4 workers: ranges finish in the order 2, 4, 1, 3; if pages are concatenated in completion order rather than sorted, the PDF is silently out of order — a correctness bug the parallel version introduced and the sequential one could not have.
- • Concurrency guarantees a blocked task does not hold the execution context. It does not guarantee any increase in computing capacity.
- • Parallelism guarantees simultaneous execution given independent work and free cores. It does not guarantee speedup — the serial fraction and the coordination decide that. See
[[amdahls-law]]. - • Neither guarantees ordering. Any output order you need must be re-established explicitly at the join.
- • Choosing either does not make shared state safe. It only decides which kind of unsafe you are exposed to: interleaving at suspension points, or genuinely simultaneous memory access.
- • Gateway: contention is on the connection pool and on upstream rate limits, not on the CPU. Raising concurrency raises pool waits before it raises anything else. See
[[connection-pool-saturation]]. - • Renderer on threads: contention is on memory bandwidth and last-level cache. Four cores streaming 40 MB each do not get four times the bandwidth.
- • Renderer on the event loop: contention is total — every task on the loop contends with the render for the only thread there is.
- • Mixed model: contention moves to the hand-off queue, which is where you want it, because a queue is something you can measure and bound.
- • Event-loop starvation: one CPU-bound handler stalls every other task on the loop, and the symptom is unrelated endpoints timing out.
- • Thread explosion: a thread per waiting operation at scale, producing memory pressure and scheduler overhead with no throughput gain.
- • Order loss at the join: results assembled in completion order when the domain required input order.
- • Partial-failure loss: a rejected branch in a fan-out abandons the others mid-flight, leaving orphaned in-flight work. See
[[orphaned-tasks]]. - • Misdiagnosis: CPU rises because of context switching, the team scales out, and the real ceiling — the upstream rate limit — never moves.
- • Concurrency helps whenever the wall-clock time is dominated by something the CPU is not doing: sockets, disks, databases, other people's APIs.
- • Parallelism helps whenever the wall-clock time is dominated by computing, the data can be partitioned, and there are idle cores to partition it onto.
- • Both help together on a request path that fetches (waiting) and then transforms (computing), provided the two phases are kept on different execution resources.
- • Concurrency hurts compute-bound work: it adds scheduling and suspension points and returns nothing, and on a shared loop it actively harms neighbours.
- • Parallelism hurts small work: splitting, dispatching and joining a 3 ms job across four threads reliably costs more than the 3 ms. See
[[parallel-overhead]]. - • Both hurt when the real bottleneck is downstream. Eight parallel workers against a database with ten connections produce eight workers waiting on a pool.
- • CPU time divided by wall-clock time for the operation. Near 1 means computing, so cores. Well below 1 means waiting, so concurrency. Above 1 means it is already parallel.
- • Event-loop lag — the delay between scheduling a zero-millisecond timer and it firing. Anything above a few milliseconds means something CPU-bound is on the loop.
- • Per-core utilisation, not the average. One core at 100% and three at 5% is a parallelism problem wearing an aggregate that reads 26%. See
[[cpu-saturation]]. - • Throughput as a function of concurrency limit. Waiting-bound work keeps climbing until a downstream limit; compute-bound work flattens at roughly the core count and then degrades.
- • Two models in one service means two mental models, two failure vocabularies and a hand-off boundary that must serialise data.
- • Worker boundaries cost copies. Structured cloning or pickling 40 MB to a worker can erase the parallel gain outright.
- • Choosing async colours your entire call graph: in most languages an async function can only be awaited from an async caller, and retrofitting that is a large refactor.
- • Every join needs an explicit failure policy — fail fast, collect all, or partial success — and defaulting to whatever the standard library does is how partial results silently become full ones.
- • Neither: make the sequential version faster. A better algorithm or one removed N+1 query routinely beats a 4× parallel win and adds no failure modes. See
[[performance-tradeoffs]]. - • Move the work off the request path entirely — enqueue it and answer immediately. This converts a latency problem into a throughput problem you can size.
- • Scale out processes instead of parallelising inside one. Four single-threaded processes behind a load balancer give you core utilisation with no shared memory at all, and
[[processes]]explains what that buys.
CPU parallelism simulator
Amdahl’s term, a synchronisation term, an oversubscription term and a bandwidth ceiling, each one a knob you can switch off. Real curves have more causes than four and are rarely this smooth. There is no ideal core count to read off this chart.
Three I/O calls, one thread
sequential const a = await getProfile(); const b = await getFlags(); const c = await getOrders() concurrent const [a, b, c] = await Promise.all([getProfile(), getFlags(), getOrders()]) wall clock 261 ms → 127 ms (2.06× less waiting) CPU used 21 ms → 21 ms (identical — no extra core was touched)
Concurrency lab
They come from a queueing and contention model inside Engineer Atlas. What is faithful is the behaviour: work that waits benefits from more workers, work that computes does not, a wide critical section pins parallelism near 1 no matter how many cores you buy, and arrivals past capacity produce an unbounded queue rather than a large latency. Real arrivals are burstier than this model assumes, so real systems reach every one of these walls earlier than the sliders suggest. Do not quote a millisecond from this page.
What people believe, and what is true
Async makes my code run in parallel.
Async lets a waiting task release the thread. Unless the runtime has more than one execution thread for your code, nothing runs simultaneously. See [[async-is-not-parallel]].
If it is not parallel it is not concurrent.
An event loop with 2000 open connections is intensely concurrent on one core. Concurrency is about how many things are in progress, not how many are executing.
Parallel always means concurrent.
A SIMD kernel is parallel with no tasks, no scheduler and no interleaving. It needs no synchronisation because there is nothing to synchronise.
Go deeper
Overview
Waiting-bound work wants concurrency; compute-bound work wants parallelism. Identify which before choosing a library.
Practical
Compute CPU time over wall time for the hot operation. Near 1 means buy cores. Far below 1 means overlap the waiting. Mixed means split the phases and give each the right model.
Advanced
The two choices have asymmetric blast radius. Under-concurrency on an I/O service degrades gracefully into queueing. Compute on a shared event loop degrades catastrophically, because the cost lands on unrelated work that has no way to defend itself.