The question this answers
Given three independent pieces of work, what are my actual options for running them, and how do they differ?
Three photos uploaded in one request. Each must be decoded, resized to a thumbnail and written to object storage before the request can be answered.
The three resize operations share nothing while they run — separate pixel buffers, separate output keys. They do share one thing: the counter that tracks how many are still outstanding, because someone has to decide when the request is finished.
Each of the three photos produces exactly one thumbnail, and the request is reported complete exactly once, after the third thumbnail exists — regardless of the order the work runs in or how it is spread across cores.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Three pieces of work, three possibilities
Sequential means the machine runs A to completion, then B, then C. There is one order, it is the order you wrote, and while A waits four milliseconds for the disk the core does nothing. The behaviour is completely determined by the source, which is why sequential code is the easiest code in the world to reason about and the reason you should keep it until you have a reason not to.
Concurrent means A, B and C are all *in progress* at the same time, but not necessarily *executing* at the same time. One core switches between them: it runs A until A blocks on the disk, runs B until B blocks, runs C, comes back to whichever is ready. Three tasks are alive; one instruction stream is advancing. Wall-clock time drops because the waiting overlapped, not because more computing happened.
Parallel means A, B and C execute at the same instant on three different cores. Three instruction streams advance simultaneously. Wall-clock time drops because more computing happened per unit of time. This is the only one of the three that requires hardware you might not have.
The timeline below is the concurrent case, and it is worth staring at: the core lane is busy almost the whole time, but no task lane is running for more than a fraction of it. That gap — busy core, mostly-waiting tasks — is what concurrency is for. See [[overlapping-progress]] for what each of those lane colours actually means to the scheduler.
The same work, three shapes, three bills
None of these is "the fast one". Each is fastest for a different reason and each fails differently. Sequential is bounded by the sum of every step including every wait. Concurrent is bounded by the sum of the *computing* plus the longest remaining wait. Parallel is bounded by the slowest chunk plus whatever coordination you added to split and rejoin.
The row that surprises people is the last one. Sequential code has no interleavings to reason about, so it has no interleaving bugs. The moment you pick either of the other two you have bought yourself a class of failure that your tests will pass through cleanly. That is not an argument against concurrency; it is the price, and [[concurrency-has-a-cost]] puts the whole invoice in one place.
| Shape | What the machine does | Bounded by | Wins when | What it costs you |
|---|---|---|---|---|
| Sequential | One task runs to completion, then the next | sum of all compute + all waiting | the work is tiny, or ordering between the items genuinely matters | nothing — no coordination, no interleavings, fully deterministic |
| Concurrent (1 core) | Tasks take turns; a task that waits yields the core | sum of compute + the last outstanding wait | the tasks spend most of their time waiting on something else | interleavings at every suspension point; shared state now needs thought |
| Parallel (3 cores) | Three instruction streams advance in the same instant | the slowest chunk + split/join overhead | the tasks spend most of their time computing and are independent | real simultaneous memory access, plus split, join and merge overhead |
| Concurrent AND parallel | A pool of threads, each interleaving many tasks | whichever of the two ceilings you hit first | mixed workloads at server scale — the common real answer | both of the above, and a scheduler you no longer fully model |
Where the shared counter kills you
The three resize jobs share nothing, which is why they parallelise so cleanly. But something has to notice that all three are done. The obvious implementation is a counter: start it at three, each finishing job decrements it, and whoever brings it to zero sends the response.
That counter is now shared mutable state touched by three tasks, and remaining -= 1 is not one operation. It is a read, a subtract and a write, with room between each for another task to run. The schedule below is a legal execution of correct-looking code in which every thumbnail is written successfully and the request never completes.
This is the shape of essentially every bug in this domain: the work was fine, the coordination was not. [[interleavings]] and [[reasoning-about-races]] teach how to find these schedules deliberately rather than by waiting for production to find them for you.
| # | Task A (photo 1) | Task B (photo 2) | Task C (photo 3) | State |
|---|---|---|---|---|
| 1 | · | · | thumbnail written; read remaining (3) | remaining=3 written=1 |
| 2 | · | · | write remaining = 2 | remaining=2 written=1 |
| 3 | · | · | test remaining == 0 → false; return | remaining=2 written=1 |
| 4 | thumbnail written; read remaining (2) | · | · | remaining=2 written=2 |
| 5 | · | thumbnail written; read remaining (2) | · | remaining=2 written=3 |
| 6 | write remaining = 1 | · | · | remaining=1 written=3 |
| 7 | · | write remaining = 1 | · | remaining=1 written=3 ✕ All three thumbnails exist but remaining is 1. Two decrements collapsed into one, so the counter can never reach 0. |
| 8 | test remaining == 0 → false; return | · | · | remaining=1 written=3 |
| 9 | · | test remaining == 0 → false; return | · | remaining=1 written=3 |
Key points
- Sequential, concurrent and parallel are three different things: one order, overlapping progress, and simultaneous execution.
- Concurrency is about *structure* — several things in progress. Parallelism is about *execution* — several things running in the same instant.
- Concurrency cuts wall clock by overlapping waiting; parallelism cuts it by doing more computing per unit of time. They fix different problems.
- Independent work parallelises cleanly. It is the coordination you bolt on — a counter, a flag, a result list — that introduces the bugs.
- A read-modify-write of a shared counter is three operations, and another task can run between any two of them.
- Sequential code has no interleavings and therefore no interleaving bugs. That is a real feature, and giving it up should be a decision.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • Sequential: the runtime executes one instruction stream; each call returns before the next begins, and a blocking wait idles the core.
- • Concurrent: the runtime holds several tasks, runs one until it blocks or is preempted, saves its state, and resumes another that is ready.
- • Parallel: the OS places runnable threads on distinct cores, and those cores retire instructions in the same clock cycles. See
[[scheduling-problem]]. - • The switch point is where interleaving becomes possible: a blocking syscall, an
await, a timer interrupt, or in parallel execution, any instruction boundary at all. - • Completion detection — "are all three done?" — is a separate piece of shared state from the work itself, and needs its own reasoning.
- • A runs fully, then B, then C — the sequential schedule; the counter is decremented three times with no overlap and the invariant holds trivially.
- • A starts, blocks on storage, B starts, blocks, C starts, blocks; completions arrive in the order C, A, B — a legal concurrent schedule with a completion order that does not match submission order.
- • A reads remaining (2); B reads remaining (2); A writes 1; B writes 1 — one decrement vanishes and the request never completes.
- • All three write their thumbnails in the same instant on three cores; if the counter is a plain integer, two cores can perform the read-modify-write with genuinely overlapping memory access, which is a data race and not merely a race condition. See
[[data-races]].
- • Concurrency guarantees that a task which is waiting does not prevent another task from running. It guarantees nothing about which task runs next, or for how long.
- • Parallelism guarantees more instructions retired per second given enough independent work. It guarantees nothing about your program getting faster, because your program may not be compute-bound.
- • Neither guarantees ordering. If the response must list thumbnails in upload order, you sort at the end; you do not assume completion order.
- • Neither makes any shared read-modify-write atomic. Both make it more likely that you notice.
- • The three resize jobs contend for nothing while computing — separate buffers, separate outputs.
- • They contend for the completion counter at exactly one instant each, which is enough.
- • On one core they contend for the core itself, but only when more than one is runnable, which for I/O-heavy work is rare.
- • On three cores they contend for memory bandwidth: three decoders streaming pixel data can saturate the bus before they saturate the ALUs. See
[[memory-bandwidth-limits]].
- • Lost update on the completion counter: two decrements become one, the counter never hits zero, the request hangs.
- • Nondeterministic ordering: results assembled in completion order, so the response body differs between runs and a snapshot test flaps.
- • Partial failure invisibility: photo 2 throws, nobody decrements, and the request hangs for the same reason but a different cause.
- • Over-decrement: a retried task decrements twice, the counter passes zero, and the response is sent before photo 3 exists.
- • Three photos each waiting 300 ms on object storage: sequentially about 900 ms of mostly-idle waiting, concurrently roughly the longest one.
- • Three photos each needing 400 ms of decode on a four-core machine: parallel execution genuinely cuts wall clock.
- • Anywhere the units of work are already independent and you were only running them in sequence because a
forloop is the default.
- • Three 8 KB avatars that resize in 2 ms each: the task setup costs more than the work, and the sequential loop is both faster and correct by construction.
- • When the "independent" items are not — thumbnail 2 needing metadata that thumbnail 1 writes makes the concurrency a race with extra steps.
- • When the downstream cannot take it: three concurrent PUTs per request times 500 requests per second is 1500 concurrent PUTs, and the storage provider has an opinion. See
[[parallelism-overloads-dependencies]].
- • Wall clock versus CPU time for the request handler. Wall clock much larger than CPU time means waiting, which means concurrency has room; roughly equal means computing, which means you need cores.
- • Request duration distribution before and after — and specifically p99, because overlapping work moves the mean long before it moves the tail.
- • Count of completed thumbnails against count of completed requests. A steady divergence is the hung-counter bug, and it shows up nowhere else.
- • Whether the response body ordering is stable across 100 identical requests. If it is not, something is assembling results in completion order.
- • You now need a completion protocol — a counter, a latch, a
Promise.all— and it is shared state with its own correctness argument. - • Stack traces stop telling the whole story: the interesting frame is on a different task, and the code that scheduled it has already returned.
- • Error handling forks. Sequential code has one place a failure surfaces; concurrent code has one per task plus the join point, and swallowed task exceptions become invisible.
- • The test suite no longer covers the failure. Interleaving bugs pass a green build and appear under production timing.
- • Keep the loop sequential. For three small items on a path that is not hot, this is very often the correct engineering answer and it costs nothing to reason about.
- • Push the work off the request entirely: accept the upload, enqueue three jobs, answer immediately. See
[[background-jobs]]and[[async-job-pattern]]— the latency problem disappears rather than being parallelised. - • Batch the operation if the downstream supports it. One request that uploads three objects beats three concurrent requests on every axis including cost.
Two increments, twenty schedules: find the one that loses an update
| # | Task A — counter++ | Task B — counter++ | State |
|---|---|---|---|
| 1 | rA ← counter | · | counter=0 rA=0 rB=0 |
| 2 | rA ← rA + 1 | · | counter=0 rA=1 rB=0 |
| 3 | counter ← rA | · | counter=1 rA=1 rB=0 |
| 4 | · | rB ← counter | counter=1 rA=1 rB=1 |
| 5 | · | rB ← rB + 1 | counter=1 rA=1 rB=2 |
| 6 | · | counter ← rB | counter=2 rA=1 rB=2 |
Scheduler timeline
What people believe, and what is true
Concurrency means things happen at the same time.
Concurrency means several things are in progress. Parallelism means several things are executing in the same instant. One core running twenty connections is concurrent and not parallel at all.
Concurrency makes the program faster.
It makes waiting overlap. If the program is not waiting, concurrency adds switching cost and returns nothing. See [[classifying-the-work]].
The tasks are independent, so there is nothing to synchronise.
The work is independent; the completion tracking is not. Almost every fan-out bug lives in the join, not in the branches.
Go deeper
Overview
Three jobs can run one after another, take turns on one core, or run at once on several cores. Those are sequential, concurrent and parallel, and they solve different problems.
Practical
Ask what the work is doing while the wall clock advances. If it is waiting, overlap it. If it is computing, spread it. If it is neither for very long, leave the loop alone.
Advanced
The independence of the work does not transfer to the coordination. A fan-out of N independent tasks introduces exactly one new piece of shared state — the completion state — and that is where the interleaving bugs concentrate.