Learn Concurrency & Parallelism

Three photos need thumbnails. They can run one after another, they can take turns on one core, or they can run at the same instant on three cores. Those are three different things, and almost every concurrency argument is really an argument about which one someone meant.

Q · Given three independent pieces of work, what are my actual options for running them, and how do they differ?

A system can be concurrent and not parallel, parallel and not concurrent, both, or neither. Operating Systems defines the two words; this lesson is about picking. Get it wrong and you will add threads to a waiting problem, or an event loop to a computing problem, and both feel like doing something.

Q · Does this workload need overlapping progress, simultaneous execution, both, or neither?

A request blocked for 500 ms on a database leaves a core doing nothing 500 million times over. Concurrency is the mechanism for giving that core to somebody else. Everything else in the model — event loops, coroutines, async I/O — is an implementation of that one idea.

Q · What is a thread actually doing while it waits 500 ms for a database, and what could it be doing instead?

One billion floats to sum. One core does it in 400 ms. Four cores should do it in 100 ms and will actually do it in about 130. Parallelism exists because a single core stopped getting faster around 2005 and the only remaining lever was more of them.

Q · When the work is pure computation and one core is already saturated, what is left?

Before you can choose an execution model you have to know what the work is doing with the wall clock. Compression, resizing, encryption, parsing and numerics compute. Network calls, database queries, disk reads and third-party APIs wait. Most real handlers do both, in phases, and the phases want different models.

Q · Is this unit of work spending its wall clock on a core or in a wait queue, and what does that dictate about how to run it?

Seven questions — bound, sharing, independence, ordering, task count, duration, isolation — and seven answers: sequential, threads, processes, async, worker pool, message passing, parallel algorithm. Every answer comes with why it wins, what it costs and how it fails, because a recommendation without those three is a preference.

Q · Given what I now know about this work, which execution model should carry it — and what am I signing up for?

A task is running, ready, waiting or blocked, and exactly one of those four uses a core. "Concurrent" means several tasks are in some state other than finished. Almost every confusing latency number resolves once you know which of the four a task was in and for how long.

Q · When we say several tasks are making progress at once, what is each of them actually doing at a given instant?

The counterweight to everything else in this domain. Coordination costs cycles, context switches cost cache, tasks cost memory, and the bugs cost you the one property you relied on — that a passing test means the code works. Concurrency is worth buying often. It is never free.

Q · What does concurrency cost, and how do I decide whether this workload can afford it?

Separate address spaces, explicit communication, contained crashes. The kernel mechanics live in Operating Systems; what matters here is the design consequence — a process model makes shared mutable state impossible by construction and makes every piece of sharing you actually need visible in the code.

Q · What does running work in separate processes actually buy me, and what does it stop being able to protect?

What a thread is belongs to Operating Systems. What matters here is the consequence of the design: every thread in a process can reach every object every other thread can reach, with no ceremony, no declaration and no error. Sharing is the default, and defaults are what you forget to check.

Q · If every thread can reach every object, which of those objects actually needs protecting, and how do I know?

Six dimensions decide it: isolation, memory sharing, startup cost, communication cost, crash impact and access to cores. Operating Systems compares the mechanisms; this lesson is the choice, and the choice is usually made by the crash column rather than the performance one.

Q · For this unit of work, do I want an execution stream that shares everything or one that shares nothing?

The abstraction most engineers skip past. A task is a unit of work the runtime may schedule; a thread is an execution stream the OS schedules. Ten thousand tasks can live on four threads. Everything confusing about async — why it scales, why it stalls, why single-threaded code still races — follows from this one gap.

Q · When I spawn ten thousand tasks, what actually exists — and what is running?

An ordinary function has two control points — call and return. A coroutine has four: call, suspend, resume, return. That single addition is what lets one thread hold ten thousand in-progress operations, and it is also what silently removes the atomicity your code was relying on without ever saying so.

Q · What does it mean for a function to pause in the middle, and who decides when it resumes?

In standard CPython builds, one lock serialises bytecode execution, so CPU-bound threads do not scale across cores — while I/O-bound threads and asyncio work fine, because the lock is released around blocking calls. Processes have separate interpreters and do scale. Free-threaded builds exist and change this, and which one you have is a property of your build.

Q · Why does adding threads to a CPU-bound Python program not make it faster, and what does?

JavaScript *execution* is one event loop per agent — but the runtime around it is thoroughly multi-threaded: Node does file and crypto work on a thread pool, network I/O through the kernel's readiness API, and both Node and browsers offer real parallelism through workers with message passing.

Q · If JavaScript runs one piece of code at a time, what exactly is doing the other work — and where does real parallelism come from?

std::thread, std::async, std::mutex, std::atomic — and, since C++11, a formally defined memory model in which a data race is undefined behaviour. Not "you get a stale value": undefined, meaning the compiler was entitled to assume it could not happen and optimised on that basis.

Q · What does C++ actually promise about concurrent memory access, and what happens when I break the promise?

A queue of ready callbacks drained by one logical execution context, with the runtime performing I/O somewhere else. That shape buys you freedom from data races on your own heap — and buys you nothing at all against race conditions across await points, which is where the bugs actually are.

Q · If one event loop drains every callback in turn, what can still change between the two halves of my handler?

The syntax reads like blocking code; the execution model is start → suspend → return control → other work runs → resume. Everything you must reason about lives in the gap: state can change across an await, and the code around it stops being atomic the moment you add one.

Q · What actually happens at an `await`, and what may have changed by the time the next line runs?

A handle to a result that does not exist yet: pending, then settled with a value or an error, once and permanently. The interesting questions are who runs the work, when it starts, what happens if nobody looks at the handle, and whether the result can be consumed twice.

Q · What does a handle to unfinished work actually promise me, and who is doing the work while I hold it?

Real OS threads in Node, each with its own event loop, its own heap and no shared objects by default. You move CPU work off the main loop by sending a message; the cost is that "sending" means structured-cloning or transferring, and the moment you reach for SharedArrayBuffer you get real data races back.

Q · How do I get CPU work off the event loop without giving up the safety the single-loop model was buying me?

The browser version of the same bargain, with a harder constraint: the main thread also renders. Anything over roughly 50 ms on it is a dropped frame or an unresponsive click, so the question is not "is this slow?" but "does this belong on the thread that paints?".

Q · Which work must leave the thread that renders, and what can actually cross the boundary?

An honest decision, with real costs on both sides. The loop wins on many waiting tasks, cheap per-task memory and a simpler shared-state model. Threads win on blocking libraries, parallel CPU work and code that is easier to read and debug. Most production systems end up hybrid, and that is not a failure of the choice.

Q · For this workload, does an event loop or a thread-per-task model actually cost less — in latency, memory, and engineer-hours?

Marking a function async does not make it run on another core. Async lets one execution context switch between tasks *while they are suspended* — so it buys you nothing at all when the tasks never suspend. A CPU-bound loop inside an async function is a CPU-bound loop that also allocates a state machine.

Q · Why did wrapping this CPU-heavy function in `async` and awaiting them all together change nothing?

One synchronous 200 ms handler does not cost 200 ms — it costs 200 ms multiplied by everything that was waiting. The work is correct, the endpoint is fast in isolation, and the symptom appears on completely unrelated routes, which is why this is diagnosed late and blamed on the wrong service.

Q · Why did one slow synchronous function make every unrelated endpoint slow at the same time?

The whole difficulty of concurrency compresses into one sentence: two tasks reach the same state, and at least one of them changes it. Remove either half — the reaching or the changing — and every lesson after this one becomes unnecessary.

Q · Two tasks can both reach the same state — when does that create a coordination requirement, and when does it create none at all?

x = x + 1 is not one operation. It is read, modify, write, and the runtime may put another task between any two of them. Learning to enumerate those orderings — and to recognise the one that loses an update — is the single skill this domain is built on.

Q · If two tasks each run three indivisible steps, what are the possible orderings, and which of them produce a wrong answer?

Synchronization exists to preserve invariants, and nothing else. "Add a mutex" is not a design decision until you can finish the sentence "so that ___ is never observed to be false". This lesson is about writing that sentence first, because it determines the primitive, the region and the test.

Q · What must remain true under every possible interleaving — and how does naming it decide which primitive you need?

Operating Systems defines what a critical section is. The engineering skill is finding the *minimal* one: the smallest region that must be indivisible for the invariant to hold. Too wide and you serialise work that did not need serialising; too narrow and the invariant breaks while every access is dutifully locked.

Q · What is the smallest region of this function that must be protected, and how do I know I have not made it one statement too small?

A race condition is correctness that depends on timing. Finding one is not a matter of staring harder — it is a repeatable procedure: list the shared accesses, put a task switch after each, and ask what the other task could do in that gap. This lesson turns the check-then-act shape into a drill you can run on any diff.

Q · Given a function I did not write, how do I systematically find the schedule that breaks it?

Two different words for two different things, used interchangeably by almost everyone. A race condition is a logical bug: correctness depends on timing. A data race is a *memory model* violation: unsynchronized conflicting access to one location, at least one of which writes. You can have either without the other, and in C++ the second one is undefined behaviour rather than a wrong answer.

Q · What exactly is the difference between a race condition and a data race, and why does the distinction change what I am allowed to assume?

Some operations look indivisible in source and are not. counter++, list.append(x), dict[k] = dict[k] + 1, if not present then insert — each is one expression and several steps. This lesson is about learning what your language actually promises, which is almost always less than the syntax suggests and occasionally more.

Q · Which operations does my language actually promise are indivisible, and which merely look that way?

Run the same concurrent program twice on the same input and you may get two different answers. Sometimes that is correct and even desirable; sometimes it is the bug. This lesson is about telling those apart, and about what nondeterminism does to the value of a passing test.

Q · When is "the output differs between runs" acceptable, and when is it the symptom I am chasing?

A mutex gives you one thing: at most one task inside the region at a time. Operating Systems covers how that is implemented. What matters here is which invariant a given mutex is protecting, what it costs the tasks that wait, and the long list of things engineers expect from it that it does not provide.

Q · Which invariant does this particular mutex protect, and which of the guarantees I am assuming does it actually give me?

The single highest-yield concurrency review question: what is inside this lock that does not need to be? A lock held across a network call has a hold time set by someone else's p99, which makes their outage your outage and their latency your throughput ceiling.

Q · What is inside this critical section that does not belong there, and what does keeping it there cost?

Many readers or one writer. The idea is obviously good and the practice frequently is not: a read/write lock has a more expensive uncontended path than a mutex, it can starve writers, and it only pays when reads dominate *and* the region is long enough for the concurrency to matter. Most of the time a plain mutex or an immutable snapshot wins.

Q · Does allowing readers to proceed in parallel actually pay here, or am I adding overhead and a starvation risk for nothing?

A semaphore is a counter with a waiting room. Its natural use is not mutual exclusion but *resource limiting*: at most N tasks may be doing this at once, where N is the size of something real — a connection pool, an upload buffer, a downstream partner's rate limit. The failure that matters is the leaked permit.

Q · How do I express "at most N of these at a time", and what happens to the permit when the task in the middle throws?

A binary semaphore and a mutex both admit one task at a time, which is where the similarity ends. A mutex has an owner; a semaphore has a count. That single difference decides reentrancy, priority inheritance, who is allowed to release, whether the primitive can signal across tasks, and which failures are possible at all.

Q · When the two primitives look identical at N = 1, what actually differs, and which failures does each one make possible?

A mutex answers "may I touch this?". A condition variable answers "is it worth touching yet?". It lets a task release the lock and sleep until another task says the state changed — and every correct use of one is built around a predicate checked in a loop, under the lock, both before waiting and after waking.

Q · How does a task wait for a *condition* on shared state without holding the lock and without spinning?

A condition variable stores nothing. If the state changes and the notification is sent at a moment when nobody is waiting, that notification is discarded — and the task that arrives a microsecond later waits forever for an event that already happened. The fix is not a bigger buffer or a retry; it is checking the predicate under the same lock that guards the state change.

Q · Why does a task wait forever for a condition that is already true, and what exactly must be under the lock to prevent it?

A condition variable may return from wait without any notification having been sent. That is permitted behaviour in POSIX, in C++, in Java and in Python — and even where it is not, another task can consume the state between the notify and your wake. Both reasons lead to the same rule: re-check the predicate in a loop, always.

Q · Why must the predicate be re-checked after `wait` returns, and which languages and APIs does that apply to?

The flagship pattern of the domain: one side creates work, a queue holds it, another side runs it. Everything interesting is in the queue — how big it is, what happens when it is full or empty, and who is allowed to notice that the producer has stopped.

Q · When one part of a system creates work faster than another can run it, what exactly must the two sides agree on?

An unbounded queue is not a queue without backpressure. It is a queue whose backpressure mechanism is the OOM killer, whose signal is a process restart, and whose latency is unbounded long before memory runs out.

Q · If the producer is faster than the consumer and the queue has no limit, what actually stops it — and how does that failure present at 3 a.m.?

The conversation a slow component has to have with a fast one. Without it the chain is: producer rate exceeds consumer rate, queue grows, memory grows, latency rises, something dies. Backpressure is the design question of how the slow side says "not so fast" in a way the fast side is forced to hear.

Q · When a downstream stage cannot keep up, how does that fact travel back to whatever is producing the work — and what does the producer do about it?

A typed conduit with send, receive and — the part people get wrong — close. Capacity decides whether a send is a rendezvous or a buffered handoff; closing is how a consumer learns that no more data is coming, and getting it wrong is how pipelines hang on shutdown.

Q · What does a channel guarantee that a shared queue does not, and how does a receiver learn that the sender is finished?

Instead of synchronizing access to shared state, transfer the data and let exactly one party own it at a time. You give up zero-copy sharing and a single global ordering; you get a system with no shared mutable state to protect, which is a different and much smaller problem.

Q · Can we make the synchronization unnecessary by moving the data instead of protecting it — and what does that trade cost?

Private state, a mailbox, and strictly sequential message processing. Inside an actor there is no concurrency at all, which is why there are no locks — and the costs are message overhead, ordering that is weaker than it looks, and the fact that remote actors turn this into a distributed-systems problem.

Q · What do you get by making a piece of state single-threaded by construction, and what does the mailbox cost you?

Before you pick a queue implementation, write down the four requirements: thread safety under how many producers and consumers, what ordering you actually need, whether operations block, and what the capacity is. Most queue bugs are a requirement nobody stated, not an implementation that was wrong.

Q · What are the actual requirements on the queue between these two stages, and which of them is the one nobody wrote down?

Stop accepting, finish what is in flight, signal the consumers, wait for the workers, then exit. Every step is a decision — and what happens to the work still sitting in the queue is a policy you choose, not an accident you discover during a deploy.

Q · When this process is told to stop, what happens to the work in flight, the work in the queue, and the workers blocked waiting for more?

Thread A holds the lock on account 7 and waits for account 12; thread B holds account 12 and waits for account 7. Neither thread is broken, neither lock is broken, and neither will ever run again. The bug is not in either thread — it is the cycle between them.

Q · Two threads are alive, each holding a lock it acquired correctly, and neither will ever make progress again — how do I find the cycle that did it?

Mutual exclusion, hold-and-wait, no preemption, circular wait. All four must hold simultaneously for a deadlock to exist — which makes the list useful for exactly one thing: choosing which one you are going to break, and paying that specific price.

Q · A deadlock needs four things to be true at once — which one is cheapest for me to make false in this particular system?

Five techniques, ordered by leverage rather than by cleverness: use fewer locks, impose an order, never hold a lock across a blocking call, acquire with a timeout, and let a higher-level primitive own the coordination. The first three cost nothing at runtime; the last two buy safety with an error path.

Q · Given a system that could deadlock, which fix gives the most safety for the least ongoing cost?

The practical answer to deadlock, and it fits in one sentence: whenever two accounts must be locked, lock the lower id first. The cycle is not detected or escaped — it becomes impossible to construct, in every schedule, for free.

Q · How do I make circular wait unconstructable rather than merely unlikely?

Both threads are running. Both are executing useful-looking code. Neither has completed anything in four minutes. Livelock is deadlock with a busy CPU graph — and because everything is "working", it is the liveness failure your dashboards are least likely to catch.

Q · Both threads are runnable and neither is blocked, so why has nothing finished?

The system is making progress. Throughput is at target, no thread is deadlocked, and one particular task has been waiting eleven minutes. Starvation is the failure where the aggregate is healthy and a specific participant never wins.

Q · The system is progressing and one task never gets to run — what keeps taking its turn?

A fair lock hands the resource to whoever waited longest. It bounds the worst case, and it is usually slower — often much slower — than the unfair lock it replaced. That trade is the lesson: fairness is bought with throughput, and you should know the price before you pay it.

Q · Should this lock hand the resource to the longest waiter, and what does that decision cost me?

A high-priority task waits on a lock held by a low-priority task, and a medium-priority task — which needs neither the lock nor anything else — preempts the holder and keeps it off the CPU. The highest-priority work in the system is now blocked behind the priority it outranks.

Q · Why is my highest-priority task waiting on work that a medium-priority task keeps interrupting?

Eight cores, one lock, one thread doing useful work and seven parked. The machine reports 12% CPU and the service is at its throughput ceiling. Observability teaches you to spot this from p99; this lesson is about why it happens and the three things that reduce it.

Q · Seven of my eight threads are waiting on one lock — what is the cost, and which of my options actually reduces it?

One slow lock holder makes every other thread queue behind it. That much is obvious. The part that surprises people is what happens next: when the slow operation ends, the queue does not disperse — the threads now arrive in lockstep and keep re-forming the queue long after the original cause is gone.

Q · The slow query finished ten minutes ago — why is the lock still backed up?

Eight threads, eight cores, one global lock. The code is parallel, the hardware is parallel, and the effective parallelism is 1.0 — every thread spends its life queueing for the same mutex. The program is concurrent in structure and serial in execution.

Q · The work is spread across eight threads and the speedup is 1.05 — where did the parallelism go?

Eight cores, sixty-four CPU-bound threads. Every thread gets an eighth of a core, every one takes eight times longer, and the machine spends a measurable share of its capacity switching between them and refilling caches that the previous thread just evicted.

Q · I have eight cores and sixty-four runnable threads — what exactly does the extra fifty-six cost me?

The register save and restore is about a microsecond and it is not the expensive part. The expensive part is the cache, TLB and branch-predictor state the incoming thread destroys, which the outgoing thread must then rebuild — a cost that is paid later, is invisible in any switch counter, and routinely exceeds the switch itself several times over.

Q · What does a context switch actually cost me, and why is the number I was told too small?

The counterweight lesson for the whole domain. Four cores, one thousand CPU-bound threads: the work does not go faster, it goes slower, and it consumes gigabytes to do so. Threads are not a performance knob — they are a way of expressing concurrency, and concurrency is not parallelism.

Q · If threads make things faster, what is the right number — and why does that question have no general answer?

while (!ready) {} occupies a core at 100% doing nothing, and on an oversubscribed machine it actively prevents the thread that would set ready from running. This is not an argument against all spinning — bounded spinning is a real technique — it is an argument against unbounded spinning where a blocking wait belongs.

Q · What does spinning on a flag cost, and when is waiting without blocking ever the right choice?

An atomic operation is one no other thread can observe half-done. That buys exactly one thing: a single memory location read-modify-written without a lock. It does not buy an invariant spanning two locations, and it does not buy check-then-act.

Q · Which operations does an atomic variable actually make indivisible, and which ones does it leave exposed?

CAS writes only if the location still holds the value you read. That turns an arbitrary read-modify-write into an optimistic loop: read, compute, attempt, and on failure re-read and recompute. The loop is the point — and so is the fact that it can spin.

Q · How does compare-and-swap turn a multi-step update into something safe without a lock, and what does the retry cost?

Two atomic variables give you two indivisible operations, not one. Every invariant that relates them lives in the gap between the two operations, and no amount of making each one more atomic closes that gap.

Q · Why do two individually correct atomic operations still break the invariant that spans them?

Lock-free means the system as a whole always advances: no thread can be blocked forever by another thread being descheduled, killed or slow. It says nothing about throughput, latency or simplicity, and it is routinely chosen for the wrong reason.

Q · What does calling an algorithm lock-free actually promise, and what does it deliberately not promise?

One head pointer and one CAS gives you a working push. The educational version fits on a screen and is genuinely instructive — and it is not production code, because everything it leaves out is where lock-free structures actually go wrong.

Q · How does a CAS on a single head pointer implement a whole stack operation, and what does the simplified version leave out?

Lock-free guarantees that some thread completes. Wait-free guarantees that every thread completes, in a bounded number of its own steps. The gap between "some" and "every" is where one unlucky thread starves while every dashboard says the system is healthy.

Q · If lock-free guarantees progress, why can one particular thread still make none?

Your CAS expected A, found A, and succeeded. In between, the value became B and then A again — and the world it pointed at is no longer the world you read. Value equality was never evidence that nothing happened.

Q · The CAS succeeded and the value was exactly what I expected — how can the structure still be corrupt?

Threads do not necessarily observe memory operations in the order your source code wrote them. A memory model is the contract that says which orders are possible — and the language's contract is a different object from the CPU's.

Q · What does a memory model actually define, and why is the language's model not the CPU's?

Happens-before is the relation that answers "will the other thread see it?". It is not about wall-clock time. It is a partial order built from specific paired constructs, and two operations with no path between them are unordered no matter which one ran first.

Q · What actually makes a write in one thread visible to a read in another?

Two independent reorderers sit between your source and what another thread observes. Both are allowed to move memory operations as long as *your own thread's* observable behaviour is unchanged. Synchronization is the only thing that constrains either of them.

Q · Who reordered my code — the compiler, the CPU, or neither — and what actually stops it?

A barrier says which of your memory operations may not move across this point. It does not "flush the cache" — caches are already coherent. Getting that distinction right is what separates a correct ordering argument from a plausible-sounding one.

Q · What does a memory barrier actually constrain, and why is "flush the cache" the wrong mental model?

Two threads update two different counters and never touch each other's data. The counters sit in the same cache line, so every update takes the line away from the other core. Correctness is untouched; the speedup you added a thread for never appears.

Q · Two threads never touch the same variable — why did adding the second thread make it slower?

Each core has its own cache, and the hardware keeps them consistent for you. That consistency is not free: a location written by many cores generates traffic on every write, which is why one shared counter can cap the throughput of a sixteen-core machine.

Q · What does a shared write actually cost when every core has its own cache?

Making a reference visible is not the same as making the object visible. A reader can hold a perfectly valid pointer to an object whose fields it cannot yet see — a half-constructed object, from the reader's point of view, with no null and no error to warn it.

Q · How does one thread safely hand a newly constructed object to another without the other seeing it half-built?

Check without the lock, lock, check again, initialise. It looks like a pure optimization and it is the standard example of why memory-model reasoning matters — the fast path can hand out a reference to an object it cannot see. The fix exists in every language and is different in each.

Q · Why is the obvious "check, lock, check again" lazy initialization broken, and why is the fix language-specific?

A pool is not a performance trick, it is a *bound*. Tasks go into a queue, a fixed set of workers takes them out, and the number of things running at once stops being a function of how many requests arrived. Everything interesting about pools is about what queues behind them.

Q · Why hand work to a fixed set of workers instead of starting a thread for every task?

There is no formula. CPU-bound work relates to core count; waiting-bound work can support many more workers than cores; and both numbers are bounded by something downstream that has its own limit. This lesson gives you the variables, the failure signatures at each end, and the instruction to measure.

Q · How many workers should this pool have, and why can nobody hand me the number?

The pool shape does not care what a worker is. Threads, processes, containers and remote machines all give you the same queueing structure — and radically different startup costs, failure modes and answers to the question "what happens to the task the dead worker was holding?"

Q · The queueing shape is identical whether a worker is a thread or a machine — so what actually changes?

Give every worker its own deque, let it push and pop its own end without synchronizing, and when it runs dry let it steal from the far end of somebody else's. Near-zero coordination in the common case, automatic load balancing in the bad case — paid for with cache locality and one genuinely hard race at the last element.

Q · Why does a runtime give each worker a private queue instead of sharing one, and what does an idle worker do about it?

Every worker busy, the queue growing, wait time climbing. Saturation is not a CPU problem and often not even a pool problem — it is arrival rate exceeding completion rate, and Little's Law tells you exactly what the wait will be before you measure it.

Q · All eight workers are busy and the queue is growing — is the pool too small, or is something else the real constraint?

Ten thousand tasks, a semaphore with fifty permits, at most fifty running. The pattern is four lines; the design decisions are where the limit lives, what number it holds, what happens to task fifty-one, and the release you forgot to put in a finally block.

Q · I have ten thousand independent tasks and firing them all at once destroys something downstream — where exactly do I put the limit, and how do I pick it?

Split the work, run the pieces, wait for all of them, combine. The split is easy and the combine is arithmetic — the join is the part that carries the correctness, because it is the only place a happens-before edge exists between the children and the parent that reads their results.

Q · What exactly does the join give me, beyond "the children finished"?

Sum a billion numbers by summing chunks and combining the partials. The decomposition is trivial and the requirement is not: the combine must be associative, or "the same computation" produces a different answer depending on how the runtime happened to schedule it.

Q · Which reductions can be parallelised safely, and what exactly does the combine operation have to satisfy?

Not a product and not a framework — a computational shape. Transform each input independently, group the intermediates by key, reduce each group. Once you can see the shape you find it everywhere: in a SQL GROUP BY, in a browser tab counting words, in a metrics pipeline, in eight lines of local code.

Q · What is the actual computational pattern here, independent of any system that implements it?

Map, reduce, scan, sort and divide-and-conquer are the primitives almost every parallel program is built from. The useful skill is not implementing them — your library already did — it is recognising which shape a problem has, and knowing that scan is parallelisable at all.

Q · Which computations have a parallel form at all, and how do I recognise the shape of the one in front of me?

Splitting, scheduling, synchronizing and combining are work the sequential version never does. For small tasks that overhead exceeds the task, and the parallel version is measurably slower on more hardware — which is why every parallel decomposition needs a size below which it stops decomposing.

Q · Why is my parallel version slower than the sequential one, on the same machine, with more cores working?

The part that cannot be parallelised sets a ceiling on everything else. If a tenth of the job must happen in sequence, an infinite number of cores still cannot make it more than ten times faster — and long before infinity, each extra core is buying almost nothing.

Q · I parallelised the expensive loop and the job is only twice as fast on sixteen cores — where did the rest of the speedup go?

The complement to Amdahl, not the refutation. Amdahl fixes the problem and asks how much faster a bigger machine makes it. Gustafson fixes the time budget and asks how much bigger a problem the machine lets you solve — and for that question the scaling looks almost linear.

Q · If bigger machines barely help a fixed job, why does everyone keep buying them?

Work is everything the computation has to do; span is the longest chain of steps that must happen one after another. Work divided by workers is the optimistic answer, span is the floor, and no scheduler, runtime or core count gets you below the span.

Q · How much parallelism does this computation actually contain, before I go looking for a machine to run it on?

Draw the edges and the parallelism reveals itself. A depends on nothing; B, C and D depend on A; E depends on D. That graph tells you exactly which tasks can run simultaneously, which chain sets the finish time, and which task is worth optimising — before you write a scheduler or buy a core.

Q · Given a set of tasks and what each one needs, which of them can actually run at the same time?

A single instruction applies the same operation to four, eight or sixteen adjacent elements at once. It is parallelism with no threads, no locks and no interleavings — and it evaporates the moment the loop body branches per element or the next iteration reads what the last one wrote.

Q · When does my loop turn into vector instructions, and what in the loop body stops that from happening?

Two different shapes of "at the same time". Data parallelism runs one operation over many elements; task parallelism runs different operations at once. They fail differently, share state differently, and scale differently — and most real systems are both at different levels.

Q · Is this the same operation applied to many items, or different operations that happen to be independent — and which one am I actually being offered?

A GPU offers thousands of execution units for work that is uniform, arithmetic-dense and enormous. The reasoning question is almost never "can this run on a GPU" — it is whether the work is big enough and uniform enough to repay moving the data there and back.

Q · Is this work uniform enough, arithmetic-dense enough and large enough to be worth shipping to a separate device and back?

Split the work into stages and let item 4 be read while item 3 is parsed, item 2 is processed and item 1 is written. Throughput rises to the rate of the slowest stage. The time any individual item takes does not improve at all — and usually gets slightly worse.

Q · Can different items occupy different stages at the same time, and what does that actually buy — throughput, latency, or neither?

One incoming request issues N calls concurrently and aggregates the answers. It converts a sum of latencies into a maximum — and simultaneously multiplies your load on everything downstream by N, which is the half nobody plans for.

Q · What does issuing N calls concurrently instead of sequentially buy me, and what does it cost the systems on the other end?

Query every shard in parallel and merge the answers. The result arrives at the speed of the slowest shard, which means the whole request inherits the *tail* of every shard it touched — and querying 100 shards turns a one-in-a-hundred slow response into a two-in-three one.

Q · If every shard is fast 99% of the time, how often is a query across all of them fast — and what do I do about the answer?

Every participant stops at the line until all of them arrive, then all of them continue. It is the primitive for phased computation — and its two characteristic failures are a participant that never arrives, which hangs everyone, and code that reads the next phase's data before the barrier, which produces a wrong answer with no error.

Q · How do N workers agree that a phase is finished before any of them starts the next one?

One-shot and asymmetric: waiters wait, workers count down, and when the count reaches zero the gate opens permanently. 3 → 2 → 1 → 0 and it never resets — which is both the whole appeal and the thing people get wrong when they reach for it expecting a barrier.

Q · How does a coordinator wait for N pieces of work to finish, without knowing which one finishes last?

Start N tasks, wait for the collection, aggregate the results. The easy part is the fan-in; the lesson is the failure behaviour — Promise.all rejects the instant one task fails while the other N−1 keep running unsupervised, and every language offers a different, incompatible way to say "tell me about all of them".

Q · When one of five parallel calls fails, what happens to the other four — and what does my caller actually learn?

Three independent awaits in a row take the sum of their latencies for no reason at all. The fix is four characters of syntax and it is genuinely correct — but read the warning before you ship it, because the very next lesson is about the database this fix knocks over.

Q · These three calls do not depend on each other — why does the endpoint take the sum of their latencies?

One request became one hundred parallel database queries. The endpoint got three times faster on the developer's laptop and the database fell over in production. This is the counterweight to the previous lesson: parallelising a fan-out does not reduce work, it concentrates it, and the system that absorbs the concentration is never the one you were optimising.

Q · The endpoint got faster and the database got slower — where did the load actually go?

One event wakes ten thousand waiting tasks and every one of them does the same thing to the same resource in the same millisecond. The trigger is usually benign — a cache entry expiring, a connection restored, a scheduled job at :00 — and the response is always the same shape: spread it, batch it, bound it, or do it once for everyone.

Q · Why does one small event produce a load spike ten thousand times its size, and how do I stop the crowd forming?

One hundred callers need the same value. Issue one fetch and give everyone the same result. The implementation is fifteen lines and two of them are the ones that go wrong: what the dedup key includes, and what happens to the shared in-flight entry when the fetch fails.

Q · A hundred concurrent callers want the same value — how do I make that one fetch, and what do they all get when it fails?

Child tasks belong to a scope, and the scope does not exit until every child has completed or been cancelled. The alternative — tasks that outlive the thing that started them — is how a request handler returns while three of its tasks are still writing to a response that is already closed.

Q · Who owns this task, and what is guaranteed to have happened to it by the time the function that started it returns?

The user closed the tab. Should the database query still run? The HTTP call to the payment provider? The 40-second report generation? Cancellation is a first-class design question, and the uncomfortable answer is that in most runtimes a task that never checks is not cancelled — it is merely marked.

Q · When nobody wants this result any more, what should stop — and what actually will?

Cancel the parent, cancel the children, clean up on the way out. The tree is easy to draw and hard to keep intact, because every layer that does not forward the signal is a subtree that keeps running — and because cancellation is cooperative, an arriving cancel is a request, not an event.

Q · When the root operation is cancelled, which of the twelve tasks it transitively started actually stop — and in what order does cleanup run?

Start, wait, deadline exceeded, give up. The part almost everyone misses: a timeout without cancellation just stops waiting — the work keeps running, keeps its connection, and keeps its share of the CPU, while the caller has already retried.

Q · When this operation takes too long, what exactly happens — to the caller, and to the operation?

A timeout is a duration from now; a deadline is an instant. The difference only matters once there is a chain — and then it matters enormously, because a propagated deadline is how the fourth service in a call chain knows there are 40 milliseconds left rather than starting a fresh 2-second budget.

Q · Does each hop get its own fresh budget, or do they all share one that is already partly spent?

Fire-and-forget with no owner: nobody awaits it, nobody cancels it, and its exception is swallowed. The task runs, fails, and the only trace is a warning on a stream nobody reads — which is why this bug survives for months in code that looks completely ordinary.

Q · This task has no one waiting for it. Who stops it, who bounds it, and where does its exception go?

Mutable shared data is a synchronization problem. Immutable shared data is not a problem at all — any number of readers can touch it simultaneously with no lock, no ordering rule and no critical section. The strongest move in concurrency is removing the need for it.

Q · If the data can never change after construction, what is left to synchronize?

Copying buys isolation and pays in memory and copy time. Sharing buys efficiency and pays in synchronization. This is a genuine decision with a crossover point, not a style preference — and the crossover moves with payload size, task count and how much of the data each task actually touches.

Q · Should each task get its own copy of this data, or should they all point at one?

Readers share one stable version and pay nothing. A writer builds a new version off to the side and swaps the pointer. Nobody blocks anybody. The kernel uses the same idea for fork; a database uses it for MVCC; here it is a way to make a mutable structure behave like an immutable one.

Q · How do I let readers proceed without a lock while a writer is changing the same structure?

Read version 5, compute the change, write only if the row is still at version 5. If it moved, somebody else got there first — retry or fail. No locks held, no blocking, and a failure mode that arrives all at once: retry storms under high contention.

Q · Can I update shared state without holding a lock while I decide what to write?

Take the lock, make the change, release the lock. Conflicts are prevented instead of detected, the loser waits instead of redoing work, and latency becomes predictable — which is exactly the right trade when contention is high or a conflict is expensive to undo.

Q · When is it right to make everyone else wait rather than let them race and retry?

Four numbers decide it: how often two writers collide, how expensive a conflict is to resolve, how long the state must stay stable, and what a retry costs. Optimistic wins comfortably at low contention and degrades badly — not gradually — as contention rises. That curve is the lesson.

Q · At what contention level does letting writers race and retry become worse than making them queue?

"If the singleton is missing, create it" is a check-then-act, and two threads running it produce two singletons — or, far worse, one thread handing out a reference to an object the other thread has not finished building. Once-initialization primitives exist because this is genuinely hard to get right by hand.

Q · What happens when two threads both discover that the thing they need does not exist yet?

Thirteen shapes that recur in every concurrent system, each with the problem it solves, the structure it imposes, what it costs and how it fails. Recognising the shape is most of the work — the implementation is almost always already in your standard library.

Q · What shape does this coordination problem already have a known answer for?

Thirteen things that look reasonable in review and fail in production. Each one is tempting for a specific reason — it is simple, it is fast to write, it makes the test pass — and each one produces a specific, nameable failure. Knowing the temptation is what makes the pattern recognisable in your own code.

Q · Which concurrency constructs look correct in review and fail only under production load?

Busy waiting is usually the wrong answer, and sometimes it is decisively the right one. When the expected wait is genuinely shorter than a context switch, spinning wins — and when it is not, spinning burns a core to make everyone slower. Context decides, and this is one of the few places in the domain where the correct answer depends on the hardware.

Q · When is burning CPU in a loop cheaper than letting the scheduler put the thread to sleep?

A type or operation is thread-safe when its documented guarantees permit correct concurrent use under stated conditions. Not "it has locks" — locks are one implementation. And the trap that catches everyone: a thread-safe method does not make a sequence of thread-safe calls atomic.

Q · What does the phrase "this class is thread-safe" actually promise me?

Can this function be safely entered again before a previous call has finished? That is a different question from whether two threads can call it at once, and the two properties are independent — a function can be either, both, or neither. Confusing them produces self-deadlocks and corrupted state that look like race conditions and are not.

Q · What happens if this function is called again — by a callback, a signal, or itself — before the first call returns?

Promise.all over ten thousand items, or a thread per request with no ceiling. The concurrency limit becomes whatever the input happens to contain, which is not a limit. Sockets, memory, file descriptors, downstream capacity and rate limits all have real ceilings, and you will find whichever one is lowest.

Q · What number limits how many of these run at once — and is that number one I chose?

One worker: 1x. Two: 1.8x. Four: 3.2x. Eight: 4.5x. The missing speedup is not lost to one cause — it is serial work, synchronization, memory bandwidth, coherence traffic and scheduling, each taking a share, and each with a different tell.

Q · I doubled the workers and got 1.4x. Which of the five things that eat speedup is eating mine?

Cores multiplied; the path to RAM did not. Some workloads stop scaling at four workers not because the CPU ran out but because the memory subsystem did — and the tell is that adding cores stops helping while CPU utilization still reads 90%.

Q · Adding workers stopped helping but every core still looks busy — am I out of CPU, or out of memory bandwidth?

A single-threaded loop walks memory in order and every access is nearly free. Split it across eight workers that migrate between cores, interleave their indices and share a last-level cache, and the same total work can move more bytes and take longer.

Q · Why did splitting this loop across eight workers move more memory and run slower than the single-threaded version?

On a multi-socket machine, "RAM" is several pools with different distances. A worker reading memory attached to its own socket is fast; reading memory attached to the other socket goes over an interconnect and costs meaningfully more — and which pool a page lives in was usually decided by whichever thread touched it first.

Q · My parallel job is fast on a single-socket machine and slow on a bigger dual-socket one — why would more hardware be worse?

Tell the scheduler a thread may only run on certain cores. It removes migration, keeps caches warm and cuts latency variance — and it hands you a scheduling decision the OS was making better than you will, on hardware you may not be running on next quarter.

Q · Should I pin these threads to specific cores, and what am I giving up if I do?

Floating-point addition is not associative, so a parallel reduction adds the same numbers in a different order and can produce a different answer — a different one again at a different worker count. No race, no bug, no lost update. Just arithmetic that does not obey the law you assumed it did.

Q · Why does my parallel sum disagree with the sequential one, and disagree differently at 2, 4 and 8 workers?

Concurrent execution is nondeterministic wherever ordering is unspecified. Sometimes that is fine and sometimes it is the bug — and the difference is whether the output you promised was a set or a sequence. Making a parallel computation reproducible is always possible and never free.

Q · The same input produced two different outputs. Is that a bug, or did I promise something I never actually specified?

No ordering, FIFO per producer, causal, total. Each is a different promise about what one observer may see relative to another, each costs progressively more parallelism, and almost every ordering bug is a system that was sold one level and assumed the next one up.

Q · Which ordering does this queue actually guarantee, which one does my code assume, and what would the stronger one cost?

Request rate, error rate and duration describe the work. They say nothing about whether forty threads are asleep in front of one lock. Concurrency observability adds a second axis — where the work is *waiting* — and there are exactly six signals worth the cardinality.

Q · Which signals tell me my system is concurrency-limited rather than slow?

A lock held for 2ms with a p99 wait of 500ms is contended. Neither number says so on its own — 2ms is a fine critical section and 500ms could be a slow dependency. The ratio is the signal, and it is the one number that tells you whether to shrink the section or reduce the arrivals.

Q · Given a lock's hold time and wait time, how do I tell contention from slow work?

A snapshot of every thread's state and stack. Twenty threads parked in the same lock frame is not twenty problems — it is one, and the dump names it. The skill is reading state and stack together, and knowing that one dump is a photograph while two are a story.

Q · The service is stalled and CPU is near zero. What does a snapshot of every thread tell me?

Four OS threads, all idle. Eleven thousand tasks, all pending. A thread dump reports a healthy process; the truth is that every task is parked on a downstream that stopped answering. Async runtimes need a dump of the tasks, because the thread is no longer the unit of work.

Q · My async service is stalled but every OS thread is idle. Where do I look?

A CPU profiler samples threads that are on a processor. A thread waiting for a lock is not on a processor, so it contributes nothing to the profile — and the flame graph of a fully contended service looks empty. Concurrency profiling measures the time you were *not* running, and attributes it to a stack.

Q · The flame graph is nearly empty and the service is slow. What is a CPU profile structurally unable to show me?

A race detector watches memory accesses and reports two that conflict with no happens-before edge between them. That is a *data race* — a precisely defined thing. It is not the same as a race condition, and the gap between the two is where the bugs that survive a clean detector run live.

Q · A detector reports my program is race-free. What class of bug does that actually rule out?

You add a log line to find out what is happening, and the bug stops happening. That is not bad luck — the log line took a lock, allocated, and did a syscall, which reordered the schedule and closed the window. Concurrency bugs resist the normal debugging loop because the loop's first step perturbs the thing being measured.

Q · Why does adding a print statement make my concurrency bug disappear, and what do I do instead?

If you record every nondeterministic input — the order threads were scheduled, which lock was granted to whom, what the clock said, what came back from the network — you can replay the failing execution exactly. It turns a one-in-forty-thousand bug into a file. It is also not free, and knowing what it costs is half the lesson.

Q · Can I capture a failing interleaving well enough to run it again on demand?

A concurrency test that passes is a statement about one schedule out of an astronomical number. To learn anything, run the same scenario thousands of times with randomized scheduling, injected delays and load — deliberately manufacturing the schedules a normal run will not produce until it is in production.

Q · My concurrency test passes. What have I actually learned?

Thread-per-request, thread pool, event loop, async tasks, worker processes. This is a decision with inputs — how much of a request is waiting, how much is computing, how many connections are open at once, and how much complexity the team can carry — and the inputs select the answer.

Q · Which concurrency model should this server use, and what decides it?

Give each request its own thread and let it block. The code is straight-line, the stack trace is the request, and the debugger works. It is the best mental model in the list — and it degrades on three specific axes: memory per thread, scheduler cost at high thread counts, and one connection held per request.

Q · Why is the simplest server model also the one that stops working first, and at what point exactly?

Forty thousand sockets, one thread, and a readiness mechanism that says which of them can be served without blocking. The cost per idle connection drops to a buffer and a descriptor — and in exchange, every handler on that thread is now responsible for the latency of every other connection.

Q · How does one thread serve forty thousand connections, and what does that thread now owe everyone?

Every production server of any size runs an event loop for I/O, a worker pool for blocking and CPU work, and OS threads underneath both. "Threads or async" is a question about one layer of a stack that always has three, and the interesting engineering is at the boundaries between them.

Q · If every real server mixes models, what is actually being decided and where do the bugs live?

Every mainstream UI framework restricts view mutation to a single thread, and it is not laziness — a view tree read by a layout pass while another thread mutates it has no coherent state to draw. So background work runs elsewhere and returns results to that thread, and the whole discipline follows from a frame budget.

Q · Why do UI frameworks insist that only one thread may touch the view, and what does that force on everything else?

Transactions, locks and MVCC give you strong guarantees about rows. They give you nothing about the in-process cache you populated from those rows, the counter you kept in a variable, or the check you performed in application code between two statements. Knowing exactly where the guarantee ends is the lesson.

Q · My database handles concurrency. Which of my concurrency problems does that actually solve?

On one machine, coordination is shared memory: a lock is an instruction sequence and it either succeeds or you wait. Across machines there is no shared memory, only messages — which can be lost, delayed, duplicated or answered by a node that has already died. Locking stops being a primitive and becomes a protocol with failure modes.

Q · Which of my single-machine concurrency intuitions survive when the state is on another machine?

The single most common concurrency mistake in production systems: a lock, a counter, a cache or a "have we already done this?" flag that lives in one process's memory, protecting an invariant that spans every process. It works perfectly in development, where there is one instance, and fails the day you scale to two.

Q · My code is correctly locked and it still double-processed. What did the lock actually protect?

An agent holding a search tool, a database tool and a payments API decides, each turn, what to invoke. Some of those calls are independent and some are not, and the model does not know which. Deciding what may overlap is a dependency question, a rate-limit question, a cost question and — for anything that writes — a safety question.

Q · Given a set of tool calls an agent wants to make, which of them are allowed to overlap?

Running three tool calls at once instead of in sequence turns three round trips into one. It also triples the instantaneous load on whatever they hit, opens the possibility of conflicting writes, raises the spend for calls whose results get discarded, and makes the run nondeterministic. All four costs are real; the latency win usually still wins for reads.

Q · What exactly am I buying and paying for when I let the agent run tool calls in parallel?

A summarizer and a fact-checker both read version 4 of a document, both edit it, both write. One edit survives. This is the lost update, unchanged since the first bank-balance example — except the writers are nondeterministic, the schedule is chosen by a model, and the losing edit is plausible enough that nobody notices it went missing.

Q · Two agent tasks are editing the same document. What stops one of them from silently erasing the other?

The user presses stop. The model call can be aborted, the queued tool calls can be dropped, the in-flight ones will finish whether you want them to or not — and the three emails already sent cannot be un-sent. Cancellation is a request to stop future work, never an undo of work already done.

Q · A user cancels a long agent run. What stops, what finishes anyway, and what is already irreversible?