Concurrency & Parallelism Roadmap

From telling overlapping progress apart from simultaneous execution, through shared state, synchronization, deadlock and contention, to parallel decomposition, memory visibility and the judgement to decide that a system does not need any of it.

0 / 154 roadmap lessons mastered0%

1
Concurrency, parallelism, tasks and threads
0 / 16 mastered
Separate "these things overlap" from "these things run at the same instant", and pick the unit of execution from the shape of the work rather than from habit.
What Concurrency Actually Is Which One Does This Workload Need?Why Concurrency Exists: Waiting Why Parallelism Exists: Compute Classifying the Work: Computing or Waiting?Choosing an Execution Model What "Making Progress" Actually Means Concurrency Is Always Bought With Complexity Processes: Isolation You Cannot Accidentally Break Threads: One Address Space, Several Instruction Streams Thread or Process?A Task Is Not a Thread Coroutines: Functions That Can Pause Python: Threads, Processes and the GIL JavaScript: One Event Loop per Agent, Not One Thread per Runtime C++: Threads, Atomics and a Memory Model With Teeth
Outcome · You can look at a piece of work, say whether it is waiting-bound or compute-bound, and name the execution model that follows — including the case where the answer is "run it sequentially".
Prove it — The scheduler lab →
2
Shared state, races and critical sections
0 / 15 mastered
Name the invariant before naming a primitive, and find the schedule that breaks it.
Shared Mutable State Interleavings: The Schedule Is Part of the Program Invariants: Name It Before You Lock It Finding the Critical Section Reasoning About Races: A Method, Not an Instinct Data Race Is Not Race Condition The Atomicity Illusion Nondeterminism: Same Input, Different Output Immutability as a Concurrency Strategy Copy or Share?Copy-on-Write as a Concurrency Strategy Optimistic Concurrency Control Pessimistic Concurrency Optimistic vs Pessimistic Initialization Races
Outcome · Given a function that two threads call, you can write down what must stay true, produce an interleaving that violates it, and say whether the fix is a lock or the removal of the sharing.
Prove it — Break the counter →
3
Mutexes, semaphores and condition variables
0 / 11 mastered
Choose the primitive from the guarantee you need, and hold it over the smallest region that preserves the invariant.
Mutexes: What They Protect and What They Do Not Lock Scope: What You Hold It Across Read/Write Locks, Honestly Semaphores: Counting Permits as a Resource Limit Semaphore versus Mutex: Not the Same Primitive Condition Variables: Waiting Until a Predicate Is True Lost Wakeups: The Notify That Arrived Before the Wait Spurious Wakeups: Why It Is `while`, Not `if`Spin Locks What "Thread-Safe" Actually Means Reentrancy
Outcome · You can say what a mutex guarantees and what it does not, why a condition variable is always waited on in a loop, and why "thread-safe" is a claim about a type's operations rather than a property you can sprinkle on.
Prove it — Fix the counter →
4
Async, promises, coroutines and event loops
0 / 16 mastered
Understand what suspending a task actually does, what keeps running while it is suspended, and who owns a task once it is started.
Event Loops as a Concurrency Model Await Is a Yield Point Futures & Promises Worker Threads Web Workers Event Loop or Threads?Async Is Not Parallelism Blocking the Event Loop Promise.all & gather The Sequential Await Trap Structured Concurrency Cancellation Cancellation Propagation Timeouts Deadlines vs Timeouts Orphaned Tasks
Outcome · You can explain why an await in a loop is a latency bug and an unbounded Promise.all is a capacity bug, and why a task with no owner is the async equivalent of a leaked thread.
Prove it — Stall the event loop →
5
Thread pools, worker pools, producer/consumer and backpressure
0 / 17 mastered
Treat concurrency as a budget you set rather than a number that emerges, and decide what happens to the work that does not fit.
Producer / Consumer Bounded vs Unbounded Queues Backpressure Channels Message Passing The Actor Model Concurrent Queues Draining a Pipeline Thread Pools Sizing a Thread Pool Worker Pools Beyond Threads Work Stealing Pool Saturation Bounding Concurrency Unbounded Concurrency Thundering Herd Single-Flight Coalescing
Outcome · You can size a pool by reasoning rather than by folklore, and you can say exactly what a full queue should do — block, drop, or reject — and defend the choice.
Prove it — Set a concurrency budget →
6
Deadlocks, starvation, livelock and contention
0 / 15 mastered
Explain how a system full of correct code stops making progress, and how to make the failure structurally impossible rather than unlikely.
Deadlock The Four Conditions Preventing Deadlock Lock Ordering Livelock Starvation Fairness Priority Inversion What Contention Actually Costs Lock Convoys False Parallelism Oversubscription The Cost of a Context Switch More Threads Is Not More Speed Busy Waiting
Outcome · You can draw a wait-for graph from a stack trace, name which of the four deadlock conditions your fix removes, and explain why the eight-core box was slower than the two-core one.
Prove it — Deadlock lab →
7
Atomics, memory ordering and lock-free
0 / 15 mastered
Learn what an atomic operation actually makes indivisible, why another thread might not see your write at all, and why lock-free is a progress guarantee rather than a speed claim.
Atomics: What Is Actually Indivisible Compare-and-Swap and the Retry Loop Atomics Are Not Magic Lock-Free Is a Progress Guarantee A Lock-Free Stack, and What the Teaching Version Omits Wait-Free vs Lock-Free: Whose Progress Is Guaranteed The ABA Problem: The Value Came Back What a Memory Model Defines Happens-Before: The Edge That Makes a Write Visible Reordering: The Compiler and the CPU Both Do It Memory Barriers Constrain Ordering, Not Caches False Sharing: Different Variables, Same Cache Line What a Shared Write Costs Safe Publication: Handing Over a Finished Object Double-Checked Locking: The Canonical Cautionary Tale
Outcome · You can explain why counter.fetch_add(1) is safe and if (!cache) cache = build() on an atomic pointer is not, and why the fix for double-checked locking is a memory-model question, not a syntax one.
Prove it — Watch a CAS loop retry →
8
Parallel algorithms, fork/join and SIMD
0 / 18 mastered
Decompose a computation into independent work, and predict the ceiling before writing any of it.
Fork/Join Parallel Reduce The Map/Reduce Pattern Parallel Algorithms Parallel Overhead Amdahl's Law Gustafson's Law Work and Span Dependency Graphs SIMD: One Instruction, Many Elements Task Parallelism vs Data Parallelism GPU Parallelism: Thousands of Lanes, One Bus Pipeline Parallelism: Different Items, Different Stages Fan-Out / Fan-In: One Request Becomes N Scatter/Gather and the Tail You Inherit Barriers Latches & Countdowns Parallelism Moves the Load Downstream
Outcome · You can compute the best speedup a decomposition can ever reach from its serial fraction and its span, and you know when the parallel version will lose to the sequential one.
Prove it — Scaling lab →
9
Parallel performance, debugging and production concurrency
0 / 31 mastered
Receive "it is slow and occasionally wrong, and it only happens in production" and answer it with evidence instead of guesses.
Why Eight Cores Give You Four and a Half Memory Bandwidth: More Cores, Same Bus Parallelism Can Destroy Locality NUMA: Not All Memory Costs the Same Thread Affinity: Pinning, and What It Costs You Reduction Ordering: The Sum Changed When the Worker Count Did Determinism: Same Input, Same Output?Ordering Guarantees: Four Levels, Four Prices What to Instrument in a Concurrent System Hold Time, Wait Time, and the Ratio Between Them Reading a Thread Dump Task Dumps: When the Threads Look Idle and Nothing Is Moving Off-CPU Time: The Thing a CPU Profiler Cannot See Race Detectors: What They Find, and What They Structurally Cannot Heisenbugs: The Bug That Leaves When You Look at It Deterministic Replay: Making the Schedule Reproducible Stress Testing: A Test That Passed Once Proves Nothing Choosing a Concurrency Model for a Server Thread per Request: The Model That Reads Like Ordinary Code Event-Driven Servers: Many Connections, One Loop Hybrid Runtimes: It Was Never Threads Versus Async UI Concurrency: One Thread Owns the Screen The Database Solves Concurrency For Its Data, Not For Your Memory What Changes When the Shared State Is on Another Machine A Mutex on Server A Does Nothing About Server B The Concurrency Pattern Catalogue Concurrency Anti-Patterns Can These Tool Calls Run At the Same Time?Parallel Tool Calls: Lower Latency, and Four Bills Two Agents, One Document Cancelling an Agent Run: What Actually Stops
Outcome · You can instrument a concurrent system so the next failure leaves evidence, and you can design one whose concurrency model you are able to defend line by line — including the parts you chose to leave sequential.
Prove it — Concurrency capstone →