The question this answers
When I spawn ten thousand tasks, what actually exists — and what is running?
A crawler with 10 000 URLs in flight, running on a four-thread runtime on a four-core machine, maintaining a shared visited-set and a per-host request budget.
The visited set, the per-host budget map, and the output writer. All of them are reachable from every task, and — critically — tasks that appear to be "on one thread" can still interleave with each other at every suspension point.
Each URL is fetched at most once, and no host receives more concurrent requests than its budget allows — under any interleaving of the ten thousand tasks over the four threads.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Three layers, not two
The mental model most people carry has two layers: my code, and threads. The real stack has three. A *task* (a future, a coroutine, a goroutine, a promise chain) is a value the runtime owns, holding a resumption point and its captured locals. A *thread* is an OS-scheduled execution stream. A *core* is hardware. The runtime maps tasks onto threads; the OS maps threads onto cores; and neither mapping is one-to-one.
The numbers make the point. Ten thousand OS threads is roughly 8 GB of stack reservation and a scheduler run queue nobody wants. Ten thousand tasks is a few megabytes of heap, because a suspended task stores only the locals live across its suspension point rather than an entire stack. That difference is the whole reason a modern server can hold a hundred thousand connections open.
The mapping is also dynamic. A task can begin on thread 2, suspend, and resume on thread 3 — which matters more than it sounds, because anything bound to a thread rather than to a task (thread-local storage, a re-entrant lock's owner identity, an OS thread's priority) is not stable across a suspension point. That is a real and frequently-hit bug class in work-stealing runtimes.
What one thread looks like from underneath
Zoom into a single thread of the runtime and the picture is a sequence of short task fragments, not a task. The thread picks a ready task, runs it until it suspends, picks another. A "task" as the programmer wrote it — fetch, parse, store — appears on the thread as three or four unrelated slivers separated by other tasks' slivers.
This is where the scaling comes from and where the stall comes from, in the same mechanism. Because each fragment is short, one thread can service hundreds of tasks and none of them waits long. Because the runtime cannot take the thread back mid-fragment — most of these runtimes are cooperative — one long fragment freezes every task assigned to that thread. Four threads and one 300 ms CPU fragment means a quarter of your crawler stops. See [[blocking-the-event-loop]] and [[overlapping-progress]].
The lane for task 4 in the timeline is the one to read carefully: it runs on thread 0, suspends, and resumes on thread 1. Nothing in the source suggests that a function moved threads halfway through, and anything the code stored in thread-local storage before the suspension point is simply gone afterwards.
One thread does not mean no interleaving
Here is the consequence people miss, and it is the reason this lesson exists. Tasks interleave at suspension points *even when they share one thread*. A read-modify-write that spans an await is not atomic, on any number of threads, and the fact that "it is all one thread" is not a defence — it is only a defence against data races, which are a different thing.
The schedule below is two crawler tasks enforcing a per-host budget of two concurrent requests. Both read the current count, both find it under the limit, both await DNS resolution, both increment and proceed. Three requests go to a host budgeted for two. On one thread. With no locks anywhere, because "there is only one thread, so what would a lock do?"
The fix is the usual one and it is worth stating in task terms: either do not span a suspension point with a read-modify-write, or hold something across it that other tasks respect. In an async runtime that something is an async-aware primitive — an async mutex or a semaphore — and specifically not an OS mutex, which would block the whole thread and take every other task on it down with the blockage. [[semaphores-and-permits]] and [[bounding-concurrency]] are the shape of the real fix here.
| # | Task 1 — GET example.com/a | Task 2 — GET example.com/b | Runtime (thread 0) | State |
|---|---|---|---|---|
| 1 | · | · | thread 0 picks task 1 | inFlight[example.com]=2 budget=2 |
| 2 | a request completes → inFlight = 1 | · | · | inFlight[example.com]=1 budget=2 |
| 3 | read inFlight (1); 1 < 2 → may proceed | · | · | inFlight[example.com]=1 |
| 4 | await dns.resolve("example.com") — SUSPENDS | · | · | inFlight[example.com]=1 |
| 5 | · | · | thread 0 picks task 2 — same thread, no parallelism | inFlight[example.com]=1 |
| 6 | · | read inFlight (1); 1 < 2 → may proceed | · | inFlight[example.com]=1 |
| 7 | · | await dns.resolve("example.com") — SUSPENDS | · | inFlight[example.com]=1 |
| 8 | DNS resolved; resume; inFlight = 2; open connection | · | · | inFlight[example.com]=2 |
| 9 | · | DNS resolved; resume; inFlight = 3; open connection | · | inFlight[example.com]=3 ✕ Three concurrent requests to a host budgeted for two. Both tasks acted on a check made before either increment, on a single thread, with no parallelism anywhere. |
| 10 | · | · | host returns 429; crawler backs off for the whole domain | inFlight[example.com]=3 rateLimited=yes |
Key points
- Three layers: tasks, threads, cores. The runtime maps tasks to threads, the OS maps threads to cores, and neither mapping is one-to-one.
- A suspended task stores only the locals live across the suspension point; a thread reserves a whole stack. That is why tasks scale to hundreds of thousands and threads do not.
- A task can resume on a different thread from the one it started on, so thread-local storage and thread-identity-based locks are unsafe across a suspension point.
- Tasks interleave at suspension points even on one thread. Single-threaded prevents data races, not race conditions.
- A read-modify-write spanning an
awaitis not atomic on any number of threads. - A CPU fragment that never suspends holds its runtime thread until it returns, freezing every task assigned to that thread.
- Use async-aware synchronisation in async code: an OS mutex blocks the thread and takes every other task on it down with it.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • Calling an async function creates a task object: a resumption point plus the locals that must survive suspension, allocated on the heap.
- • The runtime places ready tasks in a queue and hands them to worker threads; each thread runs one task until it suspends or returns.
- • Suspension saves the resumption point and releases the thread. The task is now a value on the heap that no thread is executing.
- • An external event — I/O readiness, a timer, a resolved future — marks the task ready and it re-enters the queue.
- • Any worker thread may pick it up, so the task may resume on a different thread. Work-stealing runtimes make this common rather than rare.
- • The task runs to its next suspension point, and the cycle repeats until it returns.
- • T1 checks the budget, increments, then awaits — the correct ordering, in which no other task can observe the gap because there is no gap.
- • T1 checks, awaits, T2 checks, both increment — the over-admission above, on one thread with no parallelism.
- • T2 parses 900 KB of HTML without suspending: every task assigned to that runtime thread makes no progress for the duration, including tasks with nothing to do with parsing.
- • T4 writes a request id into thread-local storage, awaits, resumes on another thread, and reads an empty slot — or worse, another task's value. See
[[context-propagation]]. - • A task takes an OS mutex and then awaits while holding it; the thread is blocked, the runtime cannot reclaim it, and if all worker threads do this the runtime deadlocks with zero tasks running.
- • The runtime guarantees a suspended task resumes at its suspension point with its own locals intact.
- • It does not guarantee the same thread, nor any particular thread, unless the task is explicitly pinned.
- • It does not guarantee that a task runs promptly after becoming ready — that depends on queue depth and on whether some other task is monopolising a worker thread.
- • Single-threaded execution guarantees the absence of data races. It guarantees nothing about race conditions across suspension points.
- • Task count guarantees nothing about parallelism. Ten thousand tasks on a one-thread runtime execute strictly one at a time. See
[[async-is-not-parallel]]. - • Most such runtimes are cooperative: they guarantee no preemption, which means a non-suspending fragment holds its thread for as long as it likes.
- • Contention for runtime worker threads: ready tasks queue, and that queueing is invisible unless the runtime exports a lag metric.
- • Contention on the shared visited-set and budget map, which is real regardless of thread count because tasks interleave at suspension points.
- • A CPU fragment contends for its worker thread against every task assigned to it — contention with no lock, no queue and no metric.
- • Work-stealing reduces imbalance across threads and increases cache misses, because a stolen task resumes on a core whose cache knows nothing about it. See
[[cache-locality-concurrency]].
- • Check-then-act across a suspension point, producing over-admission, double-processing or lost updates on a single thread.
- • Thread starvation from a non-suspending CPU fragment, stalling every task on that worker.
- • Runtime deadlock when every worker thread is blocked on an OS-level primitive that only another task could release.
- • Thread-local state silently lost or crossed after a task migrates threads.
- • Unbounded task spawning: tasks are cheap, so nothing stops you creating a million, and the heap runs out. See
[[unbounded-concurrency]]. - • Orphaned tasks: a spawned task nobody awaits, whose exception is never observed and whose work never completes. See
[[orphaned-tasks]].
- • Very high concurrency over waiting-bound work: crawlers, proxies, gateways, chat servers, anything holding tens of thousands of mostly-idle connections.
- • When per-unit memory matters: a task costs hundreds of bytes where a thread costs megabytes, and that ratio decides what fits on the box.
- • When structured lifetimes are wanted: tasks compose into groups with a parent that can cancel them all, which threads do not do naturally. See
[[structured-concurrency]].
- • CPU-bound work, where tasks add suspension points, allocation and scheduling and deliver no parallelism.
- • Code with blocking calls that cannot be made async — a synchronous driver, a native library — which will hold worker threads and defeat the model.
- • Teams new to it, because the "single-threaded so it is safe" misconception produces exactly the bug above and it is genuinely hard to see in review.
- • Debugging: a task's stack contains the runtime, not the caller that spawned it, so causality has to be reconstructed. See
[[async-task-dumps]].
- • Task count against worker-thread count. If tasks vastly exceed threads and throughput is flat, the constraint is worker threads or a monopolising fragment.
- • Runtime scheduler lag — how long a ready task waits for a worker — which is the async equivalent of run-queue delay and the only direct evidence of thread starvation.
- • Longest task fragment duration. Anything above a few milliseconds is a CPU fragment that belongs on a worker pool.
- • Live task count as a gauge. Unbounded growth here is the leading indicator of the heap exhaustion that follows.
- • Async task dumps during a stall: the distribution of suspension points names the resource everything is waiting on.
- • You must know where every suspension point is, including ones inside libraries, because each is an interleaving point in your invariants.
- • Two synchronisation vocabularies coexist and must not be mixed: async primitives suspend tasks, OS primitives block threads, and using the second in async code is a latent deadlock.
- • Anything thread-affine — thread-local storage, re-entrant lock ownership, some native handles — becomes unsafe across suspension points and needs task-local context instead.
- • Cancellation becomes a first-class design problem, since a task that nobody awaits still runs and still has effects.
- • The async colouring problem: an async function can generally only be awaited from an async caller, so the model propagates through the whole call graph.
- • Threads, when concurrency is in the tens or low hundreds and the simplicity of blocking code is worth the stacks. See
[[threads]]. - • A bounded worker pool with blocking calls, which is easier to reason about and caps concurrency by construction.
- • Virtual or green threads, where the runtime offers them: task-like cost with thread-like blocking code, removing the colouring problem at the price of a newer runtime.
- • Fewer concurrent units. Ten thousand in-flight fetches against a host that permits two is a design that needed bounding far more than it needed tasks.
Scheduler timeline
Thread pool: utilization and queue
capacity = workers / service = 8 / 50 ms = 160.0 req/s ρ = arrivals / capacity = 120 / 160.0 = 0.750 Little L = λ × W → 0.120/ms × 59.8 ms = 7.2 in flight engine status = healthy
Two increments, twenty schedules: find the one that loses an update
| # | Task A — counter++ | Task B — counter++ | State |
|---|---|---|---|
| 1 | rA ← counter | · | counter=0 rA=0 rB=0 |
| 2 | rA ← rA + 1 | · | counter=0 rA=1 rB=0 |
| 3 | counter ← rA | · | counter=1 rA=1 rB=0 |
| 4 | · | rB ← counter | counter=1 rA=1 rB=1 |
| 5 | · | rB ← rB + 1 | counter=1 rA=1 rB=2 |
| 6 | · | counter ← rB | counter=2 rA=1 rB=2 |
Bounding concurrency with permits
| permits | goodput | mean latency | timeouts | failed of 10K |
|---|---|---|---|---|
| 1 | 25/s | 43 ms | 0.00% | 0 |
| 5 | 125/s | 43 ms | 0.00% | 0 |
| 10 | 250/s | 43 ms | 0.00% | 0 |
| 25 | 625/s | 43 ms | 0.00% | 0 |
| 50 | 1000/s | 53 ms | 0.00% | 0 |
| 100 | 1000/s | 103 ms | 0.00% | 0 |
| 200 | 1000/s | 203 ms | 0.00% | 0 |
| 350 | 1000/s | 353 ms | 0.00% | 0 |
| 500 | 0/s | 503 ms | 100.0% | 10K |
What people believe, and what is true
A task is basically a lightweight thread.
A task is a heap value the runtime resumes. It has no stack of its own between suspensions, it may run on different threads at different times, and the OS does not know it exists.
Single-threaded async code cannot race.
It cannot have data races. Every await is an interleaving point, and a read-modify-write spanning one is exactly as broken as it would be on eight threads.
Ten thousand tasks means ten thousand things happening at once.
It means ten thousand things unfinished. At most one per worker thread is executing, and on a one-thread runtime that is one.
A mutex is a mutex.
An OS mutex blocks a thread; an async mutex suspends a task. Using the first in async code blocks a worker thread and can deadlock the whole runtime.
Go deeper
Overview
Tasks are units of work the runtime schedules onto a small number of threads. Thousands of tasks, a handful of threads, a handful of cores.
Practical
Treat every await as a place where other tasks run. Do not span one with a read-modify-write, do not hold an OS lock across one, and do not run more than a few milliseconds of CPU work between two of them.
Advanced
Tasks decouple the unit of concurrency from the unit of execution, which is what makes six-figure concurrency affordable and what makes anything thread-affine unsafe. Context that must follow the work has to be task-local, not thread-local, and every runtime that offers work stealing forces this on you.