Coordination & Limits

Promise.all & gather

Start N tasks, wait for the collection, aggregate the results. The easy part is the fan-in; the lesson is the failure behaviour — Promise.all rejects the instant one task fails while the other N−1 keep running unsupervised, and every language offers a different, incompatible way to say "tell me about all of them".

▶ Run the lab

The question this answers

The question

When one of five parallel calls fails, what happens to the other four — and what does my caller actually learn?

The work

A product page that fetches inventory, pricing, reviews, recommendations and shipping estimates concurrently, then renders one response from all five.

What is shared

The result array being assembled, and — much more importantly — whatever the abandoned tasks touch after the caller has given up: connections held, retries issued, caches written, metrics emitted for a request that no longer exists.

The invariant — what must stay true under every interleaving

Either every result the caller acts on came from a successful call, or the caller knows exactly which calls failed. No task keeps mutating request-scoped state after its request has returned.

Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.

WorkCan it overlap?Can it parallelise?What is shared?What ordering?What synchronization?Where is contention?What can deadlock?What can race?What is gained?What complexity?

Four languages, four different answers to "one of them failed"

The happy path is identical everywhere: start the tasks, wait, get the results in input order. The divergence is entirely in failure, and the divergence is large enough that porting a fan-in between languages without reading this table produces a behaviour change nobody notices until an incident.

The critical property Promise.all has and its name does not suggest: rejection is immediate and the other tasks are not cancelled. They keep running, keep holding connections, keep retrying, and their eventual results — or rejections — go nowhere. In Python, gather additionally has two modes that differ in this exact respect, and TaskGroup has a third behaviour again.

Fan out five independent calls, wait for all of them, and handle a failure. — Start N independent tasks, wait for the whole collection, and decide what a partial failure means
C++LANGUAGE-SPECIFIC
1// No std::when_all in the standard library (it is in the concurrency TS).
2// The hand-rolled version makes the semantics explicit — which is the point.
3std::vector<std::future<Part>> futures;
4for (auto& fetch : fetchers)
5 futures.push_back(std::async(std::launch::async, fetch));
6
7std::vector<std::optional<Part>> parts(futures.size());
8std::vector<std::exception_ptr> errors(futures.size());
9
10for (size_t i = 0; i < futures.size(); ++i) {
11 try { parts[i] = futures[i].get(); } // blocks; rethrows this task's exception
12 catch (...) { errors[i] = std::current_exception(); }
13}
14// Note: every future is joined. Nothing is abandoned — but nothing is
15// cancelled either, so a fast failure does not save you any time.

There is no first-failure short-circuit unless you build one, and there is no cancellation to build it out of. The loop waits for every task regardless, which is allSettled semantics by default. Destroying a std::async future blocks in its destructor, so you cannot even walk away.

JavaScriptNODE.JS
1// Rejects on the FIRST failure. The other four keep running.
2const [inv, price, reviews, recs, ship] = await Promise.all([
3 getInventory(id), getPrice(id), getReviews(id), getRecs(id), getShipping(id),
4])
5// If getPrice() rejects at 20ms, this throws at 20ms — and getReviews(),
6// getRecs() and getShipping() are still in flight, still holding sockets,
7// still going to settle into nobody.
8
9// Report on all of them:
10const results = await Promise.allSettled([...]) // never rejects
11const failed = results.filter((r) => r.status === 'rejected')
12
13// First success wins, ignore the rest: await Promise.any([...])
14// First to SETTLE wins, success or failure: await Promise.race([...])

Promise.all is a fail-fast aggregator with no cancellation attached. To actually stop the abandoned work you must plumb an AbortController through every call and abort it in a finally — the language will not do it for you.

TypeScript
1// Make partial failure a value, so the compiler forces you to handle it.
2type Part<T> = { ok: true; value: T } | { ok: false; error: unknown }
3
4async function all<T extends readonly unknown[]>(
5 tasks: { [K in keyof T]: Promise<T[K]> },
6 signal: AbortSignal,
7): Promise<{ [K in keyof T]: Part<T[K]> }> {
8 const settled = await Promise.allSettled(tasks as readonly Promise<unknown>[])
9 return settled.map((r) =>
10 r.status === 'fulfilled'
11 ? { ok: true as const, value: r.value }
12 : { ok: false as const, error: r.reason },
13 ) as { [K in keyof T]: Part<T[K]> }
14}
15
16// Degrade explicitly, per part, at the render site:
17const [inv, price, reviews] = await all([getInventory(id, signal),
18 getPrice(id, signal), getReviews(id, signal)] as const, signal)
19if (!price.ok) return renderUnavailable() // pricing is essential
20const reviewBlock = reviews.ok ? renderReviews(reviews.value) : null // reviews are not

Promise<T> erases the failure type entirely, so Promise.all gives you a typed success tuple and an untyped throw. Modelling each part as a Result makes the essential-versus-optional decision explicit at the render site, which is where it belongs.

PythonCPYTHON
1# Default: first exception propagates; the OTHERS KEEP RUNNING.
2inv, price, reviews = await asyncio.gather(
3 get_inventory(id), get_price(id), get_reviews(id))
4
5# Report on all of them — exceptions come back as VALUES, in order:
6results = await asyncio.gather(*calls, return_exceptions=True)
7failed = [r for r in results if isinstance(r, BaseException)]
8
9# TaskGroup (3.11+): first failure CANCELS the siblings, then raises
10# an ExceptionGroup. This is structured concurrency — different semantics again.
11async with asyncio.TaskGroup() as tg:
12 t_inv = tg.create_task(get_inventory(id))
13 t_price = tg.create_task(get_price(id))
14# On exit: all tasks are done or cancelled. Nothing is left running.

Three behaviours in one standard library. gather() propagates the first exception and abandons its siblings; gather(return_exceptions=True) returns exceptions as ordinary values so nothing is lost; TaskGroup cancels the siblings and raises an ExceptionGroup. Only the third one leaves no orphans.

What actually differs
  • Promise.all and bare gather() both reject on the first failure and neither cancels the remaining tasks — the siblings keep running with nobody waiting for them.
  • asyncio.TaskGroup (3.11+) is the only one of these that cancels siblings on failure, which is why structured concurrency exists (Structured Concurrency).
  • Aggregate reporting has three spellings with three shapes: allSettled returns tagged objects, gather(return_exceptions=True) returns exceptions inline as values, and C++ requires you to catch per-future.
  • C++ has no short-circuit at all: joining every future is the default, so it behaves like allSettled and a fast failure saves no time.
  • Only Python's ExceptionGroup (and JS AggregateError, from Promise.any) can report *several* failures. Promise.all throws exactly one reason and silently discards any others.

What happens to the abandoned four

The rejection returns to the caller, the caller returns an error to the user, and the request is over. The other four tasks do not know that. They continue: they hold their connections until the response arrives or the socket times out, they run their retry policies, and their .then handlers execute against request-scoped state — a response object that has already been sent, a cache keyed to a request id, a span in a trace that has already been closed.

This is the Orphaned Tasks failure arriving through a completely innocuous-looking API. Under load it is not a curiosity: if 5% of requests fail fast on one dependency, that is 5% of requests leaving four abandoned in-flight calls each, which is a 20% invisible increase in concurrent load on the other four dependencies at exactly the moment one of them is already unhealthy.

The fix is cancellation, and it must be explicit in every language except Python-with-TaskGroup. Create one AbortController per request, pass its signal into every call, and abort it in a finally. The tasks then terminate at their next suspension point and stop consuming anything.

One fast failure under Promise.all, and what the other four do afterwards.ILLUSTRATIVE
Invariant · No task mutates request-scoped state after its request has returned, and abandoned work releases its resources
#Handler (Promise.all)getPrice — fails fastgetReviews — slowgetRecs — retriesState
1starts all five calls; awaits Promise.all···inFlight=5 responded=no conns=5
2·rejects at 20 ms: pricing service returned 503··inFlight=4 responded=no conns=4
3Promise.all rejects immediately; handler sends 502 and returns···inFlight=4 responded=yes conns=4
4···getRecs sees a 500, applies its retry policy, issues attempt 2inFlight=4 responded=yes conns=4
✕ Load is being generated for a request that no longer exists. At 5% failure rate this is a silent multiplier on every downstream dependency.
5··getReviews resolves at 400 ms; its .then writes to the request-scoped cache·inFlight=3 responded=yes conns=3
✕ Request-scoped state is mutated after the response was sent. If that state is a res object, the runtime throws ERR_HTTP_HEADERS_SENT into an unhandled rejection.
6···retry 2 also fails; rejects with nobody listening → unhandledRejectioninFlight=2 responded=yes conns=2
✕ The rejection has no handler because the aggregate already settled. On Node 15+ the default is to terminate the process.
Rejecting the aggregate does not stop the members. Create one AbortController per request, pass the signal into every call, and abort it in a finally — then a fast failure genuinely releases four connections instead of quietly holding them. Use allSettled when partial results are acceptable, and TaskGroup (or a task-group equivalent) when you want cancellation to be the default rather than something you remembered.

Choosing the combinator by what a partial failure means

The right combinator falls out of one question: is every part essential? If pricing is essential and reviews are not, Promise.all is wrong for both — it fails the whole page when reviews time out, and it gives you no way to render without them. Model essentiality per part, then pick.

Two operational notes that the table cannot hold. First, Promise.all over a large array is not a fan-in at all but an unbounded fan-out, and that is the subject of Parallelism Moves the Load Downstream — mapping 500 ids to 500 concurrent queries is how an application team takes down a database. Second, the aggregate's latency is the *maximum* of its members, so one slow member sets the whole response time; that is tail-latency amplification, and per-part timeouts are the only defence (Timeouts).

CombinatorSettles whenOn failureSiblingsUse when
Promise.all / gather()All succeed, or one failsRejects with the first reason onlyKeep running, uncancelledEvery part is essential and the caller should fail fast
Promise.allSettled / gather(return_exceptions=True)All settleNever rejects — failures come back as valuesAll run to completionParts are independently optional and you must report on each
Promise.anyFirst success, or all failAggregateError only if every one failsKeep running, uncancelledRedundant sources: any answer will do (mirrors, replicas)
Promise.raceFirst to settle, success or failureRejects if the first to settle rejectedKeep running, uncancelledTimeouts and cancellation patterns — rarely correct for data
asyncio.TaskGroupAll succeed, or one failsCancels siblings, raises ExceptionGroupCancelled deterministicallyDefault choice in modern Python: no orphans by construction
Per-part timeout + allSettledEvery part settles or times outEach part fails independentlyBounded by their own timeoutsRendering a page that must degrade rather than fail
Bounded map (pool of K)All settle, K at a timeDepends on the inner combinatorBounded concurrency throughoutN is large — never map a big array straight into all
Which combinator, by what a partial failure should mean.

Key points

  • Promise.all and bare gather() reject on the first failure and do not cancel the remaining tasks — they keep running with nobody waiting.
  • Abandoned tasks hold connections, run retries, and mutate request-scoped state after the response has been sent.
  • allSettled and gather(return_exceptions=True) report on every member and never short-circuit; TaskGroup cancels siblings and is the only built-in that leaves no orphans.
  • Promise.all throws exactly one reason and discards any others; only ExceptionGroup and AggregateError can report several.
  • The aggregate's latency is the maximum of its members, so one slow part sets the whole response time — per-part timeouts are the defence.
  • Decide essentiality per part first; the combinator then chooses itself.
  • Mapping a large array straight into Promise.all is an unbounded fan-out, not a fan-in.

The loop, answered

Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.

How it works
  • Each task is started before the aggregate is awaited — in JS by constructing the promise, in Python by wrapping the coroutine in a Task, in C++ by launching the async operation.
  • The combinator registers a continuation on each member and keeps a completion count plus a results array indexed by input position.
  • On each fulfilment it stores the value and decrements the outstanding count; when the count reaches zero it resolves with the ordered array.
  • On a rejection, all settles the aggregate immediately with that reason and stops caring about the rest; the members' own continuations still fire, into a settled aggregate that ignores them.
  • allSettled instead stores a tagged record for every outcome and only settles when every member has settled.
  • TaskGroup additionally holds cancel scopes for the children and cancels them on the first failure before re-raising as a group.
  • Result ordering follows input position everywhere, never completion order — which is why the array destructure is safe.
Interleavings that matter
  • getPrice rejects at 20 ms; the aggregate rejects at 20 ms; the handler responds 502; getReviews resolves at 400 ms and writes to a response object that was already sent — ERR_HTTP_HEADERS_SENT inside an unhandled rejection.
  • getRecs fails, retries per its own policy, and fails again 800 ms after the request ended; the second rejection has no handler because the aggregate already settled, and Node 15+ terminates the process by default.
  • All five succeed but getShipping takes 900 ms; the other four finished in under 30 ms; the response takes 900 ms because the aggregate waits for the maximum — nothing failed and the page is still slow.
  • With allSettled: pricing fails at 20 ms and the aggregate still waits 900 ms for shipping, so fail-fast latency is traded away for complete reporting. That trade is the reason per-part timeouts exist.
  • With TaskGroup: pricing fails at 20 ms, the other four are cancelled at their next suspension point, connections are released, and the ExceptionGroup names exactly what happened.
  • 500 ids mapped into Promise.all: 500 concurrent queries against a pool of 20, so 480 wait on the pool and each one's timeout starts ticking from the moment it was created (Parallelism Moves the Load Downstream).
What it guarantees — and does not
  • Guaranteed: results are returned in input order, never completion order.
  • Guaranteed: all settles as soon as one member rejects, so the caller is not delayed by the survivors.
  • Guaranteed: allSettled never rejects — every member gets a status record.
  • NOT guaranteed: cancellation. Promise.all, Promise.any, Promise.race and bare gather() cancel nothing.
  • NOT guaranteed: that a rejected member's error is reported. all keeps the first reason and discards the rest.
  • NOT guaranteed: bounded concurrency. The combinator waits on whatever you started; it never limits how many.
  • NOT guaranteed: that members run concurrently at all — in Python a bare coroutine list passed to gather is wrapped in Tasks, but three sequential awaits before the call are already serialised (The Sequential Await Trap).
Where contention appears
  • The aggregate creates N simultaneous consumers of every downstream resource — connection pool slots, rate-limit budget, thread-pool capacity.
  • Latency is the maximum of the members, so tail latency of the slowest dependency becomes the tail latency of the whole endpoint (Fan-Out: Waiting for the Slowest of Seven).
  • Abandoned tasks after a fast failure are invisible contention: they still occupy pool slots and rate-limit budget for a request that has already returned.
  • On an event loop, all N continuations become ready in the same tick when a shared dependency responds, so the aggregate resolution is a small burst of synchronous work.
How it fails
  • Orphaned tasks: siblings keep running after the aggregate rejects, holding resources and retrying.
  • Unhandled rejection from a sibling that fails after the aggregate has settled — process termination on modern Node.
  • Write-after-response: a late continuation mutates request-scoped state or a closed trace span.
  • Swallowed errors: all reports one reason; three other failures in the same batch are simply lost.
  • Tail-latency amplification: one slow member sets the response time for the whole aggregate.
  • Accidental serialisation in Python: coroutines awaited individually before the gather, so the "concurrent" fan-out ran one at a time.
  • Unbounded fan-out: a large input array turned into an equally large burst of downstream calls.
When it helps
  • Independent calls with no ordering dependency, where the response needs all of them and the latency win is real.
  • Fan-out/fan-in over a bounded, known-small set of dependencies — a product page, a dashboard tile, an enrichment step.
  • Redundant sources with Promise.any, where any successful answer is as good as another.
  • Batch processing with allSettled, where each item succeeds or fails independently and the caller wants a per-item report.
When it hurts
  • When the parts are not equally essential — all fails the whole page for an optional review widget.
  • When N is large, because the combinator is a multiplier on downstream load and provides no bound.
  • When the calls are not actually independent, in which case the ordering constraint you removed was load-bearing.
  • When abandoned work has side effects — writes, charges, external calls — and no cancellation is plumbed through.
  • When one member is reliably slow: the aggregate hands your endpoint that member's tail latency, permanently.
How you would know
  • Per-member latency and failure rate inside the aggregate, not just the aggregate's own. The slow member is invisible otherwise.
  • Aggregate latency versus the max of the members: a gap means loop or pool contention, not dependency slowness.
  • Count of tasks still in flight after their request has returned — the direct measurement of orphaning, and almost nobody has it.
  • Unhandled-rejection count, which is the cheap proxy for the same thing.
  • Downstream request rate divided by inbound request rate. If the ratio exceeds N, abandoned retries are inflating it.
  • Per-part timeout expiry counts, which tell you which member is setting your p99 before it starts failing outright.
Complexity it introduces
  • Cancellation must be plumbed manually through every call in most languages: an AbortController per request, a signal parameter on every function, and an abort in a finally.
  • Modelling parts as Results rather than throws makes degradation explicit and adds a type and a mapping layer to every fan-in.
  • Per-part timeouts multiply configuration: each dependency now has its own budget, and those budgets must sum to less than the request deadline (Deadlines vs Timeouts).
  • Choosing between four combinators with different failure semantics is a decision per call site, and getting it wrong is silent.
Simpler alternatives
  • asyncio.TaskGroup, or a task-group / nursery equivalent, so cancellation is the default and orphans are impossible by construction (Structured Concurrency).
  • A bounded map (concurrency limit K) instead of all over a large array — the same fan-in with a ceiling on downstream load (Bounding Concurrency).
  • Sequential execution when the calls are cheap and the dependencies fragile; N round trips is sometimes the right price for not multiplying load.
  • A single batch call — one query with an IN clause, one batch endpoint — which replaces N concurrent requests with one and is almost always better if it exists (batch-apis).
  • Server-side composition or a materialised view, when the same five things are fetched together on every request and the fan-out is a data-modelling problem in disguise.

One of five fails — what happens to the siblings?

One of five fails — what happens to the other four?
Task C rejects at 40 ms. The interesting question is not what the caller sees; it is what the siblings are doing at 41 ms.
Combinator
A · charge card
charge card
B · reserve stock
reserve stock
still running, result discarded
C · fraud check
fraud check
D · send receipt
send receipt
still running, result discarded
E · update ledger
update ledger
still running, result discarded
↑ caller resumes 40 ms
runningreadywaitingblockedidlems
caller resumes at
40 ms
caller sees
1 rejection
siblings still running after
3
unobserved work
95 ms
t+0all five tasks started
t+30A resolved · charge card
t+40C rejected → Promise.all rejects NOW; the caller's await throws
t+41B, D, E are still executing — nothing cancelled them
t+55B resolved · reserve stock — result dropped on the floor
t+70E resolved · update ledger — stock reserved and ledger updated for an order the caller believes failed
t+90D resolved · receipt sent to the customer
Promise.all           rejects on the FIRST rejection; siblings are NOT cancelled and keep running
Promise.allSettled    never rejects; resolves at 90 ms with {status, value|reason} for all five
asyncio.gather(...)   return_exceptions=False → raises at 40 ms, siblings still NOT cancelled
                      return_exceptions=True  → returns at 90 ms with the exception as a value
asyncio.TaskGroup     the structured alternative: on failure it CANCELS the siblings, then raises
The caller resumed at 40 ms; 3 siblings ran on for another 95 ms of unobserved work. `Promise.all` is a combinator over promises, not a supervisor over tasks — it decides when you stop waiting, and has no power to stop anything. The stock stays reserved, the ledger entry lands and the receipt is emailed for an order your code has already reported as failed. Worse: if one of those late siblings rejects, it rejects with nobody listening, which is an unhandled rejection (a process-level warning or crash in Node, a "Task exception was never retrieved" in asyncio). If you need the siblings to stop, you need cancellation — an `AbortController` threaded into every call, or a structured construct like `asyncio.TaskGroup` or a nursery.
SIMULATEDRUNTIME-SPECIFIC

One request, N downstream calls

One request, N downstream calls
Fanning out turns N × latency into 1 × latency — and one request per second into N requests per second. The second number is the one that takes the downstream service down.
latency
unbounded
sequential would be
800 ms
peak downstream concurrency
200
downstream busy
over 100%
Latency by concurrency limit
1 at a time800.0 ms · 20 rounds · peak 10 downstream
2 at a time400.0 ms · 10 rounds · peak 20 downstream
4 at a time200.1 ms · 5 rounds · peak 40 downstream
8 at a time · peak 80 concurrent against 64 slots — no steady state
16 at a time · peak 160 concurrent against 64 slots — no steady state
20 at a time · peak 200 concurrent against 64 slots — no steady state
What the downstream sees
calls per parent request20 · each parent request multiplies into 20
concurrent calls at peak200 · 64 slots exist
queued at the downstream136 · these are connections, buffers and threads it did not budget for
Wait per call: unbounded
10 parent requests × 20 concurrent calls each = 200 simultaneous calls against 64 slots. The downstream has no steady state here: latency is not high, it is unbounded, and in a real system this appears as connection-pool exhaustion, timeouts and a service that was healthy until an unrelated caller shipped a loop. The best limit at this configuration is 4 at a time (200 ms) — and note that it is usually not 20. Raising the limit removes rounds, which is a linear win; it also raises peak downstream concurrency, which becomes a cliff the moment the peak crosses what the downstream can hold. A limit costs you a little latency in the good case and is the only thing standing between a routine traffic bump and a self-inflicted outage in the bad one. Bound it, and set the bound from the downstream capacity you were actually granted — not from the fan-out you happen to have today, which will be larger next quarter.
SIMULATEDA burst of 10 simultaneous parent requests against a downstream of 64 concurrent slots; waits from the engine's M/M/c approximation. Real fan-out also pays serialisation, connection setup and a tail latency that grows with N — the fastest of N calls does not set your latency, the slowest does.

Bounding concurrency with permits

Bounding concurrency — the permit count protects the dependency, not you
10K tasks behind a semaphore. The downstream service can serve a fixed number at once; the permit slider decides how many you throw at it.
permitsgoodputmean latencytimeoutsfailed of 10K
1 25/s43 ms0.00%0
5 125/s43 ms0.00%0
10 250/s43 ms0.00%0
25 625/s43 ms0.00%0
50 1000/s53 ms0.00%0
100 1000/s103 ms0.00%0
200 1000/s203 ms0.00%0
350 1000/s353 ms0.00%0
500 0/s503 ms100.0%10K
in flight
50
goodput
1000/s
queueing delay added
10 ms
tasks that time out
0
50 permits against a dependency that serves 40 at a time. The extra 10 requests are not being served faster — they are sitting in the dependency's queue adding 10 ms to every latency, and 0 of the 10K tasks time out because of it. Goodput is 1000/s against a peak of 1000/s: you added concurrency and got errors, not throughput. The permit count you want is the one that keeps in-flight work at the dependency's capacity — which you measure, you do not guess.
SIMULATED40 ms service · 400 ms client timeout

What people believe, and what is true

Claim

Promise.all cancels the other tasks when one fails.

Reality

It cancels nothing. The rejection reaches your caller and the other tasks keep running, holding connections and retrying against a request that has already returned.

Claim

allSettled is the safe default.

Reality

It is safe for reporting and it removes fail-fast, so the caller now waits for the slowest member even when an essential part already failed. Pair it with per-part timeouts.

Claim

Promise.all makes my calls concurrent.

Reality

The calls were already concurrent — in JavaScript they started when the promises were constructed. Promise.all only waits. In Python, a bare coroutine list is wrapped in Tasks by gather, but three awaits before the call are already serial.

Go deeper

Overview

Start N tasks, wait for the collection, get results in input order — and decide what one failure should mean.

Practical

Decide essentiality per part. Use all when everything is essential, allSettled plus per-part timeouts when the page should degrade, and always plumb an AbortController so a failure releases the siblings' resources.

Advanced

The aggregate is a load multiplier and a tail-latency amplifier: N concurrent downstream calls, and a response time equal to the slowest of them. Bound N and bound each member's latency, or the endpoint inherits the worst behaviour of every dependency it has.

Internals

The combinator is a counter plus an indexed results array plus one continuation per member. all settles the aggregate on the first rejection and simply drops later member outcomes on the floor — which is precisely why the members' own continuations still fire and why unhandled rejections appear.

Apply it