The question this answers
These three calls do not depend on each other — why does the endpoint take the sum of their latencies?
A dashboard endpoint that awaits the user profile (80 ms), then their org (60 ms), then their notification count (40 ms), then their recent activity (120 ms) — four independent reads, one after another.
Nothing between the four calls; they read disjoint data and touch no common state. That is precisely why serialising them buys nothing — and precisely why unserialising them is safe *for correctness* while being unsafe for load.
Endpoint latency is bounded by the longest call it must make, not by the sum of the calls it happens to make; and any call that genuinely depends on an earlier result still runs after it.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Sum versus max, and where the 300 milliseconds went
Written in the obvious order, four independent awaits produce four sequential round trips: 80 + 60 + 40 + 120 = 300 ms. Started together, the endpoint takes as long as the slowest, which is 120 ms. Nothing about the calls changed; only the moment each one started.
The reason this is so common is that the sequential form is what the syntax encourages. await reads like a statement, statements read as ordered, and the order is invisible in a code review because it looks exactly like correct code — which it is. It is just slow, and the slowness is proportional to how many independent things the endpoint happens to need.
The tell in production is a latency graph where the endpoint's p50 tracks the sum of its dependency spans and each individual span is fast. In a trace waterfall it is unmistakable: a staircase, each span starting where the previous ended, with no overlap anywhere (Reading the Waterfall).
The fix, and the dependency that must stay sequential
The mechanical change is to start every independent call before awaiting any of them. In JavaScript that means constructing the promises first, because construction is what starts the work; in Python it means wrapping each coroutine in a Task or handing them all to gather, because a bare coroutine is inert (Futures & Promises).
The important discipline is separating what is genuinely independent from what is not. A call that needs user.orgId cannot start before the user arrives, and no amount of restructuring changes that — the dependency is real and the latency is the critical path (Dependency Graphs). Parallelising a dependent call by guessing the input is not a performance fix; it is a correctness bug with a fast response time.
And now the warning, because it belongs here and not three paragraphs later. This refactor multiplies concurrent downstream load by the number of calls you unserialised. The endpoint that held one connection-pool slot for 300 ms now holds four for 120 ms. Your local benchmark improves; the database sees four times the concurrent query count at the same request rate. If the endpoint is hot, this is the change that takes the database down, and it will look like the database's fault. Read Parallelism Moves the Load Downstream before deploying it, and bound the concurrency if the fan-out is over a list rather than a fixed handful.
1async function dashboard(userId: string) {2 const user = await getUser(userId) // 80ms3 const org = await getOrg(orgIdFor(userId)) // 60ms — does not need `user`4 const notifications = await getNotifications(userId) // 40ms — independent5 const activity = await getActivity(userId) // 120ms — independent6 return render({ user, org, notifications, activity })7}8// Total: 300ms. Trace shows a staircase. Each span is individually fast.1async function dashboard(userId: string, signal: AbortSignal) {2 // Construction starts the work in JS. All four are in flight after this block.3 const userP = getUser(userId, signal)4 const orgP = getOrg(orgIdFor(userId), signal)5 const notificationsP = getNotifications(userId, signal)6 const activityP = getActivity(userId, signal)7 8 const [user, org, notifications, activity] =9 await Promise.all([userP, orgP, notificationsP, activityP])10 11 // GENUINELY dependent: needs user.teamId. This one stays sequential,12 // and it is now the critical path. Parallelising it would be a bug.13 const team = await getTeam(user.teamId, signal)14 15 return render({ user, org, notifications, activity, team })16}17 18// WARNING: this endpoint now holds 4 pool slots instead of 1.19// At 200 req/s that is 800 concurrent queries where there were 200.20// If the fan-out is over a LIST, bound it — do not map an array into all():21async function enrich(ids: string[], signal: AbortSignal) {22 return mapWithConcurrency(ids, 8, (id) => getDetail(id, signal))23 // ^ a ceiling, chosen against the pool size24}Starting the independent calls together turns a sum into a maximum, and the dependent call stays where it belongs because its input does not exist any earlier. What the refactor also does — and what no compiler will tell you — is multiply concurrent downstream load by four. Fixed handfuls are usually fine; a fan-out over a list needs an explicit concurrency bound before it ships.
When the sequence was load-bearing
Not every staircase is a mistake. Sometimes the sequential order was carrying an invariant that nobody wrote down, and removing it produces a fast, wrong answer. The three cases to check before unserialising: a real data dependency (call B needs A's output), an ordering requirement on writes, and a shared resource that only tolerates one user at a time.
The schedule below is the second case, and it is the one that gets shipped. Two calls that "do not depend on each other" in the sense that neither uses the other's return value, but which write to and read from the same row. Sequential, the read observed the write. Concurrent, it observes whatever the database happened to have.
| # | recordEvent (write) | buildSummary (read) | Database row | State |
|---|---|---|---|---|
| 1 | sequential version: INSERT audit event; commits | · | · | events=4 summaryCount=- |
| 2 | · | sequential version: SELECT count(*) → 4 | · | events=4 summaryCount=4 |
| 3 | CONCURRENT version: INSERT issued at t=0, still in flight | · | · | events=3 summaryCount=- |
| 4 | · | CONCURRENT version: SELECT count(*) issued at t=0, returns 3 | · | events=3 summaryCount=3 |
| 5 | · | · | INSERT commits at t=15 ms | events=4 summaryCount=3 ✕ The summary says 3 events; 4 were recorded. Neither call used the other's return value, so the "independence" check passed — the dependency was through the database, not through the code. |
| 6 | · | response rendered with a stale count; no error anywhere | · | events=4 summaryCount=3 |
Key points
- Independent awaits in sequence cost the sum of their latencies; started together they cost the maximum.
- In JavaScript, constructing the promise starts the work — so the fix is to construct all of them before awaiting any.
- In Python a bare coroutine is inert, so the fix is
gatherorcreate_task, not merely reordering awaits. - A trace waterfall showing a staircase of fast spans with no overlap is the signature.
- The fix multiplies concurrent downstream load by the number of calls unserialised — one pool slot for 300 ms becomes four for 120 ms.
- Independence means "shares no state and imposes no ordering", not "does not use the other's return value".
- Fan-out over a list needs an explicit concurrency bound; never map a large array straight into
Promise.all.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • Each
awaitsuspends the current task until that operation settles, so the next call is not even issued until the previous one has returned. - • Total latency is therefore the sum of the round trips plus the local work between them.
- • Starting all the operations first puts them in flight simultaneously; the awaits then only collect results that are already on their way.
- • The executor is free during all of this either way — the sequential version is not using more CPU, it is using more wall-clock.
- • The aggregate settles when the last member settles, so the endpoint's latency becomes the maximum of the members plus the local work.
- • Downstream, the change converts one concurrent unit of work per request into N, held for a shorter time. Total work is identical; peak concurrency is N times higher.
- • Sequential: profile (0–80 ms), org (80–140 ms), notifications (140–180 ms), activity (180–300 ms). One pool slot held throughout, response at 300 ms.
- • Concurrent: all four issued at t≈0, all four in flight, the slowest returns at 120 ms, response at 120 ms. Four pool slots held for 120 ms.
- • Concurrent with a genuine dependency mistakenly unserialised:
getTeam(user.teamId)is started beforeuserexists, soteamIdisundefinedand the call either 404s or — worse — returns a default team. - • Concurrent write-then-read on the same row: INSERT and SELECT issued together, SELECT returns before the INSERT commits, and the summary is short by one with no error.
- • At 200 req/s: sequential holds ~60 concurrent connections on average, concurrent holds ~96 at four times the instantaneous rate — and the peak, not the average, is what exhausts a pool of 100.
- • Fan-out over a list of 500 with no bound: 500 simultaneous queries, 480 of them queued on a pool of 20, each one's timeout running from creation (Parallelism Moves the Load Downstream).
- • Guaranteed: unserialising genuinely independent calls does not change the result. The correctness argument is sound when the independence is.
- • Guaranteed: latency drops from the sum to the maximum plus scheduling overhead.
- • Guaranteed: total downstream *work* is unchanged — the same queries, the same rows.
- • NOT guaranteed: that downstream can absorb the new concurrency. Peak concurrent load is multiplied by the fan-out width.
- • NOT guaranteed: that "does not use the other's return value" means independent. Shared rows, caches and external resources create dependencies the code does not show.
- • NOT guaranteed: any bound on concurrency.
Promise.allover an array starts everything at once, always. - • NOT guaranteed: improvement at all if the calls contend for the same bottleneck — four queries against a saturated pool finish no sooner than four in sequence.
- • Peak concurrent downstream usage multiplies by the fan-out width, so the connection pool, the rate limiter and the thread pool all see N times the instantaneous demand.
- • If the four calls hit the same backend, they queue there instead of overlapping and the latency win shrinks toward zero while the load multiplication remains — the worst of both.
- • On an event loop, four continuations become ready close together, so the aggregate's resolution is a small burst rather than a spread.
- • Tail latency now belongs to the slowest member; a dependency with a bad p99 exports it to your endpoint the moment you parallelise (Fan-Out: Waiting for the Slowest of Seven).
- • Unserialising a genuine data dependency: the second call runs with
undefinedinput and returns wrong data or a plausible default. - • Unserialising a write-then-read: the read observes state from before the write, with no error.
- • Connection-pool exhaustion after the deploy, because peak concurrency rose by the fan-out factor (Parallelism Moves the Load Downstream).
- • Rate-limit rejections from a third party that was fine with one call per request and is not fine with four at once.
- • Timeout storms: all N calls start their clocks together, so a slow backend fails all of them simultaneously instead of one at a time.
- • Unhandled rejection from an abandoned member after
Promise.allrejects (Promise.all & gather).
- • A fixed, small set of genuinely independent reads on a hot endpoint — the classic dashboard or product page.
- • Calls to *different* backends, where the concurrency is spread across systems rather than concentrated on one.
- • High-latency, low-cost dependencies: a 200 ms third-party API that costs nothing to run concurrently.
- • Anywhere the trace shows a staircase and the total is dominated by round trips rather than by work.
- • When the calls share a bottleneck — one database, one pool, one rate limit — so they queue instead of overlapping.
- • When the fan-out is over a list, because the width is now data-dependent and unbounded.
- • When the ordering was carrying an invariant that nobody documented.
- • When the endpoint is hot: the multiplication applies at your peak request rate, which is exactly when the dependency has the least headroom.
- • When the "independent" calls are cheap and local; you have added scheduling to save microseconds.
- • Endpoint latency versus the sum of its dependency spans. Equal means fully serialised; equal to the max means fully overlapped.
- • The trace waterfall itself — a staircase is the diagnosis and it takes one look (Reading the Waterfall).
- • Before deploying the fix: peak concurrent downstream calls, connection-pool utilisation and pool wait time. Take the baseline first, because you will need it.
- • After deploying: the same three numbers, plus rate-limit rejection counts on any third party in the fan-out.
- • Downstream request rate divided by inbound request rate. It should rise by exactly the fan-out width; if it rises more, retries are compounding.
- • p99 of the endpoint, not p50. Parallelising moves you onto the slowest member's tail, and that shows up at p99 first.
- • The code no longer reads top-to-bottom as a sequence of steps; the reader must work out which promises are in flight at each line.
- • Error handling changes shape: one failure now abandons in-flight siblings unless cancellation is plumbed through (Promise.all & gather).
- • A concurrency bound becomes a parameter someone must choose and justify against the pool size — and it is not derivable from first principles (Bounding Concurrency).
- • The dependency analysis is now load-bearing and lives only in the author's head; a later edit that adds a shared-row dependency between two parallelised calls reintroduces the race silently.
- • One batch call instead of N concurrent ones: a single query with an
INclause, or a batch endpoint. Better latency *and* less load — strictly superior when it exists (batch-apis). - • Bounded concurrency (
mapWithConcurrency(items, K, fn)) whenever the fan-out is over a list, so the width is a decision rather than a consequence of the input size. - • Cache the slow member. If activity is 120 ms and changes rarely, caching it removes it from the critical path without touching the concurrency at all.
- • Denormalise or precompute: if the same four things are always fetched together, one read of a materialised view beats four concurrent reads.
- • Leave it sequential when the endpoint is cold, the total is acceptable, and the dependency is fragile. 300 ms on an admin page is not worth multiplying database concurrency by four.
Sequential awaits vs Promise.all
sequential 90 + 60 + 120 = 270 ms peak downstream concurrency = 40 × 1 = 40 Promise.all max(90, 60, 120) = 120 ms peak downstream concurrency = 40 × 3 = 120 throughput 148/s → 333/s downstream sees 444/s → 1000/s
One request, N downstream calls
Bounding concurrency with permits
| permits | goodput | mean latency | timeouts | failed of 10K |
|---|---|---|---|---|
| 1 | 25/s | 43 ms | 0.00% | 0 |
| 5 | 125/s | 43 ms | 0.00% | 0 |
| 10 | 250/s | 43 ms | 0.00% | 0 |
| 25 | 625/s | 43 ms | 0.00% | 0 |
| 50 | 1000/s | 53 ms | 0.00% | 0 |
| 100 | 1000/s | 103 ms | 0.00% | 0 |
| 200 | 1000/s | 203 ms | 0.00% | 0 |
| 350 | 1000/s | 353 ms | 0.00% | 0 |
| 500 | 0/s | 503 ms | 100.0% | 10K |
What people believe, and what is true
They do not use each other's return values, so they are independent.
Independence means no shared state and no required ordering. A write and a read on the same row use no return values from each other and are absolutely not independent.
Parallelising is free — it is the same work.
The same total work, arriving N times more concurrently. Peak concurrency is what exhausts pools and trips rate limits, and this refactor multiplies exactly that.
Moving the awaits around fixed it in Python too.
A Python coroutine does nothing until it is scheduled. Hoisting the calls above the awaits changes nothing at all; you need asyncio.gather or create_task.
Go deeper
Overview
Awaits in sequence cost the sum; started together they cost the maximum. Four independent 80 ms calls should not take 320 ms.
Practical
Look for a staircase in the trace. Start independent calls before awaiting any of them, keep genuinely dependent calls in order, and take a baseline of downstream pool utilisation before you deploy.
Advanced
The refactor is a load multiplier at your peak request rate. Fixed handfuls are usually safe; fan-out over a list needs an explicit bound, and the bound must be chosen against the downstream pool rather than against the input size.
Internals
Nothing changes in the executor: it was idle during the sequential waits too. What changes is the number of outstanding operations at the downstream resource, which is the quantity Little's law relates to queue length and wait time there.