MiddlewareGENERALRUNTIME-SPECIFICPROTOCOL-SPECIFICFRAMEWORK-SPECIFIC

Request Context Propagation

Getting the correlation id, principal, tenant and deadline from the outermost middleware to a log line five layers down — without an extra argument on every function, and without a global that lies.

What actually happens How to build it

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.

The question

How does per-request state reach code deep in the call stack without being threaded through every signature?

The requirement

Every log line, every database query comment, every outbound call and every emitted event should carry the correlation id and the caller's identity — from code that has no idea an HTTP request exists.

The obvious build

Store it in a module-level variable when the request arrives. The process handles one request at a time from the perspective of any given piece of code, so reading it back later is safe.

Why it breaks

It is not one request at a time. A module-level variable in a process serving concurrent requests holds whichever request wrote last, so log lines get attributed to the wrong caller (Backend Races).

How it breaks in production

It is not one request at a time. A module-level variable in a process serving concurrent requests holds whichever request wrote last, so log lines get attributed to the wrong caller (Backend Races).
On an event-loop runtime the value is overwritten at every await: request A sets it, awaits the database, request B overwrites it, A resumes and reads B's tenant (The Node Event Loop).
A thread-local works in a thread-per-request server right up to the moment work is handed to a thread pool, an executor or a parallel stream — the new thread has no value at all.
The context is silently absent rather than wrong, and code that reads ctx.tenantId ?? undefined builds a query with no tenant filter. A missing context becomes a cross-tenant read (Tenant Isolation).
The trace ends at your process boundary: the id is on every internal log line and not on the outbound HTTP call or the queued job, so the downstream service starts a new, unrelated trace (Tracing From the Backend's Side).

Requirement→API Contract→Application Logic→Data Access→External Deps→Concurrency→Failure→Security→Observability→Deployment→Scale

What is actually happening

There are exactly three ways to carry per-request state, and every stack uses one or a mix: pass it explicitly, store it in ambient storage keyed to the current execution, or resolve it from a request-scoped container.
Explicit is Go's context.Context convention — the first parameter of every function that might do I/O. It is verbose and it is impossible to lose, and it carries cancellation and deadlines as well as values (Deadlines vs Timeouts).
Ambient is Node's AsyncLocalStorage, Python's contextvars.ContextVar, Java's ThreadLocal and ScopedValue, .NET's AsyncLocal. The runtime associates a store with the current execution context and propagates it across the async boundaries it knows about.
The phrase "the async boundaries it knows about" is the whole subject. contextvars are copied into an asyncio task when the task is created — but a function dispatched to a ThreadPoolExecutor gets a fresh context unless you copy one in. ThreadLocal is lost the moment work is submitted to an executor. AsyncLocalStorage follows promises and timers created inside the store, and not a callback registered on an emitter created outside it.
Request-scoped containers — Spring's request scope, ASP.NET's IHttpContextAccessor — are usually ambient storage with dependency injection on top, so they inherit the same boundary rules.
Propagating outward is a separate problem with a standard answer: W3C Trace Context defines traceparent and tracestate headers, and OpenTelemetry instrumentation injects and extracts them. For queues the equivalent is message metadata, and it must be written at enqueue time because the consumer runs in a different process much later (OpenTelemetry Concepts).
What goes in the context is a design decision with a strong default: a small, immutable, well-typed set — correlation/trace id, principal, tenant, deadline, locale, feature-flag evaluation context. Not a mutable bag, and never a database connection or an open transaction.

Three ways to carry it

The choice is not primarily about ergonomics. It is about what happens when propagation fails, and different values have very different answers to that. Losing a correlation id degrades a trace; losing a tenant id can return another customer's data. That asymmetry is what should drive the decision.

Most real services use a mix, and that is fine as long as the split is deliberate: ambient for what degrades, explicit for what breaks.

How should per-request state reach deep code?

What happens if this value is missing or wrong at the point of use?

Explicit parameter (Go `context.Context`)

when The value must be correct: tenant, principal, deadline, cancellation. Also whenever work crosses goroutines, threads or processes.

cost Every function that might do I/O grows a parameter, and adding one to a deep call chain is a wide diff.

Ambient storage (`AsyncLocalStorage`, `contextvars`, `AsyncLocal`)

when The value improves diagnostics: correlation id, trace context, log fields. Losing it degrades rather than breaks.

cost Invisible dependency; lost silently at any boundary the runtime does not model; harder to test in isolation.

Request-scoped container (Spring, ASP.NET)

when The stack already resolves collaborators by injection and the framework owns the request scope.

cost Ambient storage in disguise, with the same boundary rules plus a container lifetime to understand.

Attach to the framework request object

when Small services where nothing below the handler needs it and nothing crosses a boundary.

cost Couples every consumer to the transport, so the same code cannot run from a queue consumer (What a Handler Is Responsible For).

Module-level variable

when Never in a concurrent server.

cost Wrong attribution under concurrency, silently, in production only (Backend Races).

Ambient context ends where you stopped instrumenting

RUNTIME-SPECIFICAsyncLocalStorage is Node's mechanism and propagates through promises, timers and callbacks created inside run(). Python's contextvars propagate into asyncio tasks at creation time, not into threads. Java ThreadLocal propagates to nothing but its own thread. The pattern — establish at the edge, re-establish at each boundary — transfers; the API and the set of automatic boundaries do not.

The failure that costs the most time is not "we never set it up". It is a context that works perfectly across the boundaries the runtime knows about and vanishes at one it does not — so 95% of your log lines carry the id and the interesting 5% do not.

The two examples below are the boundaries that catch people most often on each runtime: a worker thread on Node, and a thread-pool executor in Python. The fix in both cases is to capture the value at the boundary and re-establish it on the other side.

Set once at the edge, and mind the boundaries

1import { AsyncLocalStorage } from 'node:async_hooks'
2 
3type Ctx = Readonly<{
4  correlationId: string
5  principal?: Principal
6  tenantId?: string
7  deadlineAt: number
8}>
9 
10export const als = new AsyncLocalStorage<Ctx>()
11export const ctx = () => als.getStore()
12 
13// outermost middleware: establish it once
14app.use((req, res, next) => {
15  const value: Ctx = {
16    correlationId: safeInboundId(req.headers['x-correlation-id']) ?? randomUUID(),
17    deadlineAt: Date.now() + REQUEST_BUDGET_MS,
18  }
19  res.setHeader('x-correlation-id', value.correlationId)
20  als.run(value, next)          // everything downstream runs inside the store
21})
22 
23// the logger reads it, so no call site has to pass it
24const log = baseLogger.child({ get correlation_id() { return ctx()?.correlationId } })
25 
26// --- boundaries the runtime does NOT cross for you ---
27 
28// 1. a queue: serialise it into the message, or the consumer starts a new trace
29await queue.enqueue('send-receipt', { orderId }, {
30  headers: { correlationId: ctx()!.correlationId },
31})
32 
33// 2. a worker thread: pass it in, re-establish it on the other side
34worker.postMessage({ job, correlationId: ctx()!.correlationId })
35 
36// 3. an outbound call: inject W3C trace context
37await fetch(url, { headers: { traceparent: buildTraceparent(ctx()!) } })

The middleware is four lines and the boundaries are the work. In Python the equivalent trap is ThreadPoolExecutor: contextvars are copied into an asyncio Task at creation but not into a pool thread, so you must submit contextvars.copy_context().run(fn, ...) to carry the value across.

Fail closed on the values that matter

The last failure in this table is the reason this lesson is not filed under observability. A tenant id carried ambiently and read defensively is one lost boundary away from an unscoped query, and the code that does it looks careful — a ?? with a sensible default is exactly what a reviewer expects to see.

The rule is short: values that exist to explain what happened may default; values that exist to constrain what happens may not.

Trigger	Symptom	Cause	Response
Work handed to a worker thread or executor	Log lines in that work carry no correlation id	Ambient storage is not propagated across that boundary	Capture at the boundary and re-establish on the other side; assert it in a test
Job enqueued without context in the message	The consumer's trace is unrelated to the request that caused it	Context held in memory, not serialised	Write trace context into message metadata at enqueue (Job Queues)
Value read after an await at an uninstrumented boundary	Audit entries name the wrong principal	Context bleed between concurrent requests	Keep the context immutable; establish it with `run()` rather than assigning into a store
Tenant read as `ctx.tenantId ?? undefined`	A query returns rows from every tenant	A missing context silently disabled the filter	Raise on absence in the data-access layer; give system jobs an explicit system principal (Tenant Isolation)
Inbound `traceparent` accepted from any client	Traces merged with a caller's, or oversized headers	Untrusted input treated as trusted context	Validate and bound it; accept only from trusted hops, otherwise generate and record theirs (The Trust Boundary)
Connection or transaction stored in the context	Connections leak; transactions outlive their request	A resource lifetime attached to an ambient store	Keep the context to immutable values; pass resources explicitly (Connection Pools)

How to build it

Most important first.

Establish the context once, in the outermost useful middleware, from authenticated state and inbound headers — accepting an inbound trace id if the caller is trusted, generating one otherwise (Correlation Ids That Survive Every Hop).
Make it immutable and typed. A context that anything can mutate mid-request is shared mutable state with extra steps.
Use ambient storage for observability concerns — logging, metrics, tracing — where losing it degrades diagnostics. Use explicit parameters for anything that must be correct: tenant scoping, deadlines, cancellation.
Fail closed on absence. A data-access layer that finds no tenant in the context must raise, never fall back to an unscoped query (Multi-Tenancy).
Instrument every boundary that the runtime does not propagate across for you: thread pools, worker threads, queue producers and consumers, batching layers, and outbound HTTP clients.
Propagate outward on every egress — traceparent on HTTP calls, metadata on queue messages, a comment on database queries where the driver supports it (Distributed Tracing).
Keep the context small. Everything you add is copied per request and appears in every log line that serialises it (Cardinality: The Label That Took Down Monitoring).

What can go wrong

Failure modes

Silent loss at an uninstrumented boundary: logs simply stop carrying the id, and the gap is only visible if you measure coverage.
Context bleed: a value read after an await belongs to a different request, so a log line or an audit entry names the wrong principal.
A context grown into a grab bag, so it is copied everywhere, serialised into logs, and nobody can say what is safe to remove.
A database connection or transaction stashed in the context, which turns an ambient store into a resource-lifetime problem (Connection Pools).
Trusting an inbound traceparent from an untrusted client, letting a caller merge their requests into someone else's trace or inject huge header values.
A batching or coalescing layer that serves two callers from one upstream call and attributes both to whichever context it captured (Request Coalescing).
Ambient context used for authorisation, so a lost context becomes an unscoped query rather than a denial.

What can race

Reading ambient context after an await, on a runtime or at a boundary where it is not correctly propagated, yields another request's value — the classic context bleed, and it produces plausible, wrong data rather than an error.
Mutating a shared context object from concurrent continuations of the same request means the last writer wins and neither branch can be reasoned about; keep the context immutable.
A request-coalescing layer serving several waiters from one upstream call attributes the work to one context, so the other callers' traces have a hole (Request Coalescing).
A background task started inside a request store and outliving the response continues to report the finished request's context long after it ended.

Security

Tenant scoping from ambient context must fail closed. An absent tenant that produces an unfiltered query is a cross-tenant disclosure, and it is invisible in testing where only one tenant exists (Tenant Isolation).
The principal in the context must come from authentication only — never from a header a client can set, and never merged with values from the body (Authentication in a Backend).
Do not put secrets, tokens or full credentials in a context that gets serialised into logs; put a reference (Secrets in Logs).
Inbound trace headers are attacker-controlled unless the hop that set them is trusted; validate their format and bound their size, and consider generating your own id and recording theirs as an attribute (Trust Boundaries).
The context is what makes an audit record meaningful. If it is lost, the audit says an action occurred and not who performed it (Audit Logs for Privileged Actions).

Misreads

"A global variable is fine because we handle one request at a time." No backend handles one request at a time. This assumption is wrong on every runtime, for different reasons.
"AsyncLocalStorage and contextvars propagate everywhere." They propagate across the boundaries the runtime models. Thread pools, worker threads, native callbacks and some third-party libraries are not among them.
"Thread-locals are safe in a threaded server." Until the request touches an executor, a parallel stream, or any library that dispatches work — then the value is gone.
"Context propagation is a tracing feature." It is also how tenant, principal, deadline and cancellation reach the code that must honour them.
"Passing context explicitly is boilerplate." It is a signature that tells the truth about what a function depends on, which is why Go made it a convention rather than a library.
"If the context is missing we can fall back to a default." For diagnostics, yes. For tenant or principal, the fallback is a data leak (Multi-Tenancy).

Operating it

How you see it in production

Measure the share of log lines carrying a correlation id. It should be effectively 100%; a drop after a refactor points precisely at a lost boundary.
Measure the share of traces whose spans all share one trace id, and count orphan spans. Orphans are propagation failures with a location attached (Parents, Children and Links).
Assert propagation in tests at each known boundary: an enqueued job, a worker thread, an outbound HTTP call. These are cheap tests that catch a class of regressions nothing else will.
Log the tenant on every data-access call in development and assert it is present, so fail-closed behaviour is proven rather than assumed.
Watch context size if you serialise it; a growing object appears as growing log volume long before anyone connects the two (The Log Bill and What It Is Buying).

What changes at 10x and 100x

Ambient storage has a real cost on some runtimes — the runtime maintains a context per async operation — which is negligible next to any I/O and measurable in a hot loop doing millions of tiny async operations.
At 100x, context values become log and metric dimensions, so an extra field is an extra dimension on every line and possibly every series (Cardinality: The Label That Took Down Monitoring).
Across many services, propagation becomes the difference between one trace and forty disconnected ones; the value of the discipline rises sharply with the number of hops (Distributed Tracing).
With background jobs, the context must survive an enqueue/dequeue gap that can be hours, so it has to be serialised into the message rather than held in memory (Job Queues).

What this costs

Explicit context is verbose and impossible to lose. Ambient context is invisible and easy to lose. That is the whole trade, and it is why the recommendation splits by consequence: ambient for diagnostics, explicit for correctness.
Ambient storage makes functions depend on state that is not in their signature, which makes them harder to test and harder to reason about in isolation.
Instrumenting every boundary is ongoing work: every new async utility, thread pool or client library is a boundary until proven otherwise.
Failing closed on a missing tenant will, at some point, break a legitimate internal job that never had a tenant. That is the correct failure, and it needs an explicit system-principal path rather than a fallback to unscoped.

Where this applies

Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.

GENERALThe three strategies — explicit, ambient, container — and the rule that ambient storage stops at boundaries the runtime does not model are true everywhere.
RUNTIME-SPECIFICNode AsyncLocalStorage follows promises, timers and callbacks created inside run(); work sent to a worker thread does not inherit it. Python contextvars are copied into an asyncio Task at creation, so a task started before you set a value never sees it, and a ThreadPoolExecutor job gets a fresh context unless you pass contextvars.copy_context(). Java ThreadLocal is per-thread and lost on executor submission; ScopedValue and structured concurrency change this. Go has no ambient mechanism at all by design.
PROTOCOL-SPECIFICW3C Trace Context (traceparent, tracestate) is the interoperable format for HTTP; older stacks may still emit B3 headers, and the two must be bridged rather than assumed compatible. Queues have no standard — propagation is whatever convention you put in message metadata.
FRAMEWORK-SPECIFICSpring offers request-scoped beans and RequestContextHolder; ASP.NET Core offers IHttpContextAccessor backed by AsyncLocal. Both look like dependency injection and inherit ambient storage's boundary rules exactly.

Where the depth lives

This domain teaches the application-side mechanism and hands the rest off.

API Designrequest-ids idempotency-keys

Architecturedistributed-tracing

Securitytrust-boundaries audit-logs multi-tenant-isolation

Performancecontext-propagation correlation-ids distributed-tracing span-relationships opentelemetry trace-anatomy cardinality log-cost-and-sampling

Concurrencycancellation-propagation deadlines-vs-timeouts structured-concurrency shared-mutable-state