Request IDs: The Contract's Correlation Clause
One opaque id, minted at the edge, propagated through every hop, returned in every response — especially errors. It is the difference between "can you send a screenshot?" and finding the exact failing request in one query.
Frame the contract
API design starts with a consumer, a design question and a guarantee — never with a URL.
The problem is a join, and the id is the key
A production API request touches a CDN, a gateway, one or more services, a database and maybe a queue — each writing its own logs with its own clocks. When a consumer reports "the payment call failed around 14:30 yesterday", you are being asked to join five log systems on fuzzy timestamps and guesswork. At any real traffic volume that join is unperformable: 14:30 contains forty thousand requests, and the consumer's clock is wrong anyway.
A request id turns the join into a lookup. The edge mints an opaque id for every inbound request, every hop forwards it, every log line includes it, and the response returns it to the caller. Now the bug report contains req_01HQX4T9, and one query returns the request's whole story in order: gateway received it, service A called service B, B's database call timed out, A returned 504. The id is the primary key of the incident.
This is deliberately smaller than distributed tracing. A trace records spans, timings and causality graphs and needs sampling to survive traffic (Distributed Tracing owns that machinery); a request id is one indexed string on every log line, cheap enough to run at 100% forever. The two compose: the id gets you to the request, the trace (when sampled) shows you inside it. What the *contract* owns is the id — because it is the only one of the two the consumer can see and quote.
The contract mechanics: mint, propagate, return, repeat
Mint at the edge. The gateway (or outermost service) generates the id for every request — unique, opaque, sortable-by-time if you like ULIDs, meaningless otherwise. If the client supplied one (X-Request-Id on the way in), you may adopt it for the client's convenience, but treat it as untrusted input: cap its length, restrict its charset, and never let it collide with your uniqueness assumptions — a malicious or buggy client that sends the same id forever must not be able to corrupt your correlation. Many providers log the client's id as a separate field and mint their own regardless; both work, but pick one and document it.
Propagate on every internal hop — HTTP header downstream, message attribute onto queues, job metadata into workers, so the id survives async boundaries where the "request" outlives the HTTP exchange (see The Async Job Pattern). Return it on every response, success and failure alike, as a header. And repeat it inside the error body: engineers copy from the JSON they can see, not from headers they forgot to log, which is why request_id is a standard field of a well-designed The Error Model: Structure Over Apology. The id in the error body is the single highest-value observability clause a contract can carry, because it is the one that crosses the trust boundary into the consumer's hands.
POST /payments HTTP/1.1
Host: api.example.com
Authorization: Bearer <token>
Idempotency-Key: 5c1e…
{ "amount": 4200, "currency": "EUR", "source": "card_abc" }HTTP/1.1 502 Bad Gateway
X-Request-Id: req_01HQX4T9GJ8Z
Content-Type: application/json
{
"error": {
"code": "upstream_unavailable",
"message": "The payment processor did not respond.",
"request_id": "req_01HQX4T9GJ8Z",
"retryable": true
}
}What the clause is worth, and what it is not
The payoff compounds across every workflow that touches a specific request. Support asks every reporter for the id and pastes it into one search box — ticket resolution stops depending on which engineer remembers which log system. Consumers log the ids of their failed calls and can hand you evidence instead of anecdotes; your status page and incident reviews can name exact affected requests. Idempotency debugging gets a second key: "did my retry replay or re-execute?" is answerable by comparing the two request ids against one Idempotency Keys: The Mechanism record.
Know the limits. A request id identifies; it does not measure or explain — you still need API Metrics: Rate, Errors, Duration, Sizes for the aggregate view and structured API Logging Without Leaking for the id to join against. It is also only as good as its weakest hop: one service that drops the header, one queue producer that forgets the attribute, and the story goes dark exactly where the bug lives. Propagation is infrastructure discipline (middleware, client libraries, lint rules), not per-team goodwill.
- Mint: at the edge, for every request, opaque and unique; client-supplied ids are untrusted input — cap, sanitize, or log-separately.
- Propagate: headers on sync hops, attributes on queues, metadata on jobs — the id must survive async boundaries.
- Return: response header on every status; repeated as
request_idinside every error body. - Log: one indexed field on every line; 100% of traffic, no sampling — this is the cheap layer under tracing.
- Document: tell consumers to log it and quote it; an id nobody records correlates nothing.
Key points
- A request id converts the unperformable "join five log systems on timestamps" into a single indexed lookup.
- The lifecycle is mint at the edge → propagate every hop (including queues and jobs) → return on every response → repeat inside error bodies.
- The id in the error body is the clause that matters most: it is the piece of your observability that crosses into the consumer's hands.
- Client-supplied ids are untrusted input — adopt them cautiously (length, charset, no uniqueness assumptions) or log them as a separate field.
- Request ids run at 100% where distributed tracing samples; the id finds the request, the trace explains it.
- One hop that drops the header darkens the story exactly where the bug usually is — propagation belongs in shared middleware, not per-team convention.
Follow the failure
How the contract fails or gets misused, hop by hop — and what it costs when it completes.
- 1Team → API: ships without request ids; logs are per-service and correlated by timestamp when needed.
- 2Consumer → support: reports a failed payment "around 14:30"; support asks for screenshots and HAR files.
- 3Support → engineering: an engineer spends two hours grepping three log systems for a request that turns out to have failed at 14:47.
- 4Team → gateway: adds an id at the edge but services do not forward it; the id finds the gateway line and nothing beneath it.
- 5On-call → incident: during an outage, per-request attribution is impossible; the postmortem's timeline is reconstructed from consumer complaints.
- Support cost scales with traffic: every ticket about a specific request becomes a manual forensic project.
- Consumer trust erodes — "we cannot find your request" reads as "we do not know what our system did with your money".
- Incident response slows at the worst time: blast-radius questions ("which requests were affected?") have no queryable answer.
Design, observe, evolve
A contract decision is incomplete until you know how you would notice it failing and how it changes later.
- • Mint an opaque id at the outermost edge for 100% of requests and return it on 100% of responses, all status codes, no exceptions.
- • Make `request_id` a required field of the error envelope so the id reaches the humans who will quote it back.
- • Enforce propagation in shared middleware and internal client libraries — HTTP header, queue attribute, job metadata — so no team can silently drop it.
- • Treat inbound client ids as untrusted: length-cap, charset-restrict, and never derive uniqueness or security decisions from them.
- • Log-line coverage: the percentage of lines carrying a request id per service — a dropped-propagation hop shows up as a coverage cliff.
- • Support-ticket resolution time before and after ids ship is the business metric this clause moves.
- • During incidents, the id enables the query that matters: exact affected-request lists per time window instead of estimated percentages.
- • Adding ids is purely additive — a new header and error field break no one; changing the id *format* later is safe only because the contract says "opaque" (consumers who parsed it were violating the contract).
- • Upgrading to W3C `traceparent` alongside the request id keeps consumer-facing simplicity while joining the tracing ecosystem — carry both, document one.
- • Extend the same id into webhook deliveries and async job records so the correlation clause covers the contract's asynchronous half (see [[webhook-delivery]]).
- • A few bytes per log line and per response, multiplied by everything — negligible almost everywhere, measurable only at extreme volume.
- • Propagation middleware must exist for every internal protocol and language in the fleet; the discipline is organizational, and its absence is invisible until the first dark hop.
- • Adopting client-supplied ids helps consumers correlate but imports their bugs (duplicates, junk) into your primary key unless you sanitize or double-key.
Misconceptions
request_id from the error body. Carry both; they answer different halves of the problem.