ObservabilityGENERALPROTOCOL-SPECIFICDATABASE-SPECIFIC

An Error Taxonomy That Maps Cause to Response

Eight kinds of failure, each with a different status, a different caller action and a different owner — instead of one 500 for everything.

What actually happens How to build it

The requirement, the obvious build, and why it breaks

Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind has a problem.

The question

When something goes wrong in a handler, how do I decide what the caller should be told and what they should do about it?

The requirement

Clients keep opening tickets saying "the API returned an error". Support cannot tell whether the user typed something wrong, lacks permission, hit a conflict, or whether we are broken. Neither can our own dashboards.

The obvious build

Wrap the handler in try/catch, log the exception, return 500 Internal Server Error with { "error": "Something went wrong" }. It is uniform, it never leaks anything, and it is one line of code.

Why it breaks

A client cannot distinguish "you sent an invalid email address" from "our database is down". Both are 500, so the SDK retries both — the first will never succeed and the second gets a retry storm (Retry Storms).

How it breaks in production

A client cannot distinguish "you sent an invalid email address" from "our database is down". Both are 500, so the SDK retries both — the first will never succeed and the second gets a retry storm (Retry Storms).
Your error-rate metric and your alerting are now useless: 500s are supposed to mean "we are broken", and 90% of them are users mistyping a form field. Real outages hide inside the noise (Alert Fatigue: The Page Nobody Reads).
Support cannot triage. Every failure looks identical from outside, so every ticket becomes an engineer reading logs.
The opposite failure is just as common: returning 400 for a database timeout, which tells the caller "fix your request" for something they cannot fix, so they stop retrying a transient failure that would have succeeded.
Once every error is a 500 you also lose the one distinction that matters most operationally — whose fault is it, and does retrying help.

Requirement→API Contract→Application Logic→Data Access→External Deps→Concurrency→Failure→Security→Observability→Deployment→Scale

What is actually happening

An error carries three independent pieces of information, and collapsing them is what causes the damage: what went wrong, who can fix it, and is repeating the request useful.
Validation — the request is malformed or violates a rule. Caller-fixable, never retryable unchanged, 400 (or 422 where the shape is fine and the semantics are not).
Authentication — we do not know who you are, or the credential is expired or invalid. 401. Retryable only after obtaining a new credential.
Authorization — we know who you are and you may not do this. 403. Never retryable; retrying is the definition of not accepting the answer (Authentication vs Authorization).
Not found — the resource does not exist, or does not exist *for you*. 404. Not retryable, and deliberately indistinguishable from "exists but is not yours" when disclosure matters.
Conflict — the request is valid but the current state rejects it: a duplicate, a version mismatch, an already-cancelled order. 409 (or 412 for a failed precondition). Retryable only after reading the new state (Optimistic Concurrency).
Dependency failure — a downstream service, database or provider failed. 502 or 503. Yours to fix, and retryable with backoff.
Timeout — a dependency did not answer inside the budget. 504. Retryable, but only if the operation is idempotent, because the work may still be running (Idempotency in Backends).
Internal — a bug: a null dereference, a broken invariant, an unhandled case. 500. Retrying will not help, and this is the only category that should ever page someone.

The matrix that replaces the catch-all

The taxonomy earns its keep because each row implies a *different action by a different party*. That is the test for whether a category is real: if two categories lead to the same status, the same client behaviour and the same owner, they are one category.

Read the last two columns first. "Who fixes it" tells you whether to alert. "What the caller should do" is the entire reason the response has a status code — a client SDK, a retry policy and a support agent all read that column and nothing else.

Error kind	HTTP status	What the caller should do	Who fixes it	Alert?
Validation	400 / 422	Fix the request and resend. Never retry unchanged	The caller	No — graph it, do not page
Authentication	401	Obtain a fresh credential, then retry once	The caller	No, unless the rate jumps after a deploy
Authorization	403	Stop. Request access out of band	An administrator	No — but audit it (Audit Logs for Privileged Actions)
Not found	404	Stop, or re-read the collection	The caller	No
Conflict	409 / 412	Re-read current state, then decide whether to resend	The caller, informed by state	No, unless the rate is anomalous
Dependency failure	502 / 503	Retry with backoff and jitter	You	Yes
Timeout	504	Retry with backoff — only if the operation is idempotent	You	Yes
Internal	500	Do not retry. Report it	You, with a code change	Yes, always

Categories live in the application layer, statuses live at the edge

LANGUAGE-SPECIFICTypeScript's exhaustive Record is the mechanism here. Go would use a sentinel error set with errors.Is and lose compile-time exhaustiveness; Rust's enum plus a match gives the same guarantee more strongly; Python needs an explicit assert-never helper or a mypy exhaustiveness check to get any of it.

The most common way a taxonomy dies is being expressed as HTTP inside business logic. A service function that returns res.status(409) cannot be called from a queue consumer, a scheduled job or a test, because those have no response object. The category is a domain fact; the status is a transport rendering of it.

Raising typed errors also makes the mapping exhaustive. With a union of categories, adding a ninth forces the compiler to point at the mapper. With ad-hoc strings, adding a ninth silently falls through to 500 and nobody notices until an incident.

Categories as types, statuses as one mapping

1export type ErrorCategory =
2  | 'validation' | 'authentication' | 'authorization' | 'not_found'
3  | 'conflict' | 'dependency' | 'timeout' | 'internal'
4 
5export class AppError extends Error {
6  constructor(
7    readonly category: ErrorCategory,
8    /** stable, part of the contract: 'order_already_cancelled' */
9    readonly code: string,
10    /** safe to show a caller; never interpolate internal state */
11    message: string,
12    /** kept for logs only, never serialized into the response */
13    readonly cause?: unknown,
14  ) {
15    super(message)
16  }
17}
18 
19const STATUS: Record<ErrorCategory, number> = {
20  validation: 400, authentication: 401, authorization: 403, not_found: 404,
21  conflict: 409, dependency: 503, timeout: 504, internal: 500,
22}
23 
24// Retryability is asserted, not inferred from the status.
25const RETRYABLE: Record<ErrorCategory, boolean> = {
26  validation: false, authentication: false, authorization: false, not_found: false,
27  conflict: false, dependency: true, timeout: true, internal: false,
28}

The Record<ErrorCategory, ...> is doing real work: adding a category to the union makes both tables fail to compile until they are updated. A switch with a default would not.

Where categories get lost

In practice the taxonomy is rarely wrong on paper — it is destroyed in transit. Somewhere between the driver that knew the exact SQLSTATE and the handler that returned a status, a catch (e) widened the type and the information evaporated.

These four are the leaks worth checking for by name in any codebase that returns too many 500s.

How a known category becomes an unknown 500

Trigger	Symptom	Cause	Response
Unique constraint violated on insert	500 on a duplicate signup	The driver error was caught as a generic DB exception; SQLSTATE 23505 never inspected	Translate engine error codes to categories in the repository layer (The Repository Layer)
Payment provider returns 400 for a bad card token	Your API returns 400 to a caller whose request was fine	Downstream status forwarded verbatim	Their 4xx is your bug or your dependency failure — classify it from your side (Calling Something You Do Not Control)
HTTP client hits its deadline	500, and a duplicate charge after the client retries	Timeout collapsed into internal; outcome-unknown treated as outcome-failed	Return 504, and key the operation so a retry is safe (Idempotency Keys)
Validation library throws on an unexpected shape	500 on a malformed body	The parse step ran outside the try that maps validation errors	Parse at the boundary and raise a validation category there (Parse, Do Not Validate)

How to build it

Most important first.

Define the taxonomy as types in your application layer, not as HTTP statuses in handlers. Application code raises ConflictError; one boundary maps it to a status (Error Boundaries: Three Translations, Not One).
Give every category a stable, machine-readable code in the response body — order_already_cancelled, not just 409. Statuses are coarse; clients branch on codes (The Error Model: Structure Over Apology in API Design owns the contract shape).
Make retryability explicit rather than inferable. "Retryable" is a property you assert, and it is not the same as "safe to retry" — that second property is idempotency, and it belongs to the operation, not the error.
Split your error-rate metric by category. A dashboard where 4xx and 5xx are one line cannot show you an outage.
Alert only on the categories you own: internal, dependency and timeout. Validation and authorization errors are user behaviour and belong on a different graph.
Default unclassified exceptions to internal/500. Unknown means bug, and the fix is to classify it, not to widen the default.

What can go wrong

Failure modes

Over-classification: twenty subtypes nobody maps to a status, so handlers fall back to 500 anyway. The taxonomy is only useful if the boundary handles every branch exhaustively.
Category drift: a dependency failure raised as a validation error because the downstream service returned 400 to *you*. Their 400 is your 502 — do not forward statuses from dependencies (Calling Something You Do Not Control).
A conflict returned as 500 because the unique-constraint violation was caught as a generic database exception. The database told you the category; the code discarded it.
Using 400 for authorization to avoid confirming a resource exists. That hides the reason from legitimate clients too; prefer 404 for existence-hiding and keep 403 honest.
The mitigation failing: an error mapper that stringifies the original exception into the response, which reintroduces exactly the leak the mapper was meant to prevent (Not Leaking Your Internals).

What can race

A conflict is a race made visible: two requests both read a valid state and one commits first. The second must be told 409, not 500 — the database's serialization or unique-constraint error is the signal (Optimistic Concurrency).
A timeout races with the work it abandoned. The dependency may complete after you return 504, so a retry can produce two effects unless the operation is keyed (Idempotency Keys).

Security

The category you return is an information channel. 403 versus 404 on an object the caller does not own tells an attacker the object exists — decide deliberately, per resource, whether existence is a secret (Object-Level Authorization).
Authentication errors must not distinguish "no such user" from "wrong password". That distinction is a user-enumeration oracle (Credentials and Password Handling).
Validation errors should say which field and which rule, never why the rule exists in terms of internal state — "email already registered" is another enumeration oracle in disguise.
Rate-limit responses (429) are the one category where telling the caller more is usually right: a retry-after header prevents the hammering that the limit exists to stop (Rate Limiting).

Misreads

"4xx means the client is wrong, 5xx means we are wrong." Mostly, but a 400 you returned because *your own* code built a malformed downstream request is your bug wearing the client's status.
"Retryable means safe to retry." It does not. Retryable says repeating may succeed; safe says repeating cannot double-charge. A 504 on a non-idempotent POST is retryable and unsafe at the same time.
"Return 500 so we do not leak anything." The category is not the leak. The stack trace in the body is the leak, and you can withhold it while still returning 409.
"Timeouts are dependency failures." Related, but a timeout means you *do not know* the outcome, and a dependency failure usually means you do. That difference decides whether a compensating action is safe.

Operating it

How you see it in production

One counter, labelled by category and by route: errors_total{category, route}. If it is not split by category it will not tell you anything during an incident.
Log internal errors at error level with a stack trace and the correlation id; log validation errors at info or debug. Log level should follow ownership, not severity of tone (Log Levels Are a Convention, Not a Standard).
Track the ratio of 4xx to 5xx per route. A route whose 4xx rate jumps after a deploy usually means you broke a contract, not that users changed.
When a dependency category spikes, the useful next question is which dependency — so label the dependency error with its target, not just "external".

What changes at 10x and 100x

At 10x traffic, unclassified 500s become unreadable — the same volume of noise now buries the two real bugs. Classification is what keeps error triage constant-cost as traffic grows.
At 100x, the retry behaviour that your categories imply becomes load. Returning 503 to a million clients whose SDKs retry immediately is a self-inflicted denial of service; the category must be paired with backoff guidance (Backoff and Jitter).
Nothing about the taxonomy itself changes with scale, and that is the point: it is a design that is correct at 10 rps and still correct at 100k.

What this costs

A taxonomy is more code than a catch-all. You maintain an error type hierarchy and a mapping table, and every new failure mode has to be placed in it.
Precise categories tell honest clients more, and they tell attackers more. Some of the precision has to be deliberately spent — that is a security decision, not an oversight.
Stable machine-readable codes become part of your public contract. You cannot rename invalid_email once clients branch on it (Backward Compatibility: The Real Rules).

Where this applies

Backend advice is context-sensitive. These labels say what each claim is specific to, and where a different stack or scale would differ.

GENERALThe eight categories are transport-independent. They hold for gRPC, message consumers and internal function boundaries — only the encoding changes.
PROTOCOL-SPECIFICThe status codes below are HTTP. gRPC uses its own status enum (INVALID_ARGUMENT, PERMISSION_DENIED, FAILED_PRECONDITION, UNAVAILABLE, DEADLINE_EXCEEDED), and a queue consumer has no status at all — there the category decides ack, nack-with-retry, or dead-letter (Dead-Letter Queues).
DATABASE-SPECIFICWhich engine error means "conflict" differs: Postgres raises SQLSTATE 23505 for a unique violation and 40001 for serialization failure; MySQL raises 1062 and 1213. A generic ORM exception flattens both into one class, which is where category information is usually lost.

Where the depth lives

This domain teaches the application-side mechanism and hands the rest off.

API Designerror-model error-taxonomy status-codes validation-errors retryability partial-failure

Architecturereliability-patterns

Securityerror-handling fail-open-vs-fail-closed

Performancegolden-signals red-method alerting

Domains that do not exist yet

— Testing & Reliability Engineering — asserting that each category is produced under the right conditions is a test suite in its own right, and the one most services never write.