Error Boundaries
Every failure has a point where it stops being handled locally and becomes somebody else's problem. Choosing that point deliberately is a design decision; discovering it in production is not.
The requirement, the obvious build, and why it breaks
Every lesson starts where the work starts: someone asked for something, and the first implementation that comes to mind survives until the requirement changes.
Where should a failure stop travelling, and who owns it once it crosses that line?
An incident review finds the same failure reported four different ways: a retry inside the repository, a fallback inside the service, a catch in the controller, and a generic 500 from middleware. Nobody can say which of the four actually ran, and the request id changes at two of them.
Handle failures wherever you notice them. The repository knows about connection resets, so it retries; the service knows about declines, so it falls back; the controller knows about HTTP, so it maps. Each layer handles what it understands, which sounds like good encapsulation.
The moment two layers both handle the same failure, the behaviour is the composition of both and nobody designed the composition. The repository's three retries inside the service's two produce six attempts and a timeout budget nobody computed (Retries Are a Property of the Operation).
- The moment two layers both handle the same failure, the behaviour is the composition of both and nobody designed the composition. The repository's three retries inside the service's two produce six attempts and a timeout budget nobody computed (Retries Are a Property of the Operation).
- When a new requirement says "surface provider outages to the status page", there is no single place to add it, because the failure has already been converted to something else at two of the four handlers.
- Adding the queue consumer reveals that half the handling assumed an HTTP response existed. The failure paths were written against one transport and there was never a name for the layer that assumption lived in (Dependency Direction).
- Debugging degrades over time rather than improving: each new handler adds a place where the original exception is replaced by a summary, and the trace that would identify the root cause is three wrapper layers deep.
What limits the solution, and what must never stop being true
This domain leads with these two. A design that ignores its constraints is not a design, and an invariant nobody named is one nothing is protecting.
- The four handlers were each added by a different person solving a real problem, so removing any one of them breaks something that currently works (Characterization Tests).
- The system is one deployable with an HTTP edge, a queue consumer and a nightly job — three different outermost boundaries with three different correct behaviours.
- Support has runbooks keyed to the current messages, so the boundary can move but the observable output has to move with it deliberately (Designing the Migration).
- Exactly one place decides the final disposition of a failure for a given operation. Handling can happen earlier; *deciding* happens once.
- A failure never crosses a boundary having lost its category or its correlation id (Stable Identifiers).
- No boundary reports success for work that did not happen. Degrading is allowed; lying is not.
Who owns what, and where the seams fall
Responsibilities decide boundaries; boundaries decide what an interface has to say.
- Inner layers own *recognising* failures and attaching evidence — which dependency, which field, which id — and nothing else.
- One designated boundary per operation owns the disposition: retry, degrade, fail, or escalate (Error Modeling).
- The outermost boundary of each entry point owns the last-resort behaviour: an HTTP status, a nack-and-dead-letter, a non-zero exit (The Error Boundary is the backend mechanism).
- Nobody below the designated boundary is allowed to decide "this is fine", because that decision is invisible from above by construction.
- The right boundary is the innermost place that has enough context to choose between the possible responses. Below it, the code knows what failed but not what it means; above it, the code knows what it means but not what failed.
- For an HTTP request that is usually the application service that owns the use case, not the controller and not the repository.
- For a queue consumer it is the message handler, because that is where "retry, dead-letter or drop" is decidable (A Dead-Letter Queue Is a Workflow, Not a Bin is the backend mechanism).
- Process boundaries are hard boundaries whether you designed them or not: past them a failure is a crash, and everything about it that was not serialized is gone (What Changes at the Network Boundary).
Recognition goes down, decision goes up
The useful mental image is two opposite flows through the same layers. Evidence accumulates on the way up — which dependency, which field, how long we waited — while the authority to decide what to do about it sits at exactly one level and is exercised once.
When those two flows are confused, layers make decisions with partial information. A repository knows a connection was reset; it does not know whether this operation is idempotent, whether the caller has a deadline, or whether a stale cached answer would have been acceptable. It is not in a position to retry, and it retries anyway.
- Below the boundary: recognise, categorise, attach evidence, return. No policy.
- At the boundary: choose, apply the attempt budget, log once with the correlation id (Logging at Boundaries).
- Above the boundary: translate the outcome into whatever this transport says (Boundary Adapters).
- Three transports, one failure design — that is the change this makes cheap (Change Amplification).
The lifecycle of one failure
A failure has states, and the transitions between them are where the design lives. Writing it out makes visible the transitions teams accidentally allow — most importantly, the ones that go backwards into "handled" from a state that has already been reported.
The forbidden list is the useful half. Each entry is a real production behaviour someone shipped, and each one is invisible in a code review that looks at one layer at a time.
| From | On | To | Guard | Effect |
|---|---|---|---|---|
| Occurred | recognition in the adapter or repository | Categorised | — | attach kind and evidence |
| Categorised | return or throw through intermediate layers | AtBoundary | no intermediate layer applies policy | cause chain preserved |
| AtBoundary | kind is retryable | Retrying | operation is idempotent AND budget remains | attempt count incremented once, centrally |
| Retrying | the retry also failed | AtBoundary | — | budget decremented |
| AtBoundary | a fallback exists and is acceptable for this use case | Degraded | — | — |
| AtBoundary | no recovery available | Reported | — | log once with correlation id |
| AtBoundary | kind is InternalBug | Escalated | — | — |
- Occurred → Retrying — A retry below the boundary multiplies with the boundary's own budget: three inside two is six attempts and a timeout nobody computed. It is also invisible to the deadline the caller set (A Deadline Is Divided Across the Call Chain, Not Repeated at Every Hop).
- Occurred → Reported — Reporting before categorisation is how a failure loses its kind and reaches support as a message string, which is exactly the state the incident review was complaining about.
- Categorised → Degraded — A layer that substitutes a stale value without the boundary knowing turns a visible failure into a silent wrong answer — the worst outcome available, because monitoring shows success.
- Escalated → Degraded — Absorbing a defect as a degraded result is how a null dereference becomes a slightly lower conversion rate for six months. Bugs must not have a recovery path (Swallowed Errors).
- Reported → Retrying — Retrying after the caller has been told it failed produces duplicate work with no one watching — the classic double-charge, and the reason idempotency is decided before this state machine runs (Idempotency by Design).
The two most valuable properties fall out of the forbidden list rather than the transitions: exactly one place increments the attempt count, and no state below AtBoundary may reach a terminal state.
What the boundary actually owns
It is worth being concrete about the unit, because "the boundary" sounds architectural and it is in fact a function you can open. The test of whether it is drawn correctly is the changesWhen list: if it changes for transport reasons, it is too high; if it changes for driver reasons, it is too low.
- — Which failure kinds this operation can produce
- — Whether the operation is idempotent
- — The attempt and time budget it was given
- — Which degradations are acceptable to the business
- — The correlation id for this unit of work
- — Calls the domain and the adapters
- — Chooses retry, degrade, fail or escalate
- — Increments the single attempt counter
- — Logs the disposition once, with evidence
- — Returns an outcome value, never a transport concept
- — The failure taxonomy
- — The retry and degradation policy
- — The domain
- — The adapters it calls
- — A new failure kind is added to the taxonomy
- — The business changes what degradation is acceptable
- — The retry policy for this operation changes
Three reasons to change, and all three are about failure policy for this one operation — which is the coherent responsibility we were aiming for. The warning signs to watch for are a fourth reason appearing: if it starts changing when a status code changes, transport has leaked down; if it changes when the payment SDK upgrades, the adapter is not doing its job (Anti-Corruption Layer).
How to build it
Most important first.
- Name the boundary explicitly, in code, for each entry point. A function called
handleCheckoutthat returns an outcome is a boundary; a catch block in a controller is a habit. - Push recognition down and decision up: inner code returns or throws a categorised failure with evidence, the boundary chooses the response (An Error Taxonomy That Survives Contact).
- Allow exactly one retry policy per operation, owned at or above the boundary, so the total attempt count and the time budget are computable rather than emergent.
- Preserve the cause chain across the boundary and log once, there, with the correlation id — not at every frame on the way up (Logging at Boundaries).
- Make degradation explicit as an outcome the boundary returns, so "we served a stale price" is a value the caller can see rather than a silent substitution (Partial Failure).
- Delete the redundant handlers only after the boundary is in place and the characterization tests are green, one at a time (Incremental Migration).
What the next change costs
The field this whole domain exists for. A structure is only better if it makes the change after this one cheaper — and it is worth saying which changes it does not help.
- Adding a new response policy — "circuit-break after five provider failures" — costs one change at one boundary, and it applies to every failure of that category rather than to the sites somebody remembered.
- Adding a new entry point (a queue consumer for the same use case) costs a transport adapter and no failure logic at all, which is the change this design exists to make cheap.
- Changing which layer is the boundary is expensive and rare: it means moving decision code across a module line and re-deriving the attempt budget. Budget a week and characterization tests, not an afternoon.
- What did not get cheaper: a failure whose right response genuinely depends on transport — a timeout that should stream a partial response over HTTP and dead-letter on a queue — still needs a branch per transport, and the boundary does not remove it, it just puts it in one place.
- Concentrating disposition at one boundary makes the failure logic easy to find and easy to grow into a switch with thirty branches. The coupling did not disappear; it moved somewhere you can see it, which is better and not free.
- Pushing decisions up means inner code cannot make an obviously-correct local recovery — a cheap in-memory cache fallback now needs the boundary's permission, and sometimes that is genuinely worse.
- Logging once at the boundary is cleaner and loses the breadcrumb trail. The compensation is structured evidence on the failure itself, which is more work at every recognition site.
What can go wrong
- The boundary is declared but not enforced, so an inner retry survives the refactor and the attempt budget is still wrong — this is the most common outcome, because inner retries are invisible in a code review of the boundary.
- The boundary becomes a god handler: every operation's policy in one switch, growing a branch per feature, and the coupling that was removed from the layers reappears in one file (God Object).
- Context is lost at the boundary because the failure was translated to a summary type without a cause chain, and the incident that motivated the whole exercise is no worse but no better.
- The mitigation fails on its own: a single logging point at the boundary loses the intermediate state that made a failure diagnosable, and the team re-adds inner logging until it is back where it started (Debuggability by Design).
- The boundary depends on the taxonomy and on the policy configuration; nothing depends on the boundary except the transport adapter above it.
- Inner layers gain a dependency on the taxonomy and lose their dependencies on retry libraries, HTTP status constants and logging configuration — which is usually a net reduction in the surface of the deepest code.
- The transport adapter depends on the boundary's outcome type, which is what lets the same use case serve HTTP, a queue and a job without three copies of the failure logic (Designing a Module Interface).
- "One boundary means one try/catch in main." No. There is one boundary *per operation*, and a system with an HTTP edge, a consumer and a job has at least three, with different last-resort behaviours (The Error Boundary is the request-path version).
- "Inner layers should not handle errors." They should *recognise* them and attach evidence, which is real work. What they should not do is decide the disposition, because they cannot see the alternatives.
- "This is the same as the backend error-boundary lesson." Related and not identical: Backend Engineering owns the request path and the middleware that terminates it. This one is about where in *any* call graph a failure stops being local — the same question for a nightly job with no request at all.
- "Put the boundary at the controller." Tempting, and it makes the use case untestable without HTTP and unusable from a queue. The boundary belongs at the use case; the controller is a transport adapter above it (Boundary Adapters).
- shotgun-surgery
- swallowed-errors
Testing it, and how it ages
- Test the boundary directly with each failure category injected: assert the disposition, the attempt count and the correlation id, with no transport involved (What a Unit Is).
- Test that inner layers do *not* handle: inject a dependency failure into the repository and assert it arrives at the boundary uncategorised-by-the-repository and unretried.
- One integration test per entry point that the same domain failure produces the right transport-level result — status code, nack, exit code (Where a Test Must Be Real).
- Assert the attempt budget as a number in a test. It is the property most likely to regress silently when somebody adds a retry three layers down (Retries Are a Property of the Operation).
- Boundaries drift downward over time: someone adds a local retry to fix an urgent flake and it is never removed, so the budget assertion in the test suite is what keeps the design real.
- The first serious pressure is asynchrony. Once part of the operation moves to a queue, one boundary becomes two and the disposition of the second one is invisible to the caller of the first (The Async Job Pattern is the API-level shape).
- It stops being right when the deployable splits: each service gets its own boundary, and the cross-service disposition becomes a saga question rather than a code-structure one (Sagas: Trading Isolation for Availability is the distributed mechanism).
Where this applies
This domain's advice is contested more than most. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view rather than a caricature.
- GENERALEvery system with more than one layer has the question of where a failure stops travelling; the answer differs by transport but the question does not, which is why a job, a consumer and a request endpoint all need it answered separately.
- FRAMEWORK-SPECIFICFrameworks with global exception handlers, React error boundaries or middleware pipelines supply an outermost boundary for free and encourage teams to treat it as the only one. That is fine for last-resort behaviour and wrong for disposition: the framework boundary cannot know whether a provider timeout should be retried, because it has no idea what the operation was.
- SCALE-SPECIFICIn one process the boundary is a function and the cause chain is intact. Across services it becomes a protocol question — the caller sees a status code and a request id, not an exception — and disposition splits between the two sides with no shared stack (A Timeout Tells You Nothing About Whether It Happened is where that gets hard).
Where the depth lives
This domain teaches the codebase-level structure and hands the rest off.
- — Testing & Reliability Engineering — an attempt budget and a degradation policy are reliability decisions before they are code-structure decisions, and that domain owns how to choose the numbers.
- — System Design — once the operation spans services, "one boundary per operation" becomes a coordination problem rather than a function you can open.