Failure Path Second
With the happy path working, each discovered failure becomes an injection: break the provider, click twice, kill the process, sell the last unit — against real code — and watch what happens before deciding what should. Failure handling designed from observation, not from imagination.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
The happy path works and the failure list is long. How do you turn "handle payment failure" into specific, tested behaviour — and how do you decide what each failure should do?
Checkout creates orders. The list beside you says: declined card, provider timeout, provider down, double click, crash after charge, out of stock, refresh mid-payment. You open the checkout function to "add error handling" and realise you do not actually know what the provider sends on a timeout, or what your own code does when it dies between two writes.
Wrap everything in try/catch, add a retry loop around the provider call, and show a generic "something went wrong" message. It covers every failure on the list in one afternoon and the code looks robust.
The generic handler treats a declined card, a timeout and a bug in our own code identically, so the customer whose card was declined sees "something went wrong" and tries again — and the one whose charge actually succeeded before the timeout tries again too, and is charged twice.
- The generic handler treats a declined card, a timeout and a bug in our own code identically, so the customer whose card was declined sees "something went wrong" and tries again — and the one whose charge actually succeeded before the timeout tries again too, and is charged twice.
- The retry loop retries the one thing that must not be retried blindly — a charge — because "retry on error" was applied to the list rather than to each failure.
- Nothing was observed. The handler handles the failures the author imagined; the provider's real timeout response, the real state after a crash, the real race on the last unit, all remain unknown, and the handler is tested against mocks that return what the author expected.
- The failure list gets ticked off. Seven rows, seven catches, done — and the reconciliation gap, where a payment exists with no order, is not on the list because nobody made it happen.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Take each discovered failure and make it happen, for real, against the working happy path — before writing any handling. Use the provider's test mode to decline, to time out, to return errors; click twice with the network tab open; kill the process between the charge and the insert; put the last unit in two carts. Write down what actually happened: what the provider sent, what state the order was in, what the customer saw. This is failure injection, and it replaces imagination with evidence.
- For each observed failure, decide the required outcome as a three-part requirement — what state should exist afterwards, what should the customer see, what should be possible next — and only then write the handling. The handling is now a transition from an observed state to a required one, not a guess about an error you have never seen.
- Distinguish the failure *categories* before handling any of them, because they need different responses: the provider said no (declined — do not retry, tell the customer why); the provider did not answer (timeout — the charge may or may not exist, so query before doing anything); we died (crash — the provider's callback or a reconciliation pass must be able to finish the work); the world changed (out of stock — check before the irreversible step). A single handler cannot serve four categories with different truths.
- Make every failure path observable in production the way it was in the injection: a log line with the checkout id and the category, a status on the order, a metric. A failure that was handled but cannot be seen will be handled wrongly for months before anyone knows.
What was observed, and what was required
The table is the failure pass as a record: each row is an injection, what actually happened, why, and the requirement it produced. The cause column is the part imagination gets wrong — the timeout row's cause is "the charge succeeded and we did not know", which is not what most people picture when they write a retry loop.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Provider declines the card (test mode) | Exception; no order; customer sees a raw error | Provider said no — a definite answer, nothing to retry | No order, cart intact, the decline reason shown, retry with another card allowed |
| Provider times out (forced) | Exception after the wait; no order; the charge exists in the provider's dashboard | Provider did not answer — the charge may have happened | Query the provider by checkout id before anything else; the confirmation callback can create the order; never a second charge |
| Pay clicked twice | Two charges, two orders | Two requests with no shared identity | Checkout id as idempotency key; the second request returns the first result |
| Process killed between charge and insert | Charge exists, no order, no trace | We died between two steps that are not atomic and cannot be | The callback recreates the order from the checkout record, which must carry everything the order needs |
| Last unit in two carts | Stock negative, two confirmations | The world changed between the cart and the charge; the check was not before the irreversible step | Reserve stock in a transaction before the charge; release the reservation if the charge fails |
The same failure, asked three ways
The quality of the failure requirement follows the quality of the question. "What if payment fails?" is where discovery started; here it has to become something a test can check, and the ladder shows the climb.
why The best form separates the four things the handler has to get right — the bound, the resolver, the two possible end states, and the guard against a duplicate — and makes each one testable. The vague form hides that a timeout is not a failure but an unknown, which is the fact the whole policy hangs on.
Payment as states, not as a call
Once the failures have been observed, the checkout is no longer a function call that succeeds or throws; it is a small state machine, and every observed failure is a transition on it. The diagram is what the failure pass produced — the "unknown" state in the middle is the one the try/catch reflex could not represent.
How to do it
Most important first.
- For each failure on the list, write the injection first: how will you make it happen against real code? If you cannot make it happen, you cannot test the handling (Failure Injection).
- Run each injection against the happy path with no handling and record the observed state: provider response, order row, stock, what the page showed. Keep the recordings; they are the test fixtures.
- Group the observations into categories — said no, did not answer, we died, world changed — and decide a policy per category before touching individual failures.
- Write each handling as a requirement with an observable outcome, write the test from the recording, then write the branch (What If Payment Fails?, Duplicate Requests).
- After each branch, re-run the injection and check that the log, the status and the customer message all say the same thing about what happened.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- Injection: force a provider timeout in test mode. Observed: our call throws after the configured wait; the provider's dashboard shows the charge *succeeded*; our order was never created; the customer saw a spinner and then an error. Category: did not answer. Policy: never assume a timed-out charge failed; query the provider by our checkout id before doing anything, and let the provider's confirmation callback create the order if we missed it. Requirement: "after a provider timeout, the customer's state is resolved within a bounded time to either a paid order or a clear failure, and no second charge is made." The handling is written against the recording.
- Injection: double click on Pay. Observed, with the network tab open: two POSTs, two charges in the provider's dashboard, two orders. Category: repeated request. Policy: the checkout id created before the button rendered is the idempotency key; a second request for the same id returns the first result. Requirement written; test written from the two recorded requests; branch written; injection re-run: one charge, one order, the second response identical to the first.
- Injection: kill the process between the charge and the order insert. Observed: charge exists, no order, no log line, nothing to reconcile from. Category: we died. Policy: the provider's confirmation callback must be able to create the order on its own, which means the checkout record must carry everything the order needs. That changed the checkout table — a structural discovery that Happy Path First warned about, caught here because the injection was real.
- Injection: the last unit in two carts, both check out. Observed: both charges succeed, stock goes negative, both customers get confirmations. Category: world changed. Policy: check and reserve stock before the charge inside a transaction, and treat "reserved but charge failed" as its own release path. Two requirements, not one, and the second was not on the original list.
How you know it worked
What now exists that did not before, and what question you can now ask.
- Every failure on the list has a recording of what actually happened before handling existed, and the tests are written from the recordings.
- Failures are handled by category with a stated policy per category, and no single handler covers more than one category.
- The injection found at least one failure that was not on the list — a reservation that needs releasing, a callback that needs the checkout to carry more data.
- Each failure path produces a log line and a status that agree with what the customer was told.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?How would I make this failure actually happen against the working code — and what happened when I did?
- ?Which category is this: the external system said no, did not answer, we died, or the world changed — and what is the policy for that category?
- ?What state must exist afterwards, what does the customer see, and what can they do next?
- ?If this failure happens in production tonight, what log line or status would tell me it did?
What can go wrong
- Injection becomes theatre. The provider is mocked to return the error the author expected, which is the imagination the move was meant to replace. Inject against the real test mode, the real process, the real database.
- Every failure gets bespoke handling and no policy. Twenty branches, each subtly different in how it logs and what it tells the customer, because the categories were never named.
- The failure pass never ends. Each injection reveals another, and the store launches never. The list is sorted by cost, the V1 line is drawn, and the failures below it are deferred with a reason — Requirement Discovery covers the sorting.
- Handling is added but not observable. The timeout policy is correct and silent, and when the provider's behaviour changes nobody notices for a quarter.
- Injecting every failure for real is slower than wrapping everything in one handler; on failures you have seen many times before, the recording adds little.
- Per-category policies are more code than a generic handler and more to keep consistent; the gain is that a decline and a timeout are no longer treated as the same thing.
- Observing real failure states sometimes reveals that the happy path's shape was wrong — which is cheaper now than later, and still a rebuild of something that worked.
- "Failure path second means failure handling is lower priority." It means it is built second. Everything above the V1 line ships; the sequence exists so that the handling is built against evidence rather than imagination.
- "A retry loop is failure handling." A retry is one policy for one category — did not answer, and only when the operation is safe to repeat. Applied to a charge without an idempotency key, it is the double-charge bug with extra steps.
- "Injection is a testing technique, so it comes after the code." Here it comes before: the injection is how the requirement gets its observable outcome, and the test is written from what was observed.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALInject before handling, categorise before writing branches, make every failure path observable — this holds for any system with an external boundary or persistent state; for a pure library it reduces to constructing the invalid inputs before writing the validation.
- DOMAIN-SPECIFICThe four categories are the same everywhere, but the policies differ by domain: a store may retry a stock check and must not blindly retry a charge; a chat app may retry a send with a client id and must not retry a "delete for everyone"; a file upload may retry a chunk and must not retry a "finalise".
- ILLUSTRATIVEThe injections, the observed states and the provider's dashboard behaviour are invented to show the shape of the move; a real provider's test mode has its own ways of failing.
Where the depth lives
This domain asks the question and hands the answer off by name.