Unknown, Question, Experiment
The full detour from §14: Unknown → Specific Question → Research → Tiny Experiment → Understanding → Return to the main problem. The experiment is the step people skip, and it is the only step that produces knowledge rather than familiarity.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
You have a specific question about something you have never done. What is the smallest thing you could build that would answer it — and how do you stop it becoming a project?
The question is written: "by what path does my backend learn that a payment succeeded, and can that path deliver the same confirmation twice?" You have read the provider's page on webhooks. It describes the happy path in detail and says retries "may occur". You feel you understand it, and you also notice that you could not say what your handler will actually receive.
Read more. The documentation has a reference section, an events guide and a best-practices page; a thread on the forum discusses retries. Every page adds a little confidence, and starting the real integration on that confidence feels efficient.
Reading produces a model of what the provider says it does, not of what your handler sees. The first time a confirmation arrives before the redirect returns, or twice, the model is wrong in a way no page warned about.
- Reading produces a model of what the provider says it does, not of what your handler sees. The first time a confirmation arrives before the redirect returns, or twice, the model is wrong in a way no page warned about.
- Without an experiment there is no boundary between "researching payments" and "building payments", so the reading turns into the integration and the integration inherits every guess the reading made.
- Confidence rises with pages read, and it is not connected to anything. The engineer who has read the most is the most surprised in production.
- The main problem — checkout — waits the whole time. The detour has no end, so the return never happens.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Treat the specific question as a hypothesis and design the smallest thing that could refute it. "The provider confirms by webhook, once" is a hypothesis; a handler that logs every call it receives, plus one test-mode payment, either confirms it or shows the retry and the ordering you did not expect.
- Make the experiment tiny in a specific sense: it has no store around it. No cart, no order table, no framework you have not already used. The experiment answers one question, and everything it does not need is a way for it to fail for an unrelated reason.
- Predict the result before running it — what you expect to be logged, in what order, how many times — and write the prediction down. The distance between prediction and log is the understanding; if you do not predict, every result looks like confirmation.
- Return. The experiment produced an answer; write it in the known column, delete or archive the experiment, and go back to the line of checkout that was waiting. The experiment is not the first draft of the integration (Prototype vs Production).
The detour, stage by stage
The six stages of §14 as a pipeline. The lab at /thinking/unknowns follows the same path; the point here is the pair of stages in the middle — research and experiment — and the fact that only the second produces something that goes in the known column.
- 1Unknown
"I don't understand how the backend finds out the payment worked."
fails by Staying a feeling.
- 2Specific question
"By what path does my backend learn a payment succeeded, and can that path deliver twice?"
fails by Stopping at the topic.
- 3Research
The webhook page and the retry paragraph. Output: a hypothesis, not a fact.
fails by Reading until confident and calling that the answer.
- 4Tiny experiment
A logging route, one test payment, one resend.
fails by Growing into the integration.
- 5Understanding
Arrival order is not guaranteed; the event id deduplicates. Written on the board.
fails by Not being written down, so it is re-derived next month.
- 6Return
Checkout marks the order paid from the webhook, keyed on event id. Experiment deleted.
fails by The experiment becoming the code.
What the experiment looks like
The experiment in full. It is deliberately not a checkout: no order, no database, no framework beyond a route. The prediction is written above the code, because a prediction written after the run is a description.
1// experiment/webhook-probe.ts — delete after the question is answered2const seen: { at: number; id: string; type: string }[] = []3 4app.post('/probe/webhook', (req, res) => {5 const evt = req.body // no verification yet: that is a separate question6 seen.push({ at: Date.now(), id: evt.id, type: evt.type })7 console.log('webhook', evt.id, evt.type, 'count so far', seen.length)8 res.status(200).end() // what does the provider do if this is slow? separate question9})10 11app.get('/probe/return', (_req, res) => {12 console.log('redirect returned at', Date.now(), 'webhooks seen', seen.length)13 res.send('back')14})15// then: create one test-mode payment pointing at /probe/return, complete it,16// press "resend event" in the dashboard, and read the two logs side by side.Two lines carry the words "separate question". That is the stopping rule in code: the experiment names what it is not testing so that the temptation to test it is visible.
The experiment as a slice of nothing
It helps to see the experiment through the vertical-slice lens: it touches the layers the question touches and no others. That is also what it cannot tell you, and the honest list of what it does not prove is the list of the next unknowns.
- ProviderSends the confirmation event on completion and on manual resend.
- RouteLogs id, type and time; returns 200.
- LogShows count, order relative to the redirect, and whether the resend shares an id.
How to do it
Most important first.
- Write the question and, under it, the hypothesis you currently hold. If you have none, the research step is not finished.
- Design the experiment as the minimum that could show the hypothesis wrong. Strip anything that is not needed to observe the answer.
- Write the prediction: what will be observed, in what order, how many times. Then run it (Prediction Before Execution).
- Compare. Where the observation differs from the prediction is what you learned; record it as a fact on the board, with the date and the provider's version if it matters.
- Set a stopping rule before starting — an afternoon, a fixed list of observations — and stop at it even if the experiment is interesting (Spikes).
Worked on a concrete problem
The move has to produce something. This is what it produced.
- Question: by what path does the backend learn a payment succeeded, and can it arrive twice? Hypothesis after reading: webhook, once, after the customer is redirected back. Experiment: one route that logs the request body and timestamp; one script that creates a test-mode payment; the provider's dashboard button that resends an event. Prediction: one call, after the redirect. Observed: the webhook arrived before the redirect completed, and the resend produced an identical event id. Two facts for the known column: order of arrival is not guaranteed; the event id is the deduplication key.
- Question from the inventory list: what does the database do with two concurrent decrements of one row? Experiment: one table, one row with a quantity of one, two connections, both run "decrement if positive". Prediction: one succeeds, one fails. Observed without a constraint: both succeed and the quantity is negative. Observed with a check constraint: one fails with a constraint error. The fact: the enforcement point has to be the database or a lock, not application code that reads then writes. The experiment took less time than reading about isolation levels would have, and it produced a sentence the isolation-level reading could then explain (Invariants Under Concurrency).
- Return: the checkout handler now marks the order paid on the webhook, keyed on the event id, and ignores the redirect for anything except rendering. The experiment's route was deleted. The main problem moved one line forward, and that line is correct for a reason the engineer can state.
How you know it worked
What now exists that did not before, and what question you can now ask.
- A fact is on the board that no page told you, and you can say which observation produced it.
- The experiment fit in one file and had no dependency on the rest of the store.
- Your prediction was wrong somewhere, and you can name where. A prediction that was entirely right on a thing you had never done should make you suspicious of the prediction.
- The experiment is gone and checkout has moved forward.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?What is the hypothesis I currently hold, and what is the smallest thing that could show it wrong?
- ?What do I predict will be observed — and how many times, in what order?
- ?What does this experiment need, and what have I added that it does not?
- ?What fact goes on the board now, and which line of the main problem does it unblock?
What can go wrong
- The experiment grows. The logging route gains a database write, then an order lookup, then error handling, and by evening it is the integration — with every unexamined assumption from the reading built in. The stopping rule exists for this.
- The experiment answers a different question from the one asked. Testing that a payment can be created answers "can I call the API?", not "how does my backend learn the outcome?". Check the observation against the question before calling it done.
- No prediction is written, so the result is interpreted as confirmation of whatever the reading suggested. The log showed two events; the engineer "knew retries could happen"; nothing was learned about ordering.
- The experiment is skipped because the answer was found in the documentation. The documentation was right about the happy path and silent about the arrival order; only the run showed it.
- An experiment costs an afternoon that reading could have skipped — and on a question where the documentation is complete and exact, reading was enough. The afternoon is insurance against the questions where it was not, and you cannot always tell in advance.
- A tiny experiment proves the tiny case. The webhook that arrived once in test mode says nothing about a provider outage that replays a day of events; that is a separate question for Failure Modeling.
- Throwing the experiment away feels wasteful when it works. Keeping it as the start of the integration is how experiments become production code that nobody designed.
- "So documentation is not to be trusted." It is to be read first and tested second. The documentation told you what to look for; the experiment told you what your system will see. Both were needed.
- "An experiment is a prototype of the feature." A prototype of checkout would need a cart. The experiment needed a route and a script. The difference is the question it answers (A Prototype Answers a Question).
- "I should experiment with everything." Only with what you cannot predict. The order table needs no experiment; you have built tables. The experiment is for the column marked unknown.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALHypothesis, minimal refutation, prediction, observation, return — the loop is the same whether the unknown is a webhook, a lock, a compiler flag or a rendering behaviour.
- STAGE-SPECIFICOn greenfield work the experiment lives outside the codebase; in an existing system the experiment is often a script against a staging environment or a test that exercises one existing path, and "no store around it" becomes "no feature around it".
- ILLUSTRATIVEThe webhook that arrives before the redirect, the resend button and the negative stock quantity are invented to show the shape of an experiment; a real provider's behaviour must be observed, not inherited from this lesson.
Where the depth lives
This domain asks the question and hands the answer off by name.