AI as Debugging Partner
Before the assistant can help with a bug, you must bring four things: expected behaviour, actual behaviour, evidence, and a hypothesis. Bring fewer and it guesses; bring all four and it investigates with you.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
Something is broken and you want the assistant's help. What do you have to bring to the conversation for its help to be investigation rather than guessing, and how do you know the difference?
The store's checkout sometimes creates two orders from one click. You paste the handler into the assistant with "why does this create duplicate orders?" It offers five possible causes and a rewritten handler. You do not know which cause is yours, and the rewrite changes things you did not know were involved.
Paste the code and the symptom and ask what is wrong. The assistant has seen more code than you ever will; it will recognise the pattern. And it does produce an answer — several, in fact — which feels like the investigation has started.
Five causes is not a diagnosis; it is the list you had before asking, now with more confidence attached. Nothing you sent could distinguish between them, so nothing in the reply does.
- Five causes is not a diagnosis; it is the list you had before asking, now with more confidence attached. Nothing you sent could distinguish between them, so nothing in the reply does.
- The rewrite fixes all five causes at once, which means you will never know which one it was. The bug goes away, or seems to, and the next duplicate-order report arrives with no evidence trail (Do Not Randomly Change Things).
- The assistant filled in what you did not provide. It assumed a database, a framework and a request path, and its causes are about those; the actual cause — a retry in a layer you did not paste — is not on its list because it could not be.
- You have learned nothing about your system. The bug was a chance to find out how a click becomes an order in your store; the paste-and-rewrite turned it into a chance to find out that the assistant will rewrite handlers.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Bring four things, and do not ask until you have them: what you expected (one click, one order), what actually happened (one click, two orders, with timestamps), the evidence you have gathered (logs, requests, rows), and a hypothesis — your current best guess at the cause, however weak. The four together are a reproducible problem statement; any three are a symptom.
- The hypothesis is the one people skip, and it is the one that turns the assistant from a guesser into a partner. "I think the browser retries the request on a slow response, because both orders are within a second and the client has a timeout" gives the assistant something to argue with, confirm, or design an experiment against. Without it, the assistant supplies the hypothesis, and then you are debugging its theory instead of your system.
- Ask for the next experiment, not the fix. "Given this evidence and this hypothesis, what single observation would confirm or rule it out?" produces a probe — a log line, a header check, a query — that you run and report back. The assistant's breadth is in knowing which probes distinguish which causes; the running is yours (Predict Before You Look).
- Keep the evidence trail in your hands. Every probe you run, every result, goes into your notes before it goes into the conversation. When the bug is found, you can say how, and the next similar bug starts from the trail rather than from a paste (Logs Are Evidence, Not Thinking).
What you bring, as an unknowns board
The four items are, in this domain's terms, a board: what is known, what is assumed, and what is unknown — with each unknown turned into a question and a probe. The board below is the duplicate-order bug after evidence-gathering and before the first message to the assistant. Everything on it was produced without the tool; the tool's job starts at the probes.
- ✓Expected: one click produces one order.
- ✓Actual: two orders, same cart, same customer, created within a second; the customer reports one click.
- ✓Evidence: two POST requests with different request ids in the server log; one long request and one retry in the browser network tab; no idempotency key on either.
- ~The customer really clicked once. Plausible from the timing, unverified.
- ~The server handler is deterministic — no queue between the request and the order row. To be checked; a redelivery would look similar.
? Something retries.
becomes Does the client retry a POST on timeout, and if so does the retry carry anything the server could use to recognise it?
experiment Read the client's fetch wrapper for a retry policy; log the request body and headers of both requests and compare.
? The handler might race.
becomes If the same request arrives twice within the handler's execution time, does anything prevent two inserts?
experiment Fire the same POST twice concurrently against a test database with one cart; count the rows.
? Maybe it is the queue.
becomes Is there any asynchronous hop between the POST and the order insert that could redeliver?
experiment Trace one request end to end in a test environment and list every hop; if there is no queue, this unknown closes.
The hypothesis — the client retries and the server cannot tell — is the first unknown. Sending this board to the assistant gets probes ranked by what they distinguish; sending the handler alone gets five causes and a rewrite.
The investigation as a loop with a partner
Once the board exists, the conversation is a loop: you supply evidence and a hypothesis, the partner proposes the probe that best splits the candidates, you run it, and the result becomes new evidence. The pipeline shows who does what at each step; the assistant never runs a probe and never chooses the fix.
- 1Expected vs actual
You: two concrete statements that disagree.
fails by "Duplicates sometimes."
- 2Evidence
You: logs, requests, rows — what has been observed, not inferred.
fails by The handler source pasted as if it were evidence.
- 3Hypothesis
You: the best current guess and what made you guess it.
fails by Left to the assistant.
- 4Probe
Partner: the single observation that splits the hypothesis from its alternatives.
fails by A rewrite that addresses every alternative at once.
- 5Run and report
You: the probe's result, into your notes, then into the conversation.
fails by Continuing as if the result were known.
- 6Narrow or fix
Both: one cause consistent with everything → the smallest fix for that cause.
fails by Three fixes for three candidates.
The loop is the debugging lab at /thinking/debug with a partner in the probe-choosing seat. The partner makes the probes better; it does not make the evidence optional.
Three items against four
The comparison is the same bug sent two ways. The only difference is the hypothesis, and it changes the reply from a list to an investigation.
"One click makes two orders; here are the two requests and the handler. What causes this?" Reply: five candidate causes and a handler rewritten to address all of them.
"…I think the client retries on timeout and the server has no way to recognise the repeat, because the requests are a second apart with different ids. What would confirm that, and what else would look the same?" Reply: the retry policy to check, the cart-id probe, and the one alternative — a queue redelivery — that the same evidence would also fit.
A hypothesis gives the partner something to test rather than something to replace. The reply now contains a probe you can run and an alternative you can rule out, and the eventual fix is the one for the cause that survived — which is a diagnosis you can explain at the next incident (The Minimal Reproduction).
How to do it
Most important first.
- Write the four items as four labelled lines before opening the assistant. If "evidence" is empty, the next step is gathering some, not asking (Reproduce It First).
- Make the expected and actual behaviour concrete enough to disagree: not "duplicates sometimes" but "order ids 4411 and 4412, same cart, same customer, created within a second".
- State the hypothesis even when you are unsure; "I do not know" is not a hypothesis, but "the retry layer is my best guess because of the timing" is. Say what evidence made you guess it.
- Ask for the probe that would split your hypothesis from the alternatives, run it, and paste the result. Repeat until one cause is consistent with everything (Binary Search Over the System).
- Only when the cause is known, ask about the fix — and then ask for the smallest change that addresses that cause, so the fix carries the diagnosis with it.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The duplicate orders, brought properly. Expected: one click, one order. Actual: two orders, same cart, created within a second; the customer clicked once. Evidence: two POST requests in the server log with different request ids; the browser's network tab shows one request that took long and one retry; no idempotency key in either. Hypothesis: the client retries on timeout and the server has no way to recognise a repeat. The assistant's reply is now an investigation: confirm the client's retry policy, check whether the retry carries anything identifying, and design the probe — log the cart id on both requests and see whether they match. They do. The cause is known; the fix is an idempotency key, and it is small because the diagnosis was precise.
- The same bug with three items and no hypothesis. Expected, actual and evidence sent; "what causes this?" asked. The reply lists client retry, double submit, a race in the handler, a queue redelivery and a database without a unique constraint — all reasonable, none distinguished. The rewrite adds a constraint, a debounce and a lock. The bug stops. Which of the three fixed it is unknowable, and two of them are now permanent complexity with no diagnosis behind them (The Complexity Ledger).
- The chat app: messages arrive out of order. Expected: order of send. Actual: message B shown before A on one device only. Evidence: server timestamps show A before B; the affected device reconnected between them. Hypothesis: on reconnect the client fetches recent messages and appends them without sorting. The assistant confirms the hypothesis is consistent, suggests the probe — log the fetch response order on reconnect — and the probe confirms it. The fix is in the client's merge, and nothing in the server was touched.
How you know it worked
What now exists that did not before, and what question you can now ask.
- Every debugging conversation starts with four labelled lines, and the fourth is a hypothesis you wrote before the assistant offered one.
- The assistant's replies contain probes to run rather than rewrites to apply, and each probe result narrows the list.
- The fix, when it comes, addresses one named cause, and you can say what evidence pointed to it.
- Your notes could reconstruct the investigation without the conversation.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?What did I expect, what actually happened, and are both concrete enough to disagree with each other?
- ?What evidence do I actually have, and what is my best guess at the cause — and what made me guess it?
- ?What single observation would split my hypothesis from the alternatives?
- ?When the fix is applied, can I say which cause it addresses and what evidence pointed there?
What can go wrong
- Evidence-gathering never ends. The four items are a starting bar, not a complete case file; one log line and one hypothesis is enough to ask for the first probe.
- The hypothesis is defended instead of tested. A hypothesis exists to be split from its alternatives; if the probe rules it out, that is the method working, and the next hypothesis should come from the evidence, not from loyalty.
- The probe is asked for and then not run. The assistant proposed a log line; the conversation continued as if the result were known. Run it.
- The move is used on a bug whose cause is obvious. A stack trace pointing at a null dereference does not need four items and a partner.
- Gathering four items before asking is slower than pasting the handler, and on a bug the assistant would have recognised instantly it is slower for nothing.
- Asking for probes instead of fixes means running things, and running things on a production store has costs the paste does not.
- Writing a hypothesis you are unsure of is uncomfortable, and being told it is wrong in the first reply more so. The alternative is debugging someone else's hypothesis.
- "The assistant cannot debug without all four." It will produce output with fewer; the output is guesses. The four items are what make the output investigation, not what make the assistant respond.
- "The hypothesis has to be right." It has to exist. A wrong hypothesis with a probe that rules it out is progress; no hypothesis with five candidates is a list.
- "This replaces the debugging loop." It is the debugging loop — symptom, evidence, hypothesis, experiment — with a partner who knows more probes than you. The loop is in Debugging Is Problem Solving; this lesson is about what to bring to it.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALExpected, actual, evidence, hypothesis is what any competent human debugging partner would also ask for; the assistant differs only in being willing to proceed without them, which is why the discipline has to be yours.
- DOMAIN-SPECIFICIn a system with real consequences — payments, inventory — probes must be run against a copy or in test mode, and the "evidence" item includes what you are allowed to observe. On an internal dashboard the probe can be a query on production.
- ILLUSTRATIVEThe duplicate orders, the order ids, the timing and the chat reconnect are invented to show the shape of the investigation; no real incident is described.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — The manifesto's "Debug without AI" page at /manifesto/without-ai is the same loop run solo; run it that way first on small bugs so that the four items come naturally when the partner is present.