Duplicate Requests
A payment must not happen twice, and the double click, the browser retry, the webhook redelivery and your own retry loop all make the same request twice. Idempotency is the name for the answer; the lesson is discovering the question.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
The same checkout request arrives twice. What should happen, where is it decided, and how do you know it works?
Test-mode payments work. During a demo I clicked "Pay" twice because the button did not disable fast enough, and the provider dashboard showed two charges for one cart. I disabled the button. Then I wondered what happens when the network retries for me, and realised the button was never the problem.
Disable the button after the first click. It is the visible cause, the fix takes a minute, and the demo no longer produces two charges. A spinner on the button looks like the problem being handled.
The button was one of several ways the request repeats. Browsers retry some requests on their own; mobile networks drop responses and the app resubmits; the provider redelivers its notification; your own timeout-and-retry loop, added for resilience, resubmits the charge. Disabling the button fixed the only duplicate you had seen.
- The button was one of several ways the request repeats. Browsers retry some requests on their own; mobile networks drop responses and the app resubmits; the provider redelivers its notification; your own timeout-and-retry loop, added for resilience, resubmits the charge. Disabling the button fixed the only duplicate you had seen.
- The fix lives in the layer least able to enforce it. The client can be a script, an old tab, or a second device, and none of those saw the disabled button. The invariant "one payment per checkout" is a property of the backend and the data, and the button cannot hold it.
- Nothing distinguishes the second request from a new one, so even a backend that wanted to refuse it could not. The stall is not the missing check; it is the missing identity.
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Name the invariant first: for each checkout attempt, at most one charge and at most one order. Write it as a sentence with "at most once" in it, because that phrase is what turns "the button double-fired" into a property of the system (Invariants in an Online Store, What Must Never Break).
- Enumerate the ways a request repeats, from every layer that can send it: the human, the browser, the network, your retry code, the provider's redelivery, a background job that runs twice. The list is short and it is the same for most systems; what changes is which of them can reach the operation that must not repeat.
- Give the operation an identity that is the same on every repeat and different on every new attempt, generated before the first send. For checkout that is a key created when the cart is confirmed, not when the request is built; a key that is generated per request is a new attempt every time and protects nothing.
- Decide where the check lives and what a repeat returns. The check lives where the invariant can be enforced against concurrent repeats — usually the database, through a uniqueness constraint on the key — and the repeat returns the first attempt's result, so that the client cannot tell it was a repeat. The engineering of keys, storage and expiry is the Backend domain's; the move here is to arrive there with the invariant and the enumeration in hand (Invariants Under Concurrency).
The operation, and the identity that makes a repeat recognisable
The pseudocode shows the two halves of the move: a key created before the first attempt, and a reservation of that key that happens before anything with a lasting effect. The framework's idempotency middleware, when you get to it, is this shape with storage and expiry added.
What to notice: the reservation is an insert with a unique constraint, not a lookup followed by an insert. The difference is invisible in a sequential test and decisive when two repeats arrive together.
1on review page shown:2 attemptKey = new id # created once per checkout attempt, kept by the client3 4function checkout(cart, attemptKey):5 reserved = insert attemptKey into attempts # unique constraint on attemptKey6 if not reserved:7 return stored result for attemptKey # a repeat: same answer as the first time8 order = create order (pending) for cart9 charge = provider.charge(total, key = attemptKey)10 record charge on order11 store result for attemptKey = confirmation(order)12 return that resultThe key travels with every submit of the same attempt and changes only when the customer starts a new one. If the provider accepts a key, it is the same key — one identity for the whole attempt across both systems.
The why ladder from "we need a lock"
The first idea most people reach for is a lock: hold the cart while checkout runs so a second submit waits. The ladder shows what the lock was standing in for, and that the real requirement is satisfied by something with no waiting in it.
“We need a distributed lock on the cart during checkout.”
- ↓Why a lock? So a second checkout submit cannot run while the first is running.
- ↓Why must it not run? Because it would charge the card again and create a second order.
- ↓Why would it do that? Because it cannot tell it is a repeat of an attempt already in progress or already finished.
- ↓Why can it not tell? Because nothing identifies the attempt; each request looks new.
the claim was right when The operation cannot be given an identity — for instance an external system that must be called exactly once per second across many workers — or when the work must be serialised for a different reason, such as inventory allocation across many carts.
Where the check can live
The decision is not which mechanism is best but which layer can actually hold the invariant against every source on the repeat list. Each option stops some sources and misses others, and the honest design usually has two of them: a client-side guard for the human, and a data-level constraint for everything else.
Which layer enforces at-most-once for checkout?
when Always, as a courtesy — it stops the human and improves the experience.
cost Stops nothing that does not go through this client: a retry, a second tab, a script, a redelivery.
when A single process, low stakes, or a prototype where the point is to learn the pattern.
cost Fails silently with a second instance; a read-then-write passes two concurrent repeats.
when Any operation whose repeat costs money or data — the checkout charge, the order insert, the notification handler.
cost A key table to store, replay and expire; the client has to generate and keep the key.
when The provider supports it — most payment providers do. Use it in addition to the data-level check, never instead of it.
cost Covers only the charge; the order, the stock and the email are still yours to protect.
How to do it
Most important first.
- Write the at-most-once sentence for each operation with a lasting external effect: charge, send email, decrement stock, create shipment. Not every operation needs one; reads never do.
- For each, list who can repeat it. If the provider can redeliver a notification, the notification handler is on the list even though no human is involved.
- Choose the key's lifetime: per cart confirmation, per order, per notification id. The key must be created before the first attempt and survive the client dying.
- Put the uniqueness where concurrent repeats are serialised — the database — and test with two simultaneous requests, not two sequential ones (Example-Driven Thinking).
- Decide the repeat's response: return the stored result, not an error. A client that gets an error on a repeat will retry again.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- The store's at-most-once operations: charge a card (per checkout attempt), create an order (per checkout attempt), decrement stock (per order line), send confirmation (per order), mark paid (per provider notification). Reads of the catalog and the cart are not on the list.
- Who repeats checkout: the customer double-clicking; the browser resubmitting a form on refresh; the mobile app retrying a lost response; our own retry on provider timeout; a support agent re-running a failed job. Who repeats "mark paid": the provider redelivering, and our own job re-processing a batch after a crash.
- The key for checkout is created when the customer reaches the review page and sent with every submit of that review. The backend inserts the key into a table with a unique constraint before calling the provider; a second insert fails, and the handler returns whatever the first attempt stored. Two simultaneous submits in a test produce one row, one charge and two identical responses.
- The "mark paid" handler stores the provider's notification id with the same pattern, so the redelivered notification is recognised and the second stock decrement and email never happen.
How you know it worked
What now exists that did not before, and what question you can now ask.
- Each operation that must not repeat has a written at-most-once sentence, a key with a stated lifetime, and a place where the uniqueness is enforced.
- A test exists that submits the same checkout twice at the same time and observes one charge, one order and two matching responses.
- The word "idempotent" now names something specific in your system rather than a property you hope your endpoints have.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?Which operations in this system must happen at most once, and what is the sentence that says so?
- ?Who can send this request twice — human, browser, network, my retry, their redelivery, a job?
- ?What identity is the same on a repeat and different on a new attempt, and when is it created?
- ?Where is the uniqueness enforced against two repeats arriving at the same instant, and what does the second one receive?
What can go wrong
- Every endpoint gets an idempotency key, including reads and operations that are naturally safe to repeat, and the key table becomes the busiest table in the database. The enumeration decides which operations need it; most do not.
- The key is generated on the server per request, so every request is unique and the mechanism exists without protecting anything. The key's lifetime is the design; the table is plumbing.
- The check is done in application code with a read-then-write, and passes every sequential test while two concurrent repeats both pass the read. Concurrency is the case; test it.
- A key table is state that has to be stored, indexed and eventually expired; the stored response has to be large enough to replay and small enough to keep.
- Returning the first result to a repeat hides from the client that anything was repeated; sometimes the client wanted to know, and the design has to decide.
- "Idempotency means the request has no effect the second time." It means the second time has the same effect as the first — one charge, one order, the same response — which is stronger and more useful than "no effect".
- "A unique constraint on the order table is enough." It stops two orders; it does not stop two charges if the charge happens before the insert. The key has to be reserved before the external call, which is the ordering question from Partial Failure.
- "The provider handles this for me." Many providers accept an idempotency key on the charge call — and it is your key, with your lifetime, and their guarantee covers only their side. The notification handler and the order write are still yours.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThe enumeration of who repeats a request holds for any system with a client and a network; the operations that need at-most-once are domain-specific — in a chat app it is "send message" and in a URL shortener it is nothing at all.
- SCALE-SPECIFICOn one server with one process, an in-memory check and a database constraint are both fine; with several instances only the database — or something else shared and serialised — can hold the invariant, and the in-memory version silently stops working when the second instance starts.
- ILLUSTRATIVEThe demo, the double click and the two charges are invented; the list of repeat sources is the shape of the argument and not a complete catalogue for any real provider.
Where the depth lives
This domain asks the question and hands the answer off by name.