The Lost Update, Step by Step
A reads v1, B reads v1, A writes, B writes — and A's change is gone without an error, a log line, or a conflict. The anatomy of the most silent data-loss bug an API can have, and what a version check turns it into.
Frame the contract
API design starts with a consumer, a design question and a guarantee — never with a URL.
Anatomy of a disappearance
The lost update needs no failure to occur: every request below succeeds, returns 200, and does exactly what it was told. The damage lives entirely in the interleaving. A support agent and a billing job both perform the innocent pattern — read, modify in memory, write back — and the second write is built from a snapshot that no longer describes the world.
The width of the race window is what makes APIs uniquely exposed. Inside a database transaction the read-write gap is microseconds and the engine can detect the overlap (Concurrency Anomalies). Across an API, the gap is a human editing a form: seconds to minutes, spanning multiple stateless requests the server cannot correlate. No isolation level in your database can help, because as far as the database is concerned these are two well-separated, perfectly valid transactions. The race migrated up a layer, and only the API contract can chase it there.
Full-document writes make the blast radius total: agent and job each changed *one field*, but each PUT carried every field, so the loser's entire change set is erased — including fields the winner never intended to touch. This is why the size of what a write claims to know is a concurrency decision, not a style preference (PUT vs PATCH).
t0 Agent GET /customers/17 → { credit_limit: 5000, tier: "gold", … }
t1 Job GET /customers/17 → { credit_limit: 5000, tier: "gold", … }
t2 Agent PUT /customers/17 { credit_limit: 8000, tier: "gold", … }
→ 200 OK (limit raised after review)
t3 Job PUT /customers/17 { credit_limit: 5000, tier: "silver", … }
→ 200 OK (tier recomputed — from the t1 snapshot)
Result: credit_limit is 5000 again. The agent's approved raise
is gone. Nothing failed. Nothing was logged as a conflict.
The agent finds out when the customer calls.The same interleaving, with a version check
Replay the timeline with one addition: each read returns a version, each write must present it, and the server compares atomically (Optimistic Concurrency: Versions and If-Match covers the mechanism and who resolves). The interleaving is identical — the outcome is not. The job's write at t3 presents v1 against a resource now at v2, and the race becomes a 412 the client must consciously handle.
The failure did not disappear; it changed category. Silent data loss became an explicit conflict — from the worst detectability class to the best. The job re-fetches, recomputes the tier against the *current* state (which includes the raised limit), and resubmits. Total cost: one extra round trip on the rare contended write. The uncontended path pays one header.
PUT /customers/17 HTTP/1.1
If-Match: "v1"
Content-Type: application/json
{ "credit_limit": 5000, "tier": "silver", … }HTTP/1.1 412 Precondition Failed
ETag: "v2"
{
"error": {
"code": "version_conflict",
"message": "customers/17 changed since your read (v1 → v2).",
"current_version": "v2"
}
}
# The job refetches, recomputes tier against
# credit_limit 8000, writes with If-Match: "v2".The prevention menu — and the honesty option
Version checks are the general-purpose answer, but the menu is wider, and the cheapest fix is often structural: make writes carry *intent* instead of *state*. POST /customers/17/credit-limit-reviews {new_limit: 8000} and a tier-recompute command cannot erase each other's fields, because neither claims to know the whole record. Commands and narrow PATCHes eliminate whole classes of lost updates without any client-side conflict handling (Designing State Transitions).
For genuinely commutative updates — counters, appends, set-membership — server-side operations (increment, add-to-set) sidestep read-modify-write entirely: the server applies the operation to current state, so there is no stale snapshot to write back. And there is a legitimate bottom rung: declared last-write-wins. Some data is truly one-owner or overwrite-by-design (a device heartbeat, a user's own draft), and conflict machinery there is cost without benefit. The failure mode is not choosing LWW — it is shipping LWW *by default, undeclared*, on data where two writers matter.
| Approach | How it prevents the loss | Costs | Fits when |
|---|---|---|---|
| Version check (If-Match / version) | Stale writes rejected with 412/409 | Clients must handle conflicts (Optimistic Concurrency: Versions and If-Match) | General read-modify-write on shared resources |
| Intent-shaped writes (commands, narrow PATCH) | Writes don't carry fields they didn't change | More operations to design and document | Domain actions: adjust limit, change tier, publish |
| Server-side operations (increment, add/remove) | No client snapshot involved at all | Only fits commutative updates | Counters, tags, appends, set membership |
| Explicit lease / checkout resource | One writer at a time, visibly | Lease expiry, contention UX | Long exclusive edits: case assignment, doc locking |
| Declared last-write-wins | Nothing — the loss is accepted and documented | Real losses on multi-writer data | Single-owner or overwrite-by-design data only |
Key points
- The lost update is four successful requests and zero errors — the damage exists only in the interleaving, which is why nothing logs it.
- Database isolation cannot save you: the read-think-write span crosses stateless requests, so the race lives at the API layer and only the contract can address it.
- Full-document PUTs maximize the blast radius — the loser's entire snapshot erases fields the winner never touched.
- A version check converts silent loss into an explicit 412 — same race, opposite detectability — for one header on the happy path.
- Intent-shaped writes and server-side operations prevent whole classes of lost updates structurally, with no conflict handling at all.
- Last-write-wins is acceptable exactly once it is declared; undeclared LWW is the default you get by deciding nothing.
Lost Update Lab
Change the contract and observe which guarantee moves.
—
Without the version check, the second save silently erases the first — a lost update. With it, the stale writer gets 412 and must re-read; the contract turned a data-loss bug into a visible conflict the client can resolve.
Follow the failure
How the contract fails or gets misused, hop by hop — and what it costs when it completes.
- 1Team → API: ships read/PUT endpoints; single-writer assumption holds through launch, so no conflicts are ever seen.
- 2Product → users: adds a second writer — a batch job, a second admin role, an offline-capable mobile app.
- 3Writers → resource: interleaved read-modify-write begins; a small percentage of writes silently erase others.
- 4Users → support: "my change reverted" tickets arrive without reproduction steps; engineering finds no errors and suspects user error.
- 5Team → database: audits transactions and isolation settings — the wrong layer — and closes the investigation as unreproducible.
- Approved, audited changes (credit limits, permissions, prices) silently revert — with compliance consequences, not just inconvenience.
- Trust erodes asymmetrically: users learn the system "sometimes eats changes" and start double-checking every save, or keep shadow copies in spreadsheets.
- Debugging burns weeks because the evidence is an absence: no error, no log, just state that fails to match someone's memory.
Design, observe, evolve
A contract decision is incomplete until you know how you would notice it failing and how it changes later.
- • Require versioned writes (`If-Match` or body version) on every resource with more than one possible writer — count batch jobs and future features as writers.
- • Reshape high-conflict writes as commands or narrow PATCHes so concurrent changes to different aspects cannot collide at all.
- • Offer server-side operations for commutative updates instead of documenting "read, add one, write back".
- • Where LWW is chosen, write it into the contract per field or resource — "concurrent writes: last write wins" — so consumers can route multi-writer data elsewhere.
- • Audit-log every write with before/after and principal; lost updates then become findable after the fact even where prevention is absent.
- • On unversioned resources, flag write-after-write within a short window by different principals — that pattern is the race's fingerprint.
- • A multi-writer resource with zero 412s and active traffic means the version check is being bypassed or blind writes are still allowed.
- • Retrofit path: add versions to responses (additive) → clients adopt preconditions → enforce with `428` per endpoint — each step compatible, tracked by telemetry on blind writes ([[api-migration]]).
- • Moving a field from LWW to versioned is tightening, safe for correct clients; loosening versioned to LWW breaks the safety consumers built on and needs explicit consent.
- • Prevention is a tax on every client for a failure most requests never hit — the justification is the severity class (silent loss), not the frequency.
- • Command-shaped APIs multiply endpoints and design work compared to one generic PUT; the payoff is structural conflict immunity on the writes that matter.
- • Audit logs and write-window detection add storage and pipeline cost that looks unjustified until the first "my change reverted" investigation uses it.