The question this answers
What exactly is an eventually consistent system promising me?
Eventual consistency: if no new updates are made to a given item, all replicas will eventually converge to the same value for that item, under the system's stated assumptions — that every update is eventually delivered to every replica, and that a deterministic rule combines concurrent updates. It says nothing about how long "eventually" is, and nothing about *which* value is converged upon.
Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.
A replica knows its own value and the updates it has received. It cannot know whether it is converged — "no news" is indistinguishable from "no more updates exist" and from "the link is dead". Convergence is a property of the system in the limit; no participant can observe it, which is why anti-entropy processes must run continuously rather than until they detect completion. See Anti-Entropy: Repairing Divergence Nobody Reported.
A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.
The claim, and the two assumptions it rests on
Stated properly, eventual consistency is: if updates to an item stop, every replica of that item eventually holds the same value. That is the whole claim. It is a *liveness* property — it promises something good eventually happens — and it contains no *safety* content, meaning it forbids no particular observation at any finite time.
Two assumptions carry it, and both are engineering work rather than gifts. Delivery: every update eventually reaches every replica, which requires either a reliable propagation mechanism or a repair process that catches what propagation missed. Determinism: when replicas hold concurrent updates, the rule that combines them produces the same result regardless of the order in which each replica applies them. Formally the merge must be commutative, associative and idempotent; informally, if your merge rule depends on arrival order, your replicas will not converge and you do not have eventual consistency at all.
This is why "eventually consistent" is not a shrug. It is a claim that must be *engineered*: without anti-entropy the delivery assumption fails on cold keys, and without a well-behaved merge the determinism assumption fails on every conflict. Many systems that describe themselves as eventually consistent do not achieve it. See What "Eventually Converges" Actually Requires.
- Liveness, not safety: it says something good happens eventually, never that something bad cannot happen now.
- Delivery assumption: needs propagation *plus* repair, because propagation misses things during failures.
- Determinism assumption: the merge must be commutative, associative and idempotent, or replicas settle on different answers.
- Silent about the destination: convergence to a value says nothing about it being the right value. Last-write-wins converges perfectly while discarding updates. See Last Write Wins Is Data Loss You Chose by Default.
The honest criticism: it forbids nothing observable
The famous jab — "eventual consistency guarantees nothing, because I am eventually dead" — is glib but points at something real. Take any finite execution of any system, however anomalous. Is it consistent with eventual consistency? Yes, because the model only constrains the limit, and the run has not reached it. There is no finite observation that falsifies the claim.
That is the correct criticism, and it is a criticism about *specification*, not about staleness. A model that cannot be violated by any finite run cannot be tested, cannot be relied upon by a caller, and cannot appear as a premise in a correctness argument. It is why practitioners reach past it to models that *do* forbid observations: read-your-writes forbids losing your own write, monotonic reads forbids going backwards, causal consistency forbids seeing an effect before its cause. Each of those is checkable in finite time.
So the useful posture is not to reject eventual consistency, but to treat it as a *floor* rather than a specification. Take the eventually consistent store and add the session guarantees your application actually depends on. That combination is cheap, checkable, and covers the great majority of user-visible complaints. See Session Guarantees: The Underrated Middle Ground.
| Model | Falsifiable by a finite run? | What the counterexample looks like |
|---|---|---|
| Eventual consistencyprotocol | No | There is none — every finite history is permitted |
| Read-your-writesprotocol | Yes | A session reads after its own write and does not see it |
| Monotonic readsprotocol | Yes | A session's second read returns an older state than its first |
| Causal consistencyprotocol | Yes | A reply is visible where the message it replies to is not |
| Linearizabilityprotocol | Yes | A value regresses across two non-overlapping operations |
Why it is nonetheless the right choice constantly
None of the above makes it a bad design. Eventual consistency is what remains after you decline to coordinate, and declining to coordinate is what buys you availability during partitions, local write latency, and the ability to keep working when a dependency is down. For enormous categories of data — activity feeds, view counts, presence, search indexes, recommendation state, caches, DNS — a bounded period of disagreement costs nothing and the coordination would cost a great deal.
The practical shape of a well-built eventually consistent system therefore has four parts, and skipping any of them is where the reputation comes from: a merge rule chosen per field rather than defaulted; anti-entropy that actually runs, not just read repair; session guarantees layered on top so users are not confused by their own writes; and a measured convergence time so "eventually" is a number the team knows rather than a word.
convergence probe: write a marked value, then poll every replica until all agree p50 180 ms p95 1.4 s p99 6.2 s max (last 24h) 4 m 11 s <- during a rolling restart "eventually consistent" is not a design input. "converges within 6.2s at p99, worst observed 4m during node replacement" is.
Key points
- The claim: if updates to an item stop, replicas converge on that item. Nothing more.
- It is a liveness property with no safety content — no finite observation can violate it.
- It rests on two engineered assumptions: eventual delivery to every replica, and a deterministic (commutative, associative, idempotent) merge.
- Convergence says nothing about which value you converge to; last-write-wins converges while destroying updates.
- Treat it as a floor and layer session guarantees on top; that combination is cheap and covers most real complaints.
The chain, answered
Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.
- • A write is accepted by one replica and acknowledged without waiting for the others.
- • The update propagates to other replicas asynchronously — by streaming, gossip, or a replication log.
- • A background repair process compares replicas and fixes differences that propagation missed. See Anti-Entropy: Repairing Divergence Nobody Reported.
- • When replicas hold concurrent updates, a deterministic merge rule produces a single value on each replica independently.
- • Because the rule is deterministic and every update eventually arrives everywhere, all replicas compute the same result — that is convergence.
- • An update is lost in propagation and no repair process exists to notice.
- • The merge rule is order-dependent, so replicas settle on different values and never converge.
- • A replica is down longer than tombstone retention and reintroduces deleted data on return.
- • Anti-entropy is disabled for its I/O cost, and cold keys diverge permanently.
- • Convergence takes long enough that the application's implicit assumption of freshness is violated.
- • Permanent divergence on cold keys: read repair keeps read keys correct while unread keys disagree indefinitely. Discovered during a migration or an audit, typically years in. No alert exists because nothing is reading the data.
- • Non-convergence from an order-dependent merge: two replicas hold different values forever, each stable and each confident. Observed as a value that differs depending on which node answered, with no lag to explain it.
- • Deleted data returning: a tombstone expired before a long-absent replica came back, and repair propagated the resurrected value outward. Observed as records reappearing after deletion, sometimes months later.
- • User-visible incoherence: a user's own edit is missing on refresh because no session guarantee was layered on. Zero error rate, high support volume, unreproducible by engineers on fast replicas.
- • Unbounded convergence during operations: a rolling restart or rebalance pushes convergence time from milliseconds to minutes, and application logic written against the observed p50 breaks only during maintenance windows.
- • None on the write path. That is the entire purchase and everything else here is its consequence.
- • Repair is communication but not agreement — replicas exchange state, they do not negotiate an outcome, which is why it can run in the background without affecting availability.
- • Any invariant requiring a global check — uniqueness, a limit, a non-negative balance — is outside what this model can provide, and adding it requires real coordination. See Start From the Invariant, Not From the Architecture.
- • Reads and writes remain available on every reachable replica throughout a partition.
- • Replicas diverge for the duration and converge after heal, provided the merge rule is well-behaved.
- • Nothing is lost that the merge rule preserves — and everything is lost that it discards, which is why the merge rule is the actual design decision.
- • Detect: measure convergence time with an active probe rather than inferring it from lag, and alert on the tail rather than the median.
- • Contain: during long partitions, consider restricting writes for record classes whose merge you do not trust, rather than accumulating conflicts you cannot resolve.
- • Recover: run anti-entropy to completion after any extended outage; do not rely on read repair to catch up.
- • Reconcile: keep a conflict log so updates discarded by the merge rule can be recovered later — without it, a lost update is gone with no record it existed.
- • Verify: compare replicas over a sampled key range continuously and alert on persistent disagreement, since convergence must be checked rather than assumed.
- • Active convergence probe: write a marker, poll every replica, record time to full agreement. p50, p99 and 24-hour maximum.
- • Anti-entropy cycle completion time per replica pair, and whether it is running at all.
- • Divergence rate over a sampled key range — persistent disagreement is the signal that convergence is failing rather than slow.
- • Conflict rate and the distribution of merge outcomes, so silent discards are visible.
- • Tombstone age against the longest observed node outage.
- • Data where a period of disagreement costs nothing: feeds, counts, presence, recommendations, search indexes, caches.
- • Systems that must accept writes while partitioned or offline, including every mobile client that works without a connection.
- • Very large or very geographically spread deployments where coordination on every write is prohibitively expensive.
- • Anything with a natural merge — sets that only grow, counters, collaborative documents. See CRDTs: Deterministic Merge, Not Correct Merge.
- • Invariants: uniqueness, balances, inventory, capacity limits. The model cannot express them and no merge rule fixes that.
- • Data where a discarded update is unacceptable and no meaningful merge exists — most business records.
- • Small single-region systems where coordination is cheap and eventual consistency buys nothing but confusion.
- • Teams that adopt it as a default rather than a decision, and therefore never choose merge rules, never run anti-entropy, and never measure convergence.
- • Session guarantees layered on top — the cheapest large improvement available. See Session Guarantees: The Underrated Middle Ground.
- • Causal consistency, which forbids observable order violations while remaining available under partition. See Causal Consistency: Never Show an Effect Before Its Cause.
- • CRDTs, which make convergence to a *sensible* value guaranteed rather than merely convergence to a value. See CRDTs: Deterministic Merge, Not Correct Merge.
- • Linearizability per key for the small subset of data with real invariants, keeping eventual consistency for the rest. See Linearizability: An Operation Is an Interval, Not a Point.
- • A single-region leader-based system, if the geographic spread that motivated eventual consistency does not actually exist.
Write 11 on A. Watch it arrive — or not.
- A — serves 10 · version 0
- B — serves 10 · version 0
- C — serves 10 · version 0
| step | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| A | 10 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 |
| B | 10 | 10 | 10 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 |
| C | 10 | 10 | 10 | 10 | 10 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 | 11 |
What people believe, and what is true
Eventual consistency means you get random stale data forever.
It means that once updates to an item stop, replicas converge on it. Systems converge in milliseconds in the normal case. The real weakness is that the model forbids nothing observable — not that it delivers chaos.
Eventually consistent systems lose data.
Data loss comes from the *merge rule*, not from the model. Last-write-wins discards updates; a CRDT or an application merge does not. The model is silent on which value you converge to, which is exactly why choosing the rule is your job.
"Eventually" means soon.
The model gives no bound. Your system may well converge in 20ms at p50 and 4 minutes during a rolling restart. Only measurement turns "eventually" into a number you can design against.
Eventual consistency is what you get for free by not doing anything.
What you get for free is divergence. Convergence requires anti-entropy that runs and a merge rule that is order-independent — both are deliberate engineering, and both are commonly missing.
Go deeper
Only the levels this lesson can honestly fill — a missing level is a claim nobody had.
Overview
If updates to an item stop, all replicas eventually agree on it. That is the entire promise — no bound on when, and no claim about which value.
Practical
Do the four things that make it work: pick a merge rule per field, confirm anti-entropy actually runs, layer session guarantees on top for anything user-facing, and measure convergence so "eventually" is a p99 you can quote.
Advanced
Eventual consistency is pure liveness, so it is unfalsifiable in finite time and cannot serve as a premise in a correctness argument. The useful strengthenings add safety: *strong eventual consistency* (Shapiro et al.) requires that replicas which have received the same set of updates are in the same state — a safety property, checkable, and exactly what CRDTs provide. That is the difference between "we will agree someday" and "we agree whenever we have seen the same things", and it is the reason CRDTs are a genuinely different proposition rather than a nicer merge. See CRDTs: Deterministic Merge, Not Correct Merge and What "Eventually Converges" Actually Requires.
Apply it
- 💬 State precisely what eventual consistency guarantees. Then give a finite observation that would prove a system violates it.
- 💬 Does eventual consistency imply data loss? Where does the loss actually come from?
- 💬 How would you turn "eventually" into a number for your own system?