Replication

Quorums: What R + W > N Does and Does Not Buy

Write to W replicas, read from R, and if R + W > N the read set must intersect the write set. The arithmetic is trivially true. The conclusion people draw from it — "so I read the latest value" — holds only under assumptions that real systems routinely violate, and the assumptions are the lesson.

▶ Run the lab

The question this answers

The question

If R + W > N guarantees my read set overlaps my write set, why can I still read a stale value?

The guarantee — the property claimed, and its scope

Under a stated set of assumptions — fixed membership, no sloppy quorum, writes applied to the replicas that acknowledged them, and versioned values — a read of R replicas contacts at least one replica that acknowledged any completed write of W. The guarantee is *overlap of node sets*, and turning overlap into "I read the latest value" requires the reader to be able to tell which returned version is newest.

Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.

What a node knows — observation versus inference

A replica knows the versions it holds locally. A coordinator knows which replicas responded and what versions they returned; it does not know what the non-responding replicas hold, whether a concurrent write is in flight, or whether a replica that acknowledged a write later lost it to a crash before flushing. The intersection argument is about which nodes were *contacted*, not about which nodes hold current data.

A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.

What guarantee?What does a node know?How does it work?What can fail?How does it fail?Where is coordination?What holds under failure?How does it recover?How would you know?What is the simpler thing?
quorumreplicationoverlapassumptions

The arithmetic, and the exact claim it supports

The pigeonhole argument is genuinely airtight: if a write reached W of N replicas and a read contacts R of N, and R + W > N, then the two sets cannot be disjoint. At least one contacted replica participated in that write. This is arithmetic and it does not fail.

What the arithmetic supports is exactly this: the reader receives a response from at least one replica that acknowledged the write. Every additional step — that the replica still holds it, that the reader can identify it as the newest of the R responses, that no concurrent write muddles the picture — is a separate assumption, and each one is violated by some real deployment.

N=5, W=3, R=3 — the standard majority configurationassumption
N=5 · W=3 · R=3R + W > N  (3 + 3 > 5)overlap = 1 replica
R1W
R2W
R3⬤ both
R4R
R5R
W — write quorum (first 3)R — read quorum (last 3)⬤ both — the replica that carries the write into the read
The claim

✓ Every read quorum meets every write quorum

A reader therefore touches at least one replica that saw the write — provided everything on the right holds.

Only if — 5 assumptions
  • Membership is fixed: the same five nodes are the N for both the write and the read. Adding, removing or replacing a node changes N under you.
  • The W acknowledgements came from replicas that are members of N, not from substitute nodes accepting hinted handoffs (a sloppy quorum).
  • A replica that acknowledged a write still holds it — it did not acknowledge from memory and then crash before flushing.
  • Values carry versions or version vectors, so a reader receiving three different values can determine which supersedes which rather than guessing by timestamp.
  • The write completed. A write that was acknowledged by fewer than W and then abandoned may still be present on some replicas, and a later read may or may not see it.
⚠ Where the formula stops delivering

A sloppy quorum accepts the W acknowledgements from *any* W reachable nodes, including nodes outside the designated N for that key. The write is durable and available, and R + W > N no longer implies intersection with the home replicas, so a subsequent read of R home replicas can legitimately miss it entirely until hinted handoff completes.

Four ways the useful conclusion fails

Each of these is a real deployment behaviour, not a contrived edge case. Notice that in every one, the arithmetic still holds — it is the step from overlap to recency that breaks.

Sloppy quorums. The system prefers availability, so when home replicas are unreachable it writes to substitutes with a hint to forward later. The write is durable on W nodes; those nodes are not in the read set. Concurrent writes. Two writers each achieve W. The reader sees two versions and, without version vectors, must pick one — usually by wall-clock timestamp, which is Last Write Wins Is Data Loss You Chose by Default and discards a write. Membership change. A node is replaced during a rebalance; the N for the write and the N for the read are different sets. Partial writes. A write reaches 2 of 3 required, so it is not acknowledged, but those 2 replicas keep it. A later read may or may not see the value depending on which replicas answer — the same read repeated can flip.

SituationDoes R + W > N still hold?Do you read the latest write?
Steady state, versioned values, fixed membershipassumptionYesYes
Sloppy quorum with hinted handofftypicalYes, arithmeticallyNo — the acking nodes were not the home replicas
Two concurrent writersprotocolYesYou read both, and "latest" may not be defined
Node added or replaced between write and readassumptionYes, for different values of NNot necessarily — the sets are over different memberships
Write that did not reach W and was abandonedprotocolYesA partially-written value may appear, disappear and reappear across reads
The arithmetic holds; the conclusion does not

Choosing R and W is choosing which operation pays

With N fixed, R and W trade against each other. W=N, R=1 makes reads fast and cheap and writes fragile — any single unavailable replica blocks writes. W=1, R=N is the mirror: writes always succeed and reads need everyone up. W=R=majority is the balanced choice and the only one that tolerates the loss of a minority on both paths, which is why it is the default nearly everywhere.

The important consequence is about availability under partition. With N=3 and W=R=2, a partition that isolates one node leaves the majority side fully functional and the minority side unable to complete either reads or writes at quorum. That is not a bug — it is the system choosing consistency-ish behaviour over availability for the minority, and it is the concrete shape of the argument in CAP: What the Theorem Actually Says.

  • W=R=majority tolerates ⌊(N−1)/2⌋ failures on both paths and is the default for good reason.
  • R + W ≤ N is a legitimate configuration — it buys latency and availability and explicitly gives up the overlap property. Choose it deliberately, not by accident.
  • Increasing N without increasing W and R does not increase durability of acknowledged writes; it increases the number of places a value might be stale.
  • Latency at quorum is the W-th fastest response, not the average — so quorum latency is a tail-latency problem. See perfLinks tail-latency.

Key points

  • R + W > N guarantees the read set and write set share a node. That is all it guarantees.
  • Turning overlap into "I read the latest value" needs fixed membership, strict (non-sloppy) quorums, durable acknowledgements and versioned values.
  • Sloppy quorums keep the arithmetic and destroy the conclusion — the acknowledging nodes are not the nodes you read.
  • Concurrent writes produce multiple versions; without version vectors the reconciliation is last-write-wins, which loses data.
  • W=R=majority is the balanced default; R + W ≤ N is a valid choice that gives up overlap on purpose.

The chain, answered

Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.

How it works
  • A coordinator (a node or the client library) sends the write to all N replicas for the key.
  • It waits for W acknowledgements, then acknowledges the client; the remaining replicas are updated asynchronously or repaired later.
  • A read is sent to all N (or enough of them) and the coordinator waits for R responses.
  • The coordinator compares the returned versions: if one supersedes the others it is returned; if they are concurrent, both are returned as siblings or merged by rule.
  • Replicas found to be behind are updated in the background — read repair — and a separate anti-entropy process covers keys nobody reads. See Anti-Entropy: Repairing Divergence Nobody Reported.
What can fail at the boundary
  • Fewer than W replicas are reachable and the write cannot complete at the requested level.
  • A write reaches some replicas but not W, leaving a partially-written value that may surface on later reads.
  • A replica acknowledges and then crashes before persisting.
  • Two concurrent writes both achieve W and the system must represent two versions of the truth.
  • Membership changes mid-operation and the effective N differs between the write and the read.
How it fails — what an operator sees
  • Read flapping: the same key returns the new value and then the old one across successive reads, because a partially-written value is present on some replicas and the R responding replicas differ each time.
  • Silent overwrite: two concurrent writes are resolved by wall-clock timestamp and the write from the machine with the slower clock disappears. The operator sees a correct-looking value and one missing update.
  • Availability cliff at the quorum boundary: with N=3 and W=2, losing a second replica takes writes from fully working to completely failing with no intermediate degradation — error rate jumps from 0% to 100%.
  • Stale reads after a sloppy-quorum window: writes succeeded during a partition on substitute nodes, and reads of the home replicas miss them until hinted handoff drains. The operator sees "data appeared several minutes late" with no errors at any point.
  • Latency dominated by the slowest quorum member: p99 write latency tracks the W-th fastest replica, so one degraded disk in the cluster raises tail latency for every key it hosts.
Where coordination is required
  • Every read and every write pays a fan-out and a wait for the W-th or R-th response. This is per-operation coordination, unlike leader-based replication where the steady state costs nothing.
  • There is no agreement in the consensus sense — no node has to agree with another, the coordinator merely counts. That is why quorum replication is cheaper than consensus and provides strictly less.
  • Quorum overlap gives you a *recency* argument for single keys under assumptions. It does not give atomic multi-key operations, compare-and-set, or linearizability without additional mechanism. See Linearizability: An Operation Is an Interval, Not a Point.
What still holds under failure
  • Up to N−W replicas may be down and writes still complete; up to N−R may be down and reads still complete.
  • On the minority side of a partition with majority quorums, neither reads nor writes complete — that side is unavailable by design.
  • Acknowledged writes survive the loss of up to W−1 replicas, assuming acknowledgement implied durability on each.
How it recovers
  • Detect: track the rate of operations that failed to reach quorum, and separately the rate of reads that returned divergent versions — the second is the honest measure of how much repair work is outstanding.
  • Contain: prefer failing a write at the requested quorum over silently degrading to a lower one; a sloppy quorum should be an explicit, observable mode.
  • Recover: read repair fixes what is read; anti-entropy fixes what is not. Both are needed, and a cluster relying only on read repair accumulates permanent divergence on cold keys.
  • Reconcile: drain hinted handoffs and confirm the drain completed, since the window where writes live only on substitutes is exactly the window where reads look wrong.
  • Verify: run a periodic consistency comparison across replicas for a sampled key range, using Merkle-style comparison rather than full scans. See Merkle Trees: Finding the Difference Without Reading the Data.
How you would know
  • Rate of quorum failures by operation type, distinct from general errors.
  • Divergent-version rate on reads — how often R responses disagree — and the read-repair rate that follows.
  • Hinted-handoff queue depth and age, which is the direct measure of how much data is not where it should be.
  • Per-replica response latency distribution, since quorum latency is set by the W-th fastest and one slow node moves the whole tail.
  • Effective membership churn: how often the replica set for a key changed, because that is the assumption most often silently violated.
When it helps
  • Systems that need uniform write availability without a leader or a failover procedure.
  • Clusters where node failures are frequent enough that a leader-election gap on every failure would be too costly.
  • Workloads over independent keys with no cross-key invariants, where per-key recency is the entire requirement.
  • Deployments that want to tune the consistency/latency trade per operation rather than per cluster.
When it hurts
  • Anything needing compare-and-set, uniqueness, or multi-key atomicity — quorums do not provide these and bolting them on requires consensus anyway.
  • Low-latency reads, since every read pays a fan-out and waits on the R-th response rather than one local answer.
  • Small clusters, where the availability cliff at the quorum boundary is steep and a two-node loss is not exotic.
  • Teams that will read R + W > N as a guarantee rather than as a conditional — the configuration then encodes a belief the system does not honour.
Simpler alternatives

R + W > N, and everything the arithmetic does not say

R + W > N, and everything the arithmetic does not say
The inequality is trivial and true. Turning it into “I read the latest value” requires four further conditions, and each one fails in a way that leaves the arithmetic intact.
try
R + W vs N
3 + 3 > 5
forced overlap
1 replica
failures writes survive
2
failures reads survive
2
The write set fills from the left, the read set from the right; where they must meet is the overlapassumption
N=5 · W=3 · R=3R + W > N  (3 + 3 > 5)overlap = 1 replica
R1W
R2W
R3⬤ both
R4R
R5R
W — write quorum (first 3)R — read quorum (last 3)⬤ both — the replica that carries the write into the read
The claim

✓ Every read quorum meets every write quorum

A reader therefore touches at least one replica that saw the write — provided everything on the right holds.

Only if — 5 assumptions
  • Quorums are drawn from the same N home replicas — no sloppy quorum, no hinted handoff to a stand-in node.
  • Membership is stable: every participant agrees which N nodes hold this key while the read and the write are in flight.
  • A write that reached W replicas is durable on all W — an acknowledgement is not withdrawn by a later crash.
  • The reader can tell which of the returned values is newest — a version, a vector clock or a monotonic timestamp, not a wall clock it merely trusts.
  • Under those conditions every read quorum shares at least 1 node with every write quorum, so the last acknowledged write is visible to the read.
The guarantee, scoped
Read-your-write for a single key, under the stated assumptions: any read touching R = 3 of N = 5 replicas sees at least one replica carrying the last write acknowledged by W = 3.
What it cost
R + W = 6 > N = 5. Quorums overlap on 1 node. The price: writes survive only 2 node failures and reads only 2. Overlap is bought with availability, not with cleverness.
Where the formula stops delivering — 3 cases at these settings
  • A sloppy quorum accepts W acknowledgements from nodes outside the home set during a partition. The count is met, the overlap is not, and the read misses the write.
  • A write fails partway: fewer than W replicas acknowledged, so the client saw an error, but some replicas kept the value. A later read can return a write that was reported as failed.
  • Last-write-wins resolution with unsynchronised clocks discards the newer value because the older writer had a faster clock. The overlap happened; the read still returned stale data.
The single most useful sentence about quorums: overlap is bought with availability, not with cleverness. Every replica added to W is one fewer failure the write path tolerates, and the inequality gives nothing back for free. R + W ≤ N is a legitimate choice — it is what you pick when latency and availability matter more than reading your own write — and it should be a choice, recorded, rather than a default discovered during an incident. And note what none of these settings can fix: if the reader cannot tell which returned version is newest, the overlap delivered the right replica and the wrong answer.
assumptionEvery claim in this panel is conditional on the list `quorumAnalysis` returns. The overlap count is arithmetic; the guarantee is a statement about a system model, and the model is what production violates.

What people believe, and what is true

Claim

R + W > N guarantees you read the latest write.

Reality

It guarantees your read set includes a node that acknowledged the write. Whether you *recognise* that value as the latest, and whether that node still holds it, are separate conditions the formula says nothing about.

Claim

Quorums give you strong consistency.

Reality

They give per-key recency under assumptions. They do not give linearizability, compare-and-set, or multi-key atomicity — those need consensus. And "strong consistency" is exactly the unqualified phrase to reject. See Consistency Models: What "Consistent" Has To Mean.

Claim

A sloppy quorum is still a quorum.

Reality

It satisfies the count and abandons the set. Durability is preserved; the intersection argument is not, so reads of the home replicas can legitimately miss an acknowledged write.

Claim

Raising N raises durability.

Reality

Raising N with W fixed raises the number of replicas that may be stale. Durability of acknowledged writes is governed by W and by failure-domain independence, not by N.

Go deeper

Only the levels this lesson can honestly fill — a missing level is a claim nobody had.

Overview

Write to W nodes, read from R, and if R + W > N the two sets must share a node. That overlap is the whole mechanism.

Practical

Use W = R = majority unless you have a specific reason not to. Then check the four assumption-breakers in your deployment: is sloppy quorum enabled, are values version-vectored or timestamp-resolved, does rebalancing change membership under live traffic, and does anti-entropy actually run.

Advanced

Quorum overlap is a statement about node sets, and recency is a statement about values — bridging the two requires the reader to order the versions it receives, which is why quorum systems live or die on their versioning scheme. Note also that even a strict quorum system is not linearizable without read repair being made synchronous, because a read that observes a partially-completed write and does not repair it before returning allows a later read to observe the older value. See Linearizability: An Operation Is an Interval, Not a Point and Version Vectors: Making the Conflict Visible.

Apply it

Build it, then break it
  • 🔧 For N=5, enumerate the (R, W) pairs that satisfy overlap, and for each state the maximum number of node failures tolerated on the read path and the write path.
Interview questions
  • 💬 You run N=3, W=2, R=2 and a read returns a stale value. Give three explanations that do not involve a bug.
  • 💬 What exactly does R + W > N prove, stated as carefully as you can?
  • 💬 Why does a sloppy quorum keep durability while destroying the overlap argument?
  • 💬 Your p99 write latency doubled and no node is reporting errors. How does quorum sizing explain that?