The question this answers
What problem does consensus actually solve, and what does it leave untouched?
For a single decision: agreement (no two correct nodes decide differently), validity (the decided value was proposed by some node — it is not invented), integrity (a node decides at most once, and never un-decides), and termination (every correct node eventually decides) — where termination holds only while a majority is reachable and the network eventually delivers messages. The first three are unconditional; the fourth is not.
Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.
A node knows what it proposed, what votes it cast, and which messages have arrived at it. It never observes "the cluster has decided" directly — it *infers* a decision by counting acknowledgements from a majority. A node that has not heard from the majority does not know whether a decision was made without it; the absence of news is the one thing it can never interpret.
A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.
One decision, three machines, one of them dead
Strip the problem to its bones. Three nodes must answer one question — "who is the leader?", "is this transaction committed?", "does the cluster contain node D?" — and they must all give the same answer. If they were one machine this would be a variable assignment. Across three machines with a network that loses and delays messages, it is a research problem with a name and a literature.
The reason it is hard is not that agreeing is hard. Agreeing is easy when everyone is up and the network works: ask everyone, take the answer. It is hard because you must still agree when you cannot tell a dead node from a slow one — the condition Crashed or Just Slow: The Distinction You Cannot Make establishes — and because the protocol must never let two different answers escape into the world, not even during the confusion.
So consensus is a very specific bargain. It does not make the system fast, it does not make it available, and it does not remove partitions. It converts an unbounded set of possible disagreements into exactly one outcome, at the price of needing a majority to be present.
The four properties, and which one you actually lose
Textbooks list four properties. In production only one of them is ever surrendered, and knowing which is the whole practical skill: consensus protocols give up termination and never give up agreement. A Raft cluster with no majority does not decide wrongly — it stops deciding. Your writes hang. That is the protocol working as designed.
This is the same trade the CAP: What the Theorem Actually Says describes, seen from inside the protocol rather than from outside. A consensus system is a CP system by construction, and "CP" is a polite way of writing "it will refuse to answer rather than answer differently on two sides".
| Property | Means | Held during a partition? |
|---|---|---|
| Agreementprotocol | No two nodes decide different values | Yes — unconditionally |
| Validityprotocol | The decided value was proposed by someone | Yes — unconditionally |
| Integrityprotocol | A decision is final; nobody un-decides | Yes — unconditionally |
| Terminationassumption | Every correct node eventually decides | No — the minority side simply stalls |
Everything else is this problem in a costume
The reason consensus is worth an entire module is that a surprising number of practical problems reduce to it. They are not *like* consensus; they are provably equivalent, which means if you have solved one you have solved all of them, and if you think you have solved one cheaply you have made a mistake somewhere.
That equivalence is also a warning. When a design says "we just need one node to own this", it has not avoided consensus — it has hidden a consensus requirement inside a sentence that sounds operational. Do You Actually Need Consensus? is where we make that call deliberately.
- Leader election — decide on one value: the identity of the leader. See Leader Election: Choosing One, and Knowing You Were Chosen.
- Atomic broadcast / total order broadcast — decide, repeatedly, on the next message in a global order. See Total Order Broadcast Is Consensus Wearing a Different Hat.
- Replicated log — a sequence of consensus decisions, one per log index. This is exactly what The Raft Log: Commit Index, Divergence and Reconciliation builds.
- Distributed lock with a lease — decide who holds it for this epoch. See Distributed Locks: What They Are Actually For.
- Cluster membership — decide which set of nodes is the current configuration. See Cluster Membership: A Belief, Not a Fact.
- Atomic commit across nodes — decide commit or abort, which is why Two-Phase Commit: Buying Atomicity With a Promise is not fault-tolerant while consensus-backed commit can be.
What consensus does not do
It does not make your data consistent. It makes a *sequence of decisions* consistent; if your application writes to a replicated store outside that sequence, nothing is protected. It does not make a stale reader see fresh data — a follower serving reads can still be behind, which is why Linearizability: An Operation Is an Interval, Not a Point for reads needs extra work beyond "we run Raft".
And it does not prevent a partition. Nothing prevents a partition. Consensus decides how the system behaves during one: the majority side keeps deciding, the minority side stops. That sentence is the honest one-line summary of the whole module.
Key points
- Consensus decides one value, once, such that no two nodes ever see different answers.
- Agreement, validity and integrity are unconditional; termination is the property that is surrendered under partition.
- A consensus cluster without a reachable majority stops deciding — it does not decide wrongly.
- Leader election, replicated logs, locks, membership and atomic commit are all the same problem.
- Consensus does not prevent partitions; it defines the behaviour during one.
The chain, answered
Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.
- • A node proposes a value — "I should be leader", "entry 7 is this command".
- • The proposal is sent to all peers; each peer decides locally whether it may accept it, according to rules that make conflicting acceptances impossible.
- • The proposer counts acceptances. When a majority has accepted, the value is decided.
- • Because any two majorities of the same cluster share at least one node, no second value can also collect a majority — that single overlap is the entire safety argument.
- • The decision is then propagated; nodes that were absent learn it later by catching up, never by re-deciding.
- • Proposals are lost, so a proposer waits for acceptances that will never arrive.
- • Acceptances are lost, so a value is decided by the cluster while the proposer believes it is not.
- • Messages are delayed past the point where the sender has given up and moved on, then arrive.
- • A node crashes after accepting but before anyone learns of it, taking part of the decision’s evidence with it.
- • The cluster partitions so that no side holds a majority, and nobody can decide anything.
- • Write stall with no errors: the majority is unreachable, so proposals never commit. The operator sees client timeouts on writes, a healthy-looking process on every node, and a leader that keeps stepping down.
- • Decided-but-unknown: a value reaches a majority and the proposer times out before hearing so. The operator sees a client retry, a duplicate proposal, and a log where the "failed" operation is in fact present.
- • Election churn: nodes repeatedly propose leadership without any of them collecting a majority. The operator sees the term or epoch counter climbing quickly and throughput at zero.
- • Silent minority: an isolated node keeps serving reads from a stale local view because reads were never routed through the protocol. The operator sees no errors at all — only user complaints about vanished data.
- • One decision costs at least one round trip to a majority — the Coordination Couples Availability this domain measures in availability, not only in milliseconds.
- • The coordination is over the *decision*, not over the data. Reading a decided value can be local; making a new one cannot.
- • The cost scales with the slowest member of the fastest majority, not with the slowest member of the cluster — which is why a five-node cluster tolerates one slow node much better than a three-node one.
- • Any value already decided stays decided, and stays visible to every node that survives.
- • A minority side is not wrong — it is merely uninformed. It must not serve authoritative answers.
- • New decisions stop until a majority reforms; existing state is untouched.
- • Detect: alert on "no leader" duration and on the term/epoch counter rate of change, not only on node liveness.
- • Contain: make clients fail fast on the minority side rather than hanging, so the blast radius is a visible error rather than an exhausted thread pool.
- • Recover: restore majority connectivity; the protocol re-elects and catches up laggards without operator involvement.
- • Reconcile: nodes that were behind replay the decided sequence; they never re-vote on decisions already made.
- • Verify: check that every member reports the same committed index and the same term after recovery.
- • Current term/epoch, and its rate of increase — a fast-climbing term is the signature of an unstable cluster.
- • Time since the last successful decision, per node.
- • Number of members that acknowledged the most recent decision — this is the real availability margin.
- • Proposal latency at p99, which is dominated by the median member of a majority, not the mean of the cluster.
- • When a wrong answer is worse than no answer: who is the leader, what is the current configuration, has this transaction committed.
- • When the number of decisions is small relative to the amount of work they authorise — one election per hour authorising millions of writes is an excellent ratio.
- • When you need a fact that survives node loss without a human deciding which copy was right.
- • On the path of every business operation, where each write pays a majority round trip for a decision nobody needed. See Do You Actually Need Consensus?.
- • Across regions, where a majority round trip is bounded below by The One Number You Cannot Optimise and no amount of tuning helps.
- • When the real requirement was durability or ordering rather than agreement — a replicated log without elections would have done.
- • Single node with fast recovery: no agreement at all, availability bounded by restart time. Correct for a great many systems, and much easier to operate.
- • Primary/backup with an external arbiter (a managed lock service) — you have not removed consensus, you have rented it.
- • Convergent replication with no decisions at all: CRDTs: Deterministic Merge, Not Correct Merge-style merge, where any two states combine into a third without agreement. Only available when the invariant permits it.
- • Eventual agreement by Anti-Entropy: Repairing Divergence Nobody Reported — nodes converge in the background rather than deciding up front. Cheaper, weaker, and often sufficient.
What consensus solves, and what it leaves untouched
| Status | Why | |
|---|---|---|
| Agreement — no two correct nodes decide differentlyprotocol | holds | Two conflicting decisions would need two disjoint majorities, and majorities of the same set always intersect. Unconditional under crash-stop, at any level of message loss or delay. |
| Validity — the decided value was proposed by someoneprotocol | holds | No node invents a value. Unconditional. |
| Integrity — a node decides at most once, and never un-decidesprotocol | holds | Decisions are final. Nothing in the protocol can retract one. Unconditional. |
| Termination — every correct node eventually decidesassumption | holds | A majority is reachable and the network eventually delivers, so a decision is reached in one majority round trip. |
proposed value : "leader = n2" acknowledgements : 3 of 5 (majority = 3) decision made : yes — by a majority proposer was told : yes
What people believe, and what is true
Consensus makes the whole system strongly consistent.
It makes a specific sequence of decisions consistent. Reads served outside that sequence — from a follower, from a cache — carry no such guarantee.
A consensus cluster never has two leaders.
It can, briefly. What it guarantees is that the stale one cannot commit anything. See Split-Brain: Two Nodes, Both Certain They Are In Charge and Terms and Epochs: Making Stale Leaders Harmless.
Adding nodes makes consensus more available.
It raises the failure tolerance (a 5-node cluster survives 2 losses) but also raises the size of a majority, so every decision needs more acknowledgements and the tail gets worse.
If it decided, the client was told.
The decision and the notification are separate events. A decision can be final while the client sees a timeout — the A Timeout Tells You Nothing About Whether It Happened problem, now with a durable outcome behind it.
Go deeper
Only the levels this lesson can honestly fill — a missing level is a claim nobody had.
Overview
Consensus makes a set of machines agree on one value even though some of them fail. It needs a majority. Without a majority it stops rather than guessing.
Practical
Use it for the small number of facts that must be unambiguous — leadership, configuration, membership — and keep it out of the request path for everything else. Budget one majority round trip per decision, and make sure your alerting covers "no decisions are happening", which looks like silence rather than like errors.
Advanced
The FLP result says no deterministic protocol can guarantee termination in an asynchronous system with even one crash failure. Real protocols dodge it by using timeouts as an imperfect failure detector: they never risk safety on the detector being right, only liveness. That is why a badly tuned election timeout costs you throughput but can never cost you correctness.
Apply it
- 🔧 Write down the four properties, then for each one describe a concrete production incident caused by assuming it held when it did not.
- 🔧 Show that leader election and a replicated log are the same problem by reducing each to the other.
- 💬 Explain what a consensus protocol guarantees, and name the property it gives up under partition.
- 💬 Why does a majority quorum make two conflicting decisions impossible?
- 💬 Your five-node etcd cluster loses three nodes. What exactly happens to reads and to writes?