The question this answers
If a deposed leader keeps acting, why does the system not corrupt itself?
Any operation carrying a term lower than the receiver’s current term is rejected, unconditionally and locally. Combined with at-most-one-leader-per-term, this guarantees that at most one leader in the entire history of the cluster can commit at any log position — even while several nodes believe they lead.
Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.
A node knows the highest term it has ever seen. That is enough: it does not need to know who the leader is, whether a partition exists, or how many nodes believe they lead. Rejecting a lower term is a purely local decision requiring no communication — which is exactly why it keeps working during the partition that created the problem.
A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.
A counter that only goes up
A term (Raft), epoch (ZooKeeper/Zab), view number (Viewstamped Replication) or ballot number (Paxos) is the same idea under four names: an integer that identifies a leadership generation and never decreases. Every message carries it. Every node remembers the highest it has seen.
The rules are three lines long and they carry an astonishing amount of weight:
- A node that receives a message with a higher term adopts it immediately and reverts to follower — even if it is currently leader.
- A node that receives a message with a lower term rejects it and replies with its own current term.
- A candidate increments the term when it starts an election, so no two elections share a generation.
Why this makes the stale leader safe rather than merely unlikely
Consider the classic sequence. N1 leads in term 7. A partition isolates it. N2 and N3 elect N2 in term 8. N1 knows nothing of this and continues accepting client writes, appending them to its local log, and trying to replicate them.
Every one of N1’s AppendEntries(term=7) messages that reaches any node is rejected, because everyone reachable has moved to term 8. N1 therefore cannot reach a majority, cannot commit anything, and — critically — cannot acknowledge anything to a client, because acknowledgement requires commitment. Its local log grows a tail of entries that will be thrown away when the partition heals.
Notice what was *not* required: no node had to detect the partition, no node had to know N1 existed, and no timeout had to be accurate. The safety comes from a comparison of two integers. This is why the mechanism survives exactly the conditions that break everything else.
The term is a logical clock, not a physical one
A term orders leadership generations without any reference to wall-clock time, which is what makes it trustworthy: it does not care about Clock Skew: The Gap You Cannot Measure From Inside, NTP steps, or a virtual machine being paused for a minute. It is Lamport Clocks: Consistent With Causality, Blind to Concurrency reasoning applied to a single, very important variable.
That distinction matters when people propose replacing terms with timestamps — "just reject writes older than 5 seconds". A timestamp check depends on two machines’ clocks agreeing; a term check depends on nothing. The timestamp version fails silently under skew and produces exactly the corruption terms exist to prevent.
The one hard requirement is durability: the current term must survive a restart. A node that comes back with a forgotten term can vote twice in the same generation, which is the single most direct way to manufacture two legitimate leaders.
| System | Name | Incremented when |
|---|---|---|
| Raftprotocol | term | a candidate starts an election |
| ZooKeeper (Zab)protocol | epoch | a new leader is established |
| Viewstamped Replicationprotocol | view number | a view change begins |
| Paxosprotocol | ballot / proposal number | a proposer starts a round |
| Lock services (generic)typical | fencing token | the lock is granted to a new holder |
Where the term stops working: outside the cluster
The term protects everything that understands terms. The replicated log understands terms. Your object store does not. Your payment provider does not. Your file system does not.
So a stale leader is harmless *within* the consensus group and entirely unconstrained outside it. If the leader’s job includes "write the compacted file to S3" or "send the payout", the term does nothing, and a stale leader will happily do both while the new leader does them too.
The generalisation of the term to external resources is the fencing token, and it requires the *external resource* to participate: it must remember the highest token it has seen and reject anything lower. That is Fencing Tokens: Making the Stale Actor Safe, Not Just Unlikely, and it is the same integer comparison moved to where the effect actually lands.
Key points
- A term is a monotonically increasing leadership generation number carried on every message.
- Higher term seen → adopt it and step down. Lower term received → reject, locally, without asking anyone.
- This makes a stale leader unable to commit or acknowledge, without anyone needing to detect the partition.
- Terms are logical, not physical — immune to clock skew, unlike any timestamp-based scheme.
- The current term must be durable across restarts, or two leaders in one term become possible.
- Terms protect only participants that check them; external systems need fencing tokens.
The chain, answered
Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.
- • Each node persists
currentTerm, initialised to 0. - • A candidate increments
currentTermand persists it before requesting votes. - • Every RPC — vote request, append, heartbeat — carries the sender’s term.
- • On receipt: if
msg.term > currentTerm, setcurrentTerm = msg.term, clear the vote, become follower. - • If
msg.term < currentTerm, reject and returncurrentTermso the sender learns it is stale. - • A leader that receives a reply containing a higher term steps down immediately, before processing anything else.
- • The term is not persisted, so a restarted node re-uses a generation it has already voted in.
- • A partitioned node repeatedly increments its term in fruitless elections and returns with an inflated term, deposing a healthy leader.
- • A term counter overflows a small integer type — rare, but a real bug class in embedded implementations.
- • An implementation checks the term on some message types and not others, leaving a path for stale writes.
- • External side effects are performed on the basis of leadership, where terms have no reach.
- • Term inflation after a partition heals: the operator sees a healthy leader deposed the moment a long-isolated node rejoins, with the term jumping by hundreds. Throughput dips for one election. The fix is pre-vote.
- • Two leaders in the same term: the operator sees two nodes logging "became leader, term 41" and divergent log contents. This is impossible under the protocol, so it is proof that term durability was lost — check for disabled fsync or a container with an ephemeral data volume.
- • Stale-leader writes acknowledged to clients: an implementation that returns success on local append rather than on commit. The operator sees clients reporting successful writes that are absent after failover, with no error logged anywhere.
- • External double-effect: the operator sees the same compaction output written twice to object storage by two different nodes, or two payout requests, because the external system never checked a term.
- • Checking a term requires no coordination at all — it is a local integer comparison, which is why it works during a partition.
- • Advancing a term requires an election, which requires a majority.
- • The asymmetry is the point: acquiring authority is expensive and coordinated; rejecting stale authority is free and local.
- • A stale leader cannot commit, so no committed data is ever produced by two generations at once.
- • Uncommitted entries written by a stale leader are discarded on rejoin; a client that was never acknowledged has no claim on them.
- • The mechanism holds during arbitrary message loss, delay and reordering, because it never depends on receiving anything.
- • Detect: alert on any node reporting a term different from the cluster majority for more than a few seconds.
- • Contain: ensure the leader acknowledges clients only on commit, never on local append — this converts a stale-leader incident from data loss into a timeout.
- • Recover: on rejoin, the stale node adopts the higher term, steps down, and truncates its divergent tail automatically.
- • Reconcile: for effects already emitted outside the cluster, terms cannot help; you need the token check at the resource or an idempotent effect.
- • Verify: confirm all members converge to the same term, and audit that term state lives on durable storage on every member.
- • Current term per member, as a single graph — divergence is immediately visible.
- • Rate of term increase; a sustained climb means elections are failing, not succeeding.
- • Count of RPCs rejected for stale term, broken down by sender — the direct signal that a stale leader exists.
- • Whether the data directory holding term state is on durable, non-ephemeral storage.
- • Any time leadership can change while an old leader is still running — which is always.
- • As the general pattern for "this authority has been superseded", far beyond consensus: lease generations, configuration versions, schema versions.
- • When it creates false confidence: the term protects the log, and engineers extend that feeling of safety to side effects the term never touched.
- • When pre-vote is absent and a flapping node’s inflated term disrupts an otherwise stable cluster.
- • Timestamp-based staleness checks ("reject writes older than N seconds") — simpler, and wrong under clock skew or a paused process. Only defensible when the consequence of being wrong is small.
- • Lease-based leadership with a clock assumption: the leader holds authority for a bounded wall-clock window and stops acting when it expires. Faster reads, but now correctness depends on bounded drift. See Leases: Authority With an Expiry Date.
- • Version numbers on the *data* rather than on the leader — compare-and-swap per key, which protects individual writes without needing a leadership concept at all.
Terms: a number nobody can argue with
- n1 — still accepting client writes it will never commit
What people believe, and what is true
The term stops the old leader from running.
Nothing stops it from running. The term stops anyone from accepting its messages, which is a different and much more achievable thing.
A higher term means more recent data.
It means a more recent leadership generation. A node can hold a high term and an empty log — which is why the vote rule checks log completeness separately.
You could use a timestamp instead.
A timestamp comparison depends on two clocks agreeing. A term comparison depends on nothing, which is why it survives exactly the conditions where you need it.
Terms protect the whole system.
They protect participants that check them. Every external effect — object storage, email, payments — is outside their reach.
Go deeper
Only the levels this lesson can honestly fill — a missing level is a claim nobody had.
Overview
Leadership generations are numbered. Everyone remembers the highest number they have seen and ignores anything older. A deposed leader can talk, but nobody listens.
Practical
Persist the term before acting on it. Acknowledge clients on commit, not on local append. Graph term-per-member and alert on divergence. Remember that the protection ends at the cluster boundary — any external write needs its own token check.
Advanced
The term is a Lamport clock over a single distinguished event: leadership change. Its total order is what lets an arbitrary node resolve authority with no knowledge of topology. Because rejection is local and requires no round trip, it is the rare safety mechanism whose cost does not rise under failure — it is *cheapest* exactly when the system is most stressed.
Apply it
- 🔧 Trace what happens to a client write submitted to a stale leader, from request to eventual outcome.
- 🔧 Extend the term idea to a resource outside the cluster and show what the resource must do for it to work.
- 💬 A leader is partitioned but still receiving client writes. Why is the data not corrupted?
- 💬 Why can a term not be replaced by a wall-clock timestamp?
- 💬 You see two nodes claim leadership of term 41 in the logs. What does that tell you?