The question this answers
Does allowing readers to proceed in parallel actually pay here, or am I adding overhead and a starvation risk for nothing?
A routing table consulted on every request — roughly 50,000 reads per second — and updated when a service registration changes, roughly once every 30 seconds.
A Map<route, backend[]> plus a derived prefix index, both of which must agree. Reachable from every request handler and from the registration watcher.
A reader observes a routing table and its prefix index that were built together — never a map from generation N with an index from generation N−1 — and no reader ever observes the table mid-rebuild.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
The mechanism, and the two costs nobody mentions
A read/write lock has two modes. Any number of tasks may hold it in shared (read) mode simultaneously. Exclusive (write) mode requires that nobody else holds it in either mode. That is the whole idea, and it is genuinely the right shape for the routing table above: 50,000 concurrent readers do not conflict with each other, and serialising them behind a mutex converts a parallel workload into a serial one for no reason.
The two costs are real and routinely omitted. First, the uncontended path is more expensive than a mutex. A mutex acquire is typically one atomic compare-and-swap. A read lock must increment a shared reader count, which is also an atomic write to a shared cache line — so every reader, on every acquisition, invalidates that line in every other core's cache. With many cores and a short region, the coherence traffic from the reader count alone can exceed the work the region does. This is the surprising result that makes rwlocks lose benchmarks they "should" win.
Second, fairness is a real design decision with no free option. A reader-preferring lock lets arriving readers join while a writer waits, so under sustained read load the writer may never acquire — writer starvation, and in the routing table that means the service keeps routing to a backend that deregistered five minutes ago. A writer-preferring lock blocks arriving readers as soon as a writer queues, which fixes starvation and costs read throughput. Some implementations offer fair queueing, which costs more still. There is no configuration that avoids the trade; there is only choosing which failure you prefer.
When it pays, and the cheaper thing to try first
Three conditions must hold together for a read/write lock to be the right answer. Reads must dominate — 95% or more is the usual rule of thumb, and below roughly 90% a mutex generally wins. The critical section must be long enough that reader overlap is worth more than the extra acquisition cost; a single hash lookup is not. And the write must be frequent enough or urgent enough that you cannot simply publish a new immutable snapshot instead.
That third condition is where most rwlock proposals should die. The routing table is read 50,000 times a second and written once every 30 seconds — a ratio of 1.5 million to one. At that ratio the correct answer is not a read/write lock at all: build the new table and index off to the side, then swap one reference. Readers take no lock, pay no atomic write, and observe a consistent generation because the reference swap is a single atomic store. The writer holds a lock only against other writers, of which there is one.
That is the real lesson of this lesson. A read/write lock is the right tool for a genuinely mixed workload with a long region — an in-memory index with meaningful per-read work and writes every few seconds. For extreme read ratios, immutability beats it; for balanced ratios, a mutex beats it. The window in between is narrower than its reputation suggests.
| Situation | Mutex | Read/write lock | Immutable snapshot + swap | Recommendation |
|---|---|---|---|---|
| Short region (one map lookup), any read ratio | cheapest acquire; region is nanoseconds so serialising costs little | acquisition overhead exceeds the work protected | no lock on the read path at all | Snapshot if writes are rare, otherwise mutex. Never an rwlock. |
| Long region (walk an index, build a response), 99% reads | serialises readers that could overlap — the clear loss case | genuine win; readers overlap and the overhead amortises | best of all if the structure can be rebuilt whole | Snapshot first; rwlock if rebuilding the whole structure is too expensive. |
| Long region, 70% reads | simple, predictable, and usually faster than you expect | writers queue constantly; readers gain little | rebuild cost paid 30% of the time — too often | Mutex. |
| Extreme read ratio (1.5M:1), any region length | serialises 50k reads/sec for one write every 30 seconds | works, and every reader still pays an atomic write | readers pay nothing; one atomic store per write | Immutable snapshot with a reference swap. This is the routing-table case. |
| Writes must be seen immediately by all readers | yes | yes | no — readers may hold an older snapshot until they re-read | Mutex or rwlock. Snapshot semantics are the price of the lock-free read path. |
Writer starvation, watched happening
Starvation is not a theoretical concern with reader-preferring locks; it is the default behaviour under sustained read load. The schedule below shows a routing-table update waiting behind a stream of arriving readers. Each reader is individually brief. The writer never acquires, because there is never an instant when the reader count reaches zero.
The operational shape of this is distinctive and worth recognising: reads are fast, writes appear to hang, and the stale data grows steadily more stale. The alert that catches it is not lock wait time on average — it is *maximum* writer wait time, or better, the age of the data the writer was trying to publish.
The fixes each cost something specific. A writer-preferring lock blocks arriving readers once a writer queues, which bounds writer wait at one reader region but adds a queueing hop to the read path. A fair (FIFO) lock bounds everyone's wait and costs the most throughput. Publishing a snapshot instead removes the problem rather than bounding it: the writer never waits for readers at all, because it never needs exclusive access to what they are reading. That asymmetry is why the snapshot approach keeps winning this lesson.
| # | Reader stream (50k/sec) | Writer — publish routing generation 41 | Reader stream (continued) | State |
|---|---|---|---|---|
| 1 | acquire shared; readers = 1 | · | · | readers=1 writerQueued=no gen=40 |
| 2 | acquire shared; readers = 3 | · | · | readers=3 writerQueued=no gen=40 |
| 3 | · | request exclusive — blocks, readers > 0 | · | readers=3 writerQueued=yes gen=40 |
| 4 | · | · | acquire shared; readers = 4 | readers=4 writerQueued=yes gen=40 ✕ A reader-preferring lock admits an arriving reader while a writer is queued. The count went up, not down. |
| 5 | release shared; readers = 3 | · | · | readers=3 writerQueued=yes gen=40 |
| 6 | · | · | acquire shared; readers = 4 | readers=4 writerQueued=yes gen=40 |
| 7 | · | · | ...30 seconds of this... | readers=3–6 writerQueued=yes gen=40 ✕ Generation 41 has not been published. Traffic is still being routed to a backend that deregistered 30 seconds ago; those requests fail. |
| 8 | · | WRITER-PREFERRING VARIANT: arriving readers queue behind the writer | · | readers=3 writerQueued=yes gen=40 |
| 9 | · | readers drain to 0; acquire exclusive; publish gen 41 | · | readers=0 gen=41 |
Key points
- Shared mode for any number of readers, exclusive mode for one writer. Correct in principle, narrower in practice than its reputation.
- The uncontended read acquire is more expensive than a mutex acquire: it writes a shared reader count, so every reader invalidates the same cache line on every core.
- Rule of thumb: below roughly 90% reads, a mutex usually wins. For a very short region, a mutex wins at any read ratio.
- Reader-preferring locks starve writers under sustained read load; writer-preferring locks fix that and cost read throughput. There is no free option.
- For extreme read ratios, the better answer is usually not a lock at all — build a new immutable structure and swap one reference. Readers take nothing.
- The starvation symptom is stale data, not slow reads, so the alert must be writer wait *maximum* or data age. Averages hide it completely.
- Reader-to-writer upgrade is a deadlock generator: two readers both trying to upgrade wait for each other forever. Most APIs forbid it for this reason.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • The lock maintains a reader count and a writer flag. Acquiring shared increments the count if no writer holds or (depending on policy) is queued; acquiring exclusive waits until the count is zero and no writer holds.
- • Releasing shared decrements the count; the last reader out signals any queued writer.
- • Policy decides what happens when a reader arrives while a writer is queued: reader-preferring admits it (favouring throughput), writer-preferring queues it (favouring the writer), fair queues everyone in arrival order.
- • Both the reader count and the wait queue are shared mutable state accessed atomically, which is the source of the per-acquisition overhead.
- • Upgrading a held shared lock to exclusive cannot be done atomically without releasing first, which is why concurrent upgrade attempts deadlock and why most APIs simply do not offer it.
- • Four readers acquire shared concurrently and all proceed; a writer arrives, waits for all four to release, then publishes. The intended behaviour and the reason to use one.
- • Reader-preferring under load: readers arrive faster than they release, the count never reaches zero, and the writer waits indefinitely while readers see steadily staler data.
- • Writer-preferring: a writer queues, arriving readers are held, the current readers drain, the writer publishes. Writer wait is bounded by the longest reader region.
- • Two readers both attempt to upgrade to exclusive: each waits for the other to release its shared hold. Neither can, and the deadlock involves exactly two readers and one lock.
- • A short region with 32 readers: every acquisition writes the shared reader count, so throughput is lower than with a plain mutex despite no logical conflict between any two readers.
- • Guarantees mutual exclusion between a writer and everyone else, and no exclusion among readers.
- • Does NOT guarantee that a queued writer ever acquires, under reader-preferring policy. Starvation is permitted behaviour, not a bug in the implementation.
- • Does NOT guarantee that reads are faster than under a mutex. For short regions or moderate read ratios they are measurably slower.
- • Does NOT guarantee that a reader may upgrade to a writer. Most APIs prohibit it, and the ones that allow it document the deadlock.
- • Does NOT guarantee anything about the composition of two read-locked operations — a check under a read lock followed by an act under a write lock has the usual check-then-act gap. See Finding the Critical Section.
- • Does guarantee the same memory-visibility edge as a mutex: everything a writer wrote before releasing is visible to a subsequent reader that acquires.
- • Readers contend on the reader count even when they do not contend logically. This is the cost that surprises people and the reason the scaling curve flattens.
- • Writers contend with everything. A write-heavy workload on an rwlock is strictly worse than a mutex: same exclusion, more bookkeeping.
- • Writer starvation is contention that never resolves. It presents as data staleness rather than as latency, so ordinary contention dashboards miss it.
- • Sharding the state — one lock per route prefix rather than one for the table — reduces contention far more than any policy choice, and is the change worth making first.
- • Writer starvation under reader preference, presenting as stale data rather than as a slow lock.
- • Reader starvation under writer preference, when writes are frequent — the mirror failure, presenting as elevated read latency.
- • Upgrade deadlock — two readers both attempting to become writers.
- • Performance regression from adopting an rwlock on a short or balanced-ratio region, which is common because the change is made on intuition rather than measurement.
- • Check-then-act across a read lock and a write lock, which has the same gap as any other split region and is easy to miss because both halves are locked.
- • Recursive read acquisition deadlocking under a writer-preferring policy: a thread holding a read lock takes it again, but a writer queued in between blocks the second acquisition while the first is never released.
- • A long read region — walking an index, serialising a large structure, running a query against an in-memory table — with reads well above 90%.
- • When the structure is too large or too expensive to rebuild for a snapshot swap, so immutability is off the table.
- • When writes must be immediately visible to all subsequent readers, which snapshot publication does not give you.
- • When the read path genuinely runs on many cores simultaneously; on a single-threaded runtime it buys nothing at all, because reads never overlap.
- • Short regions. A single map lookup under a read lock is slower than the same lookup under a mutex, on every core count.
- • Balanced read/write ratios, where writers queue constantly and readers gain little.
- • Extreme read ratios, where an immutable snapshot removes the read-path cost entirely and the rwlock leaves an atomic write on every read.
- • On a single-threaded async runtime, where reader parallelism does not exist and the extra bookkeeping is pure cost.
- • When the team has not decided a preference policy, because the default differs between implementations and the failure mode differs with it.
- • Read/write ratio first. If it is below roughly 90% reads, stop — the answer is a mutex and no measurement of the lock itself will change that.
- • Region duration. If the protected region is shorter than a few hundred nanoseconds, the acquisition overhead dominates and the rwlock cannot win.
- • Benchmark against a plain mutex at your real core count and real read ratio. This comparison is cheap to run and frequently reverses the intuition that motivated the change.
- • Writer wait *maximum*, not mean, and the age of the most recently published generation. This is the only signal that catches starvation.
- • Read throughput as core count rises. If it flattens or regresses, the reader count is your bottleneck and sharding or snapshots are the way out.
- • Two acquisition modes mean every call site must choose correctly, and a read-mode acquisition around code that writes is a silent correctness bug no compiler catches.
- • The preference policy is a semantic decision that differs between implementations and is rarely documented in the calling code, so behaviour changes when the lock type is swapped.
- • The prohibition on upgrading must be known by everyone touching the code, and the natural refactor — "I already have the read lock, let me just take the write lock" — is exactly the deadlock.
- • Two extra failure modes to monitor (writer starvation, reader starvation) that a mutex simply does not have.
- • A plain mutex. Simpler, cheaper uncontended, no starvation policy, no upgrade trap. The default answer unless reads dominate heavily *and* the region is long.
- • An immutable snapshot swapped by one atomic reference store. Readers take nothing at all. The right answer for the routing table and for most configuration state. See Immutability as a Concurrency Strategy and Copy-on-Write as a Concurrency Strategy.
- • Sharding — many locks over disjoint parts of the structure — which reduces contention more than any policy tuning and keeps the simple primitive.
- • A concurrent data structure designed for the access pattern, where one exists and is well tested. See Lock-Free Is a Progress Guarantee, with its caveats about progress guarantees not being performance promises.
- • Read-copy-update style publication, where readers never synchronise and reclamation of the old version is deferred until no reader can hold it. Powerful and considerably harder to get right by hand.
Does a read/write lock actually pay?
Eight threads, one lock
Immutability lab
| # | Writer | Reader | State |
|---|---|---|---|
| 1 | account.a -= 10 | · | a=40 b=50 a+b=90 |
| 2 | · | read account.a, account.b | a=40 b=50 a+b=90 ✕ the reader observed a total of 90 — a state no writer ever intended |
| 3 | account.b += 10 | · | a=40 b=60 a+b=100 |
| 4 | · | read account.a, account.b | a=40 b=60 a+b=100 |
What people believe, and what is true
Reads are free with an rwlock.
Every read acquisition performs an atomic write to the shared reader count, invalidating that cache line on every other core. At high core counts that write is the bottleneck.
It is strictly better than a mutex for read-heavy workloads.
It is better only when reads dominate heavily *and* the region is long enough to amortise the extra acquisition cost. For short regions it loses at every read ratio.
The writer will get in eventually.
Under a reader-preferring policy with sustained read arrivals, there is no guarantee it ever does. That is permitted behaviour, and the symptom is stale data rather than a hung thread.
I hold the read lock, so I can just upgrade to write.
Most implementations forbid it, and the ones that allow it deadlock when two readers try simultaneously. Release, re-acquire exclusively, and re-validate what you read.