The question this answers
What does a shared write actually cost when every core has its own cache?
One inFlight gauge incremented on request entry and decremented on exit by sixteen threads on sixteen cores.
One counter, and — invisibly — exclusive ownership of the cache line holding it.
The gauge equals the number of requests currently in flight. Correctness is guaranteed by the atomic; the question is what that correctness costs.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Coherence is automatic, and it is not free
Two facts, held together. First: caches are coherent. You never need to do anything to make a write eventually visible to another core; the hardware guarantees all cores agree on the value of a location. This is why "flush the cache" is the wrong model for a barrier, as Memory Barriers Constrain Ordering, Not Caches says. Second: providing that guarantee costs messages between cores, and those messages are what a shared write actually buys.
The essential rule of thumb is that a line can be *read* by many cores at once, held in all of their caches simultaneously, at no ongoing cost — but it can be *written* by only one core at a time, and each write must first take exclusive ownership away from everyone else. Read-shared is nearly free. Write-shared is not.
This lesson stays at that level deliberately. The protocol states, the difference between snooping and directory-based schemes, and the interconnect topology are Computer Architecture material and are bridged rather than duplicated here. What you need to design concurrent code is the granularity — the line — and the asymmetry between reading and writing.
Access pattern determines cost
Four patterns cover almost everything, and the cost differences between them are large enough to determine an architecture. Thread-private data: no coherence traffic at all, because no other core ever wants the line. Read-mostly shared data: every core keeps a copy, and traffic occurs only on the rare write. Write-shared data: the line moves on every write, and throughput on that line caps regardless of core count. Falsely shared data: the same cost as write-shared, with none of the sharing — see False Sharing: Different Variables, Same Cache Line.
The design consequence is that the question to ask about a shared variable is not "is it synchronized?" but "how many cores write it, how often?". A configuration pointer read on every request and written once an hour is essentially free to share. A counter written on every request by every core is a scalability ceiling, and no choice of synchronization primitive changes that — atomic, mutex or lock-free all serialise on the same line.
This also explains a result that surprises people: replacing a mutex with an atomic on a heavily contended counter often does not help. The mutex was not the cost. The line was.
| Access pattern | What coherence does | Cost | Signal you would see |
|---|---|---|---|
| Thread-private | Nothing — no other core requests the line | L1 hit | Linear scaling; no coherence events |
| Read-mostly shared | A copy resides in every reader's cache; no transfer while nobody writes | L1 hit after the first read | Linear scaling; a traffic spike on each rare write |
| Write-shared | Every write takes exclusive ownership, invalidating all other copies | An interconnect round trip per write | Throughput flat or falling with core count; high CPU, low work |
| Falsely shared | Same as write-shared, on variables that are not logically shared | An interconnect round trip per write | Negative scaling in a workload with provably no sharing |
| Read-modify-write shared | Exclusive ownership per operation, and it cannot be batched away | The worst case: every operation is a transfer | A hot atomic instruction with superlinear cost growth in thread count |
Keeping the number without paying for it
The general technique is to convert a write-shared location into per-core write-private locations and pay the cost at read time instead. A sharded counter gives each thread its own line-aligned slot; increments are L1 hits with no ownership transfer, and a read sums the shards.
The price is stated honestly in the code below: reads become proportional to the shard count, and — more importantly — the sum is not a snapshot. Shards are read one at a time while others are being updated, so the total corresponds to no single instant. For a monitoring gauge that is fine and should be documented. For a value that gates admission — "reject if in-flight exceeds 100" — it may not be, and then you either accept the imprecision deliberately or keep the single hot counter and accept the ceiling. See Bounding Concurrency.
The same reasoning drives several patterns elsewhere in this domain: per-thread accumulation in False Sharing: Different Variables, Same Cache Line, work-stealing deques with per-worker queues in Work Stealing, and read-mostly configuration published by pointer swap in Copy-on-Write as a Concurrency Strategy. They are all the same move — make the common operation touch only lines this core owns.
1std::atomic<int64_t> inFlight{0};2 3void onEnter() { inFlight.fetch_add(1); } // every core, every request:4void onExit() { inFlight.fetch_sub(1); } // exclusive ownership transfer5 6int64_t current() { return inFlight.load(); } // exact, and cheap to read1struct alignas(64) Shard { std::atomic<int64_t> n{0}; };2Shard shards[NUM_SHARDS]; // one line each: no false sharing either3 4void onEnter(int id) { shards[id % NUM_SHARDS].n.fetch_add(1,5 std::memory_order_relaxed); }6void onExit(int id) { shards[id % NUM_SHARDS].n.fetch_sub(1,7 std::memory_order_relaxed); }8 9// COST, stated: O(NUM_SHARDS) per read, and the shards are read at10// different instants, so this total corresponds to no single moment.11// Correct for a monitoring gauge. Think hard before gating on it.12int64_t current() {13 int64_t t = 0;14 for (auto& s : shards) t += s.n.load(std::memory_order_relaxed);15 return t;16}The write path stops touching a line any other core wants, which is the entire cost. What you give up is a consistent snapshot: summing N shards read at N different instants produces a number that was never simultaneously true. That is an acceptable trade for observability and a deliberate decision for anything that makes a control decision from the value.
Key points
- Caches are coherent automatically. The cost of that guarantee is interconnect traffic, paid on writes to lines other cores hold.
- A line can be read by many cores at once for free; it can be written by only one at a time, and each write invalidates every other copy.
- The scalability question for a shared variable is how many cores write it and how often — not which synchronization primitive guards it.
- Replacing a mutex with an atomic on a hot counter often does not help, because the contended resource was the line, not the lock.
- The general fix is to make writes core-local and pay at read time: sharded counters, per-thread accumulation, per-worker queues.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • Each core has private L1 and usually L2 caches, with a shared last level; the hardware keeps them consistent without software involvement.
- • A line held for reading may be resident in many caches at once, with no ongoing traffic.
- • A core intending to write must obtain exclusive ownership, which invalidates every other cached copy of that line.
- • A subsequent read or write by another core must fetch the line from the current owner, costing an interconnect round trip rather than an L1 hit.
- • Because ownership is per line, the cost is determined by which line the variable sits in, not by how large or small the variable is.
- • Sixteen cores each fetch_add the same counter: the line moves sixteen times per round, and per-core throughput falls as cores are added. The count stays exact throughout.
- • Sixteen cores each fetch_add their own shard: no line ever moves, throughput scales, and a concurrent read of the total sums values from sixteen different instants.
- • Fifteen cores read a config pointer while one core writes it once an hour: the line is read-shared and effectively free for 3600 seconds, then transfers once.
- • Two cores alternate writing adjacent unrelated variables in one line: identical cost to the shared counter, with no logical sharing at all. See False Sharing: Different Variables, Same Cache Line.
- • One core writes the counter, fifteen read it every request: each write invalidates fifteen copies and each subsequent read refetches. Read-heavy does not rescue a write-shared line.
- • Promises: coherence. All cores will agree on the value of a location without any software action.
- • Promises: the cost is per line and per write, so making writes core-local removes it entirely.
- • Does NOT promise: ordering between different locations. Coherence is per location; ordering is the memory model's job. See What a Memory Model Defines.
- • Does NOT promise: that a sharded total is a consistent snapshot. Summing shards read at different instants produces a value that was never simultaneously true.
- • Does NOT promise: uniform cost. Same-socket, cross-socket and cross-die transfers differ by large factors. See NUMA: Not All Memory Costs the Same.
- • Does NOT promise: that any synchronization choice avoids it. Atomic, mutex and lock-free all serialise on the same line.
- • The contended resource is exclusive ownership of a line, and it is contended by every writing core on every write.
- • This contention involves no blocking and no waiting in the software sense, so it is invisible to lock-wait metrics and thread dumps.
- • It grows with core count, which makes it a scalability ceiling rather than a constant overhead — the symptom is that a bigger machine does not help.
- • Cross-socket traffic is substantially more expensive than within-socket, so the same code can behave very differently on two machines with the same core count. See Thread Affinity: Pinning, and What It Costs You.
- • Scalability ceiling — throughput flat or falling as cores are added, with CPU utilisation at 100%.
- • Misdiagnosis as lock contention, leading to a lock-free rewrite that changes nothing because the line was the bottleneck.
- • False sharing, which is this cost paid for no reason at all.
- • NUMA amplification — the same line contended across sockets, where each transfer costs several times more.
- • Sharded-read inconsistency — a control decision made from a summed total that no instant ever matched, producing limits that are violated under load.
- • It explains why replacing a mutex with an atomic often does nothing, which saves a great deal of wasted rewriting.
- • It gives a design rule that applies before any code is written: keep the hot path writing only to lines this core owns.
- • It explains why read-mostly shared configuration is cheap and why the copy-on-write publication pattern scales so well.
- • It converts "why does the 64-core machine not help?" from a mystery into a hypothesis with a specific test.
- • When it motivates sharding a counter that is written a hundred times a second, where the cost was never measurable and the read-consistency loss is real.
- • When it motivates reasoning about hardware in code that is nowhere near the throughput where any of this matters.
- • When approximate reads are adopted without documenting the tolerance, so a later change gates a decision on a number that was never exact.
- • Scaling curve first: measure throughput at 1, 2, 4, 8 and 16 threads. Flat or falling with CPU at 100% is the coherence signature.
- • Hardware counters where available:
perf c2con Linux identifies the specific cache line, the offsets within it, and the threads fighting over it. This is the definitive tool. - • Compare same-socket pinning against unpinned. A large improvement from pinning points at cross-socket coherence traffic. See Thread Affinity: Pinning, and What It Costs You.
- • Try the sharding experiment rather than reasoning about it: shard the counter and re-measure. It is a small change and settles the question.
- • Do not expect lock-wait metrics or thread dumps to show anything. Nothing is blocked; the threads are running and getting nothing done. See Hold Time, Wait Time, and the Ratio Between Them.
- • Sharding introduces a shard count to tune, a per-shard alignment requirement, and a read path that is O(shards).
- • The value's consistency semantics change from exact to approximate, which must be documented or it will be depended upon wrongly.
- • Performance now depends on machine topology, so a benchmark on one machine transfers poorly to another.
- • The reasoning is hardware-adjacent and does not appear in the code, so it needs a comment or it will be undone by a well-meaning simplification.
- • Do not share the value. Per-thread state combined at the end is the simplest and fastest answer where it fits. See Copy or Share?.
- • Make the shared data read-mostly by publishing immutable snapshots rather than mutating in place. See Copy-on-Write as a Concurrency Strategy and Immutability as a Concurrency Strategy.
- • Batch: update the shared counter once per hundred operations rather than once per operation, trading precision for a hundredfold traffic reduction.
- • Move the aggregation out of process — emit per-thread deltas to a metrics system that sums them. See What to Instrument in a Concurrent System.
- • Accept the ceiling. A single hot counter that caps at a throughput well above your requirement is a fine engineering answer, and cheaper than every alternative here.
Two counters, no lock, one cache line
struct Counters {
long a; // byte 0..7
long b; // byte 8..15 <- same 64-byte line as a
}; // sizeof == 16Eight threads, one lock
Why is 8 cores only 4.5×?
| workers | ideal | Amdahl only | realistic | limited by |
|---|---|---|---|---|
| 1 | 1.0× | 1.00× | 1.00× | none |
| 2 | 2.0× | 1.90× | 1.85× | serial |
| 4 | 4.0× | 3.48× | 3.19× | serial |
| 6 | 6.0× | 4.80× | 4.17× | serial |
| 8 | 8.0× | 5.93× | 4.17× | bandwidth |
| 10 | 10.0× | 6.90× | 4.17× | bandwidth |
| 12 | 12.0× | 7.74× | 4.17× | bandwidth |
| 14 | 14.0× | 8.48× | 4.17× | bandwidth |
| 16 | 16.0× | 9.14× | 4.17× | bandwidth |
What people believe, and what is true
I need a barrier to make my write visible to other cores.
Coherence makes it visible without any software action. A barrier orders your operations relative to each other; it does not cause visibility that would not otherwise happen.
The atomic is slow, so I should use a lock-free algorithm.
A lock-free algorithm writing the same line pays exactly the same coherence cost. The fix is to stop writing a shared line, not to change the primitive.
More cores will increase throughput.
On a write-shared line, more cores increase the number of ownership transfers per unit work. Throughput can decrease monotonically with core count.
Go deeper
Overview
Each core caches memory, and the hardware keeps the caches agreeing. A location many cores write forces the data to move between them constantly, which is what limits throughput.
Practical
Ask how many cores write each shared variable and how often. Read-mostly is cheap. Write-shared on the hot path is a ceiling, and the fix is layout and sharding, not a different lock.
Advanced
Convert write-shared into write-private plus a read-time combine, and state the consistency you gave up. Java's LongAdder is this pattern productised; a sharded gauge is the hand-rolled version.
Internals
The protocol tracks each line in one of a small set of states per cache, and a write requires transitioning to an exclusive state, invalidating other copies. The state machine, the snooping-versus-directory distinction and the interconnect are the architecture domain's material.