Orderingatomicread-modify-writelock prefixLL/SCfetch-and-add

Atomic Instructions: What the Hardware Actually Guarantees

An atomic read-modify-write is a single instruction that no other core can observe half-finished. That is a narrow and precise guarantee, and it is routinely mistaken for a much broader one about program correctness.

Follow the mechanism

Software view, hardware view

The gap between what you wrote and what the machine does is where this whole domain lives.

The question
What does the hardware actually promise when an operation is atomic, and what does it conspicuously not promise?
What you wrote
Atomic means safe. If I mark my shared variables atomic, my concurrent code is correct.
What the hardware does
An atomic read-modify-write is indivisible with respect to other observers of that one location, plus whatever ordering the instruction specifies. Two atomic operations in sequence are two separate indivisible events with an arbitrary gap between them.
The gap between "this instruction is indivisible" and "my logic is correct" is where a great deal of broken concurrent code lives. Atomics are a building material, not a safety property, and the most common misuse is treating a sequence of atomic operations as though the sequence were atomic.
Source→Compiler→Instructions→Front End→Execution→Registers→Caches→Memory→I/O→Behavior

The primitives, and how each ISA provides them

ISA-SPECIFICInstruction names, spurious-failure behaviour and whether an atomic implies a barrier are all per-architecture. x86-64's `LOCK` implying a full barrier is an x86 property, not a general one.

The useful atomic operations are read-modify-writes: exchange, fetch-and-add, and compare-and-swap. Each reads a location, computes a new value, and writes it back as one indivisible step. A plain load or store of a naturally-aligned word is also atomic in the narrow sense — it will not tear — but it cannot express "increment", which is where the interesting problems begin.

Architectures deliver these in two distinct styles. x86-64 uses a LOCK prefix on the instruction — LOCK XADD for fetch-and-add, LOCK CMPXCHG for compare-and-swap, with XCHG locked implicitly — and each such instruction also acts as a full memory barrier. AArch64 originally used load-linked/store-conditional pairs (LDXR/STXR): the load establishes a reservation, the store succeeds only if nothing has touched the line since, and the code retries on failure. ARMv8.1 added single-instruction LSE atomics (LDADD, CAS, SWP) that behave more like the x86 style. RISC-V offers both, with LR/SC and the A extension's atomic memory operations.

The distinction matters more than it looks. LL/SC can fail *spuriously* — an interrupt, a context switch or unrelated traffic to the same cache line can clear the reservation — so LL/SC code is always a retry loop, and forward progress is guaranteed only under constraints on what appears between the pair. Single-instruction atomics have no spurious failure. Compiling the same source to both is another job you want the compiler doing rather than yourself.

Atomic read-modify-write facilities by architecture
Operationx86-64AArch64 (ARMv8.0)AArch64 (ARMv8.1 LSE)RISC-V
ExchangeXCHG (implicitly locked)LDXR/STXR loopSWPAMOSWAP
Fetch-and-addLOCK XADDLDXR/STXR loopLDADDAMOADD
Compare-and-swapLOCK CMPXCHGLDXR/STXR loopCASLR/SC
Spurious failure possibleNoYes — reservation can be clearedNoYes for LR/SC, no for AMO
Implies a full barrierYesNo — ordering is requested separatelyNo — acquire/release variants existNo — ordering bits in the encoding

What "atomic" does not mean

The guarantee covers one instruction on one location. It says nothing about the relationship between two atomic operations, and that is the gap almost every misuse falls into. Reading an atomic counter, deciding something based on the value, and then writing an atomic result is three separate events with arbitrary interleaving between them — the classic check-then-act race, entirely intact despite every individual access being atomic.

This is why compare-and-swap exists as a primitive at all. Instead of reading, computing and writing as separate steps, CAS folds the decision into the write: change the value *only if* it is still what I observed. It converts check-then-act into a single indivisible operation, which is the whole reason lock-free algorithms are built on it rather than on atomic loads and stores. See Compare-and-Swap: The Primitive Everything Is Built On.

A second common overreach is treating atomicity as though it implied ordering. On x86-64 it happens to, because LOCK implies a full barrier. On AArch64 with LSE it does not — LDADD and LDADDA differ precisely in whether acquire ordering is requested. Code that relies on an atomic incidentally providing a barrier is relying on an x86 implementation detail, which is the pattern Hardware Memory Models Are Not Language Memory Models warns about.

Every access is atomic. The logic is still racy.
1// BROKEN: three atomic operations are not one atomic operation
2if (counter.load() < LIMIT) {        // atomic read
3    counter.fetch_add(1);            // atomic increment
4}                                    // another thread can push
5                                     // counter past LIMIT in the gap
6 
7// BROKEN: atomic, but lost updates
8int v = counter.load();              // atomic read
9v = transform(v);
10counter.store(v);                    // atomic write
11                                     // any concurrent update between
12                                     // the load and the store is lost
13 
14// CORRECT: fold the decision into the write
15int cur = counter.load();
16while (cur < LIMIT &&
17       !counter.compare_exchange_weak(cur, cur + 1)) {
18    // cur is updated with the observed value on failure; retry
19}
20 
21// CORRECT and cheaper when no condition is needed
22counter.fetch_add(1);                // one indivisible step

What atomics cost, and why the answer is "it depends who else wants the line"

An atomic read-modify-write must obtain the cache line in an exclusive state, because it cannot be indivisible if another core holds a writable copy. When the line is already exclusive to this core, that is inexpensive — on the order of a few times a normal store, plus whatever barrier semantics the instruction carries.

When another core holds the line, the cost changes character entirely. The line must be transferred, and if several cores are hammering the same location it ping-pongs between them, with every operation paying a coherence round trip. This is why an atomic counter incremented by many threads scales so poorly, and why the cure is usually to stop sharing the line — per-core counters aggregated later, or padding to separate lines as False Sharing: Independent Data, Shared Line describes.

A practical ordering follows for choosing primitives. Prefer fetch_add to a CAS loop where the operation permits it: fetch-and-add always succeeds, while a CAS loop under contention can retry many times, and each retry is another coherence transaction. Prefer per-core or sharded state to a single hot atomic. And measure, because the cost is dominated by contention rather than by the instruction — the same LOCK XADD can be nearly free or catastrophically expensive depending only on how many cores want that line.

Relative cost of an atomic read-modify-write by contention state — 1 unit ≈ one ordinary store to a line already held exclusiveMICROARCH-SPECIFIC
Plain store, line exclusive×1
Atomic RMW, line already exclusive to this core×5
Atomic RMW, line shared with one other core×50
Atomic RMW, line contended by many cores×250
CAS retry loop under heavy contention×500
Ratios, not times. Absolute latencies depend on the processor, its clock, the memory it is attached to and what else is running — publishing them would be wrong everywhere except one machine. The bars are log-scaled, so each step is larger than it looks.
Plain store, line exclusive — The baseline
Atomic RMW, line already exclusive to this core — Uncontended: cheap enough to ignore in most code
Atomic RMW, line shared with one other core — Ownership transfer on every operation
Atomic RMW, line contended by many cores — Line ping-pongs; cost grows with core count
CAS retry loop under heavy contention — Each failed attempt is another coherence round trip

Key points

  • Atomic means indivisible with respect to other observers of that location, plus whatever ordering the instruction specifies — nothing more.
  • Two atomic operations in sequence are not atomic; check-then-act races survive completely intact.
  • x86-64 uses LOCK-prefixed instructions that also imply a full barrier; AArch64 uses LL/SC or LSE atomics where ordering is requested separately.
  • LL/SC can fail spuriously, so it is always a retry loop — a difference with real consequences for hand-written code.
  • Atomic cost is dominated by contention, not by the instruction: uncontended is cheap, a ping-ponging line is orders of magnitude worse.

Follow the mechanism

The path through the machine, hop by hop — and the conclusions it invites that are wrong.

  1. 1
    Atomic instruction → cache: the core requests the target line in an exclusive state, since indivisibility requires no other writable copy exists.
  2. 2
    Coherence → other cores: their copies are invalidated, transferring ownership to this core.
  3. 3
    Core → line: the read-modify-write executes with no window in which another core can observe an intermediate state.
  4. 4
    Instruction → ordering: on x86-64 the LOCK prefix additionally acts as a full barrier; on AArch64 ordering is requested by choosing an acquire or release variant.
  5. 5
    Other cores → retry: any core that wanted the same line must now re-acquire it, which is the ping-pong that makes contended atomics expensive.
What people conclude from this — wrongly
  • • "Every variable is atomic, so the code is thread-safe" — atomicity is per-operation; multi-step logic needs CAS or a lock.
  • • "Atomics are lock-free so they are fast" — an atomic on a contended line costs far more than an uncontended mutex acquisition.
  • • "The atomic includes a barrier" — true on x86-64, false on AArch64 LSE unless the ordered variant is chosen.
  • • "CAS is the general-purpose atomic, so use it everywhere" — fetch_add is cheaper where it applies, because it cannot fail and cannot retry.

Consequences, controls and cost

What it causes
  • • A shared atomic counter becomes a scalability bottleneck well before it becomes a correctness problem, and the profile shows the cost as cache traffic rather than as a lock.
  • • Check-then-act logic built from individually atomic operations is racy, and the bug survives review because every line looks defended.
  • • Code relying on atomics to provide ordering incidentally works on x86-64 and fails on AArch64 with LSE atomics.
  • • Hand-written LL/SC loops that do too much between the load and the store may fail to make forward progress.
What you can do
  • • Use `fetch_add` and friends rather than CAS loops wherever the operation allows — they always succeed and cost one transaction.
  • • Shard hot atomics into per-core or per-thread state and aggregate on read, removing the contention rather than optimising it.
  • • Pad independently-updated atomics onto separate cache lines so they do not contend accidentally — see [[false-sharing]].
  • • State the ordering you need explicitly rather than relying on an atomic to imply a barrier, which is an x86-only behaviour.
  • • Prefer a mutex to a hand-rolled atomic protocol unless profiling shows the lock is the problem; an uncontended mutex is itself only an atomic operation.
How to see it
  • • Disassemble to see which instruction your atomic became: `LOCK XADD`, an `LDXR`/`STXR` loop, or an LSE `LDADD` — each has different cost and ordering.
  • • Scale a benchmark across core counts; an atomic bottleneck shows as throughput that flattens or falls as cores are added.
  • • Sample cache-coherence counters for the contended line to confirm the cost is ownership transfer rather than the instruction itself.
  • • Compare a CAS retry loop against `fetch_add` for the same operation under contention and count retries.
What it costs
  • • Atomics avoid the context-switch cost of a contended lock but replace it with coherence traffic that can be worse at high core counts.
  • • Sharding removes contention at the cost of memory and of a more expensive read that must aggregate.
  • • Stronger ordering on an atomic is free on x86-64 and costs a real instruction on AArch64, so portable code pays somewhere.

Scope

§224 — what these claims are specific to.

What these claims are specific to
  • ISA-SPECIFICInstruction names, spurious failure and implied ordering differ per architecture. LOCK implying a full barrier is specific to x86-64.
  • MICROARCH-SPECIFICThe relative costs are illustrative ratios that vary with core count, interconnect topology and cache hierarchy. Only the ordering of magnitudes transfers.

Misconceptions

Claim
“Making a variable atomic makes code using it thread-safe.”
Reality
It makes each individual access indivisible. Any logic spanning more than one access — read, decide, write — is still racy, which is precisely why compare-and-swap exists.
Claim
“Atomic operations are cheap because they are a single instruction.”
Reality
Cost is dominated by cache-line ownership, not instruction count. An uncontended atomic is cheap; one contended by eight cores can cost hundreds of times a plain store.
Claim
“Lock-free is always faster than a mutex.”
Reality
An uncontended mutex acquisition is itself one atomic operation. Under heavy contention a lock-free retry loop can generate far more coherence traffic than a lock that parks its waiters.

Apply it

🔍 Adding Threads Made It Slower

Where the rest of this lives

Concurrency & Parallelism
Building correct algorithms from atomic primitives

This lesson covers what the instruction guarantees. Turning those primitives into a correct queue, counter or lock — and proving linearizability — is the concurrency domain's work.

← Hardware Memory Models Are Not Language Memory ModelsCompare-and-Swap: The Primitive Everything Is Built On →