The question this answers
When is burning CPU in a loop cheaper than letting the scheduler put the thread to sleep?
Acquiring a lock that protects a four-field struct update, contended by threads on other cores, in a hot path executed millions of times a second.
The lock word itself — a single machine word that every contending core reads and writes, and therefore a single cache line that bounces between them.
Exactly one thread is inside the critical section at any moment, and every thread that wants to enter eventually does.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
The trade is spin cost against switch cost
When a thread cannot acquire a lock it has two options. It can block — hand itself to the scheduler, which parks it, runs something else, and later wakes it — or it can spin, re-reading the lock word until it becomes free. Blocking costs a trip into the kernel, a context switch out, a context switch back, and the loss of whatever cache and TLB warmth the thread had. Spinning costs exactly as much CPU as the wait lasts, and nothing else.
So the comparison is direct: if the expected wait is shorter than the round trip through the scheduler, spinning is cheaper — not marginally, but by a wide margin, because the spinner also keeps its cache warm and resumes in the same nanosecond the lock is released rather than whenever it is next scheduled. If the expected wait is longer, spinning is a catastrophe, because a spinning thread occupies a core doing nothing while the holder may be waiting for that very core.
That last clause is the asymmetry that makes this dangerous. The cost of spinning too long is not "we wasted some CPU" — it is that the spinner can prevent the holder from making progress at all, when there are more runnable threads than cores. On a single-core machine, or an oversubscribed one, a pure spin lock can hold a core for a full scheduling quantum while the holder sits in the run queue. See Busy Waiting and Oversubscription.
What a real spin lock actually looks like
A naive spin lock — a bare compare-and-swap in a tight loop — is worse than the idea suggests, for two reasons that are both about the cache. Every failed attempt is a read-modify-write, which takes the cache line exclusively and invalidates it in every other core, so N spinners generate N times the coherence traffic and slow down the holder trying to release. And on a hyperthreaded core, a tight loop starves the sibling thread of issue slots.
Production implementations fix both. Test-and-test-and-set spins on a plain read (which can be satisfied from a shared cache line, generating no coherence traffic) and only attempts the atomic write when the read suggests it might succeed. A CPU pause hint tells the processor this is a spin loop, which reduces power, avoids a memory-order violation penalty on exit, and yields issue slots to the sibling thread. Exponential backoff spreads the retries so the release is not swamped.
And almost every real lock is adaptive: it spins for a bounded number of iterations, then parks. That is the honest synthesis of this lesson — it captures the short-wait win and bounds the long-wait disaster. Most standard-library mutexes already do this, which is the single strongest argument for not writing your own. See Mutexes: What They Protect and What They Do Not.
1class AdaptiveLock {2 std::atomic<bool> held{false};3 4 public:5 void lock() {6 for (int i = 0; i < kSpinLimit; ++i) {7 // Test: a plain load, satisfiable from a SHARED cache line.8 // No coherence traffic while the lock stays held.9 if (!held.load(std::memory_order_relaxed)) {10 // Test-and-set: only now do we take the line exclusively.11 if (!held.exchange(true, std::memory_order_acquire)) return;12 }13 _mm_pause(); // x86 PAUSE: power, sibling thread, pipeline14 }15 park_until_free(); // bounded spin exhausted -> hand off to the scheduler16 }17 18 void unlock() { held.store(false, std::memory_order_release); }19};20// kSpinLimit is not a universal constant. It depends on the critical section21// length, the core count, whether the machine is oversubscribed, and the cost22// of a switch on this kernel. Measure it; do not copy it from an article.Where spinning is right, and where it is indefensible
The conditions that make spinning correct are specific and checkable. The critical section must be genuinely short — a handful of instructions, no I/O, no allocation, no second lock. There must be at least as many cores as runnable threads, so a spinner is not occupying a core the holder needs. The lock holder must not be preemptible while held, or at least very unlikely to be preempted. And the wait must be uncontended enough that the spin usually succeeds within a few iterations.
Kernel code satisfies these routinely, which is why spin locks are standard there: interrupt handlers cannot sleep, critical sections are counted in instructions, and preemption can be disabled while the lock is held. Lock-free data structure retry loops satisfy them too. Real-time systems use them to avoid scheduler jitter.
Application code almost never satisfies them, and the failure modes are severe. Spinning in a virtual machine is particularly bad: the hypervisor may deschedule the vCPU holding the lock, so every spinner burns its full quantum waiting for a thread that is not running at all — the lock-holder preemption problem, and the reason paravirtualized spin locks exist. Spinning inside a container with a CPU quota is the same failure wearing different clothes: the spinner consumes the cgroup budget the holder needs, and the whole container is throttled.
| Situation | Spin? | Why | What goes wrong if you get it backwards |
|---|---|---|---|
| Critical section is a few instructions, cores >= threads | Yes, bounded | Wait is far shorter than a context switch and the cache stays warm | Parking costs more than the wait itself, on every acquisition |
| Critical section contains any I/O | Never | The wait is milliseconds; a switch is microseconds | A core pinned for the entire duration of a network call |
| More runnable threads than cores | Never | The spinner occupies a core the holder needs to finish | Progress stops until the quantum expires — worst case on one core |
| Inside a VM or a CPU-quota container | Only with paravirt support | The holder's vCPU may be descheduled entirely | Every spinner burns a full quantum waiting for a thread that is not running |
| Interrupt context / cannot sleep | Yes — required | Blocking is not available at all in this context | A sleep in interrupt context is a kernel bug, not a slow path |
| CAS retry loop in a lock-free structure | Yes, with backoff | The retry is the algorithm; the expected retry count is small | Unbounded retries under contention become livelock — see Livelock |
| Application-level mutex, unknown workload | Use the standard mutex | It already spins briefly and then parks, tuned by people with benchmarks | A hand-rolled spin lock that is faster in the microbenchmark and worse in production |
Key points
- The comparison is expected wait time against context-switch cost. Shorter wait: spin. Longer wait: park. There is no universal answer.
- A spinning thread is not merely wasting CPU — on an oversubscribed machine it can prevent the lock holder from running at all.
- A naive CAS spin loop is worse than it looks: every failed attempt takes the cache line exclusively and slows the holder down.
- Test-and-test-and-set plus a pause hint plus backoff plus a bounded spin then park is what real implementations do, and your standard mutex probably already does it.
- Spinning in a VM or a CPU-quota container is a distinct hazard: the holder may be descheduled entirely, so no amount of spinning helps.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • A thread attempts an atomic acquire of the lock word — a compare-and-swap or an exchange.
- • On failure, instead of entering the kernel it loops, re-reading the word with a plain load so the line can stay shared.
- • A pause or yield hint per iteration reduces power draw, releases issue slots to a hyperthread sibling, and avoids a pipeline penalty on exit.
- • When the plain read suggests the lock is free, the thread retries the atomic operation; only one will succeed.
- • An adaptive lock counts iterations and, past a threshold, parks itself with the scheduler — converting to a blocking lock for the long tail.
- • The release is a plain store with release ordering, which is what makes the critical section's writes visible to the next acquirer.
- • Short section, spinning wins: T2 attempts, fails, spins 40 iterations (~100 ns), T1 releases, T2 acquires immediately with a warm cache — total overhead well under a context switch.
- • Short section, blocking loses: T2 attempts, fails, enters the kernel, is parked; T1 releases 100 ns later; T2 is woken and rescheduled some microseconds later with a cold cache — the wait was 100 ns and the mechanism cost several microseconds.
- • Long section, spinning loses: T2 spins for 40 ms while T1 completes a network call inside the lock; one core is fully occupied doing nothing, and every other runnable thread is delayed.
- • Oversubscribed: one core, T1 holds the lock and is preempted; T2 is scheduled and spins for its entire quantum; T1 cannot run to release it; no progress occurs until T2's quantum expires.
- • Virtualized: T1 holds the lock and its vCPU is descheduled by the hypervisor for 3 ms; four spinners on other vCPUs burn 12 ms of CPU collectively, and the host sees high utilization with zero work done.
- • Naive CAS storm: eight cores spin with an unconditional exchange; the lock line ping-pongs between all eight caches; T1's release store itself must acquire the line and is delayed by the contention it caused.
- • A spin lock guarantees mutual exclusion, exactly as a blocking mutex does — the correctness property is identical.
- • It guarantees the fastest possible handoff when the wait is very short, because the waiter is already running.
- • It does NOT guarantee fairness. A simple spin lock is barging by nature, so a waiter can be overtaken indefinitely — ticket and MCS locks exist to fix this at extra cost.
- • It does NOT guarantee progress on an oversubscribed system, because the spinner can starve the holder.
- • It does NOT bound CPU consumption; the cost is exactly the wait, and the wait is not bounded by the lock.
- • A bounded adaptive spin guarantees the disaster is capped at the spin limit; it does NOT make the spin limit correct for your workload.
- • Nothing about spinning makes the critical section faster — it changes only the cost of waiting for it.
- • Contention on a spin lock is contention on a single cache line, and it scales badly: N spinners generate coherence traffic proportional to N, which slows the holder.
- • Test-and-test-and-set reduces this substantially by keeping the line in shared state while the lock is held, generating traffic only at the moment of release.
- • Release itself becomes contended: the holder's store must take the line exclusively, competing with every spinner attempting an atomic operation.
- • Backoff spreads retries in time, which reduces the storm at the cost of latency for the eventual winner.
- • Queue-based locks (ticket, MCS) move each waiter onto its own cache line, converting an N-way storm into a chain of handoffs — the standard fix at high core counts, at the cost of a more complex lock.
- • CPU burn with no progress, presenting as high utilization and flat throughput.
- • Lock-holder preemption: the holder is descheduled while spinners occupy the cores, so nothing advances until a quantum expires.
- • Coherence storm from naive CAS spinning, where the contention itself delays the release.
- • Starvation under a barging spin lock, where an unlucky waiter is repeatedly overtaken.
- • Livelock in a retry loop with no backoff, where contenders keep colliding and none completes.
- • Priority inversion made worse: a low-priority holder cannot be scheduled because a high-priority spinner occupies the core.
- • Container CPU throttling, where the spinner consumes the cgroup quota that the holder needed.
- • Power and thermal cost on battery-backed or density-constrained deployments, which is a real operational concern and never shows up in a benchmark.
- • Critical sections measured in tens of nanoseconds, where a context switch is orders of magnitude more expensive than the wait.
- • Kernel and driver code where sleeping is not permitted and preemption can be disabled while the lock is held.
- • Lock-free retry loops, where spinning is the algorithm rather than a waiting strategy. See Compare-and-Swap and the Retry Loop.
- • Real-time paths where scheduler jitter is unacceptable and a bounded spin gives predictable latency.
- • Dedicated cores with at least as many cores as runnable threads, where a spinner is not stealing time from anybody who needs it.
- • Any critical section containing I/O, allocation, a second lock, or an unbounded loop.
- • Oversubscribed machines, containers with CPU quotas, and virtualized environments without paravirtualized lock support.
- • Application code generally, where the standard mutex already implements a better-tuned version of the same idea.
- • High core counts with a naive implementation, where the coherence storm makes the lock slower as you add cores.
- • Anywhere power draw matters, since a spinner is indistinguishable from useful work to every power-management heuristic.
- • Spin iterations before acquisition, as a distribution — if the tail routinely reaches your spin limit, the spin is not paying and the limit is hiding it.
- • CPU time attributed to lock acquisition, which separates "busy" from "productive" in a way that utilization alone cannot.
- • Critical section duration distribution, which is the input that decides the whole question and is almost never measured before choosing.
- • Run-queue length versus core count, the direct test of whether spinning is even permissible here.
- • Steal time or hypervisor-reported vCPU preemption, which tells you whether lock-holder preemption is live in your environment.
- • Throughput against core count: a spin lock whose throughput falls as you add cores has a coherence problem, not a tuning problem.
- • A correct spin lock needs the right memory ordering on acquire and release, a pause hint, backoff and a spin bound — four things, each easy to omit and none of them visible in testing.
- • The spin threshold is workload-, machine- and deployment-specific, so it is a tuning parameter that will be wrong somewhere.
- • Behaviour changes qualitatively between bare metal, VM and container, so a correct configuration in one environment is a pathology in another.
- • Fairness, if you need it, means a queue-based lock and a substantial jump in implementation complexity.
- • The debugging signature is unhelpful: high CPU with low throughput and no lock-wait metric, because nobody was ever blocked.
- • The standard library mutex, which on most platforms already spins briefly and then parks. This is the right answer in application code essentially always. See Mutexes: What They Protect and What They Do Not.
- • A lock-free structure, when the critical section is a single word update that compare-and-swap can express directly. See Lock-Free Is a Progress Guarantee.
- • Shrinking the critical section until contention disappears, which removes the question rather than answering it. See Lock Scope: What You Hold It Across.
- • Sharding the lock so contenders rarely meet, which is usually a bigger win than any lock implementation change.
- • A condition variable or an event, when the wait is genuinely long and the thread should not be running at all. See Condition Variables: Waiting Until a Predicate Is True.
Eight threads, one lock
compare_exchange in a loop — retries, and the pointer that lied
do {
old = counter.load(); # 1 read
next = old + 1; # compute off to the side
} while (!counter.compare_exchange(old, next)); # swap only if unchanged| # | T1 — pop() via CAS | T2 — another thread | State |
|---|---|---|---|
| 1 | old ← head (= A) | · | head=A stack=A→B→C |
| 2 | · | pop() → A | head=B stack=B→C |
| 3 | · | pop() → B | head=C stack=C |
| 4 | · | push(A) | head=A stack=A→C |
| 5 | CAS(head, A, B) → SUCCESS | · | head=B stack=B→ freed ✕ head now points at B, which was popped and freed. Node C has vanished from the stack and T1 returned a node it never observed being on top. |
| 6 | return A to the caller | · | head=B stack=corrupt |
More workers than cores
What people believe, and what is true
Spin locks are always wrong — busy waiting wastes CPU.
Busy waiting for longer than a context switch wastes CPU. Busy waiting for less than one saves it, and saves the cache warmth too. The kernel uses spin locks constantly for exactly this reason. Context decides.
Spin locks are faster than mutexes.
They are faster to acquire when the wait is very short and catastrophically slower when it is not. Most standard mutexes already spin briefly before parking, so the comparison is usually between a tuned adaptive lock and your untuned one.
A tight CAS loop is a spin lock.
It is a spin lock that generates maximum coherence traffic. Spin on a plain load and only attempt the atomic when it looks free, or the spinners will slow down the holder they are waiting for.
Adding cores makes spin locks scale better.
A naive spin lock scales *worse* with core count, because contention is on one cache line and every additional spinner adds traffic. Queue-based locks exist precisely because of this.
Go deeper
Overview
Instead of sleeping until a lock is free, keep checking in a loop. Worth it only when the wait is shorter than the cost of sleeping and waking, which in application code it rarely is.
Practical
Use your standard mutex. It almost certainly spins briefly then parks, which is the correct policy, tuned by people with benchmarks on the platforms you deploy to.
Advanced
If you must implement one: test-and-test-and-set, a pause hint, exponential backoff, and a bounded spin that falls back to parking. Then verify the environment assumption — cores at least equal to runnable threads, and no hypervisor or cgroup that can deschedule the holder.
Internals
Spinning is a cache-coherence problem in disguise. Every atomic attempt takes the line in exclusive state and invalidates every other copy; a plain-load spin can keep the line shared and generate no traffic until the release. At high core counts this is why queue locks that give each waiter its own line beat every centralized design.