The question this answers
If every thread can reach every object, which of those objects actually needs protecting, and how do I know?
A request handler running on eight threads, maintaining an in-memory rate-limit table: for each API key, a request count and a window start timestamp.
The rate-limit map itself, and — separately and more importantly — the two-field counter object inside each entry. Also the process's file descriptors, its logger, its module-level configuration and every default-mutable object anybody captured in a closure.
For each API key, count equals the number of requests admitted in the window that began at windowStart, and no request is admitted once count has reached the limit.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
What is private, and what is not
A thread gets its own stack, its own registers and its own program counter. That is the complete list. The heap, the globals, the loaded code, the file descriptors, the signal handlers and the memory mappings all belong to the process, and every thread in it sees the same ones. [[threads-intro]] and [[process-memory-layout]] cover the mechanics; the design consequence is worth stating bluntly.
Local variables holding primitives are private because they live on the stack. Local variables holding *references* are private references to shared objects — which is the distinction that produces most accidental sharing. Passing an object into a thread makes the reference local and the object shared, and nothing in most languages' syntax distinguishes that from passing a copy.
So the practical question is never "is this variable shared". It is "can two threads reach this object, and does either write to it". If the answer is yes and yes, it needs an invariant and a protection. If either is no — the object is thread-confined, or it is immutable — you need nothing, and that is the cheapest correctness there is. See [[immutability]] and [[thread-safety]].
The invariant spans two fields, so protecting one is not enough
The rate-limit entry has two fields and one invariant that ties them together: count is meaningful only relative to windowStart. Any code that resets the window and the count must do both, and no other thread may observe the moment between them.
The schedule below is what happens when it can. Thread A sees an expired window and starts a reset; thread B reads count after A wrote the new windowStart but before A zeroed count. B sees the old count against the new window, concludes the client is over its limit, and rejects a request that should have been admitted. Then A finishes the reset, and the count is wrong for the rest of the window.
Note what would not have helped. Making count an atomic integer makes each read and each write indivisible and leaves this bug completely intact, because the invariant spans two fields and two operations. Using a concurrent map makes map operations safe and leaves this bug intact for the same reason. The unit of protection is the invariant, not the field. [[atomics-are-not-magic]] and [[finding-the-critical-section]] are the follow-ups.
| # | Thread A — request for key k9 | Thread B — request for key k9 | State |
|---|---|---|---|
| 1 | read entry k9 → { count: 100, windowStart: 12:00:00 } | · | count=100 windowStart=12:00:00 now=12:01:03 |
| 2 | window expired (60s elapsed) → begin reset | · | count=100 windowStart=12:00:00 |
| 3 | write windowStart = 12:01:03 | · | count=100 windowStart=12:01:03 |
| 4 | · | read entry k9 → { count: 100, windowStart: 12:01:03 } | count=100 windowStart=12:01:03 |
| 5 | · | window fresh, count >= 100 → reject with 429 | count=100 windowStart=12:01:03 rejected=1 ✕ A request is rejected although zero requests have been admitted in the current window. count and windowStart describe different windows. |
| 6 | write count = 1 (this request admitted) | · | count=1 windowStart=12:01:03 rejected=1 |
| 7 | · | client retries; admitted normally | count=2 windowStart=12:01:03 rejected=1 |
count atomic does not fix it. Using a concurrent map does not fix it. The critical section is "read entry, decide, update entry" as one unit, because that is the span of the invariant.The fix, and what the fix costs
The corrected version below does the obvious thing — takes a per-key lock around the whole read-decide-update sequence — and it is worth being precise about why each detail is there. The lock is per key, not global, so unrelated API keys do not serialise against each other: this is sharding the lock, and it is the difference between a rate limiter and a bottleneck. The critical section contains no I/O and no allocation of consequence, because a lock held across a slow operation converts one request's latency into every waiter's. And the entry is mutated only inside the lock, with nothing escaping.
The costs are real and should be named. Every request now takes and releases a lock, which is tens of nanoseconds uncontended and rather more when a hot key is being hammered by eight threads at once — that hot key is now a serialisation point, and [[contention-costs]] is where it goes when it becomes a problem. The per-key lock map is itself shared state needing its own thread-safe construction, and creating a lock for a key that appears once is a small memory leak unless entries are evicted.
The alternative worth considering before any of this: do not share the counter. Give each thread its own counter and reconcile periodically, accepting a bounded over-admission. That is [[copy-vs-share]] reasoning, and for rate limiting it is frequently the better answer, because the requirement is usually "approximately 100 per minute" and not "exactly 100".
1// A concurrent map. Atomic counts. Still wrong.2const limits = new ConcurrentMap<string, { count: AtomicInt, windowStart: number }>()3 4function admit(key: string): boolean {5 const e = limits.get(key) // atomic map read6 if (Date.now() - e.windowStart > 60_000) {7 e.windowStart = Date.now() // <-- another thread can read8 e.count.set(0) // between these two lines9 }10 if (e.count.get() >= 100) return false // atomic read of a field that11 e.count.increment() // is now inconsistent with12 return true // windowStart13}14// Three atomic operations and a concurrent map protecting an invariant15// that spans all of them. Each step is indivisible; the sequence is not.1const locks = new ConcurrentMap<string, Mutex>() // one lock per key, not one global2const limits = new Map<string, { count: number, windowStart: number }>()3 4function admit(key: string): boolean {5 const lock = locks.computeIfAbsent(key, () => new Mutex())6 return lock.withLock(() => { // critical section = the whole7 let e = limits.get(key) // read-decide-update sequence8 if (!e) { e = { count: 0, windowStart: Date.now() }; limits.set(key, e) }9 if (Date.now() - e.windowStart > 60_000) {10 e.windowStart = Date.now(); e.count = 0 // both fields, atomically w.r.t. readers11 }12 if (e.count >= 100) return false13 e.count += 114 return true15 }) // no I/O, no logging, no awaits inside16}17// Cost: a lock per request; a hot key serialises its own traffic across all18// eight threads; the lock map needs eviction or it grows with the key space.Individually atomic operations do not compose into an atomic sequence. The invariant ties count to windowStart, so the critical section must be the whole read-decide-update, and the lock must be per key so unrelated keys do not serialise. What it costs: uncontended lock overhead on every request, a serialisation point on hot keys, and a lock map that must be bounded.
Key points
- A thread privately owns a stack, registers and a program counter. Everything else in the process is shared with every other thread.
- A local variable holding a reference is a private reference to a shared object, and nothing in the syntax marks the difference.
- The question is never "is this variable shared" but "can two threads reach this object, and does either write to it".
- Thread-confined and immutable objects need no protection at all, and arranging for that is the cheapest correctness available.
- The unit of protection is the invariant, not the field. A two-field invariant is not protected by making each field atomic.
- Concurrent collections make individual operations atomic. They do not make your read-decide-update sequence atomic.
- Shard the lock to the granularity of the invariant. One global lock around a per-key invariant turns a rate limiter into a queue.
- Never hold a lock across I/O, logging or a suspension point — that converts one operation's latency into every waiter's.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • The process creates a thread; the kernel allocates a stack and a schedulable entity that shares the process's page tables. See
[[threads-intro]]. - • The new thread begins executing a function with its own stack frame; any reference passed to it points into the same shared heap.
- • Reads and writes to shared objects go through the same virtual addresses, and on a multi-core machine they can genuinely overlap.
- • A mutex serialises entry to a region: at most one thread inside, and the release/acquire pair also establishes visibility of the writes made inside. See
[[mutex]]and[[happens-before]]. - • The scheduler may preempt a thread between any two instructions, so any read-modify-write that is not explicitly atomic is exposed.
- • A reads the entry, decides, updates, releases; B does the same afterwards — the serialised schedule the lock guarantees, and the only one that preserves the invariant.
- • A writes windowStart, B reads both fields, A writes count — the torn-invariant schedule above, producing a spurious rejection.
- • A and B both see count = 99 and both admit, so 101 requests are admitted in the window. A pure lost-update on the counter, and the one people expect.
- • A holds the per-key lock and makes a 40 ms Redis call inside it; seven other threads with the same key block for 40 ms each. The lock is correct and the latency is now serialised. See
[[lock-scope]]. - • A creates a lock for key k9 while B does the same and the map put is not atomic: two distinct lock objects for one key, so both threads "hold the lock" and the protection silently does nothing.
- • The runtime guarantees each thread has a private stack. It guarantees nothing about any object either thread can reach.
- • A mutex guarantees mutual exclusion for the region it encloses, and — in a language with a memory model — that writes made inside are visible to the next thread that acquires it.
- • It does not guarantee fairness. A thread can be starved by others repeatedly winning the lock unless the implementation promises otherwise. See
[[fairness]]. - • It does not guarantee anything about code outside the region, including a caller that reads the same fields without taking the lock.
- • A concurrent collection guarantees per-operation atomicity and nothing about sequences of operations. This is the single most common misreading of a thread-safety guarantee.
- • A hot API key serialises all eight threads onto one lock, so its throughput is one thread's worth no matter how many cores exist. See
[[hot-keys]]. - • A single global lock instead of per-key locks serialises every request in the service, which is the same bug with a much larger blast radius.
- • The shared map itself is contended on insert, and on resize far more so.
- • Adjacent counters in one array or one object can share a cache line, so unrelated keys ping-pong it between cores — contention with no lock involved. See
[[false-sharing]].
- • Lost update: two threads read the same count and both write, so one admission disappears.
- • Torn invariant: a reader observes fields updated at different times and acts on a state that never validly existed.
- • Data race: unsynchronised conflicting access with at least one write. In C++ this is undefined behaviour, not merely a wrong value. See
[[data-races]]. - • Deadlock, once there is more than one lock and the acquisition order is not fixed. See
[[lock-ordering]]. - • Lock convoy on a hot key: every thread queues, and the queue itself becomes the dominant latency. See
[[lock-convoy]]. - • Silent non-protection: two lock objects for one logical resource, so mutual exclusion never happens and everything looks correct in review.
- • Compute-bound work that genuinely must share a large mutable structure — an index, a cache, a simulation grid — where copying it per worker is not affordable.
- • Servers with tens to low hundreds of concurrent operations, where a stack per operation is affordable and the programming model is simpler than async.
- • Any runtime where threads occupy separate cores and the work partitions poorly enough that message passing would be awkward.
- • Tens of thousands of concurrent operations: a stack each is gigabytes, and tasks or an event loop are the right shape. See
[[tasks-vs-threads]]. - • Waiting-bound work at scale, where threads spend their lives blocked and you pay stacks and switches for nothing.
- • Code that can crash the process — a native decoder, a foreign function — where one bad input takes down every in-flight request.
- • CPython, for CPU-bound work, where threads do not execute bytecode on separate cores. See
[[python-threads-vs-processes]].
- • Lock wait time separately from lock hold time. Wait tells you about contention; hold tells you whether the critical section is too big. See
[[lock-wait-metrics]]. - • Contended-acquisition rate per lock. A lock that is never contended costs almost nothing and is not worth optimising.
- • Thread dumps during a stall: the distribution of RUNNABLE, WAITING and BLOCKED frames names the problem in one look. See
[[thread-dumps]]. - • A race detector — TSan, Helgrind, the Go race detector — on the test suite. It finds data races that have not yet produced a symptom. See
[[race-detectors]]. - • Per-key rejection rate against per-key admission rate, which is what would have surfaced the torn-invariant bug as "429s with count zero".
- • Every shared object acquires a documented invariant and a documented protection, and both must survive every future edit.
- • Correctness becomes non-local: a function is correct only in the context of what other threads may do between its statements.
- • More than one lock means a lock ordering, which must be written down and enforced, because deadlock is a global property nobody sees locally.
- • Lock granularity is a permanent tuning axis — too coarse serialises, too fine deadlocks and costs memory — with no correct answer independent of the workload.
- • The debugging toolchain expands: thread dumps, lock profilers, race detectors and stress tests are all now part of the project.
- • Do not share. Thread-confine the state, or give each thread its own copy and reconcile — for approximate counters this is both simpler and faster. See
[[copy-vs-share]]. - • Make it immutable. An immutable object needs no lock, and replacing a whole entry with a compare-and-swap can be simpler than mutating two fields.
- • Move the invariant to a store that has atomic primitives: a Redis
INCRwith a TTL is one round trip and one atomic operation, and it also works across processes. - • Message passing: one owner thread for the rate-limit table and a channel to it. No locks, no interleavings, at the cost of a hop. See
[[actor-model]]. - • Async on one thread, when the work is waiting-bound — though note this removes data races and not race conditions. See
[[overlapping-progress]].
Two increments, twenty schedules: find the one that loses an update
| # | Task A — counter++ | Task B — counter++ | State |
|---|---|---|---|
| 1 | rA ← counter | · | counter=0 rA=0 rB=0 |
| 2 | rA ← rA + 1 | · | counter=0 rA=1 rB=0 |
| 3 | counter ← rA | · | counter=1 rA=1 rB=0 |
| 4 | · | rB ← counter | counter=1 rA=1 rB=1 |
| 5 | · | rB ← rB + 1 | counter=1 rA=1 rB=2 |
| 6 | · | counter ← rB | counter=2 rA=1 rB=2 |
if (balance >= 100) withdraw(100) — drive it until it overdraws
balance = 100
withdraw(amount): # both tasks run this concurrently
b = read(balance) # 1
if b >= amount: # 2 <- decided on a value that may already be stale
debit(amount) # 3| # | Withdrawal A (100) | Withdrawal B (100) | State |
|---|---|---|---|
| 1 | rA ← read balance | · | balance=100 paidOut=0 |
| 2 | if rA >= 100 | · | balance=100 paidOut=0 |
| 3 | debit 100 | · | balance=0 paidOut=100 |
A mutex buys correctness with throughput
How much of the task is inside the lock?
What people believe, and what is true
I used a concurrent map, so the code is thread-safe.
The map's operations are atomic. Your read-decide-update sequence spans three of them and is not. Thread-safe containers protect the container, never your invariant.
Local variables are private, so passing data into a thread is safe.
The reference is private; the object is shared. Passing a mutable object to a thread is sharing it, and nothing in the call site says so.
Making the counter atomic fixes it.
It fixes single-field lost updates. It does nothing for an invariant spanning count and windowStart, which is the actual bug.
It never happens in testing, so it is rare enough to ignore.
It happens at window boundaries under concurrent load — a condition tests do not create and production creates constantly. Rarity in CI is not rarity in production.
Go deeper
Overview
Threads share everything except their stacks. That makes communication free and mistakes free too, because nothing marks a shared object as shared.
Practical
For each object two threads can reach, write down the invariant and the protection. Make the critical section exactly as wide as the invariant, shard the lock to match the data, and never hold one across I/O.
Advanced
Thread safety is a property of an invariant over a set of fields, not of a type. This is why "is this class thread-safe?" is unanswerable without knowing what invariant the caller needs — and why composing two thread-safe objects almost never yields a thread-safe operation.