The question this answers
What does a context switch actually cost me, and why is the number I was told too small?
A worker thread iterating over a 4 MB working set, preempted every scheduler quantum by other runnable threads on the same core.
No application state is shared. What is shared and invisible is the per-core hardware: L1/L2/L3 cache lines, TLB entries, branch-predictor history and prefetcher state — all of which the incoming thread overwrites.
A thread's observable state — registers, memory, program counter — is exactly restored across a switch. That is guaranteed and never violated. The property this lesson is about is the one the hardware does *not* preserve: the thread's performance state. It resumes logically identical and physically much slower, and reasoning about concurrency cost without that distinction produces estimates that are wrong by an order of magnitude.
Synchronization exists to preserve this sentence. If a schedule can make it false, the code is wrong no matter which primitive it uses.
Two costs, and the second is the one that matters
The mechanism of a context switch belongs to Operating Systems — see Context Switching for the kernel path: trap, save the register file, update the scheduler's bookkeeping, pick the next thread, restore its registers, possibly switch address space. That path is well optimised and costs on the order of a microsecond on modern hardware.
The cost model that matters for concurrency design has a second term that does not appear in that path at all. While the other threads ran, they filled the caches with *their* data. When your thread resumes, its working set is partly or entirely gone: every access that was an L1 hit is now an L3 hit or a memory access, its TLB entries have been evicted, and the branch predictor has been retrained on somebody else's control flow. Your thread executes the same instructions and retires them more slowly, for as long as it takes to refill.
That is why the honest cost is not a constant. It is a function of your working-set size, how many threads ran in between, and how big the last-level cache is. For a thread touching a few kilobytes the indirect cost is negligible. For a thread streaming megabytes it can dwarf the switch by an order of magnitude — and it is charged to your thread's *execution* time, not to any switch counter, so no amount of staring at context-switches/sec will reveal it.
| Component | What happens | Charged to | Scales with |
|---|---|---|---|
| Direct: save/restore | Trap to kernel, save the register file, scheduler decision, restore the incoming thread's registers. | Kernel time — visible as system CPU and in switch counters. | Number of switches. Roughly constant per switch. |
| Address-space switch | Between *processes*, the page-table root changes and TLB entries are invalidated (mitigated by ASIDs/PCIDs where available). | Kernel time, plus later user-time TLB misses. | Whether the switch crosses a process boundary. Thread-to-thread in one process skips this. See Process versus Thread. |
| Cache pollution | The incoming thread evicts the outgoing thread's lines from L1/L2 and, if enough threads run, from L3. | The *resumed thread's* user time, paid later as misses. | Working-set size and how many threads ran in between. The dominant term for data-heavy work. |
| TLB and predictor state | Translation entries evicted; branch predictor and prefetcher retrained on other code. | The resumed thread's user time, as stalls and mispredictions. | Code and data footprint; how different the other threads' behaviour was. |
| Scheduler-adjacent work | Run-queue manipulation, load balancing across cores, timer interrupts, migration to a cold core. | Kernel time, plus a cold restart if the thread lands on a different core. | Run-queue length and core count. Migration is why Thread Affinity: Pinning, and What It Costs You matters. |
Watching a thread resume cold
The timeline shows the shape you should carry in your head. A thread runs at full speed with a warm cache, is preempted, other threads run and evict its data, and it resumes — but the first part of its next slice executes at a fraction of the previous rate while it refetches. The direct switch cost is the thin band; the degraded band after it is the real bill.
This has a direct design consequence, and it is not "avoid switches". It is that switch frequency matters more than switch count in isolation, and slice length matters as much as either. A thread preempted once per 4 ms quantum amortises the refill over 4 ms of work. The same thread yielding every 50 µs — because it is polling, or because it is taking a contended lock, or because it blocks on tiny reads — pays the refill twenty thousand times a second and may spend the majority of its slice running cold.
This is also the strongest argument for a bounded pool over thread-per-item, and for tasks over threads. A user-space task switch skips the kernel trap and the address-space work entirely, and — more importantly — a runtime that keeps a small number of threads pinned to cores lets each one keep its cache state across many task switches. The concurrency stays high while the *thread* switching stays low. See A Task Is Not a Thread and Hybrid Runtimes: It Was Never Threads Versus Async.
Measuring it rather than quoting a number
Because the indirect cost depends on your working set and your hardware, the only defensible number is one you measured. Fortunately the measurement is straightforward: run the same total work at a thread count at or below core count and at a thread count well above it, and compare instructions per cycle and cache-miss rate alongside switch counts. The IPC drop is the indirect cost made visible.
The read-out below is the shape to look for. Switch count rises by two orders of magnitude, which is the number everyone looks at. The two numbers that explain the throughput loss are underneath it: last-level cache misses per instruction roughly triples, and IPC falls by more than half. Multiply that IPC drop by the whole run and it is a far bigger term than 250 000 switches at a microsecond each.
One caution about attribution, which is the most common mistake in this area. A high switch count is a *symptom* with several causes: oversubscription, frequent short blocking, lock handoffs, and timer-driven wakeups all produce it, and they have completely different fixes. Split voluntary from involuntary switches — pidstat -w reports both — because voluntary switches point at blocking and lock handoff, and involuntary ones point at Oversubscription.
8 threads 128 threads delta task-clock (ms) 13 040 16 890 +29% context-switches 2 410 248 300 x103 <- the number people quote of which voluntary 310 1 900 of which involuntary 2 100 246 400 <- preemption, i.e. oversubscription cpu-migrations 180 41 700 x232 <- resumes on a COLD core instructions 38 100 000 000 38 240 000 000 ~0% <- same work cycles 15 900 000 000 34 700 000 000 +118% <- twice as long to do it insn per cycle 2.40 1.10 -54% <- the indirect cost, visible LLC-load-misses 210 000 000 690 000 000 x3.3 <- why IPC fell dTLB-load-misses 41 000 000 158 000 000 x3.9 read it as: identical instruction count, +118% cycles. The extra time is NOT in the switch path -- 248k switches at ~1 us is ~0.25 s of a 16.9 s run. It is in the halved IPC, caused by the 3.3x cache misses the switching created. Quoting only "a context switch costs a microsecond" understates this workload's real cost by roughly an order of magnitude.
Key points
- The direct switch cost (~1 µs) is real and is usually the smaller half of the bill.
- The indirect cost is cache, TLB and predictor state destroyed by the intervening threads, charged to the resumed thread's execution time and counted by nothing.
- It scales with working-set size, so the same switch is nearly free for a small thread and very expensive for a data-heavy one — there is no single number.
- Switch *frequency* and slice length matter more than switch count: a thread that yields every 50 µs may spend most of its slice running cold.
- Split voluntary from involuntary switches — they point at blocking/handoff and at oversubscription respectively, and the fixes differ.
The loop, answered
Every field is required, which is why no lesson here can recommend concurrency without naming the interleaving that breaks it, the complexity it adds, and the simpler thing to consider first.
- • The kernel traps, saves the outgoing thread's registers, runs the scheduler, and restores the incoming thread's registers; between processes it also switches the address space.
- • The incoming thread executes and its memory accesses evict the outgoing thread's lines from the shared cache levels and its entries from the TLB.
- • When the original thread resumes, its accesses miss, so it stalls on memory and retires fewer instructions per cycle until the working set is refetched.
- • If the scheduler resumes it on a different core, even the private L1/L2 state is gone as well, which is why migration counts matter.
- • The refill cost is proportional to working-set size and inversely related to how much cache survived, so the same switch has a wildly different price for different threads.
- • One thread per core: the thread runs its full slice warm, is not preempted, and the entire cost model is irrelevant.
- • Two threads with tiny working sets sharing a core: switches are frequent but refill is nearly free, so the direct cost dominates and it is small.
- • Two threads with 4 MB working sets sharing a core: each resume refetches from memory, and the pair achieves far less than one thread's throughput on that core.
- • A thread that blocks on a 20 µs read every 50 µs: two voluntary switches per 100 µs, and it runs cold for a large fraction of every slice. The fix is batching the reads, not tuning the scheduler.
- • A thread migrated to another core on resume: even private L1/L2 state is cold, which is the case
cpu-migrationscounts and Thread Affinity: Pinning, and What It Costs You addresses.
- • The OS guarantees exact restoration of architectural state — registers, program counter, memory. Your program cannot observe a switch except through timing.
- • It guarantees nothing about microarchitectural state. Caches, TLB, branch predictors and prefetchers are explicitly not preserved, and no API asks for them to be.
- • Thread-to-thread switching within a process guarantees the address space is unchanged, which avoids the page-table switch but not cache eviction.
- • Affinity guarantees a thread runs on a chosen core, and therefore that its private caches have a chance of surviving. It does not guarantee they do, because other threads on that core still evict.
- • A switch counter guarantees you know how many switches occurred. It tells you nothing about what they cost, which is the entire point of this lesson.
- • Switching is itself a contention symptom: threads switch because more of them are runnable than there are cores, or because they are blocking on something.
- • The cache is a contended resource with no lock and no metric in your application — threads evict each other continuously and nothing in the code mentions it. See Parallelism Can Destroy Locality.
- • Lock handoffs generate voluntary switches, which is why a contended lock's real cost is a park/unpark round trip and a cold resume rather than the critical section. See What Contention Actually Costs.
- • Migration between cores or across NUMA nodes multiplies the refill cost, because memory affinity is lost as well as cache. See NUMA: Not All Memory Costs the Same.
- • Throughput loss attributed to "the algorithm" when it is entirely the cost of running eight threads per core on a data-heavy workload.
- • Under-estimating concurrency overhead by an order of magnitude because only the direct cost was counted — the most common form of this failure, and it happens in design documents rather than in production.
- • Polling loops and short blocking reads producing enormous voluntary switch counts and cold execution, with a healthy-looking CPU graph.
- • Migration storms where the scheduler moves threads between cores under load, destroying private cache state on every move.
- • Latency spikes with no code-level cause, driven by unlucky preemption of a thread holding a lock — see Priority Inversion.
- • Switching is the mechanism that makes concurrency useful at all: a thread that blocks on I/O yields its core to something else, which is a strict win and the reason preemptive multitasking exists.
- • Frequent switching is fine and correct for I/O-bound work, where the alternative is an idle core.
- • Time-slicing bounds the latency of short jobs behind long ones, which is a fairness property worth paying for. See Fairness.
- • For CPU-bound work with a large working set, where every preemption costs a refill and the aggregate is a large fraction of the run.
- • For any polling or spinning pattern that yields constantly, which pays the refill without ever accumulating useful warm-cache time. See Busy Waiting.
- • When it is used to justify an unbounded thread count — "switches are only a microsecond" is the sentence that precedes a 128-thread pool on an 8-core box.
- •
perf staton the same work at two thread counts: compare instructions, cycles, IPC and LLC misses. Identical instructions with far more cycles is the indirect cost, quantified. - •
pidstat -wfor voluntary against involuntary switches — the split identifies whether you are blocking too often or oversubscribed. - •
perf stat -e cpu-migrationsfor cold resumes on a different core, which affinity can address directly. - • Working-set size, because it determines whether any of this matters for your workload. A thread touching 8 KB is unaffected; a thread touching 8 MB is dominated by it.
- • Wall-clock throughput at the pool sizes you are actually choosing between. Every microarchitectural metric above is explanation; this one is the decision.
- • Reasoning correctly here means carrying a two-term cost model rather than a constant, and knowing your working-set size — which most teams do not.
- • Mitigations (affinity, pinning, batching, larger slices) each add configuration that is hardware-specific and does not survive a move to a different machine shape.
- • Affinity in particular fights the scheduler's load balancing, and getting it wrong leaves cores idle while pinned threads queue.
- • The measurement requires hardware counters, which are frequently unavailable in containers and on virtualised or cloud instances — a real practical obstacle.
- • Keep runnable threads at or near core count so preemption is rare and the whole cost model becomes irrelevant. See Thread Pools.
- • Use tasks over a small thread pool: user-space switches skip the kernel trap and the address-space work, and the underlying threads keep their cache state. See A Task Is Not a Thread and Coroutines: Functions That Can Pause.
- • Batch small blocking operations so a thread does more work per slice and yields less often.
- • Pin latency-critical threads to dedicated cores where the workload justifies the configuration cost. See Thread Affinity: Pinning, and What It Costs You.
- • Improve locality so the working set fits in cache and the refill is cheap even when it happens. See Parallelism Can Destroy Locality.
What people believe, and what is true
A context switch costs about a microsecond.
The direct path does. The total cost includes a refill of cache and TLB state charged to the resumed thread's execution time, which for a large working set is several times larger and appears in no switch counter.
Context switches are counted, so I can see the overhead in my metrics.
You can see the direct half. The indirect half shows up as reduced instructions per cycle in ordinary user time, which looks like your code getting slower for no reason.
Switching between threads of the same process is free because the address space does not change.
It skips the page-table switch, which is the cheapest saving. Cache and predictor pollution happen regardless, and those are the dominant terms.
Go deeper
Overview
Saving and restoring registers is fast. The slow part is that the other threads threw your data out of the cache, so when you resume you run at half speed until you have fetched it all back.
Practical
Do not quote a constant. Measure IPC and cache misses at two thread counts on your own workload; identical instruction counts with far more cycles is the cost. Split voluntary from involuntary switches to know whether to fix blocking or to shrink the pool.
Advanced
The right mental model is that a thread has two kinds of state: architectural, which the OS restores exactly, and microarchitectural, which nobody restores and which determines how fast the architectural state advances. Every concurrency design decision — pool size, task versus thread, affinity, batch size — is really a decision about how often you throw the second kind away.
Internals
This is why user-space scheduling wins for high-concurrency workloads. A coroutine switch is a stack-pointer swap and a few register saves with no kernel transition, and because the underlying OS thread stays on its core, the cache state largely survives. A runtime can then run a hundred thousand concurrent tasks over eight threads and pay the expensive kind of switch only when the OS preempts one of those eight. See Hybrid Runtimes: It Was Never Threads Versus Async and io-models.