Contention & Oversubscription
Eight cores all waiting on one lock. Convoys, false parallelism, oversubscription, the real cost of a context switch, and busy waiting — the module that explains why parallel hardware does not imply parallel throughput.
Eight cores, one lock, one thread doing useful work and seven parked. The machine reports 12% CPU and the service is at its throughput ceiling. Observability teaches you to spot this from p99; this lesson is about why it happens and the three things that reduce it.
Q · Seven of my eight threads are waiting on one lock — what is the cost, and which of my options actually reduces it?
One slow lock holder makes every other thread queue behind it. That much is obvious. The part that surprises people is what happens next: when the slow operation ends, the queue does not disperse — the threads now arrive in lockstep and keep re-forming the queue long after the original cause is gone.
Q · The slow query finished ten minutes ago — why is the lock still backed up?
Eight threads, eight cores, one global lock. The code is parallel, the hardware is parallel, and the effective parallelism is 1.0 — every thread spends its life queueing for the same mutex. The program is concurrent in structure and serial in execution.
Q · The work is spread across eight threads and the speedup is 1.05 — where did the parallelism go?
Eight cores, sixty-four CPU-bound threads. Every thread gets an eighth of a core, every one takes eight times longer, and the machine spends a measurable share of its capacity switching between them and refilling caches that the previous thread just evicted.
Q · I have eight cores and sixty-four runnable threads — what exactly does the extra fifty-six cost me?
The register save and restore is about a microsecond and it is not the expensive part. The expensive part is the cache, TLB and branch-predictor state the incoming thread destroys, which the outgoing thread must then rebuild — a cost that is paid later, is invisible in any switch counter, and routinely exceeds the switch itself several times over.
Q · What does a context switch actually cost me, and why is the number I was told too small?
The counterweight lesson for the whole domain. Four cores, one thousand CPU-bound threads: the work does not go faster, it goes slower, and it consumes gigabytes to do so. Threads are not a performance knob — they are a way of expressing concurrency, and concurrency is not parallelism.
Q · If threads make things faster, what is the right number — and why does that question have no general answer?
while (!ready) {} occupies a core at 100% doing nothing, and on an oversubscribed machine it actively prevents the thread that would set ready from running. This is not an argument against all spinning — bounded spinning is a real technique — it is an argument against unbounded spinning where a blocking wait belongs.
Q · What does spinning on a flag cost, and when is waiting without blocking ever the right choice?