Parallel Performance
Where the speedup goes: serial fractions, synchronization, memory bandwidth, cache locality destroyed by tasks bouncing between cores, NUMA, affinity, and the numerical fact that parallel reduction can change floating-point results.
One worker: 1x. Two: 1.8x. Four: 3.2x. Eight: 4.5x. The missing speedup is not lost to one cause — it is serial work, synchronization, memory bandwidth, coherence traffic and scheduling, each taking a share, and each with a different tell.
Q · I doubled the workers and got 1.4x. Which of the five things that eat speedup is eating mine?
Cores multiplied; the path to RAM did not. Some workloads stop scaling at four workers not because the CPU ran out but because the memory subsystem did — and the tell is that adding cores stops helping while CPU utilization still reads 90%.
Q · Adding workers stopped helping but every core still looks busy — am I out of CPU, or out of memory bandwidth?
A single-threaded loop walks memory in order and every access is nearly free. Split it across eight workers that migrate between cores, interleave their indices and share a last-level cache, and the same total work can move more bytes and take longer.
Q · Why did splitting this loop across eight workers move more memory and run slower than the single-threaded version?
On a multi-socket machine, "RAM" is several pools with different distances. A worker reading memory attached to its own socket is fast; reading memory attached to the other socket goes over an interconnect and costs meaningfully more — and which pool a page lives in was usually decided by whichever thread touched it first.
Q · My parallel job is fast on a single-socket machine and slow on a bigger dual-socket one — why would more hardware be worse?
Tell the scheduler a thread may only run on certain cores. It removes migration, keeps caches warm and cuts latency variance — and it hands you a scheduling decision the OS was making better than you will, on hardware you may not be running on next quarter.
Q · Should I pin these threads to specific cores, and what am I giving up if I do?
Floating-point addition is not associative, so a parallel reduction adds the same numbers in a different order and can produce a different answer — a different one again at a different worker count. No race, no bug, no lost update. Just arithmetic that does not obey the law you assumed it did.
Q · Why does my parallel sum disagree with the sequential one, and disagree differently at 2, 4 and 8 workers?
Concurrent execution is nondeterministic wherever ordering is unspecified. Sometimes that is fine and sometimes it is the bug — and the difference is whether the output you promised was a set or a sequence. Making a parallel computation reproducible is always possible and never free.
Q · The same input produced two different outputs. Is that a bug, or did I promise something I never actually specified?
No ordering, FIFO per producer, causal, total. Each is a different promise about what one observer may see relative to another, each costs progressively more parallelism, and almost every ordering bug is a system that was sold one level and assumed the next one up.
Q · Which ordering does this queue actually guarantee, which one does my code assume, and what would the stronger one cost?