You observed → the hardware is doing
The reverse index. Each row opens the lesson that explains the mechanism and what you can actually do about it.
Same code, different speed
| You observed | The hardware is doing |
|---|---|
| Two loops, same complexity, wildly different times | One has spatial locality and the other chases pointersBoth Are O(n). One Is Far Slower. → |
| Performance falls off a cliff at one input size | The working set stopped fitting in a cache levelWorking Set: Why Performance Falls Off a Cliff → |
| Sorting the input made the loop faster | The branch became predictableBranch Prediction: Guessing Well Enough to Matter → |
| Swapping the loop order changed everything | Row-major storage traversed by column touches a new line every accessTemporal Locality → |
| Works fast on one machine, slow on another | Different cache sizes, associativity or memory topology — the code did not changeISA vs Microarchitecture: The Distinction Everything Depends On → |
Memory layout
| You observed | The hardware is doing |
|---|---|
| Adding a field made the struct much bigger | Alignment forced padding; field order decides how muchPadding: Why Your Struct Is Bigger Than Its Fields → |
| Reading one field pulls in the whole record | The line is the unit of transfer, not the fieldMemory Moves in Lines, Not Variables → |
| Touching one field of many records is slow | Array-of-structs brings in fields you never readArray of Structs, or Struct of Arrays? → |
| Bytes look reversed in the file | Endianness of the writing machine or formatEndianness: Which Byte Comes First → |
| Random access is fine until the data grows | It fitted in cache; now it does notCache Thrashing: Load, Evict, Reload, Repeat → |
Multiple cores
| You observed | The hardware is doing |
|---|---|
| Threads on different data still contend | The variables share a cache lineFalse Sharing: Independent Data, Shared Line → |
| Speedup stops well before the core count | Shared bandwidth, coherence traffic, or a serial sectionWhen the Memory Bus Is the Bottleneck → |
| Doubling logical CPUs did not double throughput | SMT shares execution resources; it is not more coresSMT: Two Contexts, One Core → |
| Performance depends on which cores it lands on | NUMA locality, or lost cache warmth after migrationNUMA: Not All Memory Is Equally Far → |
| Works on x86, breaks on ARM | A weaker hardware memory model exposed a missing barrier or a data raceHardware Memory Models Are Not Language Memory Models → |
| A lock is cheap uncontended and terrible contended | Fast path is an atomic; the slow path enters the kernelWhat a Mutex Actually Does → |
Reading counters
| You observed | The hardware is doing |
|---|---|
| Low IPC, high last-level misses | Memory-bound: the core is waiting for dataBusy Is Not the Same as Working → |
| Low IPC, low misses, low mispredictions | A dependency chain — nothing can overlapDependency Graphs: The Real Shape of Your Code → |
| High TLB misses, healthy cache rates | The working set spans more pages than the TLB can mapWhen Translation Itself Is the Bottleneck → |
| IPC improved but wall time got worse | You executed more instructions; IPC alone is not performanceIPC: Instructions Per Cycle → |
| The first iterations are much slower | Cold caches, cold predictors, and on managed runtimes cold codeEvery Way a CPU Microbenchmark Lies → |
| The benchmark got slower the longer it ran | Frequency dropped under sustained loadThe First Ten Seconds Lie → |
| The optimization vanished in the real program | The microbenchmark kept everything in cache and the compiler deleted the workEvery Way a CPU Microbenchmark Lies → |
Compiler and CPU
| You observed | The hardware is doing |
|---|---|
| The compiler removed my benchmark loop | The result was unused, so it was dead codeThe Compiler Reordered It Before the CPU Did → |
| The debugger shows lines out of order | The compiler reordered before the CPU ever saw itWhy Reading the Source Cannot Tell You the Cost → |
| The loop did not vectorize | Possible aliasing, a loop-carried dependency, or non-contiguous accessAuto-Vectorization: Verify, Do Not Assume → |
Accelerators
| You observed | The hardware is doing |
|---|---|
| The GPU kernel is faster but the program is not | Transfer over the interconnect dominatesThe Transfer You Forgot to Count → |
| GPU throughput collapses on branchy code | Divergent lanes serialiseLanes, Divergence and Coalescing → |
| Inference is slow despite spare compute | It is bandwidth-bound, not compute-boundLLM Inference Is a Memory Bandwidth Problem → |
| A cloud instance is inconsistent run to run | Shared hardware, vCPU scheduling, and topology you do not controlWhat a vCPU Actually Is → |