Hardware Performance Analysis
Reading the machine: performance counters, CPI, telling compute-bound from memory-bound, and every way a microbenchmark will lie to you about frequency, caches and dead code.
Every modern CPU carries a small unit whose only job is to tally what the rest of the chip did: cycles, instructions retired, misses at each cache level, mispredicted branches, stalled cycles. It is the only direct evidence you will ever get about the hardware — and it is sampled, approximate, and named differently on every chip.
Cycles per instruction, and its reciprocal instructions per cycle, describe how smoothly work is flowing through the machine. Neither is a measure of performance. A change that halves CPI while tripling instruction count made the program slower, and a vectorized loop that runs twice as fast often shows a worse CPI than the scalar loop it replaced.
A core showing 100% utilisation may be executing a dense stream of arithmetic, or it may be stalled almost the entire time waiting for data that has not arrived. The operating system reports both as "busy". They are different problems with disjoint fixes, and only the counters can tell them apart.
A cache miss costs a great deal if the core has nothing else to do, and almost nothing if it does. Modern cores keep several misses outstanding at once, so ten independent misses can cost barely more than one — while ten dependent misses cost ten times as much. This is why miss counts alone never predict runtime.
Instructions are fetched from memory through their own cache, and that cache is small. A hot loop that fits runs at full speed; a sprawling call graph with aggressive inlining can spend a large fraction of its cycles waiting for instructions to arrive — a stall that data-focused profiling is structurally unable to see.
A microbenchmark measures what it measures, which is frequently not what you meant. The compiler deletes work whose result is unused, the caches and predictors are warm in ways production never is, the clock speed moves underneath you, and the timer itself costs more than the operation. Each of these has produced published results that were simply wrong.
The number printed on the box is a nominal figure, not an operating one. Real clock frequency moves continuously with load, thermal headroom, power budget, how many cores are active, and even which instructions are executing — wide vector code frequently runs at a lower clock than scalar code on the same chip.
Silicon has a temperature limit, and the only lever the chip has to stay under it is to slow down. A workload that starts on a cool chip runs at one speed and settles at a lower one, which is why burst performance and sustained performance are different numbers and why short benchmarks systematically flatter the machine.
Energy, not time, is the constraint that actually binds on phones, on laptops and across datacentres. The configuration that finishes soonest is frequently not the one that uses least energy, and the relationship between the two is non-linear enough that "run slower to save power" is often exactly wrong.
Nearly every technique modern CPUs use — pipelining, superscalar issue, out-of-order execution, speculation — increases the number of operations completed per unit time without reducing, and sometimes while increasing, the time any single operation takes. This is why decades of architectural progress leave a dependent chain almost exactly as slow as it was.