Follow a Memory Access
`value = arr[i];` — two instructions, and anywhere from one unit of cost to several hundred depending on state your source code cannot see. Open each level and find out where the difference lives.
One line of code: value = arr[i];. It compiles to a couple of instructions, and on a good day it costs almost nothing. On a bad day it costs several hundred times more, and nothing in the source distinguishes the two. Walk the access down the machine and find out where the difference lives.
Costs are ratios against a register access, not times. Absolute latencies depend on the processor, its clock and its memory, so publishing them would be wrong everywhere but one machine. Bars are log-scaled.
The whole access. Its cost is decided almost entirely by where the data already is — a question the source code cannot answer.
base + i × sizeof(element). Address arithmetic is folded into the addressing mode on most architectures, so it is effectively free — this is not where your time goes.
Every access goes through the MMU. A TLB hit makes this invisible.
if it misses On a TLB miss the hardware walks the page table — several dependent memory accesses, each of which can itself miss in cache. This is why a program touching many pages sparsely can be slow while its data fits in cache perfectly well.
Small, fast, private to the core. If `arr[i]` shares a line with something you touched recently, you land here and the access is over.
if it misses A miss here does not fetch your variable — it fetches the whole cache line containing it, which is why the next fifteen elements of a sequential scan are already paid for.
The arithmetic you actually wanted. Notice how small this is next to everything above it: for most real loops the computation is not the cost, the data movement is.
Open every level and account for the cost before revealing the findings. The question to hold onto: which of these does your source code let you see?