Load and Store: Why Arithmetic Happens in Registers
Almost every ISA makes you bring data into a register before you can compute with it, and write it back explicitly. That looks like bureaucracy until you notice that a load is the one instruction whose cost varies by two orders of magnitude depending on where the data happens to be.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
The load–store discipline
Most modern ISAs are load–store: arithmetic instructions operate only on registers, and separate load and store instructions move data between registers and memory. x86-64 is the notable exception among common architectures, permitting some arithmetic instructions to take a memory operand — but internally those decode into the same load-then-operate sequence (Decode: Turning Bytes Into Intent).
The reason is uniformity. If arithmetic instructions can only touch registers, every arithmetic instruction has predictable, short, fixed latency. All the variability — cache hit, cache miss, TLB miss, page fault — is confined to loads and stores, which the machine can then treat specially: issue them early, let them complete out of order, and keep executing other work while they are outstanding.
That confinement is what makes an out-of-order core possible. The machine needs a small number of unpredictable instructions it can schedule around, not arbitrary instructions that might take 300 cycles. So load–store is not bureaucracy; it is the design decision that lets everything else be fast.
total = total + prices[i]; MUL t0, i, 8 ; index * element size ADD t1, prices, t0 ; base + offset -> the address LOAD t2, [t1] ; <-- the only instruction whose cost varies ADD total, total, t2 ; register arithmetic: short, fixed latency ; and if total lives in memory rather than a register: STORE [total_addr], total ; goes through the store buffer, not straight to DRAM
The one instruction whose cost is not fixed
Every other instruction in that sequence has a latency the scheduler knows in advance. The load does not. Its cost depends on whether the line is in L1, in an outer cache, or in DRAM — and on whether the address translation is in the TLB (The TLB: A Cache for Addresses, Not Data).
The machine's response is to *not wait*. A load is issued as early as its address is known, and execution continues past it until an instruction actually needs the value. This is why Out-of-Order Execution and the memory hierarchy are inseparable topics: out-of-order execution exists in large part to hide load latency, and it succeeds exactly to the extent that there is independent work available to fill the gap.
The failure case is a load-use dependency immediately after the load: the very next instruction needs the loaded value, so there is nothing to overlap. A single such pair costs a small stall on a cache hit (Forwarding and Stalls: Paying for Dependencies). A chain of them — each load's result being the address of the next load — is Pointer Chasing: The Address You Do Not Have Yet, and it defeats the machine completely because the CPU cannot even *start* the next load until the current one returns.
Stores are not symmetric with loads
A load blocks progress when its value is needed. A store usually does not, because nothing in the program is waiting for it — the value is already known. Cores exploit this with a store buffer: the store retires into a buffer and drains to cache later, letting execution continue immediately.
That asymmetry has a consequence far beyond performance. Because a store sits in a buffer that other cores cannot see, a store followed by a load of a *different* address can appear, from another core's perspective, to have happened in the opposite order. This is not a bug and not a compiler transformation — it is the store buffer doing its job, and it is the single most common source of surprising multithreaded behaviour on hardware that is otherwise strongly ordered. Store Buffers: Where Your Writes Wait and Why Your Loads and Stores Happen Out of Order pick this up in full.
For single-threaded code the buffer is invisible: the core forwards from its own store buffer when it loads an address it recently stored, so a program always sees its own writes in order. The illusion only breaks with a second observer.
| Property | Load | Store |
|---|---|---|
| Blocks progress | When a dependent instruction needs the value | Rarely — the value is already known |
| Cost variability | Enormous: L1 hit to DRAM miss | Mostly hidden by the store buffer |
| Can be issued early | Yes, as soon as the address is known | Yes, but must not become visible before retirement |
| Visible to other cores | Reads whatever coherence provides | Only once it drains from the store buffer |
| Main hazard | Load-use dependency and pointer chasing | Reordering visible to other threads |
Key points
- Load–store architectures confine all cost variability to two instruction types, which is what makes the rest of the pipeline predictable.
- A load is the only common instruction whose latency spans two orders of magnitude depending on where the data is.
- Out-of-order execution exists largely to hide load latency, and works only when independent work is available.
- A load-use dependency immediately after a load has nothing to overlap; chained ones defeat the machine entirely.
- Stores retire into a buffer and drain later, which is invisible single-threaded and the root of surprising ordering across cores.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Address computation → load unit: base and scaled index are added to form the effective address (What `arr[i]` Actually Compiles To).
- 2Load unit → TLB: the virtual address is translated to a physical one, hitting the TLB in the common case (The TLB: A Cache for Addresses, Not Data).
- 3Load unit → L1 data cache: the line is looked up; on a hit the value is delivered in a few cycles.
- 4L1 miss → outer caches → DRAM: the request walks outward, and the core continues executing independent instructions in the meantime.
- 5Store → store buffer → cache: a store retires into the buffer and drains to cache asynchronously, so the core does not wait for it.
- • "Memory access costs a fixed amount" — it spans roughly two orders of magnitude, which is why one number for "memory" is useless.
- • "The store was slow, so the write went to RAM" — it almost certainly went to a store buffer and drained later.
- • "My code stores then loads, so other threads see that order" — only if the ordering is enforced; the store buffer makes it otherwise.
Consequences, controls and cost
- • The same arithmetic runs at wildly different speeds depending purely on where its inputs live.
- • Code with long chains of dependent loads cannot be accelerated by a faster CPU, only by a better data layout.
- • Two threads can observe each other's stores in an order neither of them wrote, without any compiler reordering involved.
- • Improve locality so loads hit cache: contiguous layout, smaller working sets, sequential access ([[spatial-locality]]).
- • Break load-use chains where possible — load several independent values before using any of them, so their latencies overlap.
- • Keep hot values in registers by reducing live variables in inner loops, letting the compiler avoid reloading them.
- • For cross-thread ordering, use explicit atomics and barriers rather than assuming store order is observed ([[memory-barriers]]).
- • Read L1, L2 and last-level cache miss counters for the loop to locate which level the loads are actually being served from.
- • Compare cycles per instruction with and without a data-layout change; loads are the usual explanation for a large IPC gap.
- • Use a memory-profiling tool that attributes misses to source lines, so you can see which specific load is expensive.
- • Layout changes that improve load locality often make code less natural to express and harder to change later.
- • Manually overlapping loads increases register pressure and can cause spills that reintroduce memory traffic.
- • Reasoning about store visibility correctly requires explicit synchronisation, which costs both performance and complexity.
Scope
§224 — what these claims are specific to.
- GENERALLoad–store separation, load cost variability and store buffering are present on every mainstream cached architecture, including x86-64, AArch64 and RISC-V.
- ISA-SPECIFICx86-64 permits memory operands on arithmetic instructions where AArch64 and RISC-V require explicit loads; internally both decompose the same way, so this is a syntactic rather than a behavioural difference.
- MICROARCH-SPECIFICThe cost ratios shown are typical of contemporary high-performance cores; absolute cycle counts and cache level counts differ per design.