The Datapath: How Values Move Through the Machine
Registers feed the ALU, the ALU feeds registers back, and a separate route runs from the register file through address calculation to the data cache and back. That loop, plus the memory path hanging off it, is the datapath — and its shape explains why instruction sets look the way they do.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
The register-to-register loop
The core of the datapath is a loop. Two register file read ports supply operands to an execution unit; the unit computes; the result travels back to a write port and lands in a destination register. That is the entire path for the majority of instructions in most programs — arithmetic, logic, comparison, moves.
Two things about this loop have consequences. First, it is short, which is why register-to-register operations are the cheapest the machine performs. Second, its capacity is finite: a register file has a limited number of read and write ports, and an execution unit can accept one operation per cycle. Superscalar designs replicate both to run multiple such loops simultaneously (Superscalar Execution), but each individual loop stays narrow.
This is also where forwarding becomes intelligible. If an instruction needs a result that a previous instruction has just computed but not yet written back, waiting for the write-then-read round trip wastes cycles. The datapath therefore includes bypass routes that carry a freshly computed result directly to a waiting unit (Forwarding and Stalls: Paying for Dependencies) — extra wires whose only purpose is to shortcut this loop.
The memory path hanging off it
Memory is not reachable from the ALU. A separate route exists: a register supplies a base address, an offset is added, the resulting address goes to the load/store unit, which consults the data cache; on a hit, the value returns and is written to a register. On a miss, the request continues down the hierarchy and the latency grows by orders of magnitude (The Memory Hierarchy).
This asymmetry — arithmetic on registers, memory reached only through dedicated load and store operations — is a load/store architecture, and it is the dominant design. It is worth noticing that x86-64 is not strictly one: many of its instructions take a memory operand directly. Internally, though, such an instruction is generally expanded into a load plus an operation, which is the control-unit expansion from the previous lesson showing up again.
The comparison below makes the practical consequence concrete. Two versions of the same computation differ only in whether a value is reloaded from memory each iteration or held in a register. The arithmetic is identical; the number of trips down the memory path is not, and on a hot loop that difference is the whole story.
1// `limit` lives in a struct the compiler cannot prove is unchanging2for i in 0..n:3 if data[i] > cfg.limit: // load cfg.limit each time4 count += 15 6// each iteration:7// load reg, [cfg + offset] <- memory path, every time8// load reg2, [data + i*4]9// cmp / branch1limit = cfg.limit // one trip down the memory path2for i in 0..n:3 if data[i] > limit: // register comparison4 count += 15 6// each iteration:7// load reg2, [data + i*4] <- the unavoidable one8// cmp / branch <- register-to-registerBoth versions perform the same comparisons. The first sends a value down the memory path on every iteration because the compiler cannot prove cfg.limit is unchanged by the loop body — a possibility that pointer aliasing keeps open. Hoisting it makes the invariance explicit and reduces the loop to one unavoidable load plus register-to-register work.
Why the datapath explains the instruction set
Once the route is visible, several instruction set decisions stop looking arbitrary. Arithmetic takes register operands because that is the short loop. Addressing modes look like base + index * scale + offset because that is exactly the computation the address-generation hardware performs (Addressing Modes: How an Index Becomes an Address). The number of operands an instruction takes reflects how many register file ports are available. Instructions are the shape they are because the datapath is the shape it is.
This also frames a genuine design tension. A wider datapath — more ports, more units, more bypass paths — allows more work per cycle, but every added path costs area and power and can lengthen the critical timing path that sets the achievable clock rate. Architecture is largely the discipline of spending that budget well, and different vendors reach different answers, which is why ISA vs Microarchitecture: The Distinction Everything Depends On matters so much.
For a working programmer the payoff is interpretive rather than actionable. You are not going to change the datapath. But when you see an inner loop whose disassembly is loads and stores rather than arithmetic, you now know that the memory path is doing the work and the arithmetic units are idle — and that is a diagnosis, not merely an observation.
| Datapath feature | Instruction-set consequence |
|---|---|
| Short register-to-register loop | Arithmetic instructions take register operands |
| Separate load/store units | Memory is accessed by explicit load and store instructions |
| Address-generation hardware | Addressing modes of the form base + index × scale + offset |
| Limited register file ports | Instructions take two or three operands, not many |
| Bypass paths | Dependent instructions can issue back to back without a write/read round trip |
| Replicated units | Multiple independent instructions can execute in the same cycle |
Key points
- The datapath is a short register→ALU→register loop with a longer, separate route to memory hanging off it.
- Register-to-register operations are cheapest because that loop is short; memory operations traverse a far longer path.
- Load/store architectures make the split explicit; even x86-64 memory operands are typically expanded internally into load plus operation.
- Bypass paths exist purely to let a dependent instruction consume a result before it is written back.
- Instruction set shape — operand counts, addressing modes — follows directly from datapath shape.
Register File & Datapath
Change an input and watch which number moves — and which one refuses to.
Run an instruction.
Arithmetic happens between registers, never directly on memory operands on most architectures. Getting data into a register is a separate load instruction — and that load is where nearly all the time goes.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Register file → execution unit: read ports supply operands to an ALU or load/store unit.
- 2ALU → register file: the result returns through a write port to the destination register.
- 3ALU → bypass network: a freshly computed result is forwarded directly to a dependent operation without waiting for writeback.
- 4Register file → address generation → load/store unit: a base register and offset are combined into an address.
- 5Load/store unit → data cache → register file: on a hit the value returns and is written back; on a miss the request descends the hierarchy.
- • "x86 does arithmetic on memory, so it has no load/store split." The split exists internally; memory operands expand into a load plus an operation.
- • "A load is one instruction, so it costs like one instruction." It costs whatever the memory hierarchy charges, which on a miss is orders of magnitude more.
- • "Forwarding makes dependencies free." It removes the writeback round trip, not the dependency; a chain of dependent operations still serialises.
Consequences, controls and cost
- • A loop dominated by loads and stores leaves arithmetic units idle regardless of how much arithmetic the source contains.
- • Values the compiler cannot prove loop-invariant are re-fetched through the memory path on every iteration.
- • Dependent instruction chains still run at a reasonable rate because forwarding removes the write-then-read round trip.
- • Hoist provably-invariant loads out of loops manually when aliasing prevents the compiler from doing it.
- • Read the inner-loop disassembly and count memory operations against arithmetic operations — the ratio is the diagnosis.
- • Reduce pointer aliasing where the language provides a way to express it, so the compiler can hoist on your behalf.
- • Beyond that, nothing — the datapath is fixed and you are writing for it, not changing it.
- • Disassemble the hot loop and count memory operations versus arithmetic; a memory-dominated body points at the memory path, not the ALU.
- • Compare a hoisted variant against the original to confirm whether repeated loads were actually costing anything.
- • Use load and store counters where available to confirm the loop is issuing the number of memory operations you expect.
- • Manual hoisting duplicates work the compiler usually does, and can go stale when the surrounding code changes.
- • Reasoning from disassembly is precise but slow, and only worth it for loops that profiling has already flagged.
Scope
§224 — what these claims are specific to.
- SIMPLIFIEDA single ALU and one memory path. Real cores have multiple execution units, several load/store units and deep queues between them.
- ISA-SPECIFICStrict load/store architectures (AArch64, RISC-V) expose the split directly; x86-64 permits memory operands and expands them internally instead.