9 lessons

Where This Shows Up

Cache-aware algorithms and tiling, what the compiler did before the CPU saw your code, side channels as a consequence of speculation, and the hardware under virtual machines and containers.

SourceCompilerInstructionsFront EndExecutionRegistersCachesMemoryI/OBehavior
Cache-Aware Algorithms
▶ lab

Two algorithms with identical asymptotic complexity can differ by an order of magnitude in wall clock, because complexity analysis counts operations and hardware charges for data movement. Blocking, compact layouts and node sizes matched to the transfer granularity are all the same idea: arrange the work so that data pays its travel cost once.

Q · Why can two algorithms with the same big-O differ by an order of magnitude in wall clock, and how do you design for the memory hierarchy rather than for the instruction count?
Matrix Tiling: Same Arithmetic, Ten Times Faster
▶ lab

The tiled matrix multiply performs exactly the same multiply-accumulate operations as the naive triple loop, in a different order. It wins because a block of each matrix is brought into cache once and used many times, instead of a row or column being re-fetched on every pass.

Q · Why does reordering the loops of a matrix multiply — without changing a single arithmetic operation — make it dramatically faster on large matrices?
The Compiler Reordered It Before the CPU Did

Between the line you wrote and the work the machine performs sit two independent reordering layers: a compiler that transforms code under the language's rules, and a processor that executes the result out of order under the architecture's rules. Each preserves its own notion of observable behaviour, and neither preserves the order you wrote.

Q · What actually happens to my source code between writing it and the CPU executing it, and who is allowed to reorder what?
Why Reading the Source Cannot Tell You the Cost

Two adjacent lines of source imply neither two instructions nor two steps in time. Source code specifies *what result is required*, and it is an excellent tool for reasoning about correctness — but it deliberately says nothing about instruction count, ordering or cost, which is exactly why measurement exists.

Q · If I cannot trust source order to tell me what the machine does, what is source-level reasoning actually good for?
Side Channels: When Performance Optimisations Leak

Every mechanism that makes a CPU fast by remembering something — caches, branch predictors, translation buffers — creates state that outlives the operation and can be observed indirectly through timing. Information leaks not through what a program outputs, but through how long other things take afterwards.

Q · How can a program leak secrets without ever outputting them, purely through the timing effects of hardware optimisations?
Spectre and Meltdown: When Speculation Crossed a Boundary

In 2018 a class of vulnerabilities showed that speculative execution — a two-decade-old performance technique — could be steered into performing accesses that architecturally never happened, while leaving microarchitectural traces that a timing side channel could read. The durable lesson is not the specific bug but its shape: a performance optimisation created a security boundary violation, and the mitigations cost real performance.

Q · How did speculative execution — a pure performance feature — turn into a security vulnerability, and what did fixing it cost?
The Hardware That Makes Virtual Machines Possible

Running a guest operating system that believes it owns the machine used to require interpreting or rewriting its privileged instructions. Hardware virtualization support added a mode below the kernel's, so a guest can run its own privileged code at native speed while the hypervisor stays in control — and a second layer of address translation so guest memory works without the hypervisor intervening on every access.

Q · What does a CPU actually provide that lets a guest operating system run privileged code at full speed without escaping its virtual machine?
What a vCPU Actually Is

A vCPU is not a core. It is a schedulable thread of execution that the hypervisor multiplexes onto physical hardware, sharing that hardware with other guests. This is why cloud instance performance varies, why steal time exists, and why the count in the instance description does not translate into a guaranteed amount of compute.

Q · When a cloud instance advertises eight vCPUs, what have I actually been given — and why does identical code sometimes run at different speeds on it?
From malloc to Cache Lines
▶ lab

An allocation call returns a pointer, but between that call and a cache line being filled sit an allocator, a virtual address space, a page fault, a physical frame chosen by the kernel and finally the hardware that transfers the line. Each layer shapes where your data lands, which is why allocation pattern becomes cache behaviour.

Q · What happens between calling an allocator and the data actually occupying a cache line, and why does allocation pattern determine cache behaviour?