4 lessons

SIMD & Vectorization

One instruction, many elements. What makes a loop vectorizable, what stops the compiler from doing it, and where data-level parallelism sits among the other kinds.

SourceCompilerInstructionsFront EndExecutionRegistersCachesMemoryI/OBehavior
Four Kinds of Parallelism

Instruction-level, data-level, thread-level and core-level parallelism are four different mechanisms with four different requirements, four different costs and four different failure modes. Most confused performance arguments come from conflating two of them.

Q · When someone says "make it parallel", which of the several available mechanisms do they actually mean — and which one does my problem admit?
SIMD: One Instruction, Many Elements
▶ lab

A vector register holds several values and a vector instruction applies one operation to all of them at once. It is the cheapest parallelism on the machine — single-threaded, race-free, and frequently left unused because a single unprovable pointer relationship disabled it.

Q · How does one instruction operate on eight numbers at once, and what does my data have to look like for that to be possible?
Vectorization: Turning a Loop Into Vector Work

The transformation from one-element-per-iteration to many, and the four conditions that have to hold for it to be legal. Most loops that fail to vectorise fail on a single unprovable assumption rather than on anything fundamental.

Q · What exactly has to be true about my loop before it can legally be turned into vector operations?
Auto-Vectorization: Verify, Do Not Assume

Compilers vectorise loops automatically, sometimes. It is a best-effort optimisation with no guarantee, it fails silently, and it can stop working after an unrelated edit — so the only responsible position is to check rather than believe.

Q · How do I find out whether the compiler actually vectorised my hot loop, rather than assuming it did?