Layoutdata-oriented designlayoutabstractionhot loopstradeoffs

Data-Oriented Design, Without the Dogma

Organise data around how it is processed rather than around how the domain is modelled. It is a real technique with real wins in hot loops — and a genuinely bad default for code where clarity matters more than cache lines.

Follow the mechanism

Software view, hardware view

The gap between what you wrote and what the machine does is where this whole domain lives.

The question
When is it worth organising code around the data's memory layout instead of around the problem domain?
What you wrote
Model the domain: an entity is an object, related things are grouped, and abstraction hides representation so it can change later.
What the hardware does
The machine processes batches of bytes from cache lines. Layout chosen for conceptual clarity often scatters the bytes a hot loop needs across many lines.
Applied to the few loops that dominate runtime, this thinking produces some of the largest available speedups. Applied everywhere, it produces unmaintainable code in exchange for improving functions that were never hot. Knowing which situation you are in is the entire skill.
Source→Compiler→Instructions→Front End→Execution→Registers→Caches→Memory→I/O→Behavior

What the technique actually is

Data-oriented design starts from a different question than object-oriented modelling. Rather than asking what an entity *is*, it asks what transformations run over the data, how often, and in what order — then chooses a layout that serves those transformations. The layout decisions that follow are usually the ones in Array of Structs, or Struct of Arrays? and Padding: Why Your Struct Is Bigger Than Its Fields: group by field rather than by entity, split hot from cold, and prefer flat contiguous storage to graphs of references.

The second, less discussed half is about batching. Processing a thousand entities in one pass over dense arrays keeps instruction cache and data cache both warm and gives the branch predictor a consistent pattern. Processing them one at a time through a virtual dispatch scatters both. The technique is as much about the shape of the loop as about the shape of the data.

The compare below is the canonical transformation: a polymorphic per-entity update becomes a batched pass over dense arrays.

Domain-shaped: one object at a time, through an indirection
1for (entity in entities) {          // array of pointers
2    entity.update(dt)               // virtual dispatch
3}
4 
5// per iteration:
6//   - load the pointer                 (one line)
7//   - load the object                  (another line, scattered on heap)
8//   - load the vtable                  (another line)
9//   - indirect call                    (often mispredicted)
10//   - touch all fields, use two        (wasted line fraction)
Processing-shaped: one dense pass per transformation
1// positions and velocities in separate dense arrays
2for (i = 0; i < n; i++) {
3    pos_x[i] += vel_x[i] * dt
4    pos_y[i] += vel_y[i] * dt
5}
6 
7// per iteration:
8//   - contiguous loads, prefetcher runs ahead
9//   - every fetched byte used
10//   - no indirect call, so nothing to mispredict
11//   - straightforwardly vectorisable

The second version does the same arithmetic. What it removes is three indirections, an unpredictable indirect branch, and most of the wasted cache-line fraction — while making the loop a candidate for vectorisation. That combination is why the technique produces large wins in simulation and numeric code.

Where it earns its cost, and where it does not

The honest accounting is that data-oriented design trades abstraction for throughput. Splitting entities into parallel arrays means there is no longer one place that represents a particle; creating and destroying entities requires keeping several arrays consistent; and the code reads as loops over indices rather than as domain operations. That is a real and ongoing maintenance cost.

It buys, in the right circumstances, several-fold throughput improvements. So the question is entirely about proportion: what fraction of runtime does this code account for, and how often does it change? A physics inner loop running millions of times per frame is worth restructuring. A configuration parser, an admin endpoint, or a workflow that runs once per request is not, no matter how satisfying the transformation would be.

The failure mode worth naming is applying it as an identity rather than a tool — restructuring an entire codebase around cache lines when a profiler would show that ninety-five percent of it never appears in a hot path. That is the same error as premature optimisation wearing different clothes, and it costs the clarity that lets you find the actual bottleneck later.

When to reach for it — and when not to
SituationWorth it?Reasoning
Inner loop over millions of entities, runs constantlyYesLayout dominates runtime; the maintenance cost is repaid continuously
Batch job scanning a large datasetYesBandwidth-bound; useful-line fraction is the limiting factor
Code the profiler shows as under 1% of runtimeNoEven a 10x win is invisible; you pay clarity for nothing
Business logic that changes weeklyNoMaintenance cost is paid repeatedly, performance benefit is negligible
Code with complex, evolving domain rulesRarelyAbstraction is doing real work; flattening it makes change expensive
A hot loop inside otherwise ordinary codeYes, locallyRestructure the loop and its data only, leave the rest alone

The transferable idea

Strip away the movement and the advocacy and one durable insight remains: layout is a design decision with performance consequences, and it is usually made implicitly. Most code inherits its layout from how the domain was modelled, without anyone asking what the hot loops will need. Simply making the decision consciously — even if you conclude the current layout is fine — is most of the value.

The second transferable idea is that this thinking scales beyond memory. Columnar database storage is data-oriented design at disk scale. Batching in a network protocol is the same instinct applied to round trips. Vectorised query execution in a database engine is precisely the "process many, not one" pattern. The mechanism differs; the reasoning is identical.

So the useful stance is neither adoption nor dismissal. It is: know the technique, know what it costs, profile before applying it, and apply it locally where the measurement justifies it. That is the same discipline this whole domain asks for — understand the machine, then decide deliberately rather than by default.

  • Make layout an explicit decision rather than an accident of domain modelling.
  • Apply locally to measured hot loops, not globally as an architectural style.
  • The same reasoning appears as columnar storage, batching and vectorised execution elsewhere.
  • Abstraction has value that does not appear in a profile; spending it should be a deliberate trade.

Key points

  • Data-oriented design means choosing layout to serve the transformations that run over the data, rather than the domain model.
  • Its two halves are dense contiguous layout and batched processing; the second matters as much as the first.
  • It trades abstraction and maintainability for throughput, which is a good trade only where the code is genuinely hot.
  • Applied globally as a style it is a costly mistake; applied locally to measured hot loops it is among the largest available wins.
  • The durable insight is that layout is a design decision usually made by accident — making it consciously is most of the benefit.

Follow the mechanism

The path through the machine, hop by hop — and the conclusions it invites that are wrong.

  1. 1
    Domain modelling → layout: entities become objects, so a hot loop's fields end up scattered across records and heap allocations.
  2. 2
    Scattered fields → cache lines: each iteration touches several lines and uses a small fraction of each.
  3. 3
    Indirection → dependent loads: pointer-based entity access adds dependent loads and unpredictable indirect branches.
  4. 4
    Restructure → dense arrays: fields the loop needs become contiguous, so every fetched byte is consumed.
  5. 5
    Batched loop → hardware: constant stride enables prefetching, removes indirect calls and permits vectorisation.
What people conclude from this — wrongly
  • • "Object-oriented code is slow." Indirection and scattered layout are slow. Objects with dense layout in cold code cost nothing worth measuring.
  • • "This should be the default style." It is a targeted optimisation. Used as a default it spends clarity everywhere to gain speed in a few places.
  • • "The transformation is obviously correct, so it does not need measuring." Layout effects depend on cache sizes and working-set size; verify rather than assume.

Consequences, controls and cost

What it causes
  • • Hot loops restructured this way commonly show several-fold throughput improvements with identical arithmetic.
  • • Codebases restructured wholesale become markedly harder to change, usually for negligible aggregate gain.
  • • Entity creation and deletion become more complex once fields live in parallel arrays.
What you can do
  • • Profile first, and restructure only the loops that measurement shows dominate runtime.
  • • Apply the transformation locally, keeping domain-shaped code at the boundaries of the hot region.
  • • Start with the cheapest variants — hot/cold field splitting and field reordering — before full decomposition.
  • • Re-measure afterwards; layout changes interact with cache sizes and can fail to help or even regress.
How to see it
  • • Profile to establish which loops actually dominate runtime before restructuring anything.
  • • Measure cache misses and achieved bandwidth before and after, not just wall time, so you know why it changed.
  • • Check whether the restructured loop vectorised, using compiler reports rather than inference.
What it costs
  • • Abstraction is lost: there is no longer a single place representing an entity, which complicates every future change.
  • • Creation, deletion and invariant maintenance across parallel arrays are error-prone in ways a single struct is not.
  • • The layout is tuned to today's access pattern, so a new access pattern can invalidate the whole arrangement.

Scope

§224 — what these claims are specific to.

What these claims are specific to
  • GENERALThe reasoning applies to any cache-based machine. The size of the win is MICROARCH-SPECIFIC and depends on cache sizes and working-set size relative to them.

Misconceptions

Claim
“Data-oriented design means never using objects.”
Reality
It means choosing layout to suit the hot transformations. Most of a codebase is not hot, and objects there cost nothing measurable.
Claim
“It is only relevant to game engines.”
Reality
The same reasoning produces columnar database storage, vectorised query execution and batched network protocols. Games popularised the name, not the idea.
Claim
“Restructuring for layout always helps.”
Reality
It helps when the working set and access pattern make layout the limiting factor. If the loop is compute-bound or already cache-resident, it changes nothing.

Apply it

🔍 One Field of Sixteen, and All of the Bandwidth

Where the rest of this lives

Programming Languages & Runtime Internals
Virtual dispatch and object representation

How much indirection an entity access costs is decided by the language and runtime — vtable dispatch, boxing and object headers all add loads that a flat array does not have.

← Array of Structs, or Struct of Arrays?