How DRAM Is Organised
DRAM is not a flat array. It is a grid of rows and columns across banks, and reading it means activating a whole row into a buffer first. Whether your next access hits that open row or forces another activation is a several-fold cost difference nothing in your code mentions.
Software view, hardware view
The gap between what you wrote and what the machine does is where this whole domain lives.
Rows, columns, banks
A DRAM chip stores bits in a grid. To read anything, the controller issues an activate command that pulls an entire row — thousands of bits — into a row buffer built from sense amplifiers. Only then can it issue read commands for specific columns within that row. When it needs a different row in the same bank, it must precharge: write the buffer back and close the row before activating the next.
Chips are divided into banks that can have different rows open simultaneously, and systems have multiple channels that operate in parallel. This is the DRAM-level source of memory parallelism: several requests can be in flight if they land on different banks and channels, which is a large part of why Misses That Overlap Are Nearly Free is possible at all.
Addresses are mapped across channels, banks, rows and columns by an interleaving scheme chosen by the platform. This mapping is why a stride that looks innocuous in your array indices can map every access onto the same bank and serialise work that should have been parallel.
| Case | What the controller does | Relative cost |
|---|---|---|
| Row hit | Read another column from the already-open row | Cheapest |
| Row empty (bank idle) | Activate the row, then read | Moderate |
| Row conflict | Precharge the open row, activate the new one, then read | Most expensive |
Why sequential access wins twice
Sequential access is already favoured by the cache hierarchy through Spatial Locality and Prefetching: The Hardware Guesses What You Will Read Next. At the DRAM level it wins a second time: consecutive addresses tend to land in the same row, so a stream of misses becomes a stream of cheap row hits after the first expensive activation.
Random access over a large region loses at both levels simultaneously. Every access misses cache, and each miss is likely to land on a different row, forcing precharge-activate cycles. This compounding is why the gap between sequential and random access on real hardware is far larger than the gap between "one cache miss" and "one cache miss" would suggest.
The trace below is schematic, not a real timing diagram — it shows the *sequence of commands*, which is the part that transfers between machines. Actual cycle counts depend on the DDR generation and the configured timings.
SEQUENTIAL STREAM (addresses within one row) ACT row 41 <- expensive: activate row into buffer READ col 0 <- cheap READ col 1 <- cheap READ col 2 <- cheap READ col 3 <- cheap one activation amortised over many reads RANDOM STREAM (addresses across rows in one bank) ACT row 41 READ col 7 PRE <- close row 41 ACT row 903 <- expensive again READ col 2 PRE <- close row 903 ACT row 12 <- and again READ col 5 every access pays full price
Refresh, and the cost you cannot avoid
DRAM stores each bit as charge in a capacitor, and charge leaks. Every row must be refreshed periodically or it loses its contents — this is the "dynamic" in DRAM, and it is the fundamental reason DRAM is cheaper and denser than the SRAM used for caches, which holds state actively and needs no refresh.
Refresh is handled by the controller and is invisible to software, but it is not free: a bank being refreshed cannot service requests. On large-capacity modules refresh overhead is a measurable fraction of available bandwidth. You cannot control it and should not try; it belongs in the mental model as a reason the memory system never delivers its theoretical peak.
The practical upshot for a programmer is small but real: the gap between a DRAM datasheet's peak bandwidth and what you can actually achieve comes from refresh, row conflicts, controller overhead and bus turnaround. Treat published peak bandwidth as a ceiling you approach, never reach.
- SRAM (caches) holds state in a latch — fast, power-hungry, physically large per bit, no refresh needed.
- DRAM (main memory) holds state as charge — dense and cheap per bit, but requires periodic refresh and has much higher access latency.
- This tradeoff is exactly why the The Memory Hierarchy exists: you cannot have capacity, speed and affordability simultaneously, so machines have layers.
| Property | SRAM (cache) | DRAM (main memory) |
|---|---|---|
| Bit storage | Cross-coupled latch, actively held | Capacitor charge, leaks over time |
| Refresh required | No | Yes, periodically, costing availability |
| Density per area | Low — several transistors per bit | High — roughly one transistor and one capacitor per bit |
| Access latency | Low | An order of magnitude or more higher |
| Used for | Caches, register files | Main memory |
Key points
- Reading DRAM requires activating an entire row into a row buffer; reads then take columns from that buffer.
- A row hit is much cheaper than a row conflict, which must precharge the open row before activating a new one.
- Banks and channels provide DRAM-level parallelism, and address interleaving decides which bank an address lands on.
- Sequential access wins twice — once in cache through locality, and again in DRAM through row hits.
- Refresh is mandatory, invisible and not free, and is part of why achieved bandwidth never reaches datasheet peak.
Follow the mechanism
The path through the machine, hop by hop — and the conclusions it invites that are wrong.
- 1Controller → bank: decode the physical address into channel, bank, row and column according to the platform's interleaving.
- 2Bank → row buffer: if the target row is not open, precharge any open row and activate the target, pulling thousands of bits into sense amplifiers.
- 3Row buffer → bus: read the requested columns; further columns from the same row are comparatively cheap.
- 4Refresh controller → banks: periodically steal cycles to refresh rows, during which the affected bank cannot answer.
- 5Bus → cache: the burst returns and fills a cache line at each level.
- • "Strided access is fine because it is regular." Regularity helps the prefetcher, but a stride that aliases onto one bank or cache set is pathological.
- • "My memory bandwidth is the number on the module." That is a theoretical peak; refresh, row conflicts and turnaround mean real workloads land well below it.
- • "DRAM is slow because it is far away." Distance matters, but the dominant cost is the activate-precharge protocol, not signal travel time.
Consequences, controls and cost
- • Strided access with an unlucky stride can map onto one bank repeatedly, serialising accesses that could have been parallel.
- • Sequential streaming approaches peak bandwidth; random access over a large region falls far short of it.
- • Achieved bandwidth is always below datasheet peak, and the gap grows with capacity and access randomness.
- • Prefer sequential or predictably strided access so consecutive requests hit open rows — the same discipline that helps the cache helps here.
- • Avoid power-of-two strides that can alias onto the same bank or set; padding an array dimension slightly often breaks the pattern.
- • For genuinely random large-scale access, increase the number of independent outstanding requests so bank parallelism can be exploited.
- • Do not attempt to manage refresh or row state directly — it is not exposed, and the correct response is to need fewer DRAM accesses.
- • Compare achieved bandwidth for a sequential stream against a random-access pattern over the same footprint; the ratio exposes row-conflict cost.
- • Vary a stride across a range and plot throughput — sharp dips at particular strides indicate bank or set aliasing.
- • Use platform memory-controller counters, where available, to read row-buffer hit rates directly.
- • Padding arrays to avoid aliasing wastes memory and can push a working set over a cache threshold, trading one problem for another.
- • Restructuring for sequential access sometimes requires materialising intermediate copies, spending bandwidth to save latency.
Scope
§224 — what these claims are specific to.
- PLATFORM-SPECIFICBank counts, channel counts, address interleaving and timing parameters vary by memory generation, module and firmware settings; DDR4 and DDR5 differ notably in bank grouping.
- SIMPLIFIEDOmits bank groups, rank interleaving, burst chop and write-to-read turnaround penalties, all of which affect real timing.