Separating Storage from Compute
Scale each independently, point many engines at one copy, pay for compute only while it runs — and pay a network, a cold start and the loss of locality for it.
Who needs this, what one row is, and why the obvious build breaks
Every lesson starts from the consumer, because designing from the source outward is this domain's characteristic mistake.
What changes when the machine holding the data is not the machine querying it, and what does that separation cost?
Anyone whose workload has to coexist with someone else's. The finance close that must not be affected by an analyst's runaway query; the ML job that needs an enormous cluster for two hours a month; the dashboard that needs steady, small, predictable compute all day (Workload Isolation).
The unit that moves is not a row but a block — a column chunk, a row group, a file range fetched over the network and cached locally. Everything about this architecture's performance is a question about how many of those blocks have to cross the wire and how many were already local (The Parquet Read Path).
Keep the data on the machines that query it, as every database has always done. Locality is real, the fastest read is one that never leaves the node, and a shared-nothing cluster with local disks is a genuinely excellent architecture for a fixed workload (What a Database Actually Is).
Storage fills up and the only way to add capacity is to add machines you did not need for compute — so you buy CPU to hold bytes (Right-Sizing Without Causing an Outage).
- Storage fills up and the only way to add capacity is to add machines you did not need for compute — so you buy CPU to hold bytes (Right-Sizing Without Causing an Outage).
- Month-end needs ten times the compute for two days. Coupled, that means resizing a cluster that also holds the data, which means moving the data, which is why nobody does it and everyone over-provisions instead (Idle Capacity: Headroom or Waste?).
- A second engine wants to read the same tables — a training job, a different SQL dialect, a notebook. Coupled, that means a copy, and a copy means divergence and reconciliation (Source of Truth).
- One team's badly written query saturates the nodes that also serve the finance dashboards, because there is one pool and it holds both the data and the CPU (Queueing: Why Systems Get Slow Before They Get Broken).
- The cluster is sized for peak and runs at a fraction of it most of the time, and shrinking it is not an option because the data lives there (Capacity Planning: Traffic to Machines).
What is actually happening
- Separation means the persistent copy lives in a shared store — object storage, or the vendor's equivalent — and compute nodes hold no authoritative data. They fetch what a query needs, cache it locally, and can be created or destroyed without a migration (Object Storage as Data Infrastructure).
- That is what makes compute elastic: adding capacity is starting processes, not rebalancing data. It is also what makes compute plural: several independent groups can read the same copy at once, isolated from each other by construction (Horizontal vs Vertical Scaling).
- It is only viable because analytical reads are large, sequential and prunable. You are not fetching a row by key across a network — you are fetching a small number of large column ranges after pruning has eliminated most of them (Predicate Pushdown, Projection Pushdown).
- The network in the middle is the cost, and it is mitigated rather than removed: aggressive pruning so less crosses it, local caches so repeated reads do not, and pushdown so filtering happens as close to the bytes as the storage layer allows. Locality therefore does not disappear, it moves — a compute group in a different region from the bucket pays cross-region latency and egress on every read, which can be worse than any coupling it removed (Source Pushdown, Egress: Moving Data Costs Money, Not Just Storing It).
- A cold compute group has an empty cache and no warm processes, so its first queries pay both a start-up cost and full remote reads. That cost is real and it is the reason "scale to zero" is not free (Startup Time & Cold Start).
- The separation is also a governance boundary: several engines reading one copy means the permissions each consumer gets depend on the engine, unless a shared catalog enforces them (Data Access Control).
One copy, many independently sized readers
The architecture is easiest to see as a before-and-after of one picture. Coupled, the data and the CPU are the same machines, so every capacity question is the same question and every workload shares one pool. Separated, the persistent copy is somewhere else and compute is a set of disposable groups pointed at it.
The consequence people reach for is elasticity. The consequence that matters more in practice is isolation: the finance close and the analyst's exploratory query no longer compete, not because a scheduler is arbitrating between them, but because they are running on different machines reading the same bytes.
The consequence people forget is the wire in the middle. Every block a query needs and does not have cached crosses a network, which is why the whole architecture rests on pruning: it is viable precisely to the extent that most of the table never has to move (Predicate Pushdown).
- The three compute groups share no CPU, no memory and no queue. That is the isolation, and it is structural rather than administrative (Multi-Tenant Isolation).
- They share one copy, so there is nothing to reconcile between them — the failure mode of the coupled alternative was always the extra copy (Source of Truth).
- The cache is where the lost locality is bought back, and its hit rate is the honest measure of whether this architecture is working for you (A 95% Hit Rate Tells You Almost Nothing).
- Scale-to-zero on the ML group is a cost decision that buys a cold start on the next query. Applying it to the BI group would be felt by every user every morning (Startup Time & Cold Start).
What actually happens on a query, and where the new costs are
The stages below are the same query lifecycle as a coupled engine, with two things inserted: a metadata read that decides what must move, and a network fetch for whatever survives that decision. Everything this architecture is good and bad at lives in those two stages.
Read the guarantees column and notice how narrow each promise is. Pruning guarantees only that skipped data could not have matched — it never guarantees that pruning happened, and pruning that silently does not apply is the single most common cause of a separated query being slow (Source Pushdown).
Notice also which stages are new failure surface. Cold start and remote fetch do not exist in a coupled system; cache staleness does not exist in a system with no cache. The architecture is a good trade and it is a trade, and its costs are concentrated in stages nobody had to think about before.
- 1Acquire compute
Starts or resumes the compute group the query is routed to.
guarantees Capacity isolated from other groups, once it is running.
fails by Cold start on an idle group, paid entirely by the first user after a quiet period (Startup Time & Cold Start).
- 2Resolve table
Reads the catalog pointer and the current snapshot metadata.
guarantees A single consistent file list for the whole query (Open Table Formats).
fails by Very many snapshots or manifests making planning slower than reading (The Lakehouse).
- 3Prune
Eliminates files using partition values and per-column statistics.
guarantees Anything skipped provably could not match. Pruning is never wrong, only absent.
fails by A predicate the engine cannot push down, or missing statistics — so the whole table crosses the network with no error (Partition Pruning).
- 4Fetch
Issues ranged reads for surviving column chunks; serves what the local cache already holds.
guarantees The bytes requested, eventually. Nothing about latency.
fails by Cross-region placement turning every fetch into a long, billed round trip (Egress: Moving Data Costs Money, Not Just Storing It).
- 5Cache
Retains fetched blocks locally for subsequent queries on this group.
guarantees Repeat reads of the same blocks avoid the network.
fails by Serving blocks from a snapshot the table has moved past, if invalidation is coarser than the commit rate (Cache Invalidation, Stampedes and Hot Keys).
- 6Execute
Decodes, filters, joins and aggregates across the group's workers.
guarantees Arithmetic correctness over the rows it was given.
fails by Skew concentrating work on one worker, which is unchanged by the separation (Data Skew).
- 7Release
Returns the result and optionally suspends the group.
guarantees Compute stops being charged when it stops running, on consumption-priced products.
fails by Suspending so eagerly that the next query pays a cold start and a cold cache, undoing the saving.
Two stages are new relative to a coupled engine — fetch and cache — and both failsBy entries for them are silent. That is where a separated platform quietly gets slow and expensive.
What the separation costs, honestly
The pattern is usually presented as a pure win, and for bursty workloads with well-laid-out data it very nearly is. The honest version has three costs, and each of them is invisible in the architecture diagram that sells it.
The first is the network. Every block that is not cached crosses a wire, and while bandwidth is plentiful, the latency of a round trip is not something bandwidth fixes. This is why layout stops being an optimisation and becomes a precondition (Latency and Bandwidth Are Different Resources).
The second is the cold start. A group that scales to zero has no processes and no cache, and its first query pays for both. The third is that "separated" is not "free of locality" — region placement, cache hit rate and pruning effectiveness are all locality questions, and getting any of them wrong costs more than the coupling ever did.
The comparison below is the same workload under both architectures, and the point is not that one wins. It is that they fail differently, and knowing which failure you have chosen is the difference between a platform you can tune and one you can only complain about.
Scales with bytes processed and with how well pruning worked. On consumption pricing this is where nearly everything lands.
The driver that does not exist in a coupled architecture. Reduced by pruning and by cache hit rate, both of which are layout and configuration decisions.
The cost of choosing latency over thrift. Grows with the number of groups, which is why isolation makes sprawl easy (Idle Capacity: Headroom or Waste?).
Zero when compute and storage are co-located and immediately significant when they are not. The most avoidable line on this list.
Continuous, independent of compute, and the reason retention can finally be a data decision instead of a hardware one.
Relative weights, not measurements. The ordering is what transfers between platforms; the magnitudes do not, and this domain never publishes a price.
Relative and unitless. Note that the second and fourth drivers are both network, and both are moved by decisions — layout and region placement — rather than by tuning the engine.
One cluster holds the data and runs both workloads. The cluster is sized for the month-end peak, so it is largely idle for three weeks. The analyst's unbounded scan competes with the close for the same CPU and buffer memory, and the only remedies are a scheduler policy and asking people to be careful. Adding capacity for the peak means adding nodes that also need data rebalanced onto them.
The close runs on its own group, sized for it, isolated by construction. The analyst runs on another. Month-end scales one group up for two days and back down. No data moves. The cost is that both groups fetch blocks over a network, both maintain their own caches, and the analyst's group pays a cold start if it has been idle since Friday.
The coupled architecture forces one capacity decision to serve two workloads with different shapes, because capacity and data are the same resource. Separating them makes capacity a per-workload decision, which is what buys both the isolation and the elasticity. The price is that reads now cross a network, so the architecture only works where pruning keeps most of the table from having to cross it (Partition Pruning).
Products differ substantially in how the separation is exposed: whether you provision a cluster by the hour or consume per query, whether several compute groups can read one storage layer, how caching is configured, and how quickly a suspended group resumes. Those differences change which of the drivers above dominates your platform. Verify against current documentation for the product and tier you actually run.
How to build it
Most important first.
- Give each workload class its own compute group sized for it, so isolation is structural rather than a scheduling policy you have to enforce (Workload Isolation).
- Put compute in the same region as the bytes, always. This one is boring and it is the most common expensive mistake in the pattern (Regions and Availability Zones).
- Design the layout so pruning does most of the work, and push filtering and projection as close to storage as the layer allows — then verify it is actually happening, because pushdown that silently does not apply is the most common reason a separated query is slow. In this architecture layout is not a performance optimisation, it is what makes the architecture viable at all (Partition Pruning, Source Pushdown, File Size and the Small-Files Problem).
- Keep long-lived compute warm for latency-sensitive workloads and let batch workloads start cold. The two have opposite answers and applying one policy to both is how you get either waste or slow dashboards (Cost vs Freshness).
- Enforce access in a shared catalog rather than per engine, or the isolation you built becomes a governance hole (The Data Catalog).
- Measure cache hit rate per compute group. It is the number that tells you whether your separation is working, and almost nobody looks at it (A 95% Hit Rate Tells You Almost Nothing).
What this actually promises
Naming the guarantee you do not have is worth more than naming the one you do — everything downstream inherits the weakest promise in the chain.
- Independent scaling: compute capacity can change without moving data, and storage can grow without adding compute. This is the guarantee the architecture exists for (Autoscaling).
- Isolation between compute groups reading the same data — one group cannot consume another's CPU or memory, because they do not share any (Multi-Tenant Isolation).
- One authoritative copy readable by several engines, so there is nothing to reconcile between them (Source of Truth).
- No guarantee about consistency between engines beyond what the table layer provides. Two engines reading at different moments read different snapshots, which is correct and surprises people (Open Table Formats).
- No guarantee of performance parity with a co-located system for small, latency-sensitive reads. The architecture is tuned for large scans and it is honest about being bad at point lookups (OLTP vs OLAP).
Can I trust it?
A green pipeline is evidence that code ran. These four fields are the evidence that the data is right.
- The architecture-specific check is a cross-engine agreement test: run the same aggregate over the same table from every engine you support, on a schedule, and assert the results match (Reconciliation).
- It catches the failure that only this architecture has — one engine ignoring row-level deletes or an evolved schema, so the same table gives two answers depending on the tool (Open Table Formats).
- It misses everything both engines get wrong identically, and it says nothing about correctness relative to the source. It is an agreement test, not a truth test.
- Separation does not change freshness. Data is as fresh as the commit that published it; where the compute runs is irrelevant to that (The Lakehouse).
- What it changes is time to first answer for an idle workload: a cold compute group must start and warm its cache, so the first query after a quiet period is slower than the tenth (Startup Time & Cold Start).
- It also makes freshness *per compute group* a real concept, because a group with an aggressive cache can serve stale blocks if its invalidation is coarser than the table's commit rate — a subtle staleness that no pipeline metric shows (Cache Invalidation, Stampedes and Hot Keys).
- Adding an engine is the evolution this architecture is built to make cheap, and it is only cheap if the new engine supports the table format completely. Verify deletes, schema evolution and statistics before promising it (Federated Query).
- Changing compute sizing is not a data migration, which is the entire point — the decision becomes reversible in a way it never was with coupled storage (Right-Sizing Without Causing an Outage).
- Moving the bytes — to another region, another bucket, another storage class — is still a real migration, because every engine, catalog and pipeline references the location (Impact Analysis).
- Losing a compute node loses in-flight queries and a warm cache, and loses no data. That is a genuinely different failure class from losing a node that held a data replica (Partial Failure).
- Recovery is therefore mostly re-running queries and re-warming caches, both of which are automatic. The architecture makes compute disposable on purpose (Stateless vs Stateful Services).
- Losing storage is a different matter entirely, and the fact that compute is disposable can make teams complacent about the layer that is not (Disaster Recovery).
What can go wrong
- Compute in a different region from the bucket, paying cross-region latency and egress on every block, discovered from a bill rather than from a monitor (Egress: Moving Data Costs Money, Not Just Storing It).
- Pruning silently not applying — a function wrapped around a partition column, or statistics that were never written — so every query fetches the whole table across the network (Partition Pruning).
- Cold-start latency on a workload that scales to zero, experienced by users as a dashboard that is slow only in the morning (Startup Time & Cold Start).
- A cache that hides a correctness problem: a compute group serving cached blocks from a snapshot the table has moved past (Cache Invalidation, Stampedes and Hot Keys).
- Isolation used as an excuse for sprawl: thirty compute groups, each idle most of the time, each too small to justify deleting (Idle Capacity: Headroom or Waste?).
- The mitigation failing: aggressive scale-to-zero configured to control idle cost, producing cold starts that push users to keep a group permanently warm, which costs more than the original waste.
- "Separating storage and compute is cheaper." It changes *what* you pay for — from provisioned capacity to consumed capacity plus network. That is a large saving for bursty workloads and can be a loss for steady ones running near capacity (Fixed vs Variable Cost).
- "The separation means locality does not matter." Locality moved; it did not vanish. Region placement, cache hit rate and pruning effectiveness are all locality questions and they decide performance here more than they did before (The Memory Hierarchy).
- "Compute is stateless, so nothing is lost when it dies." The warm cache is lost, and on a large group that is a real re-warming cost. Disposable is not the same as free (Cache Warmth and the Real Cost of Migration).
- "We can point any engine at the data now." Only engines that implement the table format completely, at the version you run. Portability is verified per engine, not implied by the architecture (Open Table Formats).
- "Elastic means we never over-provision." Elastic means over-provisioning is now a per-group decision made many times, which in practice is how platforms end up with thirty idle compute groups (Cost Attribution).
Operating it
- Cache hit rate per compute group — the single most diagnostic metric in this architecture and the one most often not collected (A 95% Hit Rate Tells You Almost Nothing).
- Bytes read from remote storage versus bytes read from local cache, per query. The ratio tells you whether pruning and caching are earning their keep (Scan Cost).
- Queue time versus execution time per compute group, which separates "this group is too small" from "this query is too big" (Queueing: Why Systems Get Slow Before They Get Broken).
- Cross-region and egress bytes, alarmed rather than reviewed, because this is the failure that is silent until the bill (Egress: Moving Data Costs Money, Not Just Storing It).
- Cold-start frequency and its latency contribution, so the scale-to-zero policy can be argued with evidence (Startup Time & Cold Start).
- At 10x concurrency, separation is what lets you add compute groups rather than a bigger cluster, and the marginal cost of an isolated group is genuinely small (Horizontal vs Vertical Scaling).
- At 100x data, the network becomes the binding constraint unless pruning improves at the same rate — which means layout work is not optional at that size, it is the architecture (Physical Data Layout).
- At high engine counts the compatibility surface grows faster than the benefit, and each additional engine is a correctness question rather than a convenience (Query Engines).
- Compute is paid for while it runs, so idle capacity becomes a policy decision rather than a sunk cost. This is the saving the architecture is sold on and it is real — for workloads that are genuinely bursty (Idle Capacity: Headroom or Waste?).
- Storage is paid for continuously and independently, which is what allows retention to be a data decision rather than a hardware one (Storage Lifecycle).
- Network is the new line item: bytes crossing between storage and compute, plus egress if they cross a region or a provider boundary. It is invisible in a coupled system and unavoidable here (Egress: Moving Data Costs Money, Not Just Storing It).
- Cache infrastructure — local SSD, memory held for block caches — is compute cost incurred to avoid network cost, and it is the lever with the best return in most separated platforms (A 95% Hit Rate Tells You Almost Nothing).
- You gain elasticity, isolation and one shared copy. You pay a network hop on every cold read, a cold start on every idle group, and the need to design layout well enough that pruning carries the architecture.
- Scale-to-zero minimises idle cost and maximises cold-start latency. Always-warm inverts both. Most platforms need both policies applied to different workloads, and applying one everywhere is the common mistake.
- Caching recovers most of the lost locality and introduces an invalidation problem. It is the right trade and it is not a free one (Cache Invalidation, Stampedes and Hot Keys).
Separation of storage and compute
Change an input and watch which number moves — and which one does not. Everything here comes from a model in this repository, not from a measurement.
| Coupled | Separated | |
|---|---|---|
| Storage held | 122880.0 GBsim | 122880.0 GBsim (one copy, shared) |
| Scaling | More storage means more nodes, whether or not you need the compute. | Each scales alone, which is the actual point. |
| Isolation | One noisy job slows every other query on the cluster. | A cluster per workload, reading the same tables. |
| What it costs | Idle capacity, and a copy per cluster. | Every read crosses a network, so caching and layout matter far more than they did. |
Where this applies
Almost nothing here is universal. These labels say what each claim is specific to, and where a different engine, format, warehouse or scale would differ.
- GENERALThe trade — elasticity and isolation bought with a network hop, a cold start and a dependence on pruning — holds for any architecture where the authoritative copy is not on the querying node, whether that is a lakehouse over object storage or a managed warehouse with a shared storage layer.
- WAREHOUSE-SPECIFICHow completely storage and compute are separated differs by product: some let you run several independently sized compute groups against one storage layer with per-second billing, others separate them internally while still exposing a provisioned cluster you size and pay for by the hour. The first makes isolation cheap, the second makes it a capacity decision.
- SCALE-SPECIFICBelow the point where one cluster comfortably holds both the data and the peak workload, coupling is simpler and faster, and the separation buys nothing but a network hop. It starts paying once workloads have genuinely different shapes or the peak is far above the median.
Where the depth lives
This domain teaches how data moves and how you know it arrived intact. It hands the rest off by name.
- — Distributed Systems owns what a shared storage layer is doing underneath to be durable and available across zones, and why a compute node that loses contact with it should fail rather than guess.
- — DevOps / Production Engineering owns provisioning compute groups as infrastructure code, the autoscaling policy that decides when a group suspends, and the review that stops a group being created in the wrong region.