Event vs Snapshot Modeling
Events record what changed; snapshots record what was true at time T. Different storage curves, different query complexity, and different questions made easy.
Who needs this, what one row is, and why the obvious build breaks
Every lesson starts from the consumer, because designing from the source outward is this domain's characteristic mistake.
Should this be modelled as a stream of changes or as periodic captures of state, and which questions does each choice make hard?
Product analytics wants transitions — what changed, in what order, how long between steps. Finance wants state — what was true at the period boundary. Machine learning wants state as of a point in time with no leakage. One model does not serve all three well, and pretending otherwise is how platforms end up with both, built badly.
An event row is one change: a state transition with a timestamp. A snapshot row is one entity in one period. The two are duals — a snapshot is a fold over events up to a boundary, and a sequence of snapshots is a lossy sampling of the events between them (Grain: What Does One Row Represent?).
Pick one. Either store every change and derive state when asked, or store state periodically and forget the changes. One model is simpler to build, simpler to explain and simpler to operate.
Events only: every state question becomes a fold over all history, and the ten dashboards that ask "how many active subscriptions" each recompute it from the beginning of time (Scan Cost).
- Events only: every state question becomes a fold over all history, and the ten dashboards that ask "how many active subscriptions" each recompute it from the beginning of time (Scan Cost).
- Events only: some state changes have no event. An administrative correction, a batch reclassification, a stock count that reconciles rather than posts — the state moved and nothing recorded a transition (What a CDC Event Contains).
- Snapshots only: "how many customers churned and came back within thirty days" is unanswerable, because a churn and a return between two snapshots is invisible (Snapshot Tables).
- Snapshots only: "how long does an order take from placed to shipped" needs the transition timestamps, and the snapshot recorded only that it was shipped by Tuesday.
- Either alone: a late-arriving correction. With events you replay and the state changes retroactively; with snapshots you either restate a closed partition or carry the correction forward, and neither is obviously right (Late-Arriving Data).
What is actually happening
- An event model stores immutable records of change: what happened, to what, when, and what it changed to. It is append-only, it grows with activity, and it is the only representation that can answer questions about sequence, duration and transition (The Event Log).
- A snapshot model stores state at boundaries. It is append-only per period, it grows with entities times periods, and it answers point-in-time questions with a lookup (Snapshot Tables).
- The relationship between them is a fold. State at time T equals the initial state plus every event up to T. That means a snapshot is derivable from events, and events are not derivable from snapshots — the arrow goes one way, which is the single most useful fact in this lesson (Event Sourcing).
- The trade is therefore storage against query complexity, and it is not symmetric. Events cost less storage and more query work; snapshots cost more storage and less query work. Because storage is the cheaper of the two resources in most platforms and query work is paid repeatedly, the common answer is to keep events as the base and materialise snapshots for the questions that need them.
- Sampling is the loss. A snapshot cannot see anything that happened and reverted between two captures, and it cannot see the order of two changes within one period. Shorter periods reduce the loss and increase the cost linearly (Windows).
- The two models also fail differently. An event model degrades by growing until folds are infeasible. A snapshot model degrades by growing until storage is the problem. The first failure is a query-cost cliff and the second is a slow storage curve, and teams notice them at very different times (What Actually Drives Data Platform Cost).
The same subscription, two ways
Take one subscription over one month. In an event model it is a handful of rows: created, upgraded, payment failed, downgraded, cancelled, each with a timestamp. In a snapshot model it is thirty rows, one per day, each recording the plan and status as of that day.
The event model is smaller and answers questions the snapshot cannot: how long between upgrade and downgrade, how many payment failures preceded the cancellation, in what order the changes happened. The snapshot answers questions the event model can only answer by folding: how many subscriptions were on the Pro plan on the 14th, and how did that count move across the month.
The comparison below is the trade stated plainly. What makes it a design decision rather than a preference is the last line: you can always build the right-hand side from the left, and you can never build the left from the right.
Thirty rows, one per day, each recording plan and status. Answers "how many were on Pro on the 14th" with a partition read. Cannot answer how many payment failures occurred, in what order the changes happened, or how long the Pro period lasted — and cannot be turned into a table that can, ever, because the transitions were never recorded.
Six event rows carrying every transition with its timestamp, plus a daily snapshot derived from them for the state questions that are asked repeatedly. Transition questions read the events; state questions read the snapshot; and any snapshot that turns out to be wrong, or at the wrong period, is rebuilt by re-folding.
The derivation is one-directional. State at any instant is the fold of events up to it, so events can always produce a snapshot; a snapshot discards the transitions between captures, so it can never produce events. Keeping the representation that can generate the other one is not a preference — it is the only choice that leaves a decision reversible.
What a snapshot cannot see
The loss is easiest to see on a timeline. Below, one subscription changes four times across two days. The daily snapshot runs at midnight and captures state at each boundary, so it sees two states and misses two transitions entirely — including one that reverted before it was ever observed.
That is not a bug in the snapshot. It is what sampling means, and it is the reason "how many customers downgraded and upgraded again within a week" is an event question that no snapshot table will ever answer, at any period granularity short enough to be affordable.
The mirror observation matters just as much. Answering "how many were on Pro at each midnight" from the events requires folding every event up to each boundary, thirty times for a month's chart. Both models are doing work the other one already did; the practical answer is to keep the events and materialise the boundaries you ask about repeatedly.
| Event | Happened | Arrived | Lands in |
|---|---|---|---|
| E1 upgraded to Pro | 2026-03-13 09:41 | 2026-03-13 09:41 | Day 1 Visible in the Day 1 snapshot: the plan at midnight is Pro. |
| E2 payment failed | 2026-03-14 03:20 | 2026-03-14 03:20 | Day 2 Status becomes past_due. Reverted before the Day 2 boundary, so the snapshot never sees it. |
| E3 payment retried, succeeded | 2026-03-14 04:05 | 2026-03-14 04:05 | Day 2 Status returns to active. E2 and E3 cancel out: the snapshot records no change at all for the day. |
| E4 downgraded to Basic | 2026-03-14 22:15 | 2026-03-14 22:15 | Day 2 Visible in the Day 2 snapshot: the plan at midnight is Basic. |
| E5 support note added | 2026-03-14 22:16 | 2026-03-16 11:02 | Day 2 Arrived two days late. The Day 2 snapshot was already written, so the stored partition and a fresh fold now disagree (Late-Arriving Data). |
Two snapshot rows record Pro then Basic. The event log records five things, two of which — a failure and its recovery — cancelled out between boundaries and are permanently invisible to the snapshot. Shortening the period to hourly would catch E2 and E3 and multiply the table by twenty-four.
Choosing, per question rather than per platform
Almost nobody picks one model for a whole platform. The realistic decision is per subject area, and the deciding factor is which questions are asked repeatedly and which axis — entities or activity — the business grows along.
The one general rule is the asymmetry: keep the events if you can get them, because everything else is derivable from them and they are not derivable from anything. That is a statement about optionality rather than about performance, and it is the reason the hybrid dominates in practice.
The options below are all legitimate. The costs are the lesson, and the third one is the answer most mature platforms converge on after arriving at it the expensive way.
Which questions are asked repeatedly, and can you actually obtain a complete event stream from the source?
when History is short, questions are mostly about transitions, and the fold still fits comfortably in a query.
cost Fold cost grows with history and is paid on every query. It is fine for two years and then suddenly is not, with no change in design to blame (Scan Cost).
when The source exposes only current state — a SaaS API with no change feed — so events are not obtainable at any price.
cost Everything between captures is permanently invisible, and no future decision can recover it. Buys point-in-time answers, which is often all that is genuinely needed (Snapshot Tables).
when Both transition and state questions are asked, and a complete event stream is available.
cost Two models, a reconciliation between them, and a fold schedule to own. Buys every question and full rebuildability — the usual destination (Raw, Staging, Curated: Layers by Purpose).
when Current state must be as fresh as ingestion, and folding all history is too slow.
cost Every query is a union of two sources with different freshness, and the seam between them is a place for double counting. Buys bounded fold cost at event freshness (Lambda Architecture).
when The process is bounded with known milestones — order placed, paid, shipped, delivered.
cost Rows are mutable, so incremental loads keyed on insert time miss the updates, and the row cannot say what the state was at a past date unless it is also versioned. Buys durations in one row with no fold (Fact Tables).
How to build it
Most important first.
- Keep the events. They are the base representation from which any snapshot can be rebuilt, and losing them makes every future modelling decision irreversible (Keeping Raw History: The Recovery Position and the Liability).
- Materialise snapshots for the state questions that are asked repeatedly, at the coarsest period those questions tolerate. Month-end is enough for a surprising amount of reporting (Snapshot Tables).
- Model the transitions you care about as first-class events with explicit timestamps, rather than inferring them by diffing consecutive snapshots — diffing works and is slower, lossier and harder to explain (Commands vs Events).
- Where a state change has no natural event, generate one at the point of detection and record that it was inferred. An event that says "observed as changed at T" is honest; silently treating detection time as event time is not (Event Time).
- Use an accumulating snapshot for a bounded process with known milestones — order placed, paid, picked, shipped, delivered — which gives durations in one row without a fold (Fact Tables).
- Decide the late-data policy once, per dataset: restate history, or carry corrections forward. Different consumers need different answers and the choice must be visible in the documentation (Late-Arriving Data).
What this actually promises
Naming the guarantee you do not have is worth more than naming the one you do — everything downstream inherits the weakest promise in the chain.
- Events guarantee a complete record of transitions only if the capture is complete. A CDC gap, a dropped batch or a change made outside the captured path leaves a hole that no fold can detect, because the fold has no way to know an event is missing (CDC Failure Modes and the Retention Deadline).
- Snapshots guarantee the state as observed at the boundary and explicitly guarantee nothing about the interval between boundaries.
- A snapshot derived from events guarantees consistency with those events at derivation time. It does not stay consistent: a late event changes the correct fold and does not change the already-written partition (Late-Arriving Data).
- Neither model guarantees ordering by itself. Events ordered by arrival are not ordered by occurrence, and a fold applied in arrival order can produce a state that never existed (CDC Ordering and Transaction Boundaries).
Can I trust it?
A green pipeline is evidence that code ran. These four fields are the evidence that the data is right.
- For events: completeness against the source. Count events per period and reconcile against the source system's own count of changes, because a fold over an incomplete event log produces a confident state that is simply wrong (Reconciliation).
- For snapshots: row count per partition and continuity per entity, which catch partial loads and missed periods (Snapshot Tables).
- Across both, the highest-value check is agreement: recompute the snapshot from the events for a closed period and assert it equals the stored snapshot. A divergence means either the event log has a gap or the snapshot was taken from a source that had already moved on, and either is worth knowing (Data Tests).
- What all of them miss: a change that produced no event and happened between two snapshots. Both models are internally consistent, they agree with each other, and neither ever observed it. The only detection is a source-side audit trail, and the honest response is to document the class of change that is invisible rather than to claim coverage (Audit Logs for Privileged Actions).
- Events can be as fresh as ingestion allows, and a state derived from them is as fresh as the fold's schedule. Splitting those two is what lets a platform serve a real-time transition feed and a daily state table from one source (Batch vs Streaming Ingestion).
- Snapshots are exactly as fresh as their period. A daily snapshot is never intraday, however fast the job runs.
- Hybrid serving — the latest snapshot plus the events since it — gives current state at event freshness with bounded fold cost, and costs a more complicated query and one more thing that can be inconsistent (Lambda Architecture).
- Adding a new event type is additive and safe; existing folds ignore it until they are taught about it, which is a silent behaviour change unless the fold fails on unknown types (Schema Evolution).
- Changing what an event means — the same type now fired in more situations — breaks every historical fold with no schema signature. This is the classic semantic change and it is more damaging in an event model than anywhere else, because history is replayed rather than merely read (Semantic Changes).
- Adding a column to a snapshot leaves history null unless the events can rebuild it, which is one of the concrete arguments for keeping events even after snapshots exist.
- Moving from snapshots to events retroactively is impossible for the past. Moving from events to snapshots is a batch job. That asymmetry should decide which one you start with (Event Sourcing).
- Events are the recovery position. Any snapshot, aggregate or derived state can be rebuilt from a complete event log, which is why retention on the log is a recovery-window decision rather than a storage one (Retention and Replay).
- A snapshot alone cannot be recovered if the source state for that instant is gone and no events exist. That is the single strongest argument for keeping both (Keeping Raw History: The Recovery Position and the Liability).
- Replay is not free of hazards: a fold that references the current dimension state, or
now(), produces a different answer on replay than it did originally, and the difference is invisible (Reprocessing vs Retrying).
What can go wrong
- A fold over an event log with a gap, producing a confident state that is wrong and internally consistent (CDC Failure Modes and the Retention Deadline).
- A snapshot missing a period, with queries interpolating over the gap silently (Missing Rows).
- Transitions inferred by diffing consecutive snapshots, which merges two changes in one period into one and loses reverted changes entirely.
- A derived snapshot and its source events diverging after a late correction, with nothing comparing them (Late-Arriving Data).
- The mitigation failing: keeping both models and letting each be maintained by a different team, so they disagree and each team trusts its own (Who Owns Data Quality).
- "Event sourcing means we do not need snapshots." It means you can always derive them. Deriving state on every query is a cost decision, and at any real history length it is the wrong one (Event Sourcing).
- "Snapshots are just denormalised events." They are a lossy sampling. The information a snapshot lacks is not compressed, it was never captured (Snapshot Tables).
- "We can reconstruct transitions by diffing snapshots." You can reconstruct *net* change per period. Two changes within a period become one, and a change that reverted becomes none.
- "Events are the source of truth, so the snapshot cannot be wrong." A snapshot derived before a late event arrived is inconsistent with the events and nothing in either table says so (Late-Arriving Data).
- "Streaming means events and batch means snapshots." Orthogonal. A batch pipeline can produce an event model and a streaming one can maintain a state table; the question here is representation, not processing mode (Batch vs Streaming Ingestion).
Operating it
- Events per period against the source's own change count — the completeness signal for the base model (Reconciliation).
- Divergence between the stored snapshot and a recomputed fold for a closed period, tracked as a number rather than a boolean (The Data Quality Dashboard).
- Fold runtime over time. It grows with history in an event-only model, and the day it stops fitting the schedule is predictable months in advance if anyone is watching (Pipeline Metrics).
- Storage growth of the snapshot against the event log, which shows the crossover where the derived table costs more than its source (Cost Attribution).
- At 10x activity, event volume grows and folds slow proportionally; snapshots are unaffected because they scale with entities, not with events.
- At 10x entities, snapshots grow and events are unaffected. The two models scale along different axes, which is why the right answer depends on which axis your business grows.
- At 100x history, event-only folds stop being feasible and incremental materialisation becomes mandatory — a snapshot plus the events since it, which is the hybrid arrived at by necessity rather than by design (Incremental Processing).
- Events cost storage proportional to activity and query cost proportional to history folded. The second grows even when the first does not (Scan Cost).
- Snapshots cost storage proportional to entities times periods and query cost proportional to the periods actually read, which partition pruning makes small (Partition Pruning).
- Keeping both costs both, and is usually still right: the events are the recovery position and the snapshots are the serving layer, which is the same argument as raw-plus-curated made one level down (Raw, Staging, Curated: Layers by Purpose).
- The expensive mistake is folding all history on every query in an event-only model. It is invisible while history is short and becomes the dominant cost line without any change in design (Compute Waste).
- Events buy complete auditability, transition questions and the ability to rebuild anything, and cost query complexity that grows with history plus the discipline to keep the log complete.
- Snapshots buy cheap point-in-time answers and reproducible closed periods, and cost storage that grows on a schedule plus permanent blindness to what happened between captures.
- Keeping both buys the union of the answers and costs the union of the operational surface, plus a reconciliation between them that somebody has to own.
Modeling lab — one grain, ten questions
Change an input and watch which number moves — and which one does not. Everything here comes from a model in this repository, not from a measurement.
| Business question | At this grain | Why |
|---|---|---|
What was revenue by country last month? Wants: One order, or one order line — either works, provided the measure is additive at that grain and is not summed twice. | answered | The measure is additive at this grain and each order is counted once. |
What is the average order value? Wants: One order. An average over lines answers a different question entirely. | answered | The denominator is orders, which is exactly what one row is. |
What was revenue by customer country at the time of each order? Wants: One order, joined to the version of the customer that was current when the order was placed. | WRONG no history | With a Type 1 dimension every historical order is attributed to the customer's current country. A customer moving from Poland to Germany silently rewrites last year's regional reports, and last month's report no longer reproduces. |
What is net revenue after refunds? Wants: One order, with refunds either netted into the measure or held as a separate signed fact at the same grain. | answered | Refunds net into the measure, or sit beside it as a signed fact at the same grain. |
What was the total account balance on each day last year? Wants: One account-day. A balance is a state, not an event, and cannot be reconstructed by summing transactions unless every transaction since account opening is retained. | WRONG | Summing transactions per day gives the daily change in balance, not the balance. The chart has the right shape and the wrong y-axis. |
What is month-three retention by signup cohort? Wants: One user-month of activity, joined to the user's signup month. | WRONG | Users who were active but did not buy are invisible, so retention is understated by exactly the non-buyers. |
What share of sessions ended in a purchase? Wants: One session — which requires a session window over events, because no source system emits a session. | unanswerable | No source system emits a session. Without a session window over events there is no denominator to divide by. |
Which products are most often bought together? Wants: One order line, with the order key retained so lines can be grouped back into baskets. | unanswerable | The most instructive failure in this lab: the model is not wrong, it is at the wrong resolution, and no query can recover what was aggregated away. |
What was yesterday's revenue, asked at 06:00 this morning? Wants: One order, in a period that is not yet closed. | with care | The grain is right and the period is not closed. Orders that happened yesterday and arrive later today are still missing at 06:00. |
What was global revenue, across markets that bill in different currencies? Wants: One order, with both the transaction amount and the converted amount stored, plus the rate and the date the rate applied. | WRONG no history | Converting at query time with today's rate makes every historical report change daily. Converting once with no record of the rate makes the number unreproducible. Both pass every type check there is. |
| Stage | One row is | Rows in | Rows out | Δ |
|---|---|---|---|---|
Source database | One order, in its current state. | 4,000 | 4,000 | — |
Change capture | One committed change to one order. | 4,000 | 4,000 | — |
Event log | One delivered change record — possibly delivered more than once. | 4,000 | 4,000 | — |
Raw landing | One line in an immutable file, exactly as received. | 4,000 | 4,000 | — |
Transformation | One order, deduplicated and windowed. | 4,000 | 4,000 | — |
Serving table | One order, with measures and dimension keys. | 4,000 | 4,000 | — |
Dashboard | One number, with the grain now invisible. | 4,000 | 1 | aggregated |
Row counts are simulatedsim. The last row is where the grain disappears: one number, with nothing on the screen recording what one row of the source meant.
| Check | Result | What the model found | Still misses |
|---|---|---|---|
Completeness Every order the source recorded for the period reached the serving table. | pass | Every order in the source for this period is present. | Duplicates that coincidentally offset losses, and any period that is not yet closed. |
Uniqueness Each order id appears exactly once in the serving table. | pass | Every order id appears exactly once. | A genuine duplicate that arrived under a new key — a producer retry with a fresh event id looks like a second order. |
Freshness The newest complete record is recent enough for the decisions this table drives. | pass | Newest complete record is 12 simulated minutes old. | Data that is perfectly fresh and completely wrong. It also fires falsely on a period where the source genuinely produced nothing. |
Validity Every amount is non-null and parses as a number. | pass | Every amount is non-null and numeric. | A value that is well-typed and wrong — a price in the wrong currency passes every type check there is. |
Distribution The shape of the day resembles the days before it, per country and in total. | pass | Largest per-country share drift 0.8pp; total volume drift 0.0%. | Slow drift, and any error that preserves the shape while changing every value inside it. |
Reconciliation Revenue summed in the serving table equals revenue summed in the source for the same closed period. | FAIL | Serving table reports 1,010,654.50 against a source total of 934,498.90. | Anything wrong identically at both ends — a bug in logic shared by the extract and the model reconciles perfectly. |
- 1Refunded orders were counted at their full value. Every row is present, unique, fresh and well-typed — and revenue is overstated.
The revenue model stops subtracting refunds.
The code does exactly what it was told, and what it was told is wrong. Every row is present, unique, fresh, well-typed and normally distributed — and the number is too high.
Where this applies
Almost nothing here is universal. These labels say what each claim is specific to, and where a different engine, format, warehouse or scale would differ.
- GENERALThe duality — a snapshot is a fold over events, and a sequence of snapshots is a lossy sampling of them — is arithmetic and holds everywhere. What varies is which side is cheap in a given platform, which is a storage-versus-compute question rather than a modelling one.
- SOURCE-SPECIFICWhether an event model is even available depends on the source: a database with CDC gives you every committed change, an application emitting domain events gives you business intent as well as the change, and a SaaS API that exposes only current records gives you nothing but the ability to take your own snapshots.
- SCALE-SPECIFICBelow a modest history length, folding events on every query is perfectly fine and building snapshots is premature. The crossover is where fold runtime stops fitting the schedule, and it arrives suddenly for a query pattern that had been fine for two years.
Where the depth lives
This domain teaches how data moves and how you know it arrived intact. It hands the rest off by name.
- — Distributed Systems owns why a replayed fold can produce a different answer than the original run, and what it takes for a derivation to be deterministic across replays.