Event Sourcing
Instead of storing balance = 100, store AccountCreated +100, PaymentMade −20, RefundReceived +20 and derive the balance by replaying the log; you gain a complete audit trail, time travel and rebuildable projections, and pay with snapshots, event versioning, GDPR pain and a model most applications do not need.
A row that stores current state destroys history: after UPDATE accounts SET balance = 100 nobody can answer "why is it 100?", "what was it on Tuesday?" or "rebuild the read model from scratch". Event sourcing stores the facts that happened, in order, as the source of truth, and treats every state as a derivation of them.
State as a fold over events
A conventional system stores accounts(id, balance) and overwrites the balance on every change. An event-sourced system stores an append-only log per aggregate: AccountCreated { initial: 100 }, PaymentMade { amount: 20 }, RefundReceived { amount: 20 }. The current balance is not stored anywhere authoritative; it is computed by replaying the events through a reducer — 100 − 20 + 20 = 100. The log is the truth; everything else is a cache of it. Writes append an event after validating the command against the replayed state; they never UPDATE.
Replay is a fold, exactly like a running total: state after event *n* is apply(state after n−1, event n). That is also why snapshots work — store the state at event 1,000 and replay only events 1,001 onward, the same trick as a Prefix Sum cached at checkpoints. Without snapshots an aggregate with a million events takes a million steps to load; with a snapshot every 100 events it takes at most 100.
1type Event =2 | { type: 'AccountCreated'; initial: number }3 | { type: 'PaymentMade'; amount: number }4 | { type: 'RefundReceived'; amount: number }5 6const apply = (balance: number, e: Event): number => {7 switch (e.type) {8 case 'AccountCreated': return e.initial9 case 'PaymentMade': return balance - e.amount10 case 'RefundReceived': return balance + e.amount11 }12}13 14async function load(accountId: string) {15 const snap = await snapshots.latest(accountId) // { seq: 1000, balance: 340 } or null16 const events = await log.after(accountId, snap?.seq ?? 0) // only the tail17 return events.reduce(apply, snap?.balance ?? 0)18}19 20async function pay(accountId: string, amount: number, expectedSeq: number) {21 const balance = await load(accountId)22 if (balance < amount) throw new Error('insufficient funds')23 await log.append(accountId, { type: 'PaymentMade', amount }, expectedSeq) // optimistic concurrency on seq24}state storage event sourcing
accounts account_events (append-only)
id balance seq type data at
acc1 100 1 AccountCreated {initial:100} 09:00
2 PaymentMade {amount:20} 09:05
3 RefundReceived {amount:20} 09:40
"why 100?" → unknown "why 100?" → replay: 100 − 20 + 20
"at 09:10?" → unknown "at 09:10?" → replay events with at ≤ 09:10 → 80What the log gives you
Auditability for free: every change is a first-class record with a timestamp, a cause and (if you store it) an actor — regulators and support teams ask "what happened to this account?" and the log *is* the answer. Temporal queries: the state at any past moment is a replay up to that moment. Rebuildable projections: every read model (CQRS) is derived, so a projector bug is fixed by correcting the code and replaying the log into a fresh store; a new screen is a new projector run over history. Debugging by replay: copy the events of one aggregate into a test and reproduce the exact sequence that led to the bad state. Integration: the events the system needs internally are the events other services want (Event-Driven Architecture), already ordered per aggregate.
The relationship to a Kafka-style log (Kafka-Style Logs: Topics, Partitions, Offsets) is close but not identity. A partitioned log gives ordered, replayable, retained events per key — an excellent transport and a plausible event store for some systems — but an event store also needs per-aggregate reads (all events for acc1, cheaply), optimistic concurrency on append (expectedSeq), and indefinite retention; a database table with (aggregate_id, seq) as the primary key is the more common event store, with the log used for publishing.
What it costs
Event versioning: events are forever, and the code that reads them changes. PaymentMade { amount } becomes PaymentMade { amount, currency } and every reducer must handle both — upcasting (transforming old event shapes into the current shape on read) is a permanent layer of the system, and a badly designed early event is a tax paid on every replay for years. Snapshots are mandatory at scale and are a cache with its own invalidation problem when the reducer changes. Queries against the log are impossible — "all accounts with balance under zero" requires a projection, which is why event sourcing drags CQRS in with it. Eventual consistency between the log and every read model is the normal state.
Deletion is the hardest. GDPR erasure requests demand removing personal data, and an immutable log is built to make removal impossible. The workable answers — crypto-shredding (encrypt each person's data with a per-person key and delete the key), storing PII outside the events by reference, or rewriting the log — are all real engineering, and none is as simple as DELETE FROM users. Finally, the mental model is unfamiliar: most developers, ORMs, admin tools and reporting systems assume current-state rows, and every one of them needs a projection to work.
| State persistence | Event sourcing | |
|---|---|---|
| Source of truth | Current rows | Append-only event log |
| History | Lost on update (unless audit tables) | Complete, by construction |
| "What was it on Tuesday?" | Not answerable | Replay up to Tuesday |
| Reads | Direct query, joins, indexes | Only via projections (CQRS required) |
| Write | UPDATE row | Append event with optimistic concurrency on seq |
| Load an aggregate | One row | Snapshot + replay tail |
| Schema change | ALTER TABLE, migrate rows | Upcasters forever; events are immutable |
| Delete personal data | DELETE | Crypto-shredding or log rewrite |
| Rebuild a read model | Re-query the tables | Replay the log — always possible |
| Tooling and familiarity | Universal | Specialised; every tool needs a projection |
Who actually needs it
Event sourcing is the right model when the history is the product: ledgers and accounting (a balance without its postings is meaningless), trading and order books, version-controlled documents, workflow engines, anything where "show me exactly what happened and why" is a core requirement rather than a nice-to-have. In those domains the events already exist in the business language — a posting, a fill, an edit — and storing them is more natural than storing the state. It also pays where audit trails would otherwise be built anyway, and in Saga Pattern orchestrators, whose state is naturally a sequence of step outcomes.
It is the wrong model for a CRUD application with a profile page, a settings form and a list of things. A conventional table with an updated_at and, where required, an audit table written by a trigger gives 90% of the audit value at 5% of the cost. The honest position: most applications need neither CQRS nor event sourcing; adopt event sourcing per aggregate where history is genuinely required, keep the rest as state, and do not rewrite a working system to it because the pattern is elegant.
Key points
- Store events, derive state: the log is the truth and every current-state view is a rebuildable cache of it.
- Replay is a fold; snapshots are cached prefixes so loading an aggregate stays bounded.
- You gain audit, temporal queries, rebuildable projections and replay-driven debugging.
- You pay with upcasting forever, mandatory snapshots and projections (CQRS), eventual consistency, and GDPR deletion via crypto-shredding.
- Use it where history is the product — ledgers, order books, workflows; most applications need neither CQRS nor event sourcing.
Event log → replay → state
How data moves through it
One request or event, hop by hop.
- 1Client → Aggregate service:
Pay(acc1, 20). - 2Aggregate → Snapshot store + Event store: load the latest snapshot and the events after it; replay to the current balance.
- 3Aggregate → Event store: validate, then
append(acc1, PaymentMade{20}, expectedSeq=3)— conflict if another writer got there first. - 4Event store → Event bus → Projectors:
PaymentMadefans out; the balance projection upsertsacc1: 80, the statement projection appends a line. - 5Client → Read model:
GET /accounts/acc1reads the projection (80), possibly a few milliseconds behind the log.
When to use — and when not
- The domain is a ledger: balances, positions, inventory movements, where the postings are the business and the total is derived.
- Regulatory or product requirements demand a complete, tamper-evident history of every change with its cause.
- Read models must be rebuildable from scratch and new projections added over history — analytics, ML features, new screens.
- Workflows and sagas whose state is naturally the sequence of what happened.
- CRUD domains where current state is all anyone asks for; an audit table covers the rest.
- Heavy personal data with strict erasure obligations and no appetite for crypto-shredding.
- A team unfamiliar with the model and without time to build snapshots, upcasters, projections and rebuild tooling before shipping.
- Retrofitting a working state-based system for elegance rather than a measured need.
Tradeoffs
The highest-complexity pattern in this module: everything is derivable and auditable, but every read is a projection, every schema change is an upcaster, and deletion is a research project.
How it fails
- No snapshots: an aggregate with 2 million events takes seconds to load and the write path times out.
- An old event shape with no upcaster: the reducer throws on replay and the aggregate cannot be loaded at all.
- Two commands append concurrently without optimistic concurrency on
seq: both validate against the same balance and overdraw the account. - Projections treated as authoritative: a command validates against a lagging read model and accepts an invalid state change.
- Personal data written into events with no erasure strategy; the first GDPR request has no technical answer.
- Events that record *decisions* instead of *facts* (
BalanceSetTo 80rather thanPaymentMade 20) lose the meaning that made the log valuable.
How it scales
- Append-only writes are cheap and sequential; the store partitions cleanly by aggregate id.
- Snapshots bound load time per aggregate; take one every N events or on a size threshold.
- Projections scale as in CQRS — many projectors, each keyed by aggregate id for ordering, scaled on lag.
- Log size grows forever; archive cold aggregates to object storage and keep snapshots hot.
How it interacts with databases, queues, caches, APIs and external systems
- Database (event store): a table keyed
(aggregate_id, seq)with a unique constraint providing optimistic concurrency; append-only by policy and by permissions. - Storage: snapshots and archived cold streams in object storage.
- Queue/log: publishes appended events to projectors and other services; a Kafka-style log doubles as the transport but rarely as the store.
- Cache/document store: read models — every query the system answers is served from one of these.
- External systems: never replayed into — replaying a log must not re-send emails or re-charge cards; projectors that call out must be guarded by a replay flag and idempotency keys.