The question this answers
Two services disagree about a customer’s address. Which one is right?
Single ownership guarantees that the question "what is the current value?" has exactly one correct answer at all times, and that a defined procedure exists for making every other copy agree with it. It does not guarantee that the other copies are currently correct — only that their correctness is well defined and repairable.
Everything below is bought to hold this sentence. "Strongly consistent" with no scope attached is a slogan, not a guarantee — read what it actually covers, and what it explicitly does not.
A component holding a copy knows what it was told and when. It cannot know whether that is still true, because the authority may have changed the value since, and no local check can detect that. This is the load-bearing distinction: the owner knows the value; every other holder knows *a past value*. Systems break when a derived holder treats its copy as current and acts on it — validating, deciding, or worst of all writing back.
A node knows its own state and the messages that arrived. Everything else is inference from evidence that was already stale. "B has not replied in five seconds" is knowledge; "B is down" is a decision — and usually the bug.
Owner and derived — the only two roles
Every piece of state in the system is in one of exactly two roles. Authoritative: this component decides the value; a write here is what makes it true. Derived: this copy exists because the authoritative value was propagated here, and if it disagrees with the owner, this copy is wrong by definition.
Derived state is everywhere and is usually not labelled: a Redis cache, a search index, a read model, a database replica, a materialised view, a denormalised column, a copy held by a downstream partner, a value embedded in an event another service stored, a field cached in a user session, a spreadsheet an analyst refreshes weekly. Every one of them can be stale, and every one of them will be at some point.
The discipline is small and its consequences are large. For each piece of state, name the owner. Label every other copy as derived. Ensure only the owner accepts writes. Once those hold, "which one is right?" is never a question, and a disagreement is a repair task rather than an investigation.
The failure is almost always a second writer
Ownership is rarely lost by design. It is lost incrementally, and there are four common routes, all of which look reasonable at the time.
The convenient direct write. An operations tool, a migration script, a support console, an admin panel writes directly to the owner’s database. It bypasses the owner’s validation, its events and its cache invalidation. Everything downstream is now stale and nothing knows.
The derived store that accepts writes. A read model gets an update endpoint "just for this one case". Now two components accept writes for the same state, and the next event from the owner overwrites the local edit — or does not, depending on ordering, which is worse because it is intermittent.
The copy that ages into authority. Service B caches an address from service A. Over time B adds a correction workflow. Nobody decided B should own addresses; it happened one pull request at a time, and now there are two answers with no defined precedence.
Bidirectional sync. Two systems each propagate changes to the other. This is not two owners — it is *no* owner, and it requires conflict resolution that almost nobody implements, so in practice the last writer wins with clock skew deciding, which is [[last-write-wins]] at its most destructive.
INC-4471 "customer address inconsistent between services"
customer-svc 14 Oak Lane updated 2026-08-19 by customer
shipping-svc 9 Elm Road updated 2026-08-21 by support agent
search index 14 Oak Lane indexed 2026-08-19
carrier API 9 Elm Road pushed 2026-08-21
first question asked: "how do we sync these?"
correct first question: "which one is authoritative?"
-> nobody could answer. shipping-svc had grown a support
correction form 8 months earlier. customer-svc was never
told. Two writers, no precedence, no conflict rule.
the parcel went to 9 Elm Road. The customer had moved.Rules that keep ownership from eroding
One writer per piece of state, enforced rather than documented. Database grants, separate credentials, or an interface that simply does not exist. Convention loses to convenience every time, and the erosion happens through people acting reasonably under deadline.
Derived stores are read-only and rebuildable. If a derived store cannot be dropped and reconstructed from the owner, it is not derived — it holds state that exists nowhere else, and you have an unacknowledged second owner. Rebuildability is the test that makes the label honest, and it is worth checking rather than assuming.
Corrections flow through the owner. A support agent fixing an address calls the owner’s API. The change then propagates like any other, and every derived copy converges. A correction applied to a derived store is a bug, not a workaround, however urgent the situation.
No bidirectional sync without an explicit conflict rule. If both sides must accept writes, that is a conflict-resolution design — version vectors, application merge, or a CRDT — not a synchronisation task. Say so and design it, or make one side authoritative.
Ownership is written down where it is discoverable. A field in the service catalogue, a comment in the schema, a grant in the database. The failure mode of undocumented ownership is that it is only discovered during the incident it caused.
| Owner | Derived store | |
|---|---|---|
| Accepts writesassumption | Yes — this is what makes a value true | No, ever |
| Enforces invariantstypical | Yes — validation lives here | No — it may not even know the rules |
| Can be rebuiltassumption | Only from backups | Yes, from the owner — and this must be tested |
| On disagreementprotocol | Is right by definition | Is wrong, and gets repaired |
| Stalenessprotocol | None — it defines current | Always possible; the window should be measured |
| Correctionstypical | Applied here | Forwarded here |
Ownership is per field, not per entity
The unit of ownership is finer than most teams assume. A "customer" is not one thing owned by one service. The profile fields belong to the customer service; the credit limit belongs to risk; the lifetime value belongs to analytics; the marketing preferences belong to the marketing platform. Insisting that one service own the whole entity produces either a service that is a dumping ground or endless arguments about who owns "the customer".
So the useful artefact is a field-level ownership map: for each field, the owner, the derived copies, how each is updated, and the acceptable staleness. It is unglamorous and it prevents a whole class of incident, because most ownership disputes evaporate once the question is asked per field rather than per noun.
Two consequences worth stating. A service can be authoritative for some fields of an entity and derived for others simultaneously — that is normal, not a smell. And an entity assembled for an API response is usually a *composition* of several owners’ fields, which means the response is a derived view, and its staleness is the maximum of its parts.
1customer.email owner=customer-svc2 derived: search-index (< 60s), crm-export (nightly), session-cache (< 5m)3 4customer.shipping_address owner=customer-svc5 derived: shipping-read-model (< 10s), carrier-api (on dispatch)6 NOTE: support corrections MUST call customer-svc. A direct write to7 shipping-read-model caused INC-4471.8 9customer.credit_limit owner=risk-svc10 derived: checkout-cache (< 30s)11 customer-svc is DERIVED for this field. It must not accept writes to it.12 13customer.lifetime_value owner=analytics (recomputed nightly)14 derived: crm-export15 NOTE: derived from orders. Never written by hand; a manual correction16 here is overwritten at 02:00 and the report silently changes.17 18customer.marketing_prefs owner=marketing-platform (external SaaS)19 derived: customer-svc (< 15m). We are DERIVED for a field on our own20 entity — legitimate, and worth labelling so nobody "fixes" it here.Key points
- Exactly one component is authoritative for each piece of state; everything else is derived.
- Derived state includes caches, indexes, read models, replicas, partner copies and analyst spreadsheets.
- Most "data inconsistency" incidents are really an unanswered question about which component is authoritative.
- Ownership erodes through a second writer: an ops script, a convenient endpoint, a copy that grew a correction flow, or bidirectional sync.
- Derived stores must be read-only and rebuildable from the owner — and the rebuild must be tested.
- Ownership is per field, not per entity; a service can be owner of some fields and derived for others.
The chain, answered
Every field here is required, which is why no lesson in this domain can recommend a design without naming what an operator sees when it fails, what survives the partition, what repairs it afterwards, and the simpler thing to consider first.
- • For each piece of state, name exactly one authoritative component.
- • Enumerate every derived copy, how it is updated, and its acceptable staleness.
- • Enforce single-writer with credentials and grants rather than convention.
- • Route every correction, including manual ones, through the owner.
- • Make each derived store droppable and rebuildable from the owner, and test that path.
- • Record the map where an engineer will find it before writing the convenient shortcut.
- • A second writer appears and no mechanism prevents it.
- • A derived store accepts a write and diverges permanently, or intermittently depending on event ordering.
- • The propagation path breaks and derived copies silently hold old values.
- • The owner is unavailable and a correction is applied to a derived store "temporarily".
- • A rebuild is attempted for the first time during an incident and does not work.
- • Two answers, no precedence: the operator sees two services reporting different values for the same field and cannot determine which is correct, because both accept writes.
- • Correction that vanishes: the operator sees a support fix applied successfully and reverted hours later without any user action, because the next event from the owner overwrote the derived store.
- • Correction that sticks in the wrong place: the operator sees one service permanently disagreeing with the owner, because the fix was applied to the derived copy and the owner never learned of it.
- • Silent propagation break: the operator sees no errors anywhere while a search index serves values that are four days old, because a consumer stopped and its lag alert was never configured.
- • Rebuild that fails when needed: the operator drops a corrupted read model to rebuild it and discovers the rebuild path has never been run, and the store held fields the owner does not have.
- • Direct-write drift: the operator finds records that violate the owner’s validation rules, written by a migration script that connected to the database directly.
- • Single ownership is a coordination-*avoidance* technique: with one writer there is nothing to agree about, which is why it scales where consensus does not.
- • The residual coordination is propagation, and it is asynchronous — derived stores converge without agreement, at the cost of a staleness window.
- • Multiple writers force real coordination: distributed locks, consensus, or conflict resolution. Choosing single ownership is choosing not to pay that.
- • Agreeing the ownership map across teams is a one-time coordination cost, paid in a meeting, that removes a recurring cost paid in incidents.
- • When the owner is unavailable, derived copies keep serving their last known values — which is usually the right behaviour, and must be labelled as possibly stale.
- • Writes are unavailable while the owner is unavailable; that is the cost of single ownership and it is a deliberate trade for consistency.
- • Propagation halts during a partition and resumes afterwards; derived stores converge once connectivity returns, provided the feed is durable.
- • Any correction applied to a derived store during the outage will be silently overwritten on convergence, or will silently persist — neither is acceptable, which is why the answer is to queue it for the owner.
- • Detect: compare each derived store against the owner on a schedule; a non-zero delta beyond the staleness window is the signal, since none of this produces errors.
- • Contain: revoke the write access that allowed a second writer, before repairing anything, or the drift resumes immediately.
- • Recover: repair the derived copies from the owner, or drop and rebuild where rebuilding is cheaper and safer.
- • Reconcile: for corrections that were applied to a derived store, replay them through the owner so they survive.
- • Verify: re-run the comparison and confirm zero delta, and confirm the grants now permit exactly one writer.
- • Writers per piece of state, from grants and from write telemetry — the direct measure of whether ownership holds.
- • Delta between owner and each derived copy, computed continuously rather than after incidents.
- • Propagation lag per derived store, compared to its declared acceptable staleness.
- • Rebuild success and duration for each derived store, exercised on a schedule rather than during an incident.
- • Count of corrections applied outside the owner — should be zero, and is a leading indicator of the next inconsistency incident.
- • Any system where the same information exists in more than one place, which is every system with a cache.
- • Incident response: with a named owner, "which is right?" is answered instantly and the incident becomes a repair.
- • Cross-team disputes about data, where per-field ownership dissolves most disagreements.
- • Genuinely collaborative state — a shared document, a multi-writer counter — where insisting on a single owner forces all writes through one point and is the wrong model. Use
[[crdts]]-style convergence instead. - • Where the owner’s availability becomes the availability of writes for a critical path, and that has not been consciously accepted.
- • Over-formalised maps for tiny systems, where the ownership is obvious and the document rots.
- • Conflict-free replicated data types, where multiple writers are genuinely required and convergence can be made automatic — for the data shapes that support it.
- • Explicit multi-master with version vectors and an application merge function: real multi-writer semantics with a designed conflict rule instead of an accidental one.
- • A single shared database with one table and no copies at all: no ownership question because there is no derivation. Fine until you need a cache.
- • Per-field ownership split across services, which is usually the right refinement when one service "owning the customer" is the point of contention.
One owner per field. Click a cell until the map is true.
| Field | crm | orders | warehouse | pricing | search-index | cache | Verdict |
|---|---|---|---|---|---|---|---|
| customer.address | 1 extra writer | ||||||
| customer.email | single owner | ||||||
| order.status | single owner | ||||||
| inventory.on_hand | single owner | ||||||
| price.current | single owner | ||||||
| order.total | no owner |
What people believe, and what is true
The services are inconsistent — we need to sync them.
Ask which is authoritative first. Without that answer, "sync" has no defined direction, and bidirectional sync is how you get permanent silent conflicts.
Each service owns its own data, so we have single ownership.
Ownership is per field. A service holding a copy of another service’s field is derived for that field, and treating it as owned is exactly how a second writer appears.
It was urgent, so we fixed it directly in the read model.
That created a second writer. The fix either gets overwritten silently or persists silently, and both outcomes are worse than the original problem.
Bidirectional sync gives us two owners.
It gives you none. Without an explicit conflict rule, the winner is decided by timing and clock skew, and the loser is discarded with no record.
We can rebuild the read model whenever we need to.
Only if that has been tested. Read models routinely accumulate fields the owner does not have, at which point they are not derived and cannot be rebuilt.
Go deeper
Only the levels this lesson can honestly fill — a missing level is a claim nobody had.
Overview
Exactly one component owns each piece of state. Caches, indexes, read models and replicas are derived — if they disagree with the owner, they are wrong.
Practical
Write a field-level ownership map: owner, derived copies, update path, acceptable staleness. Enforce single-writer with grants rather than convention. Route every correction through the owner. Make derived stores rebuildable and test the rebuild. Compare owner against derived continuously.
Advanced
Single ownership is coordination avoidance: with one writer, no agreement protocol is needed, and updates propagate as one-way facts rather than as negotiations. That is why it scales where consensus does not, and it is precisely why introducing a second writer is not a small compromise — it moves you from a system needing no coordination to one needing conflict resolution, and the transition is usually made accidentally by someone under deadline.
Apply it
- 🔧 Pick one entity in your system and write the field-level ownership map. Note every field where you cannot name a single owner.
- 🔧 For each derived store you have, try dropping and rebuilding it in a non-production environment. Record which ones fail and why.
- ⚡ An analytics service recomputes lifetime value nightly. Support corrects a value by hand in the reporting database. What happens, when, and what should have happened?
- 💬 Two services disagree about a customer’s address. What is the first question you ask?
- 💬 Why is a derived store that accepts writes worse than one that is simply stale?
- 💬 Ownership per entity or per field? Defend your answer with an example.
- 💬 A support agent needs to correct data urgently and the owning service is down. What do you do?