Semantic Changes
The schema is identical, every type checks, every test passes, and the number now means something else. revenue went from gross to net. No tool will ever detect this — only documented semantics, an owner and a changelog will.
Who needs this, what one row is, and why the obvious build breaks
Every lesson starts from the consumer, because designing from the source outward is this domain's characteristic mistake.
A field's name, type and nullability are unchanged and its values have shifted. How would anyone find out that its definition changed rather than the business?
Everyone who reads the field and everyone who reads a number derived from it, which after a few layers of modelling is most of the organisation. The critical consumer is the one who does not know the field exists: an executive reading a tile, a finance team closing a quarter, a model retraining on a feature whose definition moved under it (Feature Pipelines).
The unit is one field's definition, which is a thing with no physical representation anywhere in your platform. It exists in the producer's code, in a document if you are fortunate, and in the shared understanding of whoever last discussed it. That is precisely why it can change without leaving a trace.
Rely on the schema. The platform has types, tests, a registry and column-level lineage, and every one of them is green, so the data is under control. This is a reasonable belief and it is the belief this lesson exists to break.
revenue changes from gross to net. Same name, same integer type, same nullability, same row count, same freshness. Every dashboard in the company is now reporting a smaller number and nothing anywhere has failed (Two Dashboards, Two Numbers).
revenuechanges from gross to net. Same name, same integer type, same nullability, same row count, same freshness. Every dashboard in the company is now reporting a smaller number and nothing anywhere has failed (Two Dashboards, Two Numbers).active_userchanges from "logged in during the period" to "performed a meaningful action". The count drops by a fraction that looks exactly like a seasonal dip, and the growth chart shows a decline that is entirely an artefact of a definition (The Dimensions of Data Quality).amountmoves from minor units to major units. The type is still an integer and every value is a hundredth of what it was, which reads as a catastrophic business event rather than as a unit change.created_atchanges from the time a record entered the source system to the time the customer performed the action. The values move by hours, every daily aggregate shifts across midnight boundaries, and no partition changes shape (Event Time).- A producer starts excluding test orders from its export. Correct, well-intentioned, and it silently restates every historical comparison against a period where they were included (Reconciliation).
- A dimension's
countrychanges from billing country to shipping country. Every geographic breakdown moves, the total is unchanged, and the total is the only thing anyone reconciles (Dimension Tables).
What is actually happening
- Every automated check in a data platform operates on structure or values. A schema check compares two schemas. A type check inspects one value. A distribution test compares a set of values with a previous set of values. None of them has access to the *definition* the values are supposed to represent, because that definition is not in the data (Data Tests).
- This is not a gap in tooling that better tooling will close. It is a property of the problem: two datasets that are byte-identical can mean different things, and no function of the bytes can distinguish them. A checker cannot be written, ever (The Pipeline Succeeded. The Data Is Wrong.).
- A distribution test gets closest and still cannot resolve it. It will notice that revenue moved, and it cannot tell you whether the business changed or the definition did. Its output is a question, and answering that question requires a human who knows what the producer shipped this week (Distribution Tests).
- The change is usually made by someone acting correctly. A producer nets refunds because netting refunds is right for their purpose; nobody is being careless. That is why "be more careful" is not a mitigation — the change is not a mistake, it is a change (Data Ownership).
- Detection therefore has to be organisational rather than technical: a documented definition, a named owner, and a changelog the consumers can see. Those three are the entire defence, which is an uncomfortable answer for an engineering domain and is nonetheless the true one (Data Contracts).
The changes that leave no trace
Every other failure in this module has a fingerprint somewhere: a type mismatch, a null rate, a missing column, a rejected batch. This one does not, and the table below is the catalogue.
Read the third column. In every row the schema difference is *none* — not "small", not "subtle", none. The before and after schemas are byte-identical, which means every schema-comparison tool ever built reports no change, correctly. There is nothing for them to find.
Then read the last column, which is the closest any automated check gets. In the best cases a distribution test notices that a number moved, which is a signal that something happened and never a signal about what. In the worst cases — a unit change on a low-variance measure, a redefinition applied only to a subset — even that is inside normal daily variance and nothing moves at all.
| The change | What the producer did, reasonably | Schema difference | What breaks downstream | Closest automated signal |
|---|---|---|---|---|
revenue gross to net | Started subtracting refunds, because for their purpose net is the correct figure. | None | Every revenue figure in the company steps down; year-over-year comparisons become meaningless; the finance close disagrees with the operational system. | A level shift in the measure, indistinguishable from a bad trading week. |
active_user redefined | Tightened the definition from "logged in" to "performed a meaningful action", which is a better metric. | None | The growth chart shows a decline that is entirely definitional; a target set under the old definition is now unreachable. | A level shift in a count, which for a user metric is exactly what seasonality looks like. |
amount minor to major units | Switched the field to a decimal representation to stop rounding arguments. | None if the type stays numeric | Every monetary aggregate is off by two orders of magnitude, which reads as a catastrophic event rather than a unit change. | A very large level shift — the one case in this table an alert will definitely catch. |
created_at source time to event time | Corrected a long-standing inaccuracy by recording when the customer acted rather than when the row was written. | None | Daily aggregates shift across midnight boundaries; a subset of records changes partition; period comparisons are subtly wrong (Event Time). | Almost nothing. Row counts hold and the shift is a few hours. |
| Test orders excluded | Stopped exporting internal test traffic, which should never have been in the data. | None | A permanent step down in volume; historical comparisons now span two different populations. | A volume anomaly, which is usually attributed to the business or to an upstream outage. |
country billing to shipping | Aligned the field with how the business actually thinks about geography. | None | Every regional breakdown moves while the total is unchanged — and the total is the only figure anyone reconciles (Reconciliation). | A category-mix shift, if anyone monitors mix rather than totals. |
Following gross-to-net downstream
To see why this is expensive rather than merely annoying, follow one change through the platform. The producer nets refunds in a single deploy. Nothing rejects it, so it propagates at pipeline speed — which is to say, everywhere, by tomorrow morning.
The lineage below shows where the change is *still detectable in principle* and where it stops being detectable at all. In the raw layer the payload is different from yesterday's and a careful diff would show it. By the time it reaches a mart it is one number among many, and by the time it reaches a tile the only remaining evidence is that the number is smaller than it was.
Note the couldCorrupt column on the last three nodes. None of them is corrupting anything — every one of them is doing exactly its job, correctly, on the input it was given. That is the defining property of this failure class: there is no broken component to find (Lineage Debugging).
- Checkout service
holds The authoritative order record and the code that decides what
revenuemeans.could corrupt Nothing, from this domain's point of view. The change was deliberate, reviewed and correct for the producer's own purpose.
↑ reads from - Emitted event payload
holds The same field name, the same integer type, a smaller value.
could corrupt The last point at which a byte-level diff against yesterday would have revealed anything — and only if someone were diffing payloads, which nobody is.
↑ reads from - Raw landing zone
holds Every payload exactly as received, so both definitions are preserved on disk.
could corrupt Nothing. This layer is the only reason a restatement is possible at all (Keeping Raw History: The Recovery Position and the Liability).
↑ reads from - `stg_orders`
holds One row per order, cast and normalised.
could corrupt Nothing detectable. Types check, nulls are normal, row counts are normal. Every test written against this model passes.
↑ reads from - `fct_orders`
holds One row per order with measures and dimension keys.
could corrupt Nothing it can see. It faithfully carries a measure whose definition changed upstream (Fact Tables).
↑ reads from - `revenue_daily` mart
holds One row per country-day with revenue pre-summed.
could corrupt It aggregates two definitions into one time series, producing a series that is internally inconsistent and looks perfectly continuous.
↑ reads from - Executive dashboard tile
holds One number.
could corrupt Nothing. It renders what it is given, with total confidence, to people who will act on it (Stale Dashboards).
↑ reads from - The quarterly decision
holds A choice made about pricing, headcount or investment.
could corrupt This is the actual damage, and it is the one node no remediation plan can reach.
Seven hops, zero failures, one wrong decision. Every component behaved correctly and the composition did not, which is why the defence has to sit before the first hop rather than anywhere along the chain.
When the meaning change is a grain change
A particularly common and particularly quiet variant is a change to what one row represents. It looks like a semantic change because it is one, and it is worth separating because its symptom is different: totals move by a multiplicative factor rather than by a plausible-looking step (Grain: What Does One Row Represent?).
The trap is that a grain change is often introduced by a join rather than by a producer. A model that was one row per order gains a join to order lines, and every order-level measure is now repeated once per line. The row count moves — which sounds detectable — but a volume check compares against history, and after the change the new volume becomes the new history within a week.
The table below tracks the same measure through a pipeline where the grain changes twice. Read the breaksIf column as a list of the assumptions each stage is making about the stage before it. Every one of those assumptions is invisible in the schema, and every one of them is a semantic contract.
| Stage | One row is | Breaks if |
|---|---|---|
| Producer event | One placed order, with a single total. | The producer starts emitting one event per order line while keeping the same event name. Consumers summing the measure now double-count multi-line orders. |
| `stg_orders` | One order, deduplicated by order id. | Deduplication is done on event id rather than order id, so a producer retry with a fresh event id survives as a second order (Deduplication). |
| `fct_orders` | One order, with revenue as a measure at order grain. | A join to dim_customer whose keys are not unique fans the fact table out, and every order-level measure is multiplied by the duplication factor (Dimension Tables). |
| `fct_order_lines` | One line of one order, with a line-level revenue measure. | Someone joins it to fct_orders and sums the order-level measure, which is now repeated once per line. The total is inflated by the average lines per order and looks like growth. |
| `revenue_daily` mart | One country-day, revenue pre-summed at order grain. | A consumer joins the mart back to an order-level table and re-aggregates, double-counting a measure that was already summed (Data Marts). |
| Dashboard tile | One number, with the grain now completely invisible. | The BI tool applies its own join or filter, changing the grain after every upstream check has passed and every model has been validated. |
Every row in this table is a change in meaning with no change in schema. A grain change is the semantic change that leaves the largest footprint, and it is still not one any type system can see.
The three defences, and none of them is a tool
Since no check can detect this, the design question becomes how to make the change visible as a change. There are exactly three levers and all of them are organisational.
The first is a documented definition attached to the field, precise enough that a change to the producer's logic is a change to a written sentence. The second is a named owner of that definition, so there is someone to ask and someone accountable when the sentence goes stale. The third is a consumer-visible changelog — the only mechanism in this lesson that actively pushes information to the people who need it rather than waiting for them to look.
The fourth lever, and the strongest, is not a defence but an avoidance: do not redefine, add. A new field or a new metric with a different name makes the change visible to every consumer without anyone reading anything, because the old name keeps meaning what it always meant. It costs a column and it removes the entire failure class for that field.
This is an unsatisfying answer for an engineering domain, and it is the true one. A learner who leaves this lesson believing that better tooling would have caught it has learned the opposite of the lesson.
- Definition in the contract, in prose, next to the type: gross or net, which units, which timestamp, which population is included (Data Contracts).
- One accountable owner per measure, who owns the meaning and not merely the job that computes it (Who Owns Data Quality).
- A changelog consumers subscribe to, with definition changes as required entries (Dataset Documentation).
- New name for a new meaning — the avoidance that removes the failure rather than detecting it (The Metrics Layer).
- Level-shift monitoring as a prompt, whose output is a question to the owning team rather than an automated verdict (Distribution Tests).
- Deploy events overlaid on measure charts, so a step change that lands on a producer release is immediately suspicious (Debugging a Data Incident).
`revenue` starts netting refunds. The pull request is small, correct and reviewed. History keeps the old definition, new data has the new one, and the field name — which is what every consumer actually depends on — is unchanged.
Introduce `revenue_net_minor` alongside `revenue_gross_minor`, populate both, document each precisely, announce the addition, and remove the old one only once reads have gone to zero. History under the old definition remains queryable under its own name.
Consumers depend on names, not on documents. A redefinition changes what a name means while leaving the name in place, which is the one change that no consumer can discover without being told. Adding a name makes the change discoverable by everyone at once, at the cost of one column and a deprecation window — and it keeps the historical series internally consistent, which a redefinition destroys permanently.
How to build it
Most important first.
- Write the definition of every measure down, in prose, next to the column, and treat it as part of the contract rather than as documentation. "Gross of refunds and tax, in minor units, at order-placement time" is the sentence that makes a later change visible as a change (Dataset Documentation).
- Name one owner per measure who is accountable for the definition, not merely for the pipeline that computes it. A pipeline owner can tell you it ran; a definition owner can tell you what it means (Who Owns Data Quality).
- Publish a consumer-visible changelog for every dataset, and make a definition change a required entry in it. This is the only mechanism in this lesson that actually catches the failure, and it depends entirely on people using it.
- When a definition must change, add a new field or a new metric rather than redefining the existing one.
revenue_net_minoralongsiderevenue_gross_minorcosts a column and removes the ambiguity permanently; redefiningrevenuecosts nothing today and makes every historical comparison unanswerable. Define each metric once, in one place, so a metric is not implemented in eleven dashboards with eleven definitions that will diverge (The Metrics Layer). - Watch for the signature: a measure that steps to a new level on a specific date while row counts, null rates and category mixes are unchanged. That pattern is a definition change until someone proves it is a business event (Volume Anomalies).
- Record the transformation code version alongside the data it produced, so "what logic produced this number" is a lookup and not an archaeology project (dbt Concepts).
What this actually promises
Naming the guarantee you do not have is worth more than naming the one you do — everything downstream inherits the weakest promise in the chain.
- Nothing in a data platform guarantees the meaning of a field. Schema registries, type systems, contracts-as-files, tests and lineage all guarantee properties of structure and values, and meaning is neither (Schema Registry).
- A documented definition guarantees only that a human *could* have known. It is read by a fraction of consumers and it goes stale, and it is still the strongest instrument available.
- A changelog guarantees notice was given. It does not guarantee notice was received, and the difference between those two shows up in every post-incident review of this failure.
- A distribution test guarantees you will be told the number moved. It cannot guarantee anything about why, and roughly half the time the answer is "the business" (Distribution Tests).
Can I trust it?
A green pipeline is evidence that code ran. These four fields are the evidence that the data is right.
- The closest available check is a level-shift test on the measure: compare the measure's daily level against its own recent history and alert on a step change that is not accompanied by a change in row counts, null rates or category mix (Distribution Tests).
- It misses gradual redefinitions, changes smaller than normal daily variance, and changes that coincide with a genuine business movement — and it produces a false alarm every time the business genuinely moves, which is often.
- Its real output is not a verdict but a question addressed to a specific person: the measure stepped on Tuesday, does the owning team know why. Whether that question gets asked is the difference between a two-day incident and a two-quarter one (Data Incidents).
- A semantic change costs no freshness at all. The pipeline runs at exactly its usual cadence, which is one of the reasons it is invisible: every operational signal is nominal.
- What it costs instead is comparability across time. After an undocumented redefinition, a year-over-year comparison is meaningless and looks entirely normal, which is worse than a gap (Full Refresh vs Incremental).
- The recovery, when it happens, does cost freshness: a restatement of history under one definition or the other means a large backfill and a period during which nobody should be reading the affected tables (Backfills).
- This is the evolution case that the rest of the module cannot reach. Everything else here — compatibility direction, registries, boundary checks — assumes the change is expressible as a schema difference, and this one is not by definition.
- A definition change also applies retroactively in a way a schema change does not. If a producer starts netting refunds today, history is still gross, so the series has a discontinuity that lives in the data forever unless it is restated (Forward Compatibility).
- The stable policy is that a definition is part of a field's identity, exactly as a type is. Changing it means a new name, and a new name makes the change visible to every consumer without anyone having to be told.
- Recovery starts with a decision that is not technical: which definition is correct going forward, and is history to be restated to match it or left as a documented discontinuity.
- If history is restated, it is a full backfill of every downstream model in dependency order, validated against a closed period before publishing, and announced to everyone who was given the earlier numbers (Planning a Backfill).
- If history is left alone, the discontinuity must be recorded somewhere a consumer will encounter it — ideally as a column or an annotation on the series, not as a paragraph in a document nobody opens.
- The thing that cannot be recovered is the decisions people already made on the old numbers. That is the actual damage and it does not appear in any technical remediation plan (Trusting Data).
What can go wrong
- The change ships with no announcement, because the producer did not know the field had external consumers.
- The change is announced in a channel nobody who depends on it reads.
- The change is documented in a place that is updated after the fact, so the document is right and the notice was late.
- The level shift is noticed, attributed to the business, and closed — the most common outcome, because a business explanation is always available.
- The mitigation's own failure: a changelog that exists and is not read, which produces a confident belief in coverage that the platform does not have.
- A restatement performed to fix it, which itself changes published history and is discovered by a consumer as a second incident (What Backfills Break).
- "Schema changes are safe as long as the column still exists." The column existing is not the property that matters. This lesson is the whole counter-argument: the column exists, the type is unchanged, every check passes, and the number means something else (Breaking Schema Changes).
- "A data contract would have prevented this." Only if the contract documented the semantics and the producer treated a meaning change as a contract change. A contract that lists names and types has nothing to say here (Data Contracts).
- "Column-level lineage would have caught it." Lineage tells you which models consume the field, which is essential for the blast radius and useless for detection. It shows you where the change went, not that it happened (Column-Level Lineage).
- "The tests pass, so we are fine." Tests encode what someone thought to assert about values. A definition change alters what the values mean, which is not a thing any assertion is written against (Data Tests).
- "The number went down, so the business went down." A step change on a specific date, with unchanged row counts and null rates, is a definition change far more often than it is a business event.
- A semantic change can alter a field's classification without altering its schema — a free-text column that starts carrying customer-entered addresses becomes personal data on the day the producer changes what goes in it (PII in Pipelines).
- Definition changes to fields used in regulatory or financial reporting are restatements with obligations attached, and the data platform is frequently the last party to learn that a restatement occurred (Data Governance).
Operating it
- Level of every published measure against its own recent history, with row count and null rate on the same chart. The combination is the signature: a measure that moves while its structural neighbours do not (The Data Quality Dashboard).
- A dataset changelog with definition changes as first-class entries, subscribable by consumers (Metadata: Technical, Operational and Business).
- Deployment events from producing teams, overlaid on the measure charts. A step change that lands on a deploy is not a coincidence and this overlay turns a two-day investigation into a two-minute one (Debugging a Data Incident).
- The set of distinct definitions of a metric across the organisation, which the metrics layer can enumerate and a pile of dashboards cannot (The Metrics Layer).
- At one team, this failure barely exists: the person who changed the definition is the person reading the dashboard. It appears the moment producing and consuming are different people (Data Engineering and Backend Engineering).
- At 10x consumers, the number of people who must be told exceeds the number anyone can remember, and the changelog stops being a courtesy and becomes the mechanism.
- At 100x datasets, definitions cannot be held anywhere but in a catalog next to the data, and a definition that lives only in a team's wiki is functionally undocumented (The Data Catalog).
- Detection costs nothing technically and a great deal organisationally: the recurring work of writing definitions down, keeping them true, and reading other teams' changelogs.
- The failure costs a restatement — a full backfill of every affected model, proportional to retained history rather than to new data — plus the human cost of correcting numbers people have already used (What Actually Drives Data Platform Cost).
- Carrying two clearly named metrics instead of redefining one costs a column and a little consumer confusion, permanently. It is the cheapest option in this lesson by a wide margin.
- Writing and maintaining definitions is ongoing, unglamorous work whose benefit is entirely counterfactual. It will always be the first thing dropped under pressure and it is the only defence that works.
- Adding a new field instead of redefining one means carrying both, and carrying both means a consumer will eventually pick the wrong one. That is a smaller failure than an invisible redefinition and it is not nothing.
- Level-shift alerting on measures produces false positives whenever the business genuinely moves. Tuning it to silence those also silences the real signal, which is why its output must be a question to a person rather than an automated verdict (Alert Fatigue: The Page Nobody Reads).
Where this applies
Almost nothing here is universal. These labels say what each claim is specific to, and where a different engine, format, warehouse or scale would differ.
- GENERALThat meaning is not recoverable from bytes is a property of information rather than of any stack, so no format, engine, warehouse or registry changes it. What varies is how much organisational machinery exists around the data to compensate — a catalog with owners and changelogs, or nothing.
- ORG-SPECIFICThe whole defence is organisational: documented definitions, a named owner per measure, and a changelog consumers subscribe to. In a single-team platform these are ceremony because the producer and consumer are the same person; across thirty teams they are the only mechanism that exists.
- TOOL-SPECIFICA metrics layer moves a definition from many dashboards into one place, which makes a change to it a reviewable code change rather than an invisible one. It does not prevent the change and it does not detect an upstream producer changing what a source field means — it narrows the surface, one layer down.
Where the depth lives
This domain teaches how data moves and how you know it arrived intact. It hands the rest off by name.
- — DevOps / Production Engineering owns the deployment record this lesson depends on: knowing which producer released what, when, so a step change in a measure can be correlated with a change in code rather than attributed to the business.