Impact Analysis
The same graph read the other way. If I change this column, what breaks — answered before the change rather than discovered afterwards.
Who needs this, what one row is, and why the obvious build breaks
Every lesson starts from the consumer, because designing from the source outward is this domain's characteristic mistake.
I am about to rename, retype or drop this column. Who depends on it, and which of them will fail loudly rather than silently?
The producer holding a migration they believe is safe, and the reviewer who has to approve it. Neither can enumerate downstream consumers from memory once the platform is past a few dozen models, and both are about to find out by shipping (Breaking Schema Changes).
The unit is one proposed change to one object — a column dropped, retyped, renamed, or a model's grain altered. The output is a set of affected consumers with a severity per consumer, because "affected" without "how" is a list nobody can triage (Column-Level Lineage).
Ask in the data channel: "does anyone use orders.status?" Wait a day. Nobody objects. Ship it. This works surprisingly often, which is why it survives — and it fails in precisely the cases where the consumer is not in that channel.
The people who depend on the column are not in that channel: a finance analyst with a saved query, a machine learning pipeline in another team's repository, a reverse-ETL sync into the CRM (Data Platform Anti-Patterns).
- The people who depend on the column are not in that channel: a finance analyst with a saved query, a machine learning pipeline in another team's repository, a reverse-ETL sync into the CRM (Data Platform Anti-Patterns).
- Nobody objects because nobody knows they depend on it. The dependency runs through three intermediate models, and the consumer knows only the mart at the end (Data Lineage).
- A consumer sees the message, checks their dashboard, sees the column is not on it, and approves. The column is in a filter on a hidden page, and the dashboard now silently includes cancelled orders (Two Dashboards, Two Numbers).
- The change is a *type* change rather than a removal, so nothing errors anywhere. Every downstream cast succeeds and a subset of values becomes null (Nullability & Defaults).
- The change is semantic — the column keeps its name and type and starts meaning net rather than gross. No schema check anywhere in the platform can see it, so no impact analysis based on schemas can either (Semantic Changes).
What is actually happening
- Impact analysis is lineage traversed downstream. The graph is the same one an incident walk uses; the edges are simply followed in the other direction, and that single reversal changes it from a diagnostic tool into a preventive one (Data Lineage).
- The asymmetry between the two directions is real and matters. Upstream depth is bounded and short — five or six hops from a dashboard to a source. Downstream fan-out is unbounded and grows with every new consumer, so impact analysis gets harder every quarter while incident walks do not (Directed Graph).
- The output is only useful if it is classified by failure mode, not merely enumerated. A consumer that will raise an error is a scheduling problem; a consumer that will silently produce a different number is an incident waiting for a quarter to close. These need different notifications and different lead times (Breaking Schema Changes).
- The set of changes with no downstream signature is the important one to name: a value-meaning change, a grain change under an unchanged schema, and a filter change all leave the graph identical. Impact analysis reduces the surface of surprises; it does not eliminate it (Semantic Changes).
- Precision determines whether the output gets read. Table-level impact says "forty models are affected", which is true and unactionable; column-level impact says "two models and one dashboard tile", which someone will actually follow up (Column-Level Lineage).
- The mature form runs the analysis in the pull request, alongside the tests, and comments the affected consumer list onto the change. Impact analysis performed after a deploy is an incident report (Contract Enforcement).
The same graph, read the other way
Everything in the lineage lesson applies here with the arrows reversed, and the reversal is not symmetric in practice. Walking upstream from a dashboard, each hop has a handful of parents and the walk ends at a source within a few steps. Walking downstream from a source column, each hop has as many children as the platform has consumers, and there is no natural end.
The diagram below shows the shape. One staging column feeds one fact model, which feeds three marts, which feed dashboards, exports and a machine learning feature pipeline — plus one consumer that reads the fact table directly because someone had a deadline once (Data Platform Anti-Patterns).
Two nodes on it are drawn as unknowns. That is deliberate: every real impact graph has them, and drawing them explicitly is more honest than a clean picture that implies the enumeration is complete.
Blast radius before the change
Here is a concrete change and its analysis. A producer renames status to order_status for consistency with a naming standard. It is a good change, it passes every test in the producing repository, and the schema check downstream reports a column added and a column removed — both handled (One Vocabulary: Naming and Consistency).
The impact table below is what makes it reviewable. Read the silent column: two of the five consumers will raise an error and get fixed within an hour, and three will keep running and produce different numbers. The loud ones are not the problem. The problem is the export into the CRM, which will populate a field with nulls that an account manager will read as churn.
The correct response is almost never "do not rename". It is expand and contract: add order_status as a duplicate, migrate the three silent consumers, then drop status once the impact set for the drop is empty. One breaking change becomes two non-breaking ones and a waiting period (Expand and Contract Migrations).
- order_id BIGINT
- customer_id BIGINT
- amount_minor BIGINT
- currency CHAR(3)
- status VARCHAR
- order_date DATE
- order_id BIGINT
- customer_id BIGINT
- amount_minor BIGINT
- currency CHAR(3)
- order_status VARCHAR
- order_date DATE
change status renamed to order_status. Type, nullability and value domain unchanged. In the producing repository this is a one-line diff that passes every test.
| Consumer | Effect | How it shows up |
|---|---|---|
| `fct_orders` (declared model, references the column by name) | Compilation fails at build time with an unknown-column error. Fixed in one commit. | Loudly — it raises |
| `ops_orders_hourly` (declared model, `select * except(status)`) | The exclusion no longer matches, so order_status is now included in the output and the downstream table gains a column. | Silently — no error, wrong result |
| Executive dashboard (filters on `status = paid` in the BI layer) | The filter references a column that no longer exists; depending on the tool it either errors or silently drops the filter and starts including cancelled orders. | Silently — no error, wrong result |
| Reverse-ETL sync into the CRM (maps `status` to an account field) | The mapping resolves to nothing and writes nulls into the CRM, which account managers read as a state change on the account. | Silently — no error, wrong result |
| Finance close workbook (quarterly, saved query) | Errors on next run — which is up to a quarter away, long after everyone has forgotten this change. | Loudly — it raises |
What does the impact set look like, and how do those consumers fail?
when The impact set is empty and lineage coverage for this object has been reconciled against the query log.
cost The residual risk of an uninstrumented consumer. Acceptable for internal staging models; not for anything a consumer team reads (Data Lineage).
when Any consumer would fail silently, or any consumer is outside your team.
cost Two changes, a migration period, and a duplicated column in the meantime. Almost always the right answer for a rename (Expand and Contract Migrations).
when All affected consumers fail loudly and are known and reachable.
cost A coordination burden on consumers, and a real chance that the quarterly consumer misses the window entirely (Deprecation as a Process, Not a Label).
when The change is semantic rather than structural — the column keeps its shape and changes meaning.
cost A second published surface to maintain. It is the only option that works here, because no schema-level analysis can detect the change at all (Semantic Changes).
when The benefit is consistency and the impact set includes silent consumers you cannot enumerate confidently.
cost A naming inconsistency that will be paid for repeatedly in confusion. Sometimes the right call, and it should be a decision rather than a default (Data Contracts).
What impact analysis cannot see
A tool that produces a confident list trains its users to treat the list as complete. That is the risk this section exists to counteract, because the consumers a lineage graph cannot see are systematically the ones that hurt most: they are outside the platform, run rarely, and belong to people who are not in the engineering channel.
The checks below are the ones that partially close the gap, each with its blind spot stated. The pattern is the same as everywhere else in this domain — a portfolio of imperfect checks beats one check presented as sufficient (Data Quality).
The honest summary to give a reviewer alongside any impact report is two sentences: here is what the graph knows, and here is the class of consumer it structurally cannot know about. A report without the second sentence is more dangerous than no report.
| Check | Expresses | Catches | Still misses |
|---|---|---|---|
| Reconcile declared lineage against the engine's query log for this object | Everything that read this table recently is in the impact set. | Undeclared models, ad-hoc dashboards, scripts, and anything built outside the transformation framework (Follow the Query). | Consumers whose last run is outside the log retention window — which biases against quarterly and annual consumers, the ones with the most senior readers. |
| Query log window extended to cover a full reporting cycle | Periodic consumers are represented. | Quarterly finance queries, annual audits, seasonal reports. | First-time consumers who have not run yet, and consumers that read through a shared service account so all of them appear as one. |
| BI tool metadata integration enumerating fields used per dashboard | The last hop, where the number people complain about is computed. | Filters, calculated fields and blends that reference the column and never appear in warehouse SQL (The Metrics Layer). | Dashboards in a second BI tool nobody integrated, and anything a user downloaded to a spreadsheet. |
| Retrospective comparison of predicted versus actual breakage after each change | The analysis is accurate, measured rather than assumed. | Systematic coverage gaps, parser regressions, whole classes of consumer nobody modelled. | Silent breakage that was never detected at all — which means the metric is biased optimistic by exactly the failures this domain cares about most (The Pipeline Succeeded. The Data Is Wrong.). |
| Egress inventory: scheduled exports, reverse-ETL syncs, external shares | Data leaving the platform is enumerated even where lineage does not follow it. | CRM syncs, partner feeds, mailed extracts — the consumers with the most direct business consequence. | Anything a person set up manually with their own credentials, which is most of what exists in a young platform (Data Access Control). |
None of these five makes the impact set complete. Together they make its incompleteness *bounded and stateable*, which is the achievable goal — and stating the bound is what keeps a reviewer appropriately nervous.
How to build it
Most important first.
- Run it automatically on every change to a model or contract, and attach the result to the change under review. A tool someone must remember to open is a tool that gets opened after the outage (Data Contracts).
- Classify each affected consumer by how they will fail: hard error, silent value change, silent row-count change, or no effect. The classification is what makes the list triageable (Schema Evolution).
- Extend the graph past the warehouse. The consumers that matter most are usually the least instrumented — BI dashboards, scheduled exports, reverse-ETL syncs into operational systems, notebooks (Data Platform Anti-Patterns).
- Rank by usage, not by existence. A downstream model queried by nobody in six months is not the same finding as one behind an executive dashboard, and treating them identically is what trains people to ignore the report (Data Discovery).
- Prefer changes with no blast radius at all. Expand-and-contract — add the new column, migrate consumers, remove the old one later — converts one breaking change into two safe ones and is almost always available (Expand and Contract Migrations).
- Record the analysis with the change. During the incident three weeks later, "who did we say would be affected" is a real question and the answer should not be in a chat thread (Data Incidents).
What this actually promises
Naming the guarantee you do not have is worth more than naming the one you do — everything downstream inherits the weakest promise in the chain.
- Impact analysis guarantees coverage of the instrumented graph and nothing beyond it. Its central risk is a false negative — a consumer that exists, will break, and produced no edge — and a false negative here is indistinguishable from a clean report.
- It guarantees nothing about severity unless severity is modelled explicitly. A list of names is not a risk assessment, and the temptation to read a short list as "low risk" is strong and unjustified.
- For semantic changes it guarantees nothing at all, because no schema-derived graph can represent a change in meaning (Semantic Changes).
- Where the graph comes from query history, the guarantee is bounded by retention: a consumer that runs quarterly and last ran outside the window is invisible, and quarterly consumers are disproportionately finance (Data Lineage).
Can I trust it?
A green pipeline is evidence that code ran. These four fields are the evidence that the data is right.
- The check that actually validates this is retrospective: after any change, compare the consumers that broke with the consumers the analysis predicted. Every miss is a coverage gap with a name and a repro, and this feedback loop is the only thing that makes the tool trustworthy over time.
- Reconcile the graph against the query log for the affected object specifically. Any reader of that table in the log that is not in the impact set is a false negative, discovered before the change rather than after (Follow the Query).
- Both miss consumers who read the data outside the platform: an extract mailed monthly, a notebook on someone's laptop, a spreadsheet that pulls through a connector. Those exist in every organisation and no graph contains them.
- The graph must be current with the branch being reviewed, not with production. A change that adds a dependency is analysed against a graph that does not contain it yet, which is fine for removals and misleading for restructurings.
- Usage signals need a window long enough to include periodic consumers. A thirty-day window makes every quarterly report look abandoned, and quarterly reports are the ones with the most consequential readers (Data Discovery).
- Analysis has to complete fast enough to sit in a pull request. An impact report that takes an hour is one people ship past, and the tool has then made things worse by providing an excuse.
- This is the lesson evolution is about. Additive changes have a small blast radius unless something downstream selects
*; renames and drops have a large one; type changes have a large and mostly silent one (Schema Evolution). - The safest sequence is always expand, migrate, contract, with the contract step gated on the impact set being empty — which turns impact analysis from advice into a release gate (Expand and Contract Migrations).
- Meaning changes need a different mechanism entirely: a version on the semantic definition, announced to consumers, because the graph cannot detect them and the tests will not fail (Data Contracts).
- If a change ships and the impact set was wrong, the recovery is a revert of the producing change plus a recompute of everything downstream that ran in between — and the recompute order comes from the same graph, topologically sorted (Topological Execution).
- Silent-failure consumers are the expensive part of recovery, because their outputs have to be identified and recomputed for the whole period the change was live, which may be longer than anyone noticed (Backfills).
- Keep the previous column alongside the new one until the impact set has confirmed migration. Recovery from a rename is trivial while the old column still exists and is a restore operation once it does not (Expand and Contract Migrations).
What can go wrong
- A clean report read as "no consumers" when it means "no instrumented consumers". This is the failure that produces the incident the tool was bought to prevent, and it is made worse by the confidence the tool provides.
- Table-level output that flags every downstream model, teaching every recipient to ignore the notification within a month (Alert Fatigue: The Page Nobody Reads).
- Consumers listed without severity, so the reviewer treats a hard error and a silent restatement as equivalent and prioritises the loud one.
- The analysis runs after merge rather than before, at which point it is a list of people to apologise to.
- Usage ranking based on a short window, which classifies quarterly finance reports as dead and drops them from the report entirely.
- "The impact report was empty, so the change is safe." The report covers instrumented consumers. Empty means nothing was found, and the population it cannot see — notebooks, exports, quarterly queries — is exactly where the expensive surprises live.
- "Impact analysis is a lineage feature." It is lineage plus a severity model plus a delivery mechanism. A graph with no notion of how a consumer fails produces a list, not an analysis (Data Lineage).
- "If it does not error, it is not breaking." Silent value changes are the more expensive class by a wide margin, because they are found by a person reading a report weeks later rather than by a monitor (The Pipeline Succeeded. The Data Is Wrong.).
- "We can just announce the change." Announcements reach the people who are listening. The dependency you did not know about belongs, by definition, to someone who is not (Data Ownership).
- Impact analysis run in the other direction is how deletion requests are scoped: given a source column, enumerate every downstream dataset that carries values derived from it, and you have the work list (Deletion Requests).
- It is also the mechanism for retention enforcement. A retention rule on a source is meaningless unless the derived copies are enumerable, and enumeration is a downstream traversal (Data Retention).
- The analysis output itself names systems, teams and dashboards, which is a map of the organisation's data flows and is worth access-controlling accordingly (Data Access Control).
Operating it
- Prediction accuracy over time: predicted affected consumers versus consumers that actually broke, per change. It is the only metric that says whether the tool works (Data Incidents).
- Downstream fan-out per dataset, trended. Datasets whose fan-out is growing fastest are the ones where a contract and a deprecation policy will be needed first (Data Contracts).
- Share of breaking changes that went through an expand-and-contract sequence rather than a direct alteration (Expand and Contract Migrations).
- Count of consumers reachable only through the query log and not through declared lineage — the size of the shadow consumer population (Data Lineage).
- At ten times the consumer count, fan-out makes the raw list unreadable and ranking by usage becomes mandatory rather than a nicety.
- At a hundred times, impact analysis has to be a gate rather than a report — the volume of changes exceeds anyone's capacity to review lists, and only automated blocking on the high-severity classes keeps up (Contract Enforcement).
- Upstream depth barely changes at any scale, which is why incident walks stay tractable while impact analysis gets steadily harder. Plan for the asymmetry rather than being surprised by it.
- Traversal cost grows with downstream fan-out, which grows with consumer count rather than with data volume. This is the rare data-platform cost that is driven entirely by organisational growth (What Actually Drives Data Platform Cost).
- Precomputing descendants per object makes pull-request-time analysis fast and means the precomputation must be rebuilt on every graph change. Storage is modest; the rebuild cadence is the real constraint.
- The cost this avoids is the expensive one: an incident whose blast radius was discovered by consumers, in a period that has already been reported on (Data Incidents).
- Gating changes on impact analysis prevents silent breakage and slows every change, including the safe ones. Tuning which severities block and which merely notify is the whole design problem, and getting it wrong in the strict direction makes people route around the gate.
- Extending the graph to the BI layer and to exports is where most of the real coverage is and costs an integration per tool, each of which then needs maintaining.
- Precision costs column-level lineage, with its parser burden and permanent coverage gap. Table-level impact analysis is much cheaper and produces reports people stop reading (Column-Level Lineage).
Where this applies
Almost nothing here is universal. These labels say what each claim is specific to, and where a different engine, format, warehouse or scale would differ.
- GENERALReading the dependency graph downstream to decide blast radius is method rather than tooling, and works with a whiteboard below a few dozen models. What changes with scale is only whether a human can hold the fan-out in their head, and that limit arrives earlier than people expect.
- TOOL-SPECIFICCoverage is decided by what emits edges: a transformation framework sees its own models and nothing else, a BI integration sees dashboards, and query-log parsing sees everything that ran inside the window. Platforms with only the first will report empty impact sets for their most important consumers.
- ORG-SPECIFICWhether impact analysis should block a change or merely inform it depends on how the organisation handles data incidents. Where a wrong number is a compliance event the gate must block; where consumers are two teams who talk daily, a notification is proportionate and a gate is friction.
Where the depth lives
This domain teaches how data moves and how you know it arrived intact. It hands the rest off by name.
- — DevOps / Production Engineering owns the release mechanics this depends on: running the analysis in CI, gating a merge on its result, and rolling back the producing change when the prediction was wrong.
- — Distributed Systems owns why a consumer you cannot enumerate is the normal case rather than an oversight — there is no global registry of who is reading what, only the records each component happened to keep.