The ML Testing Stack
Data tests, feature tests, training tests, model tests, serving tests, integration tests and drift tests — seven layers because a model can fail at every one of them while every unit test stays green.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
The unit tests pass and the model is wrong in production — which tests were missing, and where in the pipeline does each kind of test belong?
A payments team's CI is green on every merge. Their fraud model has shipped three regressions in six months: a feature computed from a column that changed meaning, a training job that silently trained on two weeks of data instead of six months, and a model that returned NaN for a rare merchant type. Each was caught by a customer or an analyst, not by a test.
Test the code. If every function in the feature pipeline, the training script and the serving handler has a unit test, and the tests pass, the system is tested.
The column-meaning change was upstream in the warehouse; the feature function that consumed it was correct for its input and its unit test passed. The test never looked at the data.
- The column-meaning change was upstream in the warehouse; the feature function that consumed it was correct for its input and its unit test passed. The test never looked at the data.
- The training window bug was a config default: the job ran, produced an artifact with a plausible validation metric on the two-week window, and promoted it. No test asked whether the training set was the size it should have been.
- The
NaNoutput came from a division by a zero-count feature for a merchant type absent from training. Every function did what its test said; the model as a whole had an input it could not handle and nothing probed it. - A drift alert would have caught the column change on day one, but the only monitor was the business metric, which moved three weeks later when the chargebacks arrived (Ground-Truth Delay).
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- The surrounding model predicts chargebacks (Designing a Fraud Detection System). This lesson's target is confidence — an evidence trail that the data, the training, the model and the serving path each still do what they did when the model was promoted.
- Confidence is layered because failures are: a perfect model on broken features is broken, and a perfect serving path around a model that trained on the wrong window is broken.
- The code — feature pipeline, training script, serving handler — has ordinary unit tests: functions in, values out.
- The training data is a warehouse table refreshed nightly; the serving features come from an online store fed by a stream. Neither is under test.
- The model is a file produced by a job that runs weekly. Its behaviour is checked by a validation metric at training time and by nothing afterwards.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- A unit test fixes an input and checks an output of a function. A model's behaviour is a function of its code *and* the data it was trained on *and* the data it is asked about, so a test that fixes only the code leaves two of the three inputs free to change. The ML test stack is the set of tests that pin the other two.
- Data tests check the training and serving data against a contract: schema, nulls, ranges, cardinality, volume, and timestamps that respect prediction time (Data & Feature Tests). Feature tests check that the transformations produce the expected distribution and that the serving and training implementations agree. Training tests check that the pipeline can learn at all on a tiny set (Training Smoke Tests). Model tests check the artifact's behaviour on inputs chosen to probe invariants (Model Invariant Tests) and its robustness under damage (Robustness Testing). Serving tests check the contract between the artifact and the request path (Serving Contract Tests). Integration tests run a request end to end. Drift tests are the tests that run after deployment, continuously, because the third input — the data the model is asked about — changes without a commit (Model Monitoring).
- Each layer catches failures the layers below cannot see and has a different trigger: data tests on every data refresh, training tests on every commit, model tests on every candidate, serving tests on every deploy, drift tests on every hour. A CI that runs only on commit runs only the layers whose input is code.
- Regression protection is its own layer: a challenger must not lose the cases the champion passes, on named slices and on a golden set of hard examples (Model Regression Tests).
Three inputs, and unit tests pin one
A conventional test fixes the input and checks the output. For a pure function that covers the behaviour. A model's output depends on its code, on the data that produced its weights, and on the request it is given, and the second and third change without a commit — on every data refresh and on every retrain, and every hour in production.
The seven layers of the stack are the tests that pin the other two inputs at the boundary where each enters. They differ from unit tests in trigger — a refresh, a candidate artifact, a deploy, a clock — more than in mechanism, and a stack that runs only on commit has silently reduced itself to the one layer unit tests already cover.
- 1Data tests
On every refresh: schema, nulls, ranges, cardinality, volume, and the point-in-time check that no feature timestamp exceeds its prediction time (Data & Feature Tests).
fails by Written once against a snapshot and never re-run on refresh; or so loose that a column meaning change passes.
- 2Feature tests
The transformations produce the expected distribution, and the serving implementation matches the training one on a replay sample (Train / Serve Skew).
fails by Equivalence sampled from the happy path; rare categories never replayed.
- 3Training tests
On every commit: the pipeline runs on a tiny set, the loss falls, an artifact appears, and the model can overfit a handful of examples (Training Smoke Tests).
fails by Run on the full dataset, so it is slow, skipped and then deleted.
- 4Model tests
On every candidate: invariants hold (Model Invariant Tests), damage is tolerated (Robustness Testing), the champion's slices and golden cases are not lost (Model Regression Tests).
fails by Thresholds so tight every retrain fails; the golden set leaks into training.
- 5Serving tests
On every deploy: request and response schema, model version, latency budget, fallback, and replay equivalence with the training path (Serving Contract Tests).
fails by Tests the handler with a fixture request and never the artifact behind it.
- 6Integration tests
One request end to end through features, model and decision, against a known expected action.
fails by Happy path only; the rare segment is never in the fixture.
- 7Drift tests
Continuously: feature, prediction and outcome distributions against the training reference (Model Monitoring).
fails by Alerts on every feature every day; muted; the real shift arrives among them.
Read the trigger column as carefully as the test. A data test that runs on commit tests the fixture, not the data.
Each layer catches what the one below cannot see
The three regressions in the problem each lived in a different layer. The column-meaning change would have failed a data test on the refresh it arrived in. The two-week window would have failed a training-set volume check before the run started. The NaN would have failed a model invariant test on the candidate. None was visible to the unit tests, and all three were visible to a customer.
The layers also block each other. Bad data should stop a training run from starting; a failed invariant should stop a promotion; a failed contract should stop a deploy. The blocking is what turns a stack of tests into a gate, and it is the part most teams add last, after the first incident that a passing test should have prevented.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Warehouse column changed meaning upstream | Feature distribution shifted; model quality fell three weeks later when labels arrived. | No data test on refresh; the feature function's unit test checked code, not data. | Distribution-against-reference test on every refresh, blocking training. |
| Training window defaulted to two weeks | A promoted artifact with a plausible metric on a tiny window; poor production quality on the full population. | No check that the training set had the expected row count and date range. | Training-set volume and span assertions before the run; recorded with the artifact. |
| Rare merchant type absent from training | NaN scores in production for that segment; the downstream decision defaulted to approve. | No invariant test probing outputs on unseen or zero-count inputs. | Invariant tests on every candidate: finite output, probability in range, defined behaviour on unseen categories. |
What must remain true about the tests themselves
The stack has its own assumptions. The fixtures must stay representative: a tiny training set that no longer resembles the data, a replay sample without the rare category, a golden set that drifted into the training data. And the triggers must stay wired: a data test that was detached from the refresh job during a migration is still green and still in the dashboard.
So the stack needs the same treatment a model does — a periodic check that its inputs are what they were, and a record with each artifact of which tests actually ran against it.
Every layer runs on the event that changes its input, against fixtures that remain representative and are kept out of the training data.
holds when Triggers are wired to refresh, candidate, deploy and clock; fixtures are versioned with the model; the golden set is excluded from training by an id check; each artifact records the tests that ran.
breaks when A job migration drops a trigger; a fixture is regenerated from stale data; someone trains on the golden set; a flaky invariant is muted rather than fixed.
respond Rewire or refresh the failing layer before the next promotion; treat a muted test as a missing one.
How to build it
Most important first.
- Draw the pipeline and put a test at each boundary where an input other than code enters: the raw data landing, the feature table, the training set, the artifact, the request. The map is the design; the individual tests follow.
- Trigger each layer on the event that changes its input. Data tests on refresh, not on commit. Model tests on every new artifact, in the promotion path (Promotion Is a Checklist, Not a Score). Drift tests on a schedule against production.
- Make the tests cheap enough to run always: a tiny dataset for the training smoke test, a fixed replay sample for equivalence, a small golden set for invariants. Slow tests get skipped, and skipped tests are absent.
- Treat a test failure at any layer as blocking the layer above: bad data blocks training, a failed invariant blocks promotion, a failed contract blocks deploy.
- Own the tests where the input is owned: the data team owns data contracts, the model team owns model tests, the serving team owns contract tests — with the equivalence test jointly owned, because it is the seam.
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Which layer caught each incident in the last year, and which layer *should* have. The gap is the missing test.
- Time from a data or model change to the first failing test. When it is measured in weeks, the failure was caught by a customer.
- Coverage by layer, not by line: does every pipeline boundary have a test that inspects the data crossing it.
- Line coverage of the training script is not a measure of model test coverage. A fully covered script trains a broken model on broken data.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- Every boundary where data, an artifact or a request enters the pipeline has a test that inspects what crosses it, triggered by the event that changes it rather than only by a commit.
- A failing test at a lower layer blocks the layers above it — bad data cannot become a promoted artifact, a failed invariant cannot become a deploy.
- The test fixtures — the tiny training set, the replay sample, the golden set — are versioned and kept out of the training data, so the tests keep measuring the thing they were written to measure.
- Offline: inject one failure per layer deliberately — a null column, a shrunken window, a flipped monotonic feature, a schema change — and confirm the matching layer fails and the layers above do not run.
- On each promotion: the full stack below serving runs on the candidate, and the record of which tests ran is attached to the artifact (Model Lineage).
- Over time: a quarterly review of incidents against the layer map, adding a test wherever a customer found a failure a layer should have.
What can go wrong
- All seven layers exist and run on commit only, so the data and drift layers test a snapshot from the day the test was written.
- The model tests are so strict that every retrain fails them, they are marked flaky, and the invariant that mattered is ignored with the rest (Flaky Tests on the DevOps side).
- The integration test uses a fixture request that exercises the happy path and never the rare merchant type; the
NaNsurvives it. - The golden set is added to the training data by a well-meaning engineer improving recall, and the regression test can no longer fail.
- Seven layers are seven suites to maintain, each with fixtures that go stale as the data and model evolve; the maintenance is ongoing, not a project.
- Blocking promotion on model tests slows every retrain and will occasionally block a good model on a test that was too strict.
- Owning tests at the boundary between teams means a failure can be nobody's until the ownership is written down.
- "CI is green, so the model is tested." CI tested the code on the day's fixtures. The data changed on a refresh and the model changed on a retrain, and neither triggered CI.
- "We have a validation metric — that is the model test." A validation metric is one aggregate number on one held-out set. It does not check invariants, robustness, slices or the serving contract, and it is computed on features that may not match production.
- "Testing a model means testing the training code." Training code is the smallest of the three inputs. The data and the weights are where the failures were.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThat a model's behaviour depends on data and weights that code tests cannot pin holds for every model family and every deployment mode; the layer map applies whether the model is a linear regression or a transformer.
- SCALE-SPECIFICA team with one model and one engineer can run all seven layers as a single script; the layer map matters as an organisational structure once data, training and serving are owned by different people and the seams between them are where incidents originate.
- CONTESTEDA serious position holds that the layered stack over-invests in offline tests that cannot anticipate production, and that the highest-value tests are the online ones — shadow deployment, canary and drift monitoring — with a minimal offline gate. The strongest case for that view is that most incidents in mature systems are data changes no fixture predicted; the counter is that online detection arrives after the damage and cannot block a bad artifact from shipping.
Where the depth lives
This domain teaches the model and hands the rest off by name.
- — Testing & Reliability Engineering — the test pyramid, fixture discipline, flakiness management and the economics of slow tests are testing questions this module assumes a reader has met; the ML stack adds layers whose inputs are data and weights rather than code.