Model Regression Tests
A challenger with a better aggregate metric can still lose the cases the champion passes. A regression test holds the challenger to the champion's slices and to a golden set of known hard examples — and the golden set is a leakage risk the moment anyone trains on it.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
The challenger beats the champion on the validation metric — how do you check it has not quietly become worse on the slices and the specific cases the product depends on?
A support-routing team's new ticket classifier improved the validation metric and was promoted. Within a week the escalation desk complained: legal-threat tickets, which the old model routed correctly nearly every time, were now landing in general support. There are a few dozen such tickets a month out of tens of thousands; the aggregate did not move.
Compare the challenger to the champion on the validation set. If the metric is better, promote. Averages over a large set are reliable; a few dozen tickets are noise.
The aggregate improved because the challenger got better on the dominant queues. It got worse on legal threats, which are a fraction of a percent of the set and cannot move the average. The promotion gate measured the population and the product depends on a slice (Evaluation Slices).
- The aggregate improved because the challenger got better on the dominant queues. It got worse on legal threats, which are a fraction of a percent of the set and cannot move the average. The promotion gate measured the population and the product depends on a slice (Evaluation Slices).
- The champion's behaviour on the named hard cases — the ones the desk had painfully taught the last model through relabelling — was never encoded anywhere. It lived in the champion's weights and in the desk's memory, and the challenger had neither.
- Six months later, with a golden set in place, a well-meaning engineer added it to the training data to "make sure the model learns those cases". The next challenger passed the golden set perfectly, and the golden set stopped being able to fail (Evaluation Leakage).
- A slice threshold set tightly on a slice of thirty tickets fails every challenger, on noise; the gate is loosened to let a model through, and the loosening stays.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- The model classifies support tickets into routing queues (Classification). This lesson's target is a promotion gate: the challenger must not regress on the slices and the named cases the champion handles, whatever its aggregate says (Promotion Is a Checklist, Not a Score).
- A regression is defined per slice and per golden case, not in aggregate: "worse on legal threats" is a regression even when "better overall" is true.
- Tens of thousands of tickets a month with the queue they were finally resolved in as the label. Legal threats are a fraction of a percent; high-value-customer tickets a few percent; non-English tickets a few percent.
- A golden set: a few hundred tickets chosen by the escalation desk as the cases that must route correctly — past incidents, ambiguous phrasings, the legal-threat patterns — each with its expected queue and a note on why.
- The slice definitions: legal threat, high-value customer, language, product line, ticket length — with the sample size per slice recorded, since a slice of thirty tickets cannot support a tight threshold.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- A regression test compares the challenger against the champion on the same fixed evaluation data, with two additions to the aggregate. Slices: for each named slice, the challenger's metric must be no worse than the champion's by more than a tolerance that reflects the slice's sample size — a slice of thirty supports only a wide tolerance, and the honest response to a tiny critical slice is to grow it, not to tighten the threshold (Metric Uncertainty).
- The golden set: a curated set of specific examples with expected behaviour, each of which the challenger must get right (or must get at least as right as the champion). It encodes the cases the product cannot afford to lose: past incidents, the legal-threat patterns, the ambiguous phrasings the desk has adjudicated. It is a specification in examples, and it is versioned, owned and reviewed like one.
- Both are computed on the same fixed data for both models, so the comparison is paired: the same ticket, two predictions. Paired comparison removes the variance of the data draw and leaves the variance of the models, which is what a promotion decision is about (Champion / Challenger is the rollout side).
- The golden set has a structural weakness. It is the most valuable labelled data the team has, and the temptation to train on it is constant. The moment it enters training, the model has memorised it, the test cannot fail, and the gate is a rubber stamp — the same mechanism as tuning on the test set, applied to a small set with a product name (Never Tune on the Test Set).
Better on average, worse where it matters
Legal-threat tickets are a fraction of a percent of the volume. A model that routes every one of them wrong and every other ticket slightly better has a higher aggregate metric than one that gets them all right. The promotion gate that compares aggregates will promote it, and the average is not lying — it is answering a different question from the one the escalation desk asks.
The paired comparison on slices answers the desk's question: on the same tickets, in each named segment, is the challenger at least as good as the champion? The illustrative table below is the shape of the promotion in the problem — a clear aggregate win and a slice regression the aggregate could not have shown.
| Slice | Tickets (illustrative) | Champion correct | Challenger correct | Verdict |
|---|---|---|---|---|
| All tickets | 40,000 | 35,200 | 36,000 | aggregate gain |
| General support | 31,000 | 26,800 | 27,900 | gain |
| Billing | 6,000 | 5,500 | 5,550 | flat, within tolerance |
| High-value customer | 2,500 | 2,300 | 2,310 | flat, within tolerance |
| Legal threat | 45 | 43 | 31 | regression — blocks |
| Non-English | 400 | 330 | 300 | regression — blocks |
The golden set: a specification in examples
Slices catch regressions on segments that can be defined by a field. The cases the desk cares about are often not a segment — a phrasing that looks like a complaint and is a legal threat, a ticket in one language about a product named in another — and the only way to encode them is as examples with expected behaviour. That is the golden set: each example is a case the product has already paid for once, with its expected output and a note saying why.
It is tested by pairing: both models on every case, with the list of flips. A challenger that fails a golden case the champion passes has regressed on something specific and named, and the desk can read the case and say whether it matters. That readability is why the golden set is worth curating even when the slice metrics are in place.
1def regression_gate(champion, challenger, eval_set, slices, golden, tolerance_for):2 report = {"blocks": [], "flips": []}3 4 for name, rows in slices(eval_set).items():5 a = accuracy(champion, rows); b = accuracy(challenger, rows)6 tol = tolerance_for(len(rows)) # wider for small slices7 if b < a - tol:8 report["blocks"].append((name, len(rows), a, b))9 10 for case in golden: # never in any training set11 pa = champion.predict(case.x) == case.expected12 pb = challenger.predict(case.x) == case.expected13 if pa and not pb:14 report["flips"].append((case.id, case.why))15 16 report["blocked"] = bool(report["blocks"] or report["flips"])17 return report18 19# Enforced in the training pipeline, on every run:20assert not set(train_ids) & set(golden_ids), "golden set leaked into training"21assert not set(train_hashes) & set(golden_hashes), "golden content leaked under a new id"The two assertions at the bottom are the lesson's second half. They belong in the training pipeline, not in the gate: by the time the gate runs, a golden set that was trained on has already stopped being able to fail, and the gate would report a perfect pass.
The golden set is a leakage risk
The golden set is the best labelled data the team has: hard cases, adjudicated by experts, with reasons. Every incentive says train on it. The moment it enters a training set, the model memorises it, the pass rate goes to one hundred percent, and the regression test has become a test of whether the model can recall its training data. Nothing looks wrong; the gate is simply incapable of blocking.
The exclusion has to be enforced where training data is assembled, by id and by content hash, and asserted on every run — because a data migration, a relabelling pass or a helpful engineer will each eventually put a golden example into training under a different name.
looks like A training set that includes a few hundred expertly labelled hard cases, added to improve recall on exactly the patterns the desk complained about.
why it leaks The regression test asks whether the model handles those cases; the model has seen them with their labels. The test now measures memorisation, and the answer is always yes.
fix Exclude by id and content hash in the training pipeline, asserted on every run; refresh the golden set from new incidents so it keeps testing patterns the model has not seen.
No golden example is in any training set, the slices and golden cases still describe what the product cannot afford to lose, and the tolerances match the current slice sizes.
holds when Id and hash exclusion are asserted in the training pipeline; the golden set is refreshed from escalated incidents and reviewed; tolerances are re-derived when slice sizes change; the gate's block rate is nonzero over a year.
breaks when A migration reissues ids; a relabelling pass copies golden cases into training; the refresh process lapses; the exception process becomes the default.
respond Rebuild the golden set from cases the current models have never trained on, re-enforce the exclusion, and treat the promotions made while the gate was blind as unreviewed.
How to build it
Most important first.
- Define the slices with the product — which segments must not regress — and record each slice's sample size; set per-slice tolerances from the sample size, and invest in growing the tiny critical slices rather than tightening their thresholds.
- Build the golden set with the people who see the failures — the escalation desk, the analysts — as examples with expected behaviour and a reason. Version it; review additions; give it an owner.
- Exclude the golden set from every training set by id, enforced in the training pipeline, and assert the exclusion on every run; a golden example that appears in training is a pipeline failure (Dataset Versioning).
- Run the paired comparison — aggregate, per slice, per golden case — as a promotion gate with the results attached to the candidate; a slice regression beyond tolerance or a golden failure blocks, and a human decides whether the aggregate gain is worth an exception.
- Refresh the golden set from production incidents: every routing failure the desk escalates becomes a candidate golden example, with a review to keep the set from growing into a second validation set.
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Per-slice metric difference, challenger minus champion, with the slice's sample size and a tolerance derived from it. The slices that block are the product's, not the average.
- Golden-set pass rate for both models, and the list of cases that flipped from pass to fail — the list is what the desk reads.
- The aggregate, reported alongside, as context rather than as the gate.
- A golden-set pass rate of one hundred percent on every challenger is not success; it is the signature of a golden set that has leaked into training or has stopped being refreshed.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The golden set and the slice definitions still describe the cases the product cannot afford to lose — the desk's hard cases from last year are still this year's, or the set has been refreshed.
- No golden example has entered any training set, under any id, and the exclusion is asserted rather than assumed on every training run.
- Per-slice tolerances match the current sample sizes, so a block is a regression and not noise, and a pass is not a tolerance wide enough to hide one.
- Offline: the paired comparison on every candidate, with per-slice differences, per-case flips and the aggregate, attached to the promotion record (Model Lineage).
- On every training run: an id-overlap assertion between the training set and the golden set that fails the run on a single overlap; a content-hash check as a backstop against reissued ids.
- Over time: the golden-set refresh from escalated incidents, reviewed quarterly, and a re-derivation of slice tolerances whenever slice sizes change.
What can go wrong
- The golden set is small and static; the challenger passes it and fails a new pattern the desk has seen since, because the refresh process was never run.
- Slice tolerances are set once from a large slice and reused on a tiny one, so the tiny critical slice either always fails or never can.
- The exclusion by id is enforced against ticket ids, and a data migration reissues ids; the golden tickets re-enter training under new ids.
- The gate blocks a challenger with a large aggregate gain on one flipped golden case, the exception process is invoked, and after three exceptions the gate is advisory.
- A blocking gate on slices and golden cases will stop challengers that are better on average, and the decision to override is a product judgement that must be made each time, visibly.
- The golden set is curation work by the people who understand the failures, and it decays without a refresh process that also costs their time.
- Excluding the golden set from training withholds the most informative labelled examples from the model; the test's ability to fail is bought with those examples.
- "The challenger is better on the validation metric, so it is better." It is better on average over the validation distribution. The product depends on named slices and specific cases, and the average cannot see a fraction-of-a-percent slice getting worse.
- "Add the golden set to training so the model definitely learns those cases." Then the model has memorised them and the test can no longer fail. The golden set is a test; training on it is training on the test set with a friendlier name.
- "A slice of thirty tickets is too small to measure, so ignore it." Too small to measure tightly is not too small to matter. Report it with its uncertainty, grow it, and let the product owner decide the tolerance.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThat an aggregate metric cannot see a small slice regress follows from arithmetic and applies to every task and model; slices and a golden set are the general remedy, and the specific slices are the product's.
- DOMAIN-SPECIFICWhich slices are critical and how much regression is tolerable is a domain decision: a routing model may accept a small loss on one queue for a large gain elsewhere, while a medical or lending model may have slices where any regression is unacceptable regardless of the aggregate.
- CONTESTEDWhether a golden set should be a hard gate or a report for a human is disputed. The strongest case for a report is that a small curated set has high variance, that a single flipped case is often a labelling disagreement rather than a regression, and that a hard gate on it blocks good models and breeds exceptions; the strongest case for a gate is that the cases in the set are there because the product already paid for losing them once, and a human review that can be skipped under deadline pressure will be.
Where the depth lives
This domain teaches the model and hands the rest off by name.
- — Testing & Reliability Engineering — a golden set is a characterisation test suite for a model, and the discipline of keeping such a suite from ossifying into a rubber stamp is a testing question this lesson assumes rather than answers.