Choosing a Paradigm
Have labels? Can you get them? What do they cost, and how long until they arrive? What does "structure" mean to the business? The paradigm is the answer to those questions, not a preference.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
A team wants to "use ML" on a new problem. Which questions about labels, cost and business meaning decide whether it is supervised, unsupervised, semi-supervised or self-supervised — and when is the answer none of them?
A logistics company asks: "Trucks break down and we lose a day each time. We have telemetry from every vehicle for three years. Can ML predict breakdowns?"
Breakdowns are labelled in the maintenance system, so this is a supervised classification problem. Join telemetry to breakdown events, train a classifier, and alert when the probability crosses a threshold. If labels are patchy, use anomaly detection instead — a breakdown is an anomaly.
The structured label covers one year at the newer depots, so the supervised model learns those depots' vehicles and fails silently at the others; the free-text depots have breakdowns that were never counted as labels and so appear as negatives.
- The structured label covers one year at the newer depots, so the supervised model learns those depots' vehicles and fails silently at the others; the free-text depots have breakdowns that were never counted as labels and so appear as negatives.
- The anomaly-detection fallback flags every vehicle that is unusual — a new model of truck, a route through mountains — and most unusual vehicles do not break down. Precision at the top of the list is low and the fleet manager stops reading the alerts (Anomaly Detection).
- Neither approach asked what a pull costs against a missed breakdown, so the threshold was set by the score distribution rather than the decision.
- The offline metric was computed on the labelled year at the labelled depots, and it was fine.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- The candidate target is a breakdown event within the next N days. Whether that target exists as a label depends on whether the maintenance system recorded breakdowns consistently, and whether "breakdown" means the same thing across depots.
- The decision downstream is whether to pull a truck for inspection, which has a cost per pull and a cost per missed breakdown; the paradigm has to produce something that decision can use (Decision Before Model).
- One example could be one truck-day of telemetry — engine temperature, vibration, mileage, fault codes — joined to maintenance records. Whether the join gives a label depends on the records.
- Maintenance records exist for all depots but are free text at half of them; a structured breakdown flag exists only at the newer depots, for the last year.
- Labels for the near future do not exist yet: a truck that will break down next week is unlabelled today, and the label arrives only after the event (Ground-Truth Delay).
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- The paradigm is determined by what the learning signal can be. If a consistent, observed outcome exists for the population you will serve, supervised learning can point the gradient at it. If it exists for a small subset and the rest is the same distribution, semi-supervised can extend it. If no outcome exists and the business can say what structure it wants, unsupervised can look for that. If the data is abundant and the outcomes are many and small, a self-supervised representation can be built once and specialised.
- Each option has a cost structure: labels cost money and time; unlabelled data is cheap but needs a distribution match; a chosen objective is free but needs an external validation; a pretext needs scale and compute. The choice is a comparison of those costs against what the decision is worth.
- And the option that is often right is none of them: a rule from the maintenance manual, a threshold on a fault code, or a change to the inspection schedule (When Not to Use ML, The Rule Baseline).
The label questions, in order
The paradigm is the answer to a short sequence of questions about labels. Each answer removes options. Asked in order they usually leave one, and it is often not the one the team arrived with.
For the fleet, the answers are: an outcome exists at some depots for one year; it is inconsistent across depots; more can be obtained by structuring free text at a cost; one label means "a maintenance event a depot chose to record as a breakdown".
What learning signal does the business actually have?
when A consistent, observed outcome exists for the population you will serve, or can be obtained at a cost the decision justifies.
cost Label cost and delay; selection bias in who has a label; a label process that must be versioned and monitored.
when A small consistent labelled set exists and the large unlabelled pool is from the same distribution, as a discriminator test confirms.
cost A gold holdout the loop never touches; per-class caps; the cluster assumption, which nothing in the loss checks.
when No outcome exists, the business can say what structure it wants, and an external validation of that structure is planned and funded.
cost Distance and scaling are the label; the external validation is a campaign or a quarter of outcomes; no offline failure signal.
when Data is abundant, many downstream tasks will share a representation, and the scale and compute for pretraining exist.
cost The most expensive run; corpus bias in the representation; every downstream head pinned to the encoder version.
when A rule from domain knowledge reaches most of the value, or the labels needed to evaluate any model do not exist and must be collected first.
cost The rule is crude and the ML team is idle for a year; the label collection must be designed so it is not the rule's own selection.
Costing the paradigm against the decision
Each paradigm has a price. Supervised pays in labels; semi-supervised pays in a gold set and a distribution check; unsupervised pays in an external validation; self-supervised pays in compute and corpus curation. The price is compared with what the decision is worth, and a fleet manager can put a number on a pull and a missed breakdown.
The matrix below scores the options for the fleet problem. The scores are the shape of the argument; they will differ for another problem and should be redone rather than reused.
| Option | Quality | Latency | Cost | Interpretability | Data needed | Operational | Note |
|---|---|---|---|---|---|---|---|
| Supervised, scoped to labelled depots | Highest quality where labels exist; half the fleet uncovered. | ||||||
| Semi-supervised across all depots | Extends coverage only if depot telemetry is distributed alike; needs a gold holdout per depot. | ||||||
| Anomaly detection, no labels | Cheap and fast; flags unusual, not failing; someone must validate the top of the list. | ||||||
| Self-supervised telemetry encoder + heads | Reusable across breakdown, fuel and route tasks; a large investment before any decision improves. | ||||||
| Fault-code rule + label collection | Deploys now; collects the labels every other option needs. |
caveat The scores cannot express that the last row changes the scores of the first two within a year, or that the fleet manager's cost ratio between a pull and a missed breakdown decides whether "quality" means precision or recall. A matrix is a conversation aid, not a verdict.
The choice is an assumption too
Choosing a paradigm commits the system to a label source, a cost of labels and a definition of the decision. Each of those can move: depots migrate to structured records, labels become cheap, the inspection policy changes. When they move, the paradigm that was right becomes the paradigm that was chosen.
So the choice is documented with its reasons, and the reasons are checked on a schedule. A paradigm chosen because labels were scarce should be revisited when they are not.
The label coverage, cost and delay that made this paradigm the right one, and the decision it serves, are still what they were when it was chosen.
holds when Label coverage per depot is tracked and stable; the decision's cost ratio is stated in the model's documentation and unchanged; the external check for the paradigm's blind spot is within its expiry.
breaks when Depots migrate to structured records and labels become abundant; a new truck model enters the fleet with no labelled history; the business changes what counts as a breakdown.
respond Re-choose. A supervised model scoped to two depots should be widened when labels arrive elsewhere; an anomaly detector should be replaced when a consistent outcome exists.
How to build it
Most important first.
- Ask the label questions first, in order: does an observed outcome exist for the served population; is it consistent; can more be obtained, at what cost and delay; and what does one label mean. The answers narrow the paradigm before any model is discussed (Target Definition, Label Construction).
- If labels are partial, decide whether to buy more (structure the free-text depots' records), extend them (semi-supervised, with a distribution check), or scope the model to where they exist. Scoping is the option teams forget.
- If the business wants structure rather than an outcome, write down what structure and how it will be validated externally before choosing a clustering or an encoder (Unsupervised Learning).
- Cost the paradigm against the decision: a supervised model that needs a year of labels at every depot may be worth less than a fault-code rule deployed next week and used to collect the labels (Baselines Are Mandatory).
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- The decision number: breakdowns avoided per inspection pull, on the served fleet. Every paradigm has to produce something that can be measured against that.
- Label coverage: the share of the served population that has a consistent outcome label. Below some coverage, supervised is scoped, not chosen.
- Label delay and cost per label. These decide whether more labels can be bought, and how long a supervised model takes to be validated in production.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The label source the paradigm was chosen around — structured breakdown flags at specific depots — still covers the population the model is served on, and its coverage is tracked as depots change systems.
- The cost and delay of a label that justified the paradigm are still what they were; if labels became cheap, the choice should be revisited.
- The decision the paradigm was chosen to serve — pull for inspection at a stated cost ratio — is still the decision, and its threshold is set from that ratio rather than from a score distribution.
- Offline: label coverage per depot and per vehicle model; label consistency check by re-deriving the breakdown flag from raw records at two depots and comparing.
- Online: shadow whatever was chosen against the rule baseline for a quarter, measuring pulls and missed breakdowns per depot, including the unlabelled ones.
- Over time: re-ask the label questions annually; as the free-text depots migrate to structured records the right paradigm may change.
What can go wrong
- Labels are bought by structuring the free-text records, and the structuring is done by a model trained on the newer depots, so the new labels inherit the old bias.
- The rule baseline works well enough that the ML project is cancelled, and the label collection it was meant to bootstrap stops with it.
- The anomaly-detection approach is kept because it "needs no labels", and its alerts are validated by nobody, so a year later there is still no label.
- Scoping a supervised model to the labelled depots is honest and leaves half the fleet without a model, which the sponsor will not enjoy hearing.
- Buying labels by structuring records is slow, manual and boring, and it is often the highest-value ML work on the project.
- Starting with a rule delays the model by a year and is frequently the right call, which makes the ML team look idle.
- "We have no labels, so it must be unsupervised." No labels means either get some, scope to where they exist, or define what structure the business wants and how to validate it. Unsupervised is not a default; it is a different problem.
- "We have labels, so it is supervised." Labels that cover one depot for one year are labels for that depot and year. The paradigm is chosen for the population you will serve.
- "Anomaly detection needs no labels." It needs labels to know whether the anomalies it finds are the ones that matter, and someone has to look at the top of the list.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThat the paradigm follows from the availability, cost and meaning of labels rather than from the algorithm one prefers holds for every domain and model family.
- DOMAIN-SPECIFICIn predictive maintenance and medicine the outcome is rare, delayed and expensive to confirm, so label questions dominate; in click prediction the label is free, instant and abundant, and the paradigm question collapses to supervised with the bias of implicit feedback as the real issue (Ranking).
- CONTESTEDA serious position holds that a self-supervised representation should be built first on any large corpus, before the label questions are asked, because it is reusable across every task that follows and the label questions become cheaper on top of it. The counter is that a representation without a decision to serve is infrastructure looking for a customer, and a rule plus a label-collection plan delivers value while the corpus is still being deduplicated.
Where the depth lives
This domain teaches the model and hands the rest off by name.
- — Product management — the cost of a pull against a missed breakdown, and whether the fleet manager will act on a probability at all, are product questions that precede every paradigm question here.