Problem Formulation

Start from the decision, not the model. What event to predict, when the prediction must exist, what action follows, what each mistake costs, and whether labels can be observed at all.

Problem Formulation
▶ lab

"Users are cancelling subscriptions" is a situation, not a task. Six questions turn it into Input X → Model → Prediction ŷ → Decision, and each one skipped is a model that answers something nobody asked.

Q · Someone says "users are cancelling subscriptions, can we predict it?" What has to be decided before that sentence becomes a modelling task, and in what order?
Decision Before Model
▶ lab

The model exists to change a decision. Name the decision, its owner, its capacity and its moment first, and most model choices — target, features, metric, threshold, inference mode — are made for you.

Q · Why does naming the decision before the model fix so many later choices, and what happens to a model built for a decision nobody named?
Prediction vs Decision

`P(churn) = 0.78` is a prediction. "Offer a retention discount?" is a decision. The model produces the first; a threshold, a cost and a policy turn it into the second, and none of those three lives in the model.

Q · The model outputs a probability. What has to happen between that number and the action the business takes, and why must those steps be designed and owned separately from the model?
Target Definition

The target must encode the outcome you actually care about, at a horizon, from a moment. `churned = cancelled within 30 days of the snapshot` is a target; `churned` is not.

Q · What makes a target definition precise enough to build a dataset from, and how does an imprecise one produce a model that is correct about the wrong thing?
Label Construction

Labels are built, not found. A versioned, tested query over raw events, parameterised by the snapshot moment and the horizon, is the difference between a label and a column that happened to be there.

Q · How do you turn a target definition into a label table that is correct for every (entity, snapshot) pair, reproducible later, and provably free of information from after the snapshot?
Label Leakage

A feature that carries the answer — `cancelled_at` used to predict `will_cancel` — gives excellent offline metrics and an invalid model. Leakage is about when information exists, not which columns are forbidden.

Q · How does information from the label reach the features, why does the offline evaluation reward it, and how do you tell a leaked feature from a legitimately predictive one?
When Not to Use ML
▶ lab

A rule works; labels cannot be observed; the decision cannot use a probability; a wrong prediction has unbounded cost; the data does not exist at prediction time. Any one of these is a reason to stop, and the formulation is where you find out.

Q · Which properties of a problem make a learned model the wrong tool even when a good model could be trained, and how do you recognise each one before building it?