Problem Solving and ML
Business Problem → Target → Data → Metric → Model. The model is the last step, and the one the word "AI" makes people start from. The target is what you are actually predicting, the data is whether you can, the metric is what "good" means to the business — and the ML domain teaches each once they are named.
The situation, the reflex, and why it stalls
Every lesson starts where being stuck starts: someone has a problem, and the first move that comes to mind feels like progress.
Someone wants to "use ML" in the store. How does the loop get from that request to a formulated problem with a target, data, and a metric — or to the finding that ML is not the answer — and what does it hand to the ML domain?
The founder wants "AI recommendations" on the product page. You have a catalog, a few months of orders, and an assistant that will generate a recommender in a minute. You do not know what the recommendations are supposed to change, what the model would predict, whether the orders are enough to learn from, or how anyone would know if it worked.
Pick the model. Collaborative filtering is what recommendations are; there are libraries; the assistant can wire one up. A model producing recommendations is visible progress, and "does it work?" can be answered by looking at the recommendations and finding them plausible.
The model predicts something — probably "products bought together" — and nobody decided whether that is the thing that should change what the customer sees. The target was chosen by the library, and the business problem was never stated (What Am I Actually Trying to Achieve?).
- The model predicts something — probably "products bought together" — and nobody decided whether that is the thing that should change what the customer sees. The target was chosen by the library, and the business problem was never stated (What Am I Actually Trying to Achieve?).
- A few months of orders from a small store is very little signal spread across a large catalog. The model produces recommendations for popular products and noise for the rest; the plausibility check passes because popular products are plausible.
- "Does it work?" has no metric. Plausible is not a number; the business wanted more orders, or a bigger basket, or fewer returns, and none of those is what the model is scored on. It cannot be compared to the simplest alternative — show the best sellers — because nothing measures either.
- The simplest alternative was never tried. A rule — "show the three best sellers in the same category" — would have been an afternoon, would have had the same plausibility, and would have been the baseline the model has to beat. Without it, the model is compared to nothing (The Simplest Thing That Could Work).
The move
Precisely enough to apply it to a problem you have never seen — not a slogan.
- Run the chain: Business Problem → Target → Data → Metric → Model. The business problem is what should be different — bigger baskets, fewer abandoned carts — with an observation. The target is the specific quantity a model would predict, per example, that would move that problem if predicted well. The data is whether examples with that target exist, in enough quantity, without leaking the answer. The metric is the number that says the target is being predicted well enough to matter, connected back to the business observation. The model is last, and the ML domain chooses it.
- Before the model, build the baseline the chain makes obvious — a rule, a popularity list, the mean — and measure it on the metric. The baseline is the honest floor; a model that does not beat it has not earned its complexity (The Rule Baseline and Beating the Baseline in the ML domain).
- Ask at every step whether ML is still the answer. A target that a rule predicts well, data too thin to learn from, a metric the business cannot observe — each is a finding that the answer is a rule, more data, or a different problem, and the chain treats those as successes (When Not to Use ML).
- Hand off at the target and the metric. The ML domain teaches formulation, label construction, split strategy, baselines and evaluation; each is the depth behind one step of the chain, and the chain says which step you are at.
The chain, with the honest exits
Five steps, of which the ML domain owns the last four in depth. Each step has an exit — a finding that the answer is not a model — and the exits are results, not failures. The reflex starts at the fifth step and never sees the exits.
- 1Business Problem
What should change, with an observation the founder can see.
fails by "Add AI recommendations."
- 2Target
Per example, the quantity to predict, that would move the observation if predicted well. Exit: a rule predicts it well enough.
fails by The target the library assumes.
- 3Data
Examples with that target: how many, and was the label known at prediction time. Exit: too thin, or leaking — collect first.
fails by Whatever tables exist, joined.
- 4Metric
The number that says the target is predicted well enough to matter, connected to the observation. Exit: the business cannot observe it.
fails by The metric the library reports.
- 5Model
Chosen to beat the baseline on the metric — the ML domain's job.
fails by Chosen first; compared to nothing.
The baseline sits between Metric and Model and is not optional: it is what the model is compared to. Most of the chain's value on a small store is in the exits.
Rule, model, or neither
The decision the chain produces is rarely "which model". It is whether a model is the right response at all, and the options below are the three answers with the conditions that select each. The ML domain's "decision before model" and "when not to use ML" lessons are the depth behind the first two rows.
Given the target, the data and the metric, what should be built?
when A simple rule scores acceptably on the metric, or the data is too thin for a model to beat it, or the target is one a human can write down.
cost The rule must be maintained by hand and does not improve with data; the measurement must be kept so a model can be tried later.
when Every example carries a label known at prediction time, the count is large enough to hold out a test set, and the rule leaves a measured gap on the metric.
cost A pipeline, an evaluation, retraining, and every failure mode the ML domain lists; the model must beat the rule on held-out data, not on plausibility.
when The target cannot be written as "per something", or the metric is not observable by the business, or the observation would not move even with a perfect prediction.
cost Going back to the business problem, which is the least visible work and the most valuable.
What the recommender left unknown
The board below is the recommendations request after the chain, before any baseline was built. The unknowns are the ones that decided the chain's exit, each with the experiment that answered it.
- ✓Business problem: bigger baskets; observation: items per order.
- ✓Target: per product-page view, which shown product is added next.
- ✓Metric: fraction of shown recommendations added, tied to items per order.
- ~Customers who see a relevant product will add it. Plausible; the baseline experiment tests it.
- ~The catalog is stable enough that last month's co-purchases predict this month's.
? Is there enough data?
becomes How many products have been bought together with at least one other product more than a handful of times, and what fraction of the catalog is that?
experiment One query over order items grouped by product pair; count the pairs above a small threshold.
? Would a rule do?
becomes What does "three best sellers in the same category" score on the metric over a short experiment?
experiment Deploy the rule to a fraction of product-page views; record shown and added; compute the fraction.
? Does the metric move the business?
becomes If the fraction improves, does items per order move with it, or do customers swap rather than add?
experiment Compare items per order between views that showed recommendations and views that did not, over the same period.
The first unknown's answer — most of the catalog has no co-purchase signal — is what ended the chain at the rule. It took one query, and it would have taken a month to discover from a deployed model that recommended best sellers by accident.
How to do it
Most important first.
- Write the business problem with an observation: "customers buy one item; we want bigger baskets; observe average items per order".
- Write the target as a sentence with "per": "per product-page view, the probability the customer adds a second product from a shown set". If you cannot write the "per", there is no target yet (Target Definition in the ML domain).
- Count the examples with that target in the data you have, and check whether the label existed at prediction time or only after (Data Leakage is the ML lesson; the chain's question is simply "did we know this then?").
- Choose the metric by asking what number, if it improved, the founder would believe. Connect it to the target with a sentence.
- Build the rule baseline and score it. Only then open the ML domain to choose a model that has something to beat.
Worked on a concrete problem
The move has to produce something. This is what it produced.
- Recommendations, run through the chain. Business problem: bigger baskets; observe items per order. Target: per product-page view, which of a candidate set the customer will add next. Data: a few months of orders — thin; many products have never been bought with anything. Metric: the fraction of shown recommendations added to the cart, connected to items per order. Baseline: three best sellers from the same category, scored on the metric with a small experiment. Model: not yet. The baseline is deployed, measured, and becomes the number a model must beat when the data is thicker. The chain ended before the model, and that was the right ending (Decision Before Model).
- Fraud on checkout, where the chain reaches a model. Business problem: chargebacks; observe chargeback rate. Target: per order, the probability of a chargeback. Data: every order has a label, with delay — chargebacks arrive weeks later, so the split must respect time. Metric: chargebacks caught per legitimate order blocked, at a threshold the business chooses. Baseline: a rule on amount and address mismatch. Model: justified — the rule is beaten on held-out data — and the ML domain's lessons on class imbalance, thresholds and time-based splits are the next reading.
- The AI assistant for company documentation, where the target is the hard part. Business problem: fewer questions to the support channel; observe the count. Target: per question, the passage that answers it. Data: questions exist; the "right passage" label mostly does not. The chain's finding is that the metric needs labelled examples that have to be created, and that until they exist the system's quality is being judged by plausibility — the same trap as the recommender (Case: An AI Assistant for Company Documentation).
How you know it worked
What now exists that did not before, and what question you can now ask.
- The business problem has an observation, and the target is a "per something" sentence that would move it.
- The example count exists and the label's timing has been checked against prediction time.
- A baseline has been built and scored on the metric before any model is chosen.
- At least once, the chain has ended at "a rule" or "more data first", and that ending was treated as a result.
The questions you can now ask
The field this whole domain exists for. After this lesson, these are the questions to put to an unfamiliar problem.
- ?What should be different for the business, and what would I observe?
- ?Per what, predicting what — and would predicting it well actually move the observation?
- ?Do examples with that target exist, in what quantity, and did the label exist at prediction time?
- ?What is the simplest rule, what does it score on the metric, and has a model beaten it?
What can go wrong
- The chain is run to justify a model already wanted: the target is written to fit the library, the metric is whatever the library reports. Honest steps or no steps.
- The baseline is skipped as obviously bad. It is the floor; a model compared to nothing is a model that cannot be shown to work.
- The target is pursued to perfection while the business problem waits. A target that is "good enough to move the observation" is the goal; the ML domain's metrics say what enough is.
- The chain is applied where there is no prediction. "Use AI to write product descriptions" is a generation task with a different chain, and the target step will not fit.
- The chain frequently ends in a rule, which disappoints whoever wanted AI on the product page. The rule is measured and the model would not have been.
- Labelling and baseline-building are unglamorous work that precede any model, and on a small store may be most of the project.
- A metric connected to the business is harder to move than a model metric, and progress looks slower against it. It is the only progress the founder can see.
- "So do not use ML on a small store." Use it where the chain reaches a model with a baseline to beat — fraud did. The chain is the test, not a verdict on store size.
- "The baseline is a strawman." It is the floor and often the deployment. Best sellers by category is what many stores actually show, measured, forever.
- "This is ML engineering." It is the formulation step before it. The ML domain owns targets, leakage, splits and evaluation in depth; the chain says which of those you need next.
Where this applies
Problem-solving advice is stated as universal far more often than it is. These labels say what each method is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALBusiness Problem → Target → Data → Metric → Model applies to any predictive task; what changes by domain is how hard the target is to write — easy for fraud, hard for "helpfulness".
- DOMAIN-SPECIFICOn a store the chain usually ends in a rule for anything with thin data and reaches a model where every example is labelled — fraud, returns, demand. In a data-rich product the chain reaches a model more often and the data step becomes about leakage rather than quantity.
- ILLUSTRATIVEThe few months of orders, the best-seller baseline and the chargeback labels are invented to show the chain ending in different places; no real store's data is described.
Where the depth lives
This domain asks the question and hands the answer off by name.
- — The ML domain's problem-formulation lesson is the depth behind the first three steps; arrive with the business observation and the "per" sentence, and the formulation has something to work on.