Loss Functions
Prediction vs target → loss. MSE, binary and categorical cross-entropy, ranking losses. The loss is what the optimiser minimises; it is not the metric the business cares about, and the gap is the design.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
The optimiser minimises the loss, the business watches a metric, and the two are different functions — how is each common loss defined, what does it reward, and where does the proxy diverge from the objective?
A marketplace ranks search results and pays on conversions. "The model's loss keeps improving with every retrain and the conversion rate on search has not moved in a quarter. Which number are we actually optimising?"
Cross-entropy on clicks is the standard loss; a lower loss means a better click model, and better clicks mean better search. Retrain, watch the loss fall, ship.
The loss measures how well the model predicts clicks on impressions the old ranker produced. It falls because the model learns position bias and clickbait titles better; neither is what makes someone buy.
- The loss measures how well the model predicts clicks on impressions the old ranker produced. It falls because the model learns position bias and clickbait titles better; neither is what makes someone buy.
- A pointwise loss treats every impression independently and rewards getting the absolute probability right on each one. The business needs the *order* within a query right, and a model can improve the pointwise loss while worsening the order at the top of the page (Candidate Generation vs Ranking).
- Offline the improvement was real on its own terms — lower cross-entropy on held-out clicks — and irrelevant to conversions, which nobody computed offline because purchases are sparse and delayed (Business Metrics vs Model Metrics).
- Once the model is deployed its own ranking shapes the next training set, so the loss on the new data is computed against labels the model helped create, and it keeps falling.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- Rank listings for a query so that the ones a user will buy appear first. The label is a purchase after a click; the business metric is purchases per search.
- The team has trained a pointwise classifier on click labels with binary cross-entropy and sorts by its probability — an optimisation objective two steps removed from the business one.
- One example is one (query, listing) impression with a click label; purchases are sparse and arrive later, so the model was trained on clicks as a proxy for purchases.
- Impressions are logged only for listings the previous model ranked in the top page, so the training data is shaped by the model it will replace (Feedback Loops).
- Click-through is dominated by position: the top result is clicked far more regardless of relevance, and the label carries that bias.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- A loss is a differentiable function of the prediction and the target, averaged over examples; the optimiser follows its gradient. Mean squared error, (ŷ − y)², is the regression default: its gradient is proportional to the error, large mistakes dominate, and its minimiser is the conditional mean (MSE, RMSE and MAE).
- Binary cross-entropy, −[y·log ŷ + (1 − y)·log(1 − ŷ)], is the negative log-likelihood of a Bernoulli label under the model's probability. Its gradient with respect to the logit is ŷ − y, which is why it pairs with a sigmoid output: confident wrong answers get a large push, and its minimiser is the calibrated conditional probability (Logistic Regression).
- Categorical cross-entropy, −Σ_k y_k · log ŷ_k over classes, with a softmax output, generalises to one label among K; only the true class's log-probability contributes. Ranking losses give up the absolute probability and score pairs or lists: the pairwise hinge, max(0, margin − (s⁺ − s⁻)), penalises a relevant item scored below an irrelevant one; a listwise loss compares the model's ordering of a query's items to the labelled ordering directly.
- The metric — purchases per search, NDCG at ten, recall at the operating threshold — is usually not differentiable, or not decomposable per example, or defined on outcomes that arrive too late to train on. The loss is a differentiable stand-in chosen so that minimising it *tends* to improve the metric. The gap between the two is where a model improves on paper and not in the product (Ranking).
The common losses, written out
Each loss encodes what "wrong" costs. Squared error says a mistake of two is four times as bad as a mistake of one, so outliers dominate. Cross-entropy says a confident wrong probability is catastrophically bad — log of a number near zero — and a hedged one is mildly bad, so the model is pushed towards calibrated probabilities. A pairwise hinge says only the order of a relevant and an irrelevant item matters, and only until they are separated by a margin.
The gradient of each is the mechanism. For cross-entropy with a sigmoid, the gradient with respect to the logit is simply ŷ − y: the model is pushed by exactly its error, which is why the pairing is standard and why it appears unchanged in the backpropagation lesson.
1import numpy as np2eps = 1e-123 4def mse(y_hat, y): # regression: minimiser is the conditional mean5 return np.mean((y_hat - y) ** 2)6 7def bce(p, y): # binary: p = sigmoid(logit); dL/dlogit = p - y8 p = np.clip(p, eps, 1 - eps)9 return -np.mean(y * np.log(p) + (1 - y) * np.log(1 - p))10 11def categorical_ce(probs, y_onehot): # K classes: probs = softmax(logits)12 probs = np.clip(probs, eps, 1)13 return -np.mean(np.sum(y_onehot * np.log(probs), axis=1))14 15def pairwise_hinge(s_pos, s_neg, margin=1.0): # ranking: only the order within a query16 return np.mean(np.maximum(0.0, margin - (s_pos - s_neg)))Notice what each one cannot see. MSE does not know which errors cost money; BCE does not know that two items were shown in the same query; the hinge does not know what the probability of a click was. The loss sees only its own arguments.
The proxy and the objective
There are three functions in play and they are routinely confused. The business objective is purchases per search. The offline metric is a ranking score on held-out queries with labels. The loss is a differentiable function on individual examples that the optimiser can follow. Each is an approximation of the one above it, chosen for tractability, and each approximation has conditions under which it stops tracking the one above.
The click-trained pointwise model fails at two of the three joints at once: cross-entropy on clicks is a proxy for the ranking metric that ignores within-query order, and clicks are a proxy for purchases that carries position bias. The loss kept improving because it was a good proxy for itself.
Validation cross-entropy on held-out click impressions lower on every retrain; the team's dashboard shows a steady quarter of improvement.
Purchases per search flat within noise across the same quarter; a holdback experiment on the latest model shows no difference from the model three retrains earlier.
- 1The loss measures pointwise click prediction on the old ranker's impressions; it improves by learning position bias and title effects, which do not change the order of purchase-likely listings.
- 2Within-query order at the top of the page is what drives purchases, and a pointwise loss does not optimise it.
- 3Each retrain's data was produced by the previous model, so the loss partly measures agreement with its own predecessor.
What must stay true about the proxy
A loss is chosen once, under a belief that minimising it improves the objective. The belief is an assumption that decays: the label process changes, the model reshapes its own training data, the business objective moves. None of that is visible in the loss, which will go on falling.
The assumption to state and monitor is the correlation between the offline metric and the online outcome across releases. When that weakens, the loss has drifted from the objective, and the fix is a different loss or a different label, not a longer training run.
Improvements in the loss on the proxy label translate into improvements in the offline ranking metric, which translate into purchases per search online.
holds when The label process and its biases are stable, the loss matches the decision the product makes (order within a query), and the offline-to-online relationship has been re-measured recently.
breaks when The model's own ranking reshapes the training impressions; the objective changes to margin or retention; a UI change alters position bias; the loss optimises per-example probability where the product needs order.
respond Revisit the label and the loss before the architecture: debias or change the label, move to a within-query loss, and re-establish the offline-to-online relationship with an experiment.
How to build it
Most important first.
- Write down the business objective, then the offline metric that approximates it, then the loss that approximates the metric — and state where each approximation is known to break.
- Match the loss to the decision. If the product needs an order within a query, use a pairwise or listwise loss on within-query pairs; if it needs a calibrated probability for a threshold, use cross-entropy and check calibration; if it needs a conditional mean, MSE; a median, absolute error (Choosing a Regression Metric).
- Train on the label closest to the business outcome that the data can support — purchases where they are dense enough, clicks debiased for position where they are not — and evaluate on the outcome even when training on the proxy.
- Evaluate the offline ranking metric on held-out queries alongside the loss; promote on the metric, never on the loss (Promotion Is a Checklist, Not a Score).
- Run the online experiment on the business metric before believing any of it; the loss cannot see conversions (A/B Testing Models).
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Purchases per search in an online experiment: the objective. Everything else is a proxy whose relationship to it has to be demonstrated.
- A within-query ranking metric on held-out queries with purchase labels, offline — the number that maps to the decision the ranker makes.
- The training and validation loss per epoch — for diagnosing optimisation, not for deciding to ship.
- Do not read "loss decreased on retrain" as improvement. It means the model predicts the proxy label on the old ranker's impressions better.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- Minimising the loss on the proxy label still moves the business metric in the same direction — the relationship demonstrated in the last experiment continues to hold.
- The labels the loss is computed on are still produced by the same process, and the position bias in them is stable or corrected.
- The operating point the product uses — top ten, threshold, ordering — is the one the loss and the offline metric were chosen for.
- The training data is not so shaped by the current model that the loss is measuring agreement with itself.
- Offline: the within-query metric on purchase-labelled held-out queries, alongside the loss; a retrain that lowers the loss and not the metric does not promote.
- Online: a holdback experiment on purchases per search for every model change; the offline-to-online relationship is re-estimated each time.
- Over time: the correlation between the offline metric and the online outcome across releases; when it weakens, the proxy has drifted from the objective and the loss needs revisiting (Offline vs Online Evaluation).
What can go wrong
- A pairwise loss on click pairs learns position bias twice as hard, because most "relevant above irrelevant" pairs in the data are "higher position above lower position".
- Switching to purchase labels makes positives so sparse that the loss is dominated by the negatives and the model learns to say no; reweighting fixes the loss and distorts calibration (Class Imbalance).
- The offline ranking metric improves and conversions fall, because the metric was computed on impressions the old ranker chose and the new one surfaces listings the metric has never seen.
- The business objective changes — margin per search instead of purchases — and the loss, the metric and the labels are all still optimising the old one.
- A loss closer to the business objective — listwise on purchase labels — is sparser, noisier and slower to train on than cross-entropy on clicks; the gain is that improving it means something.
- Debiasing clicks for position corrects the label and introduces a model of position bias that is itself wrong in places.
- Promoting on an offline ranking metric rather than the loss is slower and rejects models the loss liked; those are the models that would not have moved conversions.
- "Lower loss means a better model." Lower loss means a better fit to the proxy label on the training distribution. Whether it is a better model depends on whether the proxy tracks the objective there, which is an empirical question with a history of surprising answers.
- "Use cross-entropy; it is the standard loss for classification." It is the standard loss for a calibrated probability per example. For ordering items within a query it optimises the wrong thing.
- "We cannot train on purchases, they are too sparse." You can evaluate on them, which is what decides promotion; the loss can stay on the denser proxy.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThat the loss is a differentiable stand-in for a non-differentiable or delayed objective holds for every trained model, including linear ones and tree ensembles, whose split criteria are proxies for the same reason.
- TASK-SPECIFICThe right loss follows the task: MSE or absolute error for regression, cross-entropy for a probability, pairwise or listwise losses for ranking; using a classification loss for a ranking task is the specific mistake in this lesson and does not arise for a threshold-based classifier.
- SIMPLIFIEDThe formulas are the textbook forms without reduction, weighting or numerical stabilisation, and any quantities described are for the shape of the argument; production losses add clamping, class weights and label smoothing that change the gradient.