Thresholding
The threshold is not part of the model. It is the point where a business decision about costs and capacity is written down, and it deserves an owner, a config and a review.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
Who chose 0.5, against what cost, and what happens to the business when the score distribution moves and the constant does not?
A marketplace flags listings for manual moderation before they go live. Moderators say half of what they see is fine; sellers say obviously fraudulent listings still get through. Engineering points out the model has not changed in six months. "So what changed?"
The model outputs a probability; 0.5 is the natural cut. Or, when 0.5 gives the wrong queue size, tune a constant until the queue fits, put it in the code next to the model call, and move on.
Volume doubled, so the same threshold now holds twice as many listings. Moderators fall behind, review gets shallower, and the "half of what we see is fine" complaint is the precision at a threshold nobody re-derived.
- Volume doubled, so the same threshold now holds twice as many listings. Moderators fall behind, review gets shallower, and the "half of what we see is fine" complaint is the precision at a threshold nobody re-derived.
- The category mix moved towards low-risk listings, so the score distribution slid down. The fixed threshold now catches a different slice of it — fewer of the real violations, which is the sellers' complaint.
- The threshold lives in the code next to the model call, so changing it is a deploy, and nobody on the moderation team can see what it is or ask for it to move.
- A retrain shifts the score scale slightly; the threshold is not re-derived; the queue size jumps and the incident is blamed on "the new model" when the model is fine and the constant is stale.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- Predict whether a listing violates policy and would be removed if reviewed. The label is the moderator's decision on reviewed listings, so unreviewed listings have no label and the training set is biased towards what past thresholds sent to review (Selection Bias).
- The decision is binary — hold for review or publish — and its two mistakes cost different people different amounts.
- One example is one new listing: category, price relative to category median, seller age, seller prior removals, text features from the title, image count.
- The moderation team has a fixed capacity; the threshold was set at launch so that roughly a fifth of listings were held, and has not been revisited.
- Listing volume has doubled and the category mix has shifted towards a low-risk category since launch.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- A threshold t partitions the score distribution. Everything above is flagged; the four confusion-matrix cells are the areas of the positive and negative score distributions on either side of t. Move t and all four move together: precision and recall trade against each other, and the queue size is the total area above t.
- The model defines the score distributions. The business defines the costs of a false positive and a false negative and the capacity. The threshold is where those meet, and it has a derivation — from expected cost, or from capacity — that can be recomputed whenever the inputs change (Threshold Selection).
- Because it is a function of the score distribution, a threshold is only valid for the (model version, population) it was derived on. A retrain, a drift or a volume change invalidates it silently: the constant still parses, it just means something else.
- A threshold is therefore configuration, in the same sense as a rate limit or a feature flag: owned outside the model, versioned, changeable without a model release, and logged with every decision.
What a threshold actually is
Picture the score histogram of genuinely violating listings and the histogram of fine ones. They overlap. A threshold is a vertical line through both. The area of the violating histogram to the right is recall; the share of everything to the right that is violating is precision; the total area to the right is the queue. There is no line that makes all three good, and which one to protect is not a modelling question.
This is why the threshold belongs to whoever owns the costs. A moderator-hours budget, a policy floor on recall, and the reputational cost of a fraud getting through are inputs the model has never seen.
Illustrative counts. The false-positive cell is what happens when volume doubles and the category mix slides towards low-risk: the same constant now sits in the fat part of the fine-listing histogram. Nothing about the model changed.
The threshold as configuration
The serving code should make the separation impossible to miss. The model client returns a probability and a model version. The decision layer looks up the threshold for that model version from a config the moderation team owns, applies it, and logs all four values. Changing the cut is a config change with a review, not a code deploy.
The version binding is the part people skip. A threshold derived on model v7's score distribution applied to v8's is a silent policy change; binding the config to the version turns it into an explicit one.
Who needs to change it, how often, and what has to stay consistent with it?
when A prototype, one consumer, and the cut is re-derived every training run and never touched between runs.
cost Every change is a deploy; the business cannot see or move it; a retrain silently changes policy.
when The cut is purely statistical (a fixed recall target) and re-derived automatically at training time.
cost Consistent by construction, but moving the cut between releases means cutting a release.
when The business owns costs or capacity and needs to move the cut on its own schedule, or several consumers apply different cuts to one score.
cost A second system to run, a config to own, and a version binding to test.
1type Scored = { p: number; modelVersion: string }2type Policy = { modelVersion: string; threshold: number; derivedFrom: string }3 4async function decide(listingId: string, scored: Scored, policies: PolicyStore, log: DecisionLog) {5 const policy: Policy = await policies.forModel(scored.modelVersion)6 if (!policy) throw new Error(`no threshold derived for model ${scored.modelVersion}`)7 const hold = scored.p >= policy.threshold8 await log.write({ listingId, p: scored.p, threshold: policy.threshold,9 modelVersion: scored.modelVersion, decision: hold ? 'hold' : 'publish',10 derivedFrom: policy.derivedFrom })11 return hold12}The throw is deliberate: a model version with no derived threshold is a promotion that skipped a step. derivedFrom names the sweep and the cost table the number came from, so the next person can re-derive rather than guess.
What must stay true for the constant to keep meaning something
A threshold is valid for a score distribution, a cost table and a capacity. All three drift. The cheapest early signal is the score distribution itself, because it needs no labels: a weekly histogram of served scores, compared against the one the threshold was derived on, moves before the queue does.
The other two are reviews rather than monitors. Someone has to ask the moderation team whether the capacity is still the capacity, and the policy owner whether the recall floor still stands.
The deployed threshold produces the precision, recall and queue size it was derived to produce.
holds when The served score distribution matches the derivation's; the model version is the one the threshold was derived for; costs and capacity are unchanged.
breaks when Volume or mix changes shift the score histogram; a retrain shifts the scale; capacity changes; a policy change moves the recall floor.
respond Re-derive from the current distribution and costs. Do not retrain the model to fix the queue — the queue is a threshold problem until the ranking metrics say otherwise.
How to build it
Most important first.
- Put the threshold in a decision service or a config store with an owner from the moderation team, not in the model code (Prediction vs Decision). The model returns a probability and a version; the decision layer applies the policy.
- Derive the threshold from the confusion matrix and the costs — walk it in the Threshold Explorer at
/ml/threshold— and record the derivation with the value, so it can be re-derived when the inputs change. - Log the score, the threshold, the model version and the decision for every listing, so that a queue-size change can be attributed to the score distribution or the constant (Tracing a Prediction).
- Re-derive the threshold on every model promotion and on a schedule, from the current score distribution and the current capacity; treat "the threshold was not reviewed" as a promotion blocker (Promotion Is a Checklist, Not a Score).
- Ship threshold changes behind a flag and watch the queue for a day; a threshold change is a rollout (Canary Rollout).
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- Precision and recall on reviewed listings at the current threshold, weekly. These are the moderators' and the sellers' complaints, respectively, as numbers.
- Queue size against capacity. A threshold whose queue exceeds capacity is not the threshold in effect — the backlog is.
- The score distribution itself, weekly, independent of labels: the earliest signal that the constant has gone stale.
- Overall model AUC looks relevant and does not move when the threshold goes stale; it measures the ranking, not the cut.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The score distribution on incoming listings stays close to the one the threshold was derived on; a weekly histogram of served scores checks this without labels.
- The costs and the capacity the threshold encodes are still the business's costs and capacity; a review with the owner on a schedule checks this.
- The threshold in the config store is the one applied, to the model version it was derived for; a contract test on a fixed listing pins both.
- Offline: the confusion matrix and expected cost across a threshold sweep on a held-out later period; the chosen threshold and its derivation recorded together.
- Online: queue size, precision and recall on reviewed listings at the deployed threshold, weekly; the served score histogram against the derivation's histogram.
- Over time: re-derive on every promotion and quarterly; diff the new threshold against the old and explain the movement before applying it.
What can go wrong
- The threshold moves to fit capacity and recall drops below what policy allows; nobody set a floor on recall so the "fix" is a slow policy failure.
- The decision service is fed a score from a new model version whose scale differs; the config was not versioned against the model.
- The threshold is tuned on reviewed listings only — the ones a past threshold selected — so the estimated precision is about a population the new threshold does not see (Selection Bias).
- A decision service is a second system with its own config, owner and failure modes, in exchange for the business being able to move the cut without engineering.
- A threshold that tracks capacity keeps the queue workable and lets recall float; a threshold that tracks recall keeps policy and lets the queue grow. One of them has to be the constraint.
- Logging every score and decision is the only way to audit the threshold and is also a store of moderation judgements that needs access control.
- "The model has not changed, so the flagging has not changed." The model defines the scores; the threshold defines the flags; the population defines where the scores land. Two of the three moved.
- "0.5 is the model's threshold." 0.5 is where the sigmoid crosses z = 0. It has no relationship to the cost of a wrong hold or a missed fraud.
- "We tuned the threshold at launch." A threshold is derived from a score distribution and a cost; both have expiry dates.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALEvery binary decision derived from a score has a threshold, whether it is written as a constant, a top-k cut, or a capacity; the ownership argument holds regardless of what produced the score.
- CONTESTEDA reasonable position holds that a separate decision service is over-engineering for most teams: the threshold can live in the model's own config, versioned with the artifact, and re-derived as part of every training run, which keeps model and cut consistent by construction. That is right when the cut is purely statistical; it fails when the business needs to move the cut between model releases, which is when the separate owner earns its cost.
- SIMPLIFIEDQueue sizes and any precision or recall figures here are illustrative; the point is that the threshold's meaning depends on a distribution that moves.
Where the depth lives
This domain teaches the model and hands the rest off by name.