Cold Start
A new user or item has no interaction history, so an interaction-trained model has nothing but noise for it. The answers are popularity, content, asking, and deliberately showing it — none of which is a better model.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
What does a recommender do for a user or item it has never seen, and why is the embedding of an unseen item not an answer?
A marketplace onboards thousands of new sellers a month. The seller-success team says: "new listings get no views for their first two weeks, sellers give up, and by the time the system knows a listing is good the seller has left. Fix the first two weeks."
The model has an embedding table. A new item gets a new row, initialised randomly or to the mean, and the model scores it like anything else. As interactions arrive the row is updated. Nothing special needed.
A random initialisation is a random vector; the model's score for it is noise, and noise is rarely at the top of anyone's list. The item is not shown, gets no interactions, and the row is never updated — the initialisation is permanent.
- A random initialisation is a random vector; the model's score for it is noise, and noise is rarely at the top of anyone's list. The item is not shown, gets no interactions, and the row is never updated — the initialisation is permanent.
- A mean initialisation places every new item at the centre of the catalogue, where it scores moderately for everyone and highly for no one. It is out-ranked by every established item with a real vector, with the same result.
- Offline evaluation does not notice, because new items have no held-out interactions to be evaluated on. The metric is computed entirely over items the model already knew.
- For new users the symmetric failure: a mean user vector recommends the catalogue's centroid — the most generically popular items — and a user who wanted something specific leaves before the first interaction.
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- For a listing with no interactions, an estimate of how it would perform if shown — enough to decide whether and to whom to show it. For a user with no history, an estimate of what they would engage with.
- The honest label for a new item does not exist yet; the target is a prior that gets replaced by evidence as soon as evidence is allowed to accumulate.
- Item attributes at creation: category, price, text, images, seller history. These exist on day one and are the only item-side signal.
- User attributes at signup: device, locale, acquisition channel, and whatever an onboarding flow asks. Thin, but not nothing.
- Interaction data for *other* items and users, which is what the main model learned from and what the new entity lacks.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- An interaction-trained embedding is a summary of who interacted with the entity. With no interactions the summary is empty, and whatever the row contains — random, mean, zero — is the initialisation, which carries no information about the entity. Scoring it is applying learned weights to noise (Embedding Training).
- Every genuine cold-start strategy substitutes another source of information: the population (popularity), the attributes (content), the user (onboarding questions), or deliberate experimentation (showing the item to find out). The substitution is a prior, and the design question is how fast evidence replaces it.
- The transition is the hard part. An item that has three interactions has a vector that is barely better than noise; a rule that switches from the prior to the embedding at a fixed count is a cliff, and a blend weighted by evidence is what avoids it.
Why the embedding of an unseen item is noise
The item embedding table is learned by gradient descent from interactions. An item with interactions has its row pulled toward the users who touched it; an item with none has its row exactly where it was initialised. The model has learned nothing about it and the score is the model's reaction to a random vector.
This is the same mechanism as the Embedding Explorer's warning about projection, seen from the other side: the position of a vector means something only if it was placed there by training. An untrained row has a position but no meaning (Embedding Projection Caveats).
1def item_vector(item, n_interactions, learned_vec, content_to_emb, k=20):2 # content_to_emb: a model trained on established items to map3 # attributes -> the interaction-embedding space4 prior = content_to_emb(item.attributes)5 if n_interactions == 0:6 return prior # never the random initialisation7 # weight moves toward the learned vector as evidence accumulates;8 # k is "how many interactions equal one prior's worth of confidence"9 w = n_interactions / (n_interactions + k)10 return (1 - w) * prior + w * learned_vecThe constant k is a claim about how noisy early interactions are. If the first ten interactions on a listing are the seller's friends, k should be large; if they are organic buyers, small. It cannot be tuned on the overall held-out metric, which contains no new items.
The four sources of information that are not history
Every cold-start strategy answers "what do we know instead?" Popularity uses the population. Content uses the attributes. Onboarding asks the user. Exploration shows the item and finds out. They are not alternatives; a working system uses all four in a sequence that ends with history taking over.
The choice among them is about what is cheapest to obtain for the entity in question and how much it predicts. For a new user, asking costs a screen of signup and predicts a lot; for a new listing, attributes are free and predict moderately; for both, exploration is the only source that produces real evidence.
For an entity with no interactions, which information should the first score come from?
when Nothing else is known and the segment (locale, category, channel) is informative.
cost Identical recommendations for everyone in the segment; the system learns nothing from the response.
when Rich attributes exist and a mapping to the embedding space can be trained on established entities.
cost A second model to maintain; a prior that is only as good as the attributes and mislabelled attributes are common.
when The entity is a user who can be asked, and a few answers predict a lot.
cost Signup drop-off; answers go stale and are rarely refreshed.
when The entity must accumulate real evidence and the cost of a few poor impressions is acceptable.
cost Continuous engagement cost on the reserved slots; a target for gaming by whoever creates items.
What must stay true for new entities to stop being new
The system's promise is that a new item becomes an established one within a window. That depends on the exposure reserved for it actually happening, which is a policy that other ranking rules can override without anyone deciding to.
So the assumption to monitor is not about the model; it is about the exposure. A cohort chart of new items against cumulative interactions is the check, and a flatlining cohort is the alarm.
A new item receives enough impressions within the business's window for interaction evidence to replace the prior.
holds when A reserved exposure share exists, is enforced after all other re-ranking rules, and is distributed across new items rather than concentrated on the few the prior likes best.
breaks when A promotion or business rule fills the slots the exposure share used; the prior concentrates exposure on a subset; the share is a config value nobody monitors.
respond Restore or redistribute the exposure share. Do not retrain the main model: it cannot learn about items that were never shown, whatever data it is retrained on.
| Trigger | Symptom | Cause | Response |
|---|---|---|---|
| Exposure slot filled by the prior's favourites | A few new items warm up fast; most stay cold | Exploration ranked by the content score instead of spread across new items | Sample new items for the slot with a floor on each item's share |
| Onboarding shortened for completion rate | New-user first-session engagement drops while signup completion rises | The questions that carried signal were removed | Measure onboarding on first-week retention, not completion |
| Blend weight tuned on the global metric | New items are scored as if the prior did not exist | The held-out set has no new items, so the tuner prefers ignoring the prior | Tune the blend on the new-item cohort's first-week outcomes |
How to build it
Most important first.
- Give a new item a content-derived vector — from its attributes through a model that maps attributes to the embedding space — rather than a random row, so the first score is a guess based on what the item is (Content-Based Recommendation).
- Reserve exposure for new items: a slot or a share of traffic in which new items are shown to users the content model thinks are plausible, so interactions can begin (Exploration vs Exploitation).
- For new users, ask: a short onboarding flow that selects a few interests buys more than any inference from device and locale, and its cost is measured in drop-off, which is measurable.
- Blend prior and evidence by count: the effective vector is a weighted combination of the content-derived prior and the interaction-learned vector, with the weight moving toward evidence as interactions arrive.
- Measure new-item outcomes separately: time to first impression, time to tenth interaction, and the survival of new items into the established set.
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- For items: the distribution of time-to-first-impression and time-to-N-interactions for items created this month. This is the number the seller-success team is actually asking about.
- For users: engagement in the first session and first-week retention for new users, split by whether onboarding was completed.
- Do not measure the main model's held-out metric and conclude cold start is handled. New entities are not in the held-out set.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- The attributes available at creation predict how an item will perform well enough to be a useful prior — the mapping from attributes to the embedding space learned on established items still applies to new ones.
- New items receive exposure at a rate that lets evidence accumulate within the window the business cares about, and that exposure is not silently cancelled by other ranking rules.
- The count at which interaction evidence dominates the prior reflects how noisy early interactions actually are; if the first interactions are from bots or the seller's friends, the threshold is wrong.
- Offline: hold out items by creation date and evaluate the content-derived prior on their first-week interactions; a prior that ranks them no better than random is not a prior.
- Online: an A/B test of the new-item exposure policy on time-to-first-impression and new-item survival, with overall engagement as a guardrail.
- Over time: a weekly cohort chart of new items by age against cumulative interactions; a cohort that flatlines is the cold-start problem coming back.
What can go wrong
- The new-item slot is filled by whichever new items the content model scores highest, which is the same few, and the rest of the new items are as cold as before.
- The exposure reserved for new items is switched off during a promotion and never switched back; time-to-first-impression regresses and nobody is watching it.
- The onboarding questions are optimised for completion rate until they ask nothing useful.
- The blend weight is tuned on the overall held-out metric, which prefers ignoring the prior, so new items are effectively switched back to noise.
- Reserved exposure for new items costs engagement on those slots, continuously, and the return arrives as catalogue health rather than as a metric anyone owns.
- A content-derived prior needs an attribute-to-embedding model that is one more thing to train, version and keep aligned with the main embedding space.
- Onboarding questions cost signup completion; the trade is between a worse first session for everyone and a lost signup for some.
- "The embedding table handles new items — they get a row." A row is memory, not information. The initialisation is noise and stays noise until the item is shown, which it will not be.
- "Popularity is a fine cold-start fallback for users." It is a fine fallback for measuring nothing: every new user sees the same list, and the system learns nothing about any of them from their response to it.
- "Cold start solves itself as data accumulates." Data accumulates for items that are shown. The ones that are not shown are still cold a year later.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALAny model whose representation of an entity is learned from that entity's history has nothing for an entity with no history; the specific fix differs, but the noise-in-the-embedding mechanism is the same for users, items, sellers, queries and sessions.
- DOMAIN-SPECIFICOn a marketplace or a news product, new items are the whole business and cold start is the main problem; in a film catalogue that adds a few titles a week, editorial placement handles it and the engineering investment is not worth making.
- CONTESTEDA serious position holds that reserving exposure for new items is a subsidy that costs real engagement and mostly rewards items that would have failed anyway, and that a strong content-derived prior plus organic discovery through search is enough. The counter is that without some exposure the prior is never tested and the content model never learns which attributes predicted success, so the subsidy is also the training data for the prior.
Where the depth lives
This domain teaches the model and hands the rest off by name.