advancedEmbeddings

The Index Remembers the Old Space

Decide what you would do from the brief alone, including whether you would change anything at all. Everything below it is available, but the exercise stops working if you open it first.

The brief you were given

After a scheduled retrain of the product-embedding model, the "similar items" feature started returning bizarre results for about a third of products. Rolling back the model fixed it. The team wants to know how to retrain safely, since the old model is now three months stale.

The trap — the fix that moves the metric and is not the fix

Scheduling the backfill to complete before switching the model, and treating it as a sequencing bug. The full re-embed now finishes first, the index is rebuilt, the model is switched, and the similar-items feature works — until someone notices that the ranking model downstream, which takes the product embedding as one of its features, has quietly lost accuracy, because it was trained on the old space and is now fed vectors from the new one. The sequencing fix repaired the consumer that broke loudly and left the one that broke silently.

Read this even if you are confident. It is here rather than behind a button because it is the answer most teams actually ship, it passes review, and its cost arrives weeks later when the labels do.