The New Model Adds 80 ms
Decide what you would do from the brief alone, including whether you would change anything at all. Everything below it is available, but the exercise stops working if you open it first.
A new ranking model improved the offline metric and, in canary, adds about 80 ms to p99 in two of five regions and much less in the others (illustrative). The search page budget is tight. Serving runs on the same container image in every region. The team asks whether to raise the latency budget, ship a smaller model, or add GPUs.
Distilling the model to a smaller one that runs in half the time and shipping that. The model execution stage drops from 6 ms to 3 ms, the change is real and measurable on a benchmark, and the p99 in the two slow regions drops by 3 ms. The team, reasonably, concludes the model is now as fast as it can be and requests the budget increase. The 80 ms round trip is untouched, and the fix also discarded some of the offline gain in the distillation. The framing "the model adds 80 ms" was accepted without anyone measuring where the 80 ms was.
Read this even if you are confident. It is here rather than behind a button because it is the answer most teams actually ship, it passes review, and its cost arrives weeks later when the labels do.