Same Weights, Different Features
Decide what you would do from the brief alone, including whether you would change anything at all. Everything below it is available, but the exercise stops working if you open it first.
A credit model's production approval rate dropped from 61% to 44% (illustrative) the day the serving path was migrated from a batch job to a new online feature service. The model artefact did not change. The service owners say every feature is "the same query, just faster".
Retraining the model on features pulled from the new online service, so training matches serving. It does match, the approval rate comes back to something reasonable, and the skew is gone — because the model has now been trained to interpret "0" as "not verified" and a forward-shifted address tenure as normal. The training set was built by replaying the online service over historical applicants, which computes every feature as of *now* rather than as of the application, so the retrained model is trained on point-in-time-incorrect features and its offline metric is leaked upward. The skew was closed by moving the bug into the training data.
Read this even if you are confident. It is here rather than behind a button because it is the answer most teams actually ship, it passes review, and its cost arrives weeks later when the labels do.