Drift Alert, No Loss

Decide what you would do from the brief alone, including whether you would change anything at all. Everything below it is available, but the exercise stops working if you open it first.

The brief you were given

The drift monitor on a recommendation model has fired daily for two weeks on four features. The on-call rotation is exhausted. A retrain was triggered on day three per the runbook; the alerts continued. A second retrain on day nine did the same. The team wants to raise the alert thresholds.

The trap — the fix that moves the metric and is not the fix

Raising the PSI threshold on the four features until the alerts stop. It is a one-line change, it ends the pages, and it is defensible because "these features are just noisy". Three months later a genuine upstream bug drives the same four features to a degenerate distribution, the shift is below the new threshold, and the recommendation quality falls for a week before anyone notices. The alert that would have caught it was disabled for the crime of having once been right about a change that did not matter.

Read this even if you are confident. It is here rather than behind a button because it is the answer most teams actually ship, it passes review, and its cost arrives weeks later when the labels do.