advancedML Testing

We Need a Feature Store

Decide what you would do from the brief alone, including whether you would change anything at all. Everything below it is available, but the exercise stops working if you open it first.

The brief you were given

An architecture proposal lands in review: "Three teams have had train/serve skew incidents this year. We propose a feature store as the platform fix. Six-month project, one platform engineer, migration of all feature pipelines." Leadership is inclined to approve it because the incidents were expensive.

The trap — the fix that moves the metric and is not the fix

Approving the six-month project because expensive incidents deserve a platform response. The store gets built, the migration takes nine months instead of six, and in month four a fourth incident occurs — a backfilled source table — which the store does not prevent because nothing in it versions upstream sources. The incident is read as evidence the migration must finish faster. The contract tests that would have caught three of the four incidents in an afternoon each were never written, because the platform was going to solve skew.

Read this even if you are confident. It is here rather than behind a button because it is the answer most teams actually ship, it passes review, and its cost arrives weeks later when the labels do.