A model in production cannot be reproduced from the repository. What went wrong, and what must an artifact contain?
Answer it out loud before you open anything. The value of the flags below is in comparing them to what you actually said — including whether you asked about the data before naming a model.
A regulator asks a lender to explain a decision made eight months ago by a credit model. The team has the model file, the git commit of the training code and a fixed random seed. Retraining from that commit produces a model with different coefficients and a different decision for the case in question. Nobody can say which is the one that ran.
React to this
Say what you would question, what you would trust, and what you would need to know first.
What the team has (illustrative): model_v7.pkl sha256 present, no metadata inside git commit a91f3c2 train.py, features.py seed = 1337 training query: SELECT ... FROM loans.applications WHERE applied_at < '2025-12-01' encoder for employment_type: fitted in train.py, not saved; rebuilt at serving from a hard-coded list prediction log: score and decision, no feature vector, no model version
What it is really testing
Whether the candidate knows that reproducibility is a property of the whole pipeline — data, labels, features, preprocessing, code, environment, hardware and the artefact itself — and that a seed pins one of those. The artefact question tests whether they know what serving actually needs: the weights are the smallest part.