Machine learning practice
Each challenge is a brief someone could plausibly hand you — a ticket, a dashboard, a stakeholder asking why the number dropped — followed by what is actually going on underneath it. Decide what you would do before you open anything else, and read the trap on every single one, because the trap is the fix that makes the metric move and leaves the cause in place.
Ticket from the courier app team: "Customers want to know when their parcel arrives. Build a model that predicts delivery time in hours from the shipment details. We have two years of completed deliveries." The current app shows a fixed 48-hour promise.
Slack from the analytics lead: "Great news, the churn classifier is at 97% accuracy on the test set. Marketing wants to start the win-back campaign on everyone it flags next Monday. Can you confirm it is ready?"
A colleague hands over a notebook: "Train/test split, 80/20. I tried about forty settings and kept the one with the best test accuracy. Test accuracy 0.91 — should be the number in the launch doc." There is no validation set.
The sales-ops team asks for a weekly demand forecast per SKU "using machine learning, because the spreadsheet is just last week's number". They want it in a month. A first model exists: a gradient-boosted regressor on calendar and price features.
The finance team's dashboard shows the house-price model at R² 0.89 and calls it "89% accurate". A regional manager says the model is "useless in our area" and wants it switched off there. Both are looking at the same model.
A ticket from support engineering: "The lead-scoring numbers in the CRM have not updated in nine days. The data scientist who ran it is on leave. Can someone re-run the model?" There is a notebook, a CSV on a laptop, and a pickle file with no date.
Marketing asks: "We ran k-means on customer behaviour and got five clusters. Can you confirm these are our customer segments so we can build campaigns around them?" The clusters have been named — "Loyalists", "Bargain Hunters" — from a 2D plot.