ML Engineering Roadmap

You can take a request like "predict which customers will churn" and turn it into a formulation you could defend: which event, observed when, feeding which action, with which mistake costing what — and you can say whether it needs a model at all. You can name the paradigm and the task type from where the learning signal comes from, and you can produce the rule, mean or majority baseline that any model you build later must beat before it has earned its cost. Most importantly you can walk the pipeline from raw data to feedback and point at the stage where each of the sixteen classic failures enters.

You can build a training set and say what one row represents, which filter or join introduced which bias, and why the class balance looks the way it does. You can choose a split — random, time-based, grouped, stratified — from the way the model will be used rather than from habit, and you can run a leakage audit on a feature list: for every column, when does this information exist relative to the prediction? You can engineer aggregation, temporal and encoded features and explain why each one must be reproduced identically at serving time, and you can read a feature-importance chart without concluding that anything causes anything.

You can train a linear or logistic regression, read its coefficients and residuals, and explain what the sigmoid output is and is not. You can build a confusion matrix with a price on each cell, choose a threshold from those prices rather than defaulting to 0.5, and say when accuracy, precision, recall, ROC AUC, PR AUC or a calibration curve is the number that actually answers the question. For regression you can pick between MAE, RMSE and MAPE from what a large error costs, and you can explain to a stakeholder why an illustrative "94% accuracy" on a 95%-negative dataset is worse than the majority baseline.

You can design an evaluation — cross-validation or a temporal holdout, sliced by the segments that matter, with an error bar on the headline number — and you never touch the test set to tune. You can read a learning curve and say whether the fix is more data, more capacity or more regularisation, and you can explain overfitting as a mechanism rather than a vibe. You can train and tune a random forest or a gradient-boosted ensemble on tabular data, explain why boosting fits residuals, and give a reasoned answer to "which model family" that mentions data size, latency, interpretability and cost before it mentions a library.

You can write a forward pass and a loss, derive the gradients on a computational graph, and explain why a learning rate that is too large diverges and one that is too small crawls. You can train an embedding, use cosine similarity correctly, and warn a colleague that the 2D projection shows neighbours the model does not have. You can say what a convolution, a recurrent cell and self-attention each assume about the input, and you can take a pretrained model and fine-tune it — fully or parameter-efficiently — with a defensible reason for the choice. You can run a tuning budget without ever tuning on the test set and without believing that AutoML removed the search.

You can build a forecast with a rolling-origin validation that respects time, decompose trend from seasonality, and say which horizon the forecast is good for and where it stops being one. You can design a recommender as candidate generation plus ranking, choose collaborative or content-based signals from what the cold-start problem looks like, and — the part that separates this level — explain how the recommender changes the data it will be trained on next, and how you would keep exploring so that the feedback loop does not quietly narrow the catalogue.

You can reproduce any training run from what was recorded — data version, feature version, code, config, seed — and trace a served prediction back to all of them. You can package a model as an artifact with its preprocessing inside it, promote it through a registry on quality, latency, memory and cost rather than one offline score, and choose batch, online or streaming inference from freshness, latency budget and volume. You can find train/serve skew by comparing logged serving features with recomputed training features, decide with a straight face whether a feature store is worth its cost for your team, and ship a fallback before you ship a second model.

You can read an inference cost bill and say whether the fix is a smaller model, quantization, batching or simply a CPU, because you know what memory bandwidth and VRAM decide. You can split a training run across devices — data, tensor or pipeline parallel — checkpoint it, and estimate what it costs before you launch it. You can write the tests a model needs that a unit-test suite does not provide: data and feature tests, a training smoke test, invariants, a serving contract, a regression suite against the champion. And you can report subgroup performance, explain a prediction with the caveat that the explanation is approximate, and say where predicting an outcome stops and causing it begins.

You can operate a model as a system: monitor feature, prediction and outcome distributions separately, tell data drift from concept drift from a feature bug, and treat drift as a signal to investigate rather than a trigger to retrain. You can run a champion/challenger, a shadow deployment, a canary and an A/B test, and roll back — and you can debug an incident from the business metric down to the feature that changed, then write the postmortem. You can describe an MLOps pipeline without the word Kubernetes, defend a platform decision on cost, and reason about poisoning, supply chain and inference abuse. Finally you can design recommendation, fraud, churn and search-ranking systems end to end, and say where your model ends and an agentic system begins.