Machine Learning Engineering

What a machine learning system actually is — a pipeline from raw data to feedback — and the sixteen things that go wrong in it, most of which are invisible offline.

Start from the decision, not the model. What event to predict, when the prediction must exist, what action follows, what each mistake costs, and whether labels can be observed at all.

Supervised, unsupervised, semi-supervised and self-supervised — distinguished by where the learning signal comes from, and by what each can and cannot promise.

Regression, classification, ranking, clustering, dimensionality reduction and anomaly detection — what each one outputs, and why a visually separated cluster is not a business segment.

Filtering, joining, labelling and feature creation each introduce bias or leakage. What one training example represents, and how sampling decides what the model can learn.

Train, validation and test as three different jobs, and the split strategy — random, temporal, grouped, stratified — as the decision that decides whether the metric means anything.

The deepest module. Target, temporal, entity, preprocessing, feature and evaluation leakage: every way information from the answer reaches the model, and why each makes offline metrics lie.

Aggregation, bucketing, normalisation, encoding, temporal and interaction features — each a transformation that must be reproduced identically at serving time.

Hand-engineered features against learned representations, feature selection, and importance methods — with the warning every one of them needs: importance is not causality.

Rule, mean predictor, majority class, linear model, simple tree. Mandatory before anything complex, because a model that does not beat a useful baseline has not earned its cost.

Linear and logistic regression as the models everything else is compared to: coefficients, residuals, the sigmoid, and the threshold that turns a probability into a decision.

The confusion matrix and everything derived from it — precision, recall, F1, ROC AUC, PR AUC, calibration — each with its business reading and the case where it misleads.

MSE, RMSE, MAE, R² and the caveats on MAPE: what each punishes, what each hides, and how to choose one from the cost of being wrong.

Business metrics against model metrics, offline against online, cross-validation and its temporal variant, slices — and the rule that the test set is touched once.

Underfitting, overfitting, learning curves, regularisation and early stopping — the mechanics of why a model that memorised the training set looks perfect until it meets new data.

Decision trees, random forests and gradient boosting — how each learns, why boosting fits residuals, and XGBoost and LightGBM as implementations rather than as answers.

Three mental models, their assumptions, strengths and limits — taught as ways of thinking about data rather than as API calls.

The neuron, activations, the forward pass, loss functions and backpropagation on a computational graph — the mechanism every deep learning framework hides.

Gradient descent, SGD, momentum and Adam; epochs, batches and steps; learning rate and batch size trade-offs; vanishing and exploding gradients; normalisation.

Discrete entities as dense vectors — words, users, products, documents — cosine similarity, and why a 2D projection distorts the geometry it claims to show.

Convolutions for spatial structure, recurrent models for sequences, and transformers: tokens, embeddings, self-attention, positional information.

Encoder and decoder families, pretrained models reused across tasks, transfer learning, fine-tuning and parameter-efficient adaptation — at the level of what changes and what it costs.

Hyperparameters against learned parameters; grid, random and Bayesian search; the budget; and the rule that nothing is ever tuned on the test set.

Forecasting, trend and seasonality, horizon, anomaly detection — and validation that respects time, because a random split on temporal data is leakage.

Candidate generation and ranking, collaborative and content-based filtering, cold start — and feedback loops, where the model changes the data it will be trained on next.

What every run must record, why random seeds alone do not reproduce anything, and versioning datasets, labels, features and models so lineage can be traced.

What a model artifact contains, the registry lifecycle from candidate to archived, and promotion judged on quality, latency, memory, cost and robustness rather than one offline score.

Batch, online and streaming inference, how to choose between them from freshness, latency and volume, the serving architecture, batching, and CPU against GPU.

Identical weights can fail if the features differ. Skew, feature stores as optional infrastructure, point-in-time correctness, freshness, latency breakdown and fallbacks.

Parallel compute, matrix operations, memory bandwidth and VRAM; quantization from FP32 to INT8; pruning and distillation; and what inference actually costs.

Data, model, tensor and pipeline parallelism, gradient synchronisation with all-reduce, checkpointing for recovery, and the cost of a training run.

A dedicated stack: data and feature tests, training smoke tests, model invariants, serving contracts, robustness — because a green unit test suite says nothing about a model.

Subgroup performance with no universal fairness metric, explanations that are approximate, privacy as design, and the line between predicting Y and causing it.

Data, feature, prediction and concept drift taught separately; drift that is not failure; ground truth that arrives weeks late; and performance decay diagnosed rather than assumed.

Retraining as a decision rather than a schedule, champion/challenger, shadow, canary, A/B — and rollback and fallback, which every serving system needs before it needs a second model.

Tracing one request through features, model version, prediction, decision and outcome; logging what is necessary and safe; and debugging an incident from the business metric down.

The practices that make ML reproducible, testable, deployable and observable — CI, CT and CD distinguished, platform capabilities, cloud primitives before vendors, cost.

Poisoning, artifact integrity, sensitive-data leakage, adversarial inputs, supply chain and inference abuse — defensively, and at the level of what to validate and trust.

The questions to ask before drawing boxes, then recommendation, fraud, churn and search ranking designed end to end — and the boundary with Agentic Engineering.