Linear Models
Linear and logistic regression as the models everything else is compared to: coefficients, residuals, the sigmoid, and the threshold that turns a probability into a decision.
ŷ = w·x + b. A weight per feature, a bias, a loss that says which mistakes hurt — and a set of assumptions the weights only make sense under.
The residual plot is the diagnostic; the single metric is the summary. Heteroscedasticity, extrapolation and unscaled coefficients are all visible there and invisible in the RMSE.
Linear score → sigmoid → probability, trained by gradient descent on the log loss. The threshold that turns the probability into a decision is a different step, owned by someone else.
The sigmoid turns a score into a number between 0 and 1. Whether that number is a probability is a fact about calibration on the deployment distribution, not about the function.
The threshold is not part of the model. It is the point where a business decision about costs and capacity is written down, and it deserves an owner, a config and a review.
L1 makes weights zero, L2 makes them small, elastic net does both — and all of them penalise a feature in proportion to its scale, so the scaler is part of the model.