Backpropagation
Forward pass → loss → backward pass → gradients → parameter update. The chain rule applied node by node, in reverse topological order, on the 2-2-1 network the lab runs.
The problem, the obvious approach, and why it breaks
Every lesson starts where the work starts: someone has a problem, and the first model that comes to mind looks fine offline.
How does a network compute the gradient of one loss with respect to every parameter in a single backward sweep, and what can go wrong in that sweep that the loss curve will not show?
An engineer inheriting a training pipeline: "The framework hides all of it. I need to know what loss.backward() actually does, because our gradients are NaN on one batch in a thousand and nobody can explain why."
Differentiate the loss as a function of all the weights by hand, or numerically by perturbing each parameter and re-running the forward pass. For nine parameters either works; call the framework for anything larger and trust it.
Numerical differentiation costs one forward pass per parameter. For nine parameters that is nine passes; for a million it is a million passes per step. It is a test of the gradient, not a way to compute it.
- Numerical differentiation costs one forward pass per parameter. For nine parameters that is nine passes; for a million it is a million passes per step. It is a test of the gradient, not a way to compute it.
- A hand-derived closed form for the whole network is fragile: change one activation and the entire expression changes. The framework does not derive a closed form; it composes local derivatives, and understanding that is what explains the NaN.
- The NaN comes from a node whose local derivative is undefined or overflowing on an extreme value — log of a probability that rounded to zero, or a sigmoid derivative on a logit so large that intermediate products overflow — and the backward sweep multiplies it into every upstream gradient. Trusting the framework meant not looking at the node.
- A learning rate tuned on the loss curve was pushing one hidden unit's pre-activation permanently negative on some initialisations; the ReLU's local derivative went to zero, the unit stopped receiving gradient, and the network trained to a worse minimum. The loss curve looked like "slow convergence".
What is being predicted, and from what data
This domain leads with these two. A target nobody defined precisely is a label nobody can trust, and a dataset nobody can describe is a model nobody can debug.
- The surrounding system predicts whether a two-feature input belongs to the positive class — the XOR-shaped toy the lab uses — with a sigmoid output and binary cross-entropy. The label is a 0/1 class.
- The engineering target of the lesson is the gradient itself: the vector of partial derivatives of the loss with respect to every weight and bias, computed exactly, once per step.
- One example is a pair (x₁, x₂) with a label y. The lab's network has two inputs, two ReLU hidden units, one sigmoid output: nine parameters in total — four weights and two biases in the first layer, two weights and one bias in the second.
- The pipeline in the problem statement trains a much larger network on real data; the NaN appears in the backward pass, not the forward one, on a batch containing an extreme input.
How it actually works
Precisely enough to predict its behaviour — not a framework API.
- The forward pass computes every intermediate value in topological order and keeps it: x → z₁, z₂ → h₁, h₂ → z₃ → ŷ → L. The backward pass walks the same order reversed, and at each node computes ∂L/∂node from the gradients of the nodes that consume it, times the local derivative of the consuming operation. That is the chain rule, applied one edge at a time.
- For the 2-2-1 network: ∂L/∂ŷ = −y/ŷ + (1 − y)/(1 − ŷ); through the sigmoid, ∂L/∂z₃ = ŷ − y; the output weights get ∂L/∂w₂ⱼ = (ŷ − y)·hⱼ and ∂L/∂b₂ = ŷ − y; each hidden unit receives ∂L/∂hⱼ = (ŷ − y)·w₂ⱼ; through the ReLU, ∂L/∂zⱼ = ∂L/∂hⱼ · [zⱼ > 0]; and the first-layer weights get ∂L/∂w₁ⱼᵢ = ∂L/∂zⱼ · xᵢ and ∂L/∂b₁ⱼ = ∂L/∂zⱼ.
- Every node needs only its own forward value, its local derivative, and the gradient handed back from its consumers. No node knows the whole network. That locality is why the same procedure works for a million parameters, and why a framework can build it automatically from the operations you called — reverse-mode automatic differentiation (Computational Graphs).
- The update is then θ ← θ − η · ∂L/∂θ for every parameter, with η the learning rate or an optimiser's adaptive version of it. Backpropagation computes the gradient; it does not decide what to do with it (Gradient Descent, Optimisers: SGD, Momentum, Adam).
Forward, then backward along the same graph
The network the lab runs is small enough to draw in full: two inputs, two pre-activations, two ReLU outputs, one output pre-activation, one sigmoid, one loss. Nine parameters. The forward pass fills in every node's value from left to right; the backward pass fills in every node's gradient from right to left, and the parameter gradients fall out at the edges where a weight multiplies a value.
The pipeline is the five-step loop the module is named for: forward pass, loss, backward pass, gradients, parameter update. One step of the lab is exactly one iteration of it.
- 1Forward pass
Compute every node's value in topological order and keep them all.
fails by An input outside the expected range pushes a sigmoid into saturation or a probability to exactly zero.
- 2Loss
Compare ŷ with y through the loss; a single scalar.
fails by log(0) without a clamp; a loss that is a proxy for the wrong objective.
- 3Backward pass
Walk the nodes in reverse order, handing each node's gradient to its inputs via the local derivative.
fails by A wrong local derivative in a custom layer; an overflow at one node propagates as NaN to every upstream parameter.
- 4Gradients
Read ∂L/∂θ off the edges where a parameter multiplies a value.
fails by A ReLU that is never active hands back zero; its parameters never move.
- 5Parameter update
θ ← θ − η · ∂L/∂θ, or the optimiser's version of it.
fails by A learning rate that overshoots and kills units on the next forward pass.
Every gradient in the 2-2-1 network
Written out, the backward pass is a short list. Each line uses only the forward value of the node it belongs to, the local derivative of the operation, and the gradient already computed for the node downstream. The indicator [zⱼ > 0] is the ReLU's derivative; it is where a dead unit stops the flow; and ŷ − y is the sigmoid-plus-cross-entropy pairing collapsing into the model's error.
Compare these with the lab: every node on the page shows its forward value and its ∂L/∂node, and the parameter gradients in the table are the last four lines below evaluated on the current input.
1import numpy as np2 3def forward_backward(p, x, y):4 w1, b1, w2, b2 = p5 z = w1 @ x + b1; h = np.maximum(0, z)6 z3 = w2 @ h + b2; yhat = 1 / (1 + np.exp(-z3))7 eps = 1e-12; pc = np.clip(yhat, eps, 1 - eps)8 L = -(y * np.log(pc) + (1 - y) * np.log(1 - pc))9 # backward, one edge at a time10 dz3 = yhat - y11 dw2, db2 = dz3 * h, dz312 dh = dz3 * w213 dz = dh * (z > 0)14 dw1, db1 = np.outer(dz, x), dz15 return L, (dw1, db1, dw2, db2)16 17def numerical_grad(p, x, y, i, idx, eps=1e-5):18 # perturb one parameter, re-run forward: the only independent proof the sweep is right19 plus = [q.copy() for q in p]; plus[i][idx] += eps20 minus = [q.copy() for q in p]; minus[i][idx] -= eps21 return (forward_backward(plus, x, y)[0] - forward_backward(minus, x, y)[0]) / (2 * eps)The analytic sweep is five lines. The check costs two forward passes per parameter — fine for nine, impossible for a million — which is exactly why reverse mode exists and why the check is a test rather than a method.
forward: z1 = w1[0][0]*x1 + w1[0][1]*x2 + b1[0] h1 = max(0, z1) z2 = w1[1][0]*x1 + w1[1][1]*x2 + b1[1] h2 = max(0, z2) z3 = w2[0]*h1 + w2[1]*h2 + b2 yhat = 1 / (1 + exp(-z3)) L = -( y*log(yhat) + (1-y)*log(1-yhat) ) backward (reverse topological order): dL/dyhat = -y/yhat + (1-y)/(1-yhat) dL/dz3 = dL/dyhat * yhat*(1-yhat) = yhat - y # sigmoid + BCE collapse dL/dw2[j] = dL/dz3 * h[j] # j = 0, 1 dL/db2 = dL/dz3 dL/dh[j] = dL/dz3 * w2[j] dL/dz[j] = dL/dh[j] * (1 if z[j] > 0 else 0) # ReLU: dead unit => 0 from here up dL/dw1[j][i] = dL/dz[j] * x[i] # i = 0, 1 dL/db1[j] = dL/dz[j] update: every parameter theta <- theta - lr * dL/dtheta
Where the sweep breaks, and the NaN
The one-in-a-thousand NaN is a node. Follow the sweep: the loss takes log(ŷ) and log(1 − ŷ); on an extreme input the sigmoid rounds ŷ to exactly 1.0 in floating point, log(1 − ŷ) is log(0), and the backward pass multiplies −∞ by a zero derivative somewhere upstream to produce NaN, which then reaches every parameter. Clamping ŷ, or computing the loss from z₃ directly, removes the node that fails.
The dead unit is the other break, and it is silent: [zⱼ > 0] is zero on every example, every upstream gradient through that unit is zero, and the unit's four parameters never move again. The loss keeps falling using the other unit, and the network converges to something worse. Per-unit gradient statistics see it in the first epoch; the loss curve never does.
On the inputs the pipeline sees, no node's local derivative is undefined or overflows, and each hidden unit receives a non-zero gradient on a meaningful fraction of examples.
holds when The loss clamps probabilities or works from logits; inputs are standardised so pre-activations stay in range; initialisation and learning rate keep the ReLUs active; custom layers pass a finite-difference check.
breaks when A corrupt or extreme input reaches a log or a division; mixed precision overflows an intermediate; an aggressive learning rate drives a unit's bias permanently negative; a custom backward function is edited without re-running the check.
respond Locate the node from the norms, fix the numerics or the input validation there, and re-initialise dead units rather than lowering the learning rate for the whole network.
How to build it
Most important first.
- Check the analytic gradient against finite differences on a small network before trusting a custom layer or loss; the lab's model does this for every parameter, and it is the only proof the graph is wired correctly.
- Clamp probabilities away from zero and one inside the loss, and compute cross-entropy from the logit rather than from the sigmoid output where possible, so log and division never see a rounded value (Loss Functions).
- Record gradient norms per layer and the count of NaN or infinite gradients per step; a NaN in one batch in a thousand is a specific input reaching a specific node, and the norms locate it.
- Clip the gradient norm as a safety net against a single extreme batch, and treat the clip firing as a signal to inspect the batch rather than as a fix (Vanishing and Exploding Gradients).
- Initialise so that pre-activations start where local derivatives are meaningful — ReLUs active on a reasonable fraction of inputs, sigmoids away from saturation (Initialisation and Convergence).
What to measure
Which number actually maps to the decision — and which numbers look relevant and are not.
- The maximum relative difference between analytic and numerical gradients on a small network or a single custom layer — the correctness test.
- Gradient norm per layer per step, and the rate of non-finite gradients; these locate the node where the backward sweep breaks.
- The fraction of ReLU units receiving any gradient over an epoch — a unit at zero is not training.
- Do not measure "the loss is decreasing" as evidence that backpropagation is correct. A wrong gradient that is merely correlated with the right one still decreases the loss, slowly, and wastes the run.
What must stay true after deployment
The field this whole domain exists for. A model is a set of assumptions with weights attached; these are the ones a monitor or a test should be checking.
- Every operation in the graph has a correct local derivative — verified for custom layers by finite differences, assumed for framework primitives.
- The forward values stored for the backward pass are the ones the backward pass uses, in the same precision; a change to the forward computation is a change to the gradient.
- Inputs at training time stay within a range where no node's local derivative overflows or becomes undefined, or the loss and the clip handle the exceptions.
- The hidden units are receiving gradient on a meaningful fraction of examples; a unit at zero gradient will not recover on its own.
- Offline: a gradient check on a tiny network or a single layer as a unit test, with a tolerance; the lab's model asserts analytic and numerical agreement for every parameter (Training Smoke Tests).
- During training: per-layer gradient norms, non-finite counts, and dead-unit fractions as logged metrics with thresholds that fail the run.
- Over time: a smoke test on every code change that trains a few steps on a fixed batch and asserts the loss falls and the gradients are finite, so a broken backward function cannot reach a full run unnoticed.
What can go wrong
- The clamp inside the loss hides an input that should have been rejected upstream; the run finishes and the model has learned something from a corrupt row.
- Gradient clipping is set tight enough to fire on most steps; the effective learning rate is now the clip value and the optimiser's schedule is meaningless.
- A custom layer's backward function is subtly wrong, the finite-difference check was never run, and the model trains to a mediocre result that is blamed on the data.
- Mixed-precision training overflows an intermediate gradient in half precision on rare batches; the loss scaler handles most of them and the rest are the NaN.
- Reverse-mode differentiation computes every gradient in one backward sweep at the cost of storing every forward activation until it is used; that memory is the price of the efficiency (Computational Graphs).
- Clamping, clipping and loss scaling make the sweep robust to extreme values and each one hides something: a corrupt input, an effective learning rate, an overflow.
- A finite-difference check is slow and only feasible on tiny networks; it verifies the mechanism, not the run.
- "The framework handles the gradients; I do not need to know how." Until a batch produces NaN, a custom layer trains slowly, or a unit dies, and the only way to reason about it is node by node.
- "Backpropagation is the training algorithm." It computes the gradient. Gradient descent and the optimiser decide what to do with it; the loss decides what it is a gradient of.
- "The loss went down, so the gradient is correct." A gradient with the right sign and the wrong magnitude in some layers still decreases the loss, and wastes most of the run doing it.
Where this applies
ML advice is stated as universal far more often than it is. These labels say what each claim is specific to — and where CONTESTED appears, the note gives the strongest form of the opposing view.
- GENERALThe chain rule in reverse topological order is how every differentiable model is trained, from a logistic regression to a transformer; only the nodes and their local derivatives change.
- SIMPLIFIEDThe 2-2-1 network is the lab's teaching graph: one example at a time, no batch, no optimiser beyond plain descent, no normalisation. The gradients written out are exact for that graph; a real network adds a batch axis that averages them and layers whose local derivatives are matrices.
- SIMULATEDThe numbers on the lab page come from the model in
src/ml/sim/backprop.ts, which computes forward values and gradients on the seeded 2-2-1 graph and checks them against finite differences; they are for the shape of the argument, not a measurement on any dataset.
Where the depth lives
This domain teaches the model and hands the rest off by name.