Explain backpropagation in one sentence. Then: the loss went to NaN at step 300. What happened?
Answer it out loud before you open anything. The value of the flags below is in comparing them to what you actually said — including whether you asked about the data before naming a model.
A team is fine-tuning a small transformer for intent classification. Training looks fine for a few hundred steps, then loss spikes and becomes NaN. The engineer's first fix was to restart with a different seed, which worked once and then failed again at a different step.
React to this
Say what you would question, what you would trust, and what you would need to know first.
train.log (excerpt, illustrative) step 280 loss 0.912 grad_norm 1.3 lr 3e-4 step 290 loss 0.884 grad_norm 1.1 lr 3e-4 step 296 loss 0.901 grad_norm 2.7 lr 3e-4 step 298 loss 1.640 grad_norm 41.0 lr 3e-4 step 299 loss 9.831 grad_norm 3.2e5 lr 3e-4 step 300 loss nan grad_norm nan lr 3e-4 config: AdamW, no warmup, fp16 autocast, no grad clipping, batch 64, max_len 256
What it is really testing
Whether the candidate can state the mechanism compactly — the chain rule applied backwards through a computational graph — and then use it: a NaN is a numerical event with a small set of causes, most of which are visible from the mechanism, and "try another seed" is not a diagnosis.