Learning Paradigms

Supervised, unsupervised, semi-supervised and self-supervised — distinguished by where the learning signal comes from, and by what each can and cannot promise.

Supervised Learning

A labelled target turns learning into function fitting. The model is only as right as the label, and the label was made by a process nobody wrote down.

Q · When someone hands you a table with a target column, what exactly has the model been told to learn — and who decided it?
Unsupervised Learning

No labels, so no loss against the truth. The model finds structure in whatever the features and the distance say — and nobody checked that those mean anything to the business.

Q · The data has no target column, so you want the model to "find the structure". Structure according to what, and how would you know it found the wrong one?
Semi-Supervised Learning

A few thousand labels and a few million unlabelled rows. The unlabelled data helps exactly when it comes from the same distribution as the labels — and that is the thing you cannot check with labels.

Q · Labels are scarce and unlabelled data is abundant. Under what conditions does the unlabelled data make the model better rather than more confidently wrong?
Self-Supervised Learning

The data labels itself: hide part of it and predict it back. The signal is free and abundant, which is why it works — and why the model learns whatever the corpus contains, including what you did not want.

Q · If the model is trained to reconstruct its own input, what has it learned, and why would that help with a task the pretext never mentioned?
The Learning Signal

Every paradigm is defined by where the gradient's target comes from. That source decides what the model can be wrong about without anyone noticing.

Q · If you strip away the algorithms, what actually distinguishes supervised, unsupervised, semi-supervised and self-supervised — and what does that tell you to monitor?
Choosing a Paradigm
▶ lab

Have labels? Can you get them? What do they cost, and how long until they arrive? What does "structure" mean to the business? The paradigm is the answer to those questions, not a preference.

Q · A team wants to "use ML" on a new problem. Which questions about labels, cost and business meaning decide whether it is supervised, unsupervised, semi-supervised or self-supervised — and when is the answer none of them?