SIMULATED

Decision Tree Visualizer

Grow a CART tree one level at a time on two-dimensional points and watch the boundary it draws. Training accuracy can only rise with depth; test accuracy rises, peaks and then falls — and the depth at which it falls depends on how noisy the labels are, not on the depth itself.

Problem→Target→Data→Representation→Split→Model→Training→Evaluation→Validation→Deployment→Inference→Monitoring→Drift→Retraining

A tree is a greedy search for the axis-aligned split that most reduces impurity, applied recursively. That is the whole algorithm, and it is enough to reproduce overfitting without any hand-waving: each extra level lets the tree carve out a region around a single mislabelled point, which raises training accuracy by one point and lowers test accuracy by however many future points fall in that region. Move the depth slider and read the two accuracies together. The gap between them is the number this lab exists to make you distrust.

Controls

Dataset, depth, label noise and the minimum leaf size. Change one at a time.

Class by which side of the line y = x the point falls on, with a fraction of labels flipped. Axis-aligned splits approximate the diagonal with a staircase, and every flipped label is a temptation to add a step that only fits noise.

Max depth · 3
Label noise (fraction flipped)
Min samples to split

The boundary and the tree

Filled points are the training draw; hollow rings (if shown) are the held-out draw. Each translucent rectangle is one leaf, coloured by what it predicts.

class 0 class 1
depth
3
leaves
8
train acc
84.0%
test acc
75.5%
The tree it grew
  • y < -0.66n=200 · gini=0.493 · predicts 0
    • x < -1.97n=89 · gini=0.280 · predicts 0
      • y < -2.57n=15 · gini=0.498 · predicts 0
        • leaf → class 0n=3 · gini=0.000 · predicts 0
        • leaf → class 1n=12 · gini=0.486 · predicts 1
      • y < -1.13n=74 · gini=0.193 · predicts 0
        • leaf → class 0n=58 · gini=0.128 · predicts 0
        • leaf → class 0n=16 · gini=0.375 · predicts 0
    • x < 1.54n=111 · gini=0.450 · predicts 1
      • y < 1.66n=80 · gini=0.320 · predicts 1
        • leaf → class 1n=59 · gini=0.395 · predicts 1
        • leaf → class 1n=21 · gini=0.000 · predicts 1
      • y < 2.11n=31 · gini=0.412 · predicts 0
        • leaf → class 0n=21 · gini=0.091 · predicts 0
        • leaf → class 1n=10 · gini=0.320 · predicts 1

Accuracy by depth

The same dataset, trees of every depth from 0 to 10, scored on the training draw and on a second draw the tree never saw.

47%74%100%012345678910traintest
Best test depth
Depth 3 scores 75.5% on held-out data. At depth 10 the same tree scores 64.5% on held-out data and 100.0% on the training draw.
What the gap means
Training accuracy is non-decreasing with depth by construction — a deeper tree can only refine a shallower one. Every point of training accuracy gained past depth 3 was bought by memorising a label the next draw does not share. Note that the curve uses the default minimum leaf size; the min-samples control above is the cheapest regulariser and moves the current tree only.

How to read this page honestly

What the model is, and what it deliberately refuses to be.

  • SIMULATEDThe tree is actually grown: CART with Gini impurity, every candidate midpoint scanned, the split with the lowest weighted impurity taken. Nothing about the boundary is drawn by hand. The points are synthetic draws from a seeded generator, and the test set is a second draw from the same distribution — so "test accuracy" is a real held-out measurement of a real model on data no real business produced.
  • SIMPLIFIEDTwo features, two classes, two hundred points, no pruning beyond depth and a minimum leaf size. Real trees prune by cost-complexity, handle categoricals and missing values, and are almost always used inside a forest or a boosting ensemble — where the overfitting shown here is what averaging exists to cancel.
  • DATA-SPECIFICDepth is not the villain. Pick *Two blobs* with no noise and go to depth 10: test accuracy barely moves, because there is nothing to memorise. The lesson is that depth overfits noisy labels, and the noise control is the honest half of the lab.

The lessons behind it

Decision Trees →How a Tree Chooses a Split →Overfitting →Random Forests →