The Loss Stopped Moving at 2.3
Decide what you would do from the brief alone, including whether you would change anything at all. Everything below it is available, but the exercise stops working if you open it first.
A team training a product-category classifier reports that training loss drops quickly to about 2.3 and then stays there for the rest of training regardless of epochs. Validation accuracy sits at the majority-class rate. The proposal is to "train longer and add layers".
Adding layers and training for ten times as many epochs. The run is longer and the architecture is more impressive, and the loss still sits at 2.3, because the fixed point does not depend on depth. The next step in that direction — switching to a bigger pretrained backbone — also sits at 2.3, at more expense, and the conclusion becomes "the problem is hard". The whole time, a learning rate one order of magnitude lower, or a data loader that was shuffling labels separately from inputs, was the answer, and a ten-example overfit test would have found it in a minute.
Read this even if you are confident. It is here rather than behind a button because it is the answer most teams actually ship, it passes review, and its cost arrives weeks later when the labels do.