In Fitting a Line we could solve for the best w with a formula. Most models have no formula, so training must search. Picture the loss as a landscape: each setting of the model is a position, and the loss is the height there. Training means finding low ground.
To keep it drawable we use one knob, w, and a made-up loss: loss = (w - 3) squared. The bottom of this bowl is at w = 3, where the loss is 0. We start at w = 0, where the loss is (0 - 3) squared = 9. The trainer cannot see the whole bowl, because every look at a position costs a pass over the data.
💡Good to know
For a line with two knobs, w and b, the bowl is a 3D surface. One knob keeps it to a curve we can draw.