Brightlab
brightlab.Made for curious mindsbrightlab.Made for curious mindsBrightlab
How does a network know which way to change its weights?
Training scores a network with a loss. The derivative says which way is downhill, and gradient descent takes a step that way. The chain rule, run backwards through the network, finds that slope for every weight at once: backpropagation.
Where this happens: The backward pass: a gradient for every weight.
Starting knowledge: The gradient of a straight line. Some experience of derivatives helps; the chain rule is shown with numbers first.
The animations are silent; captions carry the explanation. Pause whenever a caption changes and say what you expect next.
A loss curve for one weight. The tangent’s slope says which way is downhill; gradient descent steps against it. Three learning rates: slow, fast and too large.
One neuron drawn as a chain: multiply, add, squash with σ, score with (a − y)². Values flow forwards; local slopes are multiplied backwards to give ∂L/∂w.
Choose a loss curve, a starting weight and a learning rate, then step. Every step shows the slope and the update. Below it, check the chain rule for a single neuron against a measured nudge.
Fitting a line ŷ = w·x + b to five points. The loss is a landscape over (w, b) with elliptical contour rings; gradient descent crosses the rings to the best line.
Counterexample challenge. Find a learning rate that makes the one-bowl loss bounce between two weights forever, neither settling nor escaping.
A table of learning rates and outcomes; a hand-worked ∂L/∂w for one neuron that agrees with the lab’s nudge check; a written explanation of what the sign of a gradient tells you.
The one-weight curves and the single neuron are chosen so every number can be checked by hand. Real training estimates gradients from batches of examples, updates millions of weights and usually uses a refined optimiser such as Adam, but each update is still built from these derivatives.