0110 min
Read the slope
Locate w=0 and reason about the direction of decreasing loss.
Ask: “Why subtract the gradient here?”
Listen for: “The negative gradient points toward lower local loss.”
Brightlab
brightlab.Made for curious mindsbrightlab.Made for curious mindsBrightlab
Year 9 · Optimisation
Why can trying harder move a model further from its goal?
Go to the investigation ↓An optimiser notebook page with numeric updates, traces and a stopping policy.
Take this investigation into Python: open the interactive Marimo notebook →
Can your own update schedule reach a low loss?
Gradient descent updates a parameter by subtracting learning rate times the loss gradient. For this one-parameter quadratic, loss is (w−3)² and gradient is 2(w−3). Small steps converge slowly; suitable steps converge quickly; overly large steps oscillate or diverge. The simple surface makes each update inspectable. Real neural losses can be non-convex, noisy and much harder to optimise.
Before this lesson: Evaluate a quadratic and follow a repeated numeric update. Useful earlier investigations: A signal with weight behind it.
Gradient descent updates a parameter by subtracting learning rate times the loss gradient. For this one-parameter quadratic, loss is (w−3)² and gradient is 2(w−3). Small steps converge slowly; suitable steps converge quickly; overly large steps oscillate or diverge. The simple surface makes each update inspectable. Real neural losses can be non-convex, noisy and much harder to optimise.
Evaluate a quadratic and follow a repeated numeric update. Useful earlier investigations: A signal with weight behind it.
Work through w=0 with learning rates 0.1, 0.5 and 1.1. Prepare a graph of the quadratic. Explain that an oversize parameter triggers an explicit numerical-stop state rather than silently clipping the result.
ACARA V9 AI curriculum connection ↗ · Technologies ↗
A complete teaching sequence · 80 minutes
Locate w=0 and reason about the direction of decreasing loss.
Ask: “Why subtract the gradient here?”
Listen for: “The negative gradient points toward lower local loss.”
Calculate updates for a chosen rate before running.
Ask: “Will the next point cross the minimum?”
Listen for: “That depends on the size of the update.”
Run individual updates, inspect arrows and loss history, then reset and compare equal-step runs at different rates.
Ask: “Is a larger movement the same as more learning?”
Listen for: “No, it can increase the loss.”
Set learning rate 1 on the quadratic and observe repeated equal-distance jumps.
Ask: “Why does the loss fail to decrease?”
Listen for: “Each step overshoots by the same distance.”
Choose a fixed or decaying rate, state a stopping criterion and compare with the baseline from the same start.
Ask: “What evidence tells you to stop rather than take another step?”
Listen for: “A small gradient or sufficiently small loss change.”
Submit two calculated updates and stable/unstable traces.
Ask: “Which part of this result depends on the simple quadratic?”
Listen for: “The exact safe learning-rate range and guaranteed shape.”
Bigger steps always learn faster.
At learning rate 1, this quadratic oscillates without improvement; above 1, the distance from the minimum grows.
Specify a stable update schedule and stopping rule, then compare equal-budget traces from the same initial parameter.
Check arithmetic for two updates and interpretation of increasing loss. Ask learners to separate optimisation success from generalisation.
| Criterion | Beginning | Secure | Extending |
|---|---|---|---|
| Update mechanics | Moves toward a minimum by intuition | Calculates gradient and update | Predicts oscillation and divergence |
| Experiment design | Compares unequal starts or budgets | Uses matched starts and steps | Justifies stopping and limits transfer to complex losses |
Use the quadratic and whole-number starting parameters; provide gradient values before asking for differentiation.
Add a second parameter with a steeper curvature and investigate why one learning rate can be unstable along one axis.
All loss functions are synthetic. Divergent computations are halted explicitly and no external training job is launched.
Shared device? Turn remembering off. A project file lets you continue on another device.
Your browser is the laboratory
Calculate two updates from w=0 at a chosen learning rate before stepping the optimiser.
Start at w=0. Loss is (w−3)² and gradient is 2(w−3). Update: w ← w − learning rate × gradient. Compare rates 0.1, 0.5, 1 and 1.1 from the same start.
The experiment opens after your prediction.
There is no penalty for being surprised.
Apple Silicon · PyTorch MPS
Optimise many quadratic surfaces with different curvature on MPS/CUDA, recording stability regions and synchronised timing rather than assuming GPU speedups.
Students extend the experiment in teams, documenting parameters, outputs and limitations.
The bundle contains lesson-specific working code, a configuration file, a reactive notebook, a deterministic CPU check and hardware setup instructions. Acceleration is reported only after a tensor operation and result read-back succeed.
Download Mac Studio investigation ↓unzip y9-gradient-mac-pathway.zip -d y9-gradient-mac cd y9-gradient-mac bash setup-mac.sh source .venv/bin/activate python experiment.py --device mps --output results marimo edit notebook.py
CPU and available-device execution status is recorded in the downloaded README and validation report. DGX Spark execution requires that hardware; static validation alone does not establish GPU compatibility or performance. The browser lesson remains fully available without this extension.
NVIDIA DGX Spark · PyTorch CUDA
Optimise many quadratic surfaces with different curvature on MPS/CUDA, recording stability regions and synchronised timing rather than assuming GPU speedups.
Students extend the experiment in teams, documenting parameters, outputs and limitations.
The bundle contains lesson-specific working code, a configuration file, a reactive notebook, a deterministic CPU check and hardware setup instructions. Acceleration is reported only after a tensor operation and result read-back succeed.
Download DGX Spark investigation ↓unzip y9-gradient-dgx-pathway.zip -d y9-gradient-dgx cd y9-gradient-dgx bash run-dgx.sh # Open the localhost notebook URL printed by the container.
CPU and available-device execution status is recorded in the downloaded README and validation report. DGX Spark execution requires that hardware; static validation alone does not establish GPU compatibility or performance. The browser lesson remains fully available without this extension.
An optimiser notebook page with numeric updates, traces and a stopping policy.
Download the editable handout →