018 min
Compare the same object
Show its old and new readings without changing its label.
Ask: “Did the object type change, or its measurement?”
Listen for: “Its measurement changed.”
Brightlab
brightlab.Made for curious mindsbrightlab.Made for curious mindsBrightlab
Year 4 · Robustness
Why can an unchanged model stop working?
Go to the investigation ↓A sensor calibration field note with paired readings and residual errors.
Will yesterday’s rule still work in today’s light?
Our classifier was fitted to synthetic readings under one light condition. A new light condition adds an offset to measurements without changing object labels. The frozen model now sees different numbers. Correcting a known sensor offset can restore some predictions, but a single correction does not fix every kind of environmental change.
Before this lesson: Keep a test rule fixed while changing one environmental condition. Useful earlier investigations: What the sensor missed; The sealed test envelope.
Our classifier was fitted to synthetic readings under one light condition. A new light condition adds an offset to measurements without changing object labels. The frozen model now sees different numbers. Correcting a known sensor offset can restore some predictions, but a single correction does not fix every kind of environmental change.
Keep a test rule fixed while changing one environmental condition. Useful earlier investigations: What the sensor missed; The sealed test envelope.
Prepare paired readings for the same synthetic objects under two light settings. Mark the original training threshold. Explain that subtracting a known offset is a simplified calibration procedure.
ACARA V9 AI curriculum connection ↗ · Technologies ↗
A complete teaching sequence · 60 minutes
Show its old and new readings without changing its label.
Ask: “Did the object type change, or its measurement?”
Listen for: “Its measurement changed.”
Predict which side of the frozen threshold an offset object reaches.
Ask: “Which parts of the system are frozen?”
Listen for: “The model and the true object labels.”
Move the lighting offset and watch paired readings and error counts update. Save baseline and shifted states.
Ask: “Which errors appeared first as the offset increased?”
Listen for: “Objects closest to the old threshold.”
Use the edge case in which bright objects shift more than dark ones.
Ask: “Can subtracting one number fix unequal shifts?”
Listen for: “No, some objects need a different correction.”
Estimate an offset from a known reference, apply it and evaluate every fixed object. State what the correction assumes.
Ask: “What evidence would tell us to stop using this correction?”
Listen for: “Different residual errors across the range.”
Record condition, correction, error counts and limitation.
Ask: “Was yesterday’s test wrong?”
Listen for: “It described yesterday’s conditions, not every future one.”
A model that worked yesterday will work tomorrow.
A brightness-dependent change cannot be removed by subtracting a single constant offset.
Calibrate against a reference and report residual errors under uniform and non-uniform shifts.
Ask which quantity changed and which remained fixed. If a learner says the model learned from the shifted objects, point out that the model parameters were frozen.
| Criterion | Beginning | Secure | Extending |
|---|---|---|---|
| Shift diagnosis | Calls every error a bad model | Identifies changed input measurements | Distinguishes uniform and non-uniform shift |
| Correction evidence | Tunes until one object works | Checks all fixed objects | States calibration assumptions and failure conditions |
Use three objects and an integer offset; slide a paper strip beneath a fixed threshold.
Add a separate scale correction and test whether two reference points resolve an affine shift.
The lagoon data are invented. Do not use the model for environmental monitoring or decisions about swimming conditions.
Shared device? Turn remembering off. A project file lets you continue on another device.
Your browser is the laboratory
Add a positive lighting offset with the classifier frozen. Predict which labelled objects cross its boundary.
Original readings are 15,25,35,42,48,54,62,73,82,92. The frozen classifier splits at 50. Add a lighting offset to every reading while keeping the reference labels unchanged.
The experiment opens after your prediction.
There is no penalty for being surprised.
Apple Silicon · PyTorch MPS
Apply uniform and value-dependent shifts to a frozen model over thousands of measurements. Sweep offsets, control the correction, and compare raw and corrected error rates.
Students extend the experiment in teams, documenting parameters, outputs and limitations.
The bundle contains lesson-specific working code, a configuration file, a reactive notebook, a deterministic CPU check and hardware setup instructions. Acceleration is reported only after a tensor operation and result read-back succeed.
Download Mac Studio investigation ↓unzip y4-shift-mac-pathway.zip -d y4-shift-mac cd y4-shift-mac bash setup-mac.sh source .venv/bin/activate python experiment.py --device mps --output results marimo edit notebook.py
CPU and available-device execution status is recorded in the downloaded README and validation report. DGX Spark execution requires that hardware; static validation alone does not establish GPU compatibility or performance. The browser lesson remains fully available without this extension.
NVIDIA DGX Spark · PyTorch CUDA
Apply uniform and value-dependent shifts to a frozen model over thousands of measurements. Sweep offsets, control the correction, and compare raw and corrected error rates.
Students extend the experiment in teams, documenting parameters, outputs and limitations.
The bundle contains lesson-specific working code, a configuration file, a reactive notebook, a deterministic CPU check and hardware setup instructions. Acceleration is reported only after a tensor operation and result read-back succeed.
Download DGX Spark investigation ↓unzip y4-shift-dgx-pathway.zip -d y4-shift-dgx cd y4-shift-dgx bash run-dgx.sh # Open the localhost notebook URL printed by the container.
CPU and available-device execution status is recorded in the downloaded README and validation report. DGX Spark execution requires that hardware; static validation alone does not establish GPU compatibility or performance. The browser lesson remains fully available without this extension.
A sensor calibration field note with paired readings and residual errors.
Download the editable handout →