Year 8 · 75 minutes · Evaluation
When does a better fit become a worse prediction?
A polynomial can bend more as its degree increases. Fitting minimises squared error on the training points, including their noise. A flexible curve may follow that noise and predict poorly between points. We use a separate validation set to choose degree and keep a final test sealed until the choice is made. This is a computed polynomial fit to synthetic data, not a claim about all models.
Review squared residuals. Print the fixed training and validation points in different symbols. Rehearse degree control, lock choice and final-test reveal; explain that reset begins a new classroom demonstration, not a newly independent dataset.
Read a coordinate plot and distinguish training from testing. Useful earlier investigations: y3-holdout Use pairs for investigation, with operator/reviewer swaps after each comparison. Keep individual predictions, journals and a short oral defence so group work does not hide understanding.
Australian Curriculum Version 9 · Digital Technologies: AC9TDI8P02, AC9TDI8P10. Selected aspects only. This activity contributes evidence; it does not cover the full descriptor or achievement standard. A programming descriptor is not claimed for merely moving controls. ACARA AI curriculum connection · V9 Technologies These are planning connections, not ACARA endorsement or exhaustive descriptor alignment.
Sketch a simple curve through noisy points and compare with a curve touching every point.
Ask: “Must every wobble belong to the underlying pattern?”
Listen for: “Some might be measurement noise.”
Predict training and validation changes from degree 1 to a high degree.
Ask: “Could one error fall while another rises?”
Listen for: “Yes, if the curve starts fitting noise.”
Adjust degree, inspect the fitted curve and separate error summaries. Record three degrees with data fixed.
Ask: “Where are the largest validation residuals?”
Listen for: “Often between training points where the curve bends sharply.”
Use the clean-data edge case to show that extra capacity is not always harmful.
Ask: “Is a higher degree automatically wrong?”
Listen for: “No, it depends on data and the underlying function.”
Choose degree using validation, write a reason and lock it before revealing final-test results. Report any gap without retuning the reported final result.
Ask: “What happens if we use the final score to choose again?”
Listen for: “It becomes another validation set.”
Submit choice, training/validation comparison and one final-test result.
Ask: “What does this test fail to tell us?”
Listen for: “Performance outside the sampled range or under new conditions.”
The most detailed fit is the best model.
Increase degree with all points fixed. Predict whether training and validation errors move together.
On clean data generated by a higher-order curve, extra capacity can help; degree alone does not diagnose overfitting.
Select degree from validation evidence, lock it and report the sealed final test once.
Check the timing of the degree decision and final-test reveal. Ask for a specific residual supporting the overfitting explanation.
Compare only degree 1, 3 and 7 and use precomputed residual squares before exploring all degrees.
Add regularisation and compare capacity reduction with shrinking coefficients.
A capacity-selection report with residual sketches and a sealed-test decision record.
All curves are synthetic. No predictions about students’ grades or abilities are modelled.
Fit seven polynomial degrees to fixed noisy training data on CPU, then evaluate thousands of unseen synthetic points on MPS/CUDA. Change ridge regularisation and compare training and unseen error.
| Criterion | Beginning | Secure | Extending |
|---|---|---|---|
| Fit interpretation | Prefers the smallest training error | Uses the validation gap | Explains residual geometry and a counterexample |
| Selection protocol | Tunes repeatedly on final results | Locks a validation-based choice | Reports final uncertainty and avoids inflated claims |
Australian Curriculum Version 9 · Digital Technologies
References: AC9TDI8P02, AC9TDI8P10. Read the current source (checked 2026-09-07).
Evidence to assess: A capacity-selection report with residual sketches and a sealed-test decision record.
Selected aspects only. This activity contributes evidence; it does not cover the full descriptor or achievement standard. A programming descriptor is not claimed for merely moving controls. Moderate the supplied illustrative responses against your school unit and current achievement standard.
These are planning estimates to test with your class. A short session develops one supported claim; it does not compress the whole senior project.
| Stage | 45 minute focus | 60 minute investigation |
|---|---|---|
| Readiness and prediction | 0–5 | 0–5 |
| Trace the supplied example | 5–13 | 5–15 |
| Author and run cases | 13–25 | 15–35 |
| Counterexample and redesign | 25–35 | 35–45 |
| Explain and discuss | 35–42 | 45–55 |
| Export and handover | 42–45 | 55–60 |
For a longer project, use three 50-minute sessions. Session 1 (0–50): readiness, model, hypothesis and initial cases. Export a project and record the next test. Session 2 (50–100): reopen, check settings, author counterexamples and revise the design. Export the changed project and identify unresolved evidence. Session 3 (100–150): independent peer test, final artefact, individual explanation and moderation. If using two 60-minute sessions, stop at minute 60 after saving the first comparison; use 60–120 for redesign, independent test and defence.
Entry check: Read a coordinate plot and distinguish training from testing. Ask the learner to demonstrate it before choosing the level of support.
Preparation: allow about 15 minutes to run the starter, print the cards and check a project can be reopened. This estimate has not yet been measured in a classroom pilot.
Read the entry question aloud, model one row, and label the units. Offer the case table as a large-print sheet. Keep mathematical derivations optional until the learner can explain the comparison.
For one device, use a projector: one pair predicts, one operates, and the class records on paper. Swap roles after the first comparison. For individual access, support keyboard controls and a written table equivalent to each visual. Learners may explain orally or with an annotated diagram. Never require personal data, a recorded voice, or a photograph.
Mixed readiness: if the entry check is difficult, use the linked prerequisite and the first two case cards; retain the same central question. If secure, ask the learner to design an unseen test and state which explanation it could disprove.
Fit a polynomial by regularised least squares on training rows. Compare validation errors; final tests use a fresh seeded quadratic fixture.
Starting parameters: Degree = 2, Final test seed = 17
Coefficients: 0.005714, -0.01, 1.007143. Fresh seeded final cases revealed.
| x | y | split | prediction | squared error |
|---|---|---|---|---|
| -2 | 4 | train | 4.054286 | 0.002947 |
| -1 | 1.2 | train | 1.022857 | 0.03138 |
| 0 | -0.2 | train | 0.005714 | 0.042318 |
| 1 | 1.1 | train | 1.002857 | 0.009437 |
| 2 | 4 | train | 4.014286 | 0.000204 |
| -1.5 | 2.25 | validation | 2.286786 | 0.001353 |
| 1.5 | 2.25 | validation | 2.256786 | 4.6e-05 |
| -1.0293745435774326 | 1.123718134231515 | final | 1.083189 | 0.001643 |
| -0.16170244105160236 | -0.01442329691027219 | final | 0.033666 | 0.002313 |
| 0.8560150116682053 | 0.6993401457383334 | final | 0.73515 | 0.001282 |
| 0.6854219157248735 | 0.41273039438153925 | final | 0.472019 | 0.003515 |
| -1.3762585185468197 | 1.905771897189614 | final | 1.927094 | 0.000455 |
The downloadable answer record contains all remaining cases.
These are authored examples, not work collected from children. Assess reasoning using the lesson rubric, not whether the first prediction was correct.
Beginning: “It worked because the result looks right.” This identifies no exact case, control or measurement. Ask the learner to point to one row and say what happened.
Developing: “In the first case I recorded x: -2; y: 4; split: train; prediction: 4.054286; squared error: 0.002947.” This cites evidence, but does not yet explain how the result follows from the rule. Ask the learner to trace the relevant step.
Secure: “For the first supplied case, x: -2; y: 4; split: train; prediction: 4.054286; squared error: 0.002947. I can trace it using this mechanism: Fit a polynomial by regularised least squares on training rows. Compare validation errors; final tests use a fresh seeded quadratic fixture. My result supports a claim about these supplied cases. It does not establish that the same result holds outside them.” Look for an accurate trace, the actual settings and a bounded claim; accept equivalent oral or visual evidence.
Extending: The learner constructs and reruns a new case, reports whether the first explanation survives, and defends a revised design. Use this concrete challenge: Choose a degree using validation data, lock the choice, then evaluate a fresh seeded test set. Require the original and changed evidence and this boundary: A synthetic quadratic favours particular hypotheses. Do not tune repeatedly on final tests.
Moderation: first assess independently against each lesson criterion. Compare the exact trace or artefact that led to your judgement. Resolve differences using evidence, not polished language. Keep each learner's individual explanation even when the artefact was produced in a group.