Year 10 · 85 minutes · Retrieval
When should an assistant answer, and when should it stop?
This local assistant retrieves a supplied passage using keyword overlap, then applies an explicit evidence gate. It returns a quoted extract only when the required fact is present and the score meets a threshold; otherwise it refuses or requests clarification. No language model generates prose. Visible retrieval and refusal make a small system inspectable, but neither a score nor a citation can guarantee that the source is current or true.
Read the fictional field notes and four supplied queries: supported, absent, ambiguous and conflicting. Rehearse source removal and threshold control. Explain that this deterministic assistant is simpler than an LLM-based RAG system.
Match claims to evidence and inspect retrieved passages. Useful earlier investigations: y6-sources, y10-chunks Use pairs for investigation, with operator/reviewer swaps after each comparison. Keep individual predictions, journals and a short oral defence so group work does not hide understanding.
Australian Curriculum Version 9 · Digital Technologies: AC9TDI10P06, AC9TDI10P10. Selected aspects only. This activity contributes evidence; it does not cover the full descriptor or achievement standard. A programming descriptor is not claimed for merely moving controls. ACARA AI curriculum connection · V9 Technologies These are planning connections, not ACARA endorsement or exhaustive descriptor alignment.
Define that answers must come from the supplied notes, including dates and conditions.
Ask: “What should happen when the notes do not say?”
Listen for: “The helper should say the evidence is missing.”
Classify the four queries before running them.
Ask: “Is a similar topic enough to answer the exact question?”
Listen for: “No, the required fact must be present.”
Edit a query, inspect term matches, top passage and answerability check, then compare thresholds with sources fixed.
Ask: “Which stage rejected this query?”
Listen for: “Retrieval was weak, or the required fact was absent.”
Disable the relevant note while keeping the query unchanged.
Ask: “Should the assistant repeat its earlier answer from memory?”
Listen for: “No; it must reflect the current supplied evidence.”
Choose a threshold, run all four cases and record supported answers, false answers, refusals and false refusals. Add one adversarially phrased question.
Ask: “Could a safer threshold become too unhelpful?”
Listen for: “Yes, it can reject answerable questions.”
Document sources, gate, evaluation and limits.
Ask: “What does this lab actually compute?”
Listen for: “Lexical retrieval, explicit checks and extractive responses.”
Retrieval eliminates invented answers.
Predict supported answer, missing evidence or ambiguity for each supplied query before running the assistant.
A high-scoring passage can be about the right topic while lacking the required fact; removing a source must change answerability.
Specify an evidence gate and evaluate supported answers and false refusals on the fixed suite plus one new challenge.
Require the retrieved passage and gate reason for every reported result. Check that a citation is not treated as automatic proof of a generated claim.
Begin with fixed query buttons alongside an editable query field, then compose a new question with a partner.
Add a second conflicting dated source and design a conflict response rather than arbitrarily selecting the highest score.
An extractive assistant system card with source-specific traces and an answerability test suite.
No external model or API is called. Use only synthetic notes; do not paste student information or confidential school policies.
Evaluate thousands of structured synthetic answerability cases. Sweep retrieval thresholds and compare false answers and refusals with and without a required-fact gate; relevance scores are supplied fixtures.
| Criterion | Beginning | Secure | Extending |
|---|---|---|---|
| System trace | Treats the helper as one black box | Separates retrieval, gate and response | Diagnoses the first failing stage |
| Evaluation | Counts only successful answers | Tests missing and ambiguous evidence | Reports false answers, false refusals and residual source risk |
Australian Curriculum Version 9 · Digital Technologies
References: AC9TDI10P06, AC9TDI10P10. Read the current source (checked 2026-09-07).
Evidence to assess: An extractive assistant system card with source-specific traces and an answerability test suite.
Selected aspects only. This activity contributes evidence; it does not cover the full descriptor or achievement standard. A programming descriptor is not claimed for merely moving controls. Moderate the supplied illustrative responses against your school unit and current achievement standard.
These are planning estimates to test with your class. A short session develops one supported claim; it does not compress the whole senior project.
| Stage | 45 minute focus | 60 minute investigation |
|---|---|---|
| Readiness and prediction | 0–5 | 0–5 |
| Trace the supplied example | 5–13 | 5–15 |
| Author and run cases | 13–25 | 15–35 |
| Counterexample and redesign | 25–35 | 35–45 |
| Explain and discuss | 35–42 | 45–55 |
| Export and handover | 42–45 | 55–60 |
For a longer project, use three 50-minute sessions. Session 1 (0–50): readiness, model, hypothesis and initial cases. Export a project and record the next test. Session 2 (50–100): reopen, check settings, author counterexamples and revise the design. Export the changed project and identify unresolved evidence. Session 3 (100–150): independent peer test, final artefact, individual explanation and moderation. If using two 60-minute sessions, stop at minute 60 after saving the first comparison; use 60–120 for redesign, independent test and defence.
Entry check: Match claims to evidence and inspect retrieved passages. Ask the learner to demonstrate it before choosing the level of support.
Preparation: allow about 15 minutes to run the starter, print the cards and check a project can be reopened. This estimate has not yet been measured in a classroom pilot.
Read the entry question aloud, model one row, and label the units. Offer the case table as a large-print sheet. Keep mathematical derivations optional until the learner can explain the comparison.
For one device, use a projector: one pair predicts, one operates, and the class records on paper. Swap roles after the first comparison. For individual access, support keyboard controls and a written table equivalent to each visual. Learners may explain orally or with an annotated diagram. Never require personal data, a recorded voice, or a photograph.
Mixed readiness: if the entry check is difficult, use the linked prerequisite and the first two case cards; retain the same central question. If secure, ask the learner to design an unseen test and state which explanation it could disprove.
Answer only when required evidence exists and the contradiction flag passes. All case labels are learner-authored.
Starting parameters: Require evidence = True, Block contradictions = True
3 cases calculated from your supplied inputs. Compare the evidence with your prediction.
| question | evidence | contradiction | decision | expected | correct |
|---|---|---|---|---|---|
| When is the tour? | Tour at nine. | False | answer | answer | True |
| Is lunch free? | False | withhold | withhold | True | |
| What is entry? | Eight dollars; another source says ten. | True | withhold | withhold | True |
These are authored examples, not work collected from children. Assess reasoning using the lesson rubric, not whether the first prediction was correct.
Beginning: “It worked because the result looks right.” This identifies no exact case, control or measurement. Ask the learner to point to one row and say what happened.
Developing: “In the first case I recorded question: When is the tour?; evidence: Tour at nine.; contradiction: False; decision: answer; expected: answer; correct: True.” This cites evidence, but does not yet explain how the result follows from the rule. Ask the learner to trace the relevant step.
Secure: “For the first supplied case, question: When is the tour?; evidence: Tour at nine.; contradiction: False; decision: answer; expected: answer; correct: True. I can trace it using this mechanism: Answer only when required evidence exists and the contradiction flag passes. All case labels are learner-authored. My result supports a claim about these supplied cases. It does not establish that the same result holds outside them.” Look for an accurate trace, the actual settings and a bounded claim; accept equivalent oral or visual evidence.
Extending: The learner constructs and reruns a new case, reports whether the first explanation survives, and defends a revised design. Use this concrete challenge: Write answerability cases with expected decisions before running the policy. Include absent and contradictory evidence. Require the original and changed evidence and this boundary: This policy tests answerability logic, not natural-language entailment or generated answer correctness.
Moderation: first assess independently against each lesson criterion. Compare the exact trace or artefact that led to your judgement. Resolve differences using evidence, not polished language. Keep each learner's individual explanation even when the artefact was produced in a group.