brightlab.Made for curious mindsBrightlab
Year 10 · Retrieval
Can a cut in the wrong place change what an assistant finds?
Go to the investigation ↓A chunking policy with query-level retrieval evidence, costs and a failure case.
Take this investigation into Python: open the interactive Marimo notebook →
What information does a document boundary separate?
Retrieval systems often split documents into chunks and score each against a query. This lab uses a transparent lexical overlap score over word sets. Chunk size and overlap determine whether a fact stays with its qualifier and how much irrelevant text enters the context. Larger chunks can retain context but dilute relevance. This is not semantic embedding search; a paraphrase with no shared words can fail.
Before this lesson: Find a claim and its qualifier in a passage. Useful earlier investigations: The expedition’s shrinking memory.
Retrieval systems often split documents into chunks and score each against a query. This lab uses a transparent lexical overlap score over word sets. Chunk size and overlap determine whether a fact stays with its qualifier and how much irrelevant text enters the context. Larger chunks can retain context but dilute relevance. This is not semantic embedding search; a paraphrase with no shared words can fail.
Find a claim and its qualifier in a passage. Useful earlier investigations: The expedition’s shrinking memory.
Print the supplied reserve document with word indices and three queries. Rehearse the lexical scorer and qualifier markers. Explain the overlap constraint: overlap must be smaller than chunk size.
ACARA V9 AI curriculum connection ↗ · Technologies ↗
A complete teaching sequence · 85 minutes
Place a paper boundary between an opening time and its wet-weather exception.
Ask: “Would the first piece alone support an unconditional answer?”
Listen for: “No, the exception is missing.”
Predict the top chunk and qualifier coverage before changing chunk size.
Ask: “What remains fixed while we test this policy?”
Listen for: “Document, query and scoring formula.”
Adjust chunk size and overlap; inspect cut positions, lexical scores, retrieved words and context cost. Compare the same three queries.
Ask: “Did a higher score guarantee complete evidence?”
Listen for: “No, the qualifier may be in another chunk.”
Run the query using different wording with little lexical overlap.
Ask: “Can better chunking fix a scorer that does not recognise synonyms?”
Listen for: “Not necessarily; that is a retrieval representation limit.”
Choose size and overlap, record coverage and cost for all queries and state a failure case. Propose sentence-aware boundaries on paper.
Ask: “Why not simply retrieve the whole document?”
Listen for: “It increases cost and can include irrelevant or conflicting text.”
Include exact chunk IDs and relevant passages.
Ask: “What would another evaluator need to reproduce this?”
Listen for: “The document, queries, scorer and chunk settings.”
Larger chunks always retrieve better.
A paraphrased query with no shared keywords can fail regardless of chunk size because the scorer is lexical.
Choose a chunk/overlap policy using a fixed three-query suite; report complete-evidence recall and context word cost.
Ask learners to reproduce one overlap score and point to the exact qualifier. Do not accept top-hit score as a substitute for evidence completeness.
| Criterion | Beginning | Secure | Extending |
|---|---|---|---|
| Retrieval mechanism | Treats chunking as formatting | Connects boundaries to scored passages | Distinguishes scorer limits from boundary limits |
| Policy evaluation | Optimises one query | Compares fixed queries and cost | Reports qualifier recall and a reproducible failure |
Use a short document and two queries; provide a keyword intersection table before computing scores independently.
Implement sentence-aware splitting in the hardware extension and compare it against fixed windows without changing the test queries.
Only supplied fictional documents are indexed. No uploads, network retrieval or private school documents are used.
Shared device? Turn remembering off. A project file lets you continue on another device.
Your browser is the laboratory
Move a boundary with the query fixed. Predict whether the retrieved passage retains both a fact and its qualifier.
The reserve document begins with the gate’s 8 am dry-day opening, followed by its wet-day closure. Fixed windows start at 14 words with zero overlap. Query: “gate opens wet days”. Retrieve the two highest lexical-overlap chunks.
The experiment opens after your prediction.
There is no penalty for being surprised.
Apple Silicon · PyTorch MPS
Compare four overlap policies for a synthetic reserve document using CPU segmentation and GPU lexical query scoring. Change chunk size and measure retrieved word cost and qualifier completeness. Repeated chunks test throughput, not document diversity.
Students extend the experiment in teams, documenting parameters, outputs and limitations.
The bundle contains lesson-specific working code, a configuration file, a reactive notebook, a deterministic CPU check and hardware setup instructions. Acceleration is reported only after a tensor operation and result read-back succeed.
Download Mac Studio investigation ↓unzip y10-chunks-mac-pathway.zip -d y10-chunks-mac cd y10-chunks-mac bash setup-mac.sh source .venv/bin/activate python experiment.py --device mps --output results marimo edit notebook.py
CPU and available-device execution status is recorded in the downloaded README and validation report. DGX Spark execution requires that hardware; static validation alone does not establish GPU compatibility or performance. The browser lesson remains fully available without this extension.
NVIDIA DGX Spark · PyTorch CUDA
Compare four overlap policies for a synthetic reserve document using CPU segmentation and GPU lexical query scoring. Change chunk size and measure retrieved word cost and qualifier completeness. Repeated chunks test throughput, not document diversity.
Students extend the experiment in teams, documenting parameters, outputs and limitations.
The bundle contains lesson-specific working code, a configuration file, a reactive notebook, a deterministic CPU check and hardware setup instructions. Acceleration is reported only after a tensor operation and result read-back succeed.
Download DGX Spark investigation ↓unzip y10-chunks-dgx-pathway.zip -d y10-chunks-dgx cd y10-chunks-dgx bash run-dgx.sh # Open the localhost notebook URL printed by the container.
CPU and available-device execution status is recorded in the downloaded README and validation report. DGX Spark execution requires that hardware; static validation alone does not establish GPU compatibility or performance. The browser lesson remains fully available without this extension.
A chunking policy with query-level retrieval evidence, costs and a failure case.
Download the editable handout →