Year 12 · 55 minutes · Recorded LLM traces + native rerun
Explore measured Pythia activation patches, compare token positions and layers, and optionally repeat the experiment on native CPU or H100.
Student paper journal · Interactive notebook · Editable Python
Try the starting controls, save two runs and reopen a downloaded project. Print the student journal before class. If Python cannot load, use the supplied example below for a prediction and evidence critique; rerunning changed inputs requires the live or native notebook. Do not present paper discussion as a new model experiment. Keep an individual record even when you investigate in pairs. Record the exact settings, source and data split.
Start with 5–8 minutes of prediction, use about half the lesson for paired comparisons, then reserve at least 10 minutes for a counterexample and an individual explanation. Recorded-data analysis in the browser; optional native rerun.
Settings: {'prompt pair': 0, 'evidence': 'Recorded CPU 2026-09-07T01:31:08.751532+00:00'}
The capital of France is compared with The capital of England is
| condition | layer | position | logit_difference | p_clean_answer | p_corrupt_answer |
|---|---|---|---|---|---|
| Clean | -1 | -1 | 1.55712890625 | 0.0019448678940534592 | 0.0004098619392607361 |
| Corrupted | -1 | -1 | -2.048828125 | 0.000294992933049798 | 0.0022887922823429108 |
The table shows up to eight rows. Inspect the notebook for all values, controls and denominators.
Method and scope: Six selected capital-city prompt pairs, with no-op, zero and one seeded norm-matched random intervention control in a small pretrained research model. Not a representative accuracy evaluation, a Goodfire replication or a frontier-model result.
Assess four criteria, each 0–2: testable prediction; comparison identifying what stayed fixed; accurate use of exact evidence; counterexample and limited conclusion. 0 means absent or contradicted by the record, 1 means partly supported, and 2 means clear and supported. A surprising result earns no penalty.
Beginning response: ‘The picture looks right.’ Ask for a particular case. Developing response: names one number without its setting. Ask what comparison supports it. Secure response: states the setting and measured change and separates the notebook's result from the source paper's claim. Extending response: authors or reserves a new case, tests an alternative explanation, and revises the claim if needed. These are marking examples, not pupil data.
Extension: Add a new teacher-reviewed matched prompt pair.
System and data: Llama, Gemma and Qwen families, 2–72B, ten binding tasks
Method: Uses ablations to separate positional, lexical and reflexive retrieval; compares causal predictions as entity lists and context length change.
Task: Patch clean activations into a corrupted binding prompt, then compare multiple prompts. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: Our tiny binding transformer is independently trained and does not reproduce the paper's nine-model circuit findings.
System and data: GPT-2 Small on a known greater-than task
Method: Trains cross-layer transcoders, generates attribution graphs and compares the recovered mechanism with a previously characterised circuit.
Task: Use matched interventions and distinguish a surrogate circuit from the original model. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: Replication found both similarities and differences. Our attention patching does not reproduce CLT circuit discovery.
System and data: Community synthesis of circuit methods and findings
Method: Compares probes, sparse decompositions, attribution graphs and causal testing across published work; synthesis is not a single replication.
Task: Compare circuit discovery, intervention validation and scope across methods. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: The linked community guide was reviewed. Attribution graphs provide hypotheses with missing or uninterpretable components; interventions remain necessary.
Independent Brightlab adaptations; no Goodfire endorsement or full reproduction claim. Source methods reviewed 7 September 2026. Curriculum connections are selected planning links; confirm your school syllabus.