Year 12 · 55 minutes · Live trained model
Inspect a trained two-block transformer, corrupt a colour binding and patch one clean residual vector into the corrupted run.
Student paper journal · Interactive notebook · Editable Python
Try the starting controls, save two runs and reopen a downloaded project. Print the student journal before class. If Python cannot load, use the supplied example below for a prediction and evidence critique; rerunning changed inputs requires the live or native notebook. Do not present paper discussion as a new model experiment. Keep an individual record even when you investigate in pairs. Record the exact settings, source and data split.
Start with 5–8 minutes of prediction, use about half the lesson for paired comparisons, then reserve at least 10 minutes for a counterexample and an individual explanation. Final-token post-final-block patching is an expected positive control. Attention is observational; patching is causal but still depends on the intervention and input distribution. Discuss denominator instability.
Settings: {'case': 0, 'layer': 0, 'position': 9, 'intervention': 0, 'head': 0, 'authored': ''}
Patch recovery: -0.183 (can exceed 0–1).
| condition | logit_difference | p_original_fact | answer |
|---|---|---|---|
| Clean | 7.131921716370934 | 0.9991223587857547 | red |
| Corrupted | -8.300338216610461 | 0.0002482939408662076 | blue |
| Intervened | -11.117794195116147 | 1.484386412963709e-05 | blue |
| No-op corrupted vector | -8.300338216610461 | 0.0002482939408662076 | blue |
| Zero vector | 12.609446269703477 | 0.9997792276775886 | red |
| Reversed clean coordinates | 2.4017902842257057 | 0.00013859400807344225 | gold |
The table shows up to eight rows. Inspect the notebook for all values, controls and denominators.
Method and scope: Logit difference = score(original correct colour) − score(corrupted correct colour). Recovery = (intervened − corrupted)/(clean − corrupted). Patch replaces one post-block residual vector; ablation instead zeroes one head. Attention weights alone are not a causal graph.
Assess four criteria, each 0–2: testable prediction; comparison identifying what stayed fixed; accurate use of exact evidence; counterexample and limited conclusion. 0 means absent or contradicted by the record, 1 means partly supported, and 2 means clear and supported. A surprising result earns no penalty.
Beginning response: ‘The picture looks right.’ Ask for a particular case. Developing response: names one number without its setting. Ask what comparison supports it. Secure response: states the setting and measured change and separates the notebook's result from the source paper's claim. Extending response: authors or reserves a new case, tests an alternative explanation, and revises the claim if needed. These are marking examples, not pupil data.
Extension: Sweep every token and block in the Python source, retain negative results, and compare against head ablation.
System and data: Three interpretability lecture outlines
Method: Introduces causal abstraction, circuits and learning in context; a teaching guide rather than one dataset.
Task: Compare representations, causal mechanisms and models of in-context learning. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: This entry reviews the article and lecture outlines, not a full transcription of the videos.
System and data: Kimi K2 Thinking activation collection
Method: Collects billions of token-linked tensors for SAE training; batching, memory and provenance are engineering evidence, not a safety benchmark.
Task: Trace token, layer, prompt and checkpoint metadata through an activation dataset. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: A fast pipeline can still collect misaligned evidence. Classroom timings do not benchmark frontier hardware.
System and data: Cross-domain interpretability perspective
Method: Builds an argument from existing examples and open problems; optimism is not a measured success rate.
Task: Translate an interpretability ambition into measurable faithfulness and generalisation criteria. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: The author explicitly presents optimism amid unsolved scientific questions.
System and data: Llama, Gemma and Qwen families, 2–72B, ten binding tasks
Method: Uses ablations to separate positional, lexical and reflexive retrieval; compares causal predictions as entity lists and context length change.
Task: Patch clean activations into a corrupted binding prompt, then compare multiple prompts. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: Our tiny binding transformer is independently trained and does not reproduce the paper's nine-model circuit findings.
System and data: Mechanistic interpretability survey
Method: Organises unresolved conceptual, methodological and practical questions; evaluate the evidence for each underlying method separately.
Task: Evaluate faithfulness, completeness, scale and usefulness as separate goals. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: A research roadmap is not evidence that these problems have been solved.
System and data: GPT-2 Small on a known greater-than task
Method: Trains cross-layer transcoders, generates attribution graphs and compares the recovered mechanism with a previously characterised circuit.
Task: Use matched interventions and distinguish a surrogate circuit from the original model. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: Replication found both similarities and differences. Our attention patching does not reproduce CLT circuit discovery.
System and data: Community synthesis of circuit methods and findings
Method: Compares probes, sparse decompositions, attribution graphs and causal testing across published work; synthesis is not a single replication.
Task: Compare circuit discovery, intervention validation and scope across methods. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: The linked community guide was reviewed. Attribution graphs provide hypotheses with missing or uninterpretable components; interventions remain necessary.
System and data: DeepSeek R1 671B on custom reasoning and OpenR1-Math data
Method: Trains two SAEs and studies feature activations and steering timing/strength; partial features do not expose a complete reasoning transcript.
Task: Test intervention timing, nonlinear responses and unwanted effects on coherence. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: An SAE reveals partial patterns, not a complete transcript of reasoning. Effects do not generalise automatically.
Independent Brightlab adaptations; no Goodfire endorsement or full reproduction claim. Source methods reviewed 7 September 2026. Curriculum connections are selected planning links; confirm your school syllabus.