Year 12 · 55 minutes · Live trained SAE on synthetic data
Train a small sparse autoencoder on known mixtures. Change dictionary size, rarity and sparsity pressure, then evaluate on untouched mixtures.
Student paper journal · Interactive notebook · Editable Python
Try the starting controls, save two runs and reopen a downloaded project. Print the student journal before class. If Python cannot load, use the supplied example below for a prediction and evidence critique; rerunning changed inputs requires the live or native notebook. Do not present paper discussion as a new model experiment. Keep an individual record even when you investigate in pairs. Record the exact settings, source and data split.
Start with 5–8 minutes of prediction, use about half the lesson for paired comparisons, then reserve at least 10 minutes for a counterexample and an individual explanation. The loss curve uses training examples; the summary uses 60 held-out examples. Capacity, sparsity and interpretability are different objectives. Fixed seeds help comparisons but are not uncertainty estimates.
Settings: {'capacity': 4, 'rare': 0.1, 'penalty': 0.2, 'seed': 17}
A small ReLU sparse autoencoder is trained live on 180 known mixtures and evaluated on 60 untouched mixtures. Compare total error with rare-feature error.
| case | rare_feature_present | squared_error | nonzero_units |
|---|---|---|---|
| 1 | False | 0.3816414182290277 | 0 |
| 2 | False | 0.0 | 0 |
| 3 | False | 0.1976839984624634 | 1 |
| 4 | False | 0.2681538796281341 | 0 |
| 5 | False | 0.332246225401043 | 0 |
| 6 | True | 0.7409680959380878 | 2 |
| 7 | False | 1.3333192730283883 | 0 |
| 8 | False | 1.1519646723748889 | 1 |
The table shows up to eight rows. Inspect the notebook for all values, controls and denominators.
Method and scope: Loss = mean summed squared reconstruction error + λ × mean summed positive activations. The chart reports per-coordinate reconstruction MSE. Three known features share two dimensions. This is a small SAE experiment, not a replication of Goodfire scaling curves or block-sparse featurizers.
Assess four criteria, each 0–2: testable prediction; comparison identifying what stayed fixed; accurate use of exact evidence; counterexample and limited conclusion. 0 means absent or contradicted by the record, 1 means partly supported, and 2 means clear and supported. A surprising result earns no penalty.
Beginning response: ‘The picture looks right.’ Ask for a particular case. Developing response: names one number without its setting. Ask what comparison supports it. Secure response: states the setting and measured change and separates the notebook's result from the source paper's claim. Extending response: authors or reserves a new case, tests an alternative explanation, and revises the claim if needed. These are marking examples, not pupil data.
Extension: Repeat with five seeds and align learned dictionary directions with known generating directions.
System and data: Llama 3.1 8B layer 19 and Llama 3.3 70B layer 50
Method: Releases SAEs and evaluates sparsity, fidelity and judged steering. Check judge dependence and checkpoint/layer compatibility.
Task: Check checkpoint, layer and reconstruction quality before reusing an SAE. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: Open weights enable investigation; they do not certify feature labels or universal coverage.
System and data: Known toy representations and ResNet18 vision models
Method: Varies overlap and adversarial robustness; bidirectional causal evidence is demonstrated in toy models, with a narrower direction tested in vision.
Task: Measure cross-talk and reconstruction in a known superposition model. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: Bidirectional causal evidence is strongest in toy models; real vision-model evidence is narrower, not a universal explanation.
System and data: Synthetic manifolds, DINOv3 and SDXL
Method: Trains block-sparse featurizers, compares reconstruction and subspace coverage, and tests edits; multidimensional features are evaluated against direction-based baselines.
Task: Compare one-dimensional features with a two-dimensional cyclic representation. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: The experiments concern vision and diffusion models; the classroom geometry is an analogy for representation, not an LLM replication.
System and data: Synthetic shapes and Llama 3.1 8B activations
Method: Varies SAE capacity and examines splitting, dilution and recovery of manifolds; reconstruction alone cannot measure concept completeness.
Task: Vary dictionary size and inspect reconstruction, sparsity and feature coverage separately. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: Low reconstruction error alone does not establish semantic completeness; our small SAE is not the paper's full evaluation.
System and data: Evo 2 genomic activations
Method: Trains SAEs and compares discovered features with biological annotations; associated signals motivate hypotheses rather than clinical conclusions.
Task: Contrast feature–annotation alignment with causal steering and external biological validation. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: DNA models are not ordinary text LLMs. The 2025 report was updated to note Nature publication in March 2026.
System and data: An intermediate layer of Llama 3.3 70B
Method: Maps SAE feature relationships and demonstrates steering selected features; the displayed clusters are selected examples, not exhaustive coverage.
Task: Separate visual neighbourhoods, auto-generated labels and causal intervention results. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: A two-dimensional map is a lossy view. The older Ember demo is deprecated.
System and data: ReLU SAEs on circles and other known feature manifolds
Method: Studies how adding dictionary capacity can tile common manifolds and lower loss while leaving rare features undiscovered.
Task: Vary feature frequency and dictionary capacity; report rare-feature error alongside total reconstruction. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: The paper identifies regimes and mechanisms, not a claim that all larger SAEs get worse.
System and data: Llama-3-8B and LMSYS-Chat-1M
Method: Trains an SAE, inspects features and demonstrates activation steering; feature labels and generated examples require counterexamples and evaluation.
Task: Separate reconstruction quality, semantic labels and causal steering evidence. Record the source model/task, a baseline, the changed factor, a measurement and an alternative explanation. State exactly which part your notebook investigates and which part it does not reproduce.
Boundary: Features can overlap or duplicate. Ember API and demo links are deprecated; these labs need neither.
Independent Brightlab adaptations; no Goodfire endorsement or full reproduction claim. Source methods reviewed 7 September 2026. Curriculum connections are selected planning links; confirm your school syllabus.