brightlabYear 10 · 60 minutes
Inside a geometric calculator
How can several circles represent one number — and help a model add?
You will learn to: Calculate residues, explain how several periodic features disambiguate numbers, and distinguish a pattern in activations from causal evidence.
Your mathematical model
Follow the calculation
Before opening the researchers’ model below, build a prediction you can test. A circle keeps the remainder after complete turns.
Start 0 · add 0
Start 1 · add 3
Start 6 · add 8
Blue solid = start. Pink dashed = added number. Thick green = sum. Each angle is 2π × remainder ÷ period. These three views repeat together every 10: they cannot distinguish 7 from 17.
Can you break this number code?
Goodfire’s original research demonstration
Inspect Llama’s geometric calculator
Start with August + 6, then try 16 months after August. Follow the input and output representations on the period-2, period-5 and period-10 views. Which information survives a complete turn? Compare your mathematical prediction above with these recorded model activations.
The interactive below is hosted by Goodfire. If it does not fit your screen or your school blocks it, open the original in a full window ↗.
Source measurements and interface: Goodfire. Spark does not generate these recorded Llama results.
Goodfire’s original research demonstration
Change the internal representation
This experiment fixes the prompt at sixteen months after August. Change the available steering controls and compare the result with the baseline. This modifies internal features; it is stronger evidence than merely seeing a circle. It still does not establish every step of the model’s algorithm.
The interactive below is hosted by Goodfire. If it does not fit your screen or your school blocks it, open the original in a full window ↗.
Source measurements and interface: Goodfire. Spark does not generate these recorded Llama results.
Now investigate for yourself
Your Marimo research notebook
Our circles are exact mathematical illustrations. The embedded demos are Goodfire’s recorded Llama measurements. Our small trained network and open-model experiments are independent extensions, not a replication of the complete paper.
Teacher notes & evidence task
Starting knowledge: Addition, remainders and coordinates; sine and cosine are introduced visually.
- Predict the residue of 17 on each circle. Explain why one circle cannot identify every number.
- Use the authors’ activation explorer to compare inputs and output for 6 + 8. Record one observation for each period.
- Contrast those observations with the authors’ steering experiment. What changed inside the model?
- In Marimo, train a small network and inspect its learned hidden states. Compare it with an openly specified Fourier representation.
- Extend to a small pretrained language model. Fit a probe on training prompts, then score reserved prompts.
Evidence to collect: Submit a residue calculation, a comparison from the original demo, and one claim supported by an intervention rather than just a picture.
Research basis: Goodfire identified a shared addition mechanism in Llama 3.1 8B. Its number features include periods 2, 5 and 10. The paper tests interventions as well as observing patterns.
Goodfire research article ↗ · Full research paper ↗ · Setup and teaching guidance