Evals · Year 9 · 60 minutes

Put the bots to the test

Two homework bots both say they’re brilliant. How would you find out which one really is?

An eval (short for evaluation) is a test for an AI. Your school is choosing between two homework bots, Blitz and Sage. See how a dazzling demo can fool you, how a bot can ace a test by memorising it, and why the number and mix of questions matter. Then run your own bake-off.

Why it matters: Every AI company publishes eval scores. Knowing how evals work lets you read those scores like an expert.

By the end you can

Explain why a hand-picked demo is not a fair test, and what makes a test fair.
Explain how a bot can score well by memorising test questions, and why fresh questions fix it.
Use evidence from many tests to show how the number of questions, the topics and the marking rule change which bot looks best.

Starting knowledge: Fractions, percentages and averages. No coding or AI knowledge needed.

Watch

See the idea move

The animations are silent; captions carry the explanation. Pause whenever a caption changes and say what you expect next.

0:00 / 0:31

The demo trap

0:31 · no sound

Blitz’s company shows off three perfect answers, with confetti. Then Blitz answers 40 fresh questions your class wrote.

What happens on screen

Blitz stands in a spotlight while its company runs a demo.
Three questions appear and Blitz gets every one right: 987 × 654 = 645,498; there are 86,400 seconds in a day; 2¹⁰ = 1,024. Confetti falls: 3 out of 3!
Blitz’s answers to 40 fresh questions appear in four rows of ten: 9 right in maths, 8 in science, 7 about Australia and 2 of the trick questions. That is 26 out of 40, or 65%.
The demo score, 3 out of 3 = 100%, gets a bunch of cherries: the company tried lots of questions and showed only its best. This is cherry-picking.
Sage arrives. One stack of 40 questions goes to both bots. A fair test, or eval, gives every bot the same questions, chosen before anyone sees the answers.

0:00 / 0:34

Peeking at the answers

0:34 · no sound

A famous quiz is on the internet, so the bots read it while they trained. Blitz scores 10 out of 10 on it, then stumbles when the same question is asked in new words.

What happens on screen

Pages of text from the internet gather on the left. One is highlighted: a famous online quiz, answers included.
While they train, the bots read all the pages, the quiz included.
On the famous quiz, Blitz scores 10 out of 10 and Sage 7 out of 10.
Asked “What is the capital of Australia?”, Blitz answers “Canberra”: right.
Asked “Which city is Australia’s capital?”, Blitz answers “Sydney”: wrong. Blitz memorised the quiz’s answers instead of learning the ideas. The test had leaked into its training; this is called contamination.
On 40 fresh questions your class wrote, the order flips: Sage 31, Blitz 26.

Predict, then test

Bot bake-off

Choose which questions to ask, how many, and how to mark the answers. Run the test as often as you like and keep track of who wins.

Go further

The bigger picture

0:00 / 0:30

Luck and hidden gaps

0:30 · no sound

Ten random 5-question tests give a different winner almost every time; 20-question tests settle down. Then the 40-question scores are split up by topic.

What happens on screen

A table shows ten 5-question tests, each drawn at random from the 40 fresh questions. Blitz wins 3, Sage wins 3, and 4 are ties.
The table changes to ten 20-question tests. Sage wins 8, Blitz 1, with 1 tie.
A bar chart splits the 40 questions by topic, out of 10 each. Maths: Blitz 9, Sage 7. Science: 8 and 8. Australia: 7 and 8. Trick questions: 2 and 8.
A speech bubble shows one of Blitz’s made-up answers. Asked when Australia landed astronauts on the Moon, Blitz says “1974, on the Southern Cross mission.”
Which bot is best depends on the job. A good eval starts with a question: what do we need this bot to do?

Explain

Explain what you found

Run ten 5-question tests, then ten 20-question tests. How often did each bot win? Use your numbers to explain why a short test can’t be trusted.
Switch to the famous online quiz. Why does Blitz do so well there? What would you say to a company that boasts about its bot’s score on a famous test?
Change the marking rule so a confident wrong answer loses a point. Who wins now? Should a homework bot lose more for a wrong answer than for “I’m not sure”? Explain.
Your school wants a bot to help Year 9s with maths homework. Which bot would you pick? Back up your choice with evidence from the lab.

Counterexample challenge. Find a test where Blitz beats Sage every time you run it. Then decide: is that test unfair, or is it the right test for some job?

Teacher notes

Teaching this lesson

Suggested sequence · 60 minutes

Hook (5 min): who has seen an ad for an AI that “aces” an exam? How could you check a claim like that?
Watch “The demo trap”. Pause at the confetti: would you choose Blitz yet? Why or why not?
Watch “Peeking at the answers”. Ask: how is this like seeing the exam paper the night before?
Prediction and Bot bake-off (20 min). Pairs record ten 5-question runs and ten 20-question runs, then try the online quiz, single topics and the stricter marking rule.
Watch “Luck and hidden gaps”. The first ten 5-question tests in the lab (all topics) are the same ten shown in the animation.
Write your own eval (10 min): five fresh questions for a job the class chooses, including one trick question, and a rule for marking each answer.
Exit task (5 min): one explanation prompt.

Evidence to collect

A results table showing how often each bot won at two test sizes, with a sentence linking test size to luck; a correct explanation of contamination; a bot recommendation for one job that cites at least two pieces of lab evidence; five test questions with a marking rule.

Limits of this lesson

Blitz and Sage are pretend bots: Brightlab wrote every answer to show patterns that real AI evaluations find, namely memorised test questions (contamination), confident made-up answers (hallucinations), uneven skills and noisy small tests. No AI model runs in this lesson. Real evals use hundreds or thousands of questions and careful rules for marking open-ended answers, and sometimes use another AI as the marker, which brings its own biases. The lab runs entirely in the page; nothing is sent or stored.

Common misconceptions

“A perfect demo means a bot is reliable.” A demo shows the examples someone chose. It says little about how often the bot is right.
“A higher score means a better AI, full stop.” A score depends on which questions were asked, how many, and how answers were marked.
“If a bot sounds confident, it’s probably right.” Blitz sounds just as sure when it is wrong. Confidence is not evidence.
“Scores on famous tests are worthless.” They are useful evidence, as long as you know what was tested and whether the bot could have seen the questions.

Glossary

eval: Short for evaluation: a planned test of how well an AI does a job.
benchmark: A well-known eval that many companies report scores on.
cherry-picking: Showing only the best examples and hiding the rest.
contamination: When test questions were in a bot’s training data, so a high score may come from memory rather than skill.
fresh questions: Questions the bot cannot have seen before the test. Also called held-out questions.
sample size: How many questions a test has. With a small sample, luck matters more.
marking rule: How each answer turns into points. Decide it before the test.
hallucination: A confident answer that an AI made up.

Related Brightlab investigations

Brightlab

Opening your investigation…

Put the bots to the test

Two homework bots both say they’re brilliant. How would you find out which one really is?

By the end you can

Explain why a hand-picked demo is not a fair test, and what makes a test fair.

Explain how a bot can score well by memorising test questions, and why fresh questions fix it.

Use evidence from many tests to show how the number of questions, the topics and the marking rule change which bot looks best.

Starting knowledge: Fractions, percentages and averages. No coding or AI knowledge needed.

See the idea move

The animations are silent; captions carry the explanation. Pause whenever a caption changes and say what you expect next.

0:00 / 0:31

The demo trap

0:31 · no sound

Blitz’s company shows off three perfect answers, with confetti. Then Blitz answers 40 fresh questions your class wrote.

What happens on screen

Blitz stands in a spotlight while its company runs a demo.
Three questions appear and Blitz gets every one right: 987 × 654 = 645,498; there are 86,400 seconds in a day; 2¹⁰ = 1,024. Confetti falls: 3 out of 3!
Blitz’s answers to 40 fresh questions appear in four rows of ten: 9 right in maths, 8 in science, 7 about Australia and 2 of the trick questions. That is 26 out of 40, or 65%.
The demo score, 3 out of 3 = 100%, gets a bunch of cherries: the company tried lots of questions and showed only its best. This is cherry-picking.
Sage arrives. One stack of 40 questions goes to both bots. A fair test, or eval, gives every bot the same questions, chosen before anyone sees the answers.

0:00 / 0:34

Peeking at the answers

0:34 · no sound

A famous quiz is on the internet, so the bots read it while they trained. Blitz scores 10 out of 10 on it, then stumbles when the same question is asked in new words.

What happens on screen

Pages of text from the internet gather on the left. One is highlighted: a famous online quiz, answers included.
While they train, the bots read all the pages, the quiz included.
On the famous quiz, Blitz scores 10 out of 10 and Sage 7 out of 10.
Asked “What is the capital of Australia?”, Blitz answers “Canberra”: right.
Asked “Which city is Australia’s capital?”, Blitz answers “Sydney”: wrong. Blitz memorised the quiz’s answers instead of learning the ideas. The test had leaked into its training; this is called contamination.
On 40 fresh questions your class wrote, the order flips: Sage 31, Blitz 26.

Bot bake-off

Choose which questions to ask, how many, and how to mark the answers. Run the test as often as you like and keep track of who wins.

The bigger picture

0:00 / 0:30

Luck and hidden gaps

0:30 · no sound

Ten random 5-question tests give a different winner almost every time; 20-question tests settle down. Then the 40-question scores are split up by topic.

What happens on screen

A table shows ten 5-question tests, each drawn at random from the 40 fresh questions. Blitz wins 3, Sage wins 3, and 4 are ties.
The table changes to ten 20-question tests. Sage wins 8, Blitz 1, with 1 tie.
A bar chart splits the 40 questions by topic, out of 10 each. Maths: Blitz 9, Sage 7. Science: 8 and 8. Australia: 7 and 8. Trick questions: 2 and 8.
A speech bubble shows one of Blitz’s made-up answers. Asked when Australia landed astronauts on the Moon, Blitz says “1974, on the Southern Cross mission.”
Which bot is best depends on the job. A good eval starts with a question: what do we need this bot to do?

Explain what you found

Run ten 5-question tests, then ten 20-question tests. How often did each bot win? Use your numbers to explain why a short test can’t be trusted.

Switch to the famous online quiz. Why does Blitz do so well there? What would you say to a company that boasts about its bot’s score on a famous test?

Change the marking rule so a confident wrong answer loses a point. Who wins now? Should a homework bot lose more for a wrong answer than for “I’m not sure”? Explain.

Your school wants a bot to help Year 9s with maths homework. Which bot would you pick? Back up your choice with evidence from the lab.

Counterexample challenge. Find a test where Blitz beats Sage every time you run it. Then decide: is that test unfair, or is it the right test for some job?

Teaching this lesson

Suggested sequence · 60 minutes

Hook (5 min): who has seen an ad for an AI that “aces” an exam? How could you check a claim like that?
Watch “The demo trap”. Pause at the confetti: would you choose Blitz yet? Why or why not?
Watch “Peeking at the answers”. Ask: how is this like seeing the exam paper the night before?
Prediction and Bot bake-off (20 min). Pairs record ten 5-question runs and ten 20-question runs, then try the online quiz, single topics and the stricter marking rule.
Watch “Luck and hidden gaps”. The first ten 5-question tests in the lab (all topics) are the same ten shown in the animation.
Write your own eval (10 min): five fresh questions for a job the class chooses, including one trick question, and a rule for marking each answer.
Exit task (5 min): one explanation prompt.

Evidence to collect

Limits of this lesson

Common misconceptions

“A perfect demo means a bot is reliable.” A demo shows the examples someone chose. It says little about how often the bot is right.
“A higher score means a better AI, full stop.” A score depends on which questions were asked, how many, and how answers were marked.
“If a bot sounds confident, it’s probably right.” Blitz sounds just as sure when it is wrong. Confidence is not evidence.
“Scores on famous tests are worthless.” They are useful evidence, as long as you know what was tested and whether the bot could have seen the questions.

Glossary

eval: Short for evaluation: a planned test of how well an AI does a job.
benchmark: A well-known eval that many companies report scores on.
cherry-picking: Showing only the best examples and hiding the rest.
contamination: When test questions were in a bot’s training data, so a high score may come from memory rather than skill.
fresh questions: Questions the bot cannot have seen before the test. Also called held-out questions.
sample size: How many questions a test has. With a small sample, luck matters more.
marking rule: How each answer turns into points. Decide it before the test.
hallucination: A confident answer that an AI made up.