Audit the claim, not the confidence

Year 12 · Student investigation

What would count as evidence that a monitor really works?

Keep an individual record even when you investigate in pairs. Record the exact settings, source and data split.

Predict before seeing the answer

If a sample contains zero failures, what can we conclude? Will optimising the proxy always improve independent correctness?

Two comparisons

What did you change? What stayed the same? What did you notice?

Challenge your first explanation

Compare best-of-1 and best-of-32 on the same seed, then use a fresh seed. Report selected-policy failures, denominator and interval. Use 20 pools at failure probability 0.005 to investigate what zero failures does not establish.

Make and explain

A claim–evidence audit with denominators, a Wilson interval, proxy validation and one untested condition.

Limit the claim

This activity can show _ . It cannot tell us _ . A new case I would test is ____ .

Use fictional data. Download or print your work before leaving.