Base Rates and the "Positive Test" Paradox (Bayesian Statistics)

Bayesian statistics is about updating a belief in light of new evidence — starting from how likely something was beforehand (its "base rate") and combining that with how reliable the evidence actually is, rather than trusting the evidence in isolation. The classic place this trips people up is a positive test result: even a highly accurate test can be mostly wrong in practice if whatever it's testing for is rare enough to begin with. A tea supplier receives large shipments of tea, and a small fraction of any shipment turns out to be contaminated — a food-safety problem that has to be caught before it ships to customers. A quality-control test screens every batch, but no test is perfect: it can flag a contaminated batch correctly, flag a clean one wrongly, or miss real contamination and clear it as fine. This chart simulates a full 1,000-batch shipment against your own choice of how rare contamination is and how accurate the test is, showing what actually happens to every batch.

Set how rare contamination actually is with the prevalence slider, then how well the test catches real contamination (sensitivity) and clears genuinely clean batches (specificity). Each small square below is one batch in the shipment, coloured by what's actually true about it and what the test said. Watch the precision number as you drag prevalence down toward rare: even a test that sounds highly accurate can end up mostly crying wolf once contamination is rare enough.

What this teaches: Change how rare a condition is and how accurate a test is to see how many positive results are actually true. It teaches Bayes' theorem, base-rate neglect and the positive-test paradox.