The Twickenham projectBayesian updating: an agent that changes its mind one household at a time
A belief, a stream of answers, a rule for combining them, and a test of what a bad starting belief costs.
Census 2021 table TS045 gives the number of households with no car or van in every output area. Across the 138 areas of the four Twickenham wards there are 18,171 households, and 4,571 of them, 25.16%, have none. The agent does not know that. It asks households at random, ten at a time, and after each batch updates a Beta distribution, the standard way to hold a belief about a percentage. It stops as soon as its 95% credible interval is narrower than a target. Python runs each version of the agent 2,000 times.
An agent with no starting opinion (flat prior), 2,000 simulated runs, our Python run on Census 2021 data| Stops when its range is narrower than | Households asked (median) | Truth inside its range | Typical miss |
|---|
| 20 percentage points | 80 | 92.6% | 4.32 points |
| 10 percentage points | 290 | 94.3% | 2.02 points |
| 6 percentage points | 810 | 94.9% | 1.22 points |
With no opinion to start from, the agent is honest: its "95% sure" ranges contain the true 25.16% about 95% of the time, and halving the width of the range costs roughly four times as many households. The interesting part is what a prior does. A prior is coded as imaginary earlier answers, so "I think about 25%, but I am not sure" becomes 10 car-free households out of 40.
The same agent, target range 10 points, with different starting beliefs| Starting belief | Households asked (median) | Truth inside its range | Typical miss |
|---|
| None (flat) | 290 | 94.3% | 2.02 points |
| About 25%, held loosely | 250 | 95.6% | 1.90 points |
| About 60%, held loosely | 280 | 60.3% | 4.28 points |
| About 60%, held firmly | 10 | 0.0% | 34.0 points |
A good, loosely held prior saves work: 250 households instead of 290, with no loss of honesty. A wrong prior held loosely does real damage, because the agent stops while the false belief is still pulling on the answer, and its range misses the truth four times in ten. A wrong prior held firmly is a disaster. The agent is already so certain that it stops after the first ten households, reports a figure 34 points out, and is never right. Forced to keep going, it would still believe 35.1% after a thousand households and 28.3% after four thousand. Evidence wins in the end, but confidence decides how long that takes.
Ages 8 to 11
Guess what share of a bag of counters is red, then update the guess after each handful drawn.
Ages 11 to 15
Code the running tally for the Twickenham question in Python and watch the estimate settle.
Ages 15 and up
Add priors and credible intervals, then measure how often each version of the agent is right.
Census totals, our simulated agent
The household counts are Office for National Statistics Census 2021 data from Nomis, under the Open Government Licence. No real household was asked anything: every answer is simulated at the published rate, and the agent, priors and results are our own work.
