The Bletchley projectA bitmap index in Python for 874 census areas
Turn census facts into yes-or-no columns, store each column as one integer, and race three ways of querying them.
The learner downloads four census tables from Nomis for every output area in Milton Keynes: population density, household size, type of home and age. From them the program makes seven yes-or-no columns, for example "at least a quarter of homes are flats" or "at least a fifth of residents are under 15", plus one column saying whether the area is in Bletchley. Each column is stored three ways: as a Python list of True and False, as a NumPy array of booleans, and as a single Python integer whose bit number i is 1 when output area i says yes.
How many of the 874 output areas answer yes (our thresholds on Census 2021 tables)| Column | Areas marked yes |
|---|
| In the Bletchley built-up area | 129 |
| At least 5,000 residents per square kilometre | 355 |
| At least 35% of households are one person | 148 |
| At least 25% of homes are flats | 225 |
| At least 20% of residents are 65 or over | 234 |
| At least 40% of homes are detached | 250 |
| At least 20% of residents are under 15 | 418 |
A question such as "which Bletchley areas have many flats and many one-person households?" becomes one line: bletchley & flats & solo, followed by .bit_count() to count the ones. "Not" needs care, because Python integers have no fixed width: the learner flips the bits against a mask of 874 ones rather than using the ~ operator, which would produce a negative number. The program checks that all three methods return exactly the same answer before timing anything.
Three queries timed three ways (fastest of five repeats, Python 3.13 on our laptop; your machine will differ, the ratios much less)| Query | Matching areas | Integer bitmap | NumPy booleans | Python loop |
|---|
| Bletchley and flats and one-person households | 10 | 0.25 microseconds | 5.31 microseconds | 53.0 microseconds |
| Bletchley and many children and not detached | 61 | 0.29 microseconds | 5.61 microseconds | 58.1 microseconds |
| Dense or flats, outside Bletchley, older residents | 56 | 0.37 microseconds | 6.85 microseconds | 109.8 microseconds |
The integer bitmap answered each query between 197 and 299 times faster than looping over the rows, and more than ten times faster than NumPy, whose fixed overhead dominates on a table this small. It is also compact. One column as a Python integer takes 136 bytes; the same column as a list takes 7,832 bytes for the list alone; as a NumPy array, 874 bytes, one per area. Learners then ask the useful follow-up question: which part of the speed comes from handling 30 rows per internal step, and which from avoiding Python's loop overhead? Timing a loop in which every step does almost nothing helps separate the two.
Ages 8 to 11
Write yes and no answers as rows of lights on paper, then lay two rows on top of each other to find the "both" lights.
Ages 11 to 15
Store a small class survey as integers in Python and answer questions with &, | and bit_count().
Ages 15 and up
Build the full census bitmap index, time it against lists and NumPy, and explain the gaps.
Data and caveats
Census 2021 tables TS006, TS017, TS044 and TS007A via Nomis, and the ONS output area to built-up area lookup, under the Open Government Licence. The thresholds are ours and chosen to make interesting queries, not to describe any neighbourhood. Timings depend on the computer and the Python version; the counts do not.
