The Stafford projectAn angling agent that must learn which bait works
Simulate a day by the river, try different strategies, and measure what each one catches.
The learner gives a simulated angler three baits from Walton's book: minnow, worm and fly. Each has a hidden chance of a bite on every cast, which the agent does not know. For the exercise we invented the numbers: 10 percent for minnow, 6 percent for worm and 14 percent for fly, so the fly is secretly the winner. A day has 200 casts, and the program runs 2,000 different simulated days, each with its own random seed so the results can be repeated. A perfect angler who somehow knew the answer would expect 28 fish a day. The question is how close a learning agent can get. This is the classic multi-armed bandit problem, named after rows of slot machines.
Fish per day for each strategy over 2,000 simulated days, invented bite chances, our Python run, 28 September 2026| Strategy | Average fish a day | Days the fly was used most |
|---|
| Pick a bait at random every cast | 19.94 | About a third |
| Try each bait once, then always use the top scorer so far | 20.54 | 12.4% |
| Same, but explore at random 5% of the time | 21.44 | 26.1% |
| Explore 10% of the time | 21.95 | 35.2% |
| Explore 30% of the time | 22.18 | 53.9% |
| Upper confidence bound (UCB1) | 21.02 | 71.3% |
The obvious strategy, try each bait once and then stick with the top scorer, barely beats random casting and finds the fly on only 12.4 percent of days. The learner investigates and discovers why. On 72.7 percent of days, the very first cast with every bait catches nothing, so all three averages are zero. The program breaks the tie by taking the first bait in its list, the minnow, and because it never tries anything else, it never learns the fly is better. That is Venator's complaint turned into code: a couple of unlucky early tries decide everything. Piscator's advice, that you "must endure worse luck sometime", is really an argument for exploring.
Adding a little randomness helps. Exploring 10 percent of the time lifts the catch to 21.95 fish a day, and 30 percent reaches 22.18. A cleverer method called UCB1 gives an untested bait the benefit of the doubt until it has been tried enough, and it settles on the fly on 71.3 percent of days, yet over only 200 casts it spends so long checking that it catches fewer fish than simple exploring. There is no free lunch: the right balance depends on how long the day is. The learner finishes by rerunning everything with different invented chances to see which conclusions still hold.
Ages 8 to 11
Play a game with three hidden-chance spinners and decide when to switch.
Ages 11 to 15
Simulate the three baits in Python with random numbers and count the catches.
Ages 15 and up
Compare greedy, exploring and UCB1 agents, and find the tie-breaking bug.
Walton's words, invented chances
Quotations come from Project Gutenberg's editions of The Compleat Angler and Andrew Lang's introduction to it. The bite chances are made up for teaching, and the simulation and its results are our own work.
