The Rhyl projectFour bookings, one failure, and who cleans up
An agent with a multi-step job, four ways of handling failure, and a hundred thousand simulated day outs.
The scenario is invented, and so are its numbers. An agent is asked to arrange a group day out in Rhyl in four steps, each handled by a separate booking system: reserve a room, book a coach, buy attraction tickets, order lunch. We gave each step a chance of failing (3%, 8%, 12% and 5%), a 4% chance that a successful booking's confirmation gets lost on the way back, and a 10% chance that a cancellation request itself fails. No real business is modelled. With those numbers, all four steps succeed about three times in four, so roughly a quarter of all attempts fail somewhere along the way.
The learner runs 100,000 day outs under four policies. The first agent does nothing when a step fails: it reports the error and stops. The second cancels every earlier booking, once each. The third cancels and, if a cancellation fails, tries again, up to three times. The fourth does the same and also attaches an idempotency key to every request, a unique label that lets a booking system recognise a repeated request and return the original booking instead of making a second one.
What 100,000 simulated day outs leave behind, our Python run| Agent policy | Bookings left behind | Duplicate bookings |
|---|
| Stop on failure, undo nothing | 41,272 | 13,564 |
| Cancel earlier steps, one try each | 4,279 | 13,570 |
| Cancel, retrying up to three times | 40 | 13,444 |
| Cancel with retries and idempotency keys | 40 | 0 |
Doing nothing left 41,272 stray bookings across 22,495 failed day outs: rooms reserved and coaches booked for trips that never happened. Cancelling once cut that to 4,279, and retrying the cancellations cut it to 40. Duplicates are a separate problem with a separate fix. Whenever a confirmation was lost, the agent assumed the booking had failed and asked again, and without a key each retry made a second booking, about 13,500 of them over the whole run. Idempotency keys removed every one.
One more change costs nothing to try. If the riskiest step, the tickets, goes first, a failure there has nothing earlier to cancel. Re-ordering the steps from riskiest to safest cut the cancellation calls from 1.79 to 0.97 per failed day out, and the bookings left behind from 40 to 21. The success rate barely moves, since the same four steps still have to work; what changes is how much cleaning up a failure needs.
Ages 8 to 11
Plan a party in four steps on cards, then roll a dice to see which step fails and what must be undone.
Ages 11 to 15
Simulate the four steps in Python with random failures and count the bookings left behind.
Ages 15 and up
Add cancellations, retries and idempotency keys, then reorder the steps and measure what each change buys.
Where this comes from
The saga idea follows Garcia-Molina and Salem (1987). The day-out scenario, every failure rate, the seed (20261001) and all the counts above come from our own simulation; nothing on this page describes a real booking in Rhyl.
