The Wallasey projectTwo hundred agents, one busy tool: retry storms and how to avoid them
Time a real API, then simulate a crowd of agents and compare three ways of trying again.
The learner first calls the Department for Transport's road traffic API for Wirral's 2024 records, politely, five times with a pause between: it returns 100 records and each call takes between 0.93 and 1.26 seconds, a median of 1.07. That becomes the answer time in a simulation. The rest is a stated assumption, not anything the DfT publishes: 200 agents all need an answer at once, the tool can accept 20 calls per second, and any call beyond that is refused straight away. Perfect coordination would finish in about 10 seconds plus the answer time.
Three retry rules are compared, each run 50 times. Immediate retry tries again every tenth of a second. Exponential backoff waits a tenth of a second after the first refusal, then doubles the wait each time, up to 20 seconds. Exponential backoff with jitter uses the same growing limit, but each agent picks a random wait anywhere up to that limit.
Two hundred simulated agents sharing a tool that accepts 20 calls a second, median of 50 runs, our Python simulation, 29 September 2026| Retry rule | Last agent served | Calls sent | Calls refused | Typical agent served |
|---|
| Retry immediately | 10.2 s | 6,300 | 6,100 | 5.6 s |
| Exponential backoff | 107.2 s | 1,640 | 1,440 | 20.5 s |
| Backoff with jitter | 19.6 s | 1,354 | 1,154 | 5.8 s |
Retrying immediately gets everyone served quickly, but only by sending 6,300 calls for 200 answers, more than 30 times the real need. A real service would likely block agents behaving like that. Plain exponential backoff cuts the calls to 1,640 but makes things far slower, because every refused agent waits exactly the same time and then returns at exactly the same moment: the whole crowd surges back together, gets refused together, and doubles its wait together. This is called a thundering herd, and here it pushes the last answer out to 107.2 seconds.
Adding jitter, a random share of the wait, breaks the lockstep. The returning agents spread out, the tool stays steadily busy instead of swamped and then idle, and the crowd is served with the fewest calls of all, 1,354, while the typical agent is answered in 5.8 seconds, almost as fast as the flood. One line of randomness turns the slowest strategy into the most balanced one.
Ages 8 to 11
Act out a crowd at a door that fits a few people at a time, then try taking turns at random.
Ages 11 to 15
Simulate ten agents and a slow tool in Python and count how many retries each rule sends.
Ages 15 and up
Build the full simulation, add backoff and jitter, and measure load against waiting time.
A real API timing, a simulated crowd
The response times were measured on the Department for Transport road traffic API with a handful of spaced-out calls. The crowd, the capacity and the retry rules are a simulation of our own design; no real service was put under load.
