When Retries Make Things Worse: Protecting Your System From Overload

A shipping quote service that slows from 100 milliseconds to two seconds can break an app even if it never returns a bad response. The danger isn’t errors; it’s patience. Requests keep piling up, workers stay busy, and retry logic keeps sending more traffic to a dependency that already has less room to breathe.

The article’s core point is that waiting is work. A request can tie up a worker, a connection, memory, or a pool slot while it sits on another service. Using the example numbers in the piece, 100 requests per second at 0.1 seconds per call needs about 10 concurrent calls. At 2 seconds, that jumps to about 200. If the pool only has 50 slots, the rest either wait or get rejected. CPU can look fine while the system quietly drowns in I/O.

Retries make that worse when they’re counted in the wrong place. One user action can become three attempts. Stack three retrying layers on top of each other and the math gets ugly fast: 3 x 3 x 3, or up to 27 provider attempts for one original request. The article argues that one place should own retries, and for a quote lookup that should be the provider adapter. Anything else, including SDK or proxy retries, has to be part of that decision or turned off.

Timeouts need the same discipline. A timeout on each attempt is not the same as a deadline for the whole operation, and three separate 500 millisecond waits can burn 1.5 seconds before backoff or connection setup even enter the picture. The better approach is to use the caller’s remaining budget, carry that budget across services, and stop when there isn’t enough time left for another useful try. The article also warns that a timeout only covers what the transport timeout actually covers; DNS, TLS, pool waiting, and reads can each behave differently.

Then there’s admission control. A concurrency limit keeps too many attempts from starting in the first place, while a rate limit only controls how often they start. For the quote provider, the article wants an admission check before connection acquisition, plus separate capacity for unrelated work so one overloaded dependency doesn’t steal everything else’s lunch. It also recommends a retry budget across requests, not just a per-request cap, so a flood of original traffic can’t turn into a flood of retries.

The final shape is pretty clear: one retry owner, one deadline, bounded retries, and no sleeping while holding a dependency slot. The sample PHP code is less interesting as code than as a reminder that overload handling is a policy choice, not a hopeful shrug.