Exponential backoff
- Standard term
Exponential backoff is a way of retrying after a failure where each wait is longer than the last, often doubling, with a little randomness added and a limit on attempts.
Many failures clear within seconds, such as a briefly overloaded server. Retrying at once adds load just when the service can least handle it. The programs sending it requests, called “clients,” often failed at the same moment, so they retry at the same moment too. Doubling the wait thins the retries quickly, and a random extra wait, called “jitter,” spreads them apart.
Where you’ll meet it
Your coding agent wrote a script that sends 2,000 product descriptions to an AI service for rewriting. Partway through, the service starts answering “too many requests.” The script retries each refused one immediately, every retry is refused too, and the run stalls.
What to do
Ask the agent to add exponential backoff with jitter, and to honor the wait time the service's refusal names when it gives one. Have it save the descriptions that still fail after the last try to a file, ready for a later rerun.
Easy to mix up
| Exponential backoff | Rate limit |
|---|---|
| The client, deciding how long to wait | The service, capping how many requests it accepts |
Check yourself
Your agent's request to post an invoice fails with “account not found.” Its backoff policy allows five retries. What should it do?
Go deeperA retry can repeat finished work, and retries multiply across layers
Early Ethernet used binary exponential backoff after two machines transmitted at once. A 2015 AWS Architecture Blog post compared jitter methods and found that choosing each wait at random, anywhere up to the current limit, cut clients' total calls below plain doubling. A cap on each wait and an overall deadline keep the doubling from running to hours.
A charge or email can go out twice if the first attempt succeeded and only its reply was lost. With an idempotency key, an ID sent with the request, the service spots a repeat and carries it out once. Retries stacked inside each other multiply. If a script, the code library it calls, and the service behind it each try four times, one failure at the bottom becomes 64 calls.