Treat another attempt as another unit of load
Retries can recover from transient failures, but immediate retries against an overloaded service can make recovery harder. If several layers each retry independently, downstream attempts can multiply rather than merely add. Establish which layer owns the retry policy and cap its attempt budget. The goal is to recover useful operations without turning a temporary shortage into a larger request storm.
Increase a capped waiting interval
With a base interval of one hundred milliseconds, exponential backoff can produce ceilings of one hundred, two hundred, and four hundred milliseconds. Add a maximum ceiling so the wait does not grow without bound. Different implementations number attempts differently, so verify the first actual wait instead of relying only on a formula. A cap limits an individual wait; it does not by itself limit the total duration or total number of attempts.
Spread clients across time with jitter
If many clients retry at exactly the same intervals, they can send synchronized bursts just as a service starts recovering. Full jitter chooses a random delay between zero and the ceiling for that attempt. A four-hundred-millisecond ceiling therefore produces different waits across clients. Randomization aims to spread collective demand; it does not guarantee that every individual request becomes faster. Inject a controllable random source when testing boundary behavior and deadline calculations.
Respect the overall deadline and cancellation
Use an operation-wide deadline as well as per-attempt timeouts. A plan to sleep for two seconds is pointless when only one hundred milliseconds remain. Stop work when the upstream operation is canceled, and account for time already spent waiting or executing previous attempts. If a server provides Retry-After guidance, interpret it together with the remaining deadline and the API’s documented retry contract rather than treating it as permission to wait indefinitely.
Choose eligible failures and observe the cost
Invalid input and denied authorization usually do not improve when the same request is repeated. An uncertain write also needs an idempotency guarantee before automatic retrying is safe. Sustained overload may require admission control or a circuit breaker instead of a larger retry budget. Measure attempted requests, retry counts, completion latency, and deadline expirations together. A higher eventual success rate can conceal an unacceptable increase in backend work and user waiting.
Define an attempt budget across layers
“Three tries” can mean three total attempts or an initial attempt plus three retries. Define the contract explicitly; this example includes the initial attempt in the total of three. If each of three nested layers independently allows three attempts, the deepest service can receive as many as 3³=27 calls for one originating operation. Small local budgets can therefore multiply into substantial downstream load. Prefer a clearly owned retry policy, or propagate a remaining deadline and a shared attempt budget when ownership cannot be centralized. Record both the originating operation identity and the attempt number. This makes amplification observable and prevents a local success percentage from concealing the cost imposed on another service.
Allocate the remaining deadline
Use an illustrative one-second operation deadline, a 250 ms limit per attempt, and waits of 100 ms and 200 ms. If the first attempt consumes 250 ms, the second starts at 350 ms. Another 250 ms failure reaches 600 ms; the next wait moves the third start to 800 ms. Only 200 ms remain, so assigning another full 250 ms attempt would violate the overall budget. Pass the remaining time to an interruptible operation or decline another attempt when useful completion is unlikely. Check whether name resolution and connection setup are included in the same deadline. Use a monotonic clock for elapsed-time accounting so wall-clock corrections do not unexpectedly extend or shorten a running operation. The numbers illustrate budgeting, not a measured latency distribution.
Distinguish the ceiling from the mean wait
For ideal uniform full jitter between zero and a ceiling c, expected waiting time is c/2. A 400 ms ceiling gives a 200 ms average, not a guarantee that any particular request waits 200 ms. Changing from a fixed 400 ms delay also changes average waiting time, so a comparison intended to isolate synchronization effects should state which quantities are held constant. A controllable random source supports deterministic boundary tests. In production, however, giving every client the same seed and call sequence can synchronize their waits again. Check the delay range, maximum ceiling, total attempts and overall deadline as separate properties. No single cap automatically enforces all of these constraints, and a randomized policy still consumes backend capacity.
Classify the failure before choosing a delay
An HTTP 429 or 503 response is not a universal instruction to repeat at a hard-coded interval. Inspect the API contract, Retry-After guidance and remaining deadline. Retry-After can contain a delay in seconds or an HTTP date; a date additionally raises clock-alignment questions. Repeating an unauthorized request with unchanged credentials generally does not repair authorization. An uncertain write needs particular care: the order might have been saved before its response was lost. Exponential backoff cannot prevent creating the order twice. Use the same business operation identity and a result lookup or atomic idempotency record. Separate failure eligibility, duplicate-effect safety and load-management delay into distinct decisions rather than treating every error as the same retry problem.