Design patterns and optimization · Intermediate

Bound additional load with a retry budget

Use original requests, extra-attempt allowances and multilayer amplification to distinguish count limits, retry timing and idempotency.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. Bound additional attempts across original requests
  2. Calculate amplification and consume before retrying
  3. Define scope and replenishment
  4. Combine counts with spacing and deadlines
  5. Measure original outcomes and extra work together
  6. Attempt counts and work cost are different units
  7. Solve the remaining-budget exercise
  8. Ask Logeia about this articleCheck your understanding

Bound additional attempts across original requests

A missing order response can trigger a client retry while intermediate services also retry independently. Downstream attempts can multiply. A retry budget bounds extra attempts relative to original traffic. Read idempotency and backoff first. Even an idempotent request consumes computation, connections and locks on every attempt. Preventing duplicate effects and limiting amplification are separate responsibilities.

Use one hundred original requests and at most twenty additional attempts. If twenty requests each retry once, total attempts are 120. Decide separately whether one request may consume the entire shared allowance or also has a per-request cap. Define whether three attempts includes the original call or means three retries. Ambiguous counting conventions compound when applied independently at several layers. The useful contract states original work, extra work and the scope over which both are counted.

Calculate amplification and consume before retrying

For a three-layer chain, if each layer permits up to three attempts of its downstream operation, one original action can induce as many as 27 deepest-layer attempts. One designated retry layer instead bounds that part to three. Twenty-seven is a possible maximum history, not a prediction for every failure. Specify the retry owner and how other layers propagate failure.

Consume a shared allowance before starting an extra attempt. When twenty extra tokens are exhausted, return the appropriate failure or unknown-result state. Budget exhaustion does not prove the business operation failed; a response may have been lost after application. Status checks and result retrieval also consume downstream resources and must not become an uncounted alternate retry channel. Require both remaining deadline and available retry allowance before starting more work. These are distinct constraints, and satisfying one does not imply satisfying the other.

PolicyMaximum candidate workload
Three layers, three attempts each27 deepest attempts per original
One retry layer, three attempts3 attempts per original
100 originals plus 20 extras120 attempts overall

Define scope and replenishment

Budgets can be scoped by dependency, caller or time window. A globally coordinated counter is not always required, but a per-process budget scales its total allowance with process count. Window resets can synchronize bursts. Token replenishment based on original successful traffic needs explicit initialization, minimum allowance and maximum balance. A mechanism's name alone does not define its aggregate load.

Do not count failed retries again as new originals, or retries can replenish their own allowance. Propagate original-request identity and use one classification rule. Low traffic may justify a small fixed allowance when a pure ratio yields none, but multiplying that allowance across many processes can create a large outage load. Express the availability objective as concrete attempt counts and a time scope. This turns an attractive percentage into a policy whose worst-case effect can actually be reviewed.

Combine counts with spacing and deadlines

Backoff spaces attempts and jitter spreads callers over time, but neither alone bounds attempts during a long outage. Budget controls count; jitter controls timing. Interpret overload responses and any Retry-After semantics under the actual protocol. Authentication and invalid-input errors should not be retried as though waiting necessarily fixes them.

Do not start an attempt when remaining time cannot cover its delay and useful execution, even if tokens remain. Cancellation of already sent work is another contract: stopping the wait does not necessarily undo an applied effect. Specify how unknown results reach the caller. Combine stable operation identity, payload matching, deadline and error classification with the budget. Observe the reason a retry was rejected, rather than describing every refusal as a downstream failure. The policy limits amplification while preserving the distinctions needed for correct recovery.

Measure original outcomes and extra work together

Track originals, extra attempts, successful retries, budget refusals, deadline refusals and downstream waits separately. A high retry success fraction can still harm overall completion if it crowds out original requests. Evaluate refusal changes alongside dependency recovery and total useful completion. Choose the identifiers needed to reconstruct failures without logging every payload unnecessarily.

Verifying illustrative counts of 27 and 120 does not prove that twenty percent is the right production allowance. Capacity, failure type and user deadline determine that policy. A small fixed cap may suffice when failures are rare or final. Compare the operational cost of a shared budget with that simpler alternative. The outcome is protection of useful original work during failure, not merely installing another mechanism. Preserve the declared scope so observed numbers can be compared with the intended bound.

Attempt counts and work cost are different units

Twenty extra lookups and twenty expensive aggregations can consume different resources. Decide whether a count-only budget fits the workload or whether classes and dependencies need separate allowances. A cost-weighted policy changes the budget unit and needs its own estimation assumptions. Preserve original-versus-extra classification in either form. The acceptance record should say what was bounded, over what interval, and how an expensive attempt was charged. Otherwise a stable attempt ratio can coexist with severe resource amplification. This does not imply every system needs complex cost accounting; it identifies the condition under which the simpler count example should be reconsidered.

Solve the remaining-budget exercise

With one hundred originals, twenty extra tokens and seventeen already spent, five new retry requests can start at most three extra attempts in the same unreplenished window. Record the other two as budget refusals. Selecting the admitted three requires its own fairness or priority policy. Per-request permission to retry once would not preserve this aggregate cap.

If ten processes independently receive twenty tokens, total extra allowance can reach 200. A global requirement of twenty needs allocation or coordination; jitter does not reduce 200 to twenty. Explain count, time scope and original identity together, then continue to the lost-response lab. That executable example demonstrates why retries arise and why one business effect needs stable operation identity in addition to a load budget.

Reading and practice are recorded separately in this browser, for each language.

Learn this in order

Understand the systems behind the web · STEP 16

Choose a learning path