Failures and antipatterns · Intermediate

Cache stampede: a burst of recomputation

Distinguish a hot-key miss burst from synchronized expiry across keys and bound regeneration work.

Ask Logeia about this articleTry the three questionsReview your answers

Updated

On this page
  1. A miss can trigger expensive work
  2. One popular key
  3. Spread expiry and choose staleness
  4. Protect and test the backend
  5. Ask Logeia about this articleCheck your understanding

A miss can trigger expensive work

Immediately after a cached value expires, many concurrent requests can miss and independently trigger the same database lookup or regeneration. The backend then receives a burst of work that ordinary cache hits had been hiding. This is one form of a cache stampede.

A burst on one hot key and synchronized expiry across many different keys are related but different patterns. Inspect misses per key, backend requests and concurrent regenerations, not only aggregate hit rate. A high overall hit rate can conceal a damaging burst on one expensive item.

One popular key

Imagine a ranking refreshed once a minute and requested by a thousand users at nearly the same moment. If every miss recomputes it, the system performs a thousand copies of work although one new result would suffice. The amplification follows from independent regeneration, not from a change in the ranking itself.

Single-flight behavior can coalesce regeneration for the same key. Define waiter deadlines, regeneration failures and what happens if the coordinating worker disappears. A lock that prevents duplicate work but leaves every waiter blocked indefinitely is not a satisfactory recovery design.

Spread expiry and choose staleness

Adding jitter to many keys’ expiration times can spread synchronized expiry. It does not by itself eliminate concurrent misses for one hot key, so match the remedy to the observed pattern. Coalescing and expiry spreading address different sources of amplification.

Serving an older value while refreshing is another option when the application can tolerate it. Define an explicit staleness budget. A slightly old ranking and stale account authorization have different correctness consequences. Do not use one blanket stale-serving policy simply because it improves the cache’s apparent availability.

Protect and test the backend

Test a cold cache and the moment of expiry deliberately. Measure whether backend requests stay within capacity; measuring only warm-cache latency omits the event that produces the failure. Include the operational case where a cache is purged completely.

Bound regeneration concurrency, queue length and waiter timeouts, and avoid letting failed refreshes produce synchronized retry bursts. Check the correctness and permitted age of served results as well as speed. A successful repair protects the source system while preserving the application’s actual data contract.

Reading and practice are recorded separately in this browser, for each language.

Learn this in order

Understand the systems behind the web · STEP 16

Choose a learning path