A cache entry expiring is supposed to be the quiet, routine case — TTL did its job, the next reader just pays one cache-miss cost, as System Design's caching lesson described. Under real concurrent traffic, "the next reader" can mean thousands of simultaneous readers, and that's where the quiet case stops being quiet.
3 min read
A popular cache entry — a homepage's featured-products list, read by thousands of requests per second — expires. In the instant before any of those requests has repopulated it, every single one of those thousands of concurrent requests sees a cache miss simultaneously, and every one of them independently goes to the database to fetch the same data and repopulate the same cache key.
This is a cache stampede (also called the thundering herd problem, the same general phenomenon message queues' own retry lesson in this curriculum describes from a different angle): a cache that was absorbing near-total load from the database suddenly, briefly, absorbs none at all, and the full request volume that TTL was supposed to shield the database from arrives in one simultaneous spike instead.
def get_featured_products():
cached = cache.get("featured_products")
if cached is not None:
return cached
if cache.acquire_lock("featured_products:lock", ttl=5):
value = db.query_featured_products() # only ONE request reaches here
cache.set("featured_products", value, ttl=300)
cache.release_lock("featured_products:lock")
return value
else:
time.sleep(0.05)
return get_featured_products() # everyone else waits, then retries the cacheA distributed lock ensures only the first request to notice the miss actually queries the database; every other concurrent request waits briefly and then reads the now-repopulated cache instead of also hitting the database. This converts "N simultaneous database queries" into "1 database query plus N requests briefly waiting" — a real latency cost for those waiting requests, in exchange for not overwhelming the database at all.
Instead of every cache client treating a TTL as a hard cliff, each read that's getting close to an entry's expiration time probabilistically decides to refresh it early, with a probability that increases the closer the entry is to actually expiring. This means a hot key gets refreshed by one lucky early request well before the hard expiration most other concurrent requests would have hit simultaneously — spreading the refresh out over time instead of concentrating it at one instant.
Instead of: expire → ALL requests miss → ALL hit the database
Do: keep serving the STALE value to every request, while exactly
one background refresh fetches the new value and swaps it in
Stale-while-revalidate (a pattern HTTP's own Cache-Control header supports directly, connecting back to Cloud Computing's CDN lesson) serves the old, technically-expired value to every request while a single background process fetches a fresh one — no request ever sees a miss at all, at the cost of occasionally serving data that's a few seconds more stale than a strict TTL would allow. For most real use cases, a few extra seconds of staleness is a far smaller cost than a database overload incident.
Increasing the TTL only delays the next stampede — it doesn't change what happens the moment the entry does eventually expire under concurrent load, which remains exactly as sudden and simultaneous as before. The three fixes above each address the actual mechanism (simultaneous, uncoordinated repopulation), not just the frequency of the triggering event.
Check your understanding
A quick comprehension check — not tracked, not graded, just for you.
1. What triggers a cache stampede?
2. How does locking prevent a stampede?
3. Why doesn't simply raising the TTL actually fix the stampede risk?
Caching Strategies