Tied to Mission: Backend Depth — the caching gap from the baseline assessment.
Cache-aside pattern: app checks Redis, on miss queries Postgres, writes result back to Redis. Standard, and fine — until a hot key expires. 500 concurrent requests hit that key in the same instant. All 500 see a cache miss. All 500 query Postgres, for the same data, at once.
Cache stampede (also "thundering herd"). The failure mode is nasty because it's self-timing: it hits hardest exactly when a key is popular — the worst possible moment for the DB to take a load spike it wasn't sized for.
Cache-aside says nothing about coordination between concurrent misses. Each request independently decides "not in cache, go to DB" with zero awareness that 499 others just made the identical decision. The cache is protecting the DB from repeat traffic over time, not from simultaneous traffic at the instant of expiry.
| Technique | How it works | Tradeoff |
|---|---|---|
| Single-flight / mutex-per-key | First miss acquires a short-lived lock (SET key NX EX 5 in Redis). It computes and writes the value, then releases. Others wait briefly or serve a fallback. |
Simple, but waiting requests still add latency; lock expiry/cleanup needs care. |
| Probabilistic early expiration | Requests recompute slightly before TTL, with random jitter, so refreshes spread out instead of clustering at one instant. | Smooths load, but adds randomness to freshness — some keys refresh earlier than "needed." |
| Stale-while-revalidate | Serve the expired value immediately to all callers; exactly one background request refreshes it. | Zero latency hit for users, but they can see slightly stale data during refresh. |
"Scaling Memcache at Facebook" (NSDI '13) — the leases mechanism described there is single-flight at planet scale. See also Redis caching patterns for concrete implementation.
Questions on implementing single-flight with Redis specifically, or how this interacts with cache invalidation on writes? Ask directly.