High-Concurrency Cache Hazards: Stampedes, Avalanches, Bloom Filters & Eviction Internals
A cache under high concurrency will experience cascading database failures unless hardened against stampedes, avalanches, and penetration. This article analyzes Singleflight coalescing, mathematically derives the XFetch probabilistic early expiration algorithm, implements Cuckoo and Bloom filters, and dissects Redis approximate LRU/LFU eviction pool internals.
Caching & Distributed Concurrency
High-Concurrency Cache Hazards: Stampedes, Avalanches, Bloom Filters & Eviction Internals
A cache is not an optimization layer you sprinkle over slow queries — it is a volatile, distributed state machine with independent failure modes and eventual consistency tradeoffs. Under high concurrency, those failure modes manifest not as graceful cache misses, but as cascading thundering herds and memory exhaustion that bring down authoritative databases. Mid-level implementations treat caching as an idealized black box: if a key exists, return it; if not, query PostgreSQL and store the result with a fixed TTL. In production under thousands of concurrent requests per second, this naive model collapses under three classic anomalies: Cache Breakdown, Cache Avalanche, and Cache Penetration. Hardening a system against these hazards requires mathematical precision — from probabilistic early expiration algorithms to dynamic Cuckoo filters and Redis memory allocator tuning.
This is Part 3 of the Caching & Distributed Concurrency series. It follows Part 1 — Caching Topologies and Part 2 — Cache Invalidation Strategies.
1. Taxonomy of Fatal Cache Access Anomalies
Engineers frequently conflate the three fatal cache access failure modes, applying the wrong architectural fix to the wrong problem:
| Anomaly | Root Cause Mechanism | Production Consequence |
|---|---|---|
| 💥 Cache Breakdown (Stampede / Dogpiling) |
A single ultra-hot key expires or is deleted while receiving heavy traffic | 1,000+ concurrent requests miss and simultaneously execute the origin query |
| 🌊 Cache Avalanche | Thousands of distinct keys share the exact same fixed TTL (e.g. 3600s) | Simultaneous bulk expiration causes an overwhelming global load spike on DB |
| 🕳️ Cache Penetration | Queries for non-existent entities bypass cache (attacker or bug) | 100% of requests miss cache and hit DB, saturating connection pools |
2. Cache Stampede (Thundering Herd) Defense Engines
When an ultra-hot cached entity (e.g., home page feed, flash-sale item) expires, every incoming thread misses the cache concurrently and rushes to recompute the data from the database. This is a Cache Stampede (also known as a Dogpile or Thundering Herd).
Three distinct engineering defenses exist to eliminate stampedes:
2.1 In-Process Singleflight (Promise Coalescing)
If 200 requests for product:100 hit Pod A within the same 10ms event-loop window, Pod A should execute the database query exactly once, sharing the resulting promise across all 200 awaiting callers.
2.2 Distributed Mutex Stampede Lock
While Singleflight coalesces requests within a single pod, a distributed cluster of 50 pods still sends 50 duplicate queries to PostgreSQL. To enforce a cluster-wide single execution, use a Distributed Mutex with Double-Checked Locking:
2.3 Probabilistic Early Expiration: The XFetch Algorithm
Distributed locks add network round-trips and retry loops. The optimal, lock-free approach is Probabilistic Early Expiration (the XFetch Algorithm), published by Vattani, Slivkins, and Chierichetti.
Instead of waiting for the key to reach its TTL and expiring abruptly, the XFetch algorithm computes an early recomputation probability that increases as the key approaches expiration and as computational cost ($\Delta$) increases.
$$\Delta \cdot \beta \cdot (-\ln(\text{rand}())) > (\text{TTL} - \text{now})$$
Where:
- $\Delta$ (delta): Execution duration (in seconds/ms) required to compute the origin query.
- $\beta$ (beta): Aggressiveness coefficient ($> 0$, default:
1.0; higher values recompute earlier). - $\text{rand}()$: Uniformly distributed pseudo-random float in the range $(0, 1]$.
- $\text{TTL} - \text{now}$: Remaining time-to-live of the cached item before hard expiration.

3. Cache Penetration Defense: Cuckoo vs. Bloom Filters
Cache Penetration occurs when queries for IDs that do not exist in the database bypass the cache completely, causing a 100% database miss rate. Attackers routinely weaponize penetration by scanning UUID sequences to exhaust database connection pools.
3.1 Bloom Filters vs. Cuckoo Filters
A Bloom Filter is a space-efficient probabilistic data structure that tests whether an element is definitely not in a set ($100%$ true negative) or might be in a set ($p$ false positive rate).
However, traditional Bloom filters do not support deletion. If a product is deleted from the database, the Bloom filter cannot remove the key without rebuilding the entire filter. Cuckoo Filters solve this limitation by using cuckoo hashing to support dynamic insertions and deletions.
| Feature | Standard Bloom Filter | Cuckoo Filter |
|---|---|---|
| Deletion Support | ❌ No (requires full rebuild) | ✅ Yes ($O(1)$ dynamic delete) |
| Space Efficiency | Optimal at ~1% false positive | Higher efficiency at $p < 3%$ |
| Lookup Performance | Checks $k$ independent hashes | Checks max 2 bucket locations |
| Memory Overhead | ~9.6 bits / item ($p = 0.01$) | ~12 bits / item ($p = 0.01$) |

4. Memory Eviction Internals & Allocator Forensics
When Redis memory usage exceeds maxmemory, Redis does not crash; instead, it executes its configured Eviction Policy to purge existing keys to make room for incoming writes.
4.1 Eviction Algorithms: True vs. Approximate
| Policy | Eviction Criteria | Bias / Blindspot | Recommended Use Case |
|---|---|---|---|
allkeys-lru |
Least Recently Used across all keys | Suffers from temporary sequential scan pollution | Standard OLTP web cache with power-law access |
allkeys-lfu |
Least Frequently Used (access counter + decay) | New hot keys may be evicted before accumulating frequency | Long-running cache with clear hot/cold data distributions |
volatile-ttl |
Shortest remaining time-to-live | Evicts items near expiry regardless of popularity | Caches where TTL strictly reflects business freshness |
noeviction |
Never evict; return out-of-memory error on write | Zero memory reclamation | Redis used as a primary store, queue, or state machine |
4.2 Why Redis Uses Approximate Sampled LRU
Standard LRU implementations use a doubly linked list and a hash map ($O(1)$ operations). However, maintaining linked-list pointers consumes 16 to 24 additional bytes of memory per key. In a Redis instance storing 50 million keys, pointer metadata alone consumes over 1 GB of RAM.
To conserve memory, Redis uses Approximate Sampled LRU:
- Every Redis object stores a 24-bit timestamp of its last access time (
lruclock). - When memory is full, Redis selects a random sample of $N$ keys (
maxmemory-samples, default: 5). - The oldest key in the sample is pushed into an eviction candidate pool (size 16) sorted by idle time.
- The key with the highest idle time in the pool is evicted.
Setting maxmemory-samples = 10 in redis.conf brings Redis approximate LRU within 99% accuracy of true mathematical LRU with negligible CPU impact.
4.3 Memory Fragmentation Ratio & jemalloc
Redis relies on the jemalloc memory allocator, which allocates RAM in fixed-size power-of-two arenas (e.g., 8B, 16B, 32B, 64B). When keys with varying payload sizes are repeatedly overwritten and deleted, memory becomes fragmented:
$$\text{mem_fragmentation_ratio} = \frac{\text{used_memory_rss}}{\text{used_memory}}$$
ratio < 1.0: The operating system has swapped Redis memory to disk (severe latency cliff).ratio between 1.0 and 1.5: Healthy, normal memory allocation state.ratio > 1.5: High fragmentation; Redis is consuming 50%+ more physical RAM than the data size, risking Linux OOM-killer termination.
Summary
| Concurrency Hazard | Mechanical Defense |
|---|---|
| Cache Breakdown (Stampede) | Deploy in-process Singleflight coalescing for local pods, and implement the XFetch probabilistic early expiration algorithm for lock-free background recomputation. |
| Cache Avalanche | Apply randomized TTL jitter ($\text{TTL} = \text{base} + \text{rand}(0, \text{jitter})$) to smooth out bulk expiration spikes across time. |
| Cache Penetration | Place a Cuckoo Filter in front of the cache to short-circuit non-existent entity queries in <0.2ms, and store short-lived __NULL__ sentinels for false positives. |
| Eviction Accuracy | Configure maxmemory-samples 10 and select allkeys-lfu for frequency-dominant workloads or allkeys-lru for recency-dominant OLTP traffic. |
| Memory Fragmentation | Enable activedefrag yes and replace blocking DEL commands on BigKeys with asynchronous UNLINK to prevent event-loop stalls. |
What's Next
Now that we have hardened our caching layer against high-concurrency access hazards, Part 4: Distributed Concurrency & Locking with Redis investigates multi-node distributed coordination: single-instance SETNX PX mechanics, lease watchdogs, the Redlock algorithm, and Martin Kleppmann's Monotonic Fencing Tokens.
This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.