Siddhant Deval
Siddhant DevalAuthor
Senior Full-Stack Engineer·Sep 27, 2026·22 min read

High-Concurrency Cache Hazards: Stampedes, Avalanches, Bloom Filters & Eviction Internals

A cache under high concurrency will experience cascading database failures unless hardened against stampedes, avalanches, and penetration. This article analyzes Singleflight coalescing, mathematically derives the XFetch probabilistic early expiration algorithm, implements Cuckoo and Bloom filters, and dissects Redis approximate LRU/LFU eviction pool internals.

High-Concurrency Cache Hazards: Stampedes, Avalanches, Bloom Filters & Eviction Internals

A cache is not an optimization layer you sprinkle over slow queries — it is a volatile, distributed state machine with independent failure modes and eventual consistency tradeoffs. Under high concurrency, those failure modes manifest not as graceful cache misses, but as cascading thundering herds and memory exhaustion that bring down authoritative databases. Mid-level implementations treat caching as an idealized black box: if a key exists, return it; if not, query PostgreSQL and store the result with a fixed TTL. In production under thousands of concurrent requests per second, this naive model collapses under three classic anomalies: Cache Breakdown, Cache Avalanche, and Cache Penetration. Hardening a system against these hazards requires mathematical precision — from probabilistic early expiration algorithms to dynamic Cuckoo filters and Redis memory allocator tuning.

Architectural Note

This is Part 3 of the Caching & Distributed Concurrency series. It follows Part 1 — Caching Topologies and Part 2 — Cache Invalidation Strategies.


1. Taxonomy of Fatal Cache Access Anomalies

Engineers frequently conflate the three fatal cache access failure modes, applying the wrong architectural fix to the wrong problem:

Anomaly Root Cause Mechanism Production Consequence
💥 Cache Breakdown
(Stampede / Dogpiling)
A single ultra-hot key expires or is deleted while receiving heavy traffic 1,000+ concurrent requests miss and simultaneously execute the origin query
🌊 Cache Avalanche Thousands of distinct keys share the exact same fixed TTL (e.g. 3600s) Simultaneous bulk expiration causes an overwhelming global load spike on DB
🕳️ Cache Penetration Queries for non-existent entities bypass cache (attacker or bug) 100% of requests miss cache and hit DB, saturating connection pools
TYPESCRIPT
// ❌ Broken pattern: Naive cache-aside vulnerable to all three failure modes
export async function getProductDetails(productId: string) {
  // 1. Vulnerable to Penetration: Non-existent IDs always miss and hit DB
  const cached = await redis.get(`product:${productId}`);
  if (cached) return JSON.parse(cached);

  // 2. Vulnerable to Stampede: If this hot product expires, 5,000 concurrent requests
  // all execute this expensive multi-table JOIN simultaneously!
  const product = await db.product.findUnique({
    where: { id: productId },
    include: { reviews: true, inventory: true, seller: true },
  });

  // 3. Vulnerable to Avalanche: Setting a rigid 3600s TTL causes bulk keys to expire together
  if (product) {
    await redis.set(`product:${productId}`, JSON.stringify(product), 'EX', 3600);
  }
  return product;
}

2. Cache Stampede (Thundering Herd) Defense Engines

When an ultra-hot cached entity (e.g., home page feed, flash-sale item) expires, every incoming thread misses the cache concurrently and rushes to recompute the data from the database. This is a Cache Stampede (also known as a Dogpile or Thundering Herd).

Three distinct engineering defenses exist to eliminate stampedes:

2.1 In-Process Singleflight (Promise Coalescing)

If 200 requests for product:100 hit Pod A within the same 10ms event-loop window, Pod A should execute the database query exactly once, sharing the resulting promise across all 200 awaiting callers.

TYPESCRIPT
// ✅ Production Singleflight Group in TypeScript
export class SingleflightGroup<T> {
  private inFlight = new Map<string, Promise<T>>();

  async do(key: string, fn: () => Promise<T>): Promise<T> {
    const existing = this.inFlight.get(key);
    if (existing) {
      // Coalesce onto the in-flight execution (0 extra DB queries)
      return existing;
    }

    const promise = (async () => {
      try {
        return await fn();
      } finally {
        this.inFlight.delete(key);
      }
    })();

    this.inFlight.set(key, promise);
    return promise;
  }
}

2.2 Distributed Mutex Stampede Lock

While Singleflight coalesces requests within a single pod, a distributed cluster of 50 pods still sends 50 duplicate queries to PostgreSQL. To enforce a cluster-wide single execution, use a Distributed Mutex with Double-Checked Locking:

TYPESCRIPT
// ✅ Distributed Mutex for Cache Repopulation
export async function getWithDistributedMutex(key: string, ttlSec = 300): Promise<string> {
  let val = await redis.get(key);
  if (val) return val;

  const lockKey = `lock:${key}`;
  const lockToken = crypto.randomUUID();
  
  // Try to acquire distributed lock with 5-second lease
  const acquired = await redis.set(lockKey, lockToken, 'NX', 'PX', 5000);

  if (acquired) {
    try {
      // Re-check cache inside lock in case another worker populated it
      val = await redis.get(key);
      if (val) return val;

      const fresh = await computeExpensiveDatabaseQuery();
      await redis.set(key, fresh, 'EX', ttlSec);
      return fresh;
    } finally {
      // Atomic Lua release to guarantee we only release our own lock
      const luaScript = `
        if redis.call("get", KEYS[1]) == ARGV[1] then
          return redis.call("del", KEYS[1])
        else
          return 0
        end
      `;
      await redis.eval(luaScript, 1, lockKey, lockToken);
    }
  } else {
    // Lock contention: Wait 50ms and retry from cache
    await new Promise((resolve) => setTimeout(resolve, 50));
    return getWithDistributedMutex(key, ttlSec);
  }
}

2.3 Probabilistic Early Expiration: The XFetch Algorithm

Distributed locks add network round-trips and retry loops. The optimal, lock-free approach is Probabilistic Early Expiration (the XFetch Algorithm), published by Vattani, Slivkins, and Chierichetti.

Instead of waiting for the key to reach its TTL and expiring abruptly, the XFetch algorithm computes an early recomputation probability that increases as the key approaches expiration and as computational cost ($\Delta$) increases.

$$\Delta \cdot \beta \cdot (-\ln(\text{rand}())) > (\text{TTL} - \text{now})$$

Where:

  • $\Delta$ (delta): Execution duration (in seconds/ms) required to compute the origin query.
  • $\beta$ (beta): Aggressiveness coefficient ($> 0$, default: 1.0; higher values recompute earlier).
  • $\text{rand}()$: Uniformly distributed pseudo-random float in the range $(0, 1]$.
  • $\text{TTL} - \text{now}$: Remaining time-to-live of the cached item before hard expiration.
TYPESCRIPT
// ✅ Production XFetch Probabilistic Early Expiration Implementation
interface CachedEnvelope<T> {
  data: T;
  delta: number;   // Time taken to compute in milliseconds
  expiry: number;  // Absolute unix timestamp (ms)
}

export async function getWithXFetch<T>(
  key: string,
  ttlMs: number,
  computeFn: () => Promise<T>,
  beta = 1.0
): Promise<T> {
  const raw = await redis.get(key);
  const now = Date.now();

  if (raw) {
    const envelope: CachedEnvelope<T> = JSON.parse(raw);
    const timeRemaining = envelope.expiry - now;

    // XFetch Decision Rule: Probabilistically trigger early background refresh
    const shouldRecomputeEarly = 
      (envelope.delta * beta * -Math.log(Math.random())) > timeRemaining;

    if (!shouldRecomputeEarly) {
      return envelope.data;
    }

    // Trigger asynchronous background recomputation without blocking caller
    recomputeAndCache(key, ttlMs, computeFn).catch(console.error);
    return envelope.data;
  }

  // Hard cache miss: compute synchronously
  return await recomputeAndCache(key, ttlMs, computeFn);
}

async function recomputeAndCache<T>(key: string, ttlMs: number, computeFn: () => Promise<T>): Promise<T> {
  const start = Date.now();
  const freshData = await computeFn();
  const delta = Date.now() - start;
  const expiry = Date.now() + ttlMs;

  const envelope: CachedEnvelope<T> = { data: freshData, delta, expiry };
  // Set physical Redis TTL higher than envelope expiry to allow SWR recomputation window
  await redis.set(key, JSON.stringify(envelope), 'PX', ttlMs * 2);
  return freshData;
}
Comparison of Cache Stampede mitigation strategies evaluating Database Load, P99 Latency, and Implementation Complexity.
Comparison of Cache Stampede mitigation strategies evaluating Database Load, P99 Latency, and Implementation Complexity.

3. Cache Penetration Defense: Cuckoo vs. Bloom Filters

Cache Penetration occurs when queries for IDs that do not exist in the database bypass the cache completely, causing a 100% database miss rate. Attackers routinely weaponize penetration by scanning UUID sequences to exhaust database connection pools.

3.1 Bloom Filters vs. Cuckoo Filters

A Bloom Filter is a space-efficient probabilistic data structure that tests whether an element is definitely not in a set ($100%$ true negative) or might be in a set ($p$ false positive rate).

However, traditional Bloom filters do not support deletion. If a product is deleted from the database, the Bloom filter cannot remove the key without rebuilding the entire filter. Cuckoo Filters solve this limitation by using cuckoo hashing to support dynamic insertions and deletions.

Feature Standard Bloom Filter Cuckoo Filter
Deletion Support ❌ No (requires full rebuild) ✅ Yes ($O(1)$ dynamic delete)
Space Efficiency Optimal at ~1% false positive Higher efficiency at $p < 3%$
Lookup Performance Checks $k$ independent hashes Checks max 2 bucket locations
Memory Overhead ~9.6 bits / item ($p = 0.01$) ~12 bits / item ($p = 0.01$)
TYPESCRIPT
// ✅ Production Penetration Defense: RedisBloom Cuckoo Filter + Null Sentinel
export async function getProductProtected(productId: string): Promise<Product | null> {
  const cacheKey = `product:${productId}`;

  // 1. Query L2 Cache
  const cached = await redis.get(cacheKey);
  if (cached) {
    if (cached === '__NULL__') return null; // Null-Object Sentinel
    return JSON.parse(cached);
  }

  // 2. Query Cuckoo Filter via RedisBloom module (CF.EXISTS)
  const existsInFilter = await redis.send_command('CF.EXISTS', 'cf:products', productId);
  if (!existsInFilter) {
    // Fast path: Key is guaranteed not to exist in PostgreSQL. Return null in 0.2ms!
    return null;
  }

  // 3. Probabilistic pass: Query PostgreSQL
  const product = await db.product.findUnique({ where: { id: productId } });

  if (product) {
    await redis.set(cacheKey, JSON.stringify(product), 'EX', 3600);
  } else {
    // 4. False positive protection: Cache short-lived Null Sentinel (60s)
    await redis.set(cacheKey, '__NULL__', 'EX', 60);
  }

  return product;
}
Bloom and Cuckoo filter penetration defense pipeline alongside Redis approximate sampled LRU candidate pool scoring.
Bloom and Cuckoo filter penetration defense pipeline alongside Redis approximate sampled LRU candidate pool scoring.

4. Memory Eviction Internals & Allocator Forensics

When Redis memory usage exceeds maxmemory, Redis does not crash; instead, it executes its configured Eviction Policy to purge existing keys to make room for incoming writes.

4.1 Eviction Algorithms: True vs. Approximate

Policy Eviction Criteria Bias / Blindspot Recommended Use Case
allkeys-lru Least Recently Used across all keys Suffers from temporary sequential scan pollution Standard OLTP web cache with power-law access
allkeys-lfu Least Frequently Used (access counter + decay) New hot keys may be evicted before accumulating frequency Long-running cache with clear hot/cold data distributions
volatile-ttl Shortest remaining time-to-live Evicts items near expiry regardless of popularity Caches where TTL strictly reflects business freshness
noeviction Never evict; return out-of-memory error on write Zero memory reclamation Redis used as a primary store, queue, or state machine

4.2 Why Redis Uses Approximate Sampled LRU

Standard LRU implementations use a doubly linked list and a hash map ($O(1)$ operations). However, maintaining linked-list pointers consumes 16 to 24 additional bytes of memory per key. In a Redis instance storing 50 million keys, pointer metadata alone consumes over 1 GB of RAM.

To conserve memory, Redis uses Approximate Sampled LRU:

  1. Every Redis object stores a 24-bit timestamp of its last access time (lru clock).
  2. When memory is full, Redis selects a random sample of $N$ keys (maxmemory-samples, default: 5).
  3. The oldest key in the sample is pushed into an eviction candidate pool (size 16) sorted by idle time.
  4. The key with the highest idle time in the pool is evicted.
Pro Tip & Optimization

Setting maxmemory-samples = 10 in redis.conf brings Redis approximate LRU within 99% accuracy of true mathematical LRU with negligible CPU impact.

4.3 Memory Fragmentation Ratio & jemalloc

Redis relies on the jemalloc memory allocator, which allocates RAM in fixed-size power-of-two arenas (e.g., 8B, 16B, 32B, 64B). When keys with varying payload sizes are repeatedly overwritten and deleted, memory becomes fragmented:

$$\text{mem_fragmentation_ratio} = \frac{\text{used_memory_rss}}{\text{used_memory}}$$

  • ratio < 1.0: The operating system has swapped Redis memory to disk (severe latency cliff).
  • ratio between 1.0 and 1.5: Healthy, normal memory allocation state.
  • ratio > 1.5: High fragmentation; Redis is consuming 50%+ more physical RAM than the data size, risking Linux OOM-killer termination.
INI
# redis.conf: Enable Active Memory Defragmentation
activedefrag yes
active-defrag-ignore-bytes 100mb
active-defrag-threshold-lower 10
active-defrag-threshold-upper 30
active-defrag-cycle-min 5
active-defrag-cycle-max 50

Summary

Concurrency Hazard Mechanical Defense
Cache Breakdown (Stampede) Deploy in-process Singleflight coalescing for local pods, and implement the XFetch probabilistic early expiration algorithm for lock-free background recomputation.
Cache Avalanche Apply randomized TTL jitter ($\text{TTL} = \text{base} + \text{rand}(0, \text{jitter})$) to smooth out bulk expiration spikes across time.
Cache Penetration Place a Cuckoo Filter in front of the cache to short-circuit non-existent entity queries in <0.2ms, and store short-lived __NULL__ sentinels for false positives.
Eviction Accuracy Configure maxmemory-samples 10 and select allkeys-lfu for frequency-dominant workloads or allkeys-lru for recency-dominant OLTP traffic.
Memory Fragmentation Enable activedefrag yes and replace blocking DEL commands on BigKeys with asynchronous UNLINK to prevent event-loop stalls.

What's Next

Now that we have hardened our caching layer against high-concurrency access hazards, Part 4: Distributed Concurrency & Locking with Redis investigates multi-node distributed coordination: single-instance SETNX PX mechanics, lease watchdogs, the Redlock algorithm, and Martin Kleppmann's Monotonic Fencing Tokens.

Research & Synthesis Note

This article was developed with AI-assisted deep search, specification cross-referencing, and technical research synthesis.

#Performance#Cache Stampede#Bloom Filter#Redis Internals#Reliability
Siddhant Deval

Written by Siddhant Deval

Senior Full-Stack Engineer building high-scale architectures, browser performance engineering systems, and SaaS platforms.