Caching Topologies: Client, CDN, In-Memory & Distributed Redis Architecture

A cache is not an optimization layer you sprinkle over slow queries — it is a volatile, distributed state machine with independent failure modes and eventual consistency tradeoffs. Every cache hit is a loan against consistency; every cache miss is a potential stampede. The most pervasive broken pattern in backend engineering is treating caching as a monolithic, single-node key-value store slapped between an API service and a PostgreSQL database. Engineers configure a basic Redis instance, wrap database queries in redis.get() and redis.set(), and assume their data access layer is scaled. In reality, a resilient architecture is a coordinated multi-tier hierarchy spanning edge CDNs, in-process memory, and distributed clusters. When you fail to design these tiers explicitly, adding caches does not solve latency — it simply multiplies your stale-read failure domains.

1. The Multi-Tier Caching Continuum

Every millisecond of latency saved is governed by where data lives relative to the execution runtime. Caching is not a binary choice between "in memory" and "on disk" — it is a continuous spectrum spanning physical hardware, network boundaries, and process memory spaces.

// ❌ Broken pattern: Treating Redis as the sole caching layer for ultra-hot static reads
export async function getProductCatalog(tenantId: string) {
  // 5,000 requests/sec hitting this endpoint generate 5,000 TCP round-trips to Redis.
  // Network interface card (NIC) saturation and Redis single-threaded CPU bottleneck quickly emerge.
  const cached = await redis.get(`catalog:${tenantId}`);
  if (cached) return JSON.parse(cached);

  const catalog = await db.catalog.findMany({ where: { tenantId } });
  await redis.set(`catalog:${tenantId}`, JSON.stringify(catalog), 'EX', 300);
  return catalog;
}

// ✅ Resilient pattern: Tiered access — In-Process L1 (heap) short-circuits Redis L2 network I/O
import { LRUCache } from 'lru-cache';

const l1Cache = new LRUCache<string, CatalogPayload>({
  max: 500,
  ttl: 1000 * 10, // 10s local TTL to limit pod split-brain drift
});

export async function getProductCatalogTiered(tenantId: string): Promise<CatalogPayload> {
  const key = `catalog:${tenantId}`;
  
  // Tier 1: Sub-microsecond local heap lookup (0 network hops)
  const l1Hit = l1Cache.get(key);
  if (l1Hit) return l1Hit;

  // Tier 2: Millisecond distributed Redis lookup (1 network hop)
  const l2Hit = await redis.get(key);
  if (l2Hit) {
    const parsed = JSON.parse(l2Hit);
    l1Cache.set(key, parsed);
    return parsed;
  }

  // Tier 3: Authoritative Database query (Origin)
  const catalog = await db.catalog.findMany({ where: { tenantId } });
  await redis.set(key, JSON.stringify(catalog), 'EX', 300);
  l1Cache.set(key, catalog);
  return catalog;
}

1.1 The Latency & Storage Spectrum

Each caching tier operates under distinct physical constraints:

Expand

1.2 The Failure Cascade

When an upper tier suffers a high miss rate or an uncoordinated invalidation event, traffic immediately falls through to lower tiers. Because lower tiers have orders-of-magnitude smaller concurrency tolerances than upper tiers, a 10% drop in CDN cache hit ratio can increase database load by 500%, causing connection pool exhaustion. Designing a multi-tier topology requires setting capacity limits at every boundary.

2. HTTP Caching & Edge Control: RFC 9111, SWR & Surrogate-Keys

The fastest request is the one that never touches your origin infrastructure. HTTP caching at the CDN edge (Cloudflare, Fastly, AWS CloudFront) is your first line of defense, but standard Cache-Control: max-age=300 is fundamentally unsuited for dynamic API payloads where data freshness matters.

2.1 Modern Directives: s-maxage, stale-while-revalidate & immutable

HTTP specifications (RFC 9111 and RFC 5861) allow fine-grained decoupling between browser caches, shared CDN caches, and background revalidation routines.

HTTP/1.1 200 OK
Content-Type: application/json
Cache-Control: public, max-age=0, s-maxage=300, stale-while-revalidate=60, stale-if-error=86400
Surrogate-Control: max-age=86400
Surrogate-Key: tenant_42 product_9871 catalog_feed
ETag: W/"d41d8cd98f00b204e9800998ecf8427e"

• max-age=0: Instructs browser clients not to serve from local disk without checking the origin/CDN.
• s-maxage=300: Instructs shared intermediate proxies (CDNs) to consider the asset fresh for 5 minutes.
• stale-while-revalidate=60 (RFC 5861): When the 5-minute s-maxage expires, the CDN edge serves the stale cached payload immediately to the client within the 60-second window while firing an asynchronous background sub-request to the origin to refresh the cache.
• stale-if-error=86400: If the origin returns a 5xx error or times out, the CDN serves the stale payload for up to 24 hours rather than returning an error page to the user.

// ✅ Express / Fastify Middleware for Dynamic SWR Edge Headers
import { Request, Response, NextFunction } from 'express';

export function setEdgeCachingHeaders(options: {
  freshSec: number;
  swrSec: number;
  tags?: string[];
}) {
  return (_req: Request, res: Response, next: NextFunction) => {
    const { freshSec, swrSec, tags } = options;
    
    res.setHeader(
      'Cache-Control',
      `public, max-age=0, s-maxage=${freshSec}, stale-while-revalidate=${swrSec}, stale-if-error=86400`
    );

    if (tags && tags.length > 0) {
      // Cloudflare uses 'Cache-Tag', Fastly/Akamai use 'Surrogate-Key'
      res.setHeader('Surrogate-Key', tags.join(' '));
      res.setHeader('Cache-Tag', tags.join(','));
    }

    next();
  };
}

2.2 Instant Edge Purging with Surrogate-Keys

A long CDN TTL gives near-100% cache hit ratios, but updating state (e.g., price drop, product stock change) renders the edge stale. Polling or short TTLs waste origin CPU. The production pattern is Surrogate-Key (Cache-Tag) indexing:

1. When rendering GET /api/v1/products/9871, the origin tags the response with Surrogate-Key: product_9871 tenant_42.
2. The CDN indexes the cached edge object under both tags.
3. When an administrator updates Product 9871, the backend issues an asynchronous API call to the CDN's Purge API:
   BASH

   Copy

   # Instant global purge across thousands of CDN PoPs (<150ms)
   curl -X POST "https://api.cloudflare.com/client/v4/zones/${ZONE_ID}/purge_cache" \
     -H "Authorization: Bearer ${CDN_API_TOKEN}" \
     -H "Content-Type: application/json" \
     -d '{"tags": ["product_9871"]}'
   

2.3 The Vary Header Trap

A major source of CDN cache misses and memory bloat is improper use of the Vary header. When you specify Vary: User-Agent or Vary: Cookie, the CDN generates a completely distinct cache entry for every unique browser version or session token.

3. In-Process L1 vs. Distributed L2: The Pod Split-Brain Problem

In high-throughput microservices, querying Redis over a TCP socket costs 1–3ms of network latency plus serialization overhead (JSON.parse). In-process memory (heap) resolves in <1µs. However, deploying in-process caches across horizontally scaled server pods introduces a critical distributed systems hazard: multi-pod cache split-brain.

3.1 Redis RESP3 Client-Side Tracking

To eliminate pod split-brain without discarding the microsecond performance of L1 caches, Redis 6 introduced and Redis 7/8 refined Server-Assisted Client-Side Caching via the RESP3 protocol.

With RESP3 tracking:

1. When Pod 1 reads user:100 from Redis, Redis records in its internal Invalidation Table that connection ID X holds user:100.
2. Pod 1 stores user:100 in its local in-memory L1 cache.
3. When Pod 2 mutates user:100 via SET user:100 ..., Redis checks its tracking table and pushes an asynchronous invalidation message (invalidate ["user:100"]) over the TCP connection to Pod 1.
4. Pod 1 receives the message and immediately deletes user:100 from its local heap memory.

Expand

3.2 Standard Tracking vs. Broadcast (BCAST) Mode

Redis supports two tracking implementations:

// ✅ Initializing Redis RESP3 Client-Side Caching with ioredis
import Redis from 'ioredis';
import { LRUCache } from 'lru-cache';

const l1Store = new LRUCache<string, string>({ max: 10000 });

const redisClient = new Redis({
  host: 'redis.internal',
  port: 6379,
  protocol: 3, // Enable RESP3 protocol
});

// Enable client-side tracking in broadcast mode for the 'entity:' namespace
async function setupClientSideCaching() {
  await redisClient.send_command('CLIENT', 'TRACKING', 'on', 'BCAST', 'PREFIX', 'entity:');

  // Listen for push invalidation events from Redis server
  redisClient.on('message', (channel, message) => {
    // In RESP3, invalidations arrive on dedicated push handlers
  });

  redisClient.on('push', (message) => {
    if (message.type === 'invalidate') {
      const invalidatedKeys: string[] = message.data[0];
      for (const key of invalidatedKeys) {
        l1Store.delete(key);
      }
    }
  });
}

export async function getEntityFast(key: string): Promise<string> {
  // L1 Check
  const local = l1Store.get(key);
  if (local) return local;

  // L2 Fetch & Populate L1
  const remote = await redisClient.get(key);
  if (remote) {
    l1Store.set(key, remote);
  }
  return remote || '';
}

4. Distributed Cache Clustering & Topology Engineering

When working set sizes exceed the RAM capacity of a single Redis node, or when write throughput saturates single-threaded event loop capacity, the caching layer must scale horizontally across a Redis Cluster.

4.1 Hash Slots & Key Distribution

Redis Cluster does not use consistent hashing rings by default; instead, it partitions the entire dataset across fixed 16,384 Hash Slots. Every key is assigned to a slot using the CRC16 checksum:

$$\text{slot} = \text{CRC16}(\text{key}) \pmod{16384}$$

Each master node in the cluster is responsible for a contiguous subset of the 16,384 slots (e.g., Node A owns 0–5460, Node B owns 5461–10922, Node C owns 10923–16383).

4.2 Multi-Key Operations & Hash Tags

In a Redis Cluster, multi-key commands (MGET, MSET, transactions MULTI/EXEC, and Lua scripts) are strictly prohibited if the participating keys map to different hash slots. Attempting to run MGET user:100:profile user:100:orders triggers a CROSSSLOT Keys in request don't hash to the same slot exception.

To force related keys onto the exact same hash slot and cluster node, use Hash Tags ({...}):

# ❌ Without Hash Tags: Keys hash independently to random slots across nodes
SET user:100:profile "data"   -> CRC16("user:100:profile") % 16384 = Slot 8421 (Node B)
SET user:100:orders  "data"   -> CRC16("user:100:orders")  % 16384 = Slot 3109 (Node A)
# MGET user:100:profile user:100:orders -> 💥 CROSSSLOT ERROR!

# ✅ With Hash Tags: Redis hashes ONLY the string inside curly braces
SET {user:100}:profile "data" -> CRC16("user:100") % 16384 = Slot 14205 (Node C)
SET {user:100}:orders  "data" -> CRC16("user:100") % 16384 = Slot 14205 (Node C)
# MGET {user:100}:profile {user:100}:orders -> ✅ SUCCESS (both on Node C)

4.3 The Hot Key Skew Problem & Two-Tier Key Salting

A critical vulnerability in clustered caching is Key Skew. If your application has a viral product or breaking news item, hashing that key routes 100% of read traffic to a single master node. The other 99 nodes in your cluster sit idle while Node C's network bandwidth saturates and its event loop drops connections.

// ✅ Resilient Pattern: Suffix Salting for Ultra-Hot Keys
export async function getHotKeyDistributed(baseKey: string, numSalts = 16): Promise<string | null> {
  // 1. Pick a random salt partition on read to distribute load evenly across all cluster nodes
  const salt = Math.floor(Math.random() * numSalts);
  const saltedKey = `${baseKey}:salt_${salt}`;

  const cached = await redisCluster.get(saltedKey);
  if (cached) return cached;

  // 2. On miss, compute origin data and write back across all salt slots with slight TTL jitter
  const freshData = await computeExpensiveData();
  const pipeline = redisCluster.pipeline();
  
  for (let i = 0; i < numSalts; i++) {
    const jitterTtl = 300 + Math.floor(Math.random() * 30);
    pipeline.set(`${baseKey}:salt_${i}`, freshData, 'EX', jitterTtl);
  }
  await pipeline.exec();

  return freshData;
}

Summary

What's Next

Now that we have established the multi-tier caching hierarchy and cluster topologies, Part 2: Cache Write Strategies & Invalidation Architectures tackles the hardest problem in distributed state: solving the dual-write race condition, managing Write-Back crash durability, and implementing Change Data Capture (CDC) pipelines.