System Design (HLD)Caching Strategies

Caching Layers

LevelIntermediate

Duration90 mins

TopicCaching Strategies

5 / 5

Distributed Caching

Shared Cache for Scale

As applications scale beyond a single server, in-process caches become problematic: each instance maintains its own cache, leading to inconsistent data, wasted memory, and cold-start penalties when new instances spin up. Distributed caching solves these challenges by providing a shared, network-accessible cache that all application instances can use.

Distributed caches like Redis and Memcached sit between your application and your database, serving as a high-speed data layer that can handle millions of operations per second with sub-millisecond latency. But designing effective distributed caching requires understanding the trade-offs: network overhead, serialization costs, consistency models, and failure modes that don't exist with in-process caches.

This page explores distributed caching systems, their architectures, operational patterns, and the design decisions that determine whether your cache is a performance multiplier or an operational liability.

What You Will Learn

This page covers distributed cache architecture and data placement, Redis deep dive including features, data structures, and clustering, Memcached fundamentals and when to choose it, cache consistency patterns in distributed systems, cache cluster operations and failure handling, and production deployment best practices. By the end, you'll be able to design and operate distributed caching systems that scale reliably.

Distributed Cache Architecture

A distributed cache is a network service that stores key-value pairs in memory, accessible by multiple clients. Unlike in-process caches, distributed caches survive application restarts, provide consistent views across instances, and can scale horizontally.

Basic Architecture:

┌─────────────────────────────────────────────────────────────┐
│                     Application Cluster                      │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐     │
│  │Instance 1│  │Instance 2│  │Instance 3│  │Instance 4│     │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────┬─────┘     │
└───────┼─────────────┼─────────────┼─────────────┼───────────┘
        │             │             │             │
        └──────────┬──┴─────────────┴──┬──────────┘
                   │                   │
                   ▼                   ▼
        ┌──────────────────────────────────────┐
        │         Distributed Cache            │
        │  ┌─────────┐  ┌─────────┐  ┌───────┐ │
        │  │ Node 1  │  │ Node 2  │  │Node 3 │ │
        │  │ Shard A │  │ Shard B │  │Shard C│ │
        │  └─────────┘  └─────────┘  └───────┘ │
        └──────────────────────────────────────┘
                          │
                          ▼
              ┌────────────────────┐
              │     Database       │
              └────────────────────┘

In-Process vs. Distributed Cache
Aspect	In-Process Cache	Distributed Cache
Access latency	Nanoseconds	< 1 millisecond (network hop)
Consistency	Per-instance (inconsistent across cluster)	Shared (consistent view)
Memory usage	Duplicated across instances	Shared, efficient
Cold start	Cache empty on restart	Cache persists across restarts
Scalability	Limited by single process memory	Horizontal scaling
Complexity	Simple, in-process	Network, serialization, failure handling
Failure impact	Instance crash = cache lost	Node failure = partial cache loss

Data Placement Strategies:

In a cluster, how does the cache know which node holds a particular key?

1. Client-Side Routing (Consistent Hashing):

The client library determines which node to contact based on the key. No central coordinator.

// Hash the key to determine node
const nodeIndex = consistentHash(key) % nodes.length;
return nodes[nodeIndex].get(key);

2. Proxy-Based Routing:

A proxy layer (like Twemproxy, mcrouter) routes requests to the appropriate node.

Client → Proxy → Correct Cache Node
         └─→ Routes based on consistent hashing

3. Cluster-Managed Routing (Redis Cluster):

The cluster itself manages slots, and nodes redirect clients to the correct node.

Client → Any Node → MOVED redirect → Correct Node
                    └─→ Client learns and caches slot mapping

Consistent Hashing Essential

Consistent hashing ensures that adding or removing nodes only remaps a fraction of keys, not all of them. Without it, every node change would invalidate the entire cache. Redis Cluster uses a slot-based approach (16,384 hash slots) that serves the same purpose.

Redis Deep Dive

Redis (Remote Dictionary Server) is the most popular distributed cache, used by millions of installations worldwide. It goes beyond simple key-value storage, offering rich data structures, persistence options, pub/sub messaging, and atomic operations.

Core Redis Features:

Redis Capabilities

•Data Structures — Strings, Lists, Sets, Sorted Sets, Hashes, Bitmaps, HyperLogLog, Streams, Geospatial
•Atomic Operations — INCR, multi-key transactions (MULTI/EXEC), Lua scripting for complex atomics
•Expiration — Built-in TTL per key, configurable eviction policies
•Persistence — RDB snapshots, AOF (Append-Only File), or hybrid for durability
•Replication — Primary-replica for read scaling and high availability
•Clustering — Redis Cluster for horizontal sharding across nodes
•Pub/Sub — Message broadcasting for real-time features, cache invalidation

redis-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
import Redis from 'ioredis';
 
const redis = new Redis({
  host: 'localhost',
  port: 6379,
  maxRetriesPerRequest: 3,
  // Connection pooling via ioredis is automatic
});
 
// Basic caching
async function cacheExample() {
  // SET with expiration
  await redis.setex('user:123', 300, JSON.stringify({ name: 'Alice', email: 'alice@example.com' }));
  
  // GET with type safety
  const userData = await redis.get('user:123');
  const user = userData ? JSON.parse(userData) : null;
  
  // SET only if not exists (useful for locks, deduplication)
  const wasSet = await redis.setnx('lock:resource', 'holder-id');
  
  // SET with options: NX (not exists), EX (expiration)
  await redis.set('key', 'value', 'EX', 300, 'NX');
}
 
// Hash operations (structured data without JSON parsing)
async function hashExample() {
  // Store user as hash
  await redis.hset('user:123', {
    name: 'Alice',
    email: 'alice@example.com',
    loginCount: '42',
  });
  
  // Get single field
  const name = await redis.hget('user:123', 'name');
  
  // Get all fields
  const user = await redis.hgetall('user:123');
  
  // Increment field atomically
  await redis.hincrby('user:123', 'loginCount', 1);
}
 
// Sorted set for leaderboards/rankings
async function leaderboardExample() {
  // Add scores
  await redis.zadd('leaderboard:daily', 1500, 'player:123');
  await redis.zadd('leaderboard:daily', 2300, 'player:456');
  await redis.zadd('leaderboard:daily', 1800, 'player:789');
  
  // Get top 10 with scores
  const topPlayers = await redis.zrevrange('leaderboard:daily', 0, 9, 'WITHSCORES');
  // ['player:456', '2300', 'player:789', '1800', 'player:123', '1500']
  
  // Get rank of specific player
  const rank = await redis.zrevrank('leaderboard:daily', 'player:123');
}
 
// Atomic operations with Lua scripting
async function atomicOperation() {
  // Rate limiter: increment and check in one atomic operation
  const rateLimitScript = `
    local current = redis.call('INCR', KEYS[1])
    if current == 1 then
      redis.call('EXPIRE', KEYS[1], ARGV[1])
    end
    return current
  `;
  
  const requestCount = await redis.eval(
    rateLimitScript,
    1,                     // Number of keys
    'rate:user:123',       // KEYS[1]
    '60'                   // ARGV[1]: 60 second window
  );
  
  if (requestCount > 100) {
    throw new Error('Rate limit exceeded');
  }
}
 
// Pipeline for batch operations
async function pipelineExample() {
  const pipeline = redis.pipeline();
  
  // Queue multiple operations
  pipeline.get('user:123');
  pipeline.get('user:456');
  pipeline.hgetall('session:abc');
  pipeline.zrange('recent:items', 0, 9);
  
  // Execute all at once (single round trip)
  const results = await pipeline.exec();
  // [[null, 'user123data'], [null, 'user456data'], [null, {...}], [null, [...]]]
}

Prefer Pipelines for Batches

Each Redis command is a network round-trip (~0.5ms). If you need to execute 10 commands, that's 5ms of network overhead. Using pipelines sends all commands in one round-trip, dramatically reducing latency for batch operations.

Redis Cluster Architecture

Redis Cluster provides automatic sharding across multiple Redis nodes with built-in high availability through replica failover.

Redis Cluster Fundamentals:

Hash Slots:

16,384 hash slots divide the keyspace
Each node owns a subset of slots
Key's slot = CRC16(key) % 16384

Cluster Topology Example:

┌─────────────────────────────────────────────────────────────┐
│                      Redis Cluster                           │
│                                                              │
│  ┌────────────────┐  ┌────────────────┐  ┌────────────────┐ │
│  │    Node 1      │  │    Node 2      │  │    Node 3      │ │
│  │ Slots 0-5460   │  │ Slots 5461-10922│ │Slots 10923-16383││
│  │   (Primary)    │  │   (Primary)    │  │   (Primary)    │ │
│  └───────┬────────┘  └───────┬────────┘  └───────┬────────┘ │
│          │                   │                   │          │
│  ┌───────▼────────┐  ┌───────▼────────┐  ┌───────▼────────┐ │
│  │    Node 4      │  │    Node 5      │  │    Node 6      │ │
│  │   (Replica)    │  │   (Replica)    │  │   (Replica)    │ │
│  └────────────────┘  └────────────────┘  └────────────────┘ │
└─────────────────────────────────────────────────────────────┘

redis-cluster-client.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
import { Cluster } from 'ioredis';
 
// Connect to Redis Cluster
const cluster = new Cluster([
  { host: 'node1.redis.example.com', port: 6379 },
  { host: 'node2.redis.example.com', port: 6379 },
  { host: 'node3.redis.example.com', port: 6379 },
], {
  // Cluster-specific options
  scaleReads: 'slave',  // Read from replicas for read scaling
  slotsRefreshTimeout: 2000,
  redisOptions: {
    password: process.env.REDIS_PASSWORD,
    tls: {}, // Enable TLS in production
  },
});
 
// Normal operations work transparently
await cluster.set('user:123', 'data');
const data = await cluster.get('user:123');
 
// Hash tags for co-locating related keys
// Keys with same {tag} go to same slot
await cluster.set('{user:123}:profile', JSON.stringify(profile));
await cluster.set('{user:123}:settings', JSON.stringify(settings));
await cluster.set('{user:123}:sessions', JSON.stringify(sessions));
 
// Now multi-key operations work (same slot)
const results = await cluster.mget(
  '{user:123}:profile',
  '{user:123}:settings',
  '{user:123}:sessions'
);
 
// Warning: This WILL fail in cluster mode (different slots)
// await cluster.mget('user:123', 'user:456');  // CrossSlotError
 
// Pipeline with cluster (automatically groups by slot)
const pipeline = cluster.pipeline();
pipeline.set('{order:1}:status', 'pending');
pipeline.set('{order:1}:items', JSON.stringify(items));
await pipeline.exec();

Redis Deployment Options
Deployment	Sharding	HA	Use Case
Single Node	None	None	Development, small workloads
Primary + Replica	None	Manual failover	Read scaling, basic redundancy
Sentinel	None	Automatic failover	HA without sharding
Cluster	Automatic (hash slots)	Automatic failover	Large scale, sharding + HA
Managed (ElastiCache, Redis Cloud)	Provider-managed	Provider-managed	Production with minimal ops

Cluster Mode Limitations

Redis Cluster restricts multi-key operations to keys in the same slot. Use {hash tags} to co-locate related keys, or redesign to avoid cross-slot operations. Commands like KEYS, SCAN, and Lua scripts with multiple keys are affected.

Memcached Fundamentals

Memcached is a simpler, high-performance distributed cache focused purely on caching. Unlike Redis, it offers fewer features but sometimes excels for specific use cases.

Memcached Characteristics:

Memcached Strengths

•Simple key-value model (strings only)
•Multi-threaded (uses all cores efficiently)
•Predictable memory usage (slab allocator)
•No persistence = no disk I/O latency
•Mature, battle-tested at scale (Facebook, Twitter)
•Lower memory overhead per key

Memcached Limitations

•Strings only (no data structures)
•No built-in clustering (client-side sharding)
•No persistence (cache cold on restart)
•No pub/sub or scripting
•Key size limited to 250 bytes
•Value size limited to 1 MB default

memcached-usage.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
import Memcached from 'memcached';
 
// Connect to Memcached cluster (client-side consistent hashing)
const memcached = new Memcached([
  'cache1.example.com:11211',
  'cache2.example.com:11211',
  'cache3.example.com:11211',
], {
  poolSize: 10,
  retries: 2,
  timeout: 5000,
  idle: 30000,
});
 
// Promisify for async/await
const memcachedAsync = {
  get: (key: string): Promise<any> =>
    new Promise((resolve, reject) => {
      memcached.get(key, (err, data) => {
        if (err) reject(err);
        else resolve(data);
      });
    }),
  
  set: (key: string, value: any, ttl: number): Promise<boolean> =>
    new Promise((resolve, reject) => {
      memcached.set(key, value, ttl, (err) => {
        if (err) reject(err);
        else resolve(true);
      });
    }),
  
  delete: (key: string): Promise<boolean> =>
    new Promise((resolve, reject) => {
      memcached.del(key, (err) => {
        if (err) reject(err);
        else resolve(true);
      });
    }),
  
  // Atomic increment
  incr: (key: string, amount: number): Promise<number> =>
    new Promise((resolve, reject) => {
      memcached.incr(key, amount, (err, result) => {
        if (err) reject(err);
        else resolve(result);
      });
    }),
};
 
// Usage
async function cacheUser(userId: string, user: User): Promise<void> {
  // Must serialize to string (no native data structures)
  await memcachedAsync.set(
    `user:${userId}`,
    JSON.stringify(user),
    300  // TTL in seconds
  );
}
 
async function getUser(userId: string): Promise<User | null> {
  const data = await memcachedAsync.get(`user:${userId}`);
  return data ? JSON.parse(data) : null;
}
 
// Multi-get for batch operations
function multiGet(keys: string[]): Promise<Record<string, any>> {
  return new Promise((resolve, reject) => {
    memcached.getMulti(keys, (err, data) => {
      if (err) reject(err);
      else resolve(data);
    });
  });
}

Redis vs Memcached Decision Guide
Consideration	Choose Redis	Choose Memcached
Data structures needed	Yes (lists, sets, sorted sets, hashes)	No (strings only sufficient)
Persistence required	Yes (durability matters)	No (pure cache, DB is source of truth)
Pub/sub messaging	Yes (event notifications, invalidation)	Not needed
Atomic operations	Complex atomics (Lua scripts)	Simple increment/decrement
Memory efficiency	Need predictable memory	Slab allocator more predictable
Multi-threading	Single-threaded bottleneck concern	Multi-threaded for heavy load
Ecosystem	Rich tooling, large community	Simpler, focused tooling

Modern Consensus: Redis Preferred

For most new projects, Redis is the default choice due to its richer feature set and similar performance. Memcached remains relevant for pure caching workloads where its multi-threaded architecture and predictable memory usage are beneficial, especially at extreme scale.

Cache Consistency Patterns

Distributed caches introduce consistency challenges that don't exist with databases. The cache is a separate system from your source of truth, and keeping them synchronized is non-trivial.

Write Patterns for Consistency:

Cache Write Strategies
Pattern	Description	Consistency	Write Latency	Failure Mode
Cache-Aside	App manages cache separately; write to DB, invalidate cache	Eventually consistent	Fast (DB only)	Stale reads until invalidation
Write-Through	Write to cache and DB together (cache library handles)	Strong if sync	Slower (both writes)	DB failure = cache stale
Write-Behind	Write to cache; async persist to DB later	Eventually consistent	Very fast	Data loss if cache fails before persist
Read-Through	Cache fetches from DB on miss automatically	Eventually consistent	Same as cache-aside	Same as cache-aside

cache-consistency-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
// Pattern 1: Cache-Aside with Invalidation
async function updateUserCacheAside(
  userId: string, 
  updates: Partial<User>
): Promise<User> {
  // Write to database first
  const user = await db.users.update({
    where: { id: userId },
    data: updates,
  });
  
  // Invalidate cache (next read will repopulate)
  await redis.del(`user:${userId}`);
  
  return user;
}
 
// Pattern 2: Write-Through with Cache Update
async function updateUserWriteThrough(
  userId: string,
  updates: Partial<User>
): Promise<User> {
  // Write to database
  const user = await db.users.update({
    where: { id: userId },
    data: updates,
  });
  
  // Update cache with new value (not invalidate)
  await redis.setex(
    `user:${userId}`,
    300,
    JSON.stringify(user)
  );
  
  return user;
}
 
// Pattern 3: Handling the Delete-Then-Read Race
// Problem: Thread A deletes cache, Thread B reads stale DB, caches it,
//          Thread A writes to DB. Cache is now stale.
 
// Solution: Double-delete pattern
async function updateUserDoubleDelete(
  userId: string,
  updates: Partial<User>
): Promise<User> {
  const cacheKey = `user:${userId}`;
  
  // First delete
  await redis.del(cacheKey);
  
  // Write to database
  const user = await db.users.update({
    where: { id: userId },
    data: updates,
  });
  
  // Delay to allow in-flight reads to complete
  setTimeout(async () => {
    // Second delete (cleans up any stale repopulation)
    await redis.del(cacheKey);
  }, 500);
  
  return user;
}
 
// Pattern 4: Lock-based consistency
async function updateUserWithLock(
  userId: string,
  updates: Partial<User>
): Promise<User> {
  const cacheKey = `user:${userId}`;
  const lockKey = `lock:${cacheKey}`;
  
  // Acquire lock
  const lockAcquired = await redis.set(lockKey, '1', 'EX', 5, 'NX');
  if (!lockAcquired) {
    throw new Error('Could not acquire lock');
  }
  
  try {
    // Write to database
    const user = await db.users.update({
      where: { id: userId },
      data: updates,
    });
    
    // Update cache
    await redis.setex(cacheKey, 300, JSON.stringify(user));
    
    return user;
  } finally {
    // Release lock
    await redis.del(lockKey);
  }
}
 
// Pattern 5: Version-based invalidation
interface VersionedCache {
  version: number;
  data: any;
}
 
async function getUserVersioned(userId: string): Promise<User | null> {
  const currentVersion = await redis.get(`user:${userId}:version`);
  const cached = await redis.get(`user:${userId}`);
  
  if (cached) {
    const parsed: VersionedCache = JSON.parse(cached);
    if (parsed.version.toString() === currentVersion) {
      return parsed.data;
    }
    // Version mismatch - cache is stale
  }
  
  // Fetch fresh data
  const user = await db.users.findUnique({ where: { id: userId } });
  
  if (user) {
    const version = parseInt(currentVersion || '0') + 1;
    await redis.setex(`user:${userId}`, 300, JSON.stringify({
      version,
      data: user,
    }));
    await redis.set(`user:${userId}:version`, version.toString());
  }
  
  return user;
}

Cache Invalidation > Cache Update

In most cases, delete (invalidate) is safer than update (write-through). If your update fails after DB write, cache has stale data. If your delete fails, you just get a cache miss—the next read fetches fresh data. Lazy repopulation is more robust.

Cache Failure Handling

Distributed caches are network services that can fail, become slow, or lose data. Your application must handle these failures gracefully—cache unavailability should degrade performance, not cause outages.

Failure Scenarios:

Cache Failure Types and Responses
Failure Type	Symptoms	Correct Response	Incorrect Response
Cache node down	Connection refused/timeout	Fallback to DB; skip cache writes	Fail the request
Cache slow (degraded)	High latency, timeouts	Short timeout; fallback to DB	Wait indefinitely
Cache full (OOM)	Evictions increase; errors	Continue operation; tune eviction	Panic restart
Network partition	Intermittent connectivity	Circuit breaker; fallback	Endless retries
Data corruption	Deserialization errors	Delete and repopulate	Serve corrupt data

cache-resilience.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
import Redis from 'ioredis';
import CircuitBreaker from 'opossum';
 
// Configure Redis with resilience
const redis = new Redis({
  host: 'cache.example.com',
  port: 6379,
  
  // Connection resilience
  retryStrategy: (times) => {
    if (times > 3) return null;  // Stop retrying
    return Math.min(times * 200, 2000);  // Exponential backoff
  },
  
  // Command timeout
  commandTimeout: 500,  // Fast fail
  
  // Connection timeout
  connectTimeout: 3000,
  
  // Auto-reconnect
  maxRetriesPerRequest: 1,
  enableOfflineQueue: false,  // Don't queue when disconnected
});
 
// Circuit breaker for cache operations
const cacheBreaker = new CircuitBreaker(
  async (operation: () => Promise<any>) => operation(),
  {
    timeout: 1000,           // Consider call failed after 1s
    errorThresholdPercentage: 50,  // Open after 50% failures
    resetTimeout: 30000,     // Try again after 30s
    volumeThreshold: 10,     // Minimum calls before tripping
  }
);
 
cacheBreaker.on('open', () => {
  console.warn('Cache circuit breaker OPEN - falling back to database');
});
 
cacheBreaker.on('halfOpen', () => {
  console.info('Cache circuit breaker testing recovery...');
});
 
cacheBreaker.on('close', () => {
  console.info('Cache circuit breaker CLOSED - cache operational');
});
 
// Resilient cache wrapper
class ResilientCache {
  async get<T>(key: string): Promise<T | null> {
    try {
      return await cacheBreaker.fire(async () => {
        const data = await redis.get(key);
        return data ? JSON.parse(data) : null;
      });
    } catch (error) {
      // Circuit open or cache error - return null (cache miss)
      console.warn(`Cache get failed for ${key}:`, error.message);
      return null;
    }
  }
 
  async set(key: string, value: any, ttlSeconds: number): Promise<void> {
    try {
      await cacheBreaker.fire(async () => {
        await redis.setex(key, ttlSeconds, JSON.stringify(value));
      });
    } catch (error) {
      // Cache write failure is non-fatal
      console.warn(`Cache set failed for ${key}:`, error.message);
      // Don't throw - application continues without caching
    }
  }
 
  async delete(key: string): Promise<void> {
    try {
      await cacheBreaker.fire(async () => {
        await redis.del(key);
      });
    } catch (error) {
      // Invalidation failure is concerning but not fatal
      console.error(`Cache delete failed for ${key}:`, error.message);
      // Consider: queue for retry, or accept stale data with TTL
    }
  }
}
 
// Usage with graceful degradation
const cache = new ResilientCache();
 
async function getUser(userId: string): Promise<User> {
  // Try cache (gracefully handles failure)
  const cached = await cache.get<User>(`user:${userId}`);
  if (cached) {
    return cached;
  }
  
  // Fallback to database
  const user = await db.users.findUnique({ where: { id: userId } });
  
  // Attempt to cache (fire-and-forget, non-blocking)
  if (user) {
    cache.set(`user:${userId}`, user, 300).catch(() => {
      // Already logged in ResilientCache
    });
  }
  
  return user;
}

Cache is an Optimization, Not a Dependency

Never make cache availability a hard dependency. Your application should function (albeit slower) when cache is unavailable. Treat cache failures as cache misses and fall back to the database. Exceptions: if your database can't handle the load without caching, you have a capacity planning problem.

Production Deployment Best Practices

Running distributed caches in production requires attention to memory management, monitoring, security, and operational procedures.

Memory Management:

redis-production-config.conf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
# Redis production configuration
 
# Memory limits
maxmemory 4gb
maxmemory-policy allkeys-lru  # Evict any key using LRU when memory full
# Alternatives:
# - volatile-lru: Only evict keys with TTL set
# - allkeys-random: Random eviction
# - volatile-ttl: Evict keys closest to expiration
# - noeviction: Return errors when memory full
 
# Persistence (choose based on durability needs)
# Option 1: No persistence (pure cache)
save ""
appendonly no
 
# Option 2: RDB snapshots
save 900 1       # Snapshot if 1 key changed in 15 min
save 300 10      # Snapshot if 10 keys changed in 5 min
save 60 10000    # Snapshot if 10000 keys changed in 1 min
 
# Option 3: AOF (append-only file) for durability
appendonly yes
appendfsync everysec  # Sync every second (balance durability/performance)
 
# Network security
bind 10.0.0.0/8      # Only listen on private network
requirepass <strong-password>
tls-port 6379
tls-cert-file /path/to/cert.pem
tls-key-file /path/to/key.pem
 
# Connection limits
maxclients 10000
timeout 300           # Close idle connections after 5 min
 
# Slow query logging
slowlog-log-slower-than 10000  # Log commands > 10ms
slowlog-max-len 128
 
# Disable dangerous commands in production
rename-command FLUSHALL ""
rename-command FLUSHDB ""
rename-command DEBUG ""
rename-command KEYS ""  # KEYS is O(n), use SCAN instead

Key Metrics to Monitor

•Memory usage — used_memory, used_memory_rss. Alert if approaching maxmemory.
•Hit rate — keyspace_hits / (keyspace_hits + keyspace_misses). Target > 95%.
•Evictions — evicted_keys. Non-zero means memory pressure.
•Connected clients — connected_clients. Spike may indicate connection leak.
•Commands per second — instantaneous_ops_per_sec. Baseline for capacity.
•Latency — Use redis-cli --latency. Should be < 1ms for simple operations.
•Replication lag — master_link_down_since_seconds. Critical for HA.
•Slow queries — Review slowlog periodically for optimization opportunities.

redis-monitoring.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Redis monitoring commands
 
# Real-time stats
redis-cli INFO stats
redis-cli INFO memory
redis-cli INFO replication
 
# Slow query log
redis-cli SLOWLOG GET 10
 
# Memory analysis
redis-cli MEMORY DOCTOR
 
# Big keys scan (run during low traffic)
redis-cli --bigkeys
 
# Sample key analysis
redis-cli DEBUG OBJECT key-name
 
# Live monitoring
redis-cli MONITOR  # WARNING: Production impact, use briefly
 
# Latency check
redis-cli --latency
 
# Key expiration analysis
redis-cli DEBUG SLEEP 0  # Check key TTL distribution

Use Managed Services for Production

Services like AWS ElastiCache, Azure Cache for Redis, or Redis Cloud handle replication, failover, patching, and scaling. Unless you have specific requirements or dedicated DevOps expertise, managed services reduce operational burden significantly.

Advanced Distributed Caching Patterns

Beyond basic key-value caching, distributed caches enable sophisticated patterns for coordination, real-time features, and complex data management.

Pattern: Distributed Rate Limiting

advanced-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
// Pattern 1: Sliding Window Rate Limiter
async function checkRateLimit(
  userId: string,
  limit: number,
  windowMs: number
): Promise<{ allowed: boolean; remaining: number }> {
  const key = `rate:${userId}`;
  const now = Date.now();
  const windowStart = now - windowMs;
  
  // Use sorted set with timestamp as score
  const pipeline = redis.pipeline();
  
  // Remove old entries outside window
  pipeline.zremrangebyscore(key, 0, windowStart);
  
  // Add current request
  pipeline.zadd(key, now.toString(), `${now}:${Math.random()}`);
  
  // Count requests in window
  pipeline.zcount(key, windowStart.toString(), now.toString());
  
  // Set TTL for cleanup
  pipeline.expire(key, Math.ceil(windowMs / 1000));
  
  const results = await pipeline.exec();
  const requestCount = results![2][1] as number;
  
  return {
    allowed: requestCount <= limit,
    remaining: Math.max(0, limit - requestCount),
  };
}
 
// Pattern 2: Distributed Lock with Redlock
// For critical sections across multiple instances
 
import Redlock from 'redlock';
 
const redlock = new Redlock(
  [redis], // Array of Redis instances for safety
  {
    driftFactor: 0.01,
    retryCount: 10,
    retryDelay: 200,
    retryJitter: 200,
  }
);
 
async function processWithLock(resourceId: string): Promise<void> {
  let lock;
  try {
    lock = await redlock.acquire(
      [`lock:${resourceId}`],
      5000  // Lock TTL
    );
    
    // Critical section - only one instance executes this
    await performCriticalOperation(resourceId);
    
  } finally {
    if (lock) {
      await lock.release();
    }
  }
}
 
// Pattern 3: Cache-based Pub/Sub for Real-time Invalidation
const subscriber = redis.duplicate();
 
// Subscribe to invalidation channel
await subscriber.subscribe('cache:invalidate');
 
subscriber.on('message', (channel, message) => {
  const { type, id } = JSON.parse(message);
  
  // Invalidate local caches or notify clients
  switch (type) {
    case 'user':
      localCache.delete(`user:${id}`);
      break;
    case 'product':
      localCache.delete(`product:${id}`);
      websockets.broadcastToRoom(`product:${id}`, { type: 'refresh' });
      break;
  }
});
 
// Publish invalidation
async function publishInvalidation(type: string, id: string): Promise<void> {
  await redis.publish('cache:invalidate', JSON.stringify({ type, id }));
}
 
// Pattern 4: Leaderboard with Real-time Updates
class Leaderboard {
  private key: string;
  
  constructor(name: string) {
    this.key = `leaderboard:${name}`;
  }
  
  // Update score atomically
  async updateScore(playerId: string, score: number): Promise<void> {
    await redis.zadd(this.key, score.toString(), playerId);
  }
  
  // Increment score atomically
  async incrementScore(playerId: string, delta: number): Promise<number> {
    return redis.zincrby(this.key, delta, playerId);
  }
  
  // Get top N players
  async getTopPlayers(n: number): Promise<Array<{ playerId: string; score: number }>> {
    const results = await redis.zrevrange(this.key, 0, n - 1, 'WITHSCORES');
    
    const players: Array<{ playerId: string; score: number }> = [];
    for (let i = 0; i < results.length; i += 2) {
      players.push({
        playerId: results[i],
        score: parseFloat(results[i + 1]),
      });
    }
    return players;
  }
  
  // Get player rank (1-indexed)
  async getPlayerRank(playerId: string): Promise<number | null> {
    const rank = await redis.zrevrank(this.key, playerId);
    return rank !== null ? rank + 1 : null;
  }
  
  // Get nearby players (context in leaderboard)
  async getPlayerContext(
    playerId: string,
    contextSize: number = 5
  ): Promise<Array<{ playerId: string; score: number; rank: number }>> {
    const rank = await redis.zrevrank(this.key, playerId);
    if (rank === null) return [];
    
    const start = Math.max(0, rank - contextSize);
    const end = rank + contextSize;
    
    const results = await redis.zrevrange(this.key, start, end, 'WITHSCORES');
    
    const players: Array<{ playerId: string; score: number; rank: number }> = [];
    for (let i = 0; i < results.length; i += 2) {
      players.push({
        playerId: results[i],
        score: parseFloat(results[i + 1]),
        rank: start + (i / 2) + 1,
      });
    }
    return players;
  }
}

Redis as Primary Data Store

While Redis is typically a cache, some use cases can use Redis as a primary store: session storage, real-time analytics, rate limiters, leaderboards, pub/sub messaging. Enable persistence (AOF) for durability if Redis is not just a cache.

Summary and Best Practices

Distributed caching is essential for scaling applications beyond a single server. Redis and Memcached provide sub-millisecond access to shared data, enabling consistent caching across instances, reduced database load, and sophisticated features like rate limiting and real-time coordination.

Distributed Caching Best Practices

•Choose Redis unless you have specific Memcached requirements — Redis offers more features with comparable performance.
•Use Redis Cluster for horizontal scaling — When single-node capacity is insufficient. Remember hash tag constraints.
•Implement circuit breakers — Cache failures should never take down your application.
•Set appropriate timeouts — Fast fail (< 1s) is better than blocking threads waiting for slow cache.
•Use pipelines for batch operations — Reduce network round-trips; significant latency improvement.
•Configure eviction policies thoughtfully — allkeys-lru for pure caches; volatile-lru when mixing TTL and non-TTL keys.
•Monitor key metrics continuously — Hit rate, memory, evictions, latency. Alert on anomalies.
•Secure your cache — Use authentication, TLS, and private networks. Exposed Redis is a security vulnerability.
•Handle consistency explicitly — Choose invalidation strategy; understand the trade-offs of cache-aside vs write-through.
•Consider managed services — ElastiCache, Redis Cloud, etc. reduce operational burden significantly.

Module Complete:

This concludes the Caching Layers module. You've learned about caching at every level of the stack:

Browser caching — HTTP cache headers, Cache API, Service Workers
CDN caching — Edge locations, TTL strategies, invalidation
Application caching — In-process caches, memoization, cache key design
Database query caching — Buffer pools, materialized views, query result caching
Distributed caching — Redis, Memcached, cluster architectures, consistency patterns

Together, these layers form a comprehensive caching strategy that can improve performance by orders of magnitude while maintaining data consistency appropriate to your requirements.

Module Complete

You now have a comprehensive understanding of caching layers from browser to distributed cache. You can design multi-layer caching strategies that maximize performance, implement Redis and Memcached effectively, handle cache failures gracefully, and maintain appropriate consistency guarantees at each layer.

5 / 5

Loading learning content...

System Design (HLD)Caching Strategies

Caching Layers

LevelIntermediate

Duration90 mins

TopicCaching Strategies

5 / 5

Distributed Caching

Shared Cache for Scale

What You Will Learn

Distributed Cache Architecture

Basic Architecture:

┌─────────────────────────────────────────────────────────────┐
│                     Application Cluster                      │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐     │
│  │Instance 1│  │Instance 2│  │Instance 3│  │Instance 4│     │
│  └────┬─────┘  └────┬─────┘  └────┬─────┘  └────┬─────┘     │
└───────┼─────────────┼─────────────┼─────────────┼───────────┘
        │             │             │             │
        └──────────┬──┴─────────────┴──┬──────────┘
                   │                   │
                   ▼                   ▼
        ┌──────────────────────────────────────┐
        │         Distributed Cache            │
        │  ┌─────────┐  ┌─────────┐  ┌───────┐ │
        │  │ Node 1  │  │ Node 2  │  │Node 3 │ │
        │  │ Shard A │  │ Shard B │  │Shard C│ │
        │  └─────────┘  └─────────┘  └───────┘ │
        └──────────────────────────────────────┘
                          │
                          ▼
              ┌────────────────────┐
              │     Database       │
              └────────────────────┘

In-Process vs. Distributed Cache
Aspect	In-Process Cache	Distributed Cache
Access latency	Nanoseconds	< 1 millisecond (network hop)
Consistency	Per-instance (inconsistent across cluster)	Shared (consistent view)
Memory usage	Duplicated across instances	Shared, efficient
Cold start	Cache empty on restart	Cache persists across restarts
Scalability	Limited by single process memory	Horizontal scaling
Complexity	Simple, in-process	Network, serialization, failure handling
Failure impact	Instance crash = cache lost	Node failure = partial cache loss

Data Placement Strategies:

In a cluster, how does the cache know which node holds a particular key?

1. Client-Side Routing (Consistent Hashing):

The client library determines which node to contact based on the key. No central coordinator.

// Hash the key to determine node
const nodeIndex = consistentHash(key) % nodes.length;
return nodes[nodeIndex].get(key);

2. Proxy-Based Routing:

A proxy layer (like Twemproxy, mcrouter) routes requests to the appropriate node.

Client → Proxy → Correct Cache Node
         └─→ Routes based on consistent hashing

3. Cluster-Managed Routing (Redis Cluster):

The cluster itself manages slots, and nodes redirect clients to the correct node.

Client → Any Node → MOVED redirect → Correct Node
                    └─→ Client learns and caches slot mapping

Consistent Hashing Essential

Redis Deep Dive

Core Redis Features:

Redis Capabilities

•Data Structures — Strings, Lists, Sets, Sorted Sets, Hashes, Bitmaps, HyperLogLog, Streams, Geospatial
•Atomic Operations — INCR, multi-key transactions (MULTI/EXEC), Lua scripting for complex atomics
•Expiration — Built-in TTL per key, configurable eviction policies
•Persistence — RDB snapshots, AOF (Append-Only File), or hybrid for durability
•Replication — Primary-replica for read scaling and high availability
•Clustering — Redis Cluster for horizontal sharding across nodes
•Pub/Sub — Message broadcasting for real-time features, cache invalidation

redis-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
import Redis from 'ioredis';
 
const redis = new Redis({
  host: 'localhost',
  port: 6379,
  maxRetriesPerRequest: 3,
  // Connection pooling via ioredis is automatic
});
 
// Basic caching
async function cacheExample() {
  // SET with expiration
  await redis.setex('user:123', 300, JSON.stringify({ name: 'Alice', email: 'alice@example.com' }));
  
  // GET with type safety
  const userData = await redis.get('user:123');
  const user = userData ? JSON.parse(userData) : null;
  
  // SET only if not exists (useful for locks, deduplication)
  const wasSet = await redis.setnx('lock:resource', 'holder-id');
  
  // SET with options: NX (not exists), EX (expiration)
  await redis.set('key', 'value', 'EX', 300, 'NX');
}
 
// Hash operations (structured data without JSON parsing)
async function hashExample() {
  // Store user as hash
  await redis.hset('user:123', {
    name: 'Alice',
    email: 'alice@example.com',
    loginCount: '42',
  });
  
  // Get single field
  const name = await redis.hget('user:123', 'name');
  
  // Get all fields
  const user = await redis.hgetall('user:123');
  
  // Increment field atomically
  await redis.hincrby('user:123', 'loginCount', 1);
}
 
// Sorted set for leaderboards/rankings
async function leaderboardExample() {
  // Add scores
  await redis.zadd('leaderboard:daily', 1500, 'player:123');
  await redis.zadd('leaderboard:daily', 2300, 'player:456');
  await redis.zadd('leaderboard:daily', 1800, 'player:789');
  
  // Get top 10 with scores
  const topPlayers = await redis.zrevrange('leaderboard:daily', 0, 9, 'WITHSCORES');
  // ['player:456', '2300', 'player:789', '1800', 'player:123', '1500']
  
  // Get rank of specific player
  const rank = await redis.zrevrank('leaderboard:daily', 'player:123');
}
 
// Atomic operations with Lua scripting
async function atomicOperation() {
  // Rate limiter: increment and check in one atomic operation
  const rateLimitScript = `
    local current = redis.call('INCR', KEYS[1])
    if current == 1 then
      redis.call('EXPIRE', KEYS[1], ARGV[1])
    end
    return current
  `;
  
  const requestCount = await redis.eval(
    rateLimitScript,
    1,                     // Number of keys
    'rate:user:123',       // KEYS[1]
    '60'                   // ARGV[1]: 60 second window
  );
  
  if (requestCount > 100) {
    throw new Error('Rate limit exceeded');
  }
}
 
// Pipeline for batch operations
async function pipelineExample() {
  const pipeline = redis.pipeline();
  
  // Queue multiple operations
  pipeline.get('user:123');
  pipeline.get('user:456');
  pipeline.hgetall('session:abc');
  pipeline.zrange('recent:items', 0, 9);
  
  // Execute all at once (single round trip)
  const results = await pipeline.exec();
  // [[null, 'user123data'], [null, 'user456data'], [null, {...}], [null, [...]]]
}

Prefer Pipelines for Batches

Redis Cluster Architecture

Redis Cluster provides automatic sharding across multiple Redis nodes with built-in high availability through replica failover.

Redis Cluster Fundamentals:

Hash Slots:

16,384 hash slots divide the keyspace
Each node owns a subset of slots
Key's slot = CRC16(key) % 16384

Cluster Topology Example:

┌─────────────────────────────────────────────────────────────┐
│                      Redis Cluster                           │
│                                                              │
│  ┌────────────────┐  ┌────────────────┐  ┌────────────────┐ │
│  │    Node 1      │  │    Node 2      │  │    Node 3      │ │
│  │ Slots 0-5460   │  │ Slots 5461-10922│ │Slots 10923-16383││
│  │   (Primary)    │  │   (Primary)    │  │   (Primary)    │ │
│  └───────┬────────┘  └───────┬────────┘  └───────┬────────┘ │
│          │                   │                   │          │
│  ┌───────▼────────┐  ┌───────▼────────┐  ┌───────▼────────┐ │
│  │    Node 4      │  │    Node 5      │  │    Node 6      │ │
│  │   (Replica)    │  │   (Replica)    │  │   (Replica)    │ │
│  └────────────────┘  └────────────────┘  └────────────────┘ │
└─────────────────────────────────────────────────────────────┘

redis-cluster-client.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
import { Cluster } from 'ioredis';
 
// Connect to Redis Cluster
const cluster = new Cluster([
  { host: 'node1.redis.example.com', port: 6379 },
  { host: 'node2.redis.example.com', port: 6379 },
  { host: 'node3.redis.example.com', port: 6379 },
], {
  // Cluster-specific options
  scaleReads: 'slave',  // Read from replicas for read scaling
  slotsRefreshTimeout: 2000,
  redisOptions: {
    password: process.env.REDIS_PASSWORD,
    tls: {}, // Enable TLS in production
  },
});
 
// Normal operations work transparently
await cluster.set('user:123', 'data');
const data = await cluster.get('user:123');
 
// Hash tags for co-locating related keys
// Keys with same {tag} go to same slot
await cluster.set('{user:123}:profile', JSON.stringify(profile));
await cluster.set('{user:123}:settings', JSON.stringify(settings));
await cluster.set('{user:123}:sessions', JSON.stringify(sessions));
 
// Now multi-key operations work (same slot)
const results = await cluster.mget(
  '{user:123}:profile',
  '{user:123}:settings',
  '{user:123}:sessions'
);
 
// Warning: This WILL fail in cluster mode (different slots)
// await cluster.mget('user:123', 'user:456');  // CrossSlotError
 
// Pipeline with cluster (automatically groups by slot)
const pipeline = cluster.pipeline();
pipeline.set('{order:1}:status', 'pending');
pipeline.set('{order:1}:items', JSON.stringify(items));
await pipeline.exec();

Redis Deployment Options
Deployment	Sharding	HA	Use Case
Single Node	None	None	Development, small workloads
Primary + Replica	None	Manual failover	Read scaling, basic redundancy
Sentinel	None	Automatic failover	HA without sharding
Cluster	Automatic (hash slots)	Automatic failover	Large scale, sharding + HA
Managed (ElastiCache, Redis Cloud)	Provider-managed	Provider-managed	Production with minimal ops

Cluster Mode Limitations

Memcached Fundamentals

Memcached is a simpler, high-performance distributed cache focused purely on caching. Unlike Redis, it offers fewer features but sometimes excels for specific use cases.

Memcached Characteristics:

Memcached Strengths

•Simple key-value model (strings only)
•Multi-threaded (uses all cores efficiently)
•Predictable memory usage (slab allocator)
•No persistence = no disk I/O latency
•Mature, battle-tested at scale (Facebook, Twitter)
•Lower memory overhead per key

Memcached Limitations

•Strings only (no data structures)
•No built-in clustering (client-side sharding)
•No persistence (cache cold on restart)
•No pub/sub or scripting
•Key size limited to 250 bytes
•Value size limited to 1 MB default

memcached-usage.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
import Memcached from 'memcached';
 
// Connect to Memcached cluster (client-side consistent hashing)
const memcached = new Memcached([
  'cache1.example.com:11211',
  'cache2.example.com:11211',
  'cache3.example.com:11211',
], {
  poolSize: 10,
  retries: 2,
  timeout: 5000,
  idle: 30000,
});
 
// Promisify for async/await
const memcachedAsync = {
  get: (key: string): Promise<any> =>
    new Promise((resolve, reject) => {
      memcached.get(key, (err, data) => {
        if (err) reject(err);
        else resolve(data);
      });
    }),
  
  set: (key: string, value: any, ttl: number): Promise<boolean> =>
    new Promise((resolve, reject) => {
      memcached.set(key, value, ttl, (err) => {
        if (err) reject(err);
        else resolve(true);
      });
    }),
  
  delete: (key: string): Promise<boolean> =>
    new Promise((resolve, reject) => {
      memcached.del(key, (err) => {
        if (err) reject(err);
        else resolve(true);
      });
    }),
  
  // Atomic increment
  incr: (key: string, amount: number): Promise<number> =>
    new Promise((resolve, reject) => {
      memcached.incr(key, amount, (err, result) => {
        if (err) reject(err);
        else resolve(result);
      });
    }),
};
 
// Usage
async function cacheUser(userId: string, user: User): Promise<void> {
  // Must serialize to string (no native data structures)
  await memcachedAsync.set(
    `user:${userId}`,
    JSON.stringify(user),
    300  // TTL in seconds
  );
}
 
async function getUser(userId: string): Promise<User | null> {
  const data = await memcachedAsync.get(`user:${userId}`);
  return data ? JSON.parse(data) : null;
}
 
// Multi-get for batch operations
function multiGet(keys: string[]): Promise<Record<string, any>> {
  return new Promise((resolve, reject) => {
    memcached.getMulti(keys, (err, data) => {
      if (err) reject(err);
      else resolve(data);
    });
  });
}

Redis vs Memcached Decision Guide
Consideration	Choose Redis	Choose Memcached
Data structures needed	Yes (lists, sets, sorted sets, hashes)	No (strings only sufficient)
Persistence required	Yes (durability matters)	No (pure cache, DB is source of truth)
Pub/sub messaging	Yes (event notifications, invalidation)	Not needed
Atomic operations	Complex atomics (Lua scripts)	Simple increment/decrement
Memory efficiency	Need predictable memory	Slab allocator more predictable
Multi-threading	Single-threaded bottleneck concern	Multi-threaded for heavy load
Ecosystem	Rich tooling, large community	Simpler, focused tooling

Modern Consensus: Redis Preferred

Cache Consistency Patterns

Distributed caches introduce consistency challenges that don't exist with databases. The cache is a separate system from your source of truth, and keeping them synchronized is non-trivial.

Write Patterns for Consistency:

Cache Write Strategies
Pattern	Description	Consistency	Write Latency	Failure Mode
Cache-Aside	App manages cache separately; write to DB, invalidate cache	Eventually consistent	Fast (DB only)	Stale reads until invalidation
Write-Through	Write to cache and DB together (cache library handles)	Strong if sync	Slower (both writes)	DB failure = cache stale
Write-Behind	Write to cache; async persist to DB later	Eventually consistent	Very fast	Data loss if cache fails before persist
Read-Through	Cache fetches from DB on miss automatically	Eventually consistent	Same as cache-aside	Same as cache-aside

cache-consistency-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
// Pattern 1: Cache-Aside with Invalidation
async function updateUserCacheAside(
  userId: string, 
  updates: Partial<User>
): Promise<User> {
  // Write to database first
  const user = await db.users.update({
    where: { id: userId },
    data: updates,
  });
  
  // Invalidate cache (next read will repopulate)
  await redis.del(`user:${userId}`);
  
  return user;
}
 
// Pattern 2: Write-Through with Cache Update
async function updateUserWriteThrough(
  userId: string,
  updates: Partial<User>
): Promise<User> {
  // Write to database
  const user = await db.users.update({
    where: { id: userId },
    data: updates,
  });
  
  // Update cache with new value (not invalidate)
  await redis.setex(
    `user:${userId}`,
    300,
    JSON.stringify(user)
  );
  
  return user;
}
 
// Pattern 3: Handling the Delete-Then-Read Race
// Problem: Thread A deletes cache, Thread B reads stale DB, caches it,
//          Thread A writes to DB. Cache is now stale.
 
// Solution: Double-delete pattern
async function updateUserDoubleDelete(
  userId: string,
  updates: Partial<User>
): Promise<User> {
  const cacheKey = `user:${userId}`;
  
  // First delete
  await redis.del(cacheKey);
  
  // Write to database
  const user = await db.users.update({
    where: { id: userId },
    data: updates,
  });
  
  // Delay to allow in-flight reads to complete
  setTimeout(async () => {
    // Second delete (cleans up any stale repopulation)
    await redis.del(cacheKey);
  }, 500);
  
  return user;
}
 
// Pattern 4: Lock-based consistency
async function updateUserWithLock(
  userId: string,
  updates: Partial<User>
): Promise<User> {
  const cacheKey = `user:${userId}`;
  const lockKey = `lock:${cacheKey}`;
  
  // Acquire lock
  const lockAcquired = await redis.set(lockKey, '1', 'EX', 5, 'NX');
  if (!lockAcquired) {
    throw new Error('Could not acquire lock');
  }
  
  try {
    // Write to database
    const user = await db.users.update({
      where: { id: userId },
      data: updates,
    });
    
    // Update cache
    await redis.setex(cacheKey, 300, JSON.stringify(user));
    
    return user;
  } finally {
    // Release lock
    await redis.del(lockKey);
  }
}
 
// Pattern 5: Version-based invalidation
interface VersionedCache {
  version: number;
  data: any;
}
 
async function getUserVersioned(userId: string): Promise<User | null> {
  const currentVersion = await redis.get(`user:${userId}:version`);
  const cached = await redis.get(`user:${userId}`);
  
  if (cached) {
    const parsed: VersionedCache = JSON.parse(cached);
    if (parsed.version.toString() === currentVersion) {
      return parsed.data;
    }
    // Version mismatch - cache is stale
  }
  
  // Fetch fresh data
  const user = await db.users.findUnique({ where: { id: userId } });
  
  if (user) {
    const version = parseInt(currentVersion || '0') + 1;
    await redis.setex(`user:${userId}`, 300, JSON.stringify({
      version,
      data: user,
    }));
    await redis.set(`user:${userId}:version`, version.toString());
  }
  
  return user;
}

Cache Invalidation > Cache Update

Cache Failure Handling

Failure Scenarios:

Cache Failure Types and Responses
Failure Type	Symptoms	Correct Response	Incorrect Response
Cache node down	Connection refused/timeout	Fallback to DB; skip cache writes	Fail the request
Cache slow (degraded)	High latency, timeouts	Short timeout; fallback to DB	Wait indefinitely
Cache full (OOM)	Evictions increase; errors	Continue operation; tune eviction	Panic restart
Network partition	Intermittent connectivity	Circuit breaker; fallback	Endless retries
Data corruption	Deserialization errors	Delete and repopulate	Serve corrupt data

cache-resilience.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
import Redis from 'ioredis';
import CircuitBreaker from 'opossum';
 
// Configure Redis with resilience
const redis = new Redis({
  host: 'cache.example.com',
  port: 6379,
  
  // Connection resilience
  retryStrategy: (times) => {
    if (times > 3) return null;  // Stop retrying
    return Math.min(times * 200, 2000);  // Exponential backoff
  },
  
  // Command timeout
  commandTimeout: 500,  // Fast fail
  
  // Connection timeout
  connectTimeout: 3000,
  
  // Auto-reconnect
  maxRetriesPerRequest: 1,
  enableOfflineQueue: false,  // Don't queue when disconnected
});
 
// Circuit breaker for cache operations
const cacheBreaker = new CircuitBreaker(
  async (operation: () => Promise<any>) => operation(),
  {
    timeout: 1000,           // Consider call failed after 1s
    errorThresholdPercentage: 50,  // Open after 50% failures
    resetTimeout: 30000,     // Try again after 30s
    volumeThreshold: 10,     // Minimum calls before tripping
  }
);
 
cacheBreaker.on('open', () => {
  console.warn('Cache circuit breaker OPEN - falling back to database');
});
 
cacheBreaker.on('halfOpen', () => {
  console.info('Cache circuit breaker testing recovery...');
});
 
cacheBreaker.on('close', () => {
  console.info('Cache circuit breaker CLOSED - cache operational');
});
 
// Resilient cache wrapper
class ResilientCache {
  async get<T>(key: string): Promise<T | null> {
    try {
      return await cacheBreaker.fire(async () => {
        const data = await redis.get(key);
        return data ? JSON.parse(data) : null;
      });
    } catch (error) {
      // Circuit open or cache error - return null (cache miss)
      console.warn(`Cache get failed for ${key}:`, error.message);
      return null;
    }
  }
 
  async set(key: string, value: any, ttlSeconds: number): Promise<void> {
    try {
      await cacheBreaker.fire(async () => {
        await redis.setex(key, ttlSeconds, JSON.stringify(value));
      });
    } catch (error) {
      // Cache write failure is non-fatal
      console.warn(`Cache set failed for ${key}:`, error.message);
      // Don't throw - application continues without caching
    }
  }
 
  async delete(key: string): Promise<void> {
    try {
      await cacheBreaker.fire(async () => {
        await redis.del(key);
      });
    } catch (error) {
      // Invalidation failure is concerning but not fatal
      console.error(`Cache delete failed for ${key}:`, error.message);
      // Consider: queue for retry, or accept stale data with TTL
    }
  }
}
 
// Usage with graceful degradation
const cache = new ResilientCache();
 
async function getUser(userId: string): Promise<User> {
  // Try cache (gracefully handles failure)
  const cached = await cache.get<User>(`user:${userId}`);
  if (cached) {
    return cached;
  }
  
  // Fallback to database
  const user = await db.users.findUnique({ where: { id: userId } });
  
  // Attempt to cache (fire-and-forget, non-blocking)
  if (user) {
    cache.set(`user:${userId}`, user, 300).catch(() => {
      // Already logged in ResilientCache
    });
  }
  
  return user;
}

Cache is an Optimization, Not a Dependency

Production Deployment Best Practices

Running distributed caches in production requires attention to memory management, monitoring, security, and operational procedures.

Memory Management:

redis-production-config.conf
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
# Redis production configuration
 
# Memory limits
maxmemory 4gb
maxmemory-policy allkeys-lru  # Evict any key using LRU when memory full
# Alternatives:
# - volatile-lru: Only evict keys with TTL set
# - allkeys-random: Random eviction
# - volatile-ttl: Evict keys closest to expiration
# - noeviction: Return errors when memory full
 
# Persistence (choose based on durability needs)
# Option 1: No persistence (pure cache)
save ""
appendonly no
 
# Option 2: RDB snapshots
save 900 1       # Snapshot if 1 key changed in 15 min
save 300 10      # Snapshot if 10 keys changed in 5 min
save 60 10000    # Snapshot if 10000 keys changed in 1 min
 
# Option 3: AOF (append-only file) for durability
appendonly yes
appendfsync everysec  # Sync every second (balance durability/performance)
 
# Network security
bind 10.0.0.0/8      # Only listen on private network
requirepass <strong-password>
tls-port 6379
tls-cert-file /path/to/cert.pem
tls-key-file /path/to/key.pem
 
# Connection limits
maxclients 10000
timeout 300           # Close idle connections after 5 min
 
# Slow query logging
slowlog-log-slower-than 10000  # Log commands > 10ms
slowlog-max-len 128
 
# Disable dangerous commands in production
rename-command FLUSHALL ""
rename-command FLUSHDB ""
rename-command DEBUG ""
rename-command KEYS ""  # KEYS is O(n), use SCAN instead

Key Metrics to Monitor

•Memory usage — used_memory, used_memory_rss. Alert if approaching maxmemory.
•Hit rate — keyspace_hits / (keyspace_hits + keyspace_misses). Target > 95%.
•Evictions — evicted_keys. Non-zero means memory pressure.
•Connected clients — connected_clients. Spike may indicate connection leak.
•Commands per second — instantaneous_ops_per_sec. Baseline for capacity.
•Latency — Use redis-cli --latency. Should be < 1ms for simple operations.
•Replication lag — master_link_down_since_seconds. Critical for HA.
•Slow queries — Review slowlog periodically for optimization opportunities.

redis-monitoring.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# Redis monitoring commands
 
# Real-time stats
redis-cli INFO stats
redis-cli INFO memory
redis-cli INFO replication
 
# Slow query log
redis-cli SLOWLOG GET 10
 
# Memory analysis
redis-cli MEMORY DOCTOR
 
# Big keys scan (run during low traffic)
redis-cli --bigkeys
 
# Sample key analysis
redis-cli DEBUG OBJECT key-name
 
# Live monitoring
redis-cli MONITOR  # WARNING: Production impact, use briefly
 
# Latency check
redis-cli --latency
 
# Key expiration analysis
redis-cli DEBUG SLEEP 0  # Check key TTL distribution

Use Managed Services for Production

Advanced Distributed Caching Patterns

Beyond basic key-value caching, distributed caches enable sophisticated patterns for coordination, real-time features, and complex data management.

Pattern: Distributed Rate Limiting

advanced-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
// Pattern 1: Sliding Window Rate Limiter
async function checkRateLimit(
  userId: string,
  limit: number,
  windowMs: number
): Promise<{ allowed: boolean; remaining: number }> {
  const key = `rate:${userId}`;
  const now = Date.now();
  const windowStart = now - windowMs;
  
  // Use sorted set with timestamp as score
  const pipeline = redis.pipeline();
  
  // Remove old entries outside window
  pipeline.zremrangebyscore(key, 0, windowStart);
  
  // Add current request
  pipeline.zadd(key, now.toString(), `${now}:${Math.random()}`);
  
  // Count requests in window
  pipeline.zcount(key, windowStart.toString(), now.toString());
  
  // Set TTL for cleanup
  pipeline.expire(key, Math.ceil(windowMs / 1000));
  
  const results = await pipeline.exec();
  const requestCount = results![2][1] as number;
  
  return {
    allowed: requestCount <= limit,
    remaining: Math.max(0, limit - requestCount),
  };
}
 
// Pattern 2: Distributed Lock with Redlock
// For critical sections across multiple instances
 
import Redlock from 'redlock';
 
const redlock = new Redlock(
  [redis], // Array of Redis instances for safety
  {
    driftFactor: 0.01,
    retryCount: 10,
    retryDelay: 200,
    retryJitter: 200,
  }
);
 
async function processWithLock(resourceId: string): Promise<void> {
  let lock;
  try {
    lock = await redlock.acquire(
      [`lock:${resourceId}`],
      5000  // Lock TTL
    );
    
    // Critical section - only one instance executes this
    await performCriticalOperation(resourceId);
    
  } finally {
    if (lock) {
      await lock.release();
    }
  }
}
 
// Pattern 3: Cache-based Pub/Sub for Real-time Invalidation
const subscriber = redis.duplicate();
 
// Subscribe to invalidation channel
await subscriber.subscribe('cache:invalidate');
 
subscriber.on('message', (channel, message) => {
  const { type, id } = JSON.parse(message);
  
  // Invalidate local caches or notify clients
  switch (type) {
    case 'user':
      localCache.delete(`user:${id}`);
      break;
    case 'product':
      localCache.delete(`product:${id}`);
      websockets.broadcastToRoom(`product:${id}`, { type: 'refresh' });
      break;
  }
});
 
// Publish invalidation
async function publishInvalidation(type: string, id: string): Promise<void> {
  await redis.publish('cache:invalidate', JSON.stringify({ type, id }));
}
 
// Pattern 4: Leaderboard with Real-time Updates
class Leaderboard {
  private key: string;
  
  constructor(name: string) {
    this.key = `leaderboard:${name}`;
  }
  
  // Update score atomically
  async updateScore(playerId: string, score: number): Promise<void> {
    await redis.zadd(this.key, score.toString(), playerId);
  }
  
  // Increment score atomically
  async incrementScore(playerId: string, delta: number): Promise<number> {
    return redis.zincrby(this.key, delta, playerId);
  }
  
  // Get top N players
  async getTopPlayers(n: number): Promise<Array<{ playerId: string; score: number }>> {
    const results = await redis.zrevrange(this.key, 0, n - 1, 'WITHSCORES');
    
    const players: Array<{ playerId: string; score: number }> = [];
    for (let i = 0; i < results.length; i += 2) {
      players.push({
        playerId: results[i],
        score: parseFloat(results[i + 1]),
      });
    }
    return players;
  }
  
  // Get player rank (1-indexed)
  async getPlayerRank(playerId: string): Promise<number | null> {
    const rank = await redis.zrevrank(this.key, playerId);
    return rank !== null ? rank + 1 : null;
  }
  
  // Get nearby players (context in leaderboard)
  async getPlayerContext(
    playerId: string,
    contextSize: number = 5
  ): Promise<Array<{ playerId: string; score: number; rank: number }>> {
    const rank = await redis.zrevrank(this.key, playerId);
    if (rank === null) return [];
    
    const start = Math.max(0, rank - contextSize);
    const end = rank + contextSize;
    
    const results = await redis.zrevrange(this.key, start, end, 'WITHSCORES');
    
    const players: Array<{ playerId: string; score: number; rank: number }> = [];
    for (let i = 0; i < results.length; i += 2) {
      players.push({
        playerId: results[i],
        score: parseFloat(results[i + 1]),
        rank: start + (i / 2) + 1,
      });
    }
    return players;
  }
}

Redis as Primary Data Store

Summary and Best Practices

Distributed Caching Best Practices

•Choose Redis unless you have specific Memcached requirements — Redis offers more features with comparable performance.
•Use Redis Cluster for horizontal scaling — When single-node capacity is insufficient. Remember hash tag constraints.
•Implement circuit breakers — Cache failures should never take down your application.
•Set appropriate timeouts — Fast fail (< 1s) is better than blocking threads waiting for slow cache.
•Use pipelines for batch operations — Reduce network round-trips; significant latency improvement.
•Configure eviction policies thoughtfully — allkeys-lru for pure caches; volatile-lru when mixing TTL and non-TTL keys.
•Monitor key metrics continuously — Hit rate, memory, evictions, latency. Alert on anomalies.
•Secure your cache — Use authentication, TLS, and private networks. Exposed Redis is a security vulnerability.
•Handle consistency explicitly — Choose invalidation strategy; understand the trade-offs of cache-aside vs write-through.
•Consider managed services — ElastiCache, Redis Cloud, etc. reduce operational burden significantly.

Module Complete:

This concludes the Caching Layers module. You've learned about caching at every level of the stack:

Browser caching — HTTP cache headers, Cache API, Service Workers
CDN caching — Edge locations, TTL strategies, invalidation
Application caching — In-process caches, memoization, cache key design
Database query caching — Buffer pools, materialized views, query result caching
Distributed caching — Redis, Memcached, cluster architectures, consistency patterns

Together, these layers form a comprehensive caching strategy that can improve performance by orders of magnitude while maintaining data consistency appropriate to your requirements.

Module Complete

5 / 5