System Design (LLD)Cache Invalidation Strategies

Cache Invalidation Strategies

LevelIntermediate

Duration60 mins

TopicCache Invalidation Strategies

2 / 4

Event-Based Invalidation

From Passive Waiting to Active Notification

Time-based expiration has a fundamental limitation: it doesn't know when data actually changes. A cache entry might expire 5 seconds after being stored, even if the underlying data hasn't changed in years. Conversely, data might change milliseconds after caching, yet the stale value is served for the entire TTL duration.

Event-based invalidation flips this model on its head. Instead of guessing when data might be stale, we explicitly notify the cache when data does change. The source of truth emits events like "User 123 was updated" or "Product 456 was deleted," and cache layers subscribe to these events to invalidate or refresh affected entries.

This approach provides stronger consistency guarantees—cache invalidation happens in response to actual changes, not arbitrary time boundaries. But this power comes with significant complexity: event delivery, ordering guarantees, failure handling, and the challenge of knowing which cache entries are affected by a given event.

What You Will Learn

By the end of this page, you will understand how to design event-based cache invalidation systems. You'll learn event propagation architectures, cache invalidation subscription patterns, how to handle invalidation failures gracefully, and when event-based invalidation is worth the complexity it introduces.

The Event-Driven Invalidation Model

In event-based invalidation, the system is structured around three key actors:

1. Event Producers — Services that modify data and emit change events 2. Event Channel — The messaging infrastructure that delivers events 3. Cache Invalidators — Services that receive events and invalidate cache entries

The flow is conceptually simple:

[Write Operation] → [Emit Event] → [Event Channel] → [Invalidate Cache]

But each step has important design decisions that affect consistency, latency, and reliability.

Converting Mermaid diagram...

Key Insight: Eventual Consistency

The diagram above reveals an important truth: there's a window between the database commit and the cache invalidation where the cache contains stale data. This is inevitable in any distributed system unless you're willing to accept significant performance penalties (synchronous invalidation).

The question isn't whether to accept eventual consistency—it's how small you can make the inconsistency window and how you handle the edge cases.

event-invalidation-architecture.ts
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
/**
 * Event-Based Cache Invalidation Architecture
 * 
 * This example demonstrates the core components and flow
 * of an event-driven invalidation system.
 */
 
// ============================================
// Event Definitions
// ============================================
 
interface CacheInvalidationEvent {
    type: string;
    entityType: string;
    entityId: string;
    timestamp: number;
    correlationId: string;
    metadata?: Record<string, unknown>;
}
 
interface UserUpdatedEvent extends CacheInvalidationEvent {
    type: 'UserUpdated';
    entityType: 'User';
    changes?: {
        fields: string[];  // Which fields changed
    };
}
 
interface UserDeletedEvent extends CacheInvalidationEvent {
    type: 'UserDeleted';
    entityType: 'User';
}
 
interface ProductPriceChangedEvent extends CacheInvalidationEvent {
    type: 'ProductPriceChanged';
    entityType: 'Product';
    previousPrice?: number;
    newPrice: number;
}
 
// ============================================
// Event Producer (Write Service)
// ============================================
 
class UserService {
    constructor(
        private userRepository: UserRepository,
        private eventPublisher: EventPublisher,
    ) {}
 
    async updateUser(userId: string, updates: Partial<User>): Promise<User> {
        // 1. Perform the write
        const updatedUser = await this.userRepository.update(userId, updates);
 
        // 2. Emit invalidation event
        const event: UserUpdatedEvent = {
            type: 'UserUpdated',
            entityType: 'User',
            entityId: userId,
            timestamp: Date.now(),
            correlationId: generateCorrelationId(),
            changes: {
                fields: Object.keys(updates),
            },
        };
 
        // Important: Fire-and-forget vs. transactional
        // Option A: Fire-and-forget (faster, less reliable)
        this.eventPublisher.publish('cache.invalidation', event)
            .catch(err => console.error('Event publish failed:', err));
 
        // Option B: Transactional (slower, more reliable)
        // await this.eventPublisher.publishWithinTransaction(
        //     'cache.invalidation', 
        //     event
        // );
 
        return updatedUser;
    }
 
    async deleteUser(userId: string): Promise<void> {
        await this.userRepository.delete(userId);
 
        const event: UserDeletedEvent = {
            type: 'UserDeleted',
            entityType: 'User',
            entityId: userId,
            timestamp: Date.now(),
            correlationId: generateCorrelationId(),
        };
 
        await this.eventPublisher.publish('cache.invalidation', event);
    }
}
 
// ============================================
// Event Channel Interface
// ============================================
 
interface EventPublisher {
    publish(topic: string, event: CacheInvalidationEvent): Promise<void>;
}
 
interface EventSubscriber {
    subscribe(
        topic: string,
        handler: (event: CacheInvalidationEvent) => Promise<void>
    ): void;
}
 
// ============================================
// Cache Invalidation Service
// ============================================
 
class CacheInvalidationService {
    private handlers = new Map<string, (event: CacheInvalidationEvent) => Promise<void>>();
 
    constructor(
        private eventSubscriber: EventSubscriber,
        private cache: CacheClient,
    ) {
        this.registerHandlers();
        this.startListening();
    }
 
    private registerHandlers(): void {
        // User-related invalidations
        this.handlers.set('UserUpdated', async (event) => {
            const userEvent = event as UserUpdatedEvent;
            
            // Invalidate user cache
            await this.cache.delete(`user:${userEvent.entityId}`);
            
            // If profile fields changed, invalidate profile cache too
            if (userEvent.changes?.fields.includes('name') ||
                userEvent.changes?.fields.includes('avatar')) {
                await this.cache.delete(`profile:${userEvent.entityId}`);
            }
            
            // Invalidate any aggregates that include this user
            await this.cache.deleteByPattern(`*:includes:user:${userEvent.entityId}*`);
        });
 
        this.handlers.set('UserDeleted', async (event) => {
            // More aggressive invalidation on delete
            await this.cache.deleteByPattern(`user:${event.entityId}*`);
            await this.cache.deleteByPattern(`*:user:${event.entityId}*`);
        });
 
        this.handlers.set('ProductPriceChanged', async (event) => {
            const productEvent = event as ProductPriceChangedEvent;
            
            // Invalidate product cache
            await this.cache.delete(`product:${productEvent.entityId}`);
            
            // Invalidate category listings (price affects sorting)
            await this.cache.deleteByPattern('category:*:products');
            
            // Invalidate search results (might include price)
            await this.cache.deleteByPattern('search:*:products');
        });
    }
 
    private startListening(): void {
        this.eventSubscriber.subscribe('cache.invalidation', async (event) => {
            const handler = this.handlers.get(event.type);
            
            if (handler) {
                try {
                    await handler(event);
                    console.log(`Invalidated cache for ${event.type}:${event.entityId}`);
                } catch (error) {
                    console.error(`Invalidation failed for ${event.type}:${event.entityId}:`, error);
                    // Could retry or send to dead letter queue
                }
            } else {
                console.warn(`No handler for event type: ${event.type}`);
            }
        });
    }
}
 
// Type definitions for the example
interface User { id: string; name: string; email: string; }
interface UserRepository {
    update(userId: string, updates: Partial<User>): Promise<User>;
    delete(userId: string): Promise<void>;
}
interface CacheClient {
    delete(key: string): Promise<void>;
    deleteByPattern(pattern: string): Promise<void>;
}
declare function generateCorrelationId(): string;

Design Decision: Granularity of Events

Event granularity is a critical design choice. Fine-grained events (UserNameChanged, UserEmailChanged) enable precise invalidation but increase complexity. Coarse-grained events (UserUpdated) are simpler but may cause unnecessary invalidations. Start coarse and refine based on observed cache churn.

Event Propagation Architectures

How events travel from producers to invalidators significantly impacts system reliability, latency, and operational complexity. Three primary architectures dominate in practice.

Message Queue (Kafka, RabbitMQ, SQS)

The most common architecture. Events are published to a message queue and consumed by cache invalidation services.

Advantages:

Durable: Events survive producer/consumer restarts
Decoupled: Producers and consumers operate independently
Scalable: Multiple consumers can parallelize invalidation
Reliable: At-least-once delivery guarantees

Disadvantages:

Higher latency (10ms - 100ms typical)
Ordering only guaranteed per partition
Operational complexity of running message infrastructure
Potential for duplicate events (at-least-once semantics)

kafka-invalidation.ts
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
/**
 * Kafka-Based Cache Invalidation
 */
import { Kafka, Consumer, Producer, EachMessagePayload } from 'kafkajs';
 
const kafka = new Kafka({ brokers: ['kafka:9092'] });
 
class KafkaCacheInvalidator {
    private consumer: Consumer;
    private producer: Producer;
 
    constructor(private cache: CacheClient) {
        this.consumer = kafka.consumer({ 
            groupId: 'cache-invalidation-service',
            // Ensure ordering within entity by partitioning on entityId
        });
        this.producer = kafka.producer();
    }
 
    async start(): Promise<void> {
        await this.consumer.connect();
        await this.consumer.subscribe({ 
            topic: 'data-changes',
            fromBeginning: false, // Only new events
        });
 
        await this.consumer.run({
            eachMessage: async (payload: EachMessagePayload) => {
                const event = JSON.parse(payload.message.value!.toString());
                
                try {
                    await this.handleEvent(event);
                } catch (error) {
                    // Failed to process - will be retried or sent to DLQ
                    console.error('Invalidation failed:', error);
                    throw error; // Re-throw to prevent commit
                }
            },
        });
    }
 
    private async handleEvent(event: DataChangeEvent): Promise<void> {
        const cacheKeys = this.deriveCacheKeys(event);
        
        // Batch delete for efficiency
        await Promise.all(cacheKeys.map(key => this.cache.delete(key)));
        
        console.log(`Invalidated ${cacheKeys.length} keys for ${event.type}`);
    }
 
    private deriveCacheKeys(event: DataChangeEvent): string[] {
        // Map event type to affected cache keys
        // This is where domain knowledge about caching structure lives
        switch (event.entityType) {
            case 'User':
                return [
                    `user:${event.entityId}`,
                    `profile:${event.entityId}`,
                    `user:permissions:${event.entityId}`,
                ];
            case 'Product':
                return [
                    `product:${event.entityId}`,
                    'catalog:featured',  // Aggregates may need invalidation
                ];
            default:
                return [`${event.entityType.toLowerCase()}:${event.entityId}`];
        }
    }
}
 
interface DataChangeEvent {
    type: string;
    entityType: string;
    entityId: string;
    timestamp: number;
}

Event Propagation Architecture Comparison
Architecture	Latency	Reliability	Complexity	Best For
Message Queue	10-100ms	High (durable)	Medium	Primary invalidation, cross-service
Redis Pub/Sub	<1ms	Low (fire-forget)	Low	Local cache sync, supplementary
CDC	1-10s	Very High	High	Guaranteed capture, legacy systems

Hybrid Architecture

Production systems often combine approaches: CDC captures all changes reliably, publishes to Kafka for durable delivery, and Redis Pub/Sub provides fast local cache sync. This layered approach maximizes both reliability and speed.

Mapping Events to Cache Keys: The Domain Challenge

The trickiest part of event-based invalidation isn't propagating events—it's knowing which cache keys to invalidate when a given event arrives. This mapping requires deep understanding of your caching structure and data relationships.

The Challenge: Derived Data

Caches often store derived or aggregated data. When a Product changes, you need to invalidate:

The product's direct cache entry (product:123)
Search results that might include this product (search:query:widgets)
Category pages listing this product (category:electronics)
User wishlists containing this product (user:456:wishlist)
Price comparison aggregates (category:electronics:price-range)

Missing any of these leads to serving stale data. Invalidating too aggressively destroys your cache hit rate.

cache-key-mapping.ts
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
/**
 * Cache Key Mapping Strategies
 * 
 * Three approaches to mapping events to cache keys,
 * each with different tradeoffs.
 */
 
// ============================================
// Strategy 1: Static Mapping (Simple but Limited)
// ============================================
 
/**
 * Hard-coded rules mapping entity types to cache key patterns.
 * Simple to implement but doesn't scale with complexity.
 */
class StaticCacheKeyMapper {
    private rules: Map<string, (entityId: string) => string[]> = new Map([
        ['User', (id) => [
            `user:${id}`,
            `profile:${id}`,
            `user:${id}:permissions`,
        ]],
        ['Product', (id) => [
            `product:${id}`,
            // Note: We can't know which categories without querying!
        ]],
        ['Order', (id) => [
            `order:${id}`,
        ]],
    ]);
 
    getKeysForEvent(entityType: string, entityId: string): string[] {
        const mapper = this.rules.get(entityType);
        return mapper ? mapper(entityId) : [];
    }
}
 
// ============================================
// Strategy 2: Inverse Index (Powerful but Complex)
// ============================================
 
/**
 * Maintain an index of which cache keys contain which entities.
 * When an entity changes, look up all keys that reference it.
 */
class InverseIndexCacheKeyMapper {
    private index = new Map<string, Set<string>>(); // entityRef -> cacheKeys
 
    /**
     * When caching a value, register all entities it contains
     */
    registerCacheEntry(
        cacheKey: string,
        containedEntities: { type: string; id: string }[]
    ): void {
        for (const entity of containedEntities) {
            const entityRef = `${entity.type}:${entity.id}`;
            
            if (!this.index.has(entityRef)) {
                this.index.set(entityRef, new Set());
            }
            this.index.get(entityRef)!.add(cacheKey);
        }
    }
 
    /**
     * When uncaching or invalidating, clean up the index
     */
    unregisterCacheEntry(cacheKey: string): void {
        for (const [entityRef, keys] of this.index) {
            keys.delete(cacheKey);
            if (keys.size === 0) {
                this.index.delete(entityRef);
            }
        }
    }
 
    /**
     * Get all cache keys that reference this entity
     */
    getKeysForEntity(entityType: string, entityId: string): string[] {
        const entityRef = `${entityType}:${entityId}`;
        const keys = this.index.get(entityRef);
        return keys ? Array.from(keys) : [];
    }
}
 
// Usage of Inverse Index:
const indexMapper = new InverseIndexCacheKeyMapper();
 
// When caching a search result that includes products
const searchResults = { query: 'blue widgets', products: [
    { id: 'prod-1', name: 'Blue Widget A' },
    { id: 'prod-2', name: 'Blue Widget B' },
]};
 
indexMapper.registerCacheEntry(
    'search:blue-widgets:page-1',
    [
        { type: 'Product', id: 'prod-1' },
        { type: 'Product', id: 'prod-2' },
    ]
);
 
// When product prod-1 is updated:
const keysToInvalidate = indexMapper.getKeysForEntity('Product', 'prod-1');
// Returns: ['search:blue-widgets:page-1']
 
 
// ============================================
// Strategy 3: Tag-Based Invalidation
// ============================================
 
/**
 * Tag cache entries with metadata, then invalidate by tag.
 * Similar to inverse index but uses Redis or cache-native features.
 */
class TagBasedCacheClient {
    constructor(private redis: Redis) {}
 
    /**
     * Set with tags - allows invalidation by any tag
     */
    async setWithTags(
        key: string,
        value: unknown,
        ttlSeconds: number,
        tags: string[]
    ): Promise<void> {
        const pipeline = this.redis.pipeline();
        
        // Store the value
        pipeline.setex(key, ttlSeconds, JSON.stringify(value));
        
        // Add key to tag sets
        for (const tag of tags) {
            pipeline.sadd(`tag:${tag}`, key);
            pipeline.expire(`tag:${tag}`, ttlSeconds + 60); // Tag outlives entries slightly
        }
        
        // Store tags on the key for cleanup
        pipeline.sadd(`key:${key}:tags`, ...tags);
        pipeline.expire(`key:${key}:tags`, ttlSeconds);
        
        await pipeline.exec();
    }
 
    /**
     * Invalidate all entries with a given tag
     */
    async invalidateByTag(tag: string): Promise<number> {
        const tagKey = `tag:${tag}`;
        const keysToDelete = await this.redis.smembers(tagKey);
        
        if (keysToDelete.length === 0) {
            return 0;
        }
        
        const pipeline = this.redis.pipeline();
        
        // Delete all tagged entries
        for (const key of keysToDelete) {
            pipeline.del(key);
            pipeline.del(`key:${key}:tags`);
        }
        
        // Delete the tag set itself
        pipeline.del(tagKey);
        
        await pipeline.exec();
        return keysToDelete.length;
    }
 
    async get<T>(key: string): Promise<T | null> {
        const value = await this.redis.get(key);
        return value ? JSON.parse(value) : null;
    }
}
 
// Usage of Tag-Based Invalidation:
const tagCache = new TagBasedCacheClient(redis);
 
// Cache product with category tags
await tagCache.setWithTags(
    'product:123',
    { id: '123', name: 'Widget', price: 9.99 },
    3600,
    ['product:123', 'category:electronics', 'featured']
);
 
// Cache category listing with product tags
await tagCache.setWithTags(
    'category:electronics:page-1',
    [{ id: '123' }, { id: '456' }],
    3600,
    ['category:electronics', 'product:123', 'product:456']
);
 
// When product 123 changes, invalidate by tag:
const invalidatedCount = await tagCache.invalidateByTag('product:123');
// Invalidates both product:123 AND category:electronics:page-1

Cache Key Mapping Strategy Comparison
Strategy	Accuracy	Memory Overhead	Complexity	Query Cost
Static Mapping	Low (misses derived)	None	Low	None
Inverse Index	High	Medium-High	Medium	Per-cache write
Tag-Based	High	Medium	Medium	Per-cache write

The Cascade Problem

Be careful with cascading invalidations. If updating a Category invalidates all Products in that category, and each Product invalidation triggers Category aggregate invalidation, you can create invalidation storms that defeat caching entirely. Use circuit breakers and rate limiting on invalidation paths.

Failure Handling: When Invalidation Fails

In distributed systems, failures are inevitable. Event-based invalidation introduces several failure modes that must be explicitly handled.

Failure Modes:

Event loss: Message queue loses events (extremely rare with proper config)
Consumer failure: Invalidator crashes before processing event
Cache unreachable: Cache cluster unavailable when invalidation tried
Partial invalidation: Some keys invalidated, others fail
Event duplication: Same event processed multiple times

Each requires different handling strategies.

failure-handling.ts
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
/**
 * Robust Event-Based Invalidation with Failure Handling
 */
 
interface InvalidationResult {
    success: boolean;
    keysInvalidated: string[];
    keysFailed: string[];
    error?: Error;
}
 
class ResilientCacheInvalidator {
    private deadLetterQueue: DeadLetterQueue;
    private processedEvents: Set<string> = new Set(); // Dedup window
    
    constructor(
        private cache: CacheClient,
        private config: InvalidatorConfig,
    ) {
        this.deadLetterQueue = new DeadLetterQueue(config.dlqTopic);
    }
 
    /**
     * Main entry point - processes invalidation event with full error handling
     */
    async processEvent(event: InvalidationEvent): Promise<InvalidationResult> {
        const eventId = event.correlationId;
        
        // 1. Idempotency check - handle duplicate events
        if (this.processedEvents.has(eventId)) {
            console.log(`Duplicate event ${eventId}, skipping`);
            return { success: true, keysInvalidated: [], keysFailed: [] };
        }
        
        const keysToInvalidate = this.deriveKeysFromEvent(event);
        const keysInvalidated: string[] = [];
        const keysFailed: string[] = [];
        
        // 2. Attempt invalidation with retries
        for (const key of keysToInvalidate) {
            const success = await this.invalidateWithRetry(key);
            if (success) {
                keysInvalidated.push(key);
            } else {
                keysFailed.push(key);
            }
        }
        
        // 3. Handle partial failures
        if (keysFailed.length > 0) {
            if (keysFailed.length === keysToInvalidate.length) {
                // Complete failure - send to DLQ for later retry
                await this.deadLetterQueue.send(event, 'complete_failure');
                return {
                    success: false,
                    keysInvalidated,
                    keysFailed,
                    error: new Error('All invalidations failed'),
                };
            } else {
                // Partial failure - log and potentially alert
                console.warn(`Partial invalidation: ${keysFailed.length} failed`, keysFailed);
                
                // Send failed keys as separate event
                await this.deadLetterQueue.send({
                    ...event,
                    metadata: { ...event.metadata, failedKeys: keysFailed },
                }, 'partial_failure');
            }
        }
        
        // 4. Mark event as processed (with expiration)
        this.markAsProcessed(eventId);
        
        return { success: keysFailed.length === 0, keysInvalidated, keysFailed };
    }
 
    /**
     * Invalidate with exponential backoff retry
     */
    private async invalidateWithRetry(key: string): Promise<boolean> {
        const maxRetries = this.config.maxRetries;
        const baseDelay = this.config.retryDelayMs;
        
        for (let attempt = 0; attempt <= maxRetries; attempt++) {
            try {
                await this.cache.delete(key);
                return true;
            } catch (error) {
                if (attempt === maxRetries) {
                    console.error(`Failed to invalidate ${key} after ${maxRetries} retries`);
                    return false;
                }
                
                // Exponential backoff with jitter
                const delay = baseDelay * Math.pow(2, attempt) * (0.5 + Math.random());
                await this.sleep(delay);
            }
        }
        
        return false;
    }
 
    /**
     * Track processed events to handle duplicates
     * Uses sliding window to bound memory
     */
    private markAsProcessed(eventId: string): void {
        this.processedEvents.add(eventId);
        
        // Clean up old entries (simple sliding window)
        if (this.processedEvents.size > this.config.dedupWindowSize) {
            const iterator = this.processedEvents.values();
            // Remove oldest entries
            for (let i = 0; i < this.config.dedupWindowSize / 2; i++) {
                const oldest = iterator.next().value;
                if (oldest) this.processedEvents.delete(oldest);
            }
        }
    }
 
    private deriveKeysFromEvent(event: InvalidationEvent): string[] {
        // Implementation depends on your key mapping strategy
        return [`${event.entityType.toLowerCase()}:${event.entityId}`];
    }
 
    private sleep(ms: number): Promise<void> {
        return new Promise(resolve => setTimeout(resolve, ms));
    }
}
 
/**
 * Dead Letter Queue handler for failed invalidations
 */
class DeadLetterQueue {
    constructor(private topic: string) {}
 
    async send(event: InvalidationEvent, reason: string): Promise<void> {
        const dlqEvent = {
            originalEvent: event,
            failureReason: reason,
            failedAt: Date.now(),
            retryCount: (event.metadata?.retryCount as number || 0) + 1,
        };
 
        // Publish to DLQ topic for manual or automated retry
        await publishToKafka(this.topic, dlqEvent);
        
        // Alert if retry count is high
        if (dlqEvent.retryCount > 3) {
            await sendAlert(`Invalidation failing repeatedly for ${event.entityType}:${event.entityId}`);
        }
    }
}
 
/**
 * Fallback: TTL as Safety Net
 * 
 * Even with event-based invalidation, always set a TTL.
 * If all invalidation mechanisms fail, TTL ensures eventual consistency.
 */
class SafetyNetCache {
    private readonly SAFETY_TTL_SECONDS = 3600; // 1 hour fallback
 
    constructor(private cache: CacheClient) {}
 
    async set<T>(key: string, value: T, ttlSeconds?: number): Promise<void> {
        // Always apply safety TTL, even if caller doesn't specify
        const effectiveTtl = ttlSeconds ?? this.SAFETY_TTL_SECONDS;
        const maxTtl = Math.min(effectiveTtl, this.SAFETY_TTL_SECONDS);
        
        await this.cache.setex(key, maxTtl, JSON.stringify(value));
    }
}
 
interface InvalidationEvent {
    type: string;
    entityType: string;
    entityId: string;
    timestamp: number;
    correlationId: string;
    metadata?: Record<string, unknown>;
}
 
interface InvalidatorConfig {
    maxRetries: number;
    retryDelayMs: number;
    dedupWindowSize: number;
    dlqTopic: string;
}

The Belt-and-Suspenders Approach

Best practice: Combine event-based invalidation with a TTL safety net. Event-based provides fast consistency, while TTL ensures that even if events are lost, stale data is eventually refreshed. The TTL should be long enough not to undermine your event system (e.g., 1 hour vs. typical event propagation of seconds).

When to Use Event-Based Invalidation

Event-based invalidation adds complexity. It's not always the right choice. Understanding when the complexity is justified—and when simpler approaches suffice—is crucial for pragmatic system design.

Use Event-Based When

•Low latency consistency required — Users expect to see their changes immediately (e.g., profile updates, cart modifications)
•Irregular update patterns — Data changes unpredictably, making TTL-based invalidation wasteful or stale
•High write-to-read ratio — Writes are frequent enough that stale reads significantly impact UX
•Multi-system caches — Multiple services cache the same data and need coordinated invalidation
•Regulatory requirements — Compliance mandates that stale data exposure is minimized
•Already have event infrastructure — You're already using Kafka/RabbitMQ for other purposes

Stick with TTL When

•Static or slowly-changing data — Configuration, reference data that changes weekly/monthly
•High read-to-write ratio — Reads vastly outnumber writes (1000:1+), so staleness is rare
•Eventual consistency is acceptable — Analytics, recommendations, non-critical aggregates
•Simple architecture — Single service, single cache, no distributed coordination needed
•Limited infrastructure — Adding message queues would significantly increase operational burden
•Predictable update patterns — Data refreshes on a schedule (hourly reports, daily batches)

Decision Matrix: Event-Based vs TTL-Only Invalidation
Data Type	Update Pattern	Consistency Need	Recommended Strategy
User profile	On user action	Immediate	Event-based + short TTL
Product catalog	Hourly import	Minutes acceptable	TTL (sync with import)
Session data	Continuous	Immediate	Event-based (logout) + sliding TTL
Search results	Continuous	Minutes acceptable	Long TTL + event on major changes
Feature flags	Rare manual updates	Minutes acceptable	TTL + manual invalidation API
Analytics	Batch computed	Hours acceptable	Long TTL only

Progressive Adoption

Start with TTL-only. Add event-based invalidation for specific high-impact cache types where staleness causes visible problems. Don't architect for event-based invalidation across all caches until you've proven the value and built the operational muscle.

Summary: Event-Based Invalidation

Event-based invalidation provides stronger consistency guarantees than time-based expiration by proactively notifying caches when data changes. However, it introduces significant complexity in event propagation, key mapping, and failure handling.

Key Takeaways

•Event-based invalidation is proactive — Changes trigger immediate cache updates, minimizing staleness windows.
•Multiple architectures exist — Message queues for reliability, Redis Pub/Sub for speed, CDC for guaranteed capture. Choose based on your consistency and latency requirements.
•Key mapping is the hard part — Knowing which cache keys to invalidate when an entity changes requires careful design of inverse indexes or tag-based systems.
•Failures must be handled explicitly — Retries, dead letter queues, and idempotency checks are essential for production systems.
•Always keep TTL as a safety net — Even with perfect event delivery, TTL ensures eventual consistency if events fail.
•Complexity must be justified — Event-based invalidation isn't always necessary. Use for high-impact, consistency-critical caches; stick with TTL for everything else.

What's next:

We've covered time-based and event-based invalidation. But what happens when cached data evolves structurally—when the shape of the data changes, not just its values? The next page explores cache versioning, a strategy for handling schema evolution and gradual rollouts without breaking caches.

Page Complete

You now understand event-based cache invalidation: the architectural patterns, key mapping strategies, failure handling approaches, and decision criteria for when to adopt this more complex but more consistent approach.

2 / 4

Loading learning content...

System Design (LLD)Cache Invalidation Strategies

Cache Invalidation Strategies

LevelIntermediate

Duration60 mins

TopicCache Invalidation Strategies

2 / 4

Event-Based Invalidation

From Passive Waiting to Active Notification

What You Will Learn

The Event-Driven Invalidation Model

In event-based invalidation, the system is structured around three key actors:

The flow is conceptually simple:

[Write Operation] → [Emit Event] → [Event Channel] → [Invalidate Cache]

But each step has important design decisions that affect consistency, latency, and reliability.

Converting Mermaid diagram...

Key Insight: Eventual Consistency

The question isn't whether to accept eventual consistency—it's how small you can make the inconsistency window and how you handle the edge cases.

event-invalidation-architecture.ts
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
/**
 * Event-Based Cache Invalidation Architecture
 * 
 * This example demonstrates the core components and flow
 * of an event-driven invalidation system.
 */
 
// ============================================
// Event Definitions
// ============================================
 
interface CacheInvalidationEvent {
    type: string;
    entityType: string;
    entityId: string;
    timestamp: number;
    correlationId: string;
    metadata?: Record<string, unknown>;
}
 
interface UserUpdatedEvent extends CacheInvalidationEvent {
    type: 'UserUpdated';
    entityType: 'User';
    changes?: {
        fields: string[];  // Which fields changed
    };
}
 
interface UserDeletedEvent extends CacheInvalidationEvent {
    type: 'UserDeleted';
    entityType: 'User';
}
 
interface ProductPriceChangedEvent extends CacheInvalidationEvent {
    type: 'ProductPriceChanged';
    entityType: 'Product';
    previousPrice?: number;
    newPrice: number;
}
 
// ============================================
// Event Producer (Write Service)
// ============================================
 
class UserService {
    constructor(
        private userRepository: UserRepository,
        private eventPublisher: EventPublisher,
    ) {}
 
    async updateUser(userId: string, updates: Partial<User>): Promise<User> {
        // 1. Perform the write
        const updatedUser = await this.userRepository.update(userId, updates);
 
        // 2. Emit invalidation event
        const event: UserUpdatedEvent = {
            type: 'UserUpdated',
            entityType: 'User',
            entityId: userId,
            timestamp: Date.now(),
            correlationId: generateCorrelationId(),
            changes: {
                fields: Object.keys(updates),
            },
        };
 
        // Important: Fire-and-forget vs. transactional
        // Option A: Fire-and-forget (faster, less reliable)
        this.eventPublisher.publish('cache.invalidation', event)
            .catch(err => console.error('Event publish failed:', err));
 
        // Option B: Transactional (slower, more reliable)
        // await this.eventPublisher.publishWithinTransaction(
        //     'cache.invalidation', 
        //     event
        // );
 
        return updatedUser;
    }
 
    async deleteUser(userId: string): Promise<void> {
        await this.userRepository.delete(userId);
 
        const event: UserDeletedEvent = {
            type: 'UserDeleted',
            entityType: 'User',
            entityId: userId,
            timestamp: Date.now(),
            correlationId: generateCorrelationId(),
        };
 
        await this.eventPublisher.publish('cache.invalidation', event);
    }
}
 
// ============================================
// Event Channel Interface
// ============================================
 
interface EventPublisher {
    publish(topic: string, event: CacheInvalidationEvent): Promise<void>;
}
 
interface EventSubscriber {
    subscribe(
        topic: string,
        handler: (event: CacheInvalidationEvent) => Promise<void>
    ): void;
}
 
// ============================================
// Cache Invalidation Service
// ============================================
 
class CacheInvalidationService {
    private handlers = new Map<string, (event: CacheInvalidationEvent) => Promise<void>>();
 
    constructor(
        private eventSubscriber: EventSubscriber,
        private cache: CacheClient,
    ) {
        this.registerHandlers();
        this.startListening();
    }
 
    private registerHandlers(): void {
        // User-related invalidations
        this.handlers.set('UserUpdated', async (event) => {
            const userEvent = event as UserUpdatedEvent;
            
            // Invalidate user cache
            await this.cache.delete(`user:${userEvent.entityId}`);
            
            // If profile fields changed, invalidate profile cache too
            if (userEvent.changes?.fields.includes('name') ||
                userEvent.changes?.fields.includes('avatar')) {
                await this.cache.delete(`profile:${userEvent.entityId}`);
            }
            
            // Invalidate any aggregates that include this user
            await this.cache.deleteByPattern(`*:includes:user:${userEvent.entityId}*`);
        });
 
        this.handlers.set('UserDeleted', async (event) => {
            // More aggressive invalidation on delete
            await this.cache.deleteByPattern(`user:${event.entityId}*`);
            await this.cache.deleteByPattern(`*:user:${event.entityId}*`);
        });
 
        this.handlers.set('ProductPriceChanged', async (event) => {
            const productEvent = event as ProductPriceChangedEvent;
            
            // Invalidate product cache
            await this.cache.delete(`product:${productEvent.entityId}`);
            
            // Invalidate category listings (price affects sorting)
            await this.cache.deleteByPattern('category:*:products');
            
            // Invalidate search results (might include price)
            await this.cache.deleteByPattern('search:*:products');
        });
    }
 
    private startListening(): void {
        this.eventSubscriber.subscribe('cache.invalidation', async (event) => {
            const handler = this.handlers.get(event.type);
            
            if (handler) {
                try {
                    await handler(event);
                    console.log(`Invalidated cache for ${event.type}:${event.entityId}`);
                } catch (error) {
                    console.error(`Invalidation failed for ${event.type}:${event.entityId}:`, error);
                    // Could retry or send to dead letter queue
                }
            } else {
                console.warn(`No handler for event type: ${event.type}`);
            }
        });
    }
}
 
// Type definitions for the example
interface User { id: string; name: string; email: string; }
interface UserRepository {
    update(userId: string, updates: Partial<User>): Promise<User>;
    delete(userId: string): Promise<void>;
}
interface CacheClient {
    delete(key: string): Promise<void>;
    deleteByPattern(pattern: string): Promise<void>;
}
declare function generateCorrelationId(): string;

Design Decision: Granularity of Events

Event Propagation Architectures

How events travel from producers to invalidators significantly impacts system reliability, latency, and operational complexity. Three primary architectures dominate in practice.

Message Queue (Kafka, RabbitMQ, SQS)

The most common architecture. Events are published to a message queue and consumed by cache invalidation services.

Advantages:

Durable: Events survive producer/consumer restarts
Decoupled: Producers and consumers operate independently
Scalable: Multiple consumers can parallelize invalidation
Reliable: At-least-once delivery guarantees

Disadvantages:

Higher latency (10ms - 100ms typical)
Ordering only guaranteed per partition
Operational complexity of running message infrastructure
Potential for duplicate events (at-least-once semantics)

kafka-invalidation.ts
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
/**
 * Kafka-Based Cache Invalidation
 */
import { Kafka, Consumer, Producer, EachMessagePayload } from 'kafkajs';
 
const kafka = new Kafka({ brokers: ['kafka:9092'] });
 
class KafkaCacheInvalidator {
    private consumer: Consumer;
    private producer: Producer;
 
    constructor(private cache: CacheClient) {
        this.consumer = kafka.consumer({ 
            groupId: 'cache-invalidation-service',
            // Ensure ordering within entity by partitioning on entityId
        });
        this.producer = kafka.producer();
    }
 
    async start(): Promise<void> {
        await this.consumer.connect();
        await this.consumer.subscribe({ 
            topic: 'data-changes',
            fromBeginning: false, // Only new events
        });
 
        await this.consumer.run({
            eachMessage: async (payload: EachMessagePayload) => {
                const event = JSON.parse(payload.message.value!.toString());
                
                try {
                    await this.handleEvent(event);
                } catch (error) {
                    // Failed to process - will be retried or sent to DLQ
                    console.error('Invalidation failed:', error);
                    throw error; // Re-throw to prevent commit
                }
            },
        });
    }
 
    private async handleEvent(event: DataChangeEvent): Promise<void> {
        const cacheKeys = this.deriveCacheKeys(event);
        
        // Batch delete for efficiency
        await Promise.all(cacheKeys.map(key => this.cache.delete(key)));
        
        console.log(`Invalidated ${cacheKeys.length} keys for ${event.type}`);
    }
 
    private deriveCacheKeys(event: DataChangeEvent): string[] {
        // Map event type to affected cache keys
        // This is where domain knowledge about caching structure lives
        switch (event.entityType) {
            case 'User':
                return [
                    `user:${event.entityId}`,
                    `profile:${event.entityId}`,
                    `user:permissions:${event.entityId}`,
                ];
            case 'Product':
                return [
                    `product:${event.entityId}`,
                    'catalog:featured',  // Aggregates may need invalidation
                ];
            default:
                return [`${event.entityType.toLowerCase()}:${event.entityId}`];
        }
    }
}
 
interface DataChangeEvent {
    type: string;
    entityType: string;
    entityId: string;
    timestamp: number;
}

Event Propagation Architecture Comparison
Architecture	Latency	Reliability	Complexity	Best For
Message Queue	10-100ms	High (durable)	Medium	Primary invalidation, cross-service
Redis Pub/Sub	<1ms	Low (fire-forget)	Low	Local cache sync, supplementary
CDC	1-10s	Very High	High	Guaranteed capture, legacy systems

Hybrid Architecture

Mapping Events to Cache Keys: The Domain Challenge

The Challenge: Derived Data

Caches often store derived or aggregated data. When a Product changes, you need to invalidate:

The product's direct cache entry (product:123)
Search results that might include this product (search:query:widgets)
Category pages listing this product (category:electronics)
User wishlists containing this product (user:456:wishlist)
Price comparison aggregates (category:electronics:price-range)

Missing any of these leads to serving stale data. Invalidating too aggressively destroys your cache hit rate.

cache-key-mapping.ts
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
/**
 * Cache Key Mapping Strategies
 * 
 * Three approaches to mapping events to cache keys,
 * each with different tradeoffs.
 */
 
// ============================================
// Strategy 1: Static Mapping (Simple but Limited)
// ============================================
 
/**
 * Hard-coded rules mapping entity types to cache key patterns.
 * Simple to implement but doesn't scale with complexity.
 */
class StaticCacheKeyMapper {
    private rules: Map<string, (entityId: string) => string[]> = new Map([
        ['User', (id) => [
            `user:${id}`,
            `profile:${id}`,
            `user:${id}:permissions`,
        ]],
        ['Product', (id) => [
            `product:${id}`,
            // Note: We can't know which categories without querying!
        ]],
        ['Order', (id) => [
            `order:${id}`,
        ]],
    ]);
 
    getKeysForEvent(entityType: string, entityId: string): string[] {
        const mapper = this.rules.get(entityType);
        return mapper ? mapper(entityId) : [];
    }
}
 
// ============================================
// Strategy 2: Inverse Index (Powerful but Complex)
// ============================================
 
/**
 * Maintain an index of which cache keys contain which entities.
 * When an entity changes, look up all keys that reference it.
 */
class InverseIndexCacheKeyMapper {
    private index = new Map<string, Set<string>>(); // entityRef -> cacheKeys
 
    /**
     * When caching a value, register all entities it contains
     */
    registerCacheEntry(
        cacheKey: string,
        containedEntities: { type: string; id: string }[]
    ): void {
        for (const entity of containedEntities) {
            const entityRef = `${entity.type}:${entity.id}`;
            
            if (!this.index.has(entityRef)) {
                this.index.set(entityRef, new Set());
            }
            this.index.get(entityRef)!.add(cacheKey);
        }
    }
 
    /**
     * When uncaching or invalidating, clean up the index
     */
    unregisterCacheEntry(cacheKey: string): void {
        for (const [entityRef, keys] of this.index) {
            keys.delete(cacheKey);
            if (keys.size === 0) {
                this.index.delete(entityRef);
            }
        }
    }
 
    /**
     * Get all cache keys that reference this entity
     */
    getKeysForEntity(entityType: string, entityId: string): string[] {
        const entityRef = `${entityType}:${entityId}`;
        const keys = this.index.get(entityRef);
        return keys ? Array.from(keys) : [];
    }
}
 
// Usage of Inverse Index:
const indexMapper = new InverseIndexCacheKeyMapper();
 
// When caching a search result that includes products
const searchResults = { query: 'blue widgets', products: [
    { id: 'prod-1', name: 'Blue Widget A' },
    { id: 'prod-2', name: 'Blue Widget B' },
]};
 
indexMapper.registerCacheEntry(
    'search:blue-widgets:page-1',
    [
        { type: 'Product', id: 'prod-1' },
        { type: 'Product', id: 'prod-2' },
    ]
);
 
// When product prod-1 is updated:
const keysToInvalidate = indexMapper.getKeysForEntity('Product', 'prod-1');
// Returns: ['search:blue-widgets:page-1']
 
 
// ============================================
// Strategy 3: Tag-Based Invalidation
// ============================================
 
/**
 * Tag cache entries with metadata, then invalidate by tag.
 * Similar to inverse index but uses Redis or cache-native features.
 */
class TagBasedCacheClient {
    constructor(private redis: Redis) {}
 
    /**
     * Set with tags - allows invalidation by any tag
     */
    async setWithTags(
        key: string,
        value: unknown,
        ttlSeconds: number,
        tags: string[]
    ): Promise<void> {
        const pipeline = this.redis.pipeline();
        
        // Store the value
        pipeline.setex(key, ttlSeconds, JSON.stringify(value));
        
        // Add key to tag sets
        for (const tag of tags) {
            pipeline.sadd(`tag:${tag}`, key);
            pipeline.expire(`tag:${tag}`, ttlSeconds + 60); // Tag outlives entries slightly
        }
        
        // Store tags on the key for cleanup
        pipeline.sadd(`key:${key}:tags`, ...tags);
        pipeline.expire(`key:${key}:tags`, ttlSeconds);
        
        await pipeline.exec();
    }
 
    /**
     * Invalidate all entries with a given tag
     */
    async invalidateByTag(tag: string): Promise<number> {
        const tagKey = `tag:${tag}`;
        const keysToDelete = await this.redis.smembers(tagKey);
        
        if (keysToDelete.length === 0) {
            return 0;
        }
        
        const pipeline = this.redis.pipeline();
        
        // Delete all tagged entries
        for (const key of keysToDelete) {
            pipeline.del(key);
            pipeline.del(`key:${key}:tags`);
        }
        
        // Delete the tag set itself
        pipeline.del(tagKey);
        
        await pipeline.exec();
        return keysToDelete.length;
    }
 
    async get<T>(key: string): Promise<T | null> {
        const value = await this.redis.get(key);
        return value ? JSON.parse(value) : null;
    }
}
 
// Usage of Tag-Based Invalidation:
const tagCache = new TagBasedCacheClient(redis);
 
// Cache product with category tags
await tagCache.setWithTags(
    'product:123',
    { id: '123', name: 'Widget', price: 9.99 },
    3600,
    ['product:123', 'category:electronics', 'featured']
);
 
// Cache category listing with product tags
await tagCache.setWithTags(
    'category:electronics:page-1',
    [{ id: '123' }, { id: '456' }],
    3600,
    ['category:electronics', 'product:123', 'product:456']
);
 
// When product 123 changes, invalidate by tag:
const invalidatedCount = await tagCache.invalidateByTag('product:123');
// Invalidates both product:123 AND category:electronics:page-1

Cache Key Mapping Strategy Comparison
Strategy	Accuracy	Memory Overhead	Complexity	Query Cost
Static Mapping	Low (misses derived)	None	Low	None
Inverse Index	High	Medium-High	Medium	Per-cache write
Tag-Based	High	Medium	Medium	Per-cache write

The Cascade Problem

Failure Handling: When Invalidation Fails

In distributed systems, failures are inevitable. Event-based invalidation introduces several failure modes that must be explicitly handled.

Failure Modes:

Event loss: Message queue loses events (extremely rare with proper config)
Consumer failure: Invalidator crashes before processing event
Cache unreachable: Cache cluster unavailable when invalidation tried
Partial invalidation: Some keys invalidated, others fail
Event duplication: Same event processed multiple times

Each requires different handling strategies.

failure-handling.ts
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
/**
 * Robust Event-Based Invalidation with Failure Handling
 */
 
interface InvalidationResult {
    success: boolean;
    keysInvalidated: string[];
    keysFailed: string[];
    error?: Error;
}
 
class ResilientCacheInvalidator {
    private deadLetterQueue: DeadLetterQueue;
    private processedEvents: Set<string> = new Set(); // Dedup window
    
    constructor(
        private cache: CacheClient,
        private config: InvalidatorConfig,
    ) {
        this.deadLetterQueue = new DeadLetterQueue(config.dlqTopic);
    }
 
    /**
     * Main entry point - processes invalidation event with full error handling
     */
    async processEvent(event: InvalidationEvent): Promise<InvalidationResult> {
        const eventId = event.correlationId;
        
        // 1. Idempotency check - handle duplicate events
        if (this.processedEvents.has(eventId)) {
            console.log(`Duplicate event ${eventId}, skipping`);
            return { success: true, keysInvalidated: [], keysFailed: [] };
        }
        
        const keysToInvalidate = this.deriveKeysFromEvent(event);
        const keysInvalidated: string[] = [];
        const keysFailed: string[] = [];
        
        // 2. Attempt invalidation with retries
        for (const key of keysToInvalidate) {
            const success = await this.invalidateWithRetry(key);
            if (success) {
                keysInvalidated.push(key);
            } else {
                keysFailed.push(key);
            }
        }
        
        // 3. Handle partial failures
        if (keysFailed.length > 0) {
            if (keysFailed.length === keysToInvalidate.length) {
                // Complete failure - send to DLQ for later retry
                await this.deadLetterQueue.send(event, 'complete_failure');
                return {
                    success: false,
                    keysInvalidated,
                    keysFailed,
                    error: new Error('All invalidations failed'),
                };
            } else {
                // Partial failure - log and potentially alert
                console.warn(`Partial invalidation: ${keysFailed.length} failed`, keysFailed);
                
                // Send failed keys as separate event
                await this.deadLetterQueue.send({
                    ...event,
                    metadata: { ...event.metadata, failedKeys: keysFailed },
                }, 'partial_failure');
            }
        }
        
        // 4. Mark event as processed (with expiration)
        this.markAsProcessed(eventId);
        
        return { success: keysFailed.length === 0, keysInvalidated, keysFailed };
    }
 
    /**
     * Invalidate with exponential backoff retry
     */
    private async invalidateWithRetry(key: string): Promise<boolean> {
        const maxRetries = this.config.maxRetries;
        const baseDelay = this.config.retryDelayMs;
        
        for (let attempt = 0; attempt <= maxRetries; attempt++) {
            try {
                await this.cache.delete(key);
                return true;
            } catch (error) {
                if (attempt === maxRetries) {
                    console.error(`Failed to invalidate ${key} after ${maxRetries} retries`);
                    return false;
                }
                
                // Exponential backoff with jitter
                const delay = baseDelay * Math.pow(2, attempt) * (0.5 + Math.random());
                await this.sleep(delay);
            }
        }
        
        return false;
    }
 
    /**
     * Track processed events to handle duplicates
     * Uses sliding window to bound memory
     */
    private markAsProcessed(eventId: string): void {
        this.processedEvents.add(eventId);
        
        // Clean up old entries (simple sliding window)
        if (this.processedEvents.size > this.config.dedupWindowSize) {
            const iterator = this.processedEvents.values();
            // Remove oldest entries
            for (let i = 0; i < this.config.dedupWindowSize / 2; i++) {
                const oldest = iterator.next().value;
                if (oldest) this.processedEvents.delete(oldest);
            }
        }
    }
 
    private deriveKeysFromEvent(event: InvalidationEvent): string[] {
        // Implementation depends on your key mapping strategy
        return [`${event.entityType.toLowerCase()}:${event.entityId}`];
    }
 
    private sleep(ms: number): Promise<void> {
        return new Promise(resolve => setTimeout(resolve, ms));
    }
}
 
/**
 * Dead Letter Queue handler for failed invalidations
 */
class DeadLetterQueue {
    constructor(private topic: string) {}
 
    async send(event: InvalidationEvent, reason: string): Promise<void> {
        const dlqEvent = {
            originalEvent: event,
            failureReason: reason,
            failedAt: Date.now(),
            retryCount: (event.metadata?.retryCount as number || 0) + 1,
        };
 
        // Publish to DLQ topic for manual or automated retry
        await publishToKafka(this.topic, dlqEvent);
        
        // Alert if retry count is high
        if (dlqEvent.retryCount > 3) {
            await sendAlert(`Invalidation failing repeatedly for ${event.entityType}:${event.entityId}`);
        }
    }
}
 
/**
 * Fallback: TTL as Safety Net
 * 
 * Even with event-based invalidation, always set a TTL.
 * If all invalidation mechanisms fail, TTL ensures eventual consistency.
 */
class SafetyNetCache {
    private readonly SAFETY_TTL_SECONDS = 3600; // 1 hour fallback
 
    constructor(private cache: CacheClient) {}
 
    async set<T>(key: string, value: T, ttlSeconds?: number): Promise<void> {
        // Always apply safety TTL, even if caller doesn't specify
        const effectiveTtl = ttlSeconds ?? this.SAFETY_TTL_SECONDS;
        const maxTtl = Math.min(effectiveTtl, this.SAFETY_TTL_SECONDS);
        
        await this.cache.setex(key, maxTtl, JSON.stringify(value));
    }
}
 
interface InvalidationEvent {
    type: string;
    entityType: string;
    entityId: string;
    timestamp: number;
    correlationId: string;
    metadata?: Record<string, unknown>;
}
 
interface InvalidatorConfig {
    maxRetries: number;
    retryDelayMs: number;
    dedupWindowSize: number;
    dlqTopic: string;
}

The Belt-and-Suspenders Approach

When to Use Event-Based Invalidation

Use Event-Based When

•Low latency consistency required — Users expect to see their changes immediately (e.g., profile updates, cart modifications)
•Irregular update patterns — Data changes unpredictably, making TTL-based invalidation wasteful or stale
•High write-to-read ratio — Writes are frequent enough that stale reads significantly impact UX
•Multi-system caches — Multiple services cache the same data and need coordinated invalidation
•Regulatory requirements — Compliance mandates that stale data exposure is minimized
•Already have event infrastructure — You're already using Kafka/RabbitMQ for other purposes

Stick with TTL When

•Static or slowly-changing data — Configuration, reference data that changes weekly/monthly
•High read-to-write ratio — Reads vastly outnumber writes (1000:1+), so staleness is rare
•Eventual consistency is acceptable — Analytics, recommendations, non-critical aggregates
•Simple architecture — Single service, single cache, no distributed coordination needed
•Limited infrastructure — Adding message queues would significantly increase operational burden
•Predictable update patterns — Data refreshes on a schedule (hourly reports, daily batches)

Decision Matrix: Event-Based vs TTL-Only Invalidation
Data Type	Update Pattern	Consistency Need	Recommended Strategy
User profile	On user action	Immediate	Event-based + short TTL
Product catalog	Hourly import	Minutes acceptable	TTL (sync with import)
Session data	Continuous	Immediate	Event-based (logout) + sliding TTL
Search results	Continuous	Minutes acceptable	Long TTL + event on major changes
Feature flags	Rare manual updates	Minutes acceptable	TTL + manual invalidation API
Analytics	Batch computed	Hours acceptable	Long TTL only

Progressive Adoption

Summary: Event-Based Invalidation

Key Takeaways

•Event-based invalidation is proactive — Changes trigger immediate cache updates, minimizing staleness windows.
•Multiple architectures exist — Message queues for reliability, Redis Pub/Sub for speed, CDC for guaranteed capture. Choose based on your consistency and latency requirements.
•Key mapping is the hard part — Knowing which cache keys to invalidate when an entity changes requires careful design of inverse indexes or tag-based systems.
•Failures must be handled explicitly — Retries, dead letter queues, and idempotency checks are essential for production systems.
•Always keep TTL as a safety net — Even with perfect event delivery, TTL ensures eventual consistency if events fail.
•Complexity must be justified — Event-based invalidation isn't always necessary. Use for high-impact, consistency-critical caches; stick with TTL for everything else.

What's next:

Page Complete

2 / 4