Cache Fallbacks — Fallback Patterns | OneNoughtOne

0/107

System Design (HLD)Fallback Patterns

Fallback Patterns

LevelAdvanced

Duration90 mins

TopicFallback Patterns

3 / 5

Cache Fallbacks

When Yesterday's Data Beats No Data

In 2017, Amazon S3 experienced a significant outage in the US-East-1 region that cascaded across the internet. Thousands of websites and services went down—but some remained functional despite their complete reliance on S3. The difference? Those services had implemented cache fallbacks, serving cached content when S3 became unreachable. Cache fallbacks represent a sophisticated evolution beyond static default values. Rather than returning predetermined generic data, cache fallbacks serve actual user data from a previous successful fetch. The data may be stale, but it's real—and often, slightly stale real data is vastly preferable to no data at all or generic placeholders.

What You Will Learn

This page provides comprehensive coverage of cache fallback strategies—from the fundamental patterns to sophisticated multi-tier caching architectures designed for resilience. You'll learn how to architect caches specifically for fallback purposes, manage the freshness-availability trade-off, implement stale-while-revalidate patterns, and navigate the complex decisions around when cached data is acceptable versus when it's dangerous.

Understanding Cache Fallbacks

A cache fallback uses previously cached data when the primary data source is unavailable. Unlike static defaults that provide generic placeholder values, cache fallbacks serve actual data that was valid at some point in the past. The fundamental trade-off: Cache fallbacks trade data freshness for system availability . You're accepting that users might see slightly outdated information in exchange for the ability to continue serving requests when upstream services fail. When this trade-off makes sense: The data changes infrequently relative to typical outage duration Stale data is better than no data for user experience Users can understand and work around data staleness The cache contains meaningful data (not empty from a cold start) When this trade-off is unacceptable: Data accuracy is critical for safety, financial, or compliance reasons Stale data could lead to irreversible user actions The data changes rapidly relative to cache TTL Cache consistency is critical for system correctness

Static Defaults vs. Cache Fallbacks

Static Defaults	Cache Fallbacks
Fixed, predetermined values	Previously real data
Always available (no cold start)	Requires cache to be populated
Generic, not user-specific	User-specific when cached that way
Never stale (never was fresh)	May be stale (was fresh once)
Simple implementation	Complex cache management
No user context required	Maintains user context during failures

The cache fallback mindset: Traditionally, caches are designed for performance—reducing latency and load on primary data sources. Cache fallbacks reframe caching as a resilience mechanism . The cache becomes a buffer against upstream failures, not just a performance optimization. This mindset shift has architectural implications: Cache TTLs may be set longer than needed for pure performance Cache invalidation becomes more conservative Cache infrastructure requires higher availability guarantees Cache population strategies prioritize coverage over efficiency

Cache Architecture for Resilience

Designing caches for fallback purposes requires different architectural considerations than designing purely for performance. A resilience-focused cache must be: Highly available — The cache itself cannot be a single point of failure Broadly populated — Cold caches provide no fallback value Appropriately retained — Data must survive long enough to cover outages Observable — You must know cache coverage and freshness

Resilience Cache Architecture Principles

•Multi-tier caching — Implement multiple cache layers (local/L1, distributed/L2, persistent/L3) with progressively longer TTLs. Local cache handles most requests; distributed cache survives instance restarts; persistent cache survives cache cluster failures.
•Cache replication — Replicate cache across regions and availability zones. If your cache is in a single AZ, an AZ outage eliminates both your primary data sources AND your fallback.
•Eager cache population — Don't wait for user requests to populate cache. Background jobs should pre-warm cache for popular and critical data paths.
•Extended TTL for fallback — Maintain separate 'fallback TTL' beyond normal expiration. Data expired for normal serving can still be used during outages.
•Cache persistence — For critical fallback data, persist cache to durable storage. In-memory caches can be reconstructed after restarts without hitting primary sources.
•Degraded source awareness — Cache should know when upstream is unhealthy. Serve stale data during outages, refresh aggressively when upstream recovers.

Multi-Tier Resilient Cache Configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
// Multi-tier cache configuration optimized for resilience
interface ResilientCacheConfig {
  // L1: Local in-process cache (fastest, smallest)
  local: {
    maxSize: number;          // e.g., 1000 items
    ttlSeconds: number;       // e.g., 60 seconds
  };
  
  // L2: Distributed cache (Redis/Memcached)
  distributed: {
    ttlSeconds: number;       // e.g., 300 seconds (5 minutes)
    fallbackTtlSeconds: number; // e.g., 3600 seconds (1 hour) - stale but usable
  };
  
  // L3: Persistent fallback cache (database or object storage)
  persistent: {
    ttlSeconds: number;       // e.g., 86400 seconds (24 hours)
    enabled: boolean;         // Enable only for critical paths
  };
}
 
// Example configuration for a product catalog
const productCatalogCache: ResilientCacheConfig = {
  local: {
    maxSize: 10000,     // Hot products cached locally
    ttlSeconds: 30,     // Quick refresh for price changes
  },
  distributed: {
    ttlSeconds: 300,          // 5 minute freshness target
    fallbackTtlSeconds: 7200, // Serve up to 2 hour stale during outage
  },
  persistent: {
    ttlSeconds: 86400,  // 24 hour backup in S3/database
    enabled: true,      // Product data is critical
  }
};

The Netflix Approach

Netflix maintains multiple cache tiers specifically for resilience. Their EVCache layer handles normal operation, but they also maintain a 'stale cache' that retains data beyond normal TTL specifically for use during upstream outages. This stale cache is never used under normal operation—it exists purely for resilience.

Stale-While-Revalidate Pattern

The Stale-While-Revalidate (SWR) pattern is a sophisticated caching strategy that serves cached data immediately while asynchronously refreshing in the background. This pattern is exceptionally powerful for fallback scenarios because it prioritizes availability while still maintaining eventual freshness. How SWR works: Request arrives, cache hit found If cache entry is within 'fresh' window: serve immediately, no refresh If cache entry is within 'stale-but-usable' window: serve immediately, trigger background refresh If cache entry is beyond 'stale-but-usable' window: treat as cache miss During upstream outages, the background refresh fails, but the stale data is still served, providing resilience.

Stale-While-Revalidate Implementation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
interface SWRCacheEntry<T> {
  data: T;
  cachedAt: number;      // Timestamp when cached
  refreshedAt: number;   // Timestamp of last successful refresh
}
 
interface SWRConfig {
  freshThresholdMs: number;    // Serve without refresh
  staleThresholdMs: number;    // Serve but trigger refresh
  maxStaleMs: number;          // Beyond this, treat as miss
}
 
class SWRCache<T> {
  constructor(
    private cache: Cache,
    private fetcher: () => Promise<T>,
    private config: SWRConfig
  ) {}
 
  async get(key: string): Promise<{ data: T; status: 'fresh' | 'stale' | 'miss' }> {
    const entry = await this.cache.get<SWRCacheEntry<T>>(key);
    const now = Date.now();
 
    // Case 1: Cache miss - fetch synchronously
    if (!entry) {
      const data = await this.fetchAndCache(key);
      return { data, status: 'miss' };
    }
 
    const age = now - entry.refreshedAt;
 
    // Case 2: Fresh - serve immediately
    if (age < this.config.freshThresholdMs) {
      return { data: entry.data, status: 'fresh' };
    }
 
    // Case 3: Stale but usable - serve and refresh
    if (age < this.config.maxStaleMs) {
      // Trigger background refresh (don't await)
      this.backgroundRefresh(key);
      return { data: entry.data, status: 'stale' };
    }
 
    // Case 4: Too stale - treat as miss
    try {
      const data = await this.fetchAndCache(key);
      return { data, status: 'miss' };
    } catch (error) {
      // Even though too stale, if fetch fails, still serve stale
      // This is the fallback power of SWR
      if (entry) {
        logger.warn('Serving very stale cache due to fetch failure', { key, age });
        metrics.increment('cache.very_stale_fallback');
        return { data: entry.data, status: 'stale' };
      }
      throw error;
    }
  }
 
  private async fetchAndCache(key: string): Promise<T> {
    const data = await this.fetcher();
    const now = Date.now();
    await this.cache.set(key, {
      data,
      cachedAt: now,
      refreshedAt: now
    });
    return data;
  }
 
  private backgroundRefresh(key: string): void {
    // Fire and forget - errors are logged but don't affect response
    this.fetchAndCache(key).catch(error => {
      logger.warn('Background refresh failed', { key, error: error.message });
      metrics.increment('cache.background_refresh_failed');
    });
  }
}

HTTP Cache-Control: stale-while-revalidate The HTTP specification includes native support for SWR via the Cache-Control header: Cache-Control: max-age=60, stale-while-revalidate=3600 This header tells caches: Serve the response fresh for 60 seconds After 60 seconds, serve stale while revalidating in background for up to 1 hour After 1 hour of staleness, treat as miss Browsers, CDNs, and proxy caches that support this directive automatically implement the SWR pattern, providing resilience at the edge.

SWR Libraries

Many modern frontend frameworks include SWR libraries (React's SWR, TanStack Query, Apollo Client) that implement this pattern client-side. These libraries serve cached data immediately and refresh in the background, providing excellent perceived performance and inherent resilience to API failures.

Managing Data Staleness

The central challenge in cache fallbacks is staleness management. Data that was accurate two hours ago may be dangerously outdated—or it may be perfectly fine. Understanding and managing staleness is essential for effective cache fallbacks. Staleness dimensions:

Staleness Analysis Framework
Data Type	Acceptable Staleness	Staleness Risk	Strategy
User profile info	Hours to days	Low - rarely changes	Aggressive caching with long fallback TTL
Product catalog	Minutes to hours	Medium - prices/availability change	Moderate caching with staleness indicators
Inventory levels	Seconds to minutes	High - affects purchase decisions	Short cache, conservative fallback messaging
Stock prices	Seconds	Critical - financial impact	Minimal caching, clear stale indicators, may refuse to serve
Account balance	Not acceptable	Critical - financial decisions	No cache fallback - show error instead

Staleness indicators to users: When serving stale data, transparency with users is critical. Common patterns include: Timestamp display : 'Last updated 5 minutes ago' Visual indicators : Grayed-out or faded styling for stale content Explicit messaging : 'Showing cached data. Information may be outdated.' Auto-refresh hints : 'Refresh to see latest data' Staleness badges : Small icons indicating data freshness Programmatic staleness handling: Downstream systems need to know data freshness for their own decision-making. Include staleness metadata in responses:

Staleness Metadata in Responses
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
interface CachedResponse<T> {
  data: T;
  metadata: {
    source: 'fresh' | 'cache' | 'stale_cache' | 'fallback';
    cachedAt?: Date;
    maxAge?: number;           // Original TTL
    age?: number;              // How old the data is
    staleReason?: string;      // Why we're serving stale
    refreshAttempted?: Date;   // Last refresh attempt time
    nextRefresh?: Date;        // When refresh will be tried
  };
}
 
// Controller returns this structure
async function getProduct(productId: string): Promise<CachedResponse<Product>> {
  const cacheKey = `product:${productId}`;
  const cached = await cache.getWithMetadata<Product>(cacheKey);
 
  // Fresh cache hit
  if (cached && cached.age < cached.maxAge) {
    return {
      data: cached.value,
      metadata: {
        source: 'cache',
        cachedAt: cached.timestamp,
        maxAge: cached.maxAge,
        age: cached.age
      }
    };
  }
 
  // Try to fetch fresh
  try {
    const product = await productService.fetch(productId);
    await cache.set(cacheKey, product, { maxAge: 300 });
    return {
      data: product,
      metadata: { source: 'fresh' }
    };
  } catch (error) {
    // Fetch failed - use stale cache if available
    if (cached) {
      return {
        data: cached.value,
        metadata: {
          source: 'stale_cache',
          cachedAt: cached.timestamp,
          age: cached.age,
          staleReason: 'upstream_unavailable',
          refreshAttempted: new Date()
        }
      };
    }
    throw error; // No cache, no fresh - must fail
  }
}

Cache Population Strategies

A cache provides no fallback value if it's empty. For cache fallbacks to work, the cache must be populated before failures occur. This requires deliberate cache population strategies. Reactive vs. Proactive Population:

Cache Population Approaches

Reactive (On-Demand)	Proactive (Pre-Population)
Cache populated when first requested	Cache pre-populated before requests
Cold cache on startup	Warm cache on startup
Gaps in fallback coverage	Comprehensive fallback coverage
Simple implementation	Complex orchestration needed
No wasted cache space	May cache unused data
First user sees latency/errors	All users get fallback protection

Proactive population strategies:

Cache Pre-Population Patterns

•Start-up cache warming — On service startup, populate cache with critical data before accepting traffic. This ensures no cold-cache fallback gaps for the most important data paths.
•Background refresh jobs — Scheduled jobs continuously refresh cache entries before they expire. This maintains cache freshness without relying on user requests.
•Event-driven population — Subscribe to data change events and update cache proactively. When a product is updated, immediately refresh the product cache entry.
•Traffic-pattern-based warming — Analyze traffic patterns and pre-populate cache for items likely to be requested soon. If users typically browse after login, warm user-specific caches during authentication.
•Geographical pre-replication — Populate caches in regions before traffic shifts there. If you're launching in a new region, warm caches before directing traffic.
•Tiered population priority — Prioritize caching for Tier 1 (critical) data over Tier 3 (nice-to-have) data. Critical paths should always have fallback available.

Background Cache Warmer
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
class CacheWarmer {
  private warmingQueue: Queue;
  
  constructor(
    private cache: Cache,
    private dataSource: DataSource,
    private config: CacheWarmerConfig
  ) {
    this.warmingQueue = new Queue('cache-warming');
    this.schedulePeriodicWarming();
  }
 
  // Warm cache for a specific key
  async warmKey(key: string, fetcher: () => Promise<any>): Promise<void> {
    const existing = await this.cache.get(key);
    const age = existing ? Date.now() - existing.timestamp : Infinity;
 
    // Only warm if approaching staleness threshold
    if (age > this.config.warmingThresholdMs) {
      try {
        const data = await fetcher();
        await this.cache.set(key, data, { maxAge: this.config.ttlSeconds });
        metrics.increment('cache.warmed', { key: this.keyPattern(key) });
      } catch (error) {
        logger.warn('Cache warming failed', { key, error: error.message });
        metrics.increment('cache.warming_failed', { key: this.keyPattern(key) });
      }
    }
  }
 
  // Scheduled warming of critical data
  private schedulePeriodicWarming(): void {
    // Run every minute
    setInterval(async () => {
      // Warm product catalog top 1000 products
      const topProducts = await this.dataSource.getTopProducts(1000);
      for (const product of topProducts) {
        await this.warmingQueue.add('warm-product', { productId: product.id });
      }
 
      // Warm user preferences for recently active users
      const recentUsers = await this.dataSource.getRecentlyActiveUsers(10000);
      for (const user of recentUsers) {
        await this.warmingQueue.add('warm-user-prefs', { userId: user.id });
      }
 
      metrics.gauge('cache.warming_queue_size', this.warmingQueue.length);
    }, 60000);
  }
 
  // Process warming jobs
  async processWarmingJob(job: { type: string; data: any }): Promise<void> {
    switch (job.type) {
      case 'warm-product':
        await this.warmKey(
          `product:${job.data.productId}`,
          () => this.dataSource.getProduct(job.data.productId)
        );
        break;
      case 'warm-user-prefs':
        await this.warmKey(
          `user:prefs:${job.data.userId}`,
          () => this.dataSource.getUserPreferences(job.data.userId)
        );
        break;
    }
  }
 
  private keyPattern(key: string): string {
    return key.replace(/:[a-f0-9-]+/g, ':*');
  }
}

Cache Fallback During Outages

When an outage occurs, the cache fallback system must seamlessly transition from normal operation to fallback mode. This transition involves different behaviors for reads, writes, and cache management. Read path during outages: During upstream outages, read path behavior changes: Detection : Recognize that upstream is unhealthy (timeouts, errors, circuit breaker open) Fallback activation : Switch to serving from cache without attempting upstream Extended TTL : Allow serving data beyond normal staleness thresholds Background retry : Periodically probe upstream for recovery Recovery : Resume normal operation when upstream is healthy

Outage-Aware Cache Read Pattern
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
class OutageAwareCache<T> {
  private upstreamHealthy = true;
  private lastHealthCheck = 0;
  private healthCheckIntervalMs = 5000;
 
  constructor(
    private cache: Cache,
    private fetcher: (key: string) => Promise<T>,
    private config: OutageCacheConfig
  ) {}
 
  async get(key: string): Promise<T | null> {
    // Check if we should probe upstream health
    if (!this.upstreamHealthy && this.shouldProbeHealth()) {
      this.probeUpstreamHealth();
    }
 
    // If upstream is healthy, try normal fetch
    if (this.upstreamHealthy) {
      return this.normalFetch(key);
    }
 
    // Upstream unhealthy - fallback mode
    return this.fallbackFetch(key);
  }
 
  private async normalFetch(key: string): Promise<T | null> {
    // First check cache
    const cached = await this.cache.get<CacheEntry<T>>(key);
    if (cached && !this.isStale(cached)) {
      return cached.data;
    }
 
    // Cache miss or stale - fetch from upstream
    try {
      const data = await this.fetcher(key);
      await this.cache.set(key, { data, timestamp: Date.now() });
      return data;
    } catch (error) {
      // Upstream failed - might be start of outage
      this.handleUpstreamFailure(error);
      
      // Return stale cache if available
      if (cached) {
        metrics.increment('cache.stale_fallback');
        return cached.data;
      }
      
      throw error;
    }
  }
 
  private async fallbackFetch(key: string): Promise<T | null> {
    // In fallback mode, only use cache - don't hit upstream
    const cached = await this.cache.get<CacheEntry<T>>(key);
    
    if (cached) {
      const staleness = Date.now() - cached.timestamp;
      
      // Check if within extended fallback threshold
      if (staleness < this.config.fallbackMaxStaleMs) {
        metrics.increment('cache.fallback_hit', {
          staleness_bucket: this.stalenessBucket(staleness)
        });
        return cached.data;
      }
      
      // Beyond fallback threshold - data too old
      metrics.increment('cache.fallback_too_stale');
      // Depending on policy, either return anyway with warning or return null
      if (this.config.serveVeryStale) {
        logger.warn('Serving very stale data', { key, staleness });
        return cached.data;
      }
    }
 
    return null;
  }
 
  private handleUpstreamFailure(error: Error): void {
    this.upstreamHealthy = false;
    this.lastHealthCheck = Date.now();
    logger.warn('Upstream failure detected, entering fallback mode');
    metrics.increment('cache.fallback_mode_entered');
  }
 
  private shouldProbeHealth(): boolean {
    return Date.now() - this.lastHealthCheck > this.healthCheckIntervalMs;
  }
 
  private async probeUpstreamHealth(): Promise<void> {
    this.lastHealthCheck = Date.now();
    try {
      await this.fetcher('__health_probe__');
      this.upstreamHealthy = true;
      logger.info('Upstream recovered, exiting fallback mode');
      metrics.increment('cache.fallback_mode_exited');
    } catch {
      // Still unhealthy
    }
  }
 
  private isStale(entry: CacheEntry<T>): boolean {
    return Date.now() - entry.timestamp > this.config.freshTtlMs;
  }
 
  private stalenessBucket(staleness: number): string {
    if (staleness < 60000) return '<1min';
    if (staleness < 300000) return '1-5min';
    if (staleness < 900000) return '5-15min';
    if (staleness < 3600000) return '15-60min';
    return '>1hour';
  }
}

Write Path During Outages

Cache fallbacks primarily address read operations. Write operations during outages require different patterns: queuing for later processing, optimistic writes with reconciliation, or explicit failure with retry guidance. Don't let cache fallbacks mask write failures.

Distributed Cache Considerations

In distributed systems, cache fallbacks introduce additional complexity around consistency, replication, and failover. A distributed cache optimized for fallback scenarios must handle these challenges. Consistency challenges:

Distributed Cache Fallback Challenges

•Stale replica reads — Different cache nodes may have data of different ages. During fallback, you might serve older data from one request and newer data from the next.
•Network partitions — If the cache cluster is partitioned, some nodes may fall back while others don't. Users hitting different nodes get different experiences.
•Cache invalidation during outage — If data changes in the (unavailable) source during an outage, the cache becomes incorrect. When the source recovers, the cache holds stale data.
•Thundering herd on recovery — When upstream recovers, all nodes simultaneously try to refresh cache, potentially overwhelming the just-recovered upstream.
•Clock skew — Distributed systems have clock differences. Staleness calculations based on timestamps may be inconsistent across nodes.

Mitigation strategies: Read-repair during fallback : When serving stale data, also trigger an async check against other cache replicas for fresher data. Versioned cache entries : Include version numbers in cache entries. During fallback, prefer higher versions even if slightly older by timestamp. Coordinated invalidation : Use a distributed coordination service for invalidations. If the source updates, invalidation can still propagate via the coordinator. Jittered refresh on recovery : Add random delays to recovery refreshes to avoid thundering herd. Logical timestamps : Use logical clocks (Lamport timestamps, vector clocks) rather than wall-clock time for staleness tracking.

Redis as Fallback Cache

Redis Cluster with read replicas provides a good foundation for cache fallbacks. Use READONLY commands against replicas for fallback reads, reducing load on primaries. Configure replicas with higher persistence than primaries to maintain fallback data during primary failures. Sentinel or Cluster mode handles automatic failover.

System Design (HLD)Fallback Patterns

Fallback Patterns

LevelAdvanced

Duration90 mins

TopicFallback Patterns

3 / 5

Cache Fallbacks

When Yesterday's Data Beats No Data

In 2017, Amazon S3 experienced a significant outage in the US-East-1 region that cascaded across the internet. Thousands of websites and services went down—but some remained functional despite their complete reliance on S3. The difference? Those services had implemented cache fallbacks, serving cached content when S3 became unreachable. Cache fallbacks represent a sophisticated evolution beyond static default values. Rather than returning predetermined generic data, cache fallbacks serve actual user data from a previous successful fetch. The data may be stale, but it's real—and often, slightly stale real data is vastly preferable to no data at all or generic placeholders.

What You Will Learn

This page provides comprehensive coverage of cache fallback strategies—from the fundamental patterns to sophisticated multi-tier caching architectures designed for resilience. You'll learn how to architect caches specifically for fallback purposes, manage the freshness-availability trade-off, implement stale-while-revalidate patterns, and navigate the complex decisions around when cached data is acceptable versus when it's dangerous.

Understanding Cache Fallbacks

A cache fallback uses previously cached data when the primary data source is unavailable. Unlike static defaults that provide generic placeholder values, cache fallbacks serve actual data that was valid at some point in the past. The fundamental trade-off: Cache fallbacks trade data freshness for system availability . You're accepting that users might see slightly outdated information in exchange for the ability to continue serving requests when upstream services fail. When this trade-off makes sense: The data changes infrequently relative to typical outage duration Stale data is better than no data for user experience Users can understand and work around data staleness The cache contains meaningful data (not empty from a cold start) When this trade-off is unacceptable: Data accuracy is critical for safety, financial, or compliance reasons Stale data could lead to irreversible user actions The data changes rapidly relative to cache TTL Cache consistency is critical for system correctness

Static Defaults vs. Cache Fallbacks

Static Defaults	Cache Fallbacks
Fixed, predetermined values	Previously real data
Always available (no cold start)	Requires cache to be populated
Generic, not user-specific	User-specific when cached that way
Never stale (never was fresh)	May be stale (was fresh once)
Simple implementation	Complex cache management
No user context required	Maintains user context during failures

The cache fallback mindset: Traditionally, caches are designed for performance—reducing latency and load on primary data sources. Cache fallbacks reframe caching as a resilience mechanism . The cache becomes a buffer against upstream failures, not just a performance optimization. This mindset shift has architectural implications: Cache TTLs may be set longer than needed for pure performance Cache invalidation becomes more conservative Cache infrastructure requires higher availability guarantees Cache population strategies prioritize coverage over efficiency

Cache Architecture for Resilience

Designing caches for fallback purposes requires different architectural considerations than designing purely for performance. A resilience-focused cache must be: Highly available — The cache itself cannot be a single point of failure Broadly populated — Cold caches provide no fallback value Appropriately retained — Data must survive long enough to cover outages Observable — You must know cache coverage and freshness

Resilience Cache Architecture Principles

•Multi-tier caching — Implement multiple cache layers (local/L1, distributed/L2, persistent/L3) with progressively longer TTLs. Local cache handles most requests; distributed cache survives instance restarts; persistent cache survives cache cluster failures.
•Cache replication — Replicate cache across regions and availability zones. If your cache is in a single AZ, an AZ outage eliminates both your primary data sources AND your fallback.
•Eager cache population — Don't wait for user requests to populate cache. Background jobs should pre-warm cache for popular and critical data paths.
•Extended TTL for fallback — Maintain separate 'fallback TTL' beyond normal expiration. Data expired for normal serving can still be used during outages.
•Cache persistence — For critical fallback data, persist cache to durable storage. In-memory caches can be reconstructed after restarts without hitting primary sources.
•Degraded source awareness — Cache should know when upstream is unhealthy. Serve stale data during outages, refresh aggressively when upstream recovers.

Multi-Tier Resilient Cache Configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
// Multi-tier cache configuration optimized for resilience
interface ResilientCacheConfig {
  // L1: Local in-process cache (fastest, smallest)
  local: {
    maxSize: number;          // e.g., 1000 items
    ttlSeconds: number;       // e.g., 60 seconds
  };
  
  // L2: Distributed cache (Redis/Memcached)
  distributed: {
    ttlSeconds: number;       // e.g., 300 seconds (5 minutes)
    fallbackTtlSeconds: number; // e.g., 3600 seconds (1 hour) - stale but usable
  };
  
  // L3: Persistent fallback cache (database or object storage)
  persistent: {
    ttlSeconds: number;       // e.g., 86400 seconds (24 hours)
    enabled: boolean;         // Enable only for critical paths
  };
}
 
// Example configuration for a product catalog
const productCatalogCache: ResilientCacheConfig = {
  local: {
    maxSize: 10000,     // Hot products cached locally
    ttlSeconds: 30,     // Quick refresh for price changes
  },
  distributed: {
    ttlSeconds: 300,          // 5 minute freshness target
    fallbackTtlSeconds: 7200, // Serve up to 2 hour stale during outage
  },
  persistent: {
    ttlSeconds: 86400,  // 24 hour backup in S3/database
    enabled: true,      // Product data is critical
  }
};

The Netflix Approach

Netflix maintains multiple cache tiers specifically for resilience. Their EVCache layer handles normal operation, but they also maintain a 'stale cache' that retains data beyond normal TTL specifically for use during upstream outages. This stale cache is never used under normal operation—it exists purely for resilience.

Stale-While-Revalidate Pattern

The Stale-While-Revalidate (SWR) pattern is a sophisticated caching strategy that serves cached data immediately while asynchronously refreshing in the background. This pattern is exceptionally powerful for fallback scenarios because it prioritizes availability while still maintaining eventual freshness. How SWR works: Request arrives, cache hit found If cache entry is within 'fresh' window: serve immediately, no refresh If cache entry is within 'stale-but-usable' window: serve immediately, trigger background refresh If cache entry is beyond 'stale-but-usable' window: treat as cache miss During upstream outages, the background refresh fails, but the stale data is still served, providing resilience.

Stale-While-Revalidate Implementation
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
interface SWRCacheEntry<T> {
  data: T;
  cachedAt: number;      // Timestamp when cached
  refreshedAt: number;   // Timestamp of last successful refresh
}
 
interface SWRConfig {
  freshThresholdMs: number;    // Serve without refresh
  staleThresholdMs: number;    // Serve but trigger refresh
  maxStaleMs: number;          // Beyond this, treat as miss
}
 
class SWRCache<T> {
  constructor(
    private cache: Cache,
    private fetcher: () => Promise<T>,
    private config: SWRConfig
  ) {}
 
  async get(key: string): Promise<{ data: T; status: 'fresh' | 'stale' | 'miss' }> {
    const entry = await this.cache.get<SWRCacheEntry<T>>(key);
    const now = Date.now();
 
    // Case 1: Cache miss - fetch synchronously
    if (!entry) {
      const data = await this.fetchAndCache(key);
      return { data, status: 'miss' };
    }
 
    const age = now - entry.refreshedAt;
 
    // Case 2: Fresh - serve immediately
    if (age < this.config.freshThresholdMs) {
      return { data: entry.data, status: 'fresh' };
    }
 
    // Case 3: Stale but usable - serve and refresh
    if (age < this.config.maxStaleMs) {
      // Trigger background refresh (don't await)
      this.backgroundRefresh(key);
      return { data: entry.data, status: 'stale' };
    }
 
    // Case 4: Too stale - treat as miss
    try {
      const data = await this.fetchAndCache(key);
      return { data, status: 'miss' };
    } catch (error) {
      // Even though too stale, if fetch fails, still serve stale
      // This is the fallback power of SWR
      if (entry) {
        logger.warn('Serving very stale cache due to fetch failure', { key, age });
        metrics.increment('cache.very_stale_fallback');
        return { data: entry.data, status: 'stale' };
      }
      throw error;
    }
  }
 
  private async fetchAndCache(key: string): Promise<T> {
    const data = await this.fetcher();
    const now = Date.now();
    await this.cache.set(key, {
      data,
      cachedAt: now,
      refreshedAt: now
    });
    return data;
  }
 
  private backgroundRefresh(key: string): void {
    // Fire and forget - errors are logged but don't affect response
    this.fetchAndCache(key).catch(error => {
      logger.warn('Background refresh failed', { key, error: error.message });
      metrics.increment('cache.background_refresh_failed');
    });
  }
}

HTTP Cache-Control: stale-while-revalidate The HTTP specification includes native support for SWR via the Cache-Control header: Cache-Control: max-age=60, stale-while-revalidate=3600 This header tells caches: Serve the response fresh for 60 seconds After 60 seconds, serve stale while revalidating in background for up to 1 hour After 1 hour of staleness, treat as miss Browsers, CDNs, and proxy caches that support this directive automatically implement the SWR pattern, providing resilience at the edge.

SWR Libraries

Many modern frontend frameworks include SWR libraries (React's SWR, TanStack Query, Apollo Client) that implement this pattern client-side. These libraries serve cached data immediately and refresh in the background, providing excellent perceived performance and inherent resilience to API failures.

Managing Data Staleness

The central challenge in cache fallbacks is staleness management. Data that was accurate two hours ago may be dangerously outdated—or it may be perfectly fine. Understanding and managing staleness is essential for effective cache fallbacks. Staleness dimensions:

Staleness Analysis Framework
Data Type	Acceptable Staleness	Staleness Risk	Strategy
User profile info	Hours to days	Low - rarely changes	Aggressive caching with long fallback TTL
Product catalog	Minutes to hours	Medium - prices/availability change	Moderate caching with staleness indicators
Inventory levels	Seconds to minutes	High - affects purchase decisions	Short cache, conservative fallback messaging
Stock prices	Seconds	Critical - financial impact	Minimal caching, clear stale indicators, may refuse to serve
Account balance	Not acceptable	Critical - financial decisions	No cache fallback - show error instead

Staleness indicators to users: When serving stale data, transparency with users is critical. Common patterns include: Timestamp display : 'Last updated 5 minutes ago' Visual indicators : Grayed-out or faded styling for stale content Explicit messaging : 'Showing cached data. Information may be outdated.' Auto-refresh hints : 'Refresh to see latest data' Staleness badges : Small icons indicating data freshness Programmatic staleness handling: Downstream systems need to know data freshness for their own decision-making. Include staleness metadata in responses:

Staleness Metadata in Responses
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
interface CachedResponse<T> {
  data: T;
  metadata: {
    source: 'fresh' | 'cache' | 'stale_cache' | 'fallback';
    cachedAt?: Date;
    maxAge?: number;           // Original TTL
    age?: number;              // How old the data is
    staleReason?: string;      // Why we're serving stale
    refreshAttempted?: Date;   // Last refresh attempt time
    nextRefresh?: Date;        // When refresh will be tried
  };
}
 
// Controller returns this structure
async function getProduct(productId: string): Promise<CachedResponse<Product>> {
  const cacheKey = `product:${productId}`;
  const cached = await cache.getWithMetadata<Product>(cacheKey);
 
  // Fresh cache hit
  if (cached && cached.age < cached.maxAge) {
    return {
      data: cached.value,
      metadata: {
        source: 'cache',
        cachedAt: cached.timestamp,
        maxAge: cached.maxAge,
        age: cached.age
      }
    };
  }
 
  // Try to fetch fresh
  try {
    const product = await productService.fetch(productId);
    await cache.set(cacheKey, product, { maxAge: 300 });
    return {
      data: product,
      metadata: { source: 'fresh' }
    };
  } catch (error) {
    // Fetch failed - use stale cache if available
    if (cached) {
      return {
        data: cached.value,
        metadata: {
          source: 'stale_cache',
          cachedAt: cached.timestamp,
          age: cached.age,
          staleReason: 'upstream_unavailable',
          refreshAttempted: new Date()
        }
      };
    }
    throw error; // No cache, no fresh - must fail
  }
}

Cache Population Strategies

A cache provides no fallback value if it's empty. For cache fallbacks to work, the cache must be populated before failures occur. This requires deliberate cache population strategies. Reactive vs. Proactive Population:

Cache Population Approaches

Reactive (On-Demand)	Proactive (Pre-Population)
Cache populated when first requested	Cache pre-populated before requests
Cold cache on startup	Warm cache on startup
Gaps in fallback coverage	Comprehensive fallback coverage
Simple implementation	Complex orchestration needed
No wasted cache space	May cache unused data
First user sees latency/errors	All users get fallback protection

Proactive population strategies:

Cache Pre-Population Patterns

•Start-up cache warming — On service startup, populate cache with critical data before accepting traffic. This ensures no cold-cache fallback gaps for the most important data paths.
•Background refresh jobs — Scheduled jobs continuously refresh cache entries before they expire. This maintains cache freshness without relying on user requests.
•Event-driven population — Subscribe to data change events and update cache proactively. When a product is updated, immediately refresh the product cache entry.
•Traffic-pattern-based warming — Analyze traffic patterns and pre-populate cache for items likely to be requested soon. If users typically browse after login, warm user-specific caches during authentication.
•Geographical pre-replication — Populate caches in regions before traffic shifts there. If you're launching in a new region, warm caches before directing traffic.
•Tiered population priority — Prioritize caching for Tier 1 (critical) data over Tier 3 (nice-to-have) data. Critical paths should always have fallback available.

Background Cache Warmer
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
class CacheWarmer {
  private warmingQueue: Queue;
  
  constructor(
    private cache: Cache,
    private dataSource: DataSource,
    private config: CacheWarmerConfig
  ) {
    this.warmingQueue = new Queue('cache-warming');
    this.schedulePeriodicWarming();
  }
 
  // Warm cache for a specific key
  async warmKey(key: string, fetcher: () => Promise<any>): Promise<void> {
    const existing = await this.cache.get(key);
    const age = existing ? Date.now() - existing.timestamp : Infinity;
 
    // Only warm if approaching staleness threshold
    if (age > this.config.warmingThresholdMs) {
      try {
        const data = await fetcher();
        await this.cache.set(key, data, { maxAge: this.config.ttlSeconds });
        metrics.increment('cache.warmed', { key: this.keyPattern(key) });
      } catch (error) {
        logger.warn('Cache warming failed', { key, error: error.message });
        metrics.increment('cache.warming_failed', { key: this.keyPattern(key) });
      }
    }
  }
 
  // Scheduled warming of critical data
  private schedulePeriodicWarming(): void {
    // Run every minute
    setInterval(async () => {
      // Warm product catalog top 1000 products
      const topProducts = await this.dataSource.getTopProducts(1000);
      for (const product of topProducts) {
        await this.warmingQueue.add('warm-product', { productId: product.id });
      }
 
      // Warm user preferences for recently active users
      const recentUsers = await this.dataSource.getRecentlyActiveUsers(10000);
      for (const user of recentUsers) {
        await this.warmingQueue.add('warm-user-prefs', { userId: user.id });
      }
 
      metrics.gauge('cache.warming_queue_size', this.warmingQueue.length);
    }, 60000);
  }
 
  // Process warming jobs
  async processWarmingJob(job: { type: string; data: any }): Promise<void> {
    switch (job.type) {
      case 'warm-product':
        await this.warmKey(
          `product:${job.data.productId}`,
          () => this.dataSource.getProduct(job.data.productId)
        );
        break;
      case 'warm-user-prefs':
        await this.warmKey(
          `user:prefs:${job.data.userId}`,
          () => this.dataSource.getUserPreferences(job.data.userId)
        );
        break;
    }
  }
 
  private keyPattern(key: string): string {
    return key.replace(/:[a-f0-9-]+/g, ':*');
  }
}

Cache Fallback During Outages

When an outage occurs, the cache fallback system must seamlessly transition from normal operation to fallback mode. This transition involves different behaviors for reads, writes, and cache management. Read path during outages: During upstream outages, read path behavior changes: Detection : Recognize that upstream is unhealthy (timeouts, errors, circuit breaker open) Fallback activation : Switch to serving from cache without attempting upstream Extended TTL : Allow serving data beyond normal staleness thresholds Background retry : Periodically probe upstream for recovery Recovery : Resume normal operation when upstream is healthy

Outage-Aware Cache Read Pattern
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
class OutageAwareCache<T> {
  private upstreamHealthy = true;
  private lastHealthCheck = 0;
  private healthCheckIntervalMs = 5000;
 
  constructor(
    private cache: Cache,
    private fetcher: (key: string) => Promise<T>,
    private config: OutageCacheConfig
  ) {}
 
  async get(key: string): Promise<T | null> {
    // Check if we should probe upstream health
    if (!this.upstreamHealthy && this.shouldProbeHealth()) {
      this.probeUpstreamHealth();
    }
 
    // If upstream is healthy, try normal fetch
    if (this.upstreamHealthy) {
      return this.normalFetch(key);
    }
 
    // Upstream unhealthy - fallback mode
    return this.fallbackFetch(key);
  }
 
  private async normalFetch(key: string): Promise<T | null> {
    // First check cache
    const cached = await this.cache.get<CacheEntry<T>>(key);
    if (cached && !this.isStale(cached)) {
      return cached.data;
    }
 
    // Cache miss or stale - fetch from upstream
    try {
      const data = await this.fetcher(key);
      await this.cache.set(key, { data, timestamp: Date.now() });
      return data;
    } catch (error) {
      // Upstream failed - might be start of outage
      this.handleUpstreamFailure(error);
      
      // Return stale cache if available
      if (cached) {
        metrics.increment('cache.stale_fallback');
        return cached.data;
      }
      
      throw error;
    }
  }
 
  private async fallbackFetch(key: string): Promise<T | null> {
    // In fallback mode, only use cache - don't hit upstream
    const cached = await this.cache.get<CacheEntry<T>>(key);
    
    if (cached) {
      const staleness = Date.now() - cached.timestamp;
      
      // Check if within extended fallback threshold
      if (staleness < this.config.fallbackMaxStaleMs) {
        metrics.increment('cache.fallback_hit', {
          staleness_bucket: this.stalenessBucket(staleness)
        });
        return cached.data;
      }
      
      // Beyond fallback threshold - data too old
      metrics.increment('cache.fallback_too_stale');
      // Depending on policy, either return anyway with warning or return null
      if (this.config.serveVeryStale) {
        logger.warn('Serving very stale data', { key, staleness });
        return cached.data;
      }
    }
 
    return null;
  }
 
  private handleUpstreamFailure(error: Error): void {
    this.upstreamHealthy = false;
    this.lastHealthCheck = Date.now();
    logger.warn('Upstream failure detected, entering fallback mode');
    metrics.increment('cache.fallback_mode_entered');
  }
 
  private shouldProbeHealth(): boolean {
    return Date.now() - this.lastHealthCheck > this.healthCheckIntervalMs;
  }
 
  private async probeUpstreamHealth(): Promise<void> {
    this.lastHealthCheck = Date.now();
    try {
      await this.fetcher('__health_probe__');
      this.upstreamHealthy = true;
      logger.info('Upstream recovered, exiting fallback mode');
      metrics.increment('cache.fallback_mode_exited');
    } catch {
      // Still unhealthy
    }
  }
 
  private isStale(entry: CacheEntry<T>): boolean {
    return Date.now() - entry.timestamp > this.config.freshTtlMs;
  }
 
  private stalenessBucket(staleness: number): string {
    if (staleness < 60000) return '<1min';
    if (staleness < 300000) return '1-5min';
    if (staleness < 900000) return '5-15min';
    if (staleness < 3600000) return '15-60min';
    return '>1hour';
  }
}

Write Path During Outages

Cache fallbacks primarily address read operations. Write operations during outages require different patterns: queuing for later processing, optimistic writes with reconciliation, or explicit failure with retry guidance. Don't let cache fallbacks mask write failures.

Distributed Cache Considerations

In distributed systems, cache fallbacks introduce additional complexity around consistency, replication, and failover. A distributed cache optimized for fallback scenarios must handle these challenges. Consistency challenges:

Distributed Cache Fallback Challenges

•Stale replica reads — Different cache nodes may have data of different ages. During fallback, you might serve older data from one request and newer data from the next.
•Network partitions — If the cache cluster is partitioned, some nodes may fall back while others don't. Users hitting different nodes get different experiences.
•Cache invalidation during outage — If data changes in the (unavailable) source during an outage, the cache becomes incorrect. When the source recovers, the cache holds stale data.
•Thundering herd on recovery — When upstream recovers, all nodes simultaneously try to refresh cache, potentially overwhelming the just-recovered upstream.
•Clock skew — Distributed systems have clock differences. Staleness calculations based on timestamps may be inconsistent across nodes.

Mitigation strategies: Read-repair during fallback : When serving stale data, also trigger an async check against other cache replicas for fresher data. Versioned cache entries : Include version numbers in cache entries. During fallback, prefer higher versions even if slightly older by timestamp. Coordinated invalidation : Use a distributed coordination service for invalidations. If the source updates, invalidation can still propagate via the coordinator. Jittered refresh on recovery : Add random delays to recovery refreshes to avoid thundering herd. Logical timestamps : Use logical clocks (Lamport timestamps, vector clocks) rather than wall-clock time for staleness tracking.

Redis as Fallback Cache

Redis Cluster with read replicas provides a good foundation for cache fallbacks. Use READONLY commands against replicas for fallback reads, reducing load on primaries. Configure replicas with higher persistence than primaries to maintain fallback data during primary failures. Sentinel or Cluster mode handles automatic failover.

Cache Fallbacks

When Yesterday's Data Beats No Data

Understanding Cache Fallbacks

Static Defaults vs. Cache Fallbacks

Cache Architecture for Resilience

Stale-While-Revalidate Pattern

Managing Data Staleness

Cache Population Strategies

Cache Population Approaches

Cache Fallback During Outages

Distributed Cache Considerations

There’s more to understand.

Cache Fallbacks

When Yesterday's Data Beats No Data

Understanding Cache Fallbacks

Static Defaults vs. Cache Fallbacks

Cache Architecture for Resilience

Stale-While-Revalidate Pattern

Managing Data Staleness

Cache Population Strategies

Cache Population Approaches

Cache Fallback During Outages

Distributed Cache Considerations

There’s more to understand.