Content Caching: The Intelligence Behind Efficient Delivery

A global CDN with 300 edge locations serving 50 million requests per second faces a fundamental challenge: how do you store the right content at each location to maximize cache hits while minimizing staleness?

The answer to this question determines whether a CDN achieves 95% cache efficiency or 50%—the difference between extraordinary user experience and mediocre performance. Caching is where the physical infrastructure we've studied becomes intelligent, making real-time decisions about what to store, where to store it, and when to expire it.

Caching is deceptively complex. On the surface, it's simple: store a copy of content closer to users. In practice, it involves intricate decisions about cache keys, TTLs, invalidation strategies, consistency guarantees, and storage hierarchies—each choice affecting performance, freshness, and cost.

CDN caching is built upon the HTTP caching model defined in RFC 7234. Understanding these fundamentals is essential for effective CDN configuration and troubleshooting.

The cacheability decision:

HTTP defines when a response can be cached and for how long through response headers:

1. Cache-Control — The primary mechanism for cache directives
2. Expires — Legacy absolute expiration time (superseded by Cache-Control)
3. ETag — Entity tag for validation-based caching
4. Last-Modified — Timestamp for validation-based caching

Cache-Control directive reference:

Validation-based caching:

When cache content expires (TTL reached), the cache can validate whether the content has changed rather than fetching a full copy:

Client → CDN: GET /resource
               If-None-Match: "abc123"     # ETag from previous response
               If-Modified-Since: Tue, 15 Jan 2025 10:00:00 GMT

CDN → Origin: GET /resource
              If-None-Match: "abc123"
              If-Modified-Since: Tue, 15 Jan 2025 10:00:00 GMT

Origin → CDN: HTTP/1.1 304 Not Modified
              ETag: "abc123"
              Cache-Control: public, max-age=3600

CDN → Client: HTTP/1.1 304 Not Modified
              ETag: "abc123"
              Cache-Control: public, max-age=3600

Key insight: The 304 response has no body—just headers. For large resources, conditional validation saves significant bandwidth while ensuring freshness.

Enterprise CDNs employ multi-tiered cache hierarchies that balance access speed, storage capacity, and origin offload. Understanding this hierarchy is essential for optimizing cache efficiency.

The typical four-tier cache hierarchy:

Tier 1: Memory Cache (RAM)

The fastest tier stores the most frequently accessed content in server RAM:

• Capacity: 100-512GB per server (subset of total content)
• Access time: <1 microsecond
• Hit rate: 30-50% of requests (for popular content distributions)
• Eviction policy: LRU (Least Recently Used) or LFU (Least Frequently Used)
• Persistence: Lost on server restart

Tier 2: Local Disk Cache (NVMe SSD)

The primary persistent cache tier stores the working set:

• Capacity: 30-100TB per server (larger content catalog)
• Access time: 50-200 microseconds (random read)
• Hit rate: 40-60% of remaining requests
• Eviction policy: LRU with popularity scoring
• Persistence: Survives server restart; enables instant cache warming

Tier 3: Origin Shield

A regional cache tier that aggregates requests from multiple edge servers:

• Purpose: Protect origin from thundering herd; improve cache efficiency
• Behavior: Edge servers fetch from shield on cache miss (not directly from origin)
• Location: 3-10 global shield locations (regional coverage)
• Benefit: 10 edges with 80% CHR each combine to 98% origin offload via shield

Cache tier selection logic:

When a request arrives, the edge server checks tiers in order:

1. Check RAM cache → HIT? Serve immediately (fastest path)
                   → MISS? Check next tier

2. Check SSD cache  → HIT? Serve and promote to RAM
                   → MISS? Check shield tier

3. Check Shield    → HIT? Serve and cache locally
                   → MISS? Fetch from origin

4. Fetch Origin    → Cache at shield + local SSD + RAM (if hot)
                   → Serve to client

Promotion and demotion:

• Content accessed frequently is promoted up tiers (SSD → RAM)
• Content not accessed recently is demoted down tiers or evicted
• Each tier runs independent eviction based on its constraints
• Hot content exists in all tiers simultaneously; cold content only on disk

The cache key is the unique identifier used to store and retrieve cached content. Cache key design directly determines hit rate—poor key design can cause cache fragmentation that devastates performance.

Default cache key components:

By default, most CDNs construct cache keys from:

Scheme + Host + Path + Query String
https://example.com/images/logo.png?v=123

The cache key problem:

This default key can cause issues when variations don't indicate different content:

• ?utm_source=google vs ?utm_source=facebook — Same content, different keys!
• Query parameter order: ?a=1&b=2 vs ?b=2&a=1 — Same content, different keys!
• Tracking parameters: ?_=1705350000 (cache-busting timestamps)

Result: Multiple cache entries for identical content → lower CHR → higher origin load.

Cache key best practices:

1. Audit your cache keys: Use CDN analytics to identify URLs with unexpectedly low CHR; investigate cache key fragmentation.

2. Minimize key cardinality: Every unique cache key is a separate entry. Lower cardinality = higher hit rate.

3. Separate static and dynamic: Static assets should have simple keys (just URL). Dynamic content may need additional attributes.

4. Test cache key changes: Incorrect key configuration can cause users to receive wrong content. Test thoroughly in staging.

5. Document your strategy: Cache key configuration is operational knowledge that must be maintained.

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton

Cache invalidation—ensuring cached content is removed or refreshed when the authoritative source changes—is notoriously challenging. CDNs offer multiple invalidation strategies, each with tradeoffs between immediacy, cost, and operational complexity.

Beyond basic TTL-based caching, production CDNs implement sophisticated patterns to handle edge cases, optimize efficiency, and maintain consistency under challenging conditions.

Request coalescing deep dive:

Request coalescing (also called request collapsing or request deduplication) is essential for handling viral content and flash crowds:

┌──────────────────────────────────────────────────────────┐
│                   Edge Server Timeline                    │
├──────────────────────────────────────────────────────────┤
│                                                          │
│ t=0ms:   Request A arrives for /viral-video.mp4 (MISS)  │
│          → Origin request initiated                      │
│                                                          │
│ t=5ms:   Request B arrives for /viral-video.mp4         │
│          → Sees pending request, JOINS WAIT GROUP        │
│                                                          │
│ t=10ms:  Request C arrives for /viral-video.mp4         │
│          → Joins same wait group                         │
│                                                          │
│ t=100ms: Origin response received                        │
│          → Content cached                                │
│          → All waiting requests (A, B, C) satisfied      │
│                                                          │
│ Result: 3 client requests, 1 origin request              │
│ Without coalescing: 3 client requests, 3 origin requests │
└──────────────────────────────────────────────────────────┘

For viral content with 10,000 simultaneous requests, coalescing reduces origin load from 10,000 requests to 1. This is why CDNs can handle flash crowds that would devastate any origin server.

While CDNs excel at static content, increasingly they cache dynamic content—personalized pages, API responses, and real-time data. This requires careful strategy to balance freshness, personalization, and efficiency.

The dynamic content challenge:

Dynamic content varies by user, time, or context. A product page might include:

• Static content: Product description, images (cache forever)
• Semi-dynamic: Price, availability (cache 60 seconds)
• User-specific: Cart status, recommendations (don't cache on CDN)

Pattern: HTTP Cache Fragments with Edge Composition

Modern CDNs can assemble pages from independently cached fragments:

┌─────────────────────────────────────────────────────────────┐
│                         Full Page                            │
├─────────────────────────────────────────────────────────────┤
│  ┌────────────────────────────────────────────────────┐     │
│  │     Header Fragment (shared, TTL: 1 hour)          │     │
│  └────────────────────────────────────────────────────┘     │
│  ┌──────────────────────┐  ┌──────────────────────────┐    │
│  │ User Menu Fragment   │  │ Search Fragment (shared) │    │
│  │ (private, no cache)  │  │ (TTL: 1 minute)          │    │
│  └──────────────────────┘  └──────────────────────────┘    │
│  ┌────────────────────────────────────────────────────┐     │
│  │     Content Fragment (shared, TTL: 5 minutes)      │     │
│  └────────────────────────────────────────────────────┘     │
│  ┌────────────────────────────────────────────────────┐     │
│  │     Footer Fragment (shared, TTL: 1 hour)          │     │
│  └────────────────────────────────────────────────────┘     │
└─────────────────────────────────────────────────────────────┘

• Most fragments cached and shared across all users
• User Menu fetched client-side (JavaScript) or excluded from cache
• Page delivered fast; small personalized parts lazy-loaded

With content cached across 200+ global locations, maintaining consistency is a significant challenge. Different users may see different content versions depending on which edge server they hit and the local cache state.

The consistency challenge:

Consider this timeline:

t=0:    Price is $10.00; cached globally with TTL=300s
t=60s:  Price updated to $12.00 at origin
t=61s:  User A (Tokyo edge) sees $10.00 (cached)
t=62s:  User B (NYC edge) sees $12.00 (edge had cache miss)
t=180s: User A still sees $10.00; User B sees $12.00

Result: Same product, different prices for 4+ minutes

Consistency models for CDN caching:

Implementing bounded staleness:

For most applications, bounded staleness provides acceptable consistency at reasonable cost:

1. Set s-maxage to maximum acceptable staleness (e.g., 60 seconds)
2. Use stale-while-revalidate for background updates
3. For critical updates, trigger instant purge via API
4. Result: Content is at most 60 seconds stale except when explicitly purged

The version field pattern:

For content that must be consistent across API responses, include a version field:

{
  "product_id": 123,
  "name": "Widget",
  "price": 12.00,
  "cache_version": "2025-01-17T10:30:00Z",
  "_meta": {
    "cached_at": "2025-01-17T10:32:15Z",
    "edge_location": "tokyo-01"
  }
}

Clients can compare cache_version across responses and detect inconsistencies. If critical actions require consistency, clients can request with Cache-Control: no-cache to bypass CDN.