Netflix Streaming Service Design

Delivering video at Netflix scale confronts fundamental physics constraints. Light in fiber optics travels at approximately 200,000 kilometers per second—about two-thirds the speed of light in vacuum. A round trip from New York to Tokyo covers roughly 22,000 kilometers, requiring a minimum of 110 milliseconds just for the signal to travel—before any server processing, queueing, or content delivery.

This latency is immutable. No amount of engineering can make electrons travel faster. The only solution is to bring content closer to users.

Furthermore, streaming 50+ terabits per second from centralized data centers would require backbone network capacity that simply doesn't exist. The internet's core network would collapse. The only feasible architecture pushes content to the edge—as close to end users as physically possible.

This page explores how Netflix architected a content delivery system that effectively shrinks the globe, placing content within milliseconds of virtually every subscriber while maintaining consistency, freshness, and cost efficiency.

Netflix's content delivery follows a hierarchical architecture where content flows from origin storage through multiple tiers before reaching end users. Each tier serves a specific purpose in the content distribution pipeline.

The origin layer is where content begins its journey. This isn't just storage—it's a sophisticated content preparation and management system that transforms raw master files into optimized delivery formats.

Per-Title Encoding:

Netflix pioneered per-title encoding—analyzing each piece of content individually to determine optimal bitrate ladders. A static shot of two people talking can look perfect at 1 Mbps. An action sequence with explosions needs 10 Mbps for equivalent quality.

Instead of one-size-fits-all encoding profiles, Netflix's system:

1. Analyzes content complexity per scene (motion, texture, dark scenes)
2. Runs thousands of encoding trials at different bitrate/quality combinations
3. Builds a custom encoding ladder that maximizes quality per bit for that specific title
4. Can achieve 50%+ bitrate savings over static encoding while maintaining quality

This is why Netflix can stream 4K over connections where competitors struggle with HD—they're more efficient per bit.

Open Connect Appliances (OCAs) are purpose-built servers optimized for video delivery. Unlike general-purpose CDN edge nodes, these machines are designed with a single mission: stream video as efficiently as possible.

Capacity Per OCA:

A single OCA can serve:

• 100+ Gbps of video traffic
• 30,000+ concurrent streams
• Millions of requests per hour

With 15,000+ OCAs globally, Netflix has over 1,500 Tbps of edge capacity—more than most countries' total internet capacity.

Storage Tiering Within OCA:

Even within a single server, content is tiered:

1. Hot Tier (Memory): Most popular content (top 100 titles) kept in RAM. Instant serving.
2. Warm Tier (SSD): Popular content (top 10,000 files) on fast flash storage. Sub-millisecond access.
3. Cold Tier (or Missing): Rarely accessed content may not be on this OCA. Fetched from fill server on demand.

Effective cache hit rates exceed 95% for edge locations—meaning 95% of requests are served entirely from local storage with no upstream fetch.

With thousands of edge locations, Netflix must intelligently route each viewer to the optimal server. This isn't simple geographic proximity—the routing system considers dozens of factors to maximize quality while minimizing cost.

Steering Mechanisms:

Netflix uses multiple mechanisms to direct traffic:

1. DNS-Based Steering Initial server selection via DNS. Client resolves netflix.com and receives IP addresses of recommended CDN servers. DNS responses are customized per-request based on client IP's location and network.

2. HTTP Redirect Steering After initial connection, manifest files contain URLs pointing to specific CDN servers. The control plane can dynamically update these between segments to rebalance load or recover from failures.

3. Client-Side Selection The Netflix player can probe multiple candidate servers and select based on measured performance. This provides ultimate flexibility but adds complexity.

4. BGP Anycast (Limited Use) For some high-level routing, anycast IPs route to the topologically nearest server. Less precise but very fast for initial connection.

Unlike reactive caching (fetch on demand), Netflix proactively pushes content to edge servers before users request it. This is possible because video streaming is uniquely predictable—new content is known in advance.

Fill During Night Hours:

Netflix schedules most cache filling during local off-peak hours (typically 2-6 AM). Benefits:

1. Network Capacity — Internet backbone has abundant capacity at night. Filling is cheap and fast.
2. Edge Server Headroom — OCAs not busy streaming can dedicate resources to receiving fill traffic.
3. ISP Relationships — Using off-peak capacity is positive for ISP partnerships—Netflix isn't competing with daytime traffic.

Mathematical Modeling:

The cache filling system solves an optimization problem:

• Minimize total egress from origin (bandwidth cost)
• Maximize cache hit rate at edge (user experience)
• Subject to storage constraints (not all content fits on all servers)
• Subject to fill bandwidth constraints (can't transfer faster than links allow)

This is fundamentally a variant of the facility location problem with time-varying demand.

Unlike most distributed systems where consistency is critical, video content is immutable once encoded. Episode 5 of a show doesn't change. This simplifies consistency dramatically—but introduces other challenges around content updates, removals, and version management.

Content Addressing:

Netflix uses content-addressable storage at the edge. File names include content hash:

movie_12345_video_1080p_h264_hash_a7b3c9d2e1f4.mp4

Benefits of content addressing:

• Deduplication — Identical content stored once regardless of how many manifests reference it
• Cache Invalidation — Update manifest to point to new hash; old content naturally becomes unreachable
• Corruption Detection — Hash mismatch indicates storage error; re-fetch from upstream
• CDN Transparency — CDN doesn't need to understand content semantics; just serve files by name

Manifest Version Control:

Manifests (the 'table of contents' for adaptive streaming) are versioned separately from content. When anything changes:

1. Generate new manifest version
2. Push to edge caches (small file, fast propagation)
3. New playback sessions get new manifest
4. In-progress sessions continue with old manifest (graceful transition)
5. Old manifest expires after TTL (typically hours)

With 15,000+ servers across 1,000+ locations, failures are constant—not exceptional. The architecture must treat failure as normal and maintain service through all but the most catastrophic scenarios.

Chaos Engineering:

Netflix pioneered Chaos Engineering—intentionally injecting failures to verify resilience. Famous examples:

• Chaos Monkey — Randomly terminates production instances to ensure services handle server failure
• Chaos Kong — Simulates entire region failure to verify cross-region failover
• Latency Monkey — Injects artificial latency to test timeout handling

For the CDN specifically:

• Randomly pull servers from rotation to verify traffic redistribution
• Simulate fill server failures to verify edge caches serve independently
• Test content invalidation propagation speed

Recovery Time Objectives:

Content delivery is Netflix's largest infrastructure cost after content licensing itself. Understanding the economics explains many architectural decisions.

ISP Embedding Economics:

When Netflix places servers inside an ISP's network:

For Netflix:

• Eliminates transit costs (often $0.50-5.00 per Mbps)
• Lower latency = better quality = happier subscribers
• Reduced origin egress as content served locally

For ISPs:

• Reduced backbone traffic (Netflix is 15%+ of their traffic)
• Better customer experience = less churn
• Free hardware (Netflix provides and maintains OCAs)

This is why Netflix offers Open Connect free to ISPs—it's still cheaper than paying transit. Major ISPs have hundreds of OCAs embedded in their networks.

Cost-Per-Stream Optimization:

All architectural decisions ultimately aim to minimize cost-per-stream while maintaining quality:

• Better encoding → lower bitrate → lower bandwidth cost
• Higher cache hit rate → less origin egress
• ISP embedding → near-zero transit cost
• Server efficiency → more streams per dollar of hardware

Netflix's effective cost per stream is estimated at fractions of a cent—remarkable for delivering gigabytes of video per viewing session.