There's a deep irony in load balancing: the very component designed to eliminate single points of failure can itself become the single point of failure.

Consider a system with 100 highly available backend servers, each with 99.9% uptime. Together, they could theoretically achieve astronomical availability. But if all their traffic flows through a single load balancer with 99.9% uptime, the entire system's availability is capped at that 99.9%—roughly 8.76 hours of downtime per year.

The math is unforgiving:

• System availability = min(component availabilities) when components are in series
• If LB availability = 99.9%, then system availability ≤ 99.9%, regardless of backend redundancy

This means that investing in backend redundancy while neglecting load balancer high availability is architectural waste. The load balancer's reliability must match or exceed the availability targets of the entire system.

To design for high availability, we must first understand the failure modes of load balancers. These failures fall into several categories:

The Cascading Failure Problem:

The most dangerous failure mode is the cascade. Consider:

1. You have 2 load balancers, each handling 50% of traffic
2. Load balancer A fails
3. All traffic shifts to load balancer B (now at 100%)
4. Load balancer B, sized for 50%, becomes overloaded
5. Load balancer B's performance degrades, then it crashes
6. Complete outage

This is why high-availability load balancing isn't just about having a spare—it's about having capacity headroom across all instances to absorb failures.

The active-passive (also called hot standby) pattern is the simplest approach to load balancer high availability. One load balancer actively handles all traffic while another stands by, ready to take over if the primary fails.

How Active-Passive Works:

1. Virtual IP (VIP): A floating IP address that clients connect to. This IP is currently bound to the active load balancer.

2. Heartbeat Monitoring: The passive load balancer continuously monitors the active one through heartbeat messages, typically using protocols like VRRP (Virtual Router Redundancy Protocol) or keepalived.

3. Failover: When the passive detects the active has failed (missed heartbeats, health check failures), it takes over the VIP. Network switches learn the new MAC-to-IP mapping, and traffic flows to the new active.

4. Recovery: When the original primary recovers, it can either stay passive (non-preemptive) or reclaim the active role (preemptive).

The active-active pattern runs multiple load balancers simultaneously, all handling traffic. This eliminates the wasted capacity of active-passive while also enabling horizontal scaling.

How Active-Active Works:

The challenge of active-active is: how do clients find the load balancers? There are several approaches:

Approach 1: DNS Round-Robin

DNS returns multiple A records (IP addresses) for the load balancer hostname. Clients choose one (usually randomly or round-robin). If that load balancer fails, clients eventually retry with another IP.

• Pros: Simple, widely supported
• Cons: DNS caching delays failover (minutes), no health awareness

Approach 2: BGP Anycast

All load balancers advertise the same IP address via BGP routing. Network routers automatically send traffic to the 'nearest' load balancer. If one fails, BGP reconverges.

• Pros: Fast failover (seconds), automatic geographic routing
• Cons: Requires network control, complex operations

Approach 3: External Load Balancer Layer

A higher-tier load balancer (like a cloud provider's NLB or GLB) distributes traffic across your load balancer pool.

• Pros: Managed, health-aware, fast failover
• Cons: Added layer, cost, dependency on provider

Approach 4: Client-Side Load Balancing

Clients are given a list of load balancer IPs and implement their own load balancing logic, including failover.

• Pros: No single point of failure, fast failover
• Cons: Complex client logic, harder to update addresses

State Synchronization Challenge:

Active-active creates a challenge: if a client's request goes to LB1 first and then LB2, will LB2 have the session state?

Solutions:

1. Stateless load balancing: Design so session state isn't needed at the LB. Use sticky sessions at the application layer or externalize session state to Redis.

2. State replication: Load balancers synchronize state between themselves (HAProxy supports this, as do some commercial solutions). Adds complexity and latency.

3. Consistent hashing/affinity: Use client IP or other attributes to consistently route the same client to the same load balancer. Only failed LB's sessions disrupt.

DNS-based failover uses the Domain Name System to direct traffic away from failed load balancers (or entire regions). While DNS isn't traditionally thought of as a load balancing layer, modern DNS services provide sophisticated health checking and traffic management.

How DNS-Based Failover Works:

DNS Failover Limitations:

1. TTL Caching Delays Failover

DNS responses are cached by resolvers, browsers, and operating systems. Even with a 60-second TTL:

• Some resolvers ignore TTLs and cache longer
• Existing connections aren't affected by DNS changes
• Real-world failover often takes 1-5 minutes, not seconds

2. No Connection-Level Awareness

DNS only affects new connections. Existing TCP connections to a failed load balancer will hang until timeout. Applications must implement connection-level failover.

3. No Load Awareness

DNS failover typically only knows healthy/unhealthy, not 'overloaded.' A load balancer at 99% CPU is still 'healthy' from DNS perspective.

Mitigation Strategies:

Cloud providers have essentially 'solved' the load balancer SPOF problem with managed services. Understanding how they achieve this helps you design better, whether you use their services or build your own.

How Cloud Load Balancers Achieve HA:

AWS ALB/NLB Architecture:

• Runs on a cluster of EC2 instances invisible to you
• Deployed across all AZs in the region you select
• Auto-scales based on traffic (within minutes)
• Health checks backends and removes unhealthy ones
• Uses internal DNS to distribute traffic across LB nodes

Google Cloud Load Balancing:

• Uses Anycast: one IP address, served from hundreds of PoPs globally
• Traffic enters the Google network at the nearest PoP
• Google's internal network routes to backends
• No single point of failure by design

Key Insight: The Load Balancer Is a Fleet, Not a Box

Cloud load balancers aren't single machines—they're fleets of machines behind a managed abstraction. This is why they achieve 99.99% SLAs: any individual machine failure is invisible.

High availability isn't just about having backups—it's about ensuring those backups have sufficient capacity to handle failure scenarios. This requires careful capacity planning.

Capacity Calculation Example:

Scenario: You handle 100K requests/second at peak.

Single load balancer capacity: 60K requests/second (fully utilized)

Option 1: Active-Passive (N+1)

• 2 load balancers (1 active, 1 passive)
• Active handles 100K req/s at ~167% theoretical capacity ❌
• Not viable—single LB can't handle load

Option 2: Active-Active (2N)

• 4 load balancers, each handles 25K req/s (~42% capacity)
• If 2 fail, remaining 2 handle 50K req/s each (~83% capacity) ✓
• If 1 fails, remaining 3 handle 33K req/s each (~55% capacity) ✓

Option 3: N+1 with larger instances

• 2 load balancers, each capable of 100K req/s
• Normal: each handles 50K (~50% capacity)
• One fails: survivor handles 100K (~100% capacity) ⚠️ (risky, no headroom)

Recommendation: N+2 or 2N for critical systems.

Technical architecture is only half the battle. Operational practices determine whether your HA design actually delivers its promised reliability.

Common Operational Failure Patterns:

Let's consolidate the key concepts from this page and the module as a whole:

Module 1 Summary: What Is Load Balancing?

Over these four pages, we've established a comprehensive understanding of load balancing:

1. Definition and Purpose — Load balancing distributes traffic across resources to optimize utilization, maximize throughput, minimize latency, and avoid overload.

2. Benefits — Availability (through redundancy and failover), performance (through distribution and optimization), and flexibility (through abstraction and operational agility).

3. Placement — Load balancing occurs at multiple tiers: edge (external traffic), middle (service-to-service), and data (database/cache access).

4. High Availability — The load balancer itself must be made highly available through redundancy patterns, careful capacity planning, and operational discipline.