Load Balancer Comparison - Learning Module

Loading content...

0/273

HAProxy — High Performance, TCP/HTTP

The Gold Standard for High-Performance Proxying

When organizations require maximum throughput, minimal latency, and rock-solid reliability from their load balancing infrastructure, the conversation inevitably turns to HAProxy. First released in 2001 by Willy Tarreau, HAProxy (High Availability Proxy) has evolved into the definitive solution for high-performance TCP and HTTP load balancing, trusted by some of the world's most demanding platforms including GitHub, Stack Overflow, Reddit, Tumblr, and Instagram.

Unlike general-purpose web servers that evolved into load balancers, HAProxy was designed from inception with a singular focus: move traffic between clients and servers with maximum efficiency and reliability. This laser focus manifests in every architectural decision, configuration option, and performance benchmark.

HAProxy routinely achieves millions of sessions per second and saturates 100 Gbps network links on modern hardware. It maintains session rates that exceed what most applications can realistically generate, making it effectively infinitely scalable for practical purposes. Understanding HAProxy deeply means understanding what's possible at the limits of network performance.

Learning Objectives

By completing this page, you will understand HAProxy's threading model and event architecture, master its configuration paradigm for TCP and HTTP proxying, comprehend its unique features like stick tables and runtime API, and recognize when HAProxy is the optimal choice over alternatives.

Architecture Deep Dive: Multi-Threaded Event Processing

HAProxy's architecture has evolved significantly over its two-decade history, culminating in a sophisticated multi-threaded event-driven model that extracts maximum performance from modern multi-core systems.

Historical Evolution:

Pre-1.8 (single-process): Like early NGINX, HAProxy ran a single event loop. Performance scaled by running multiple independent processes, each binding to shared ports via SO_REUSEPORT.
HAProxy 1.8+ (multi-threaded): Introduced native multi-threading within a single process, enabling shared memory for stick tables, counters, and connection state. This is the modern, recommended deployment model.

The multi-threaded architecture provides significant advantages:

Multi-Threading Advantages

•Shared State — Stick tables, rate limiting counters, and session data are shared across all threads without external synchronization
•Memory Efficiency — Single process memory footprint versus multiple isolated processes
•Atomic Operations — Configuration reloads can preserve existing connections without disruption
•Simpler Operations — Single PID to manage, cleaner systemd integration, unified logging
•Connection Locality — Threads can be pinned to CPU cores for cache efficiency

The Event-Driven Core:

Each HAProxy thread runs an independent event loop, using epoll (Linux), kqueue (BSD), or platform-appropriate mechanisms. Unlike NGINX's request-oriented processing, HAProxy operates on connection state machines:

New connection arrives → allocate session structure
State machine initialized → waiting for client data
Data received → parse protocol (TCP/HTTP), evaluate ACLs
Routing decision made → connect to backend or serve cached response
Bidirectional data relay → optimized memory management
Connection close → cleanup, logging, statistics update

This state machine model enables HAProxy to handle connections at any protocol stage efficiently, including incomplete HTTP requests, slow clients, and long-lived connections.

Thread Configuration

haproxy

global
    # Thread configuration
    nbthread 4                      # Number of threads (typically = CPU cores)
    cpu-map auto:1/1-4 0-3          # Pin threads 1-4 to CPU cores 0-3
    
    # Process and privilege management
    user haproxy
    group haproxy
    chroot /var/lib/haproxy
    
    # Performance tuning
    maxconn 100000                  # Global maximum connections
    tune.ssl.default-dh-param 2048
    tune.bufsize 16384              # Buffer size per connection
    tune.maxrewrite 1024
    
    # Logging
    log /dev/log local0
    log /dev/log local1 notice
    
    # Stats socket for runtime API
    stats socket /var/run/haproxy.sock mode 660 level admin expose-fd listeners
    stats timeout 30s

CPU Pinning for Maximum Performance

Use cpu-map to pin HAProxy threads to specific CPU cores, ensuring cache locality and reducing context switch overhead. For NUMA systems, ensure threads are pinned to cores on the same NUMA node as the network interfaces they serve to avoid cross-node memory access penalties.

Configuration Paradigm: Frontends, Backends, and ACLs

HAProxy's configuration model is built around three core concepts: frontends (where connections arrive), backends (where connections are forwarded), and ACLs (access control lists that drive routing decisions). This separation provides exceptional flexibility for complex routing scenarios.

HAProxy Load Balancing Configuration

haproxy

# Global settings (shown previously)
global
    nbthread 4
    maxconn 100000
    log stdout format raw local0
 
# Default settings applied to all frontends/backends
defaults
    mode http                       # HTTP mode (vs tcp)
    log global
    option httplog                  # Detailed HTTP logging
    option dontlognull              # Don't log health checks
    option http-server-close        # Close server connection after response
    option forwardfor               # Add X-Forwarded-For header
    option redispatch               # Retry on different server if connection fails
    retries 3                       # Retry count before giving up
    
    timeout connect 5s              # Time to establish backend connection
    timeout client 50s              # Max inactivity on client side
    timeout server 50s              # Max inactivity on server side
    timeout http-request 10s        # Max time for complete HTTP request
    timeout http-keep-alive 10s     # Max idle time for keep-alive
    timeout queue 30s               # Max time waiting in queue
 
# Frontend: Entry point for client connections
frontend http_front
    bind *:80
    bind *:443 ssl crt /etc/haproxy/certs/combined.pem alpn h2,http/1.1
    
    # Redirect HTTP to HTTPS
    http-request redirect scheme https unless { ssl_fc }
    
    # ACL definitions for routing
    acl is_api path_beg /api/
    acl is_static path_beg /static/ /assets/ /images/
    acl is_websocket hdr(Upgrade) -i websocket
    acl is_health path /health /ready /live
    
    # Route based on ACLs
    use_backend api_servers if is_api
    use_backend static_servers if is_static
    use_backend ws_servers if is_websocket
    use_backend health_servers if is_health
    default_backend app_servers
 
# Backend: Pool of servers to handle requests
backend api_servers
    balance roundrobin
    option httpchk GET /health HTTP/1.1\r\nHost:\ api.internal
    http-check expect status 200
    
    server api1 10.0.1.10:8080 check inter 5s fall 3 rise 2 weight 100
    server api2 10.0.1.11:8080 check inter 5s fall 3 rise 2 weight 100
    server api3 10.0.1.12:8080 check inter 5s fall 3 rise 2 weight 100
    server api4 10.0.1.13:8080 check inter 5s fall 3 rise 2 backup
 
backend app_servers
    balance leastconn
    option httpchk GET /health
    
    cookie SERVERID insert indirect nocache
    
    server app1 10.0.2.10:8080 check cookie s1
    server app2 10.0.2.11:8080 check cookie s2
    server app3 10.0.2.12:8080 check cookie s3
 
backend static_servers
    balance uri                     # Route based on URI hash for cache efficiency
    hash-type consistent
    
    server static1 10.0.3.10:80 check
    server static2 10.0.3.11:80 check
 
backend ws_servers
    balance source                  # Stick WebSocket connections by IP
    
    timeout tunnel 1h              # Long timeout for WebSocket
    
    server ws1 10.0.4.10:8080 check
    server ws2 10.0.4.11:8080 check

Configuration Analysis:

defaults section: Establishes baseline behavior inherited by all frontends and backends. Override individual settings as needed.
bind with SSL: HAProxy terminates TLS with excellent performance. The alpn parameter enables HTTP/2 negotiation.
ACL composition: ACLs can reference headers (hdr), paths (path_beg, path_end, path_reg), cookies, IP addresses, and more. Multiple conditions combine with implicit AND; use or for explicit OR.
use_backend: Directives evaluated in order; first match wins. default_backend handles unmatched requests.
Health checking: option httpchk sends HTTP requests rather than just TCP connection checks. The fall and rise parameters control failure/recovery thresholds.
Cookie insertion: cookie SERVERID insert adds a session cookie for server affinity, enabling stateful application support.

Mode TCP vs HTTP

HAProxy operates in two primary modes: mode http for Layer 7 processing (inspecting HTTP headers, cookies, etc.) and mode tcp for Layer 4 passthrough (minimal processing, maximum throughput). TCP mode is used for non-HTTP protocols like MySQL, PostgreSQL, Redis, or when SSL passthrough is required.

Load Balancing Algorithms

HAProxy implements a comprehensive set of load balancing algorithms, including several unique to its platform. The choice of algorithm significantly impacts load distribution, cache efficiency, and failure behavior.

HAProxy Load Balancing Algorithms
Algorithm	Keyword	Behavior	Optimal Use Case
Round Robin	roundrobin	Cycles through servers sequentially, respecting weights	General HTTP workloads, homogeneous backends
Static Round Robin	static-rr	Round robin without dynamic weight adjustment	When weight stability is critical
Least Connections	leastconn	Routes to server with fewest active connections	Long-lived connections, variable request duration
Source IP Hash	source	Hashes client IP for consistent routing	Session persistence without cookies
URI Hash	uri	Hashes request URI for consistent routing	Caching proxies, CDN backends
URL Parameter	url_param	Hashes specific URL parameter	Application-controlled routing (e.g., ?user_id=)
Header Hash	hdr(name)	Hashes specific header value	Route by User-Agent, API version, etc.
Random	random	Selects random server (optionally weighted)	Large pools, avoiding hotspots
First Available	first	Uses first available server up to maxconn	Cost optimization (fill servers before scaling)

Algorithm Examples

haproxy

# Weighted Least Connections — ideal for heterogeneous backends
backend api_weighted
    balance leastconn
    server api-large 10.0.1.10:8080 weight 200    # 4-core machine
    server api-small1 10.0.1.11:8080 weight 100   # 2-core machines
    server api-small2 10.0.1.12:8080 weight 100
 
# URI-based consistent hashing for cache distribution
backend cache_cluster
    balance uri
    hash-type consistent          # Minimize rehashing on server changes
    hash-balance-factor 150       # Allow 150% of average load per server
    
    server cache1 10.0.2.10:11211 check
    server cache2 10.0.2.11:11211 check
    server cache3 10.0.2.12:11211 check
 
# URL parameter-based routing for user sharding
backend user_shards
    balance url_param user_id check_post
    hash-type consistent
    
    server shard1 10.0.3.10:8080 check
    server shard2 10.0.3.11:8080 check
    server shard3 10.0.3.12:8080 check
    server shard4 10.0.3.13:8080 check
 
# First-available for cost optimization (fill before scale)
backend cost_optimized
    balance first
    
    server primary 10.0.4.10:8080 maxconn 1000 check
    server secondary 10.0.4.11:8080 maxconn 1000 check
    server tertiary 10.0.4.12:8080 maxconn 1000 check
 
# Random with power-of-two choices (HAProxy 2.4+)
backend random_p2c
    balance random(2)             # Pick 2 random, choose least loaded
    
    server s1 10.0.5.10:8080 check
    server s2 10.0.5.11:8080 check
    server s3 10.0.5.12:8080 check
    server s4 10.0.5.13:8080 check

Consistent Hashing Deep Dive:

The hash-type consistent directive implements consistent hashing with bounded load via hash-balance-factor. When a server is added or removed, only a fraction of keys are remapped, preserving cache locality. The balance factor (default: 0 = disabled) limits how overloaded a single server can become before shedding load to others.

For example, with hash-balance-factor 150, a server will accept up to 150% of the average load. If it exceeds this threshold, the consistent hashing algorithm spills requests to the next server in the ring. This provides a balance between cache efficiency and load distribution.

Algorithm-Server Count Interaction

Some algorithms behave poorly with few servers. random with 2 servers is effectively 50/50 regardless of load. uri hashing with 2 servers provides no distribution benefit and high collision rates. Match algorithm complexity to your server pool size.

Advanced Health Checking

HAProxy implements comprehensive health checking capabilities that surpass most competitors. Unlike NGINX open source (which relies on passive checking), HAProxy includes active health checking as a core feature.

Comprehensive Health Check Configuration

haproxy

# TCP health checks (Layer 4)
backend database_pool
    mode tcp
    balance leastconn
    
    # Simple TCP connect check
    server db1 10.0.1.10:5432 check inter 3s fall 3 rise 2
    server db2 10.0.1.11:5432 check inter 3s fall 3 rise 2
    
    # check: Enable health checking
    # inter: Check interval
    # fall: Consecutive failures before marking DOWN
    # rise: Consecutive successes before marking UP
 
# HTTP health checks (Layer 7)
backend api_servers
    mode http
    balance roundrobin
    
    # Define HTTP health check request
    option httpchk GET /health HTTP/1.1\r\nHost:\ localhost
    
    # Expected response (any of these)
    http-check expect status 200
    # http-check expect string "healthy"
    # http-check expect rstatus ^2[0-9][0-9]$
    
    server api1 10.0.2.10:8080 check inter 5s fall 3 rise 2 slowstart 30s
    server api2 10.0.2.11:8080 check inter 5s fall 3 rise 2 slowstart 30s
    
    # slowstart: Gradually increase load on recovered server over N seconds
 
# Advanced multi-step health checks (HAProxy 2.2+)
backend complex_api
    mode http
    
    # Multi-step check sequence
    option httpchk
    http-check connect
    http-check send meth GET uri /health ver HTTP/1.1 hdr Host localhost
    http-check expect status 200
    http-check connect port 8081
    http-check send meth GET uri /metrics ver HTTP/1.1 hdr Host localhost
    http-check expect status 200
    
    server api1 10.0.3.10:8080 check
 
# Agent health checks (external health agent)
backend agent_checked
    mode http
    
    server app1 10.0.4.10:8080 check agent-check agent-port 8888 agent-inter 5s
    
    # Agent returns: "up", "down", "drain", "maint", or weight percentage
 
# DNS-based health checks for dynamic backends
backend dns_dynamic
    mode http
    balance roundrobin
    
    server-template srv 10 _api._tcp.service.consul:8080 check resolvers consul
    
resolvers consul
    nameserver consul 127.0.0.1:8600
    accepted_payload_size 8192
    hold valid 10s

Health Check Types and Use Cases

•TCP Connect Check — Validates that the port is accepting connections. Fast but doesn't verify application health. Use for database connections, cache servers.
•HTTP Check — Sends HTTP request and validates response status/content. Preferred for web services. Can verify application initialization is complete.
•Multi-Step Check — Executes sequence of checks (connect, request, validate). Use when health requires multiple endpoint verification.
•Agent Check — External process reports health status via dedicated port. Enables complex health logic (disk space, queue depth) outside HAProxy.
•Slow Start — Gradually increases traffic to recovered servers. Prevents cold cache thundering herd and allows JIT compilation to warm up.

Health Check Intervals and Timeouts

Set inter (check interval) based on your acceptable detection time. With inter 3s and fall 3, worst-case detection is 9 seconds. For critical services, use inter 1s fall 2 for 2-second detection. However, aggressive checking increases backend load—balance based on backend capacity and criticality.

Stick Tables: Distributed Session Intelligence

One of HAProxy's most powerful and unique features is stick tables—in-memory key-value stores that track connection metadata, enable session persistence, and power advanced rate limiting. Stick tables are shared across all threads and can be replicated between HAProxy peers for high availability.

Core Capabilities:

Stick Table Use Cases

•Session Persistence — Remember which backend served a client for subsequent requests
•Rate Limiting — Track request counts per client IP, user ID, or API key
•Abuse Detection — Count error rates, detect connection floods
•Connection Tracking — Monitor concurrent connections per client
•Bandwidth Tracking — Track bytes in/out per client for metering

Stick Table Examples

haproxy

# Session persistence via stick table
backend app_sticky
    mode http
    balance roundrobin
    
    # Define stick table: track by source IP, expire after 30 minutes
    stick-table type ip size 1m expire 30m
    
    # Store server assignment
    stick on src
    
    server app1 10.0.1.10:8080 check
    server app2 10.0.1.11:8080 check
    server app3 10.0.1.12:8080 check
 
# Rate limiting with stick tables
frontend http_front
    bind *:80
    mode http
    
    # Track request rate by source IP
    stick-table type ip size 100k expire 1m store http_req_rate(10s),conn_cur,bytes_out_rate(1m)
    
    # Update counters
    http-request track-sc0 src
    
    # Rate limit: max 100 requests per 10 seconds
    acl rate_limit_exceeded sc_http_req_rate(0) gt 100
    
    # Concurrent connection limit
    acl too_many_connections sc_conn_cur(0) gt 50
    
    # Bandwidth limit: 10 MB/minute
    acl bandwidth_exceeded sc_bytes_out_rate(0) gt 10485760
    
    # Apply limits
    http-request deny deny_status 429 if rate_limit_exceeded
    http-request deny deny_status 429 if too_many_connections
    http-request deny deny_status 503 if bandwidth_exceeded
    
    default_backend app_servers
 
# API key rate limiting
frontend api_front
    bind *:443 ssl crt /etc/haproxy/certs/api.pem
    mode http
    
    # Track by API key header
    stick-table type string len 64 size 100k expire 1h store http_req_rate(1m),http_err_rate(1m)
    
    http-request track-sc0 req.hdr(X-API-Key)
    
    # Tiered rate limits (check from strictest to most lenient)
    acl free_tier req.hdr(X-API-Tier) -i free
    acl pro_tier req.hdr(X-API-Tier) -i pro
    
    acl free_limit_exceeded sc_http_req_rate(0) gt 100
    acl pro_limit_exceeded sc_http_req_rate(0) gt 1000
    
    # Block if exceeded for tier
    http-request deny deny_status 429 if free_tier free_limit_exceeded
    http-request deny deny_status 429 if pro_tier pro_limit_exceeded
    
    default_backend api_servers
 
# Stick table replication between peers
peers haproxy_peers
    peer haproxy1 10.0.0.1:10000
    peer haproxy2 10.0.0.2:10000
 
backend app_replicated
    stick-table type ip size 1m expire 30m peers haproxy_peers
    stick on src
    
    server app1 10.0.1.10:8080 check
    server app2 10.0.1.11:8080 check

Stick Table Data Types:

Stick tables can track various metrics per entry:

conn_cnt — Total connections
conn_cur — Current concurrent connections
conn_rate(period) — Connection rate over period
http_req_cnt — Total HTTP requests
http_req_rate(period) — Request rate over period
http_err_cnt — Total HTTP errors (4xx/5xx)
http_err_rate(period) — Error rate over period
bytes_in_cnt / bytes_out_cnt — Total bytes
bytes_in_rate(period) / bytes_out_rate(period) — Bandwidth rate
gpc0, gpc1 — General purpose counters
server_id — Assigned server for persistence

Peer Replication:

The peers directive enables real-time replication of stick table entries between HAProxy instances. This provides session persistence and rate limiting that survives failover between load balancers—a critical capability for high availability deployments.

Memory Sizing

Stick tables consume memory proportional to their size and stored fields. A table with size 1m storing 10 fields per entry might consume 100+ MB of RAM. Monitor actual usage via the stats socket and size appropriately for your traffic patterns.

Runtime API: Dynamic Control Without Reloads

HAProxy's Runtime API (accessed via the stats socket) enables dynamic configuration changes without service interruption. This capability distinguishes HAProxy from NGINX open source, which requires configuration reloads for most changes.

Capabilities via Runtime API:

Runtime API Operations

•Server management — Enable/disable servers, adjust weights, change addresses
•Connection draining — Gracefully drain connections before maintenance
•Rate limit adjustment — Modify stick table entries and thresholds
•Statistics retrieval — Query real-time metrics programmatically
•ACL updates — Add/remove entries from dynamic ACLs and maps
•Session management — Kill specific sessions, clear tables

Runtime API Usage Examples
bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# Connect to HAProxy stats socket
echo "show info" | socat stdio /var/run/haproxy.sock
 
# View all servers in a backend
echo "show servers state" | socat stdio /var/run/haproxy.sock
 
# Disable a server for maintenance
echo "disable server api_servers/api1" | socat stdio /var/run/haproxy.sock
 
# Enable a server
echo "enable server api_servers/api1" | socat stdio /var/run/haproxy.sock
 
# Set server to drain mode (finish existing connections, reject new)
echo "set server api_servers/api1 state drain" | socat stdio /var/run/haproxy.sock
 
# Change server weight dynamically
echo "set server api_servers/api1 weight 50" | socat stdio /var/run/haproxy.sock
 
# Add new server dynamically (requires server-template)
echo "add server api_servers/api4 10.0.1.14:8080 check" | socat stdio /var/run/haproxy.sock
 
# View stick table contents
echo "show table http_front" | socat stdio /var/run/haproxy.sock
 
# Clear rate limit for specific IP
echo "clear table http_front key 192.168.1.100" | socat stdio /var/run/haproxy.sock
 
# View current statistics in CSV format
echo "show stat" | socat stdio /var/run/haproxy.sock
 
# View active sessions
echo "show sess" | socat stdio /var/run/haproxy.sock
 
# Gracefully terminate a specific session
echo "shutdown session 0x7f8a9c001234" | socat stdio /var/run/haproxy.sock

Integration Patterns:

The Runtime API enables powerful automation patterns:

Automated Deployment: CI/CD pipelines can drain servers, deploy code, run health checks, and restore traffic without HAProxy reloads.
Auto-Scaling Integration: Cloud auto-scaling events can dynamically add/remove servers from backends via API calls.
Circuit Breakers: External monitoring can automatically disable backends experiencing high error rates.
Chaos Engineering: Deliberately disable servers during game days to test failover behavior.

dataplaneapi for REST Access

HAProxy's dataplaneapi provides a RESTful interface over the stats socket, enabling integration with orchestration tools, service meshes, and custom applications without socket programming. It's the recommended approach for production automation.

Performance Characteristics

HAProxy's performance is legendary in the industry, routinely achieving metrics that push hardware limits rather than software limits. Understanding these characteristics helps with capacity planning and benchmark interpretation.

HAProxy Performance Reference (Modern Hardware)
Metric	TCP Mode	HTTP Mode	Bottleneck
New sessions/sec	500,000 - 2,000,000+	200,000 - 500,000+	CPU (connection setup)
Concurrent connections	1,000,000 - 10,000,000+	500,000 - 2,000,000+	Memory, file descriptors
Throughput	100+ Gbps	50+ Gbps	Network interface, CPU
Latency overhead	< 50μs	< 100μs	Processing complexity
Memory per connection	~1 KB (idle)	~2-4 KB (with buffers)	Increases with SSL, buffers

Performance Comparison with NGINX:

In head-to-head benchmarks, HAProxy typically demonstrates:

10-30% higher throughput in TCP mode proxying
Similar performance in HTTP mode with simple routing
Significantly lower memory per connection
Lower latency in Layer 4 scenarios
Better performance under connection-heavy workloads

However, NGINX may outperform when serving static content directly (which HAProxy doesn't do), or when heavy response transformation is required.

Why HAProxy Is Faster:

Performance Optimization Techniques

•Zero-copy data transfer — Data moves directly between sockets without user-space copying where possible
•Memory pooling — Pre-allocated memory pools eliminate malloc/free overhead during request processing
•Lock-free data structures — Thread communication uses atomic operations, not mutexes
•Optimized HTTP parsing — Hand-tuned HTTP parser optimized for common request patterns
•Minimal abstraction layers — Direct system call usage, fewer indirection layers than general-purpose software
•Splice for large transfers — Uses kernel splice() for large HTTP bodies, bypassing user space entirely

Benchmark Reality

Published benchmarks often test synthetic scenarios that exaggerate differences. In real-world deployments, both HAProxy and NGINX perform adequately for most use cases. Choose based on features, operational requirements, and team expertise rather than marginal benchmark differences.

When to Choose HAProxy

HAProxy excels in specific scenarios where its unique capabilities provide decisive advantages.

Choose HAProxy When

•Maximum Layer 4 performance is critical (database proxying, real-time systems)
•You need advanced rate limiting with stick tables
•Runtime API for dynamic configuration is required
•High-availability with session replication is needed
•Detailed traffic metrics and logging are important
•TCP load balancing for non-HTTP protocols (Redis, PostgreSQL, etc.)
•You need active health checking in open-source software

Consider Alternatives When

•You need to serve static files directly (use NGINX)
•Complex Lua scripting or modules are required (use NGINX/OpenResty)
•You want integrated web serving + proxying (use NGINX)
•Cloud-native features like xDS configuration (use Envoy)
•gRPC with advanced traffic management (use Envoy)
•You prefer fully managed solutions (use cloud LBs)

Summary:

HAProxy represents the pinnacle of purpose-built load balancing software. Its singular focus on proxying—without the distractions of web serving, content caching, or module ecosystems—yields exceptional performance and reliability.

The stick table infrastructure, runtime API, and comprehensive health checking make HAProxy the preferred choice for:

High-traffic platforms requiring maximum efficiency
Infrastructure teams with strong operational practices
Scenarios requiring advanced session persistence or rate limiting
Environments where every microsecond of latency matters

In the next page, we'll explore Envoy—the modern, cloud-native proxy that reimagines load balancing for the service mesh era.

Page Complete

You now possess comprehensive knowledge of HAProxy as a load balancing solution—from its multi-threaded architecture to stick tables, runtime API, and performance characteristics. Next, we'll explore Envoy and its unique approach to modern, cloud-native traffic management.

HAProxy — High Performance, TCP/HTTP

The Gold Standard for High-Performance Proxying

Learning Objectives

Architecture Deep Dive: Multi-Threaded Event Processing

Historical Evolution:

Pre-1.8 (single-process): Like early NGINX, HAProxy ran a single event loop. Performance scaled by running multiple independent processes, each binding to shared ports via SO_REUSEPORT.
HAProxy 1.8+ (multi-threaded): Introduced native multi-threading within a single process, enabling shared memory for stick tables, counters, and connection state. This is the modern, recommended deployment model.

The multi-threaded architecture provides significant advantages:

Multi-Threading Advantages

•Shared State — Stick tables, rate limiting counters, and session data are shared across all threads without external synchronization
•Memory Efficiency — Single process memory footprint versus multiple isolated processes
•Atomic Operations — Configuration reloads can preserve existing connections without disruption
•Simpler Operations — Single PID to manage, cleaner systemd integration, unified logging
•Connection Locality — Threads can be pinned to CPU cores for cache efficiency

The Event-Driven Core:

New connection arrives → allocate session structure
State machine initialized → waiting for client data
Data received → parse protocol (TCP/HTTP), evaluate ACLs
Routing decision made → connect to backend or serve cached response
Bidirectional data relay → optimized memory management
Connection close → cleanup, logging, statistics update

This state machine model enables HAProxy to handle connections at any protocol stage efficiently, including incomplete HTTP requests, slow clients, and long-lived connections.

Thread Configuration

haproxy

global
    # Thread configuration
    nbthread 4                      # Number of threads (typically = CPU cores)
    cpu-map auto:1/1-4 0-3          # Pin threads 1-4 to CPU cores 0-3
    
    # Process and privilege management
    user haproxy
    group haproxy
    chroot /var/lib/haproxy
    
    # Performance tuning
    maxconn 100000                  # Global maximum connections
    tune.ssl.default-dh-param 2048
    tune.bufsize 16384              # Buffer size per connection
    tune.maxrewrite 1024
    
    # Logging
    log /dev/log local0
    log /dev/log local1 notice
    
    # Stats socket for runtime API
    stats socket /var/run/haproxy.sock mode 660 level admin expose-fd listeners
    stats timeout 30s

CPU Pinning for Maximum Performance

Configuration Paradigm: Frontends, Backends, and ACLs

HAProxy Load Balancing Configuration

haproxy

# Global settings (shown previously)
global
    nbthread 4
    maxconn 100000
    log stdout format raw local0
 
# Default settings applied to all frontends/backends
defaults
    mode http                       # HTTP mode (vs tcp)
    log global
    option httplog                  # Detailed HTTP logging
    option dontlognull              # Don't log health checks
    option http-server-close        # Close server connection after response
    option forwardfor               # Add X-Forwarded-For header
    option redispatch               # Retry on different server if connection fails
    retries 3                       # Retry count before giving up
    
    timeout connect 5s              # Time to establish backend connection
    timeout client 50s              # Max inactivity on client side
    timeout server 50s              # Max inactivity on server side
    timeout http-request 10s        # Max time for complete HTTP request
    timeout http-keep-alive 10s     # Max idle time for keep-alive
    timeout queue 30s               # Max time waiting in queue
 
# Frontend: Entry point for client connections
frontend http_front
    bind *:80
    bind *:443 ssl crt /etc/haproxy/certs/combined.pem alpn h2,http/1.1
    
    # Redirect HTTP to HTTPS
    http-request redirect scheme https unless { ssl_fc }
    
    # ACL definitions for routing
    acl is_api path_beg /api/
    acl is_static path_beg /static/ /assets/ /images/
    acl is_websocket hdr(Upgrade) -i websocket
    acl is_health path /health /ready /live
    
    # Route based on ACLs
    use_backend api_servers if is_api
    use_backend static_servers if is_static
    use_backend ws_servers if is_websocket
    use_backend health_servers if is_health
    default_backend app_servers
 
# Backend: Pool of servers to handle requests
backend api_servers
    balance roundrobin
    option httpchk GET /health HTTP/1.1\r\nHost:\ api.internal
    http-check expect status 200
    
    server api1 10.0.1.10:8080 check inter 5s fall 3 rise 2 weight 100
    server api2 10.0.1.11:8080 check inter 5s fall 3 rise 2 weight 100
    server api3 10.0.1.12:8080 check inter 5s fall 3 rise 2 weight 100
    server api4 10.0.1.13:8080 check inter 5s fall 3 rise 2 backup
 
backend app_servers
    balance leastconn
    option httpchk GET /health
    
    cookie SERVERID insert indirect nocache
    
    server app1 10.0.2.10:8080 check cookie s1
    server app2 10.0.2.11:8080 check cookie s2
    server app3 10.0.2.12:8080 check cookie s3
 
backend static_servers
    balance uri                     # Route based on URI hash for cache efficiency
    hash-type consistent
    
    server static1 10.0.3.10:80 check
    server static2 10.0.3.11:80 check
 
backend ws_servers
    balance source                  # Stick WebSocket connections by IP
    
    timeout tunnel 1h              # Long timeout for WebSocket
    
    server ws1 10.0.4.10:8080 check
    server ws2 10.0.4.11:8080 check

Configuration Analysis:

defaults section: Establishes baseline behavior inherited by all frontends and backends. Override individual settings as needed.
bind with SSL: HAProxy terminates TLS with excellent performance. The alpn parameter enables HTTP/2 negotiation.
ACL composition: ACLs can reference headers (hdr), paths (path_beg, path_end, path_reg), cookies, IP addresses, and more. Multiple conditions combine with implicit AND; use or for explicit OR.
use_backend: Directives evaluated in order; first match wins. default_backend handles unmatched requests.
Health checking: option httpchk sends HTTP requests rather than just TCP connection checks. The fall and rise parameters control failure/recovery thresholds.
Cookie insertion: cookie SERVERID insert adds a session cookie for server affinity, enabling stateful application support.

Mode TCP vs HTTP

Load Balancing Algorithms

HAProxy Load Balancing Algorithms
Algorithm	Keyword	Behavior	Optimal Use Case
Round Robin	roundrobin	Cycles through servers sequentially, respecting weights	General HTTP workloads, homogeneous backends
Static Round Robin	static-rr	Round robin without dynamic weight adjustment	When weight stability is critical
Least Connections	leastconn	Routes to server with fewest active connections	Long-lived connections, variable request duration
Source IP Hash	source	Hashes client IP for consistent routing	Session persistence without cookies
URI Hash	uri	Hashes request URI for consistent routing	Caching proxies, CDN backends
URL Parameter	url_param	Hashes specific URL parameter	Application-controlled routing (e.g., ?user_id=)
Header Hash	hdr(name)	Hashes specific header value	Route by User-Agent, API version, etc.
Random	random	Selects random server (optionally weighted)	Large pools, avoiding hotspots
First Available	first	Uses first available server up to maxconn	Cost optimization (fill servers before scaling)

Algorithm Examples

haproxy

# Weighted Least Connections — ideal for heterogeneous backends
backend api_weighted
    balance leastconn
    server api-large 10.0.1.10:8080 weight 200    # 4-core machine
    server api-small1 10.0.1.11:8080 weight 100   # 2-core machines
    server api-small2 10.0.1.12:8080 weight 100
 
# URI-based consistent hashing for cache distribution
backend cache_cluster
    balance uri
    hash-type consistent          # Minimize rehashing on server changes
    hash-balance-factor 150       # Allow 150% of average load per server
    
    server cache1 10.0.2.10:11211 check
    server cache2 10.0.2.11:11211 check
    server cache3 10.0.2.12:11211 check
 
# URL parameter-based routing for user sharding
backend user_shards
    balance url_param user_id check_post
    hash-type consistent
    
    server shard1 10.0.3.10:8080 check
    server shard2 10.0.3.11:8080 check
    server shard3 10.0.3.12:8080 check
    server shard4 10.0.3.13:8080 check
 
# First-available for cost optimization (fill before scale)
backend cost_optimized
    balance first
    
    server primary 10.0.4.10:8080 maxconn 1000 check
    server secondary 10.0.4.11:8080 maxconn 1000 check
    server tertiary 10.0.4.12:8080 maxconn 1000 check
 
# Random with power-of-two choices (HAProxy 2.4+)
backend random_p2c
    balance random(2)             # Pick 2 random, choose least loaded
    
    server s1 10.0.5.10:8080 check
    server s2 10.0.5.11:8080 check
    server s3 10.0.5.12:8080 check
    server s4 10.0.5.13:8080 check

Consistent Hashing Deep Dive:

Algorithm-Server Count Interaction

Advanced Health Checking

Comprehensive Health Check Configuration

haproxy

# TCP health checks (Layer 4)
backend database_pool
    mode tcp
    balance leastconn
    
    # Simple TCP connect check
    server db1 10.0.1.10:5432 check inter 3s fall 3 rise 2
    server db2 10.0.1.11:5432 check inter 3s fall 3 rise 2
    
    # check: Enable health checking
    # inter: Check interval
    # fall: Consecutive failures before marking DOWN
    # rise: Consecutive successes before marking UP
 
# HTTP health checks (Layer 7)
backend api_servers
    mode http
    balance roundrobin
    
    # Define HTTP health check request
    option httpchk GET /health HTTP/1.1\r\nHost:\ localhost
    
    # Expected response (any of these)
    http-check expect status 200
    # http-check expect string "healthy"
    # http-check expect rstatus ^2[0-9][0-9]$
    
    server api1 10.0.2.10:8080 check inter 5s fall 3 rise 2 slowstart 30s
    server api2 10.0.2.11:8080 check inter 5s fall 3 rise 2 slowstart 30s
    
    # slowstart: Gradually increase load on recovered server over N seconds
 
# Advanced multi-step health checks (HAProxy 2.2+)
backend complex_api
    mode http
    
    # Multi-step check sequence
    option httpchk
    http-check connect
    http-check send meth GET uri /health ver HTTP/1.1 hdr Host localhost
    http-check expect status 200
    http-check connect port 8081
    http-check send meth GET uri /metrics ver HTTP/1.1 hdr Host localhost
    http-check expect status 200
    
    server api1 10.0.3.10:8080 check
 
# Agent health checks (external health agent)
backend agent_checked
    mode http
    
    server app1 10.0.4.10:8080 check agent-check agent-port 8888 agent-inter 5s
    
    # Agent returns: "up", "down", "drain", "maint", or weight percentage
 
# DNS-based health checks for dynamic backends
backend dns_dynamic
    mode http
    balance roundrobin
    
    server-template srv 10 _api._tcp.service.consul:8080 check resolvers consul
    
resolvers consul
    nameserver consul 127.0.0.1:8600
    accepted_payload_size 8192
    hold valid 10s

Health Check Types and Use Cases

•TCP Connect Check — Validates that the port is accepting connections. Fast but doesn't verify application health. Use for database connections, cache servers.
•HTTP Check — Sends HTTP request and validates response status/content. Preferred for web services. Can verify application initialization is complete.
•Multi-Step Check — Executes sequence of checks (connect, request, validate). Use when health requires multiple endpoint verification.
•Agent Check — External process reports health status via dedicated port. Enables complex health logic (disk space, queue depth) outside HAProxy.
•Slow Start — Gradually increases traffic to recovered servers. Prevents cold cache thundering herd and allows JIT compilation to warm up.

Health Check Intervals and Timeouts

Stick Tables: Distributed Session Intelligence

Core Capabilities:

Stick Table Use Cases

•Session Persistence — Remember which backend served a client for subsequent requests
•Rate Limiting — Track request counts per client IP, user ID, or API key
•Abuse Detection — Count error rates, detect connection floods
•Connection Tracking — Monitor concurrent connections per client
•Bandwidth Tracking — Track bytes in/out per client for metering

Stick Table Examples

haproxy

# Session persistence via stick table
backend app_sticky
    mode http
    balance roundrobin
    
    # Define stick table: track by source IP, expire after 30 minutes
    stick-table type ip size 1m expire 30m
    
    # Store server assignment
    stick on src
    
    server app1 10.0.1.10:8080 check
    server app2 10.0.1.11:8080 check
    server app3 10.0.1.12:8080 check
 
# Rate limiting with stick tables
frontend http_front
    bind *:80
    mode http
    
    # Track request rate by source IP
    stick-table type ip size 100k expire 1m store http_req_rate(10s),conn_cur,bytes_out_rate(1m)
    
    # Update counters
    http-request track-sc0 src
    
    # Rate limit: max 100 requests per 10 seconds
    acl rate_limit_exceeded sc_http_req_rate(0) gt 100
    
    # Concurrent connection limit
    acl too_many_connections sc_conn_cur(0) gt 50
    
    # Bandwidth limit: 10 MB/minute
    acl bandwidth_exceeded sc_bytes_out_rate(0) gt 10485760
    
    # Apply limits
    http-request deny deny_status 429 if rate_limit_exceeded
    http-request deny deny_status 429 if too_many_connections
    http-request deny deny_status 503 if bandwidth_exceeded
    
    default_backend app_servers
 
# API key rate limiting
frontend api_front
    bind *:443 ssl crt /etc/haproxy/certs/api.pem
    mode http
    
    # Track by API key header
    stick-table type string len 64 size 100k expire 1h store http_req_rate(1m),http_err_rate(1m)
    
    http-request track-sc0 req.hdr(X-API-Key)
    
    # Tiered rate limits (check from strictest to most lenient)
    acl free_tier req.hdr(X-API-Tier) -i free
    acl pro_tier req.hdr(X-API-Tier) -i pro
    
    acl free_limit_exceeded sc_http_req_rate(0) gt 100
    acl pro_limit_exceeded sc_http_req_rate(0) gt 1000
    
    # Block if exceeded for tier
    http-request deny deny_status 429 if free_tier free_limit_exceeded
    http-request deny deny_status 429 if pro_tier pro_limit_exceeded
    
    default_backend api_servers
 
# Stick table replication between peers
peers haproxy_peers
    peer haproxy1 10.0.0.1:10000
    peer haproxy2 10.0.0.2:10000
 
backend app_replicated
    stick-table type ip size 1m expire 30m peers haproxy_peers
    stick on src
    
    server app1 10.0.1.10:8080 check
    server app2 10.0.1.11:8080 check

Stick Table Data Types:

Stick tables can track various metrics per entry:

conn_cnt — Total connections
conn_cur — Current concurrent connections
conn_rate(period) — Connection rate over period
http_req_cnt — Total HTTP requests
http_req_rate(period) — Request rate over period
http_err_cnt — Total HTTP errors (4xx/5xx)
http_err_rate(period) — Error rate over period
bytes_in_cnt / bytes_out_cnt — Total bytes
bytes_in_rate(period) / bytes_out_rate(period) — Bandwidth rate
gpc0, gpc1 — General purpose counters
server_id — Assigned server for persistence

Peer Replication:

Memory Sizing

Runtime API: Dynamic Control Without Reloads

Capabilities via Runtime API:

Runtime API Operations

•Server management — Enable/disable servers, adjust weights, change addresses
•Connection draining — Gracefully drain connections before maintenance
•Rate limit adjustment — Modify stick table entries and thresholds
•Statistics retrieval — Query real-time metrics programmatically
•ACL updates — Add/remove entries from dynamic ACLs and maps
•Session management — Kill specific sessions, clear tables

Runtime API Usage Examples
bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
# Connect to HAProxy stats socket
echo "show info" | socat stdio /var/run/haproxy.sock
 
# View all servers in a backend
echo "show servers state" | socat stdio /var/run/haproxy.sock
 
# Disable a server for maintenance
echo "disable server api_servers/api1" | socat stdio /var/run/haproxy.sock
 
# Enable a server
echo "enable server api_servers/api1" | socat stdio /var/run/haproxy.sock
 
# Set server to drain mode (finish existing connections, reject new)
echo "set server api_servers/api1 state drain" | socat stdio /var/run/haproxy.sock
 
# Change server weight dynamically
echo "set server api_servers/api1 weight 50" | socat stdio /var/run/haproxy.sock
 
# Add new server dynamically (requires server-template)
echo "add server api_servers/api4 10.0.1.14:8080 check" | socat stdio /var/run/haproxy.sock
 
# View stick table contents
echo "show table http_front" | socat stdio /var/run/haproxy.sock
 
# Clear rate limit for specific IP
echo "clear table http_front key 192.168.1.100" | socat stdio /var/run/haproxy.sock
 
# View current statistics in CSV format
echo "show stat" | socat stdio /var/run/haproxy.sock
 
# View active sessions
echo "show sess" | socat stdio /var/run/haproxy.sock
 
# Gracefully terminate a specific session
echo "shutdown session 0x7f8a9c001234" | socat stdio /var/run/haproxy.sock

Integration Patterns:

The Runtime API enables powerful automation patterns:

Automated Deployment: CI/CD pipelines can drain servers, deploy code, run health checks, and restore traffic without HAProxy reloads.
Auto-Scaling Integration: Cloud auto-scaling events can dynamically add/remove servers from backends via API calls.
Circuit Breakers: External monitoring can automatically disable backends experiencing high error rates.
Chaos Engineering: Deliberately disable servers during game days to test failover behavior.

dataplaneapi for REST Access

Performance Characteristics

HAProxy Performance Reference (Modern Hardware)
Metric	TCP Mode	HTTP Mode	Bottleneck
New sessions/sec	500,000 - 2,000,000+	200,000 - 500,000+	CPU (connection setup)
Concurrent connections	1,000,000 - 10,000,000+	500,000 - 2,000,000+	Memory, file descriptors
Throughput	100+ Gbps	50+ Gbps	Network interface, CPU
Latency overhead	< 50μs	< 100μs	Processing complexity
Memory per connection	~1 KB (idle)	~2-4 KB (with buffers)	Increases with SSL, buffers

Performance Comparison with NGINX:

In head-to-head benchmarks, HAProxy typically demonstrates:

10-30% higher throughput in TCP mode proxying
Similar performance in HTTP mode with simple routing
Significantly lower memory per connection
Lower latency in Layer 4 scenarios
Better performance under connection-heavy workloads

However, NGINX may outperform when serving static content directly (which HAProxy doesn't do), or when heavy response transformation is required.

Why HAProxy Is Faster:

Performance Optimization Techniques

•Zero-copy data transfer — Data moves directly between sockets without user-space copying where possible
•Memory pooling — Pre-allocated memory pools eliminate malloc/free overhead during request processing
•Lock-free data structures — Thread communication uses atomic operations, not mutexes
•Optimized HTTP parsing — Hand-tuned HTTP parser optimized for common request patterns
•Minimal abstraction layers — Direct system call usage, fewer indirection layers than general-purpose software
•Splice for large transfers — Uses kernel splice() for large HTTP bodies, bypassing user space entirely

Benchmark Reality

When to Choose HAProxy

HAProxy excels in specific scenarios where its unique capabilities provide decisive advantages.

Choose HAProxy When

•Maximum Layer 4 performance is critical (database proxying, real-time systems)
•You need advanced rate limiting with stick tables
•Runtime API for dynamic configuration is required
•High-availability with session replication is needed
•Detailed traffic metrics and logging are important
•TCP load balancing for non-HTTP protocols (Redis, PostgreSQL, etc.)
•You need active health checking in open-source software

Consider Alternatives When

•You need to serve static files directly (use NGINX)
•Complex Lua scripting or modules are required (use NGINX/OpenResty)
•You want integrated web serving + proxying (use NGINX)
•Cloud-native features like xDS configuration (use Envoy)
•gRPC with advanced traffic management (use Envoy)
•You prefer fully managed solutions (use cloud LBs)

Summary:

The stick table infrastructure, runtime API, and comprehensive health checking make HAProxy the preferred choice for:

High-traffic platforms requiring maximum efficiency
Infrastructure teams with strong operational practices
Scenarios requiring advanced session persistence or rate limiting
Environments where every microsecond of latency matters

In the next page, we'll explore Envoy—the modern, cloud-native proxy that reimagines load balancing for the service mesh era.

Page Complete