101
0/228
Loading content...
If datacenter architecture is the skeleton and redundancy the immune system, then traffic patterns are the circulatory system—the flows of data that animate the entire infrastructure. Every design decision, from topology to capacity planning, ultimately serves to move data efficiently from where it originates to where it's needed.
But not all traffic is equal. Data moves in distinct patterns, each with different characteristics, requirements, and challenges:
Understanding these patterns is essential because the network that handles north-south traffic efficiently may be entirely wrong for east-west patterns, and vice versa. The revolution in datacenter networking over the past decade—the shift from three-tier to leaf-spine—was driven precisely by a fundamental change in traffic patterns: the explosion of east-west communication in distributed systems.
By the end of this page, you will understand the fundamental traffic patterns in modern datacenters, how application architectures shape traffic characteristics, the tools and techniques for traffic analysis and engineering, and how traffic considerations influence network design decisions.
North-South traffic refers to data flows that cross the datacenter boundary—traffic entering from or exiting to external networks. This includes user requests from the internet, API calls to external services, connections to partner networks, and WAN traffic to other datacenter locations.
Ingress (Client → Datacenter):
Egress (Datacenter → Client):
North-south traffic traverses multiple network layers:
Diurnal patterns: User traffic follows human activity cycles—peaks during business hours (for enterprise apps) or evenings (for consumer media). Understanding these patterns enables:
Geographic patterns: Traffic volume varies by region based on:
Session behavior: North-south traffic is typically session-oriented:
Traditional web applications had highly asymmetric traffic: small ingress requests, large egress responses. But modern applications—video calls, file uploads, collaborative editing—are increasingly symmetric or even ingress-heavy. Network designs must accommodate the actual ratio, not assumptions based on legacy patterns.
East-West traffic refers to data flows between servers within the datacenter. While invisible to end users, this internal traffic often constitutes 70-80% of total datacenter bandwidth in modern distributed systems.
Distributed computing:
Storage operations:
Cluster operations:
ML/AI workloads:
Volume: Typically 3-10x greater than north-south traffic in modern deployments
Latency sensitivity: Internal services often have tight latency budgets
Traffic patterns:
Connection patterns:
Traditional three-tier networks were designed for north-south traffic: few uplinks from access to aggregation, and fewer still to core. When east-west traffic exploded, these networks choked. Two servers in adjacent racks might have megabits of bandwidth between them if traffic had to traverse congested aggregation layers. Leaf-spine networking exists specifically because east-west traffic demanded consistent, high-bandwidth, low-latency connectivity between all servers.
Different application architectures generate dramatically different traffic patterns. Understanding these patterns is essential for network capacity planning and optimization.
Traffic characteristics:
Traffic characteristics:
Traffic amplification: A single user request may multiply internally:
Traffic characteristics:
Example MapReduce shuffle:
Traffic characteristics:
Traffic characteristics:
| Application Type | N-S Ratio | E-W Ratio | Key Pattern | Network Sensitivity |
|---|---|---|---|---|
| Traditional Web | High | Low | Client-server | Moderate |
| Microservices | Low | High | Many fan-out, fan-in | High (latency) |
| MapReduce/Spark | Low | Very High | All-to-all shuffle | Very High (bandwidth) |
| Distributed Storage | Moderate | High | Replication, repair | High (bandwidth) |
| ML Training | Low | Very High | All-reduce | Extreme (latency + BW) |
| CDN Origin | High | Low | Large object serving | Moderate (throughput) |
Network capacity requirements should be derived from actual or projected application traffic patterns, not industry averages. Profile your workloads: measure actual traffic volumes, latency requirements, and communication patterns. A network optimized for microservices may underperform for ML training, and vice versa.
You can't optimize what you can't measure. Traffic analysis provides the visibility needed to understand patterns, plan capacity, and troubleshoot problems.
SNMP Counters:
Flow Telemetry (NetFlow, sFlow, IPFIX):
Deep Packet Inspection (DPI):
Streaming Telemetry:
Bandwidth utilization:
Traffic matrix:
Flow characteristics:
Latency measurements:
As TLS encryption becomes ubiquitous (including for east-west traffic in zero-trust architectures), deep packet inspection becomes impossible. Traffic analysis must rely on metadata (IP addresses, ports, flow sizes, timing) rather than content. Encrypted traffic analysis using machine learning is an emerging field addressing this challenge.
Traffic engineering (TE) is the science of optimizing how traffic flows through the network. While ECMP provides automatic load distribution, sophisticated environments use explicit traffic engineering to improve performance beyond what default routing achieves.
ECMP (Equal-Cost Multi-Path):
Weighted ECMP (WCMP):
Segment Routing (SR) / SR-MPLS / SRv6:
Distributed TE (traditional):
Centralized TE (SDN-enabled):
Leaf-spine networks provide natural TE through ECMP, but optimizations include:
Flow scheduling:
Workload placement:
Application-aware routing:
Advanced traffic engineering adds operational complexity. Simple ECMP may be 90% optimal with 10% of the complexity of explicit TE. Implement advanced TE only when measurement shows clear bottlenecks that simpler approaches can't address. The best traffic engineering is often better topology design that doesn't need explicit path control.
Not all congestion is visible at the scale of minutes or seconds. Micro-bursts—short traffic spikes lasting microseconds to milliseconds—can cause packet drops and latency spikes even when average utilization is low.
Cause: Network links have finite capacity, but traffic arrives in bursts. When multiple flows simultaneously send data:
Why averages lie:
Definition: Incast occurs when many sources simultaneously send data to one destination, overwhelming the receiver's incoming link or switch buffer.
Common incast scenarios:
Impact:
Hardware approaches:
Protocol approaches:
Application approaches:
TCP's default timeout (200ms-1s) is catastrophic for datacenter incast. When packets drop during incast, affected flows timeout and retransmit. By then, the burst is over and the network is idle—but the application waited a timeout period for the retransmit. This 'incast collapse' can make applications thousands of times slower than expected. DCTCP and microsecond-scale timeouts address this.
Traffic patterns directly inform network design decisions. Understanding your traffic characteristics guides choices in topology, capacity, placement, and features.
High east-west traffic →
Latency-sensitive traffic →
Large flow (elephant) dominated →
Bursty/synchronized traffic →
Beyond network design, placing workloads intelligently reduces traffic:
Rack-local placement:
Pod-level affinity:
Anti-affinity for redundancy:
Topology-aware scheduling:
Optimize traffic locality in priority order: (1) Same server - memory speed, (2) Same rack - microseconds, low cost, (3) Same pod - sub-milliseconds, moderate cost, (4) Same DC - milliseconds, higher cost, (5) Different DC - tens of milliseconds, highest cost. Every tier up in this hierarchy reduces latency and increases available bandwidth.
Understanding traffic patterns is fundamental to datacenter networking. We've explored the characteristics of north-south and east-west traffic, how applications shape traffic demands, measurement and analysis techniques, and how traffic considerations drive network design and optimization.
Module Complete:
Congratulations! You've completed Module 1: Datacenter Overview. You now understand the comprehensive foundation of modern datacenter networking—from physical architecture through topology, scalability, redundancy, and traffic patterns. This knowledge prepares you for deeper exploration of cloud networking, virtualization, load balancing, and the advanced topics covered in the remaining modules of this chapter.
You now understand datacenter traffic patterns comprehensively—the fundamental flows that animate datacenter infrastructure. Combined with the architecture, topology, scalability, and redundancy knowledge from earlier pages, you have a complete picture of how modern datacenters work. The next module explores cloud networking in detail.