Enterprise Network Design

When a user in the Tokyo office opens a document stored on a server in New York, their request traverses a complex web of carrier networks, submarine cables, and routing infrastructure spanning 10,000+ kilometers. When that file loads in seconds rather than minutes, it's because of deliberate Wide Area Network (WAN) design decisions made by network architects who understood how to optimize connectivity across vast distances.

WAN design is the discipline of connecting geographically distributed sites—branch offices, datacenters, cloud providers, and partners—into a cohesive enterprise network. Unlike campus networks where you control every cable and switch, WAN design involves navigating carrier services, managing costs that can exceed millions annually, and engineering solutions that perform reliably across infrastructure you don't own.

A Wide Area Network (WAN) connects geographically dispersed locations using telecommunications carrier infrastructure. Unlike Local Area Networks (LANs) where organizations own and control the physical infrastructure, WANs traverse carrier networks—introducing dependencies on external providers, service level agreements, and fundamentally different economics.

WAN vs. LAN: Key Differences:

WAN Design Objectives:

Effective WAN design balances multiple competing objectives:

1. Performance: Minimize latency, maximize available bandwidth, and ensure consistent application response times.

2. Reliability: Provide redundant paths to eliminate outages from single component failures. Target 99.9-99.99% availability.

3. Security: Protect data in transit across untrusted carrier infrastructure through encryption and access control.

4. Cost Efficiency: Optimize bandwidth investments; WAN costs often exceed $1M annually for large enterprises.

5. Scalability: Support business growth without architectural redesign. Add sites, increase bandwidth, extend to cloud.

6. Manageability: Enable centralized visibility and control despite distributed infrastructure.

7. Agility: Provision and modify connectivity quickly to support business changes.

The relative priority of these objectives varies by organization. A financial trading firm prioritizes performance and reliability above cost. A retail chain with 5,000 stores may prioritize cost efficiency and scalability.

WAN topology determines how sites interconnect and how traffic flows between locations. The choice profoundly impacts performance, cost, resilience, and operational complexity.

1. Hub-and-Spoke (Star) Topology

All remote sites (spokes) connect to a central hub (typically headquarters or primary datacenter). Traffic between spokes traverses the hub.

2. Full Mesh Topology

Every site has direct connectivity to every other site. Provides optimal path selection and maximum resilience.

Characteristics:

• Number of connections: n(n-1)/2 for n sites
• Example: 10 sites = 45 connections; 50 sites = 1,225 connections
• Optimal path selection: direct route between any two sites
• Maximum resilience: multiple paths survive any single failure
• Very high cost and complexity beyond 10-15 sites

3. Partial Mesh Topology

Strategic subset of sites have direct connectivity based on traffic patterns and business requirements. Balance between hub-and-spoke simplicity and full mesh optimization.

Design Approach:

• Analyze traffic patterns: which sites communicate most frequently/heavily
• Connect high-traffic site pairs directly
• Regional hubs with local full mesh; inter-regional via hub-to-hub
• Accept sub-optimal paths for low-traffic site pairs

4. Hierarchical/Regional Hub Topology

Large enterprises often adopt multi-tier WANs with regional aggregation:

• Tier 1: Global backbone connecting primary datacenters and HQ
• Tier 2: Regional hubs (EMEA, APAC, Americas) aggregating local sites
• Tier 3: Local sites connect to nearest regional hub

This hierarchy limits inter-continental traffic, keeps latency-sensitive applications regional, and enables localized management while maintaining global connectivity.

Enterprises purchase WAN connectivity from telecommunications carriers (AT&T, Verizon, BT, NTT, etc.). Understanding carrier service offerings is essential for procurement decisions:

Layer 1 Services: Physical Transport

Layer 2 Services: Ethernet and Beyond

Layer 3 Services: Managed Routing

MPLS VPN (IP VPN)

The dominant managed WAN service for enterprises. Carrier manages routing within their MPLS backbone; customer receives IP connectivity.

Three MPLS VPN Classes:

1. Class of Service (CoS) with SLA: Carrier guarantees latency, jitter, and packet loss for different traffic classes. Essential for real-time applications (voice, video).

2. Standard: Best-effort forwarding within carrier network. Lower cost but no performance guarantees.

3. Burstable: Committed Information Rate (CIR) with ability to burst above during low utilization. Pay base rate plus overages.

Typical MPLS VPN SLAs:

• Latency: ≤50ms domestic, ≤100-200ms continental, ≤300ms global
• Jitter: ≤10ms
• Packet Loss: ≤0.1%
• Availability: 99.9% (8.76 hours/year downtime) to 99.99% (52 minutes/year)

Internet as WAN Transport

The public internet, once considered unsuitable for enterprise WAN, is increasingly viable as primary or hybrid transport:

Internet Advantages:

• Cost: 5-10x cheaper per Mbps than MPLS
• Availability: Ubiquitous, even in developing regions
• Bandwidth: Can exceed affordable MPLS bandwidth
• Provisioning: Days vs. weeks for carrier services
• Cloud optimization: Direct paths to cloud/SaaS providers

Internet Challenges:

• No SLAs: Performance varies with congestion, peering, and routing changes
• Security: Must encrypt all traffic; exposed to DDoS attacks
• Latency variability: Routing changes can cause path changes
• Last-mile diversity: Business internet often shares physical plant with MPLS

Hybrid WAN Strategy: Most organizations now deploy hybrid WAN—combining MPLS (for critical, real-time applications) with internet (for cloud access, bandwidth-heavy transfers, backup). SD-WAN orchestrates traffic steering between these transports.

Routing protocol selection and design profoundly impact WAN behavior—convergence time, path selection, scalability, and troubleshooting complexity. WAN routing must address challenges absent from campus environments: variable link quality, asymmetric paths, and scale across potentially thousands of sites.

Routing Protocol Options for Enterprise WAN:

OSPF WAN Design Considerations:

OSPF is commonly used for enterprise WANs, particularly over MPLS VPN where it interfaces with carrier-provided PE-CE routing.

Key Design Principles:

1. Area Design: Place all sites in Area 0 for simple deployments (< 100 sites). For larger WANs, use multiple areas with hub sites as ABRs.

2. Network Types:

   • MPLS VPN: Point-to-multipoint → OSPF Point-to-Point network type (avoids DR/BDR election)
   • Broadcast Ethernet: Use Point-to-Point subinterfaces or carefully manage DR election

3. Summarization: Summarize at area boundaries to reduce LSA propagation and routing table size.

4. Stub Areas: Configure stub or totally-stubby areas for branch sites that don't need full routing table.

5. Timer Tuning: Default OSPF Hello (10s) / Dead (40s) timers may be too slow. Consider 1s/4s for fast convergence, balanced against stability.

BGP for WAN Design:

BGP, while traditionally an internet routing protocol, increasingly appears in enterprise WANs:

When to Use BGP:

• Multi-homed internet connectivity (mandatory for receiving provider routes)
• MPLS VPN PE-CE routing (common carrier requirement)
• Large-scale WAN (BGP scales better than IGPs for 500+ sites)
• SD-WAN overlays (many SD-WAN platforms use BGP internally)
• Multi-datacenter environment requiring traffic engineering

BGP WAN Design Principles:

• Use iBGP within the enterprise, eBGP toward carriers
• Implement route reflectors for iBGP scalability (avoid full mesh)
• Apply consistent BGP communities for policy (traffic engineering, backup paths)
• Tune BGP timers for faster convergence (BFD recommended)
• Filter routes carefully—don't accept/advertise unintended prefixes

WAN links are expensive and constrained compared to LAN. Performance optimization techniques extract maximum value from WAN investments and ensure acceptable application experience despite distance and bandwidth limitations.

Quality of Service (QoS)

QoS ensures critical applications receive priority under congestion. Without QoS, a large file transfer can starve a voice call, causing unacceptable quality.

QoS Building Blocks:

WAN Optimization (WAN-X)

WAN optimization appliances (Riverbed Steelhead, Cisco WAAS, Silver Peak) improve application performance over WAN through:

1. Data Deduplication: Eliminate redundant data across transfers. Byte-level deduplication can reduce transferred data by 70-95% for repeated patterns (documents, code, backups).

2. Compression: Compress unique data using LZ-based algorithms. Additional 40-60% reduction on compressible content.

3. Protocol Optimization: Overcome protocol inefficiencies:

   • CIFS/SMB: Prefetch, read-ahead, compound commands
   • HTTP: Connection pooling, persistent connections
   • TCP: Overcome bandwidth-delay product limitations

4. SSL/TLS Optimization: Decrypt, optimize, re-encrypt for protocols with end-to-end encryption. Requires certificate management.

5. Application Caching: Cache frequently accessed content locally (video streams, software updates).

WAN Optimization Deployment Models:

• In-Path: Appliances inline between LAN and WAN; intercept all traffic
• Out-of-Path (WCCP): Redirect traffic via WCCP; appliance failure doesn't break connectivity
• Cloud-Based: Optimization as cloud service; no on-premises appliances

WAN circuits traverse infrastructure outside your control—carrier networks, third-party fiber, shared conduits, and internet exchanges. Resilience design must account for failures at multiple levels.

Failure Domains and Redundancy Strategies:

Carrier Diversity Best Practices:

1. True Diversity Assessment: Carriers often share underlying infrastructure. Two "different" circuits may travel the same fiber. Request diversity reports showing physical path separation.

2. Entrance Facility Separation: For critical sites, bring circuits into different building entrance points, ideally on different sides of the building.

3. Last-Mile Technology Mix: Combine fiber and LTE, or different fiber providers, to reduce common-mode failures.

4. Carrier Selection Strategy: Use different carriers for primary and backup. Reduces exposure to carrier-specific outages or business issues.

Active-Active vs. Active-Standby:

The enterprise WAN is undergoing fundamental transformation. Traditional architectures—designed when applications lived in enterprise datacenters—struggle with cloud-centric traffic patterns. Two converging trends are reshaping WAN design:

SD-WAN: Software-Defined Wide Area Network

SD-WAN (introduced in Branch Connectivity, expanded here) transforms WAN architecture by:

1. Abstracting Transport: Multiple underlying transports (MPLS, internet, LTE) appear as a unified fabric. Traffic steered based on application requirements and real-time path quality.

2. Centralizing Control: Policies defined once in central controller, deployed automatically to all sites. Enables enterprise-wide consistency and rapid change.

3. Enabling Direct Cloud Access: Sites access cloud/SaaS directly via local internet breakout, reducing latency and avoiding backhaul costs.

4. Simplifying Operations: Zero-touch deployment, automated configuration, and comprehensive visibility reduce operational burden.

SASE: Secure Access Service Edge

SASE (pronounced "sassy"), defined by Gartner in 2019, converges SD-WAN with cloud-delivered security into a unified service:

SASE Components:

• SD-WAN: Network connectivity and optimization
• CASB (Cloud Access Security Broker): Visibility and control for SaaS applications
• SWG (Secure Web Gateway): Web filtering and threat protection
• ZTNA (Zero Trust Network Access): Application access without traditional VPN
• FWaaS (Firewall as a Service): Cloud-delivered firewall capabilities

SASE Benefits:

• Consistent security regardless of user/site location
• Reduced attack surface (cloud enforcement, not perimeter)
• Simplified architecture (fewer on-premises appliances)
• Better cloud/SaaS performance (security at cloud edge)
• Support for remote/hybrid workforce

We've covered substantial ground in WAN architecture. Let's consolidate the key takeaways:

What's Next:

With campus, branch, and WAN design understood, we'll next explore Security Zones—how to segment enterprise networks to contain threats, enforce access policies, and align network architecture with security requirements. Security zone design integrates network and security architecture into a cohesive defense strategy.