API Gateway - Learning Module

Loading content...

0/273

Request Routing

The Traffic Director of Your Distributed System

Every request that enters your microservices ecosystem must find its way to the right service. This seemingly simple task—routing—is the foundational responsibility of any API Gateway. Yet beneath this simplicity lies remarkable complexity: path matching, header-based decisions, weighted distribution, canary deployments, and dynamic service discovery.

Request routing determines not just where traffic goes, but how it gets there. A well-designed routing layer enables zero-downtime deployments, gradual rollouts, A/B testing, and graceful degradation. A poorly designed one creates operational nightmares and deployment risks.

This page provides a comprehensive exploration of request routing strategies, from basic path matching to advanced traffic management patterns used by the world's most sophisticated distributed systems.

What You Will Learn

By the end of this page, you will master the complete spectrum of routing strategies—static and dynamic, simple and sophisticated. You'll understand how to implement traffic splitting for deployment strategies, use header-based routing for feature flags, and design routing rules that scale from dozens to thousands of services.

Routing Fundamentals

At its core, routing is a decision: given an incoming request with specific characteristics, which backend service should handle it? This decision is based on routing rules that match request attributes against defined criteria.

The Anatomy of a Routing Decision:

Request arrives at the gateway with URL, headers, method, and body
Matcher evaluates the request against defined routing rules
First matching rule (or highest priority rule) determines the target
Request is forwarded to the selected backend service
Response flows back through the gateway to the client

The sophistication of your routing layer determines what attributes can be matched, how rules are prioritized, and how dynamic the routing decisions can be.

Routing Match Criteria
Criterion	Description	Example Use Case
Path	URL path pattern matching	`/users/*` → user-service
Method	HTTP method (GET, POST, etc.)	POST `/orders` → order-write-service
Host	Request host header	`api.mobile.example.com` → mobile-backend
Headers	Custom or standard headers	`X-API-Version: 2` → v2-service
Query Parameters	URL query string values	`?region=eu` → eu-deployment
Source IP	Client IP address or CIDR range	Internal IPs → internal-gateway
Cookie	Request cookie values	`beta=true` → beta-features-service
Request Body	JSON path or field matching	`$.priority = 'high'` → priority-queue

Rule Ordering Matters

Most gateways evaluate rules in order, using the first match. This means specific rules must come before general ones. /users/admin should appear before /users/*, or the general pattern will match first. Some gateways support explicit priority numbers to avoid ordering dependencies.

Path-Based Routing

Path-based routing is the most common and intuitive routing strategy. The URL path determines which service handles the request. This approach maps naturally to RESTful API designs where resource paths indicate domain boundaries.

Path Matching Patterns:

Path Routing Configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# Kong Gateway route configuration
routes:
  # Exact path match
  - name: health-check
    paths:
      - /health
    service: health-service
    
  # Prefix match (default behavior)
  - name: user-routes
    paths:
      - /api/users
    service: user-service
    strip_path: true  # Remove /api/users prefix before forwarding
    
  # Regex path match
  - name: order-by-id
    paths:
      - ~/api/orders/[0-9]+$  # Regex pattern
    service: order-service
    
  # Wildcard with path parameter capture
  - name: product-details
    paths:
      - /api/products/{productId}
    service: product-service
    
  # Multiple paths to same service
  - name: legacy-compatibility
    paths:
      - /v1/customers
      - /api/customers
      - /customers
    service: customer-service

Path Rewriting:

Often, the external API path differs from the internal service path. For example, clients call /api/v2/users, but the user-service expects just /users. Path rewriting transforms the request path before forwarding.

External Path	Rewrite Rule	Internal Path
`/api/v2/users/123`	Strip `/api/v2`	`/users/123`
`/mobile/orders`	Replace `/mobile` with `/api`	`/api/orders`
`/old-endpoint`	Map to new path	`/new-endpoint`

Path Security Considerations

Be cautious with path normalization. Path traversal attacks (../) can bypass routing rules if not properly handled. Ensure your gateway normalizes paths and rejects malformed URLs before routing decisions are made.

Header-Based Routing

Header-based routing enables sophisticated traffic management without changing URL structures. This is essential for versioning, feature flags, A/B testing, and multi-tenant routing.

Common Header Routing Patterns:

Header Routing Use Cases

•API Versioning — API-Version: 2 routes to v2 implementation while API-Version: 1 (or missing) routes to v1
•Feature Flags — X-Feature: dark-mode routes to service with feature enabled for gradual rollout
•Tenant Isolation — X-Tenant-ID: acme-corp routes to dedicated infrastructure for enterprise customers
•Client Type — X-Client-Type: mobile routes to mobile-optimized backend with different response formats
•Debug/Testing — X-Debug: true routes to instrumented service version for troubleshooting
•Geographic Routing — X-Region: eu-west routes to geographically appropriate data center

Header-Based Routing Examples
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# Envoy Proxy header-based routing
virtual_hosts:
- name: api_service
  domains: ["api.example.com"]
  routes:
    # Route based on API version header
    - match:
        prefix: "/users"
        headers:
          - name: "API-Version"
            exact_match: "2"
      route:
        cluster: user_service_v2
        
    - match:
        prefix: "/users"
        headers:
          - name: "API-Version"
            exact_match: "1"
      route:
        cluster: user_service_v1
        
    # Default to v1 if no version header
    - match:
        prefix: "/users"
      route:
        cluster: user_service_v1
        
    # Premium tier routing
    - match:
        prefix: "/search"
        headers:
          - name: "X-Subscription-Tier"
            exact_match: "premium"
      route:
        cluster: search_service_premium
        timeout: 30s
        
    - match:
        prefix: "/search"
      route:
        cluster: search_service_standard
        timeout: 10s
        
    # Beta features routing
    - match:
        prefix: "/"
        headers:
          - name: "X-Beta-Features"
            present_match: true
      route:
        cluster: beta_service

Header Injection for Downstream Services

Beyond routing, gateways often inject headers that downstream services can use. Common patterns include: X-Request-ID for correlation, X-User-ID from decoded JWT, X-Forwarded-For for client IP, and X-Trace-ID for distributed tracing. This simplifies service implementations by providing pre-processed context.

Traffic Splitting and Weighted Routing

Traffic splitting enables gradual rollouts by directing percentages of traffic to different service versions. This is essential for canary deployments, blue-green deployments, and A/B testing.

The Deployment Evolution:

All traffic (100%) → Current stable version (v1)
Canary phase (5%) → 5% to new version (v2), 95% to v1
Early rollout (25%) → Increase v2 traffic if metrics are healthy
Majority rollout (75%) → Most traffic to v2, v1 as fallback
Full deployment (100%) → All traffic to v2, v1 decommissioned

Traffic Splitting Configuration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# Istio VirtualService for canary deployment
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: product-service
spec:
  hosts:
    - product-service
  http:
    - route:
        # 90% of traffic to stable version
        - destination:
            host: product-service
            subset: stable
          weight: 90
        # 10% of traffic to canary
        - destination:
            host: product-service
            subset: canary
          weight: 10
---
apiVersion: networking.istio.io/v1beta1
kind: DestinationRule
metadata:
  name: product-service
spec:
  host: product-service
  subsets:
    - name: stable
      labels:
        version: v1
    - name: canary
      labels:
        version: v2

Session Stickiness Considerations:

When splitting traffic, you often want the same user to consistently reach the same version. Otherwise, a user might experience UI inconsistencies or data issues between versions.

Stickiness Method	How It Works	Trade-offs
Cookie-based	Gateway sets a cookie indicating the assigned version	Requires cookie support; versioned by browser
Header-based	Route based on user ID or session ID in headers	Requires client to send consistent ID
IP-based	Hash client IP to determine version	NAT can cause uneven distribution
Consistent Hashing	Hash key deterministically maps to version	More complex but robust

Monitoring During Splits

Traffic splitting is only useful if you can compare versions. Ensure your observability stack can segment metrics by version: error rates, latency percentiles, and business metrics. Automated rollback based on metrics is the gold standard for safe deployments.

Dynamic Routing and Service Discovery

In static routing, backend addresses are hardcoded in configuration. This works for small, stable deployments but fails at scale. When services auto-scale, addresses change constantly. When containers restart, ports are reassigned. Dynamic routing solves this by integrating with service discovery.

Service Discovery Models:

Client-Side Discovery

•Gateway queries service registry (Consul, Eureka, etcd)
•Receives list of available instances
•Gateway performs load balancing across instances
•Gateway handles instance health tracking
•Pros: More control, can implement custom logic
•Cons: Gateway must implement discovery client

Server-Side Discovery

•Gateway routes to a load balancer (AWS ALB, Kubernetes Service)
•Load balancer handles discovery and distribution
•Gateway configuration uses stable DNS names
•Platform manages instance registration
•Pros: Simpler gateway, platform-managed
•Cons: Additional network hop, less control

Service Discovery Integration
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# Kong with Consul service discovery
upstreams:
  - name: user-service
    # Instead of static targets, use Consul
    host_header: user-service.service.consul
    slots: 10000
    
services:
  - name: user-api
    host: user-service.service.consul
    port: 8080
    protocol: http
    # Kong will resolve via DNS to get current instances
    
# DNS-based discovery configuration
dns_resolver:
  - 10.0.0.2:8600  # Consul DNS interface
dns_order:
  - SRV
  - A
dns_stale_ttl: 1  # Seconds before re-resolving

Discovery Failure Handling

When service discovery is unavailable, your gateway must not lose existing routes. Implement caching of last-known-good endpoints, graceful degradation, and alerts for discovery failures. A failed discovery query shouldn't bring down routing entirely.

Advanced Routing Patterns

Beyond basic path and header matching, sophisticated systems employ advanced routing patterns for specific use cases.

Content-Based Routing:

Routing decisions based on request body content. This is more expensive (requires body parsing) but enables powerful patterns:

Content-Based Routing
Kong Plugin
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
-- Kong plugin for content-based routing
local cjson = require "cjson"
 
function route_by_content(conf)
    local body = kong.request.get_body()
    
    if not body then
        return nil
    end
    
    -- Parse JSON body
    local ok, json = pcall(cjson.decode, body)
    if not ok then
        return nil
    end
    
    -- Route based on order priority
    if json.priority == "express" then
        kong.service.set_upstream("express-processing")
        return
    end
    
    -- Route based on payment method
    if json.payment_method == "crypto" then
        kong.service.set_upstream("crypto-payment-processor")
        return
    end
    
    -- Route based on order value for fraud checking
    if json.total_amount > 10000 then
        kong.service.set_upstream("high-value-order-service")
        return
    end
    
    -- Default routing
    kong.service.set_upstream("standard-order-service")
end

Geographic Routing:

Routing based on client location for compliance, latency optimization, or data residency requirements:

Client Location	Routing Decision	Rationale
EU countries	EU data center	GDPR compliance - data must stay in EU
Asia Pacific	Singapore region	Latency optimization
US Government	GovCloud	Compliance requirements
Unknown/VPN	Default region	Fallback behavior

More Advanced Patterns

•Shadow/Mirror Traffic — Clone requests to a secondary service for testing without affecting production responses. Useful for validating new implementations.
•Request Cloning — Send same request to multiple services, return first response. Useful for redundancy or performance comparison.
•Circuit Breaker Routing — Automatically route around failing services to fallback endpoints.
•Time-Based Routing — Different routing during business hours vs. off-hours, or during known high-traffic events.
•Tenant-Aware Sharding — Route tenants to specific service instances based on tenant ID hash for data locality.

Shadow Traffic for Safe Testing

Shadow traffic (also called 'dark launching') lets you test new services with production traffic without risk. The gateway mirrors requests to both old and new services but only returns responses from the old service. You can compare behavior and performance before switching traffic.

Routing Performance Considerations

Every routing decision adds latency. At scale, even milliseconds matter. Understanding the performance implications of routing strategies is crucial for high-throughput systems.

Routing Complexity Hierarchy (Fastest to Slowest):

Routing Strategy Performance Comparison
Strategy	Typical Latency Added	Why
Exact path match	~0.01-0.1ms	Simple hash lookup, O(1)
Prefix path match	~0.1-0.5ms	Trie or prefix tree traversal
Regex path match	~0.5-2ms	Regex engine execution, backtracking risk
Header-based routing	~0.1-0.3ms	Header parsing + match logic
Cookie-based routing	~0.2-0.5ms	Cookie parsing overhead
Body content routing	~1-10ms	Body buffering + parsing (JSON/XML)
External decision service	~5-50ms	Network call to routing service

Optimization Strategies:

Performance Optimizations

•Rule ordering — Place most frequently matched rules first; avoid evaluating all rules for common paths
•Compiled regex — Pre-compile regular expressions at startup, not per-request
•Route caching — Cache routing decisions for identical requests (path + relevant headers)
•Avoid body parsing — Only parse request bodies when absolutely necessary for routing
•Efficient data structures — Use radix trees for path matching, hash maps for exact matches
•Connection pooling — Reuse connections to backend services to avoid connection setup overhead

Regex Complexity

Complex regular expressions with backtracking can cause catastrophic performance issues (ReDoS attacks). Limit regex usage to simple patterns, set execution timeouts, and consider using literal string matching with wildcards instead of full regex when possible.

Summary: Request Routing Mastery

Request routing is the foundation of API Gateway functionality. Let's consolidate the key insights from this comprehensive exploration:

Key Takeaways

•Multiple matching criteria — Path, headers, query parameters, and body content can all inform routing decisions, with different performance characteristics.
•Rule ordering matters — Most gateways use first-match semantics; organize rules from specific to general.
•Path rewriting enables decoupling — External API paths can differ from internal service expectations; gateways bridge this gap.
•Header routing enables sophistication — API versioning, feature flags, A/B testing, and tenant isolation all leverage header-based decisions.
•Traffic splitting enables safe deployments — Canary and blue-green deployments depend on weighted routing with proper session stickiness.
•Dynamic routing scales — Service discovery integration eliminates hardcoded addresses; essential for auto-scaling environments.
•Advanced patterns exist — Shadow traffic, geographic routing, and content-based routing solve specialized needs.
•Performance matters — Simple routing strategies are fast; body parsing and external calls add significant latency.

What's Next:

With routing fundamentals established, the next page explores Authentication Aggregation—how the API Gateway centralizes identity verification, validates tokens, and propagates authenticated context to backend services, eliminating redundant security logic across your microservices.

Page Complete

You now possess comprehensive knowledge of request routing in API Gateways. From basic path matching to advanced traffic splitting, you can design routing strategies that enable sophisticated deployment patterns while maintaining performance and operational clarity.