Computer NetworksContainerized Networking

Containerized Networking

LevelAdvanced

Duration90 mins

TopicContainerized Networking

3 / 5

Kubernetes Networking

Networking at Orchestration Scale

Kubernetes doesn't just run containers—it orchestrates entire distributed systems. With thousands of pods across hundreds of nodes, traditional port-based container networking breaks down. How do you address a specific pod among millions? How do you route traffic to services that scale up and down dynamically? How do you enforce security policies in a constantly changing environment?

Kubernetes answers these questions with a fundamentally different networking model: every pod gets its own IP address, there's no NAT between pods, and sophisticated abstractions (Services, Ingress, Network Policies) handle discovery, load balancing, and security. Understanding this model is essential for anyone operating Kubernetes in production.

What You Will Learn

By the end of this page, you will understand Kubernetes' fundamental networking requirements, the pod networking model, how Services abstract ephemeral pod IPs into stable endpoints, Ingress controllers for HTTP routing, Network Policies for security, and how CNI plugins implement these abstractions. You'll see how Kubernetes networking differs fundamentally from Docker's approach.

Kubernetes Network Requirements

Kubernetes imposes three fundamental requirements on any compliant networking implementation. These requirements are non-negotiable—any CNI plugin or network solution must satisfy them:

Kubernetes Networking Requirements

•All pods can communicate with all other pods without NAT — Any pod can reach any other pod's IP address directly. No port mapping, no address translation. The source IP seen by the destination is the true source pod IP.
•All nodes can communicate with all pods without NAT — Node processes (like kubelet, kube-proxy) can reach pod IPs directly. This enables health checks, log collection, and debugging.
•The IP that a pod sees itself as is the same IP that others see it as — No split-horizon DNS or IP masquerading. Pods have consistent network identity across the cluster.

Why these requirements matter:

These requirements simplify application development dramatically. Traditional container networking (like Docker's published ports) forces applications to be aware of the mapping between container ports and host ports. Kubernetes' flat network means:

Applications use standard ports (web servers on 80/443, databases on their native ports)
Service discovery is simpler—just reach an IP
Security policies can be based on actual source IPs
Debugging is easier—tcpdump shows real addresses
Existing applications work without modification

The cost is implementation complexity: creating a flat network across potentially thousands of nodes requires sophisticated solutions (VXLAN, BGP, eBPF, etc.) provided by CNI plugins.

Docker vs Kubernetes Networking Model
Aspect	Docker (Bridge)	Kubernetes
Pod/Container IP visibility	Private to host	Globally routable in cluster
Cross-host communication	Requires port mapping or overlay	Native; any pod reaches any pod
Port usage	Host port conflicts possible	Each pod has full port space
NAT	Required for external access	No NAT within cluster
Service discovery	Per-network DNS or manual	Cluster-wide DNS + Services
IP per workload	Often shared (port mapping)	Every pod has unique IP

The Power of No NAT

NAT breaks many protocols (SIP, FTP active mode, some game protocols) and complicates debugging. Kubernetes' no-NAT requirement means any network protocol works without modification, and packet captures show real addresses—invaluable for production troubleshooting.

Pod Networking Deep Dive

A pod is Kubernetes' fundamental unit of deployment—one or more containers that share network namespace, IPC namespace, and storage. From a networking perspective, all containers in a pod share the same network stack: they have the same IP address and can communicate via localhost.

Pod network namespace structure:

When the kubelet creates a pod, it:

Creates a network namespace for the pod (via the container runtime)
Invokes the configured CNI plugin to set up networking
The CNI plugin creates a veth pair, assigns an IP, sets up routing
All containers in the pod join this namespace

Converting Mermaid diagram...

Inter-container communication within a pod:

Containers in the same pod communicate via localhost. This enables powerful patterns:

Sidecar proxies (like Envoy in Istio) intercept traffic on localhost
Log collectors read from shared volumes and expose metrics on localhost:9090
Init containers prepare the network or data before main containers start

One IP address, many ports:

All containers in a pod share the IP address but must coordinate on ports. Container A on port 8080 and Container B on port 9090 both use 10.244.1.5, but different ports. This is intentional—it encourages cohesive, tightly-coupled containers that form a logical unit.

pod-networking-demo.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
apiVersion: v1
kind: Pod
metadata:
  name: multi-container-pod
spec:
  containers:
  - name: web
    image: nginx:alpine
    ports:
    - containerPort: 80
    # Web server accessible on pod IP:80
 
  - name: metrics
    image: prom/prometheus:latest
    ports:
    - containerPort: 9090
    # Prometheus accessible on pod IP:9090
    # Also reachable from web container via localhost:9090
 
  - name: debug
    image: nicolaka/netshoot
    command: ["sleep", "infinity"]
    # Debug container shares same network namespace
    # Can see/debug network traffic from other containers

pod-network-inspection.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# View pod's assigned IP address
kubectl get pod multi-container-pod -o wide
# NAME                  READY   STATUS    IP           NODE
# multi-container-pod   3/3     Running   10.244.1.5   node-1
 
# All containers see the same network configuration
kubectl exec multi-container-pod -c web -- ip addr
kubectl exec multi-container-pod -c metrics -- ip addr
kubectl exec multi-container-pod -c debug -- ip addr
# All output: 10.244.1.5 (same IP!)
 
# Container can reach others via localhost
kubectl exec multi-container-pod -c debug -- curl localhost:80
# Returns nginx response from 'web' container
 
kubectl exec multi-container-pod -c debug -- curl localhost:9090
# Returns Prometheus response from 'metrics' container
 
# Pod-to-pod communication (across cluster)
kubectl exec multi-container-pod -c debug -- curl 10.244.2.10:8080
# Directly reaches another pod's IP

Kubernetes Services: Stable Network Endpoints

Pods are ephemeral—they're created, destroyed, and rescheduled constantly. Their IP addresses change with each incarnation. Kubernetes Services solve this problem by providing stable network endpoints that abstract away pod ephemerality.

A Service is a stable IP address (and DNS name) that load-balances traffic across a dynamic set of pods selected by labels. Clients connect to the Service; Kubernetes routes traffic to healthy pods.

Kubernetes Service Types
Type	ClusterIP	External Access	Use Case
ClusterIP (default)	Assigned from cluster CIDR	None (internal only)	Internal microservice communication
NodePort	Yes + Node IP:NodePort	Any node IP:30000-32767	Development; simple external access
LoadBalancer	Yes + External LB IP	Cloud provider LB	Production external services
ExternalName	None (CNAME)	DNS alias to external	Bridge to external services
Headless	None (ClusterIP: None)	Direct pod IPs via DNS	StatefulSets; direct pod access

kubernetes-services.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
# ClusterIP Service (default) - Internal only
apiVersion: v1
kind: Service
metadata:
  name: backend-api
spec:
  type: ClusterIP  # Can be omitted (default)
  selector:
    app: backend
    tier: api
  ports:
    - name: http
      port: 80          # Service port (what clients use)
      targetPort: 8080  # Pod port (what app listens on)
      protocol: TCP
---
# NodePort Service - External access via node IPs
apiVersion: v1
kind: Service
metadata:
  name: web-nodeport
spec:
  type: NodePort
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30080  # Optional; auto-assigned if omitted
---
# LoadBalancer Service - Cloud LB provisioning
apiVersion: v1
kind: Service
metadata:
  name: web-lb
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
    - port: 443
      targetPort: 8443
  # Cloud controller provisions external LB
---
# Headless Service - Direct pod IPs
apiVersion: v1
kind: Service
metadata:
  name: database
spec:
  clusterIP: None  # Makes it headless
  selector:
    app: postgres
  ports:
    - port: 5432
# DNS returns all pod IPs, not a single VIP

How Services work: kube-proxy and iptables

Kubernetes Services are typically implemented by kube-proxy, which runs on every node and programs the data plane:

iptables mode (default):

kube-proxy watches the API server for Service/Endpoints changes
For each Service, it creates iptables rules
Traffic to ClusterIP is DNAT'd to a pod IP (random selection)
Return traffic is handled by connection tracking

IPVS mode (high performance):

Uses Linux IPVS (IP Virtual Server) kernel module
Hash-based load balancing (O(1) vs O(n) for iptables)
Multiple scheduling algorithms (round-robin, least connections, etc.)
Better performance for clusters with thousands of Services

service-inspection.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# View Service details
kubectl get svc backend-api -o wide
# NAME          TYPE        CLUSTER-IP      PORT(S)   SELECTOR
# backend-api   ClusterIP   10.96.145.82    80/TCP    app=backend,tier=api
 
# View endpoints (actual pod IPs behind the Service)
kubectl get endpoints backend-api
# NAME          ENDPOINTS
# backend-api   10.244.1.4:8080,10.244.2.7:8080,10.244.3.2:8080
 
# DNS resolution for Services
kubectl run dns-test --rm -it --image=busybox -- nslookup backend-api
# Server:    10.96.0.10
# Address:   10.96.0.10:53
# Name:      backend-api.default.svc.cluster.local
# Address:   10.96.145.82
 
# Full DNS pattern: <service>.<namespace>.svc.<cluster-domain>
 
# View kube-proxy's iptables rules (on a node)
sudo iptables -t nat -L KUBE-SERVICES -n | grep backend-api
# Shows DNAT rules routing ClusterIP to pod IPs

Service DNS Patterns

Services get DNS entries in the format: <service-name>.<namespace>.svc.cluster.local. Within the same namespace, use just <service-name>. Across namespaces, use <service-name>.<namespace>. For headless services, DNS returns A records for each pod.

Ingress: HTTP-Layer Routing

Services provide Layer 4 (TCP/UDP) load balancing. For Layer 7 (HTTP/HTTPS) routing—hostname-based routing, path-based routing, TLS termination—Kubernetes provides the Ingress resource.

An Ingress defines HTTP routing rules mapping external requests to internal Services. An Ingress Controller (nginx, Traefik, HAProxy, AWS ALB, etc.) implements these rules—Kubernetes itself doesn't include one by default.

Converting Mermaid diagram...

ingress-example.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: main-ingress
  annotations:
    # Ingress controller-specific annotations
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
spec:
  ingressClassName: nginx  # Which controller handles this
  tls:
  - hosts:
    - api.example.com
    - web.example.com
    secretName: example-tls  # Kubernetes Secret with TLS cert
  rules:
  # Host-based routing
  - host: api.example.com
    http:
      paths:
      - path: /v1
        pathType: Prefix
        backend:
          service:
            name: api-v1
            port:
              number: 80
      - path: /v2
        pathType: Prefix
        backend:
          service:
            name: api-v2
            port:
              number: 80
  # Different host
  - host: web.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-frontend
            port:
              number: 80
      - path: /static
        pathType: Prefix
        backend:
          service:
            name: cdn-service
            port:
              number: 80

Ingress Controller capabilities:

Popular Ingress controllers provide features beyond basic routing:

Feature	nginx-ingress	Traefik	HAProxy	AWS ALB
TLS termination	✅	✅	✅	✅
Rate limiting	✅	✅	✅	✅
OAuth/OIDC auth	✅	✅	✅	✅
WebSocket support	✅	✅	✅	✅
gRPC support	✅	✅	✅	✅
Canary deployments	✅	✅	✅	Limited
Custom error pages	✅	✅	✅	Limited
Request tracing	Via integration	Built-in	Via integration	AWS X-Ray

Gateway API: The Future

Kubernetes Gateway API is the next-generation replacement for Ingress, providing more expressive, extensible, and role-oriented configuration. It supports TCP/UDP routing, traffic splitting, and cross-namespace routing—capabilities limited in Ingress. As of 2024, Gateway API is GA and recommended for new deployments.

Network Policies: Microsegmentation

By default, Kubernetes allows all pods to communicate with all other pods—a flat, permissive network. In production, this is often undesirable. Network Policies implement microsegmentation, controlling which pods can communicate with which others.

Network Policies are firewall rules at the pod level: you specify which ingress (incoming) and egress (outgoing) traffic is allowed. All other traffic is denied by the default-deny policy.

network-policies.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
# Default deny all ingress traffic to database pods
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: database-deny-all
  namespace: production
spec:
  podSelector:
    matchLabels:
      tier: database
  policyTypes:
  - Ingress
  # No ingress rules = deny all ingress
---
# Allow backend pods to access database pods on port 5432
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: database-allow-backend
  namespace: production
spec:
  podSelector:
    matchLabels:
      tier: database
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          tier: backend
    ports:
    - protocol: TCP
      port: 5432
---
# Allow frontend to reach backend API only
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-allow-frontend
  namespace: production
spec:
  podSelector:
    matchLabels:
      tier: backend
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          tier: frontend
    - namespaceSelector:
        matchLabels:
          environment: production
    ports:
    - protocol: TCP
      port: 8080
---
# Restrict egress: backend can only reach database and external APIs
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-egress
  namespace: production
spec:
  podSelector:
    matchLabels:
      tier: backend
  policyTypes:
  - Egress
  egress:
  - to:
    - podSelector:
        matchLabels:
          tier: database
    ports:
    - protocol: TCP
      port: 5432
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 10.0.0.0/8    # Block internal IPs
        - 172.16.0.0/12
        - 192.168.0.0/16
    ports:
    - protocol: TCP
      port: 443

Network Policy enforcement:

Network Policies are enforced by the CNI plugin, not by Kubernetes itself. Not all CNI plugins support Network Policies:

CNI Plugin	Network Policy Support
Calico	✅ Full support (L3-L4)
Cilium	✅ Full + L7 policies
Weave	✅ Full support
Flannel	❌ No support*
AWS VPC CNI	✅ With Calico addon
Azure CNI	✅ With Azure Network Policy

*Flannel requires adding Calico for policy enforcement

Zero-Trust Architecture

In production, always implement default-deny policies and explicitly allow required traffic. Start by denying all ingress/egress, then incrementally add allow rules. This 'zero-trust' approach prevents lateral movement if a pod is compromised.

Network Policy Best Practices

•Start with deny-all — Create namespace-wide default-deny policies first, then add specific allow rules.
•Label consistently — Network policies select pods by labels. Consistent labeling (tier, app, version) enables precise policies.
•Limit egress — Egress policies are often overlooked but critical. Restrict outbound traffic to prevent data exfiltration.
•Test policies before production — Use tools like kubectl-np-viewer or network policy simulators to verify behavior.
•Monitor policy enforcement — CNI plugins like Cilium provide metrics on allowed/denied connections.

Kubernetes DNS: CoreDNS

Every Kubernetes cluster runs a DNS server (typically CoreDNS) that provides name resolution for Services and Pods. DNS is fundamental to service discovery—applications use DNS names, not IP addresses, to find each other.

DNS records Kubernetes creates:

Kubernetes DNS Records
Resource	DNS Format	Record Type	Returns
Service	`<svc>.<ns>.svc.cluster.local`	A/AAAA	ClusterIP
Headless Service	`<svc>.<ns>.svc.cluster.local`	A/AAAA	All pod IPs
Pod (headless svc)	`<pod-ip-dashed>.<svc>.<ns>.svc.cluster.local`	A/AAAA	Pod IP
StatefulSet Pod	`<pod-name>.<svc>.<ns>.svc.cluster.local`	A/AAAA	Specific pod IP
Service Port	`_<port>._<proto>.<svc>.<ns>.svc.cluster.local`	SRV	Port + target
External (headless)	CNAME	CNAME	External hostname

dns-exploration.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# View CoreDNS pods
kubectl get pods -n kube-system -l k8s-app=kube-dns
# NAME                       READY   STATUS    RESTARTS   AGE
# coredns-558bd4d5db-abcde   1/1     Running   0          15d
# coredns-558bd4d5db-fghij   1/1     Running   0          15d
 
# View CoreDNS ConfigMap
kubectl get configmap coredns -n kube-system -o yaml
 
# DNS resolution examples from a pod
kubectl run dns-test --rm -it --image=busybox:1.28 -- sh
 
# Within the same namespace
nslookup my-service
# Name:      my-service
# Address:   10.96.100.50
 
# Cross-namespace
nslookup postgres.database.svc.cluster.local
# Name:      postgres.database.svc.cluster.local
# Address:   10.96.200.100
 
# StatefulSet pods (stable DNS per pod)
nslookup postgres-0.postgres.database.svc.cluster.local
# Returns specific pod IP - crucial for stateful apps
 
# SRV records for port discovery
nslookup -query=SRV _http._tcp.my-service.default.svc.cluster.local
 
# External name service
nslookup external-db.default.svc.cluster.local
# Returns CNAME: rds.aws.amazon.com

Pod DNS configuration:

Each pod's /etc/resolv.conf is configured to use the cluster DNS:

nameserver 10.96.0.10     # CoreDNS ClusterIP
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

The search directive allows short names: my-service automatically expands to my-service.default.svc.cluster.local.

The ndots:5 option means names with fewer than 5 dots are tried with search domains first. This can add latency for external lookups (like api.github.com, which has only 2 dots). For performance-sensitive applications, consider customizing this.

DNS Performance Tuning

High-traffic clusters often hit DNS rate limits. Solutions: 1) Use NodeLocal DNSCache (runs DNS cache on each node), 2) Scale CoreDNS replicas, 3) Reduce ndots value, 4) Use FQDNs with trailing dot to bypass search expansion.

CNI Plugins: Implementing the Network

Kubernetes doesn't implement pod networking itself—it delegates to CNI (Container Network Interface) plugins. A CNI plugin is responsible for:

Assigning IP addresses to pods
Programming routes so pods can communicate
Enforcing Network Policies (if supported)
Managing network namespaces and virtual interfaces

Popular CNI Plugins Comparison
CNI Plugin	Networking Mode	Network Policy	Notable Features
Calico	BGP (L3) or VXLAN	✅ Advanced	High performance; enterprise support; eBPF mode
Cilium	eBPF (kernel bypass)	✅ L3-L7	API-aware policies; observability; service mesh
Flannel	VXLAN overlay	❌	Simple; lightweight; requires Calico for policies
Weave	VXLAN + mesh	✅	Encryption; multicast; simple setup
AWS VPC CNI	Native VPC networking	✅ (addon)	Pod IPs from VPC; no overlay; high performance
Azure CNI	Native VNET	✅ (addon)	Pod IPs from VNET; Azure integration
Antrea	Open vSwitch	✅	VMware-backed; Windows support; observability

CNI plugin architectures:

Overlay networks (VXLAN, Geneve):

Encapsulate pod traffic in UDP packets
Traverse any underlay network (even across cloud regions)
Overhead: ~50 bytes per packet, slight CPU cost
Examples: Flannel VXLAN, Weave, Calico VXLAN mode

Routed networks (BGP):

Pod CIDRs advertised via BGP to network infrastructure
No encapsulation overhead
Requires BGP-capable network (or integration)
Examples: Calico BGP mode, MetalLB

Native cloud integration:

Pod IPs assigned from cloud VPC/VNET
Pods directly routable from cloud resources
Cloud IAM and security groups apply directly to pods
Examples: AWS VPC CNI, Azure CNI, GKE VPC-native

Choosing a CNI Plugin

For cloud environments: use the cloud-native CNI (AWS VPC CNI, Azure CNI, GKE) for best performance and integration. For on-prem or advanced needs: Calico for performance and policies, Cilium for eBPF observability and L7 policies. For simplicity: Flannel + Calico (policy only).

Summary: Kubernetes Networking Mastery

We've covered Kubernetes networking comprehensively—from fundamental requirements to advanced policy enforcement.

Key Takeaways

•Flat network, no NAT — Every pod gets a routable IP; any pod can reach any other pod directly.
•Pods share network namespace — All containers in a pod have the same IP, communicate via localhost.
•Services abstract pod ephemerality — ClusterIP, NodePort, LoadBalancer provide stable endpoints.
•Ingress enables L7 routing — Hostname and path-based routing, TLS termination, advanced features.
•Network Policies enforce microsegmentation — Define which pods can communicate with which others.
•CoreDNS provides service discovery — Automatic DNS entries for Services and Pods.
•CNI plugins implement networking — Calico, Cilium, Flannel, etc. provide the actual network.

What's next:

Kubernetes networking provides the foundation for pod communication. In the next page, we'll explore Service Meshes—an advanced pattern that adds observability, security, and traffic management at the application layer, using sidecar proxies to intercept and control all pod network traffic.

Page Complete

You now understand Kubernetes networking deeply—from the flat pod network to Services, Ingress, Network Policies, and DNS. These concepts form the foundation for operating any Kubernetes cluster in production.

3 / 5

Loading learning content...

Computer NetworksContainerized Networking

Containerized Networking

LevelAdvanced

Duration90 mins

TopicContainerized Networking

3 / 5

Kubernetes Networking

Networking at Orchestration Scale

What You Will Learn

Kubernetes Network Requirements

Kubernetes imposes three fundamental requirements on any compliant networking implementation. These requirements are non-negotiable—any CNI plugin or network solution must satisfy them:

Kubernetes Networking Requirements

•All pods can communicate with all other pods without NAT — Any pod can reach any other pod's IP address directly. No port mapping, no address translation. The source IP seen by the destination is the true source pod IP.
•All nodes can communicate with all pods without NAT — Node processes (like kubelet, kube-proxy) can reach pod IPs directly. This enables health checks, log collection, and debugging.
•The IP that a pod sees itself as is the same IP that others see it as — No split-horizon DNS or IP masquerading. Pods have consistent network identity across the cluster.

Why these requirements matter:

Applications use standard ports (web servers on 80/443, databases on their native ports)
Service discovery is simpler—just reach an IP
Security policies can be based on actual source IPs
Debugging is easier—tcpdump shows real addresses
Existing applications work without modification

The cost is implementation complexity: creating a flat network across potentially thousands of nodes requires sophisticated solutions (VXLAN, BGP, eBPF, etc.) provided by CNI plugins.

Docker vs Kubernetes Networking Model
Aspect	Docker (Bridge)	Kubernetes
Pod/Container IP visibility	Private to host	Globally routable in cluster
Cross-host communication	Requires port mapping or overlay	Native; any pod reaches any pod
Port usage	Host port conflicts possible	Each pod has full port space
NAT	Required for external access	No NAT within cluster
Service discovery	Per-network DNS or manual	Cluster-wide DNS + Services
IP per workload	Often shared (port mapping)	Every pod has unique IP

The Power of No NAT

Pod Networking Deep Dive

Pod network namespace structure:

When the kubelet creates a pod, it:

Creates a network namespace for the pod (via the container runtime)
Invokes the configured CNI plugin to set up networking
The CNI plugin creates a veth pair, assigns an IP, sets up routing
All containers in the pod join this namespace

Converting Mermaid diagram...

Inter-container communication within a pod:

Containers in the same pod communicate via localhost. This enables powerful patterns:

Sidecar proxies (like Envoy in Istio) intercept traffic on localhost
Log collectors read from shared volumes and expose metrics on localhost:9090
Init containers prepare the network or data before main containers start

One IP address, many ports:

pod-networking-demo.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
apiVersion: v1
kind: Pod
metadata:
  name: multi-container-pod
spec:
  containers:
  - name: web
    image: nginx:alpine
    ports:
    - containerPort: 80
    # Web server accessible on pod IP:80
 
  - name: metrics
    image: prom/prometheus:latest
    ports:
    - containerPort: 9090
    # Prometheus accessible on pod IP:9090
    # Also reachable from web container via localhost:9090
 
  - name: debug
    image: nicolaka/netshoot
    command: ["sleep", "infinity"]
    # Debug container shares same network namespace
    # Can see/debug network traffic from other containers

pod-network-inspection.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# View pod's assigned IP address
kubectl get pod multi-container-pod -o wide
# NAME                  READY   STATUS    IP           NODE
# multi-container-pod   3/3     Running   10.244.1.5   node-1
 
# All containers see the same network configuration
kubectl exec multi-container-pod -c web -- ip addr
kubectl exec multi-container-pod -c metrics -- ip addr
kubectl exec multi-container-pod -c debug -- ip addr
# All output: 10.244.1.5 (same IP!)
 
# Container can reach others via localhost
kubectl exec multi-container-pod -c debug -- curl localhost:80
# Returns nginx response from 'web' container
 
kubectl exec multi-container-pod -c debug -- curl localhost:9090
# Returns Prometheus response from 'metrics' container
 
# Pod-to-pod communication (across cluster)
kubectl exec multi-container-pod -c debug -- curl 10.244.2.10:8080
# Directly reaches another pod's IP

Kubernetes Services: Stable Network Endpoints

A Service is a stable IP address (and DNS name) that load-balances traffic across a dynamic set of pods selected by labels. Clients connect to the Service; Kubernetes routes traffic to healthy pods.

Kubernetes Service Types
Type	ClusterIP	External Access	Use Case
ClusterIP (default)	Assigned from cluster CIDR	None (internal only)	Internal microservice communication
NodePort	Yes + Node IP:NodePort	Any node IP:30000-32767	Development; simple external access
LoadBalancer	Yes + External LB IP	Cloud provider LB	Production external services
ExternalName	None (CNAME)	DNS alias to external	Bridge to external services
Headless	None (ClusterIP: None)	Direct pod IPs via DNS	StatefulSets; direct pod access

kubernetes-services.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
# ClusterIP Service (default) - Internal only
apiVersion: v1
kind: Service
metadata:
  name: backend-api
spec:
  type: ClusterIP  # Can be omitted (default)
  selector:
    app: backend
    tier: api
  ports:
    - name: http
      port: 80          # Service port (what clients use)
      targetPort: 8080  # Pod port (what app listens on)
      protocol: TCP
---
# NodePort Service - External access via node IPs
apiVersion: v1
kind: Service
metadata:
  name: web-nodeport
spec:
  type: NodePort
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 80
      nodePort: 30080  # Optional; auto-assigned if omitted
---
# LoadBalancer Service - Cloud LB provisioning
apiVersion: v1
kind: Service
metadata:
  name: web-lb
spec:
  type: LoadBalancer
  selector:
    app: web
  ports:
    - port: 443
      targetPort: 8443
  # Cloud controller provisions external LB
---
# Headless Service - Direct pod IPs
apiVersion: v1
kind: Service
metadata:
  name: database
spec:
  clusterIP: None  # Makes it headless
  selector:
    app: postgres
  ports:
    - port: 5432
# DNS returns all pod IPs, not a single VIP

How Services work: kube-proxy and iptables

Kubernetes Services are typically implemented by kube-proxy, which runs on every node and programs the data plane:

iptables mode (default):

kube-proxy watches the API server for Service/Endpoints changes
For each Service, it creates iptables rules
Traffic to ClusterIP is DNAT'd to a pod IP (random selection)
Return traffic is handled by connection tracking

IPVS mode (high performance):

Uses Linux IPVS (IP Virtual Server) kernel module
Hash-based load balancing (O(1) vs O(n) for iptables)
Multiple scheduling algorithms (round-robin, least connections, etc.)
Better performance for clusters with thousands of Services

service-inspection.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# View Service details
kubectl get svc backend-api -o wide
# NAME          TYPE        CLUSTER-IP      PORT(S)   SELECTOR
# backend-api   ClusterIP   10.96.145.82    80/TCP    app=backend,tier=api
 
# View endpoints (actual pod IPs behind the Service)
kubectl get endpoints backend-api
# NAME          ENDPOINTS
# backend-api   10.244.1.4:8080,10.244.2.7:8080,10.244.3.2:8080
 
# DNS resolution for Services
kubectl run dns-test --rm -it --image=busybox -- nslookup backend-api
# Server:    10.96.0.10
# Address:   10.96.0.10:53
# Name:      backend-api.default.svc.cluster.local
# Address:   10.96.145.82
 
# Full DNS pattern: <service>.<namespace>.svc.<cluster-domain>
 
# View kube-proxy's iptables rules (on a node)
sudo iptables -t nat -L KUBE-SERVICES -n | grep backend-api
# Shows DNAT rules routing ClusterIP to pod IPs

Service DNS Patterns

Ingress: HTTP-Layer Routing

Services provide Layer 4 (TCP/UDP) load balancing. For Layer 7 (HTTP/HTTPS) routing—hostname-based routing, path-based routing, TLS termination—Kubernetes provides the Ingress resource.

Converting Mermaid diagram...

ingress-example.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: main-ingress
  annotations:
    # Ingress controller-specific annotations
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/proxy-body-size: "50m"
spec:
  ingressClassName: nginx  # Which controller handles this
  tls:
  - hosts:
    - api.example.com
    - web.example.com
    secretName: example-tls  # Kubernetes Secret with TLS cert
  rules:
  # Host-based routing
  - host: api.example.com
    http:
      paths:
      - path: /v1
        pathType: Prefix
        backend:
          service:
            name: api-v1
            port:
              number: 80
      - path: /v2
        pathType: Prefix
        backend:
          service:
            name: api-v2
            port:
              number: 80
  # Different host
  - host: web.example.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: web-frontend
            port:
              number: 80
      - path: /static
        pathType: Prefix
        backend:
          service:
            name: cdn-service
            port:
              number: 80

Ingress Controller capabilities:

Popular Ingress controllers provide features beyond basic routing:

Feature	nginx-ingress	Traefik	HAProxy	AWS ALB
TLS termination	✅	✅	✅	✅
Rate limiting	✅	✅	✅	✅
OAuth/OIDC auth	✅	✅	✅	✅
WebSocket support	✅	✅	✅	✅
gRPC support	✅	✅	✅	✅
Canary deployments	✅	✅	✅	Limited
Custom error pages	✅	✅	✅	Limited
Request tracing	Via integration	Built-in	Via integration	AWS X-Ray

Gateway API: The Future

Network Policies: Microsegmentation

Network Policies are firewall rules at the pod level: you specify which ingress (incoming) and egress (outgoing) traffic is allowed. All other traffic is denied by the default-deny policy.

network-policies.yaml
YAML
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
# Default deny all ingress traffic to database pods
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: database-deny-all
  namespace: production
spec:
  podSelector:
    matchLabels:
      tier: database
  policyTypes:
  - Ingress
  # No ingress rules = deny all ingress
---
# Allow backend pods to access database pods on port 5432
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: database-allow-backend
  namespace: production
spec:
  podSelector:
    matchLabels:
      tier: database
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          tier: backend
    ports:
    - protocol: TCP
      port: 5432
---
# Allow frontend to reach backend API only
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-allow-frontend
  namespace: production
spec:
  podSelector:
    matchLabels:
      tier: backend
  policyTypes:
  - Ingress
  ingress:
  - from:
    - podSelector:
        matchLabels:
          tier: frontend
    - namespaceSelector:
        matchLabels:
          environment: production
    ports:
    - protocol: TCP
      port: 8080
---
# Restrict egress: backend can only reach database and external APIs
apiVersion: networking.k8s.io/v1
kind: NetworkPolicy
metadata:
  name: backend-egress
  namespace: production
spec:
  podSelector:
    matchLabels:
      tier: backend
  policyTypes:
  - Egress
  egress:
  - to:
    - podSelector:
        matchLabels:
          tier: database
    ports:
    - protocol: TCP
      port: 5432
  - to:
    - ipBlock:
        cidr: 0.0.0.0/0
        except:
        - 10.0.0.0/8    # Block internal IPs
        - 172.16.0.0/12
        - 192.168.0.0/16
    ports:
    - protocol: TCP
      port: 443

Network Policy enforcement:

Network Policies are enforced by the CNI plugin, not by Kubernetes itself. Not all CNI plugins support Network Policies:

CNI Plugin	Network Policy Support
Calico	✅ Full support (L3-L4)
Cilium	✅ Full + L7 policies
Weave	✅ Full support
Flannel	❌ No support*
AWS VPC CNI	✅ With Calico addon
Azure CNI	✅ With Azure Network Policy

*Flannel requires adding Calico for policy enforcement

Zero-Trust Architecture

Network Policy Best Practices

•Start with deny-all — Create namespace-wide default-deny policies first, then add specific allow rules.
•Label consistently — Network policies select pods by labels. Consistent labeling (tier, app, version) enables precise policies.
•Limit egress — Egress policies are often overlooked but critical. Restrict outbound traffic to prevent data exfiltration.
•Test policies before production — Use tools like kubectl-np-viewer or network policy simulators to verify behavior.
•Monitor policy enforcement — CNI plugins like Cilium provide metrics on allowed/denied connections.

Kubernetes DNS: CoreDNS

DNS records Kubernetes creates:

Kubernetes DNS Records
Resource	DNS Format	Record Type	Returns
Service	`<svc>.<ns>.svc.cluster.local`	A/AAAA	ClusterIP
Headless Service	`<svc>.<ns>.svc.cluster.local`	A/AAAA	All pod IPs
Pod (headless svc)	`<pod-ip-dashed>.<svc>.<ns>.svc.cluster.local`	A/AAAA	Pod IP
StatefulSet Pod	`<pod-name>.<svc>.<ns>.svc.cluster.local`	A/AAAA	Specific pod IP
Service Port	`_<port>._<proto>.<svc>.<ns>.svc.cluster.local`	SRV	Port + target
External (headless)	CNAME	CNAME	External hostname

dns-exploration.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
# View CoreDNS pods
kubectl get pods -n kube-system -l k8s-app=kube-dns
# NAME                       READY   STATUS    RESTARTS   AGE
# coredns-558bd4d5db-abcde   1/1     Running   0          15d
# coredns-558bd4d5db-fghij   1/1     Running   0          15d
 
# View CoreDNS ConfigMap
kubectl get configmap coredns -n kube-system -o yaml
 
# DNS resolution examples from a pod
kubectl run dns-test --rm -it --image=busybox:1.28 -- sh
 
# Within the same namespace
nslookup my-service
# Name:      my-service
# Address:   10.96.100.50
 
# Cross-namespace
nslookup postgres.database.svc.cluster.local
# Name:      postgres.database.svc.cluster.local
# Address:   10.96.200.100
 
# StatefulSet pods (stable DNS per pod)
nslookup postgres-0.postgres.database.svc.cluster.local
# Returns specific pod IP - crucial for stateful apps
 
# SRV records for port discovery
nslookup -query=SRV _http._tcp.my-service.default.svc.cluster.local
 
# External name service
nslookup external-db.default.svc.cluster.local
# Returns CNAME: rds.aws.amazon.com

Pod DNS configuration:

Each pod's /etc/resolv.conf is configured to use the cluster DNS:

nameserver 10.96.0.10     # CoreDNS ClusterIP
search default.svc.cluster.local svc.cluster.local cluster.local
options ndots:5

The search directive allows short names: my-service automatically expands to my-service.default.svc.cluster.local.

DNS Performance Tuning

CNI Plugins: Implementing the Network

Kubernetes doesn't implement pod networking itself—it delegates to CNI (Container Network Interface) plugins. A CNI plugin is responsible for:

Assigning IP addresses to pods
Programming routes so pods can communicate
Enforcing Network Policies (if supported)
Managing network namespaces and virtual interfaces

Popular CNI Plugins Comparison
CNI Plugin	Networking Mode	Network Policy	Notable Features
Calico	BGP (L3) or VXLAN	✅ Advanced	High performance; enterprise support; eBPF mode
Cilium	eBPF (kernel bypass)	✅ L3-L7	API-aware policies; observability; service mesh
Flannel	VXLAN overlay	❌	Simple; lightweight; requires Calico for policies
Weave	VXLAN + mesh	✅	Encryption; multicast; simple setup
AWS VPC CNI	Native VPC networking	✅ (addon)	Pod IPs from VPC; no overlay; high performance
Azure CNI	Native VNET	✅ (addon)	Pod IPs from VNET; Azure integration
Antrea	Open vSwitch	✅	VMware-backed; Windows support; observability

CNI plugin architectures:

Overlay networks (VXLAN, Geneve):

Encapsulate pod traffic in UDP packets
Traverse any underlay network (even across cloud regions)
Overhead: ~50 bytes per packet, slight CPU cost
Examples: Flannel VXLAN, Weave, Calico VXLAN mode

Routed networks (BGP):

Pod CIDRs advertised via BGP to network infrastructure
No encapsulation overhead
Requires BGP-capable network (or integration)
Examples: Calico BGP mode, MetalLB

Native cloud integration:

Pod IPs assigned from cloud VPC/VNET
Pods directly routable from cloud resources
Cloud IAM and security groups apply directly to pods
Examples: AWS VPC CNI, Azure CNI, GKE VPC-native

Choosing a CNI Plugin

Summary: Kubernetes Networking Mastery

We've covered Kubernetes networking comprehensively—from fundamental requirements to advanced policy enforcement.

Key Takeaways

•Flat network, no NAT — Every pod gets a routable IP; any pod can reach any other pod directly.
•Pods share network namespace — All containers in a pod have the same IP, communicate via localhost.
•Services abstract pod ephemerality — ClusterIP, NodePort, LoadBalancer provide stable endpoints.
•Ingress enables L7 routing — Hostname and path-based routing, TLS termination, advanced features.
•Network Policies enforce microsegmentation — Define which pods can communicate with which others.
•CoreDNS provides service discovery — Automatic DNS entries for Services and Pods.
•CNI plugins implement networking — Calico, Cilium, Flannel, etc. provide the actual network.

What's next:

Page Complete

3 / 5