Understanding Kubernetes Architecture

Kubernetes has become the de facto standard for container orchestration, powering workloads from the smallest startups to the largest enterprises and hyperscalers. But beneath its powerful abstractions lies a carefully designed distributed system with distinct components, each serving a critical purpose.

Understanding Kubernetes components isn't just academic knowledge—it's essential for debugging production issues, capacity planning, security hardening, and designing resilient architectures. When your application deployment fails, knowing whether the problem lies with the API Server, Scheduler, Controller Manager, or kubelet fundamentally changes your troubleshooting approach.

Kubernetes follows a control plane / data plane architecture pattern common in distributed systems. This separation provides clear boundaries between the components that make decisions and the components that execute work.

The Control Plane (also called the master components) is the brain of the cluster. It maintains the desired state, makes scheduling decisions, responds to cluster events, and exposes the API. The control plane doesn't run your application workloads—it orchestrates them.

The Data Plane (the worker nodes) is where your containers actually run. Each node hosts the components necessary to run Pods and communicate with the control plane. The data plane executes the decisions made by the control plane.

Why this separation matters:

This architectural division enables several critical capabilities:

1. Scalability: Control plane and data plane can scale independently. You can add hundreds of worker nodes without proportionally scaling control plane components.

2. Isolation: Control plane failures don't immediately kill running workloads. If the API server goes down, existing Pods continue running—you just can't make changes.

3. Security: The control plane can be isolated in separate network segments, reducing attack surface. Worker nodes only need limited access to specific control plane endpoints.

4. Maintenance: Control plane components can be upgraded or restarted with minimal impact on running workloads.

The kube-apiserver is the central hub of all Kubernetes communication. Every interaction with the cluster—whether from kubectl, controllers, the scheduler, or kubelets—goes through the API server. It's not just a gateway; it's the single source of truth for the cluster's current state.

Core Responsibilities:

1. API Endpoint Exposure: Serves the Kubernetes REST API over HTTPS, handling CRUD operations for all Kubernetes objects (Pods, Services, ConfigMaps, etc.)

2. Authentication & Authorization: Validates the identity of all requests (via certificates, tokens, etc.) and enforces RBAC policies to determine what actions are permitted.

3. Admission Control: Runs admission controllers that can mutate or validate requests before they're persisted. This is where policies like resource quotas, pod security standards, and webhook-based validations are enforced.

4. etcd Gateway: Serves as the only component that directly communicates with etcd. All cluster state changes go through the API server to etcd.

5. Watch Mechanism: Supports efficient watches that allow clients to subscribe to changes rather than polling. This is how controllers learn about new or modified resources.

Scalability Characteristics:

The API server is designed to be horizontally scalable. In high-availability setups, multiple API server instances run behind a load balancer. Because all state is stored in etcd, any API server instance can handle any request. This stateless design is critical for production deployments.

Key Performance Considerations:

• Request throttling: The API server implements priority and fairness to prevent any single client from overwhelming the server
• Watch efficiency: Long-running watch connections allow clients to receive events without constant polling
• Caching: Frequently accessed data is cached to reduce etcd load
• Index-based queries: Efficient label selectors use indexes for fast lookups

etcd is a distributed, consistent key-value store that serves as Kubernetes' backing store for all cluster data. Every object you create—every Pod, Service, ConfigMap, Secret—is persisted in etcd. It's not just storage; it's the foundation of Kubernetes' consistency guarantees.

Why etcd?

Kubernetes requires a storage system with very specific properties:

1. Strong Consistency: When the API server writes a Pod spec, that write must be immediately visible to all readers. etcd provides this through the Raft consensus algorithm.

2. Watch Support: Controllers need to react to changes. etcd's native watch capability allows efficient event notification without polling.

3. Distributed & Fault-Tolerant: etcd can tolerate node failures while maintaining consistency. A 3-node cluster survives 1 failure; a 5-node cluster survives 2.

4. Transactional Operations: Compare-and-swap operations enable safe concurrent updates—critical for controllers competing to update resources.

Data Organization in etcd:

Kubernetes organizes data in etcd using a hierarchical key structure:

/registry/pods/<namespace>/<pod-name>
/registry/services/<namespace>/<service-name>
/registry/secrets/<namespace>/<secret-name>
/registry/deployments/<namespace>/<deployment-name>

This organization enables efficient prefix-based watches (watch all pods in a namespace) and range queries.

The Raft Consensus Algorithm:

etcd uses Raft for distributed consensus. Here's how it works at a high level:

1. Leader Election: One node is elected leader; all writes go through the leader
2. Log Replication: The leader replicates entries to followers before committing
3. Commit: Once a majority acknowledges, the entry is committed and applied
4. Failover: If the leader fails, a new election occurs within milliseconds

Performance Tuning Considerations:

• Disk I/O: etcd is write-intensive; use SSDs with low latency. Network-attached storage can introduce unacceptable delays.
• Network Latency: Keep etcd nodes close (same availability zone ideally). Cross-AZ latency impacts commit times.
• Compaction: etcd stores historical revisions; periodic compaction prevents unbounded storage growth.
• Resource Isolation: Run etcd on dedicated nodes in large clusters to prevent resource contention with other workloads.

The kube-scheduler watches for newly created Pods that have no assigned node and selects an appropriate node for them to run on. This is a non-trivial problem—the scheduler must consider resource requirements, affinity/anti-affinity rules, taints and tolerations, data locality, and many other factors.

The Scheduling Process:

Scheduling happens in two phases:

1. Filtering Phase (Predicates) The scheduler filters out nodes that cannot run the Pod:

• Does the node have sufficient CPU and memory?
• Does the Pod tolerate the node's taints?
• Does the node satisfy node affinity requirements?
• Are required ports available?
• Does the node's labels match node selectors?

2. Scoring Phase (Priorities) Among feasible nodes, the scheduler scores each to find the optimal placement:

• Balanced resource utilization
• Data locality (volume zones)
• Pod anti-affinity spread
• Custom priority functions

Scheduler Extensibility:

The default scheduler can be extended through several mechanisms:

• Scheduling Profiles: Configure multiple scheduling behaviors and select per-Pod
• Scheduler Plugins: The scheduling framework allows injecting custom logic at various extension points
• Custom Schedulers: You can run completely separate schedulers alongside the default one

Common Scheduling Failures:

The kube-controller-manager runs a collection of controllers that watch the cluster state and work to move the current state toward the desired state. This is the heart of Kubernetes' declarative model—you specify what you want, and controllers make it happen.

The Controller Pattern:

Every controller follows the same pattern:

1. Watch: Subscribe to relevant resource changes via the API server
2. Observe: When changes occur, read the current state
3. Compare: Calculate the difference between current and desired states
4. Act: Take actions to reconcile the difference
5. Repeat: Continue the loop indefinitely

Key Controllers Bundled in kube-controller-manager:

Reconciliation in Action—ReplicaSet Example:

Consider a ReplicaSet with replicas: 3:

1. The ReplicaSet controller watches ReplicaSets and Pods
2. When notified, it counts Pods matching the ReplicaSet's selector
3. If count < 3: Create (3 - count) new Pods
4. If count > 3: Delete (count - 3) excess Pods
5. If count == 3: No action needed

This loop runs continuously. If a Pod dies, the controller creates a replacement. If you scale to 5, the controller creates 2 more. The desired state is always maintained.

Leader Election:

In HA setups with multiple control plane nodes, only one instance of kube-controller-manager actively runs controllers—the leader. Others are on standby. If the leader fails, another instance acquires the leader lock and takes over. This prevents conflicting actions from multiple controllers.

The cloud-controller-manager (CCM) contains controllers that interact with cloud provider APIs. This component was extracted from kube-controller-manager to allow cloud providers to develop their integration at their own pace without being tied to Kubernetes release cycles.

Controllers in the CCM:

1. Node Controller (Cloud Portion)

   • Initializes nodes with cloud-specific metadata (zone, instance type, etc.)
   • Detects when cloud instances are deleted and removes corresponding Node objects
   • Updates node addresses from cloud provider

2. Route Controller

   • Configures cloud provider routes for inter-node networking
   • Ensures Pod network routes exist between nodes

3. Service Controller

   • Provisions cloud load balancers for type: LoadBalancer services
   • Updates load balancer configs as service endpoints change
   • Cleans up load balancers when services are deleted

Provider Implementations:

Each major cloud provider maintains their own CCM:

• AWS: cloud-provider-aws
• GCP: cloud-provider-gcp
• Azure: cloud-provider-azure

Running Without CCM:

On bare-metal or on-premises deployments, you typically don't run a CCM. Without it:

• LoadBalancer services stay in Pending state (use MetalLB or similar for bare-metal LB)
• Node zone/region labels must be applied manually
• No automatic cloud resource cleanup

The kubelet is the primary node agent—it runs on every worker node and is responsible for ensuring containers are running as specified. It's the component that actually makes Pods come to life.

Core Responsibilities:

1. Pod Lifecycle Management

   • Receives Pod specs (via API server or static Pod files)
   • Instructs container runtime to create/start/stop containers
   • Monitors container health and restarts failed containers
   • Reports Pod and container status back to API server

2. Volume Management

   • Mounts required volumes before containers start
   • Handles volume plugins (CSI drivers, etc.)
   • Unmounts volumes when Pods terminate

3. Resource Enforcement

   • Configures cgroups to enforce resource limits
   • Triggers eviction when node resources are exhausted
   • Reports node capacity and allocatable resources

4. Health Monitoring

   • Executes liveness, readiness, and startup probes
   • Updates container status based on probe results
   • Removes failed containers and signals for replacement

Container Runtime Interface (CRI):

kubelet doesn't run containers directly—it delegates to a container runtime via CRI. This abstraction allows swapping runtimes:

Pod Eviction:

When node resources are exhausted, kubelet evicts Pods based on priority:

1. BestEffort Pods (no requests/limits) are evicted first
2. Burstable Pods (requests < limits) are next
3. Guaranteed Pods (requests == limits) are evicted last

kube-proxy runs on every node and implements the Kubernetes Service concept. When you create a Service, kube-proxy configures the node's networking rules to direct traffic to the appropriate Pods.

What kube-proxy Does:

1. Watches Services and Endpoints: Subscribes to changes via API server
2. Configures Packet Forwarding: Sets up rules to intercept service ClusterIP traffic and forward to Pod IPs
3. Load Balances: Distributes traffic across endpoint Pods
4. Handles NodePort Services: Opens ports on nodes for external access

Operating Modes:

kube-proxy can operate in different modes with significant performance implications:

How Service Traffic Flows (iptables mode):

When a Pod accesses a ClusterIP service:

1. Pod sends packet to 10.96.0.1:80 (service ClusterIP)
2. iptables rule intercepts the packet (DNAT)
3. Random Pod endpoint is selected (e.g., 10.244.1.5:8080)
4. Packet destination is rewritten to the selected Pod IP
5. Packet is routed to the target Pod
6. Return traffic follows conntrack for consistent source NAT

IPVS Mode Advantages:

IPVS uses hash tables for O(1) rule lookup versus O(n) iptables chain traversal:

• Better performance with thousands of services
• Multiple load balancing algorithms (round-robin, least connections, etc.)
• Kernel-level connection tracking
• Better metrics and observability

The container runtime is the software responsible for actually running containers. While often overlooked, choosing and understanding your container runtime affects security, performance, and compatibility.

The OCI Standard:

The Open Container Initiative (OCI) defines standards for container images and runtimes:

• Image Spec: Container image format
• Runtime Spec: How to run a container (process, mounts, cgroups, etc.)

OCI compliance means different runtimes are interchangeable at the low level.

Runtime Layers:

Container runtimes exist in layers:

kubelet
    ↓ (CRI - Container Runtime Interface)
High-Level Runtime (containerd, CRI-O)
    ↓ (OCI Runtime Spec)
Low-Level Runtime (runc, crun, kata-runtime)
    ↓
Linux Kernel (namespaces, cgroups, seccomp)

containerd Architecture:

containerd (the most common high-level runtime) provides:

• Image pull and push: Registry interaction, layer deduplication
• Snapshot management: Copy-on-write filesystem layers
• Container lifecycle: Create, start, stop, pause, resume
• Content storage: Image and layer caching
• Namespace isolation: Multiple independent container groups

Security Runtimes:

For enhanced isolation (multi-tenant clusters, untrusted code), specialized runtimes provide stronger boundaries:

• gVisor: Intercepts syscalls in userspace, providing a security boundary without full VM overhead
• Kata Containers: Runs each Pod in a lightweight VM, hardware-level isolation
• Firecracker: MicroVM technology (AWS Lambda/Fargate), millisecond boot times

Now that we understand each component individually, let's trace a complete workflow to see how they work together. We'll follow the lifecycle of a Deployment from creation to running Pods.

Step-by-Step: Creating a Deployment

Key Observations:

1. etcd is touched only by the API server: All other components communicate via API server watches

2. Controllers are event-driven but level-triggered: They react to changes but always reconcile based on current state, not event history

3. No component directly calls another: Communication is via shared state in etcd, accessed through the API server

4. Asynchronous by design: Each controller operates independently; there's no synchronous orchestration

5. Self-healing emerges from reconciliation: If a Pod dies, ReplicaSet controller notices and creates a replacement—no central coordinator needed

We've explored every core component of the Kubernetes architecture. Let's consolidate the key takeaways:

What's Next:

Now that you understand the components, the next page dives into the core Kubernetes objects—Pods, Deployments, and Services. You'll see how these abstractions build on the component architecture to provide powerful application management capabilities.