Network Software - Learning Module

Loading content...

0/228

Operating System Support

The OS: The Network's Invisible Conductor

Behind every network application—from the simplest curl command to the most sophisticated distributed system—stands the operating system, providing the essential infrastructure that makes network communication possible. The OS manages network interfaces, implements protocol stacks, multiplexes connections, enforces security policies, and provides the APIs that applications use to communicate.

Without OS networking support, applications would need to directly control hardware, implement their own TCP/IP stacks, and handle the complexity of managing concurrent connections—an impractical burden that would make network programming nearly impossible. This page explores the deep integration between operating systems and networking, revealing the machinery that makes modern networked computing work.

What You Will Learn

By the end of this page, you will understand how operating systems implement network stacks, the role of system calls in network operations, how the kernel manages network state and resources, virtual networking concepts including network namespaces and virtual interfaces, and how modern containerization leverages OS networking features.

The Kernel Network Stack

The kernel network stack is the core of the operating system's networking capability. It implements the protocol layers (TCP/IP), manages connections, handles packet routing, and interfaces with network device drivers.

Why Networking Lives in the Kernel:

Performance: Kernel code runs with direct hardware access and optimized memory operations. Context switching between user and kernel space is expensive; keeping hot paths in the kernel minimizes overhead.
Resource Management: The kernel must arbitrate network resources among multiple applications—allocating ports, managing buffers, scheduling packet transmission.
Security: Network operations require controlled access. The kernel enforces permissions, preventing unauthorized applications from sniffing traffic or binding to privileged ports.
Shared State: Routing tables, ARP caches, and connection state must be globally consistent. The kernel provides a single source of truth.
Hardware Abstraction: Different NICs require different drivers, but applications need a uniform interface. The kernel provides this abstraction.

Converting Mermaid diagram...

Major Components of the Linux Network Stack:

Socket Layer:

Implements BSD socket API
Manages file descriptors for network connections
Demultiplexes data to correct sockets

Transport Protocols:

TCP: connection-oriented, reliable, ordered
UDP: connectionless, unreliable, fast
SCTP: multi-homing, multi-streaming
Raw sockets for custom protocols

IP Layer:

IPv4 and IPv6 implementation
Fragmentation and reassembly
ICMP handling

Routing Subsystem:

Forwarding Information Base (FIB)
Policy routing
Multipath routing
Route caching

Netfilter:

Packet filtering (iptables, nftables)
NAT
Connection tracking
Packet mangling

Network Device Layer (netdev):

Interface abstraction
Queue discipline (qdisc) for traffic shaping
Multiple transmit queues (XPS)
Receive packet steering (RPS)

System Calls for Networking

Applications communicate with the kernel network stack through system calls (syscalls)—controlled entry points that transition from user mode to kernel mode. Network-specific syscalls implement the socket API.

The Socket Lifecycle:

socket() → bind() → listen() → accept() → recv()/send() → close()

Each step involves a syscall that transitions to kernel mode, performs operations on kernel data structures, and returns results to user space.

Essential Network System Calls
System Call	Purpose	Key Parameters
socket()	Create a communication endpoint	domain (AF_INET, AF_UNIX), type (SOCK_STREAM, SOCK_DGRAM), protocol
bind()	Assign address to socket	sockfd, address structure, address length
listen()	Mark socket as passive (accept connections)	sockfd, backlog (queue size)
accept()	Accept incoming connection	sockfd, client address (output), address length
connect()	Initiate connection to server	sockfd, server address, address length
send()/write()	Transmit data	sockfd, buffer, length, flags
recv()/read()	Receive data	sockfd, buffer, length, flags
close()	Close socket	file descriptor
setsockopt()	Configure socket options	sockfd, level, option, value, length
select()/poll()/epoll	I/O multiplexing	file descriptor sets, timeout

System Call Overhead:

Every syscall incurs overhead:

Mode Switch: CPU transitions from user mode to kernel mode (~100-500 CPU cycles)
Context Save: User registers saved; kernel stack activated
Security Checks: Kernel validates permissions and parameters
Execution: Kernel performs the requested operation
Context Restore: User state restored; return to user mode

For high-frequency operations, this overhead matters. This is why modern systems provide:

epoll — Amortize syscall cost across many connections
io_uring — Batched syscalls through shared ring buffers
sendmmsg/recvmmsg — Send/receive multiple messages in one syscall
Kernel bypass — Eliminate syscalls entirely for extreme performance

socket_lifecycle.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
/* Complete socket lifecycle demonstration */
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
 
int main() {
    // 1. Create socket - returns file descriptor
    int sockfd = socket(AF_INET,     // IPv4
                        SOCK_STREAM,  // TCP
                        0);           // Protocol (auto)
    
    // 2. Bind to address
    struct sockaddr_in addr = {
        .sin_family = AF_INET,
        .sin_port = htons(8080),
        .sin_addr.s_addr = INADDR_ANY
    };
    bind(sockfd, (struct sockaddr*)&addr, sizeof(addr));
    
    // 3. Listen for connections (backlog = 128)
    listen(sockfd, 128);
    
    // 4. Accept connection
    struct sockaddr_in client_addr;
    socklen_t client_len = sizeof(client_addr);
    int client_fd = accept(sockfd, 
                           (struct sockaddr*)&client_addr,
                           &client_len);
    
    // 5. Send and receive data
    char buffer[4096];
    ssize_t n = recv(client_fd, buffer, sizeof(buffer), 0);
    if (n > 0) {
        send(client_fd, buffer, n, 0);  // Echo
    }
    
    // 6. Close connections
    close(client_fd);
    close(sockfd);
    
    return 0;
}

File Descriptor Unification

Unix's 'everything is a file' philosophy extends to sockets. After socket() returns, the socket is accessed through a file descriptor, using familiar operations like read(), write(), and close(). This unification enables powerful patterns like redirecting socket data to files or using select() to wait on both file and network I/O.

Kernel Buffer Management

The kernel manages vast amounts of memory for network operations—buffers for packet storage, queues for pending transmissions, and data structures for connection state. Efficient buffer management is critical for network performance.

The Socket Buffer (sk_buff):

In Linux, the sk_buff (socket buffer) is the fundamental data structure for network packets. Every packet flowing through the stack is wrapped in an sk_buff that contains:

Data pointers: Head, data, tail, end—allowing efficient header prepending/stripping
Packet metadata: Protocol, device, timestamps, checksums
Control information: Routing decisions, fragmentation state
Queue linkage: Pointers for list management

Converting Mermaid diagram...

Buffer Lifecycle:

Receive Path:

Driver allocates sk_buff from pool
DMA fills buffer with packet data
Driver sets up metadata and passes to netdev layer
Packet moves up the stack; headers stripped, data pointer advances
Finally delivered to socket receive queue
Application reads data; sk_buff freed

Transmit Path:

Application writes data; kernel allocates sk_buff
Transport layer adds header (TCP/UDP), data pointer moves back
Network layer adds IP header
Link layer adds Ethernet header
Driver queues for transmission
After transmission complete, sk_buff freed

Socket Buffers (Different Concept):

Confusingly, there are also socket buffers—per-socket kernel memory for buffering data:

Receive buffer (SO_RCVBUF): Holds incoming data until application reads it
Send buffer (SO_SNDBUF): Holds outgoing data until acknowledged (for TCP)

These buffers determine how much data can be 'in flight' and affect throughput for high-latency links.

Key Buffer-Related Socket Options
Option	Purpose	Typical Values	Impact
SO_RCVBUF	Receive buffer size	64KB - 16MB	Affects receive window, throughput on high-latency links
SO_SNDBUF	Send buffer size	64KB - 16MB	Limits data in flight, affects write() behavior
TCP_NODELAY	Disable Nagle algorithm	0 or 1	Reduces latency for small writes
SO_RCVLOWAT	Minimum receive data	1 (default)	recv() blocks until this much data available
SO_SNDLOWAT	Minimum send space	1 (default)	send() blocks until this much buffer available

Buffer Bloat

Excessively large buffers can cause 'buffer bloat'—high latency as packets queue behind backed-up traffic. This is counterintuitive: more buffering reduces throughput and responsiveness. Modern systems use active queue management (CoDel, FQ-CoDel) to balance throughput and latency.

Network Configuration and Interfaces

The operating system exposes network configuration through multiple interfaces, allowing administrators and applications to view and modify network state.

Configuration Mechanisms:

Traditional Tools (Linux):

ifconfig — Legacy interface configuration
route — Legacy routing table management
netstat — Network statistics and connections

Modern Tools (iproute2):

ip link — Interface configuration
ip addr — Address management
ip route — Routing tables
ip neigh — ARP/ND cache
ss — Socket statistics (replaces netstat)

Configuration Files:

/etc/network/interfaces (Debian)
/etc/sysconfig/network-scripts/ (RHEL)
NetworkManager, systemd-networkd for dynamic management

network_commands.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# Modern Linux network configuration examples
 
# View all interfaces
ip link show
 
# Set interface up/down
ip link set eth0 up
ip link set eth0 down
 
# Add IP address
ip addr add 192.168.1.100/24 dev eth0
 
# View routing table
ip route show
 
# Add default route
ip route add default via 192.168.1.1
 
# View ARP cache
ip neigh show
 
# View open sockets
ss -tuln
 
# View TCP connections with process info
ss -tp
 
# View network statistics
ip -s link show eth0
 
# Create VLAN interface
ip link add link eth0 name eth0.100 type vlan id 100
 
# Create bridge
ip link add br0 type bridge
ip link set eth0 master br0

Programmatic Configuration:

Applications can configure networking through:

Netlink Sockets:

Primary kernel-userspace communication channel for network configuration
Used by ip commands internally
Supports routing, interfaces, addresses, firewall rules
Allows monitoring (rtnetlink for route changes)

ioctl():

Legacy interface still supported
SIOCGIFADDR, SIOCSIFADDR for addresses
Being replaced by netlink

sysfs and procfs:

/sys/class/net/<if>/ — Interface parameters
/proc/net/ — Statistics, connection tables
Read-only for most information

Configuration Persistence:

Kernel network state is volatile—it's lost on reboot. Persistence requires:

Configuration files read at boot
Network management daemons (NetworkManager, systemd-networkd)
Script-based initialization

Network Namespaces Preview

Linux network namespaces allow multiple independent network stacks on one machine. Each namespace has its own interfaces, routing tables, and firewall rules. This is fundamental to container networking—each container gets its own namespace, appearing to have dedicated network hardware.

Virtual Network Devices

Modern operating systems provide virtual network devices—software-based network interfaces that behave like physical NICs but exist entirely in software. These enable powerful networking topologies for virtualization, containers, and testing.

Types of Virtual Network Devices:

Linux Virtual Network Devices

•Loopback (lo) — Traffic never leaves the host; used for local inter-process communication. Always exists, typically 127.0.0.1/::1.
•Bridge (br0) — Software Layer 2 switch connecting multiple interfaces. VMs and containers attach to bridges to communicate.
•veth (Virtual Ethernet) — Paired virtual interfaces; packets sent to one appear on the other. Used to connect network namespaces.
•TUN/TAP — User-space network drivers. TUN for IP packets (VPNs), TAP for Ethernet frames (VM networking).
•macvlan/ipvlan — Create virtual interfaces with unique MAC (macvlan) or shared MAC (ipvlan) off a parent interface.
•VXLAN/GENEVE — Overlay networks tunneling L2 over L3. Used in cloud/datacenter networking.
•dummy — Loopback-like interface for testing or as anchor for addresses.

Converting Mermaid diagram...

Virtual Device Use Cases:

Container Networking: Containers typically have:

Own network namespace with private network stack
veth pair connecting to host bridge
NAT or routing to reach external networks
This is what Docker, Kubernetes, and other container runtimes do

Virtual Machine Networking: VMs typically have:

TAP device backing their virtual NIC
TAP connected to host bridge
Bridge connected to physical NIC or used for host-only networking

VPN Implementation: VPNs typically use:

TUN device receiving decrypted packets from VPN software
Routing table directing traffic through TUN
VPN software encrypts and sends via physical network

Testing and Development:

Create complex topologies without hardware
Network namespaces simulate multi-host environments
Inject failures, delays, packet loss with netem

Performance Considerations

Virtual devices add overhead—packets traverse additional layers of kernel code. For performance-critical workloads, consider: macvlan/ipvlan (lower overhead than bridge), SR-IOV for VMs (hardware-assisted virtualization), or kernel bypass (DPDK in containers with specialized drivers).

Network Namespaces

Network namespaces are a Linux kernel feature that provides isolated network stacks. Each namespace has independent:

Network interfaces
IP addresses
Routing tables
Firewall rules (iptables/nftables)
Socket bindings
ARP/neighbor cache

Processes in different namespaces see completely different network environments, even on the same physical machine. This is the foundation of container networking.

Namespace Operations:

namespace_operations.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# Create a new network namespace
ip netns add myns
 
# List namespaces
ip netns list
 
# Run command in namespace
ip netns exec myns ip addr show
 
# Create veth pair to connect namespaces
ip link add veth0 type veth peer name veth1
 
# Move one end into namespace
ip link set veth1 netns myns
 
# Configure interfaces
ip addr add 10.0.0.1/24 dev veth0
ip link set veth0 up
 
ip netns exec myns ip addr add 10.0.0.2/24 dev veth1
ip netns exec myns ip link set veth1 up
ip netns exec myns ip link set lo up
 
# Now 10.0.0.1 can ping 10.0.0.2
ping 10.0.0.2
 
# The namespace can ping back
ip netns exec myns ping 10.0.0.1
 
# Delete namespace (and all its interfaces)
ip netns delete myns

How Containers Use Namespaces:

When you run a container:

Container runtime creates namespace — New, empty network stack
Creates veth pair — Links container namespace to host
Configures container interface — Assigns IP, sets up routes
Connects host side — Usually to a bridge (docker0, cni0)
Sets up NAT/routing — For external connectivity
Container process runs — In the new namespace, sees only its interfaces

The container believes it has its own dedicated network stack—because, from its perspective, it does.

Beyond Containers:

Network namespaces are useful for:

Testing: Simulate multi-host environments on one machine
Security: Isolate network-facing services
VPN routing: Run specific apps through VPN without affecting others
Network function development: Test routing, firewalling without hardware

Namespace Persistence

Network namespaces persist as long as at least one process uses them OR they're mounted to a file (ip netns creates /var/run/netns/<name>). When all processes exit and the mount is removed, the namespace and all its interfaces are destroyed. This automatic cleanup is one of the features that makes containers practical.

Traffic Control and QoS

Operating systems provide traffic control (TC) capabilities to manage how packets are scheduled, shaped, and prioritized. This enables Quality of Service (QoS) policies that ensure critical traffic gets bandwidth while limiting less important traffic.

The Linux Traffic Control (tc) Subsystem:

Linux implements traffic control through three components:

Queuing Disciplines (qdiscs) — Algorithms that decide packet transmission order
Classes — Divisions within qdiscs for categorizing traffic
Filters — Rules that classify packets into classes

Common Queuing Disciplines
Qdisc	Type	Purpose	Use Case
pfifo_fast	Classless	Default; 3 priority bands	General purpose
fq_codel	Classless	Fair queuing with controlled delay	Reducing buffer bloat
htb	Classful	Hierarchical token bucket	Bandwidth allocation, guarantees
tbf	Classless	Token bucket filter	Rate limiting
prio	Classful	Priority queuing	Prioritize interactive traffic
netem	Classless	Network emulation	Testing: add delay, loss, reordering
cake	Classless	Common Applications KE	Home routers, buffer bloat

traffic_control_examples.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# View current qdisc
tc qdisc show dev eth0
 
# Add rate limiting (1 Mbit)
tc qdisc add dev eth0 root tbf rate 1mbit burst 32kbit latency 400ms
 
# Add fair queuing with controlled delay (reduce buffer bloat)
tc qdisc replace dev eth0 root fq_codel
 
# Add network emulation (100ms delay, 1% loss)
tc qdisc add dev eth0 root netem delay 100ms loss 1%
 
# Hierarchical Token Bucket for bandwidth allocation
tc qdisc add dev eth0 root handle 1: htb default 30
 
# Create classes
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit  # High priority
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 30mbit  # Normal
tc class add dev eth0 parent 1:1 classid 1:30 htb rate 20mbit  # Low priority
 
# Add filters to classify traffic
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 \
    match ip dport 22 0xffff flowid 1:10  # SSH to high priority
 
# Delete qdisc
tc qdisc del dev eth0 root

QoS Use Cases:

Enterprise Networks:

Voice/video traffic prioritized over bulk data
Critical business applications guaranteed bandwidth
Guest WiFi limited to prevent interference

Service Providers:

Customer rate limiting per contract
Premium tier prioritization
Fair usage policies

Home Networks:

Gaming traffic low-latency
Video streaming vs. downloads
Work-from-home traffic protection

Container Orchestration:

Per-pod bandwidth limits
Priority classes for critical workloads
Fair sharing between tenants

TC Complexity

The tc subsystem is extremely powerful but has notoriously complex syntax and semantics. Incorrect configuration can completely break networking. In production, prefer higher-level tools (systemd, container runtimes, network management APIs) that generate correct tc rules. Use tc directly only when you need fine-grained control.

Summary: Operating System Support

We've explored the deep integration between operating systems and networking—from kernel architecture to the abstractions that enable modern cloud-native applications.

Key Takeaways

•The kernel network stack is central — It implements protocols, manages state, and provides the abstraction layer between applications and hardware.
•System calls are the API — Applications interact with networking through socket system calls, trading user/kernel transitions for kernel services.
•Buffer management is critical — sk_buff structures and socket buffers directly impact performance and must be sized appropriately.
•Configuration has many interfaces — From legacy ifconfig to modern ip commands to netlink sockets, multiple paths exist to configure networking.
•Virtual devices enable virtualization — Bridges, veths, TUN/TAP, and overlays create the networking fabric for VMs and containers.
•Network namespaces provide isolation — Each namespace has independent network state, enabling container networking and security isolation.
•Traffic control enables QoS — The tc subsystem provides powerful traffic shaping, though with complexity that warrants higher-level tools.

What's Next:

With an understanding of how operating systems support networking, the next page examines Network Services—the infrastructure services like DNS, DHCP, and time synchronization that applications depend on. These services operate at the system level, providing foundational capabilities that higher-level applications take for granted.

Page Complete

You now understand how operating systems implement network stacks, the role of system calls in network operations, kernel buffer management, network configuration mechanisms, virtual network devices and namespaces, and traffic control for QoS. This knowledge is essential for debugging network issues, optimizing performance, and working with containers and virtualization.

Operating System Support

The OS: The Network's Invisible Conductor

What You Will Learn

The Kernel Network Stack

Why Networking Lives in the Kernel:

Performance: Kernel code runs with direct hardware access and optimized memory operations. Context switching between user and kernel space is expensive; keeping hot paths in the kernel minimizes overhead.
Resource Management: The kernel must arbitrate network resources among multiple applications—allocating ports, managing buffers, scheduling packet transmission.
Security: Network operations require controlled access. The kernel enforces permissions, preventing unauthorized applications from sniffing traffic or binding to privileged ports.
Shared State: Routing tables, ARP caches, and connection state must be globally consistent. The kernel provides a single source of truth.
Hardware Abstraction: Different NICs require different drivers, but applications need a uniform interface. The kernel provides this abstraction.

Converting Mermaid diagram...

Major Components of the Linux Network Stack:

Socket Layer:

Implements BSD socket API
Manages file descriptors for network connections
Demultiplexes data to correct sockets

Transport Protocols:

TCP: connection-oriented, reliable, ordered
UDP: connectionless, unreliable, fast
SCTP: multi-homing, multi-streaming
Raw sockets for custom protocols

IP Layer:

IPv4 and IPv6 implementation
Fragmentation and reassembly
ICMP handling

Routing Subsystem:

Forwarding Information Base (FIB)
Policy routing
Multipath routing
Route caching

Netfilter:

Packet filtering (iptables, nftables)
NAT
Connection tracking
Packet mangling

Network Device Layer (netdev):

Interface abstraction
Queue discipline (qdisc) for traffic shaping
Multiple transmit queues (XPS)
Receive packet steering (RPS)

System Calls for Networking

The Socket Lifecycle:

socket() → bind() → listen() → accept() → recv()/send() → close()

Each step involves a syscall that transitions to kernel mode, performs operations on kernel data structures, and returns results to user space.

Essential Network System Calls
System Call	Purpose	Key Parameters
socket()	Create a communication endpoint	domain (AF_INET, AF_UNIX), type (SOCK_STREAM, SOCK_DGRAM), protocol
bind()	Assign address to socket	sockfd, address structure, address length
listen()	Mark socket as passive (accept connections)	sockfd, backlog (queue size)
accept()	Accept incoming connection	sockfd, client address (output), address length
connect()	Initiate connection to server	sockfd, server address, address length
send()/write()	Transmit data	sockfd, buffer, length, flags
recv()/read()	Receive data	sockfd, buffer, length, flags
close()	Close socket	file descriptor
setsockopt()	Configure socket options	sockfd, level, option, value, length
select()/poll()/epoll	I/O multiplexing	file descriptor sets, timeout

System Call Overhead:

Every syscall incurs overhead:

Mode Switch: CPU transitions from user mode to kernel mode (~100-500 CPU cycles)
Context Save: User registers saved; kernel stack activated
Security Checks: Kernel validates permissions and parameters
Execution: Kernel performs the requested operation
Context Restore: User state restored; return to user mode

For high-frequency operations, this overhead matters. This is why modern systems provide:

epoll — Amortize syscall cost across many connections
io_uring — Batched syscalls through shared ring buffers
sendmmsg/recvmmsg — Send/receive multiple messages in one syscall
Kernel bypass — Eliminate syscalls entirely for extreme performance

socket_lifecycle.c
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
/* Complete socket lifecycle demonstration */
#include <sys/socket.h>
#include <netinet/in.h>
#include <unistd.h>
 
int main() {
    // 1. Create socket - returns file descriptor
    int sockfd = socket(AF_INET,     // IPv4
                        SOCK_STREAM,  // TCP
                        0);           // Protocol (auto)
    
    // 2. Bind to address
    struct sockaddr_in addr = {
        .sin_family = AF_INET,
        .sin_port = htons(8080),
        .sin_addr.s_addr = INADDR_ANY
    };
    bind(sockfd, (struct sockaddr*)&addr, sizeof(addr));
    
    // 3. Listen for connections (backlog = 128)
    listen(sockfd, 128);
    
    // 4. Accept connection
    struct sockaddr_in client_addr;
    socklen_t client_len = sizeof(client_addr);
    int client_fd = accept(sockfd, 
                           (struct sockaddr*)&client_addr,
                           &client_len);
    
    // 5. Send and receive data
    char buffer[4096];
    ssize_t n = recv(client_fd, buffer, sizeof(buffer), 0);
    if (n > 0) {
        send(client_fd, buffer, n, 0);  // Echo
    }
    
    // 6. Close connections
    close(client_fd);
    close(sockfd);
    
    return 0;
}

File Descriptor Unification

Kernel Buffer Management

The Socket Buffer (sk_buff):

In Linux, the sk_buff (socket buffer) is the fundamental data structure for network packets. Every packet flowing through the stack is wrapped in an sk_buff that contains:

Data pointers: Head, data, tail, end—allowing efficient header prepending/stripping
Packet metadata: Protocol, device, timestamps, checksums
Control information: Routing decisions, fragmentation state
Queue linkage: Pointers for list management

Converting Mermaid diagram...

Buffer Lifecycle:

Receive Path:

Driver allocates sk_buff from pool
DMA fills buffer with packet data
Driver sets up metadata and passes to netdev layer
Packet moves up the stack; headers stripped, data pointer advances
Finally delivered to socket receive queue
Application reads data; sk_buff freed

Transmit Path:

Application writes data; kernel allocates sk_buff
Transport layer adds header (TCP/UDP), data pointer moves back
Network layer adds IP header
Link layer adds Ethernet header
Driver queues for transmission
After transmission complete, sk_buff freed

Socket Buffers (Different Concept):

Confusingly, there are also socket buffers—per-socket kernel memory for buffering data:

Receive buffer (SO_RCVBUF): Holds incoming data until application reads it
Send buffer (SO_SNDBUF): Holds outgoing data until acknowledged (for TCP)

These buffers determine how much data can be 'in flight' and affect throughput for high-latency links.

Key Buffer-Related Socket Options
Option	Purpose	Typical Values	Impact
SO_RCVBUF	Receive buffer size	64KB - 16MB	Affects receive window, throughput on high-latency links
SO_SNDBUF	Send buffer size	64KB - 16MB	Limits data in flight, affects write() behavior
TCP_NODELAY	Disable Nagle algorithm	0 or 1	Reduces latency for small writes
SO_RCVLOWAT	Minimum receive data	1 (default)	recv() blocks until this much data available
SO_SNDLOWAT	Minimum send space	1 (default)	send() blocks until this much buffer available

Buffer Bloat

Network Configuration and Interfaces

The operating system exposes network configuration through multiple interfaces, allowing administrators and applications to view and modify network state.

Configuration Mechanisms:

Traditional Tools (Linux):

ifconfig — Legacy interface configuration
route — Legacy routing table management
netstat — Network statistics and connections

Modern Tools (iproute2):

ip link — Interface configuration
ip addr — Address management
ip route — Routing tables
ip neigh — ARP/ND cache
ss — Socket statistics (replaces netstat)

Configuration Files:

/etc/network/interfaces (Debian)
/etc/sysconfig/network-scripts/ (RHEL)
NetworkManager, systemd-networkd for dynamic management

network_commands.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# Modern Linux network configuration examples
 
# View all interfaces
ip link show
 
# Set interface up/down
ip link set eth0 up
ip link set eth0 down
 
# Add IP address
ip addr add 192.168.1.100/24 dev eth0
 
# View routing table
ip route show
 
# Add default route
ip route add default via 192.168.1.1
 
# View ARP cache
ip neigh show
 
# View open sockets
ss -tuln
 
# View TCP connections with process info
ss -tp
 
# View network statistics
ip -s link show eth0
 
# Create VLAN interface
ip link add link eth0 name eth0.100 type vlan id 100
 
# Create bridge
ip link add br0 type bridge
ip link set eth0 master br0

Programmatic Configuration:

Applications can configure networking through:

Netlink Sockets:

Primary kernel-userspace communication channel for network configuration
Used by ip commands internally
Supports routing, interfaces, addresses, firewall rules
Allows monitoring (rtnetlink for route changes)

ioctl():

Legacy interface still supported
SIOCGIFADDR, SIOCSIFADDR for addresses
Being replaced by netlink

sysfs and procfs:

/sys/class/net/<if>/ — Interface parameters
/proc/net/ — Statistics, connection tables
Read-only for most information

Configuration Persistence:

Kernel network state is volatile—it's lost on reboot. Persistence requires:

Configuration files read at boot
Network management daemons (NetworkManager, systemd-networkd)
Script-based initialization

Network Namespaces Preview

Virtual Network Devices

Types of Virtual Network Devices:

Linux Virtual Network Devices

•Loopback (lo) — Traffic never leaves the host; used for local inter-process communication. Always exists, typically 127.0.0.1/::1.
•Bridge (br0) — Software Layer 2 switch connecting multiple interfaces. VMs and containers attach to bridges to communicate.
•veth (Virtual Ethernet) — Paired virtual interfaces; packets sent to one appear on the other. Used to connect network namespaces.
•TUN/TAP — User-space network drivers. TUN for IP packets (VPNs), TAP for Ethernet frames (VM networking).
•macvlan/ipvlan — Create virtual interfaces with unique MAC (macvlan) or shared MAC (ipvlan) off a parent interface.
•VXLAN/GENEVE — Overlay networks tunneling L2 over L3. Used in cloud/datacenter networking.
•dummy — Loopback-like interface for testing or as anchor for addresses.

Converting Mermaid diagram...

Virtual Device Use Cases:

Container Networking: Containers typically have:

Own network namespace with private network stack
veth pair connecting to host bridge
NAT or routing to reach external networks
This is what Docker, Kubernetes, and other container runtimes do

Virtual Machine Networking: VMs typically have:

TAP device backing their virtual NIC
TAP connected to host bridge
Bridge connected to physical NIC or used for host-only networking

VPN Implementation: VPNs typically use:

TUN device receiving decrypted packets from VPN software
Routing table directing traffic through TUN
VPN software encrypts and sends via physical network

Testing and Development:

Create complex topologies without hardware
Network namespaces simulate multi-host environments
Inject failures, delays, packet loss with netem

Performance Considerations

Network Namespaces

Network namespaces are a Linux kernel feature that provides isolated network stacks. Each namespace has independent:

Network interfaces
IP addresses
Routing tables
Firewall rules (iptables/nftables)
Socket bindings
ARP/neighbor cache

Processes in different namespaces see completely different network environments, even on the same physical machine. This is the foundation of container networking.

Namespace Operations:

namespace_operations.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
# Create a new network namespace
ip netns add myns
 
# List namespaces
ip netns list
 
# Run command in namespace
ip netns exec myns ip addr show
 
# Create veth pair to connect namespaces
ip link add veth0 type veth peer name veth1
 
# Move one end into namespace
ip link set veth1 netns myns
 
# Configure interfaces
ip addr add 10.0.0.1/24 dev veth0
ip link set veth0 up
 
ip netns exec myns ip addr add 10.0.0.2/24 dev veth1
ip netns exec myns ip link set veth1 up
ip netns exec myns ip link set lo up
 
# Now 10.0.0.1 can ping 10.0.0.2
ping 10.0.0.2
 
# The namespace can ping back
ip netns exec myns ping 10.0.0.1
 
# Delete namespace (and all its interfaces)
ip netns delete myns

How Containers Use Namespaces:

When you run a container:

Container runtime creates namespace — New, empty network stack
Creates veth pair — Links container namespace to host
Configures container interface — Assigns IP, sets up routes
Connects host side — Usually to a bridge (docker0, cni0)
Sets up NAT/routing — For external connectivity
Container process runs — In the new namespace, sees only its interfaces

The container believes it has its own dedicated network stack—because, from its perspective, it does.

Beyond Containers:

Network namespaces are useful for:

Testing: Simulate multi-host environments on one machine
Security: Isolate network-facing services
VPN routing: Run specific apps through VPN without affecting others
Network function development: Test routing, firewalling without hardware

Namespace Persistence

Traffic Control and QoS

The Linux Traffic Control (tc) Subsystem:

Linux implements traffic control through three components:

Queuing Disciplines (qdiscs) — Algorithms that decide packet transmission order
Classes — Divisions within qdiscs for categorizing traffic
Filters — Rules that classify packets into classes

Common Queuing Disciplines
Qdisc	Type	Purpose	Use Case
pfifo_fast	Classless	Default; 3 priority bands	General purpose
fq_codel	Classless	Fair queuing with controlled delay	Reducing buffer bloat
htb	Classful	Hierarchical token bucket	Bandwidth allocation, guarantees
tbf	Classless	Token bucket filter	Rate limiting
prio	Classful	Priority queuing	Prioritize interactive traffic
netem	Classless	Network emulation	Testing: add delay, loss, reordering
cake	Classless	Common Applications KE	Home routers, buffer bloat

traffic_control_examples.sh
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
# View current qdisc
tc qdisc show dev eth0
 
# Add rate limiting (1 Mbit)
tc qdisc add dev eth0 root tbf rate 1mbit burst 32kbit latency 400ms
 
# Add fair queuing with controlled delay (reduce buffer bloat)
tc qdisc replace dev eth0 root fq_codel
 
# Add network emulation (100ms delay, 1% loss)
tc qdisc add dev eth0 root netem delay 100ms loss 1%
 
# Hierarchical Token Bucket for bandwidth allocation
tc qdisc add dev eth0 root handle 1: htb default 30
 
# Create classes
tc class add dev eth0 parent 1: classid 1:1 htb rate 100mbit
tc class add dev eth0 parent 1:1 classid 1:10 htb rate 50mbit  # High priority
tc class add dev eth0 parent 1:1 classid 1:20 htb rate 30mbit  # Normal
tc class add dev eth0 parent 1:1 classid 1:30 htb rate 20mbit  # Low priority
 
# Add filters to classify traffic
tc filter add dev eth0 protocol ip parent 1:0 prio 1 u32 \
    match ip dport 22 0xffff flowid 1:10  # SSH to high priority
 
# Delete qdisc
tc qdisc del dev eth0 root

QoS Use Cases:

Enterprise Networks:

Voice/video traffic prioritized over bulk data
Critical business applications guaranteed bandwidth
Guest WiFi limited to prevent interference

Service Providers:

Customer rate limiting per contract
Premium tier prioritization
Fair usage policies

Home Networks:

Gaming traffic low-latency
Video streaming vs. downloads
Work-from-home traffic protection

Container Orchestration:

Per-pod bandwidth limits
Priority classes for critical workloads
Fair sharing between tenants

TC Complexity

Summary: Operating System Support

We've explored the deep integration between operating systems and networking—from kernel architecture to the abstractions that enable modern cloud-native applications.

Key Takeaways

•The kernel network stack is central — It implements protocols, manages state, and provides the abstraction layer between applications and hardware.
•System calls are the API — Applications interact with networking through socket system calls, trading user/kernel transitions for kernel services.
•Buffer management is critical — sk_buff structures and socket buffers directly impact performance and must be sized appropriately.
•Configuration has many interfaces — From legacy ifconfig to modern ip commands to netlink sockets, multiple paths exist to configure networking.
•Virtual devices enable virtualization — Bridges, veths, TUN/TAP, and overlays create the networking fabric for VMs and containers.
•Network namespaces provide isolation — Each namespace has independent network state, enabling container networking and security isolation.
•Traffic control enables QoS — The tc subsystem provides powerful traffic shaping, though with complexity that warrants higher-level tools.

What's Next:

Page Complete