MTU Concept

Every network technology has a fundamental constraint: the maximum amount of data it can carry in a single transmission. This limit—the Maximum Transmission Unit (MTU)—is perhaps the most practically significant concept in network engineering, directly affecting performance, reliability, and whether packets successfully traverse the Internet at all.

MTU is not merely a technical specification—it's the bridge between abstract network protocols and physical transmission realities. Understanding MTU deeply transforms you from someone who follows configuration guides to someone who can diagnose mysterious connectivity failures, optimize network performance, and design robust systems.

Maximum Transmission Unit (MTU) is the largest size of a protocol data unit (PDU) that can be transmitted in a single network-layer transaction. But this simple definition masks considerable complexity.

Layer considerations:

MTU operates at the boundary between the network layer and data link layer. It represents the maximum payload the data link layer can carry—which becomes the maximum datagram size for the network layer.

Each network technology specifies its own MTU based on historical, technical, and economic factors. Understanding these values and their origins helps predict fragmentation scenarios.

Why 1500 bytes became the standard:

When Ethernet was developed at Xerox PARC in the 1970s, engineers faced competing constraints:

1. Efficiency — Larger frames have better payload-to-overhead ratio
2. Collision recovery — On shared coax, larger frames mean longer recovery from collisions
3. Buffer memory — RAM was expensive; larger frames require more buffer space
4. Latency fairness — Larger frames delay other traffic longer

The 1500-byte choice optimized the collision-recovery tradeoff for 10 Mbps Ethernet on coaxial cable. Despite modern switched Ethernet having no collision concerns, backward compatibility has preserved 1500 bytes as the standard.

Jumbo frames (9000 bytes):

Data centers often enable jumbo frames for significant performance benefits:

• 6× higher efficiency for bulk transfers
• Reduced CPU interrupt load
• Lower latency for large transfers

However, jumbo frames require end-to-end support and are NOT internet-compatible.

Rather than blindly sending large packets and hoping for the best, modern hosts use Path MTU Discovery (RFC 1191 for IPv4, RFC 8201 for IPv6) to determine the optimal packet size for a given destination.

How PMTUD Works:

PMTUD Algorithm Step-by-Step:

1. Send with DF flag set — Host sends packets with the Don't Fragment (DF) flag, starting with interface MTU

2. Router encounters MTU limit — If packet exceeds outgoing interface MTU, router cannot fragment due to DF

3. ICMP response sent — Router sends ICMP Type 3, Code 4 (Fragmentation Needed and DF Set) with next-hop MTU

4. Host updates cache — Source records the reduced MTU in its routing/PMTU cache

5. Retry with smaller packets — Host retransmits using the discovered MTU

6. Iterative discovery — Process repeats until packet reaches destination (path may have multiple reduced MTU points)

When PMTUD fails, the symptoms can be bewildering: connections establish but data transfer hangs, some websites work but others don't, small requests succeed but large responses fail. Understanding failure modes is essential for diagnosis.

Classic symptoms of PMTU black hole:

• TCP connection establishes successfully (SYN/SYN-ACK/ACK are small)
• Small HTTP requests work (GET requests, small responses)
• Large transfers hang (file downloads, large web pages)
• SSH works but SCP hangs
• Ping succeeds but application fails

Why TCP is particularly affected:

TCP uses MSS (Maximum Segment Size) which depends on PMTU. If PMTUD fails:

• TCP sends segments at interface MTU size
• Segments are dropped at bottleneck link
• TCP retransmits at same size (no ICMP feedback)
• Connection times out after multiple retries

Given PMTUD's fragility, various alternatives and workarounds have been developed:

Tunneling protocols wrap packets in additional headers, reducing effective MTU. Understanding this overhead is critical for VPN design and troubleshooting.

Calculating tunnel MTU:

Tunnel effective MTU = Underlying MTU - Tunnel overhead

Example: IPsec VPN over DSL:

• Base MTU (DSL/PPPoE): 1492 bytes
• IPsec ESP overhead: 60 bytes (typical)
• Effective MTU: 1492 - 60 = 1432 bytes
• TCP MSS: 1432 - 40 = 1392 bytes

Nested tunnels compound the problem:

• Original Ethernet: 1500 bytes MTU
• VPN tunnel: 1500 - 60 = 1440 bytes
• Inner GRE tunnel: 1440 - 24 = 1416 bytes
• Final TCP MSS: 1416 - 40 = 1376 bytes

Each tunnel layer consumes overhead that cannot carry user payload.

Systematic MTU diagnosis requires understanding what tools are available and how to interpret their output.