Ipv4 Header Format - Learning Module

Loading content...

0/240

Total Length Field

Defining Packet Boundaries

In a continuous stream of bytes flowing through network interfaces, how does a device know where one packet ends and another begins? At the link layer, Ethernet frames have their own length mechanisms, but at the IP layer, every packet must carry its own size information. The Total Length field serves this critical purpose—telling every device that handles the packet exactly how many bytes comprise the complete IP datagram.

This 16-bit field, occupying bytes 2 and 3 of the IPv4 header, might seem straightforward: just a number indicating packet size. But its implications extend deeply into fragmentation mechanics, Maximum Transmission Unit (MTU) considerations, packet validation, and security. Misinterpreting or mishandling this field has been the source of countless network bugs and security vulnerabilities.

What You Will Learn

By the end of this page, you will master the Total Length field completely: its exact encoding, valid value ranges, relationship with IHL and payload, role in fragmentation and reassembly, MTU interactions, and essential validation requirements for secure packet processing.

Field Position and Encoding

The Total Length field occupies bytes 2 and 3 of the IPv4 header (bit positions 16-31), immediately following the Type of Service byte.

Total Length Field Position in IPv4 Header
Bytes	Bit Positions	Field Name	Size
0	0-7	Version + IHL	8 bits
1	8-15	Type of Service	8 bits
2-3	16-31	Total Length	16 bits
4-5	32-47	Identification	16 bits

Total Length Field Specifications

•Size: 16 bits (2 bytes)
•Position: Bytes 2-3 (bit positions 16-31)
•Encoding: Unsigned 16-bit integer, big-endian (network byte order)
•Unit: Bytes (octets)
•Minimum Value: 20 (minimum IPv4 header with no data)
•Maximum Value: 65,535 (2¹⁶ - 1)
•Includes: IP header + IP options + payload data

Critical Definition:

The Total Length field specifies the total size of the IP datagram in bytes, including:

The IP header (20-60 bytes, as specified by IHL)
The IP payload (data carried by the IP packet)

The formula is:

Total Length = IP Header Length + Payload Length
Payload Length = Total Length - (IHL × 4)

This means if you have a Total Length of 1500 bytes and IHL of 5 (20-byte header), the payload is 1480 bytes.

Network Byte Order

Like all multi-byte fields in Internet protocols, Total Length uses big-endian encoding. The most significant byte comes first (byte 2), followed by the least significant byte (byte 3). A value of 500 bytes would be encoded as 0x01F4, stored as [0x01, 0xF4] in order.

Value Constraints and Validation

The Total Length field has both theoretical and practical constraints that must be validated for secure packet processing.

Theoretical Constraints:

Minimum Value: 20 bytes

An IPv4 packet must contain at least the minimum 20-byte header
Total Length < 20 is impossible and indicates corruption
Total Length < IHL × 4 is logically inconsistent

Maximum Value: 65,535 bytes

The 16-bit field can express values 0-65,535
This defines the maximum possible IP datagram size
In practice, such large datagrams require fragmentation

Total Length Validation Rules
Condition	Action	Rationale
Total Length < 20	Discard packet	Cannot be a valid IPv4 header
Total Length < IHL × 4	Discard packet	Header claims to be larger than packet
Total Length > actual bytes received	Discard packet	Packet truncated in transit
Total Length < actual bytes received	Trim padding	Link layer may have added padding
Total Length = 0	Discard packet	Invalid (RFC 791 requirement)

Practical Constraints:

MTU Limitations: While the theoretical maximum is 65,535 bytes, practical maximum sizes depend on the network path:

Ethernet MTU: 1500 bytes (standard), 9000 bytes (jumbo frames)
PPPoE: 1492 bytes (8 bytes of PPPoE overhead)
VPN tunnels: 1400-1472 bytes typically (encapsulation overhead)
Internet path: 576 bytes is the guaranteed minimum MTU any host must accept

Packets exceeding the path MTU must be fragmented or will be dropped (if DF bit is set).

total_length_validation.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
def validate_total_length(packet: bytes) -> dict:
    """
    Comprehensive validation of the IPv4 Total Length field.
    
    Returns:
        Dictionary with validation results and parsed information.
    
    Raises:
        ValueError: If the packet fails critical validation.
    """
    actual_length = len(packet)
    
    # Basic minimum length check
    if actual_length < 20:
        raise ValueError(f"Packet too short: {actual_length} bytes (minimum 20)")
    
    # Extract IHL from first byte
    ihl = packet[0] & 0x0F
    header_length = ihl * 4
    
    # Validate IHL
    if ihl < 5:
        raise ValueError(f"Invalid IHL: {ihl} (minimum 5)")
    
    # Extract Total Length from bytes 2-3 (big-endian)
    total_length = (packet[2] << 8) | packet[3]
    
    # Validation checks
    results = {
        'total_length': total_length,
        'ihl': ihl,
        'header_length': header_length,
        'actual_received': actual_length,
        'warnings': [],
        'valid': True
    }
    
    # Check 1: Total Length vs minimum
    if total_length < 20:
        raise ValueError(f"Total Length {total_length} < 20 bytes")
    
    # Check 2: Total Length vs header length
    if total_length < header_length:
        raise ValueError(
            f"Total Length {total_length} < Header Length {header_length}"
        )
    
    # Check 3: Total Length vs actual received bytes
    if total_length > actual_length:
        raise ValueError(
            f"Packet truncated: Total Length {total_length}, "
            f"received {actual_length}"
        )
    
    # Check 4: Padding detection (not an error, but noteworthy)
    if total_length < actual_length:
        padding = actual_length - total_length
        results['warnings'].append(
            f"Link layer padding detected: {padding} bytes"
        )
        results['padding'] = padding
    
    # Calculate payload length
    results['payload_length'] = total_length - header_length
    results['data_offset'] = header_length
    
    return results
 
# Example usage
packet = bytes([
    0x45, 0x00,       # Version=4, IHL=5, ToS=0
    0x00, 0x3C,       # Total Length = 60 bytes
    0x1A, 0x2B,       # Identification
    0x40, 0x00,       # Flags + Fragment Offset
    0x40, 0x06,       # TTL=64, Protocol=TCP
    0x00, 0x00,       # Header Checksum (placeholder)
    0xC0, 0xA8, 0x01, 0x64,  # Source IP: 192.168.1.100
    0x0A, 0x00, 0x00, 0x01   # Dest IP: 10.0.0.1
]) + bytes(40)  # 40 bytes of payload
 
info = validate_total_length(packet)
print(f"Total Length: {info['total_length']} bytes")
print(f"Header: {info['header_length']} bytes")
print(f"Payload: {info['payload_length']} bytes")
# Output:
# Total Length: 60 bytes
# Header: 20 bytes
# Payload: 40 bytes

The Padding Problem

Ethernet has a minimum frame size of 64 bytes (46 bytes of payload). IP packets smaller than 46 bytes will have padding added at the link layer. The IP layer MUST use the Total Length field to determine the actual packet size, discarding any trailing padding. Failing to do so causes checksum errors and data corruption.

Total Length and Fragmentation

The Total Length field plays a crucial role in IP fragmentation and reassembly. When an IP datagram is too large for a link's MTU, it must be fragmented into smaller pieces. Each fragment is itself a complete IP datagram with its own Total Length value.

Fragmentation Mechanics:

When a router fragments a datagram:

Original datagram: Total Length = Header + All Data
Each fragment: Total Length = Header + Fragment Data
Fragment Offset: Indicates where this fragment's data fits in the original
More Fragments (MF) flag: Set on all fragments except the last

Key Insight: Each fragment's Total Length reflects that fragment's size, not the original datagram's size. The original size must be reconstructed during reassembly by examining all fragments.

Fragmentation Example: 4000-byte Datagram, 1500-byte MTU
Fragment	Total Length	Fragment Offset	MF Flag	Data Bytes
Original	4000	N/A	N/A	3980 (4000-20 header)
Fragment 1	1500	0	1	1480
Fragment 2	1500	185 (1480/8)	1	1480
Fragment 3	1040	370 (2960/8)	0	1020

Calculation Details:

Original: 4000 bytes total = 20-byte header + 3980 bytes data

Fragment 1:
  - Header: 20 bytes (copied from original)
  - Data: 1480 bytes (must be multiple of 8)
  - Total Length: 1500 bytes
  - Fragment Offset: 0 (first fragment)
  - MF Flag: 1 (more fragments follow)

Fragment 2:
  - Header: 20 bytes
  - Data: 1480 bytes
  - Total Length: 1500 bytes
  - Fragment Offset: 1480/8 = 185
  - MF Flag: 1 (more fragments follow)

Fragment 3:
  - Header: 20 bytes
  - Data: 3980 - 1480 - 1480 = 1020 bytes
  - Total Length: 1040 bytes
  - Fragment Offset: 2960/8 = 370
  - MF Flag: 0 (last fragment)

Reconstructing Original Length

To compute the original datagram size from fragments, find the last fragment (MF=0) and calculate: Original Length = (Last Fragment Offset × 8) + Last Fragment's Data Length + Header Length. In the example: (370 × 8) + 1020 + 20 = 2960 + 1020 + 20 = 4000 bytes.

MTU and Path MTU Discovery

The Maximum Transmission Unit (MTU) defines the largest packet a network link can carry. Total Length must not exceed the MTU of any link in the path, or the packet will either be fragmented or dropped.

Common MTU Values
Technology	MTU (bytes)	Total Length Limit	Notes
Ethernet	1500	1500	De facto Internet standard
Jumbo Frames	9000	9000	Data centers, requires end-to-end support
PPPoE DSL	1492	1492	8 bytes PPPoE overhead
GRE Tunnel	1476	1476	24 bytes GRE overhead
IPsec ESP	1438	1438	Varies with algorithm
IPv6-in-IPv4	1480	1480	20 bytes outer IPv4 header
Internet minimum	576	576	RFC 791 guaranteed minimum
Loopback	65535	65535	No physical constraints

Path MTU Discovery (PMTUD):

Path MTU Discovery (RFC 1191) enables hosts to determine the smallest MTU along a path without fragmentation:

Sender sets DF (Don't Fragment) bit on all outgoing packets
Router with smaller MTU receives packet larger than its MTU
Router drops packet and sends ICMP "Fragmentation Needed" (Type 3, Code 4)
ICMP includes the MTU of the limiting link
Sender reduces packet size and retries
Process repeats until packets traverse the full path

Total Length's Role in PMTUD:

The sender must adjust Total Length to not exceed the discovered Path MTU. For TCP, this means reducing the Maximum Segment Size (MSS). For UDP, applications must limit datagram sizes.

PMTUD Calculation
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
# Path MTU Discovery Example
# Typical calculation for TCP over various transports
 
def calculate_max_total_length(path_mtu: int) -> dict:
    """
    Calculate maximum IP Total Length for a given path MTU.
    """
    return {
        'path_mtu': path_mtu,
        'max_total_length': path_mtu,
        'ip_header': 20,
        'tcp_header': 20,
        'tcp_options_typical': 12,  # Timestamps
        'max_tcp_payload': path_mtu - 20 - 20 - 12,
        'mss_without_options': path_mtu - 40,
        'mss_with_timestamps': path_mtu - 52
    }
 
# Common scenarios
scenarios = [
    ('Ethernet', 1500),
    ('PPPoE', 1492),
    ('GRE Tunnel', 1476),
    ('IPsec ESP (AES-256)', 1438),
]
 
for name, mtu in scenarios:
    info = calculate_max_total_length(mtu)
    print(f"{name} (MTU {mtu}):")
    print(f"  Max TCP MSS: {info['mss_with_timestamps']} bytes")
    print(f"  Max Total Length: {info['max_total_length']} bytes")
    print()
 
# Output:
# Ethernet (MTU 1500):
#   Max TCP MSS: 1448 bytes
#   Max Total Length: 1500 bytes
#
# PPPoE (MTU 1492):
#   Max TCP MSS: 1440 bytes
#   Max Total Length: 1492 bytes
#
# GRE Tunnel (MTU 1476):
#   Max TCP MSS: 1424 bytes
#   Max Total Length: 1476 bytes
#
# IPsec ESP (MTU 1438):
#   Max TCP MSS: 1386 bytes
#   Max Total Length: 1438 bytes

PMTUD Black Holes

Many firewalls incorrectly block ICMP messages, including 'Fragmentation Needed.' This creates PMTUD black holes where large packets simply disappear. Symptoms include connections that stall after a few small packets, or complete failure for applications sending large datagrams. Solutions include MSS clamping at borders and TCP probing (RFC 4821).

Jumbograms and IPv6 Considerations

The 16-bit Total Length field limits IPv4 datagrams to 65,535 bytes. While this was enormous when IPv4 was designed, modern high-speed networks and specialized applications can benefit from larger packets. IPv6 addresses this limitation.

IPv4's 65,535-Byte Limit:

With a 16-bit field, the maximum expressible value is 2¹⁶ - 1 = 65,535 bytes. This includes both header and payload, so the maximum payload is approximately 65,515 bytes (with a 20-byte header).

Practical Impact:

Adequate for most applications: Even jumbo frames (9000 bytes) are far below this limit
Large file transfers: Applications segment data into multiple packets anyway
Memory mapping scenarios: Some specialized applications could benefit from larger packets

IPv6 Jumbograms (RFC 2675):

IPv6 introduces the concept of jumbograms for packets larger than 65,535 bytes:

Payload Length field: IPv6 has a 16-bit Payload Length field (not Total Length)
Jumbo Payload option: When Payload Length = 0, a Hop-by-Hop extension header contains a 32-bit Jumbo Payload Length
Maximum size: 4,294,967,295 bytes (2³² - 1)
Use case: High-performance computing, specialized data center applications

IPv4 Total Length vs IPv6 Payload Length
Aspect	IPv4 Total Length	IPv6 Payload Length
Field size	16 bits	16 bits
What it measures	Header + Payload	Payload only (excludes header)
Maximum value	65,535 bytes	65,535 bytes (regular)
Header excluded?	No, header included	Yes, 40-byte base header excluded
Jumbogram support	No	Yes, via extension header (32 bits)
Maximum with jumbo	N/A	4,294,967,295 bytes

Why IPv6 Excludes Header from Length

IPv6's Payload Length excludes the 40-byte base header because the base header has a fixed size—there's no need to include it in count. IPv4's variable-length header (20-60 bytes due to options) necessitated including the header in Total Length. This IPv6 design simplifies processing and provides marginally more payload capacity.

Security Implications

The Total Length field has been exploited in numerous attacks over the decades. Understanding these vulnerabilities is essential for secure network programming and operations.

Historical Vulnerabilities

•Ping of Death (1996): Malformed packets with Total Length > 65,535 bytes via fragmentation caused buffer overflows when reassembled. Fragments with offset × 8 + data length > 65,535 triggered crashes.
•Teardrop Attack (1997): Overlapping fragments with inconsistent Total Length values caused kernel crashes during reassembly. Fragment 2's offset overlapped Fragment 1's data.
•Land Attack: Packets with source=destination and crafted Total Length caused some systems to enter infinite loops.
•Oversized Packet Attacks: Sending packets larger than interface buffers caused stack overflows in network drivers.
•Total Length Mismatch: Packets where Total Length didn't match actual data length exploited weak validation to bypass security devices.

Modern Security Considerations:

1. Input Validation: Always validate Total Length before allocating buffers or copying data:

Total Length ≥ IHL × 4 (header size)
Total Length ≤ actual received bytes
Total Length ≤ reasonable maximum (e.g., 65535 or MTU)

2. Reassembly Limits: Modern systems limit fragment reassembly:

Maximum fragments per datagram
Maximum reassembly buffer size per source
Timeout for incomplete reassembly
Total Length consistency across fragments

3. IDS/IPS Evasion: Attackers craft Total Length values to confuse intrusion detection:

Claim smaller size than actual payload
Create fragments that reassemble to different content on different systems

secure_parsing.py
Python
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
import struct
from typing import Optional
 
class IPv4ValidationError(Exception):
    """Raised when IPv4 packet validation fails."""
    pass
 
def secure_parse_ipv4(
    data: bytes,
    max_length: int = 65535,
    require_complete: bool = True
) -> dict:
    """
    Securely parse IPv4 header with Total Length validation.
    
    Args:
        data: Raw packet bytes
        max_length: Maximum acceptable Total Length
        require_complete: If True, require all claimed data present
    
    Returns:
        Parsed header information
    
    Raises:
        IPv4ValidationError: On any validation failure
    """
    received_bytes = len(data)
    
    # Basic sanity check
    if received_bytes < 20:
        raise IPv4ValidationError(
            f"Packet too short for IPv4: {received_bytes} bytes"
        )
    
    # Extract and validate Version/IHL
    version_ihl = data[0]
    version = (version_ihl >> 4) & 0x0F
    ihl = version_ihl & 0x0F
    
    if version != 4:
        raise IPv4ValidationError(f"Not IPv4: version = {version}")
    
    if ihl < 5:
        raise IPv4ValidationError(f"Invalid IHL: {ihl}")
    
    header_length = ihl * 4
    
    if header_length > received_bytes:
        raise IPv4ValidationError(
            f"Header length {header_length} exceeds received {received_bytes}"
        )
    
    # Extract and validate Total Length
    total_length = struct.unpack('!H', data[2:4])[0]  # Big-endian unsigned short
    
    # Check: Total Length >= header
    if total_length < header_length:
        raise IPv4ValidationError(
            f"Total Length {total_length} < Header Length {header_length}"
        )
    
    # Check: Total Length within limits
    if total_length > max_length:
        raise IPv4ValidationError(
            f"Total Length {total_length} exceeds maximum {max_length}"
        )
    
    # Check: Total Length <= received bytes (packet not truncated)
    if require_complete and total_length > received_bytes:
        raise IPv4ValidationError(
            f"Packet truncated: Total Length {total_length}, "
            f"received {received_bytes}"
        )
    
    # Calculate derived values safely
    payload_length = total_length - header_length
    
    # Determine actual data bounds
    data_start = header_length
    data_end = min(total_length, received_bytes)
    actual_payload = data_end - data_start
    
    return {
        'version': version,
        'ihl': ihl,
        'header_length': header_length,
        'total_length': total_length,
        'payload_length': payload_length,
        'received_bytes': received_bytes,
        'header': data[:header_length],
        'payload': data[data_start:data_end],
        'has_padding': received_bytes > total_length,
        'is_truncated': total_length > received_bytes
    }

Never Trust the Length Field Blindly

The Total Length field is attacker-controlled data. Use it only for interpretation, never for direct memory allocation without bounds checking. Calculate buffer sizes from actual received bytes, then validate against Total Length for consistency.

Practical Packet Analysis

Let's examine how the Total Length field appears in real packet captures and how to use it for troubleshooting.

Wireshark Analysis

Wireshark

# Wireshark display of Total Length field
 
Internet Protocol Version 4, Src: 192.168.1.100, Dst: 93.184.216.34
    0100 .... = Version: 4
    .... 0101 = Header Length: 20 bytes (5)
    Differentiated Services Field: 0x00
    Total Length: 1500
    Identification: 0x1a2b
    Flags: 0x4000, Don't fragment
    Fragment offset: 0
    Time to Live: 64
    Protocol: TCP (6)
    Header checksum: 0x1234 [correct]
    Source: 192.168.1.100
    Destination: 93.184.216.34
 
# Note: Wireshark shows "Total Length: 1500" directly
 
# Wireshark filters for Total Length:
ip.len == 1500          # Exact match (maximum Ethernet)
ip.len > 1500           # Jumbo frames or fragmentation needed
ip.len < 100            # Small packets (ACKs, keepalives)
ip.len >= 576           # Above minimum Internet MTU
ip.len < 46             # Likely padding present
 
# Detecting MTU issues:
icmp.type == 3 && icmp.code == 4   # Fragmentation Needed messages
 
# Comparing claimed vs actual (Wireshark calculates this):
frame.len - 14 != ip.len  # 14 = Ethernet header
                          # Indicates padding or truncation

Troubleshooting with Total Length:

Common Troubleshooting Scenarios

•Path MTU issues: If connections stall with large data, compare Total Length against expected MTU; look for ICMP Fragmentation Needed messages that might be blocked
•Jumbo frame mismatch: If Total Length > 1500 but frames are being dropped, verify jumbo frame support is enabled end-to-end
•Padding confusion: Small packets with Total Length < 46 bytes will have Ethernet padding; verify applications read Total Length bytes, not frame length
•Fragmentation debugging: When reassembly fails, verify Total Length of each fragment; look for overlapping or gapped fragments
•Performance analysis: Distribution of Total Length values reveals traffic patterns; large values indicate bulk transfers, small values indicate interactive traffic

tcpdump Analysis
Shell
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
# Using tcpdump to inspect Total Length
 
# Show verbose output including length
$ tcpdump -n -v -i eth0 ip
IP (tos 0x0, ttl 64, id 12345, offset 0, flags [DF], proto TCP (6), length 1500)
    192.168.1.100.43210 > 93.184.216.34.80: ...
#                                                                    ^^^^^^
#                                           The 'length' field is Total Length
 
# Filter by packet size
$ tcpdump -n 'ip and greater 1500'   # Packets exceeding Ethernet MTU
$ tcpdump -n 'ip and less 100'        # Small packets
 
# Show hex dump to see raw bytes
$ tcpdump -n -X 'ip and host 192.168.1.100'
0x0000:  4500 05dc 1a2b 4000 4006 ...
         ^^   ^^^^
         │    │
         │    └── Total Length = 0x05DC = 1500 bytes
         └─────── Version=4, IHL=5
 
# Capture fragmented packets
$ tcpdump -n 'ip[6:2] & 0x1fff != 0'  # Non-zero fragment offset
$ tcpdump -n 'ip[6] & 0x20 != 0'      # MF flag set

Quick Mental Math

Common Total Length values to recognize: 40-44 = TCP ACK (empty), 52-56 = TCP ACK with timestamps, 576 = minimum Internet MTU, 1500 = Ethernet MTU, ~1492 = PPPoE. Values like 1000, 1460, 1448 often indicate application-controlled segment sizes.

Summary: Total Length Field

We've comprehensively examined the Total Length field—its encoding, constraints, and critical role in packet processing. Let's consolidate the essential knowledge:

Key Takeaways

•Total Length occupies bytes 2-3: A 16-bit big-endian unsigned integer specifying the complete IP datagram size in bytes.
•Includes header and payload: Total Length = IHL × 4 + Payload. Use this to calculate payload size and data offset.
•Valid range is 20-65,535: Minimum is a header with no data; maximum is constrained by the 16-bit field.
•MTU limits practical size: Most Internet traffic uses Total Length ≤ 1500 (Ethernet MTU). Larger packets require fragmentation or Path MTU Discovery.
•Each fragment has its own Total Length: Fragments are independent datagrams; the original size is reconstructed during reassembly.
•Link layer padding must be ignored: Ethernet pads small frames; IP layer must use Total Length, not frame length, to determine data boundaries.
•Security-critical validation required: Always validate Total Length against IHL, actual received bytes, and reasonable maximums before processing.

What's Next:

With Total Length mastered, we proceed to the Identification field—a 16-bit value that, along with Flags and Fragment Offset, forms the foundation of IP fragmentation and reassembly. Understanding Identification is essential for analyzing fragmented traffic and understanding reassembly attacks.

Page Complete

You now possess comprehensive understanding of the Total Length field—its encoding, validation requirements, MTU interactions, fragmentation role, and security implications. This knowledge is foundational for packet parsing, network programming, and protocol analysis.