Computer NetworksNetwork Layer Security

Authentication Header (AH)

LevelIntermediate

Duration55 mins

TopicNetwork Layer Security

4 / 5

Anti-Replay Protection: Defeating Packet Replay Attacks

The Threat of Replay Attacks

Imagine an attacker who cannot break your encryption, cannot forge your authentication, but can still cause significant harm to your system. This isn't hypothetical—it's the reality of replay attacks, one of the most insidious threats in network security.

A replay attack works by capturing legitimate, properly authenticated packets and retransmitting them at a later time. The packets are genuine; the authentication is valid; but the context has changed. A funds transfer command replayed becomes a duplicate transfer. An authentication token replayed grants unauthorized access. A configuration command replayed reverts critical settings.

The Authentication Header's anti-replay service defeats these attacks through a deceptively simple mechanism: unique sequence numbers combined with a sliding window algorithm. This page explores this mechanism in depth, from fundamental concepts through implementation details and edge cases.

Learning Objectives

By the end of this page, you will: understand how replay attacks work and why they're dangerous; know how sequence numbers and sliding windows prevent replay; be able to configure and tune anti-replay parameters; understand Extended Sequence Numbers (ESN) for high-bandwidth links; and recognize scenarios where anti-replay considerations affect deployment.

Understanding Replay Attacks

Before examining the protection mechanism, we must understand the threat it counters. Replay attacks exploit a fundamental property of network authentication: a valid authentication code remains valid regardless of when or how many times it's presented.

Attack Mechanics:

Capture Phase: Attacker intercepts packets on the network (passive eavesdropping or man-in-the-middle position)
Storage Phase: Captured packets are stored for later use
Replay Phase: Attacker retransmits captured packets at a strategically chosen time

The attack succeeds because the receiving system validates the authentication (it's genuinely valid) and processes the packet as if it were freshly transmitted.

Converting Mermaid diagram...

Real-World Replay Attack Scenarios:

Replay Attack Impact Examples
Domain	Attack Vector	Potential Damage
Financial Systems	Replay fund transfer authorizations	Duplicate transactions, fund theft
Authentication	Replay login success packets	Unauthorized access to accounts
Industrial Control	Replay control commands	Incorrect equipment operation, safety hazards
VPN Sessions	Replay session establishment	Hijack or duplicate VPN connections
Database Operations	Replay write operations	Data corruption, duplicate entries
Routing Protocols	Replay routing updates	Traffic misdirection, black holes
E-commerce	Replay purchase confirmations	Duplicate orders, inventory fraud

Why Integrity Protection Isn't Enough:

Integrity verification (the ICV check) confirms packets haven't been modified—but replayed packets haven't been modified. They're perfect copies of legitimate traffic. HMAC-SHA-256 verifies that the packet originated from someone with the shared secret, but it cannot detect that this same packet was already processed minutes, hours, or days ago.

Attack Variations:

Simple Replay: Retransmit the exact captured packet
Delayed Replay: Wait for strategic timing (end of financial period, system maintenance window)
Selective Replay: Only replay certain packet types (commands vs. data)
Out-of-Order Replay: Disrupt sequences by replaying earlier packets
Inter-Session Replay: Attempt to replay packets from expired sessions into new ones

Replay Attacks Are Protocol-Agnostic

Replay attacks can affect any authentication mechanism that doesn't include freshness or uniqueness guarantees. TLS uses sequence numbers per session. Kerberos includes timestamps in tickets. OAuth tokens have expiration times. These all address the same fundamental vulnerability that AH's anti-replay mechanism counters.

The Sequence Number Mechanism

AH defeats replay attacks through a mandatory Sequence Number field combined with receiver-side tracking. Each packet in a Security Association carries a unique, monotonically increasing sequence number that ensures no two valid packets ever have the same identifier.

Sender Behavior:

The sender maintains a 32-bit counter for each Security Association:

Sender Sequence Number Handling
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
// Security Association state (sender side)
struct SA_Sender {
    uint32_t sequenceNumber;     // Current counter value
    bool     useESN;             // Extended Sequence Number enabled?
    uint32_t sequenceNumberHigh; // Upper 32 bits for ESN
    // ... other SA parameters
};
 
function sendPacket(sa, packet) {
    // Increment BEFORE use—0 is never transmitted
    if (sa.useESN) {
        // 64-bit counter
        uint64_t seqNum = (sa.sequenceNumberHigh << 32) | sa.sequenceNumber;
        seqNum++;
        sa.sequenceNumber = seqNum & 0xFFFFFFFF;
        sa.sequenceNumberHigh = seqNum >> 32;
    } else {
        // 32-bit counter
        sa.sequenceNumber++;
    }
    
    // Check for overflow
    if (sa.sequenceNumber == 0 && !sa.useESN) {
        // Counter has wrapped—MUST establish new SA
        triggerRekey(sa);
        return ERROR_MUST_REKEY;
    }
    
    // Insert sequence number into AH header
    packet.ah.sequenceNumber = sa.sequenceNumber;
    
    // Compute ICV with sequence number included
    packet.ah.icv = computeICV(sa, packet);
    
    transmit(packet);
}

Critical Sender Rules:

Counter starts at 0, but first transmitted sequence number is 1
Counter MUST NOT wrap around (32-bit limit = 2³² - 1 = 4,294,967,295 packets)
If counter would overflow, new SA MUST be established before transmitting
Sequence number is included in ICV computation—it cannot be modified without invalidating authentication
Each packet gets a unique sequence number—no reuse within an SA

Sequence Number Space:

For a 32-bit sequence number:

32-bit Sequence Number Exhaustion Times
Packet Rate	Time to Exhaust
1,000 packets/second	~50 days
10,000 packets/second	~5 days
100,000 packets/second	~12 hours
1,000,000 packets/second	~72 minutes
10,000,000 packets/second	~7.2 minutes

High-Bandwidth Considerations:

At 10 Gbps with minimum-sized packets (84 bytes on wire for 64-byte payload with Ethernet overhead):

Maximum rate: ~14.88 million packets/second
Time to exhaust 32-bit space: ~5 minutes

This is why Extended Sequence Numbers (ESN) were introduced—we'll cover them shortly.

Protection Properties:

Uniqueness: Each packet has a globally unique (within the SA) identifier
Authentication: Sequence number is authenticated—cannot be modified
Ordering: Monotonic increase enables detection of reordering and replays
Freshness: Old sequence numbers are rejected, ensuring packet recency

Why Not Timestamps?

Timestamps might seem like a natural anti-replay mechanism, but they require synchronized clocks across all endpoints—a significant deployment challenge. Sequence numbers are locally generated and require no synchronization. The receiver only needs to track which numbers it has seen, not when they were sent.

The Sliding Window Algorithm

The receiver can't simply require packets to arrive in order—network reordering is common. Instead, AH uses a sliding window algorithm that accepts packets within a range of sequence numbers while rejecting duplicates and stale packets.

Window Concept:

The receiver maintains:

Window size (W): Number of sequence numbers to track (typically 32 or 64)
Right edge (R): Highest sequence number received and validated
Window contents: Bitmap tracking which sequence numbers within the window have been received

Acceptable sequence numbers fall in the range: [R - W + 1, R]

Converting Mermaid diagram...

Legend:

✓ (Green): Already received and validated
□ (Yellow): Not yet received but within window (acceptable)
Old (Red): Below window, rejected
New (Blue): Above window, will advance window if ICV validates

Receiver Algorithm:

Sliding Window Anti-Replay
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
struct SA_Receiver {
    uint32_t  rightEdge;   // Highest validated sequence number
    uint64_t  bitmap;      // Bit i set = (rightEdge - i) received
    uint32_t  windowSize;  // Typically 32 or 64
};
 
function checkReplay(sa, seqNum) {
    // Case 1: Sequence number is above the window (new highest)
    if (seqNum > sa.rightEdge) {
        return ACCEPT_PENDING_ICV;  // Will advance window after ICV validates
    }
    
    // Case 2: Sequence number is below the window (too old)
    if (seqNum < sa.rightEdge - sa.windowSize + 1) {
        return REJECT_TOO_OLD;
    }
    
    // Case 3: Sequence number is within the window
    uint32_t offset = sa.rightEdge - seqNum;  // Position in bitmap
    
    if (sa.bitmap & (1ULL << offset)) {
        return REJECT_DUPLICATE;  // Already received
    }
    
    return ACCEPT_PENDING_ICV;  // Within window, not yet received
}
 
function updateWindow(sa, seqNum) {
    // Called ONLY after ICV validation succeeds
    
    if (seqNum > sa.rightEdge) {
        // Advance the window
        uint32_t shift = seqNum - sa.rightEdge;
        
        if (shift >= sa.windowSize) {
            // New packet is so far ahead, clear entire bitmap
            sa.bitmap = 1;  // Only the new packet is marked
        } else {
            // Shift bitmap and mark new packet
            sa.bitmap = (sa.bitmap << shift) | 1;
        }
        
        sa.rightEdge = seqNum;
    } else {
        // Within window—just mark it
        uint32_t offset = sa.rightEdge - seqNum;
        sa.bitmap |= (1ULL << offset);
    }
}

Window Size Selection:

The window size W determines how much out-of-order delivery is tolerated:

Window Size Trade-offs
Window Size	Memory	Tolerance	Common Use Case
32 packets	4 bytes + uint32	Moderate reordering	Low-latency networks, LAN
64 packets	8 bytes + uint32	Higher reordering	Default recommendation
128 packets	16 bytes + uint32	Significant reordering	High-latency paths, satellite
1024 packets	128 bytes + uint32	Extreme reordering	Specialized applications

State Machine Summary:

Converting Mermaid diagram...

ICV Before Window Update

The window is ONLY updated after successful ICV validation. This is critical—otherwise attackers could corrupt the window by sending forged packets with specific sequence numbers, causing legitimate packets to be rejected as duplicates.

Extended Sequence Numbers (ESN)

For high-bandwidth connections, 32-bit sequence numbers exhaust too quickly. Extended Sequence Numbers (ESN) provide a 64-bit sequence space while maintaining backward-compatible packet format.

ESN Design:

Full sequence number: 64 bits (16 exabytes of unique packets)
Transmitted: Only lower 32 bits (in AH header, as before)
Upper 32 bits: Maintained internally, reconstructed at receiver
ICV computation: Uses full 64-bit sequence number

Why Not Just Transmit 64 Bits?

Maintaining the 32-bit field in the header:

Preserves backward compatibility with existing implementations
Reduces packet overhead by 4 bytes per packet
At 10 Gbps, saving 4 bytes per packet = 400+ Mbps bandwidth savings

Sender ESN Handling:

ESN Sender Processing
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
function sendWithESN(sa, packet) {
    // Increment full 64-bit counter
    sa.seqLow++;
    if (sa.seqLow == 0) {
        sa.seqHigh++;  // Carry to upper 32 bits
        if (sa.seqHigh == 0) {
            // 64-bit overflow—theoretically impossible in practice
            return ERROR_COUNTER_EXHAUSTED;
        }
    }
    
    // Only lower 32 bits go in the packet
    packet.ah.sequenceNumber = sa.seqLow;
    
    // ICV computed over full 64-bit sequence
    // Upper 32 bits appended to authenticated data (not transmitted)
    packet.ah.icv = computeICVWithESN(sa, packet, sa.seqHigh);
    
    transmit(packet);
}

Receiver ESN Reconstruction:

The receiver must determine the upper 32 bits without receiving them. This is possible because:

Packets generally arrive in order (minor reordering)
The receiver knows approximately which 64-bit range the sender is in
The 32-bit window provides bounds on how far a packet can be from expected

ESN Receiver Reconstruction
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
function reconstructESN(sa, receivedSeqLow) {
    // Receiver tracks: Tl (low 32 bits of right edge), Th (high 32 bits)
    uint32_t Tl = sa.rightEdge & 0xFFFFFFFF;
    uint32_t Th = sa.rightEdge >> 32;
    uint32_t W  = sa.windowSize;
    
    // Determine which "era" the packet is from
    uint32_t seqHigh;
    
    if (receivedSeqLow >= Tl - W + 1 || Tl < W - 1) {
        // Case 1: Received seq is >= window bottom
        // Packet is in current era OR just wrapped
        if (receivedSeqLow > Tl) {
            // Packet is ahead of right edge in same era
            seqHigh = Th;
        } else if (receivedSeqLow >= Tl - W + 1) {
            // Packet is within window in same era
            seqHigh = Th;
        } else {
            // Packet appears to be from previous era (wrapped)
            // This happens when low bits wrapped but high bits didn't yet advance
            seqHigh = Th + 1;
        }
    } else if (receivedSeqLow < Tl - W + 1) {
        // Below window in current era—might be from next era
        if (receivedSeqLow < W && Tl > 0xFFFFFFFF - W) {
            // Packet from future era (receiver's low bits about to wrap)
            seqHigh = Th + 1;
        } else {
            // Truly old packet—will be rejected by anti-replay
            seqHigh = Th;
        }
    }
    
    return ((uint64_t)seqHigh << 32) | receivedSeqLow;
}

ESN in ICV Computation:

When ESN is enabled, the authentication covers the full 64-bit sequence number. The upper 32 bits are appended to the authenticated data (after the payload) for ICV computation—but they are NOT transmitted in the packet.

ICV_Input = IP_Header || AH_Header || Payload || SeqNumHigh(32 bits)

This ensures that even though the upper bits aren't transmitted, they're authenticated. A receiver who reconstructs the wrong upper bits will compute an incorrect ICV and reject the packet.

ESN Exhaustion:

With 64-bit sequence numbers:

64-bit ESN Exhaustion Times
Packet Rate	Time to Exhaust
1 million packets/second	~584,942 years
100 million packets/second	~5,849 years
1 billion packets/second	~585 years
100 billion packets/second	~5.8 years

ESN Negotiation

ESN is negotiated during IKE SA establishment. Both endpoints must agree to use ESN. Enable ESN by default for new deployments—the overhead is minimal, and it eliminates the risk of sequence number exhaustion requiring emergency rekeying.

Anti-Replay Configuration and Tuning

While anti-replay is fundamental to AH security, its configuration can be tuned for specific network conditions. Understanding the parameters helps optimize security and performance.

Configuration Parameters:

Anti-Replay Configuration Options
Parameter	Typical Values	Effect of Larger	Effect of Smaller
Window Size	32-1024	More tolerance for reordering; more memory	Less memory; rejects more reordered packets
ESN	On/Off	Eliminates 32-bit exhaustion; slight CPU overhead	May require frequent rekeying at high rates
Sequence Number Check	Enabled/Disabled	Full replay protection	Vulnerable to replay; for compatibility only

Configuring Window Size:

The window size should accommodate expected network reordering:

Measure reordering: Use tools like iperf3 or specialized network analyzers
Determine 99th percentile: What's the maximum reorder distance seen?
Add safety margin: Set window to 1.5-2x the measured maximum
Monitor rejections: If legitimate packets are rejected, increase window

Example Scenario:

Corporate VPN over Internet:

Measured reordering: Usually 0-5 packets, occasionally 20-30
99th percentile: 25 packets
Recommended window: 64 packets (provides 2.5x margin)

Platform-Specific Configuration:

Configuration Examples
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Linux (ip xfrm / StrongSwan)
# Set replay window for SA
ip xfrm state add ... replay-window 64
 
# StrongSwan (ipsec.conf)
conn myvpn
    replay_window=64
    extended_sequence_numbers=yes
 
# Cisco IOS
crypto ipsec security-association replay window-size 1024
 
# Juniper JunOS
set security ipsec security-association anti-replay-window-size 64
 
# Windows (PowerShell / netsh)
# Windows uses 64 by default, configurable via registry
# HKLM\SYSTEM\CurrentControlSet\Services\PolicyAgent\Oakley
# ReplayWindowSize (DWORD)

Disabling Anti-Replay (NOT Recommended):

RFC 4302 allows disabling anti-replay for backward compatibility, but this is strongly discouraged. Legitimate reasons to disable are extremely rare:

Legacy systems that don't support anti-replay
Debugging specific interoperability issues
Multicast scenarios where per-receiver sequence tracking is impractical

If you must disable:

Document the security implication explicitly
Implement alternative replay protection at the application layer
Monitor for suspicious traffic patterns that might indicate replay attempts
Plan upgrade to supporting implementation

Never Disable for Performance

The anti-replay check is computationally trivial—a few integer comparisons and bitmap operations. It adds nanoseconds to packet processing. Claims that disabling anti-replay improves performance are unfounded. Always keep it enabled.

Edge Cases and Special Scenarios

While the sliding window algorithm handles most scenarios, certain edge cases require special consideration.

1. SA Renegotiation Race Condition:

When an SA is being renegotiated (rekeyed), packets may arrive on both old and new SAs:

Converting Mermaid diagram...

Mitigation: Implementations maintain both old and new SAs during transition. Each SA has its own sequence number space and window. The old SA times out after a grace period.

2. Multipath and Load Balancing:

When traffic is load-balanced across multiple paths with different latencies, reordering can be extreme:

Path A: 10ms latency
Path B: 50ms latency
Packets 1, 2, 3 → Path A
Packets 4, 5, 6 → Path B
Arrival order: 1, 2, 3, 7, 8, 9, 4, 5, 6

With window = 8, packets 4, 5, 6 might fall outside the window when they arrive.

Mitigation: Increase window size or use hash-based load balancing that keeps flows on the same path.

3. System Restart:

If the receiver restarts mid-SA:

Sequence tracking state may be lost
Attacker could replay packets from before the restart
Some implementations persist anti-replay state to disk; most don't

Mitigation: Trigger SA renegotiation on restart. Discard current SA and establish new SAs with fresh sequence number spaces.

4. Clock/Counter Desynchronization:

If sender's sequence counter is ahead of receiver's expectation (e.g., due to transmission before receiver was ready):

Legitimate packets may be rejected as "too new"
Anti-replay logic doesn't have concept of "too new" but window advancement issues may occur

Mitigation: Ensure both endpoints complete SA establishment before transmitting data. IKE handshake completion guarantees readiness.

Edge Case Summary and Mitigations
Scenario	Symptom	Mitigation
SA Rekeying	Packets on old SA rejected after new SA established	Maintain overlapping validity; old SA grace period
Multipath Reorder	Legitimate packets rejected as too old	Increase window size; use flow-based ECMP
Receiver Restart	Replay vulnerable until new SA	Trigger rekey on restart; persist state if critical
Network Partition	Large sequence gap on recovery	Rekey if gap exceeds window; monitor for attacks
Packet Loss	Gaps in sequence (not a problem)	Normal operation; window accommodates

Monitoring Anti-Replay

Most IPSec implementations expose counters for anti-replay failures. Monitor these metrics. A sudden increase in 'replay detected' or 'too old' events may indicate either network issues (reordering beyond window) or active attacks.

Security Analysis of Anti-Replay

Let's analyze the security guarantees provided by AH's anti-replay mechanism and understand its limits.

What Anti-Replay Guarantees:

Security Guarantees

•No exact replays — A packet with sequence number N, once accepted, cannot be accepted again with the same sequence number
•No stale replays — Packets with sequence numbers older than the window are rejected, preventing delayed replay
•Authenticated uniqueness — Sequence numbers are authenticated; attackers cannot forge new sequence numbers
•Bounded acceptance — Only packets within the window are candidates for acceptance, limiting vulnerability window

What Anti-Replay Does NOT Guarantee:

Limitations

•Cross-SA replay — Different SAs have independent sequence spaces; packets from SA1 can't be replayed into SA2, but this is due to different keys, not anti-replay
•Pre-window replay — Before any packets are received, the window hasn't been established; first packet has no anti-replay check (but does have ICV)
•Future packet forgery — Anti-replay doesn't prevent forging new packets (that's ICV's job); it only prevents reuse of existing packets
•Application-layer replay — If the same application operation can be performed with different packet contents, anti-replay doesn't help

Attack Resilience Analysis:

Scenario 1: Attacker captures and immediately replays

Packet with sequence N is captured
Attacker replays packet with same sequence N
If original not yet received: Replay might succeed (but attacker doesn't know this)
If original already received: Rejected as duplicate
Risk: Minimal; attacker has short window and uncertain success

Scenario 2: Attacker stores packets and replays later

Packets captured over time
Attacker replays hours/days later
All sequence numbers now below window
100% rejection rate
Risk: None

Scenario 3: Attacker attempts selective replay

Attacker wants to replay specific high-value packet
Must replay within window (before window advances past it)
Must replay before original is received
Success probability: Very low for targeted attacks

Conclusion:

The anti-replay mechanism provides strong protection that, combined with ICV authentication, makes replay attacks impractical. The primary vulnerability—the brief window between packet transmission and reception—doesn't offer attackers meaningful exploitation opportunities.

Defense in Depth

Anti-replay is one layer in a defense-in-depth strategy. Critical applications should implement additional replay protections: transaction IDs at application layer, timestamps in application protocols, idempotent operation design. AH anti-replay protects the transport; applications should protect their semantics.

Summary: Anti-Replay Mastery

We've comprehensively explored AH's anti-replay mechanism. Let's consolidate the essential concepts:

Key Takeaways

•Replay attacks bypass authentication — Valid packets remain valid; uniqueness/freshness must be tracked separately from authenticity
•Sequence numbers provide uniqueness — Each packet has a unique, authenticated, monotonically increasing sequence number in its SA
•Sliding window tolerates reordering — Accepts packets within a configurable range; rejects duplicates and stale packets
•Window updates only after ICV validation — Prevents attackers from corrupting window state with forged packets
•ESN handles high bandwidth — 64-bit sequence space prevents exhaustion; only 32 bits transmitted to save bandwidth
•Configuration matters — Window size should match network reordering characteristics; always enable anti-replay
•Edge cases require attention — SA rekeying, multipath, and restarts need careful handling to maintain protection

What's Next:

We conclude our Authentication Header module with an examination of AH limitations. Despite its strengths in integrity and authentication, AH has specific constraints—particularly around confidentiality, NAT compatibility, and modern deployment scenarios—that have influenced its role in contemporary network security architectures.

Page Complete

You now understand how replay attacks work, how AH's sequence numbers and sliding window defeat them, how ESN extends protection to high-bandwidth scenarios, and how to configure and troubleshoot anti-replay in production environments.

4 / 5

Loading learning content...

Computer NetworksNetwork Layer Security

Authentication Header (AH)

LevelIntermediate

Duration55 mins

TopicNetwork Layer Security

4 / 5

Anti-Replay Protection: Defeating Packet Replay Attacks

The Threat of Replay Attacks

Learning Objectives

Understanding Replay Attacks

Attack Mechanics:

Capture Phase: Attacker intercepts packets on the network (passive eavesdropping or man-in-the-middle position)
Storage Phase: Captured packets are stored for later use
Replay Phase: Attacker retransmits captured packets at a strategically chosen time

The attack succeeds because the receiving system validates the authentication (it's genuinely valid) and processes the packet as if it were freshly transmitted.

Converting Mermaid diagram...

Real-World Replay Attack Scenarios:

Replay Attack Impact Examples
Domain	Attack Vector	Potential Damage
Financial Systems	Replay fund transfer authorizations	Duplicate transactions, fund theft
Authentication	Replay login success packets	Unauthorized access to accounts
Industrial Control	Replay control commands	Incorrect equipment operation, safety hazards
VPN Sessions	Replay session establishment	Hijack or duplicate VPN connections
Database Operations	Replay write operations	Data corruption, duplicate entries
Routing Protocols	Replay routing updates	Traffic misdirection, black holes
E-commerce	Replay purchase confirmations	Duplicate orders, inventory fraud

Why Integrity Protection Isn't Enough:

Attack Variations:

Simple Replay: Retransmit the exact captured packet
Delayed Replay: Wait for strategic timing (end of financial period, system maintenance window)
Selective Replay: Only replay certain packet types (commands vs. data)
Out-of-Order Replay: Disrupt sequences by replaying earlier packets
Inter-Session Replay: Attempt to replay packets from expired sessions into new ones

Replay Attacks Are Protocol-Agnostic

The Sequence Number Mechanism

Sender Behavior:

The sender maintains a 32-bit counter for each Security Association:

Sender Sequence Number Handling
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
// Security Association state (sender side)
struct SA_Sender {
    uint32_t sequenceNumber;     // Current counter value
    bool     useESN;             // Extended Sequence Number enabled?
    uint32_t sequenceNumberHigh; // Upper 32 bits for ESN
    // ... other SA parameters
};
 
function sendPacket(sa, packet) {
    // Increment BEFORE use—0 is never transmitted
    if (sa.useESN) {
        // 64-bit counter
        uint64_t seqNum = (sa.sequenceNumberHigh << 32) | sa.sequenceNumber;
        seqNum++;
        sa.sequenceNumber = seqNum & 0xFFFFFFFF;
        sa.sequenceNumberHigh = seqNum >> 32;
    } else {
        // 32-bit counter
        sa.sequenceNumber++;
    }
    
    // Check for overflow
    if (sa.sequenceNumber == 0 && !sa.useESN) {
        // Counter has wrapped—MUST establish new SA
        triggerRekey(sa);
        return ERROR_MUST_REKEY;
    }
    
    // Insert sequence number into AH header
    packet.ah.sequenceNumber = sa.sequenceNumber;
    
    // Compute ICV with sequence number included
    packet.ah.icv = computeICV(sa, packet);
    
    transmit(packet);
}

Critical Sender Rules:

Counter starts at 0, but first transmitted sequence number is 1
Counter MUST NOT wrap around (32-bit limit = 2³² - 1 = 4,294,967,295 packets)
If counter would overflow, new SA MUST be established before transmitting
Sequence number is included in ICV computation—it cannot be modified without invalidating authentication
Each packet gets a unique sequence number—no reuse within an SA

Sequence Number Space:

For a 32-bit sequence number:

32-bit Sequence Number Exhaustion Times
Packet Rate	Time to Exhaust
1,000 packets/second	~50 days
10,000 packets/second	~5 days
100,000 packets/second	~12 hours
1,000,000 packets/second	~72 minutes
10,000,000 packets/second	~7.2 minutes

High-Bandwidth Considerations:

At 10 Gbps with minimum-sized packets (84 bytes on wire for 64-byte payload with Ethernet overhead):

Maximum rate: ~14.88 million packets/second
Time to exhaust 32-bit space: ~5 minutes

This is why Extended Sequence Numbers (ESN) were introduced—we'll cover them shortly.

Protection Properties:

Uniqueness: Each packet has a globally unique (within the SA) identifier
Authentication: Sequence number is authenticated—cannot be modified
Ordering: Monotonic increase enables detection of reordering and replays
Freshness: Old sequence numbers are rejected, ensuring packet recency

Why Not Timestamps?

The Sliding Window Algorithm

Window Concept:

The receiver maintains:

Window size (W): Number of sequence numbers to track (typically 32 or 64)
Right edge (R): Highest sequence number received and validated
Window contents: Bitmap tracking which sequence numbers within the window have been received

Acceptable sequence numbers fall in the range: [R - W + 1, R]

Converting Mermaid diagram...

Legend:

✓ (Green): Already received and validated
□ (Yellow): Not yet received but within window (acceptable)
Old (Red): Below window, rejected
New (Blue): Above window, will advance window if ICV validates

Receiver Algorithm:

Sliding Window Anti-Replay
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
struct SA_Receiver {
    uint32_t  rightEdge;   // Highest validated sequence number
    uint64_t  bitmap;      // Bit i set = (rightEdge - i) received
    uint32_t  windowSize;  // Typically 32 or 64
};
 
function checkReplay(sa, seqNum) {
    // Case 1: Sequence number is above the window (new highest)
    if (seqNum > sa.rightEdge) {
        return ACCEPT_PENDING_ICV;  // Will advance window after ICV validates
    }
    
    // Case 2: Sequence number is below the window (too old)
    if (seqNum < sa.rightEdge - sa.windowSize + 1) {
        return REJECT_TOO_OLD;
    }
    
    // Case 3: Sequence number is within the window
    uint32_t offset = sa.rightEdge - seqNum;  // Position in bitmap
    
    if (sa.bitmap & (1ULL << offset)) {
        return REJECT_DUPLICATE;  // Already received
    }
    
    return ACCEPT_PENDING_ICV;  // Within window, not yet received
}
 
function updateWindow(sa, seqNum) {
    // Called ONLY after ICV validation succeeds
    
    if (seqNum > sa.rightEdge) {
        // Advance the window
        uint32_t shift = seqNum - sa.rightEdge;
        
        if (shift >= sa.windowSize) {
            // New packet is so far ahead, clear entire bitmap
            sa.bitmap = 1;  // Only the new packet is marked
        } else {
            // Shift bitmap and mark new packet
            sa.bitmap = (sa.bitmap << shift) | 1;
        }
        
        sa.rightEdge = seqNum;
    } else {
        // Within window—just mark it
        uint32_t offset = sa.rightEdge - seqNum;
        sa.bitmap |= (1ULL << offset);
    }
}

Window Size Selection:

The window size W determines how much out-of-order delivery is tolerated:

Window Size Trade-offs
Window Size	Memory	Tolerance	Common Use Case
32 packets	4 bytes + uint32	Moderate reordering	Low-latency networks, LAN
64 packets	8 bytes + uint32	Higher reordering	Default recommendation
128 packets	16 bytes + uint32	Significant reordering	High-latency paths, satellite
1024 packets	128 bytes + uint32	Extreme reordering	Specialized applications

State Machine Summary:

Converting Mermaid diagram...

ICV Before Window Update

Extended Sequence Numbers (ESN)

For high-bandwidth connections, 32-bit sequence numbers exhaust too quickly. Extended Sequence Numbers (ESN) provide a 64-bit sequence space while maintaining backward-compatible packet format.

ESN Design:

Full sequence number: 64 bits (16 exabytes of unique packets)
Transmitted: Only lower 32 bits (in AH header, as before)
Upper 32 bits: Maintained internally, reconstructed at receiver
ICV computation: Uses full 64-bit sequence number

Why Not Just Transmit 64 Bits?

Maintaining the 32-bit field in the header:

Preserves backward compatibility with existing implementations
Reduces packet overhead by 4 bytes per packet
At 10 Gbps, saving 4 bytes per packet = 400+ Mbps bandwidth savings

Sender ESN Handling:

ESN Sender Processing
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
function sendWithESN(sa, packet) {
    // Increment full 64-bit counter
    sa.seqLow++;
    if (sa.seqLow == 0) {
        sa.seqHigh++;  // Carry to upper 32 bits
        if (sa.seqHigh == 0) {
            // 64-bit overflow—theoretically impossible in practice
            return ERROR_COUNTER_EXHAUSTED;
        }
    }
    
    // Only lower 32 bits go in the packet
    packet.ah.sequenceNumber = sa.seqLow;
    
    // ICV computed over full 64-bit sequence
    // Upper 32 bits appended to authenticated data (not transmitted)
    packet.ah.icv = computeICVWithESN(sa, packet, sa.seqHigh);
    
    transmit(packet);
}

Receiver ESN Reconstruction:

The receiver must determine the upper 32 bits without receiving them. This is possible because:

Packets generally arrive in order (minor reordering)
The receiver knows approximately which 64-bit range the sender is in
The 32-bit window provides bounds on how far a packet can be from expected

ESN Receiver Reconstruction
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
function reconstructESN(sa, receivedSeqLow) {
    // Receiver tracks: Tl (low 32 bits of right edge), Th (high 32 bits)
    uint32_t Tl = sa.rightEdge & 0xFFFFFFFF;
    uint32_t Th = sa.rightEdge >> 32;
    uint32_t W  = sa.windowSize;
    
    // Determine which "era" the packet is from
    uint32_t seqHigh;
    
    if (receivedSeqLow >= Tl - W + 1 || Tl < W - 1) {
        // Case 1: Received seq is >= window bottom
        // Packet is in current era OR just wrapped
        if (receivedSeqLow > Tl) {
            // Packet is ahead of right edge in same era
            seqHigh = Th;
        } else if (receivedSeqLow >= Tl - W + 1) {
            // Packet is within window in same era
            seqHigh = Th;
        } else {
            // Packet appears to be from previous era (wrapped)
            // This happens when low bits wrapped but high bits didn't yet advance
            seqHigh = Th + 1;
        }
    } else if (receivedSeqLow < Tl - W + 1) {
        // Below window in current era—might be from next era
        if (receivedSeqLow < W && Tl > 0xFFFFFFFF - W) {
            // Packet from future era (receiver's low bits about to wrap)
            seqHigh = Th + 1;
        } else {
            // Truly old packet—will be rejected by anti-replay
            seqHigh = Th;
        }
    }
    
    return ((uint64_t)seqHigh << 32) | receivedSeqLow;
}

ESN in ICV Computation:

ICV_Input = IP_Header || AH_Header || Payload || SeqNumHigh(32 bits)

This ensures that even though the upper bits aren't transmitted, they're authenticated. A receiver who reconstructs the wrong upper bits will compute an incorrect ICV and reject the packet.

ESN Exhaustion:

With 64-bit sequence numbers:

64-bit ESN Exhaustion Times
Packet Rate	Time to Exhaust
1 million packets/second	~584,942 years
100 million packets/second	~5,849 years
1 billion packets/second	~585 years
100 billion packets/second	~5.8 years

ESN Negotiation

Anti-Replay Configuration and Tuning

While anti-replay is fundamental to AH security, its configuration can be tuned for specific network conditions. Understanding the parameters helps optimize security and performance.

Configuration Parameters:

Anti-Replay Configuration Options
Parameter	Typical Values	Effect of Larger	Effect of Smaller
Window Size	32-1024	More tolerance for reordering; more memory	Less memory; rejects more reordered packets
ESN	On/Off	Eliminates 32-bit exhaustion; slight CPU overhead	May require frequent rekeying at high rates
Sequence Number Check	Enabled/Disabled	Full replay protection	Vulnerable to replay; for compatibility only

Configuring Window Size:

The window size should accommodate expected network reordering:

Measure reordering: Use tools like iperf3 or specialized network analyzers
Determine 99th percentile: What's the maximum reorder distance seen?
Add safety margin: Set window to 1.5-2x the measured maximum
Monitor rejections: If legitimate packets are rejected, increase window

Example Scenario:

Corporate VPN over Internet:

Measured reordering: Usually 0-5 packets, occasionally 20-30
99th percentile: 25 packets
Recommended window: 64 packets (provides 2.5x margin)

Platform-Specific Configuration:

Configuration Examples
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Linux (ip xfrm / StrongSwan)
# Set replay window for SA
ip xfrm state add ... replay-window 64
 
# StrongSwan (ipsec.conf)
conn myvpn
    replay_window=64
    extended_sequence_numbers=yes
 
# Cisco IOS
crypto ipsec security-association replay window-size 1024
 
# Juniper JunOS
set security ipsec security-association anti-replay-window-size 64
 
# Windows (PowerShell / netsh)
# Windows uses 64 by default, configurable via registry
# HKLM\SYSTEM\CurrentControlSet\Services\PolicyAgent\Oakley
# ReplayWindowSize (DWORD)

Disabling Anti-Replay (NOT Recommended):

RFC 4302 allows disabling anti-replay for backward compatibility, but this is strongly discouraged. Legitimate reasons to disable are extremely rare:

Legacy systems that don't support anti-replay
Debugging specific interoperability issues
Multicast scenarios where per-receiver sequence tracking is impractical

If you must disable:

Document the security implication explicitly
Implement alternative replay protection at the application layer
Monitor for suspicious traffic patterns that might indicate replay attempts
Plan upgrade to supporting implementation

Never Disable for Performance

Edge Cases and Special Scenarios

While the sliding window algorithm handles most scenarios, certain edge cases require special consideration.

1. SA Renegotiation Race Condition:

When an SA is being renegotiated (rekeyed), packets may arrive on both old and new SAs:

Converting Mermaid diagram...

Mitigation: Implementations maintain both old and new SAs during transition. Each SA has its own sequence number space and window. The old SA times out after a grace period.

2. Multipath and Load Balancing:

When traffic is load-balanced across multiple paths with different latencies, reordering can be extreme:

Path A: 10ms latency
Path B: 50ms latency
Packets 1, 2, 3 → Path A
Packets 4, 5, 6 → Path B
Arrival order: 1, 2, 3, 7, 8, 9, 4, 5, 6

With window = 8, packets 4, 5, 6 might fall outside the window when they arrive.

Mitigation: Increase window size or use hash-based load balancing that keeps flows on the same path.

3. System Restart:

If the receiver restarts mid-SA:

Sequence tracking state may be lost
Attacker could replay packets from before the restart
Some implementations persist anti-replay state to disk; most don't

Mitigation: Trigger SA renegotiation on restart. Discard current SA and establish new SAs with fresh sequence number spaces.

4. Clock/Counter Desynchronization:

If sender's sequence counter is ahead of receiver's expectation (e.g., due to transmission before receiver was ready):

Legitimate packets may be rejected as "too new"
Anti-replay logic doesn't have concept of "too new" but window advancement issues may occur

Mitigation: Ensure both endpoints complete SA establishment before transmitting data. IKE handshake completion guarantees readiness.

Edge Case Summary and Mitigations
Scenario	Symptom	Mitigation
SA Rekeying	Packets on old SA rejected after new SA established	Maintain overlapping validity; old SA grace period
Multipath Reorder	Legitimate packets rejected as too old	Increase window size; use flow-based ECMP
Receiver Restart	Replay vulnerable until new SA	Trigger rekey on restart; persist state if critical
Network Partition	Large sequence gap on recovery	Rekey if gap exceeds window; monitor for attacks
Packet Loss	Gaps in sequence (not a problem)	Normal operation; window accommodates

Monitoring Anti-Replay

Security Analysis of Anti-Replay

Let's analyze the security guarantees provided by AH's anti-replay mechanism and understand its limits.

What Anti-Replay Guarantees:

Security Guarantees

•No exact replays — A packet with sequence number N, once accepted, cannot be accepted again with the same sequence number
•No stale replays — Packets with sequence numbers older than the window are rejected, preventing delayed replay
•Authenticated uniqueness — Sequence numbers are authenticated; attackers cannot forge new sequence numbers
•Bounded acceptance — Only packets within the window are candidates for acceptance, limiting vulnerability window

What Anti-Replay Does NOT Guarantee:

Limitations

•Cross-SA replay — Different SAs have independent sequence spaces; packets from SA1 can't be replayed into SA2, but this is due to different keys, not anti-replay
•Pre-window replay — Before any packets are received, the window hasn't been established; first packet has no anti-replay check (but does have ICV)
•Future packet forgery — Anti-replay doesn't prevent forging new packets (that's ICV's job); it only prevents reuse of existing packets
•Application-layer replay — If the same application operation can be performed with different packet contents, anti-replay doesn't help

Attack Resilience Analysis:

Scenario 1: Attacker captures and immediately replays

Packet with sequence N is captured
Attacker replays packet with same sequence N
If original not yet received: Replay might succeed (but attacker doesn't know this)
If original already received: Rejected as duplicate
Risk: Minimal; attacker has short window and uncertain success

Scenario 2: Attacker stores packets and replays later

Packets captured over time
Attacker replays hours/days later
All sequence numbers now below window
100% rejection rate
Risk: None

Scenario 3: Attacker attempts selective replay

Attacker wants to replay specific high-value packet
Must replay within window (before window advances past it)
Must replay before original is received
Success probability: Very low for targeted attacks

Conclusion:

Defense in Depth

Summary: Anti-Replay Mastery

We've comprehensively explored AH's anti-replay mechanism. Let's consolidate the essential concepts:

Key Takeaways

•Replay attacks bypass authentication — Valid packets remain valid; uniqueness/freshness must be tracked separately from authenticity
•Sequence numbers provide uniqueness — Each packet has a unique, authenticated, monotonically increasing sequence number in its SA
•Sliding window tolerates reordering — Accepts packets within a configurable range; rejects duplicates and stale packets
•Window updates only after ICV validation — Prevents attackers from corrupting window state with forged packets
•ESN handles high bandwidth — 64-bit sequence space prevents exhaustion; only 32 bits transmitted to save bandwidth
•Configuration matters — Window size should match network reordering characteristics; always enable anti-replay
•Edge cases require attention — SA rekeying, multipath, and restarts need careful handling to maintain protection

What's Next:

Page Complete

4 / 5