TCP Three-Way Handshake

The SYN-ACK segment is a masterpiece of protocol efficiency. In a single packet, the server accomplishes two critical objectives: acknowledging the client's connection request and proposing its own sequence number for the reverse direction. This dual-purpose design reduces Round-Trip Time (RTT) overhead by combining what could have been two separate messages.

When a server receives a valid SYN, it responds with the SYN-ACK—a segment that says, in essence: "I received your request to connect (ACK), and here is my initial sequence number for data I'll send to you (SYN)." This elegant combination makes TCP's connection establishment complete in just three packets rather than four.

The SYN-ACK segment has two simultaneous meanings, encoded in two flags that are both set:

1. ACK (Acknowledgment) — "I received your SYN"

The ACK flag being set indicates that the Acknowledgment Number field is valid. This value tells the client: "I received your SYN with ISN = X, so I expect your next byte to have sequence number X + 1."

2. SYN (Synchronize) — "Here is my ISN"

The SYN flag being set indicates that the Sequence Number field contains the server's Initial Sequence Number (ISN). This tells the client: "When I start sending you data, it will begin at sequence number Y."

The Two-Way Handshake Fallacy:

A common misconception is that TCP could work with just two packets: SYN from client, SYN-ACK from server. Why isn't this sufficient?

Consider: after the server sends SYN-ACK, it knows the client wants to connect, but the client hasn't confirmed it received the server's response. The server's sequence number hasn't been acknowledged. Without the final ACK:

1. The server doesn't know if its SYN-ACK was received
2. The server's ISN hasn't been acknowledged
3. The client could have crashed, been firewalled, or changed its mind
4. The server would enter ESTABLISHED state prematurely

The three-way handshake ensures both sides have confirmed both directions of communication are functional.

The Acknowledgment Number in the SYN-ACK is deceptively simple in concept but critical in practice. When the server sets ACK = client_ISN + 1, it's making a precise statement about what it expects next.

Acknowledgment Number Semantics:

The TCP acknowledgment number indicates the next sequence number the receiver expects. It implicitly acknowledges all bytes up to (but not including) this value.

In the SYN-ACK:

Acknowledgment Number = Client's ISN + 1

This means:

• The server has received everything up to and including the client's SYN
• The SYN itself 'consumed' one sequence number (the ISN)
• The next byte the client sends should have sequence number ISN + 1

Example Acknowledgment Calculation:

Suppose the client sends a SYN with:

Sequence Number (ISN): 1000

The server's SYN-ACK will contain:

Acknowledgment Number: 1001

This tells the client: "I've received everything through sequence 1000 (including your SYN). Send byte 1001 whenever you're ready."

When the client later sends its first data segment, it will begin:

Sequence Number: 1001
Data: "GET / HTTP/1.1\r\n..."

Cumulative Acknowledgment:

TCP uses cumulative acknowledgments—the acknowledgment number indicates all contiguous bytes up to that point have been received. This is important for the handshake:

• The SYN-ACK's ACK = ISN + 1 acknowledges the SYN
• Later data ACKs acknowledge all bytes through the ACK number
• Gaps in received data don't get acknowledged until filled (without SACK)

Just as the client proposed its ISN in the SYN, the server must propose its own ISN in the SYN-ACK. This server ISN follows the same security requirements and generation methods as the client's ISN, but the values are completely independent.

Why Independent ISNs?

TCP connections are full-duplex—both sides can send data simultaneously. Each direction has its own sequence number space:

• Client → Server: Uses client's ISN as the starting point
• Server → Client: Uses server's ISN as the starting point

These two sequence spaces are entirely separate. The client's acknowledgments reference the server's sequence space, and vice versa.

Server ISN Generation:

The server generates its ISN using the same cryptographic randomization method as the client (RFC 6528):

Server_ISN = M + F(server_ip, server_port, client_ip, client_port, secret_key)

Note the parameters are the same tuple but with consistent ordering. The function F uses a secret key known only to the host, ensuring:

1. Unpredictability to attackers — Can't guess the server's ISN before receiving SYN-ACK
2. Different from client's ISN — The hash function with different secret keys produces different values
3. Consistent per connection — Same socket pair yields same offset, helping avoid old duplicate issues

Let's examine every field in a typical SYN-ACK segment, comparing it to the original SYN to understand the response pattern:

Critical Observations:

1. Flags = 0x12 — In the 8-bit flags field, bit 1 (SYN) and bit 4 (ACK) are set: 0001 0010 = 0x12

2. Port Reversal — Source and destination ports are swapped from the SYN. The server responds from its service port to the client's ephemeral port.

3. Both Sequence and Acknowledgment Valid — Unlike the pure SYN, both fields carry meaningful values. The sequence number is the server's ISN; the acknowledgment number references the client's sequence space.

4. Server's Window — The window size advertises how much data the server can receive. This may differ significantly from the client's advertised window.

The SYN-ACK is the server's opportunity to accept, modify, or reject the optional capabilities the client proposed. The negotiation follows specific rules:

Negotiation Semantics:

Option-by-Option Analysis:

Maximum Segment Size (MSS): Each side independently declares the largest segment it can receive. The effective MSS for each direction is what that direction's receiver advertised. If client says 1460 and server says 1400:

• Client sends segments with max data payload: 1400 bytes
• Server sends segments with max data payload: 1460 bytes

Window Scale: Unlike MSS, Window Scale must be present in both SYN and SYN-ACK to be enabled. If either omits it, neither side can use scaled windows. Each side's scale factor applies to windows they advertise, not receive.

SACK Permitted: This is a binary negotiation. If both include SACK Permitted, selective acknowledgments can be used. If either omits it, SACK is unavailable for the connection.

Timestamps: If both sides include the Timestamps option, subsequent segments will carry timestamps for RTT estimation and PAWS. The server echoes the client's timestamp in the echo-reply field.

Server's Option Strategy:

Servers must balance compatibility with performance:

1. Always include MSS — Without it, 536-byte segments limit throughput
2. Always include Window Scale — Essential for high-bandwidth paths
3. Always include SACK Permitted — Critical for loss recovery
4. Include Timestamps if client does — Enables RTT measurement, helps with PAWS

Servers may deliberately omit options in constrained environments:

• Embedded systems might omit Window Scale to save buffer memory
• Security appliances might strip Timestamps to prevent timing attacks
• Middleboxes might not support certain options

When the server sends the SYN-ACK, it undergoes a critical state transition in the TCP state machine. Understanding this transition is essential for debugging connection issues and understanding resource management.

State Transition Sequence:

SYN_RECEIVED State Characteristics:

In SYN_RECEIVED, the connection is half-open:

• Server has allocated connection state (TCB)
• Server's ISN has been proposed but not acknowledged
• Server is waiting for client's ACK to complete handshake
• Retransmission timer is running (will resend SYN-ACK if no ACK arrives)
• Connection cannot transfer data yet

Timeout Behavior:

If the client's ACK never arrives, the server's SYN-ACK retransmission timer expires:

1. First timeout (~3 seconds): Retransmit SYN-ACK
2. Subsequent timeouts: Exponential backoff (6s, 12s, 24s, etc.)
3. Maximum retries reached: Connection aborted, resources freed

Typical Linux defaults: 5 retries with exponential backoff, totaling ~31 seconds before giving up.

Network packets can be lost. When the client's original SYN triggers a SYN-ACK that never reaches the client, or when the client's ACK never reaches the server, retransmission mechanisms ensure the handshake can still complete.

Scenario 1: SYN-ACK Lost (Never Reaches Client)

Scenario 2: Client's ACK Lost (Server Never Gets ACK)

Key Observations:

1. Client may enter ESTABLISHED before server — The client considers the connection established upon receiving SYN-ACK, before its ACK is delivered or processed.

2. Duplicate SYN-ACKs are handled gracefully — The client responds to duplicate SYN-ACKs by retransmitting its ACK. This is idempotent and safe.

3. Exponential backoff limits network load — Retransmissions don't flood the network; they're spaced with increasing intervals.

4. Connection eventually times out — If repeated retransmissions fail, the server abandons the connection attempt.

Examining actual SYN-ACK packets reveals implementation details and helps diagnose connection issues. Here's an annotated capture:

Detailed Analysis:

1. Flags = 0x012 (SYN + ACK) — Confirms this is the handshake response

2. Raw Acknowledgment = 2812345679 — This is exactly client's ISN (2812345678) + 1, confirming SYN receipt

3. Server's ISN — Raw sequence 1517890234, completely independent of client's ISN

4. Timestamp Echo Reply = 123456789 — Server echoes the client's timestamp from the SYN. This enables the client to measure RTT immediately.

5. Window Scale = 8 — Server's scale factor differs from client's (which was 7). Each side uses its own factor for windows it advertises.

6. MSS = 1460 — Server agrees to the same MSS as client (both using standard Ethernet values)

7. SACK Permitted present — Server agrees to selective acknowledgments

Real networks present edge cases that test TCP implementations. Understanding these scenarios helps in debugging and security analysis.

Simultaneous Open Deep Dive:

Simultaneous open is a rare but valid scenario. It occurs when:

1. Host A sends SYN to Host B
2. Host B sends SYN to Host A (before receiving A's SYN)
3. Each receives the other's SYN while in SYN_SENT state

Both hosts respond with SYN-ACK:

• Each SYN-ACK carries the sender's ISN and acknowledges the peer's ISN
• Both transition: SYN_SENT → SYN_RECEIVED → ESTABLISHED

The result is a normal established connection, just with a symmetric handshake pattern. This is rare in practice because client-server timing makes simultaneous SYN unlikely.

The SYN-ACK introduces its own security considerations, distinct from those of the initial SYN:

SYN Cookies and SYN-ACK:

When SYN cookies are enabled, the server's SYN-ACK is constructed specially:

1. No TCB allocated — Server doesn't store any connection state

2. ISN encodes connection info — The sequence number in the SYN-ACK is a cryptographic hash of connection parameters including:

   • Source/destination IPs
   • Source/destination ports
   • A secret key
   • A timestamp counter
   • The client's MSS (limited encoding)

3. Stateless operation — When the client's ACK arrives, the server:

   • Validates the acknowledgment number as a valid cookie
   • Reconstructs connection parameters from the cookie
   • Creates TCB only after validation

This elegantly prevents SYN flood attacks by deferring resource allocation until the handshake completes.

We've thoroughly examined the SYN-ACK segment—the clever dual-purpose response that makes TCP connection establishment efficient. Let's consolidate the essential knowledge:

What's Next:

With the server's SYN-ACK sent, the handshake is two-thirds complete. The client has proposed its sequence number and received the server's. But the connection isn't established until the client confirms receipt. In the next page, we'll examine the final ACK—the segment that completes the three-way handshake and transitions both endpoints to the ESTABLISHED state.