HTTP Request and Response Format

Every interaction on the web—every page load, API call, image download, and form submission—consists of HTTP messages flowing between clients and servers. Understanding the precise structure of these messages is fundamental to debugging network issues, building APIs, and comprehending how the web actually works at the protocol level.

HTTP messages are simple text-based protocols (in HTTP/1.x) or binary-framed (in HTTP/2 and HTTP/3), but they share a common conceptual structure:

┌──────────────────────────────┐
│         Start Line           │  ← Request line or Status line
├──────────────────────────────┤
│         Headers              │  ← Key: Value pairs (metadata)
│         Headers              │
│         Headers              │
├──────────────────────────────┤
│                              │
│         Body                 │  ← Payload data (optional)
│         (Optional)           │
│                              │
└──────────────────────────────┘

This page provides a comprehensive, authoritative examination of HTTP message format—how requests and responses are structured, encoded, and transmitted across the network.

An HTTP request is a message sent from client to server, asking for a resource or action. In HTTP/1.1 (the most human-readable version), a request has this structure:

Method SP Request-Target SP HTTP-Version CRLF
Header-Field CRLF
Header-Field CRLF
...
CRLF
[message-body]

Where:

• SP = Space (ASCII 32)
• CRLF = Carriage Return + Line Feed (\r , ASCII 13 + 10)

The Request Line

The first line of every HTTP request contains three components:

POST /api/users HTTP/1.1
└──┘ └─────────┘ └───────┘
 │        │          └── HTTP Version
 │        └── Request Target (path + query)
 └── Method (verb)

Method: The action to perform (GET, POST, PUT, DELETE, etc.)

Request-Target: Usually the path component of the URL:

• origin-form: /path/to/resource?query=value (most common)
• absolute-form: http://example.com/path (used with proxies)
• authority-form: example.com:443 (CONNECT method only)
• asterisk-form: * (OPTIONS for the server itself)

HTTP-Version: HTTP/1.0, HTTP/1.1, or HTTP/2 (though HTTP/2 is binary, tools often display it this way)

Request Headers

Headers follow the request line, each on its own line:

Header-Name: Header-Value CRLF

Key rules:

• Header names are case-insensitive (Content-Type = content-type)
• Whitespace after the colon is optional but conventional
• Headers end with a blank line (CRLF alone)

Minimum required headers (HTTP/1.1):

GET / HTTP/1.1
Host: example.com

The Host header is required in HTTP/1.1 because:

• Multiple domains can share one IP address (virtual hosting)
• Without Host, the server wouldn't know which site you want

Request Body

After the blank line, the optional body contains the payload:

Content-Type: application/json
Content-Length: 45

{"username": "john", "password": "secret123"}

Body rules:

• Content-Length specifies exact byte count
• Transfer-Encoding: chunked allows streaming without knowing total size
• GET requests typically don't have bodies (though technically allowed)
• POST, PUT, PATCH typically include bodies

An HTTP response follows a similar structure to requests, with the first line being the status line instead of the request line:

HTTP-Version SP Status-Code SP Reason-Phrase CRLF
Header-Field CRLF
Header-Field CRLF
...
CRLF
[message-body]

The Status Line

HTTP/1.1 200 OK
└───────┘ └─┘ └┘
    │      │   └── Reason Phrase (human-readable, optional)
    │      └── Status Code (3 digits)
    └── HTTP Version

HTTP-Version: Must match the highest version the server supports for that request

Status-Code: The 3-digit result code (200, 404, 500, etc.)

Reason-Phrase: Human-readable explanation

• HTTP/1.x: Present but not parsed by clients
• HTTP/2+: Not transmitted (removed from the protocol)

Response Headers

Response headers provide metadata about the response:

Date: Sat, 18 Jan 2025 12:00:00 GMT        # When response was generated
Server: nginx/1.24.0                        # Server software (optional)
Content-Type: application/json; charset=utf-8  # Body format
Content-Length: 156                         # Body size in bytes
Cache-Control: private, max-age=3600        # Caching instructions
ETag: "abc123def456"                        # Resource version identifier

Date header: Required for responses with caching implications. Format: RFC 5322 date (e.g., Sat, 18 Jan 2025 12:00:00 GMT).

Response Body

The body appears after the blank line, with size determined by:

1. Content-Length header (fixed size)
2. Transfer-Encoding: chunked (streaming)
3. Connection close (HTTP/1.0 style; deprecated)

The message body can be transmitted in several ways, each with different characteristics and use cases.

Content-Length Encoding (Fixed Size)

The simplest approach—declare the exact byte count:

Content-Length: 156

{"id": 12345, "name": "John Doe", ...}

Pros:

• Client knows exactly when body ends
• Allows connection reuse (HTTP/1.1 keep-alive)
• Required for requests in HTTP/1.1

Cons:

• Must know size before sending
• Can't stream dynamically generated content

Chunked Transfer Encoding (Streaming)

For dynamic content where size isn't known in advance:

Transfer-Encoding: chunked

1a
This is the first chunk.
15
This is chunk two.
0

Format:

chunk-size (hex) CRLF
chunk-data CRLF
...
0 CRLF
CRLF

Content-Encoding (Compression)

Separate from transfer encoding, content encoding compresses the body:

Content-Type: application/json
Content-Encoding: gzip
Content-Length: 312   # Compressed size

(gzip-compressed JSON data)

Common encodings:

• gzip — GNU zip, widely supported
• deflate — zlib compression (less common)
• br — Brotli, best compression ratio for text
• identity — No compression (default)

Combining with chunked:

Transfer-Encoding: chunked
Content-Encoding: gzip

(chunked, gzip-compressed data)

Body Length Determination Priority

When determining body length, HTTP clients follow this priority:

1. 1xx, 204, 304 responses: No body, always
2. HEAD response: No body (but headers may include Content-Length)
3. Transfer-Encoding: chunked: Use chunk lengths
4. Content-Length: Use specified byte count
5. Connection: close: Read until connection closes (HTTP/1.0 only)

Important: Transfer-Encoding takes precedence over Content-Length. If both present, Content-Length should be ignored.

Let's examine complete HTTP conversations for common scenarios.

While HTTP/1.1 uses text-based messages, HTTP/2 and HTTP/3 use binary framing for efficiency. The semantic meaning remains the same, but the wire format is fundamentally different.

HTTP/2 Pseudo-Headers

HTTP/2 replaces the request line and status line with pseudo-headers (prefixed with :):

Request pseudo-headers:

:method: GET
:scheme: https
:authority: api.example.com
:path: /api/users?id=123

Response pseudo-headers:

:status: 200

Note: The reason phrase ("OK", "Not Found") is not transmitted in HTTP/2+.

Binary Framing

HTTP/2 divides messages into frames:

+-----------------------------------------------+
|                Frame Header (9 bytes)          |
+-----------------------------------------------+
|  Length (24)  |  Type (8)  |  Flags (8)  |    |
+-----------------------------------------------+
|                  Stream Identifier (32)        |
+-----------------------------------------------+
|                  Frame Payload                 |
+-----------------------------------------------+

Frame types:

• DATA: Request/response body
• HEADERS: HTTP headers
• PRIORITY: Stream priority
• SETTINGS: Connection settings
• PUSH_PROMISE: Server push
• GOAWAY: Graceful shutdown
• WINDOW_UPDATE: Flow control
• CONTINUATION: Large header continuation

Header Compression

HTTP/1.1: Headers sent as plain text on every request, often repeating (Host, User-Agent, Cookie, etc.)

HTTP/2 (HPACK): Headers are:

• Indexed in a table (common headers have numeric codes)
• Huffman-encoded (character-level compression)
• Incrementally updated (only send changes)

HTTP/3 (QPACK): Similar to HPACK but designed to work with QUIC's independent streams (avoids head-of-line blocking on header compression).

Multiplexing and Streams

# HTTP/1.1: Sequential requests
Connection ─────┬─── Request 1 ─── Response 1
                ├─── Request 2 ─── Response 2  (must wait)
                └─── Request 3 ─── Response 3  (must wait)

# HTTP/2: Multiplexed streams over single connection
Connection ─────┬─── Stream 1: Request 1 ────────────
                ├─── Stream 3: Request 2 ─────────
                ├─── Stream 5: Request 3 ──────────
                └─── (responses interleaved)

Each stream is independent; a slow response doesn't block others.

Understanding how HTTP connections work is essential for performance optimization and debugging.

HTTP/1.0: Connection per Request

Original HTTP opened a new TCP connection for each request:

1. TCP handshake (3 packets)
2. Send HTTP request
3. Receive HTTP response
4. TCP teardown (4 packets)
5. Repeat for next request

Problem: TCP handshake and slow-start made this extremely inefficient for pages with many resources.

HTTP/1.1: Keep-Alive

HTTP/1.1 made persistent connections the default:

Connection: keep-alive  # Default in HTTP/1.1

Multiple requests can use the same connection:

1. TCP handshake
2. Request 1 → Response 1
3. Request 2 → Response 2
4. Request 3 → Response 3
...
N. Connection idle → close

Connection header values:

• keep-alive: Maintain connection (default)
• close: Close after this response

Pipelining (Failed Experiment)

HTTP/1.1 also defined pipelining—sending multiple requests without waiting for responses:

Client: Request 1
Client: Request 2 (without waiting)
Client: Request 3 (without waiting)
Server: Response 1
Server: Response 2
Server: Response 3

Problem: Responses must come in order. A slow Response 1 blocks Response 2 and 3 (head-of-line blocking). Most browsers disabled pipelining.

Browser Connection Limits

Browsers limit concurrent connections per domain:

Workarounds (HTTP/1.1 era):

• Domain sharding: static1.example.com, static2.example.com
• Sprite images, concatenated CSS/JS

HTTP/2 solution: Single connection, unlimited multiplexed streams (typically 100-256 concurrent).

Connection: close

To explicitly close after a response:

# Request
GET /final-request HTTP/1.1
Host: example.com
Connection: close

# Response
HTTP/1.1 200 OK
Connection: close
Content-Length: 42

(body)
(connection closed after response)

Mastering debugging tools is essential for working with HTTP at the protocol level.

Common Debugging Scenarios

"Why isn't my request working?"

1. Check the status code first
2. Examine response headers (Content-Type, error details)
3. Look at response body for error messages
4. Verify request headers (Authorization, Content-Type)
5. Compare with a working request

"Why isn't caching working?"

1. Check Cache-Control header in response
2. Verify Vary header isn't too broad
3. Look for no-cache/no-store directives
4. Confirm ETag or Last-Modified present
5. Check if browser is in "disable cache" mode

"Why is my request slow?"

1. Use browser Network tab to see timing breakdown:
   • DNS lookup time
   • TCP connection time
   • TLS handshake time
   • Time to First Byte (TTFB)
   • Content download time
2. Check Content-Encoding (is compression enabled?)
3. Look for redirect chains
4. Verify connection reuse (keep-alive)