Http Methods And Status - Learning Module

Loading content...

0/240

HTTP Request and Response Format

The Anatomy of HTTP Messages

Every interaction on the web—every page load, API call, image download, and form submission—consists of HTTP messages flowing between clients and servers. Understanding the precise structure of these messages is fundamental to debugging network issues, building APIs, and comprehending how the web actually works at the protocol level.

HTTP messages are simple text-based protocols (in HTTP/1.x) or binary-framed (in HTTP/2 and HTTP/3), but they share a common conceptual structure:

┌──────────────────────────────┐
│         Start Line           │  ← Request line or Status line
├──────────────────────────────┤
│         Headers              │  ← Key: Value pairs (metadata)
│         Headers              │
│         Headers              │
├──────────────────────────────┤
│                              │
│         Body                 │  ← Payload data (optional)
│         (Optional)           │
│                              │
└──────────────────────────────┘

This page provides a comprehensive, authoritative examination of HTTP message format—how requests and responses are structured, encoded, and transmitted across the network.

Learning Objectives

By the end of this page, you will: • Understand the complete structure of HTTP requests and responses • Parse and construct HTTP messages manually • Recognize differences between HTTP/1.1, HTTP/2, and HTTP/3 message formats • Debug HTTP issues using raw message analysis • Apply best practices for message construction in APIs

HTTP Request Format

An HTTP request is a message sent from client to server, asking for a resource or action. In HTTP/1.1 (the most human-readable version), a request has this structure:

Method SP Request-Target SP HTTP-Version CRLF
Header-Field CRLF
Header-Field CRLF
...
CRLF
[message-body]

Where:

SP = Space (ASCII 32)
CRLF = Carriage Return + Line Feed (\r , ASCII 13 + 10)

Complete HTTP Request Example
HTTP
1
2
3
4
5
6
7
8
9
10
11
POST /api/users HTTP/1.1
Host: api.example.com
Content-Type: application/json
Content-Length: 89
Accept: application/json
Accept-Encoding: gzip, deflate, br
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
User-Agent: MyApp/1.0 (Windows NT 10.0; Win64; x64)
X-Request-ID: 7a3b2c1d-4e5f-6789-abcd-ef0123456789
 
{"username": "johndoe", "email": "john@example.com", "preferences": {"theme": "dark"}}

The Request Line

The first line of every HTTP request contains three components:

POST /api/users HTTP/1.1
└──┘ └─────────┘ └───────┘
 │        │          └── HTTP Version
 │        └── Request Target (path + query)
 └── Method (verb)

Method: The action to perform (GET, POST, PUT, DELETE, etc.)

Request-Target: Usually the path component of the URL:

origin-form: /path/to/resource?query=value (most common)
absolute-form: http://example.com/path (used with proxies)
authority-form: example.com:443 (CONNECT method only)
asterisk-form: * (OPTIONS for the server itself)

HTTP-Version: HTTP/1.0, HTTP/1.1, or HTTP/2 (though HTTP/2 is binary, tools often display it this way)

Request Headers

Headers follow the request line, each on its own line:

Header-Name: Header-Value CRLF

Key rules:

Header names are case-insensitive (Content-Type = content-type)
Whitespace after the colon is optional but conventional
Headers end with a blank line (CRLF alone)

Minimum required headers (HTTP/1.1):

GET / HTTP/1.1
Host: example.com

The Host header is required in HTTP/1.1 because:

Multiple domains can share one IP address (virtual hosting)
Without Host, the server wouldn't know which site you want

Request Body

After the blank line, the optional body contains the payload:

Content-Type: application/json
Content-Length: 45

{"username": "john", "password": "secret123"}

Body rules:

Content-Length specifies exact byte count
Transfer-Encoding: chunked allows streaming without knowing total size
GET requests typically don't have bodies (though technically allowed)
POST, PUT, PATCH typically include bodies

Raw Request Construction

You can send raw HTTP requests using tools like netcat or telnet:

echo -e "GET / HTTP/1.1\r
Host: example.com\r
\r
" | nc example.com 80

Or use curl --verbose to see the exact request being sent:

curl -v https://api.example.com/users

HTTP Response Format

An HTTP response follows a similar structure to requests, with the first line being the status line instead of the request line:

HTTP-Version SP Status-Code SP Reason-Phrase CRLF
Header-Field CRLF
Header-Field CRLF
...
CRLF
[message-body]

Complete HTTP Response Example
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
HTTP/1.1 200 OK
Date: Sat, 18 Jan 2025 12:00:00 GMT
Server: nginx/1.24.0
Content-Type: application/json; charset=utf-8
Content-Length: 156
Cache-Control: private, max-age=3600
ETag: "abc123def456"
X-Request-ID: 7a3b2c1d-4e5f-6789-abcd-ef0123456789
Connection: keep-alive
 
{
  "id": 12345,
  "username": "johndoe",
  "email": "john@example.com",
  "createdAt": "2025-01-18T12:00:00Z",
  "status": "active"
}

The Status Line

HTTP/1.1 200 OK
└───────┘ └─┘ └┘
    │      │   └── Reason Phrase (human-readable, optional)
    │      └── Status Code (3 digits)
    └── HTTP Version

HTTP-Version: Must match the highest version the server supports for that request

Status-Code: The 3-digit result code (200, 404, 500, etc.)

Reason-Phrase: Human-readable explanation

HTTP/1.x: Present but not parsed by clients
HTTP/2+: Not transmitted (removed from the protocol)

Response Headers

Response headers provide metadata about the response:

Date: Sat, 18 Jan 2025 12:00:00 GMT        # When response was generated
Server: nginx/1.24.0                        # Server software (optional)
Content-Type: application/json; charset=utf-8  # Body format
Content-Length: 156                         # Body size in bytes
Cache-Control: private, max-age=3600        # Caching instructions
ETag: "abc123def456"                        # Resource version identifier

Date header: Required for responses with caching implications. Format: RFC 5322 date (e.g., Sat, 18 Jan 2025 12:00:00 GMT).

Response Body

The body appears after the blank line, with size determined by:

Content-Length header (fixed size)
Transfer-Encoding: chunked (streaming)
Connection close (HTTP/1.0 style; deprecated)

Response Body Rules by Status Code
Status Code	Body Allowed?	Notes
1xx Informational	No	Never contains a body
2xx Success	Usually	204 No Content has no body
3xx Redirection	Usually not	May contain redirect message
4xx Client Error	Usually	Error details in body
5xx Server Error	Usually	Error details in body
204 No Content	Never	Must not have body
304 Not Modified	Never	Client uses cached body

Message Body Encoding

The message body can be transmitted in several ways, each with different characteristics and use cases.

Content-Length Encoding (Fixed Size)

The simplest approach—declare the exact byte count:

Content-Length: 156

{"id": 12345, "name": "John Doe", ...}

Pros:

Client knows exactly when body ends
Allows connection reuse (HTTP/1.1 keep-alive)
Required for requests in HTTP/1.1

Cons:

Must know size before sending
Can't stream dynamically generated content

Chunked Transfer Encoding (Streaming)

For dynamic content where size isn't known in advance:

Transfer-Encoding: chunked

1a
This is the first chunk.
15
This is chunk two.
0

Format:

chunk-size (hex) CRLF
chunk-data CRLF
...
0 CRLF
CRLF

Chunked Transfer Encoding Example
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
HTTP/1.1 200 OK
Content-Type: text/plain
Transfer-Encoding: chunked
 
7\r
 
Hello, \r
 
6\r
 
World!\r
 
0\r
 
\r
 
 
# Decoded result: "Hello, World!"
 
# Chunk breakdown:
# "7" = 7 bytes in hex (7 in decimal)
# "Hello, " = 7 bytes
# "6" = 6 bytes in hex (6 in decimal)  
# "World!" = 6 bytes
# "0" = Final chunk (0 bytes = end of body)

Content-Encoding (Compression)

Separate from transfer encoding, content encoding compresses the body:

Content-Type: application/json
Content-Encoding: gzip
Content-Length: 312   # Compressed size

(gzip-compressed JSON data)

Common encodings:

gzip — GNU zip, widely supported
deflate — zlib compression (less common)
br — Brotli, best compression ratio for text
identity — No compression (default)

Combining with chunked:

Transfer-Encoding: chunked
Content-Encoding: gzip

(chunked, gzip-compressed data)

Body Length Determination Priority

When determining body length, HTTP clients follow this priority:

1xx, 204, 304 responses: No body, always
HEAD response: No body (but headers may include Content-Length)
Transfer-Encoding: chunked: Use chunk lengths
Content-Length: Use specified byte count
Connection: close: Read until connection closes (HTTP/1.0 only)

Important: Transfer-Encoding takes precedence over Content-Length. If both present, Content-Length should be ignored.

HTTP Request Smuggling

Disagreements between how different servers interpret Content-Length and Transfer-Encoding can lead to HTTP Request Smuggling attacks.

If a frontend proxy sees:

Content-Length: 44
Transfer-Encoding: chunked

And interprets Content-Length, while the backend interprets chunked, an attacker can "smuggle" a second request inside the first.

Prevention: Reject requests with both headers, or ensure consistent handling across all servers in the chain.

Complete Request/Response Examples

Let's examine complete HTTP conversations for common scenarios.

Scenario 1: Simple GET Request
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
# Request
GET /api/products/42 HTTP/1.1
Host: api.store.com
Accept: application/json
Accept-Encoding: gzip, deflate, br
User-Agent: Mozilla/5.0 Chrome/120.0.0.0
Authorization: Bearer token123
If-None-Match: "v1-etag-abc"
 
# Response (Cache Hit - Not Modified)
HTTP/1.1 304 Not Modified
Date: Sat, 18 Jan 2025 12:00:00 GMT
ETag: "v1-etag-abc"
Cache-Control: private, max-age=3600
 
# Response (Cache Miss - Full Response)
HTTP/1.1 200 OK
Date: Sat, 18 Jan 2025 12:00:00 GMT
Content-Type: application/json; charset=utf-8
Content-Encoding: gzip
Content-Length: 412
ETag: "v2-etag-xyz"
Cache-Control: private, max-age=3600
Vary: Accept-Encoding
 
(gzip-compressed JSON body)

Scenario 2: POST with JSON Body
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# Request
POST /api/orders HTTP/1.1
Host: api.store.com
Content-Type: application/json
Content-Length: 156
Accept: application/json
Authorization: Bearer token123
Idempotency-Key: order-7a3b2c1d-unique
 
{
  "productId": 42,
  "quantity": 2,
  "shippingAddress": {
    "city": "San Francisco",
    "zip": "94102"
  }
}
 
# Response (Created)
HTTP/1.1 201 Created
Date: Sat, 18 Jan 2025 12:00:00 GMT
Content-Type: application/json
Content-Length: 234
Location: /api/orders/78901
X-Request-ID: req-abc123
 
{
  "orderId": 78901,
  "status": "pending",
  "total": 59.98,
  "createdAt": "2025-01-18T12:00:00Z",
  "links": {
    "self": "/api/orders/78901",
    "payment": "/api/orders/78901/pay"
  }
}

Scenario 3: File Upload (Multipart)
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
# Request
POST /api/documents HTTP/1.1
Host: api.example.com
Content-Type: multipart/form-data; boundary=----Boundary123
Content-Length: 2456
Authorization: Bearer token123
 
------Boundary123
Content-Disposition: form-data; name="title"
 
Annual Report 2024
------Boundary123
Content-Disposition: form-data; name="file"; filename="report.pdf"
Content-Type: application/pdf
 
%PDF-1.7
(binary PDF content...)
%%EOF
------Boundary123--
 
# Response
HTTP/1.1 201 Created
Date: Sat, 18 Jan 2025 12:00:00 GMT
Content-Type: application/json
Location: /api/documents/doc-xyz789
 
{
  "documentId": "doc-xyz789",
  "title": "Annual Report 2024",
  "filename": "report.pdf",
  "size": 2048576,
  "mimeType": "application/pdf",
  "uploadedAt": "2025-01-18T12:00:00Z"
}

Scenario 4: Error Response
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
# Request
POST /api/users HTTP/1.1
Host: api.example.com
Content-Type: application/json
Content-Length: 45
 
{"email": "invalid", "password": "123"}
 
# Response (Validation Error)
HTTP/1.1 422 Unprocessable Entity
Date: Sat, 18 Jan 2025 12:00:00 GMT
Content-Type: application/problem+json
Content-Length: 312
 
{
  "type": "https://api.example.com/errors/validation",
  "title": "Validation Failed",
  "status": 422,
  "detail": "Request body contains invalid data",
  "instance": "/api/users",
  "errors": [
    {"field": "email", "code": "invalid_format", "message": "Invalid email format"},
    {"field": "password", "code": "too_short", "message": "Password must be at least 8 characters"}
  ]
}

HTTP/2 and HTTP/3 Message Format

While HTTP/1.1 uses text-based messages, HTTP/2 and HTTP/3 use binary framing for efficiency. The semantic meaning remains the same, but the wire format is fundamentally different.

HTTP Version Message Format Comparison
Aspect	HTTP/1.1	HTTP/2	HTTP/3
Format	Text-based	Binary frames	Binary frames (QUIC)
Headers	Plaintext, each request	HPACK compressed	QPACK compressed
Multiplexing	None (pipelining failed)	Streams over single TCP	Streams over QUIC
Request line	GET /path HTTP/1.1	:method: GET, :path: /path	:method: GET, :path: /path
Status line	HTTP/1.1 200 OK	:status: 200	:status: 200
Connection	Per-request or keep-alive	Single connection, many streams	QUIC connection, many streams

HTTP/2 Pseudo-Headers

HTTP/2 replaces the request line and status line with pseudo-headers (prefixed with :):

Request pseudo-headers:

:method: GET
:scheme: https
:authority: api.example.com
:path: /api/users?id=123

Response pseudo-headers:

:status: 200

Note: The reason phrase ("OK", "Not Found") is not transmitted in HTTP/2+.

Binary Framing

HTTP/2 divides messages into frames:

+-----------------------------------------------+
|                Frame Header (9 bytes)          |
+-----------------------------------------------+
|  Length (24)  |  Type (8)  |  Flags (8)  |    |
+-----------------------------------------------+
|                  Stream Identifier (32)        |
+-----------------------------------------------+
|                  Frame Payload                 |
+-----------------------------------------------+

Frame types:

DATA: Request/response body
HEADERS: HTTP headers
PRIORITY: Stream priority
SETTINGS: Connection settings
PUSH_PROMISE: Server push
GOAWAY: Graceful shutdown
WINDOW_UPDATE: Flow control
CONTINUATION: Large header continuation

HTTP/2 Conceptual Message (as displayed by tools)
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
# HTTP/2 Request (as shown by debugging tools)
:method: POST
:scheme: https
:authority: api.example.com
:path: /api/users
content-type: application/json
accept: application/json
authorization: Bearer token123
 
{"username": "johndoe", "email": "john@example.com"}
 
# HTTP/2 Response
:status: 201
content-type: application/json
location: /api/users/12345
 
{"id": 12345, "username": "johndoe", "createdAt": "2025-01-18T12:00:00Z"}
 
# What actually goes on the wire:
# - HEADERS frame with HPACK-compressed pseudo-headers and headers
# - DATA frame with the body
# - Both use stream ID to associate request/response

Header Compression

HTTP/1.1: Headers sent as plain text on every request, often repeating (Host, User-Agent, Cookie, etc.)

HTTP/2 (HPACK): Headers are:

Indexed in a table (common headers have numeric codes)
Huffman-encoded (character-level compression)
Incrementally updated (only send changes)

HTTP/3 (QPACK): Similar to HPACK but designed to work with QUIC's independent streams (avoids head-of-line blocking on header compression).

Multiplexing and Streams

# HTTP/1.1: Sequential requests
Connection ─────┬─── Request 1 ─── Response 1
                ├─── Request 2 ─── Response 2  (must wait)
                └─── Request 3 ─── Response 3  (must wait)

# HTTP/2: Multiplexed streams over single connection
Connection ─────┬─── Stream 1: Request 1 ────────────
                ├─── Stream 3: Request 2 ─────────
                ├─── Stream 5: Request 3 ──────────
                └─── (responses interleaved)

Each stream is independent; a slow response doesn't block others.

Debugging HTTP/2 and HTTP/3

Since HTTP/2+ is binary, you can't use telnet to debug. Use:

• Browser DevTools: Network tab shows HTTP/2 as normal headers • curl: curl --http2 -v https://example.com • Wireshark: Decodes HTTP/2 frames (requires decryption for HTTPS) • nghttp: HTTP/2 command-line tool • h2load: HTTP/2 benchmarking tool

Connection Management

Understanding how HTTP connections work is essential for performance optimization and debugging.

HTTP/1.0: Connection per Request

Original HTTP opened a new TCP connection for each request:

1. TCP handshake (3 packets)
2. Send HTTP request
3. Receive HTTP response
4. TCP teardown (4 packets)
5. Repeat for next request

Problem: TCP handshake and slow-start made this extremely inefficient for pages with many resources.

HTTP/1.1: Keep-Alive

HTTP/1.1 made persistent connections the default:

Connection: keep-alive  # Default in HTTP/1.1

Multiple requests can use the same connection:

1. TCP handshake
2. Request 1 → Response 1
3. Request 2 → Response 2
4. Request 3 → Response 3
...
N. Connection idle → close

Connection header values:

keep-alive: Maintain connection (default)
close: Close after this response

Pipelining (Failed Experiment)

HTTP/1.1 also defined pipelining—sending multiple requests without waiting for responses:

Client: Request 1
Client: Request 2 (without waiting)
Client: Request 3 (without waiting)
Server: Response 1
Server: Response 2
Server: Response 3

Problem: Responses must come in order. A slow Response 1 blocks Response 2 and 3 (head-of-line blocking). Most browsers disabled pipelining.

Connection Strategies by HTTP Version
Version	Connections	Multiplexing	Head-of-Line Blocking
HTTP/1.0	One per request	No	N/A
HTTP/1.1	Persistent, 6-8 per domain	No (pipelining failed)	Yes, at HTTP level
HTTP/2	Single per domain	Yes, streams	Yes, at TCP level
HTTP/3	Single (QUIC)	Yes, independent streams	No (per-stream)

Browser Connection Limits

Browsers limit concurrent connections per domain:

Browser	Connections per domain	Total connections
Chrome	6	256
Firefox	6	256
Safari	6	Varies

Workarounds (HTTP/1.1 era):

Domain sharding: static1.example.com, static2.example.com
Sprite images, concatenated CSS/JS

HTTP/2 solution: Single connection, unlimited multiplexed streams (typically 100-256 concurrent).

Connection: close

To explicitly close after a response:

# Request
GET /final-request HTTP/1.1
Host: example.com
Connection: close

# Response
HTTP/1.1 200 OK
Connection: close
Content-Length: 42

(body)
(connection closed after response)

Debugging HTTP Messages

Mastering debugging tools is essential for working with HTTP at the protocol level.

curl Verbose Output
Bash
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
$ curl -v https://api.example.com/users/123
 
* Trying 93.184.216.34:443...
* Connected to api.example.com (93.184.216.34) port 443
* TLS 1.3 connection using TLS_AES_256_GCM_SHA384
* Server certificate: api.example.com
> GET /users/123 HTTP/2
> Host: api.example.com
> User-Agent: curl/8.4.0
> Accept: */*
>
< HTTP/2 200 
< date: Sat, 18 Jan 2025 12:00:00 GMT
< content-type: application/json
< content-length: 89
<
{"id": 123, "name": "John Doe", "email": "john@example.com"}
 
# Key:
# > = Request (client to server)
# < = Response (server to client)
# * = Connection info

HTTP Debugging Tools
Tool	Use Case	Key Features
Browser DevTools	Web debugging	Network tab, timing, headers, preview
curl	Command-line requests	`-v` verbose, `--trace` raw bytes, `-i` show headers
httpie	Human-friendly CLI	Colored output, JSON by default
Postman/Insomnia	API testing GUI	Collections, environments, history
Wireshark	Packet-level analysis	Full protocol decode, filtering
mitmproxy	HTTP proxy/intercept	Modify requests in-flight
Charles Proxy	macOS proxy	SSL proxying, throttling
Fiddler	Windows proxy	Full HTTP debugging, scripting

Common Debugging Scenarios

"Why isn't my request working?"

Check the status code first
Examine response headers (Content-Type, error details)
Look at response body for error messages
Verify request headers (Authorization, Content-Type)
Compare with a working request

"Why isn't caching working?"

Check Cache-Control header in response
Verify Vary header isn't too broad
Look for no-cache/no-store directives
Confirm ETag or Last-Modified present
Check if browser is in "disable cache" mode

"Why is my request slow?"

Use browser Network tab to see timing breakdown:
- DNS lookup time
- TCP connection time
- TLS handshake time
- Time to First Byte (TTFB)
- Content download time
Check Content-Encoding (is compression enabled?)
Look for redirect chains
Verify connection reuse (keep-alive)

Inspecting Raw Bytes

To see the exact bytes sent/received:

curl --trace - https://example.com

This shows hexadecimal dumps of the actual wire format, useful for debugging encoding issues or binary protocols.

Summary: HTTP Message Mastery

Understanding HTTP message format at the protocol level enables effective debugging, API design, and performance optimization.

Key Takeaways

•Request format: Request line + headers + blank line + body
•Response format: Status line + headers + blank line + body
•Content-Length vs Chunked: Fixed-size vs streaming transfer
•HTTP/2+ uses binary framing: Pseudo-headers replace start lines
•Persistent connections are default: Keep-alive in HTTP/1.1, single connection in HTTP/2+
•Debugging tools are essential: curl -v, browser DevTools, Wireshark for deep analysis

Module Complete

Congratulations! You have completed the HTTP Methods and Status module. You now have a comprehensive understanding of:

• HTTP methods (GET, POST, PUT, DELETE) and their semantic properties • HTTP headers for caching, security, content negotiation, and CORS • Status codes across all five categories (1xx-5xx) • Content types and MIME type structure • HTTP request and response message format

This foundation prepares you for advanced topics like HTTP/1.1, HTTP/2, HTTP/3, and HTTPS covered in subsequent modules.