HTTP Overview

When you click a link, a conversation begins—a precise, structured exchange between your browser and a distant server. This conversation follows a strict pattern: your browser asks for something (a request), and the server provides something in return (a response). This pattern—the request-response model—is the heartbeat of the web.

Every web page load, every API call, every image displayed involves this exchange. Sometimes hundreds of these conversations happen in parallel for a single page. Understanding this model deeply is essential for any engineer building web systems, debugging production issues, or optimizing application performance.

This page dissects the request-response model completely. You'll learn the exact structure of HTTP messages, the role of each component, how requests and responses relate, and the subtle behaviors that impact real-world applications. By the end, you'll read HTTP traffic like a native language.

HTTP operates on a simple but powerful pattern: clients send requests, servers send responses. This pattern defines the entire structure of web communication.

The Basic Exchange:

┌─────────┐                           ┌─────────┐
│  Client │ ───── HTTP Request ─────> │  Server │
│         │ <──── HTTP Response ───── │         │
└─────────┘                           └─────────┘

This exchange is synchronous from the client's perspective—the client sends a request and waits for a response before proceeding. The server processes the request independently and responds when ready.

Key Characteristics:

1. Client-Initiated: The server cannot spontaneously send data to the client. All communication begins with a client request. (WebSocket and Server-Sent Events extend HTTP to enable server-initiated messages.)

2. One Response Per Request: Each request receives exactly one response. The response either provides the requested resource, redirects to another location, or indicates an error.

3. Stateless: Each request-response pair is independent. The server doesn't inherently remember previous requests from the same client. (Applications layer state through cookies, tokens, and session mechanisms.)

4. Sequential or Parallel: Clients can send multiple requests. In HTTP/1.1, requests on the same connection are typically processed sequentially. HTTP/2 and HTTP/3 multiplex requests, processing them in parallel.

Why This Pattern Works:

The request-response model maps naturally to human information-seeking behavior:

• 'I want this document' → 'Here it is'
• 'Submit this form' → 'Form received, here's the result'
• 'Does this resource exist?' → 'Yes/No, here's what I know'

This intuitive mapping made the web easy to understand and use. Complex interactions compose from simple request-response pairs.

An HTTP request is a text message (in HTTP/1.1) or binary frame (in HTTP/2+) sent by the client to specify what resource it wants and how it wants it. Every request has three potential parts:

1. Request Line (or Start Line)
2. Headers
3. Body (optional)

Let's examine each with precision.

Part 1: The Request Line

The first line of every HTTP request contains three elements:

METHOD  REQUEST-URI  HTTP-VERSION
GET     /api/users/123   HTTP/1.1

• Method: The action to perform (GET, POST, PUT, DELETE, etc.). Each method has specific semantics we'll cover in a later module.

• Request URI (Uniform Resource Identifier): The path to the resource, optionally including query parameters (/search?q=term&page=2).

• HTTP Version: The protocol version (HTTP/1.0, HTTP/1.1, HTTP/2, HTTP/3). This tells the server which features the client supports.

Part 2: Headers

Following the request line are headers—key-value pairs providing metadata about the request. Headers are separated from the request line by a single newline, and from each other by newlines. An empty line marks the end of headers.

Header-Name: Header-Value
Another-Header: Another-Value
<empty line>

Headers fall into several categories:

• General Headers: Apply to the entire message (Connection, Cache-Control, Date)
• Request Headers: Specific to requests (Accept, Accept-Language, Authorization, Host, User-Agent)
• Entity Headers: Describe the body (Content-Type, Content-Length, Content-Encoding)

Part 3: Body (Message Body)

The body contains data sent to the server—form submissions, JSON payloads, file uploads. Not all requests have bodies:

• GET, HEAD, DELETE, OPTIONS: Typically no body
• POST, PUT, PATCH: Usually include a body

The body appears after the empty line following headers. Its format depends on the Content-Type header (JSON, form data, multipart, etc.).

An HTTP response mirrors the request structure: a status line, headers, and (usually) a body containing the requested resource or error information.

1. Status Line
2. Headers
3. Body (usually present)

Part 1: The Status Line

The status line contains three elements:

HTTP-VERSION  STATUS-CODE  REASON-PHRASE
HTTP/1.1      200          OK

• HTTP Version: Matches or negotiates with the request version.

• Status Code: A three-digit numeric code indicating the result. Status codes are grouped by first digit:

  • 1xx: Informational (100 Continue, 101 Switching Protocols)
  • 2xx: Success (200 OK, 201 Created, 204 No Content)
  • 3xx: Redirection (301 Moved Permanently, 302 Found, 304 Not Modified)
  • 4xx: Client Error (400 Bad Request, 401 Unauthorized, 404 Not Found)
  • 5xx: Server Error (500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable)

• Reason Phrase: Human-readable description of the status (OK, Not Found, Internal Server Error). This is largely ignored by machines but aids debugging.

Part 2: Response Headers

Response headers provide metadata about the response and instructions for the client:

• General Headers: Date, Connection, Transfer-Encoding
• Response Headers: Server, WWW-Authenticate, Location (for redirects)
• Entity Headers: Content-Type, Content-Length, Content-Encoding, Last-Modified, ETag

Part 3: Response Body

The body contains the actual resource content—HTML, JSON, images, videos, or any other data. The Content-Type header specifies how to interpret the body. Content-Length indicates the body size in bytes (or Transfer-Encoding: chunked for streaming).

Understanding the complete lifecycle of a request-response cycle reveals where performance bottlenecks occur and how to optimize web applications.

Phase 1: Connection Establishment

Before any HTTP message is sent, a TCP connection must be established (or, for HTTP/3, a QUIC connection). This involves:

1. DNS Resolution: Translate the hostname (e.g., api.example.com) to an IP address
2. TCP Handshake: The 3-way SYN, SYN-ACK, ACK exchange
3. TLS Negotiation (for HTTPS): Certificate verification and key exchange

For HTTP/1.0, a new connection is established for each request. HTTP/1.1 introduced persistent connections (keep-alive), reusing connections for multiple requests. HTTP/2 and HTTP/3 optimize further with multiplexing.

Phase 2: Request Transmission

The client sends the HTTP request over the established connection:

GET /resource HTTP/1.1
Host: example.com
<headers>
<empty line>
<body if applicable>

Transmission time depends on request size and network conditions. For simple GET requests, this is fast. For large POST bodies (file uploads), this can take significant time.

Phase 3: Server Processing

The server receives the request, processes it, and prepares a response. Processing time varies enormously:

• Static file: microseconds (read from disk/cache)
• Database query: milliseconds to seconds
• External API calls: variable, potentially seconds
• Heavy computation: seconds or more

Server-side latency is often the dominant factor in total response time.

Phase 4: Response Transmission

The server sends the response:

HTTP/1.1 200 OK
<headers>
<empty line>
<body>

Large responses (images, videos, large JSON payloads) take longer to transmit. Compression (gzip, brotli) reduces transmission time at the cost of CPU.

Phase 5: Client Processing

The client receives the response and processes it:

• Parse headers (check status, content-type, caching directives)
• Parse body (decode JSON, render HTML, display image)
• Execute any triggered actions (JavaScript, redirects)
• Update cache (store for future requests if cacheable)

Latency Breakdown:

For a typical HTTPS request:

Optimization Opportunities:

Each phase offers optimization potential:

• DNS: DNS prefetching, caching
• TCP: Connection reuse (keep-alive), TCP Fast Open
• TLS: Session resumption, TLS 1.3, OCSP stapling
• Request: Request compression, minimal headers
• Server: Caching, database optimization, async processing
• Response: Compression, CDNs, streaming
• Client: Parallel requests, caching, lazy loading

HTTP enables a sophisticated negotiation process where clients express preferences and servers respond with the best-fit representation. This is content negotiation—the mechanism by which a single URL can serve different formats, languages, or encodings.

Why Content Negotiation Matters:

A single resource might have multiple representations:

• Same API endpoint returning JSON, XML, or CSV
• Same page in English, French, or Japanese
• Same content compressed or uncompressed
• Same image in WebP, JPEG, or PNG format

Content negotiation enables servers to automatically select the most appropriate representation based on client capabilities and preferences.

Types of Content Negotiation:

1. Server-Driven Negotiation (Proactive)

The client sends preferences in request headers, and the server selects the best representation:

• Accept: Preferred content types (Accept: application/json, text/html;q=0.9)
• Accept-Language: Preferred languages (Accept-Language: en-US,en;q=0.9,fr;q=0.8)
• Accept-Encoding: Supported compression (Accept-Encoding: gzip, deflate, br)
• Accept-Charset: Supported character sets (deprecated; UTF-8 assumed)

The server examines these headers and returns the best match, or 406 Not Acceptable if no suitable representation exists.

2. Agent-Driven Negotiation (Reactive)

The server provides a list of available representations, and the client (or user) selects one. Typically implemented via:

• 300 Multiple Choices response with links to variants
• Application logic presenting format options to users

Quality Values (q-factors):

The q parameter indicates relative preference from 0 to 1 (default is 1):

Accept: application/json, application/xml;q=0.9, text/plain;q=0.5

Interpretation:

• application/json: q=1.0 (most preferred)
• application/xml: q=0.9 (second choice)
• text/plain: q=0.5 (acceptable but not preferred)

Servers use these values to select the best match when multiple representations are available.

Vary Header:

When a server performs content negotiation, it should include a Vary header in the response indicating which request headers affected the selection:

Vary: Accept, Accept-Language, Accept-Encoding

This instructs caches that the response varies based on these headers—a request with different Accept values might get a different response.

Practical Content Negotiation:

The message body carries the actual data—the resource being transferred. Understanding body handling is essential for building and debugging web applications.

Content-Type: Identifying Body Format

The Content-Type header specifies the MIME type (Multipurpose Internet Mail Extensions) of the body:

Content-Type: type/subtype; parameter=value

Examples:

• text/html; charset=utf-8 — HTML document with UTF-8 encoding
• application/json — JSON data
• application/x-www-form-urlencoded — Form data (key=value pairs)
• multipart/form-data; boundary=----FormBoundary — File uploads with form data
• image/png — PNG image
• application/octet-stream — Binary data (unknown/unspecified type)

Content-Length: Specifying Body Size

The Content-Length header indicates the body size in bytes. This is critical for:

• The receiver knowing when the body ends
• Progress indicators during uploads/downloads
• Detecting truncated transmissions

Content-Length: 15234

Chunked Transfer Encoding:

When the content size is unknown in advance (streaming, dynamic generation), HTTP/1.1 allows chunked encoding:

Transfer-Encoding: chunked

The body is sent in chunks, each prefixed with its size in hexadecimal:

25
<37 bytes of data>
1a
<26 bytes of data>
0
<end of body>

A chunk of size 0 signals the end. This enables servers to begin sending responses before the total size is known.

Content Encoding: Compression

The Content-Encoding header indicates compression applied to the body:

Content-Encoding: gzip

Common encodings:

• gzip: Widely supported, good compression
• deflate: Similar to gzip, less common
• br (Brotli): Better compression than gzip, modern browsers

Clients advertise supported encodings via Accept-Encoding; servers compress if beneficial.

Multipart Bodies: File Uploads

File uploads typically use multipart/form-data, where the body contains multiple parts separated by boundaries:

Content-Type: multipart/form-data; boundary=----FormBoundaryXYZ

------FormBoundaryXYZ
Content-Disposition: form-data; name="title"

My Document
------FormBoundaryXYZ
Content-Disposition: form-data; name="file"; filename="report.pdf"
Content-Type: application/pdf

<binary PDF content>
------FormBoundaryXYZ--

Each part has its own headers and content. The boundary (a unique string) separates parts.

How HTTP connections are established, maintained, and terminated significantly impacts performance. The evolution of connection management reflects learned lessons about efficiency.

HTTP/1.0: One Connection Per Request

Originally, each request used a fresh TCP connection:

1. Client opens connection
2. Client sends request
3. Server sends response
4. Connection closes

For a page with 50 resources, this meant 50 TCP handshakes—extremely inefficient.

HTTP/1.1: Persistent Connections (Keep-Alive)

HTTP/1.1 made persistent connections the default. Multiple requests share a connection:

Connection: keep-alive  (default, can be omitted)
Keep-Alive: timeout=5, max=100

This eliminates redundant TCP handshakes. The connection stays open for a timeout period or until either side sends Connection: close.

HTTP/1.1 Pipelining (Problematic)

HTTP/1.1 also defined pipelining—sending multiple requests without waiting for responses:

Client: Request1, Request2, Request3
Server: Response1, Response2, Response3

However, responses must arrive in request order (head-of-line blocking). A slow Response1 delays Response2 and Response3, even if they're ready. Additionally, many proxies and servers didn't implement pipelining correctly, so browsers largely disabled it.

HTTP/2: Multiplexed Streams

HTTP/2 revolutionized connection management with multiplexing:

• Single TCP connection carries multiple 'streams'
• Each stream handles one request-response pair
• Streams are independent—a slow response doesn't block others
• Frames from different streams interleave freely

This eliminates HTTP-level head-of-line blocking entirely.

HTTP/3: QUIC Streams

HTTP/3 over QUIC goes further:

• Each stream has independent reliability
• Packet loss in one stream doesn't block others
• Connection migration survives IP address changes (important for mobile)
• Built-in encryption eliminates separate TLS handshake

Connection Limits and Parallelism:

Browsers impose per-domain connection limits:

• HTTP/1.1: Typically 6 concurrent connections per domain
• HTTP/2: Typically 1 connection per domain (multiplexed)

This led to 'domain sharding' in the HTTP/1.1 era—distributing resources across multiple domains (cdn1.example.com, cdn2.example.com) to increase parallelism. With HTTP/2, domain sharding becomes counterproductive since one multiplexed connection is more efficient than many.

Connection: close

Either party can signal intention to close the connection:

Connection: close

This indicates: 'I won't send more requests' (client) or 'I won't accept more requests on this connection' (server). The connection closes after the current response completes.

Real-world applications compose multiple request-response exchanges into complex workflows. Understanding common patterns helps design efficient, correct systems.

Pattern 1: Resource Fetch

The simplest pattern—retrieve a resource:

GET /products/123 → 200 OK + product data

Optimizations include conditional requests (If-None-Match, If-Modified-Since) to avoid transferring unchanged resources.

Pattern 2: Create Resource

Create a new resource with POST:

POST /products + product data → 201 Created + Location: /products/456

The response includes the new resource's URL in the Location header.

Pattern 3: Update Resource

Modify an existing resource:

PUT /products/123 + complete product data → 200 OK + updated data
PATCH /products/123 + partial update → 200 OK + updated data

PUT replaces the entire resource; PATCH modifies only specified fields.

Pattern 4: Delete Resource

Remove a resource:

DELETE /products/123 → 204 No Content

204 indicates success without returning data.

Pattern 5: Redirect Chain

Request redirects to new location:

GET /old-page → 301 Moved Permanently + Location: /new-page
GET /new-page → 200 OK + content

Clients automatically follow redirects (browsers limit to ~20 to prevent loops).

Pattern 6: Authentication Challenge

Resources requiring authentication follow a challenge pattern:

1. GET /protected → 401 Unauthorized + WWW-Authenticate: Bearer
2. Client obtains token (via login flow)
3. GET /protected + Authorization: Bearer <token> → 200 OK

Pattern 7: Conditional Requests (Caching)

1. GET /resource → 200 OK + ETag: "abc123"
   (Client caches response)
2. GET /resource + If-None-Match: "abc123"
   → 304 Not Modified (no body, use cached version)
   → 200 OK + new body (if changed)

This pattern reduces bandwidth dramatically for unchanged resources.

Pattern 8: CORS Preflight

Cross-origin requests with non-simple characteristics trigger preflight:

1. OPTIONS /api/data (preflight check)
   → 200 + Access-Control-Allow-Origin, Access-Control-Allow-Methods
2. POST /api/data (actual request, if preflight allowed)
   → 200 OK

Composing Patterns:

Real applications combine these patterns. A single page load might involve:

• 1 HTML fetch (with redirect if needed)
• Dozens of CSS, JS, image fetches (parallel, with caching)
• API calls for dynamic data (with authentication)
• Preflight checks for cross-origin APIs

Understanding these patterns enables effective debugging and optimization.

We've thoroughly explored HTTP's request-response model—the fundamental pattern underlying all web communication. Let's consolidate the key insights:

What's Next:

The request-response model seems simple—and that simplicity is a feature. But one aspect demands deeper examination: HTTP's stateless protocol nature. The next page explores what 'stateless' truly means, why it matters for scalability, and how applications achieve statefulness despite stateless protocols.

You'll understand the design rationale behind statelessness, how cookies and tokens layer state onto HTTP, and the tradeoffs between stateless and stateful architectures.