Computer NetworksHTTP & Web

HTTP Methods and Status

LevelIntermediate

Duration90 mins

TopicHTTP & Web

2 / 5

HTTP Headers: The Metadata Layer

The Hidden Conversation Behind Every Request

Every HTTP request and response carries a hidden payload that most users never see—HTTP headers. While the URL defines what you're requesting and the body contains the content, headers form the metadata layer that controls how the communication happens.

Headers answer questions like:

What content formats can the client accept?
Is this response cacheable, and for how long?
What credentials does the client present?
What character encoding does the content use?
Should this content be downloaded or displayed?
Is this request coming from a mobile or desktop browser?

Without headers, HTTP would be limited to simple document retrieval. With headers, HTTP becomes a rich, extensible protocol capable of powering the modern web—from streaming video to real-time APIs to content delivery networks operating at planetary scale.

Learning Objectives

By the end of this page, you will: • Understand the anatomy and syntax of HTTP headers • Master essential request and response headers • Implement content negotiation using Accept headers • Apply caching headers for optimal performance • Configure security headers to protect users • Debug header-related issues effectively

Header Anatomy and Syntax

HTTP headers follow a simple, consistent syntax defined in RFC 9110. Understanding this structure is fundamental to working with HTTP at any level.

Basic Syntax

Header-Name: header-value

Rules:

Name: Case-insensitive (though conventionally Title-Case or lowercase)
Colon: Immediately follows the name (no space before)
Value: May contain spaces; leading/trailing whitespace is ignored
Line ending: CRLF (\r\n)

Header Structure in Context

GET /api/users/123 HTTP/1.1\r\n
Host: api.example.com\r\n
Accept: application/json\r\n
Authorization: Bearer token123\r\n
\r\n

The blank line (CRLF only) separates headers from the body.

Header Classification
Category	Description	Examples
General Headers	Apply to both requests and responses	Date, Connection, Cache-Control
Request Headers	Provide context about the request or client	Accept, User-Agent, Authorization, Host
Response Headers	Provide context about the response or server	Server, Set-Cookie, WWW-Authenticate
Representation Headers	Describe the body content	Content-Type, Content-Length, Content-Encoding
Payload Headers	About the payload data itself	Content-Range, Trailer

Multiple Values in Headers

Some headers accept multiple values. Two approaches exist:

1. Comma-Separated (Single Header)

Accept-Encoding: gzip, deflate, br
Cache-Control: no-cache, no-store, must-revalidate

2. Repeated Headers (Multiple Lines)

Set-Cookie: sessionId=abc123; HttpOnly
Set-Cookie: theme=dark; Path=/

Important: Not all headers support repetition. Set-Cookie specifically requires separate headers because its values contain commas for expiration dates.

Header Size Limits

While HTTP doesn't specify maximum header sizes, practical limits exist:

Component	Typical Limits
Single header line	8,192 bytes (Apache default)
Total header size	32KB - 64KB (varies by server)
Number of headers	100 headers (common limit)
Cookie header	4KB per cookie; 50 cookies per domain

Exceeding limits typically results in 431 Request Header Fields Too Large.

Header Injection Vulnerabilities

Never include untrusted user input directly in headers. CRLF characters (\r\n) in header values can inject additional headers:

User input: "malicious\r\nSet-Cookie: admin=true"

This creates an injected Set-Cookie header, potentially granting admin access. Always sanitize header values!

Essential Request Headers

Request headers communicate client capabilities, preferences, and context to the server. Understanding these headers is essential for building and debugging web applications.

Complete Request Headers Example
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
GET /api/products?category=electronics HTTP/1.1
Host: api.example.com
Accept: application/json, text/html;q=0.9, */*;q=0.1
Accept-Language: en-US,en;q=0.9,es;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Charset: UTF-8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
Cookie: sessionId=abc123; preferences=dark-mode
Referer: https://www.example.com/products
Origin: https://www.example.com
Cache-Control: no-cache
If-None-Match: "etag-abc123"
If-Modified-Since: Wed, 15 Jan 2025 12:00:00 GMT
Connection: keep-alive
X-Request-ID: 7d3f8a2b-uuid

Host Header (Required in HTTP/1.1)

The Host header is the only required header in HTTP/1.1. It specifies the domain name of the server:

Host: api.example.com
Host: api.example.com:8080  # With non-standard port

Why required? A single IP address often hosts multiple domains (virtual hosting). Without Host, the server wouldn't know which site you're accessing.

Accept Headers (Content Negotiation)

These headers enable content negotiation—the client describing what it can accept:

Header	Purpose	Example
`Accept`	Acceptable media types	`application/json, text/html`
`Accept-Language`	Preferred languages	`en-US,en;q=0.9,fr;q=0.8`
`Accept-Encoding`	Supported compression	`gzip, deflate, br`
`Accept-Charset`	Character encodings	`UTF-8, ISO-8859-1`

Quality Values (q=): Lower values indicate lower preference:

Accept: application/json;q=1.0, text/html;q=0.9, */*;q=0.1

Prefer JSON (q=1.0, maximum)
HTML acceptable (q=0.9)
Anything else as last resort (q=0.1)

User-Agent Header

Identifies the client software making the request:

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36

Uses:

Serve optimized content for specific browsers
Analytics and traffic analysis
Block known bots or scrapers

Caution: User-Agent can be easily spoofed. Don't rely on it for security decisions.

Authorization Header

Carries authentication credentials:

# Bearer Token (OAuth 2.0, JWT)
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

# Basic Authentication (Base64 encoded username:password)
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

# API Key (custom scheme)
Authorization: ApiKey sk-live-abc123

Security Note: Always use HTTPS when sending Authorization headers. Without TLS, credentials are transmitted in plain text.

Cookie Header

Sends stored cookies back to the server:

Cookie: sessionId=abc123; userId=42; theme=dark

Multiple cookies are semicolon-separated. The Cookie header provides state persistence for the stateless HTTP protocol.

Conditional Request Headers

Conditional headers enable efficient caching:

• If-None-Match: "etag" — "Only send the body if ETag changed" → 304 Not Modified • If-Modified-Since: date — "Only send if modified after this date" → 304 Not Modified • If-Match: "etag" — "Only process if ETag still matches" (used with PUT/DELETE) • If-Unmodified-Since: date — "Only process if not changed since" (concurrency control)

These headers can reduce bandwidth by 90%+ for cacheable resources.

Origin and Referer Headers

These headers indicate where the request came from:

Referer (yes, misspelled historically):

Referer: https://www.example.com/products/123

Full URL of the page that linked to this request. Used for analytics, logging, CSRF protection.

Origin:

Origin: https://www.example.com

Scheme + domain only (no path). Sent with cross-origin requests; fundamental for CORS.

Key Difference:

Referer: Full URL, stripped in many privacy-focused browsers
Origin: Domain only, always sent for cross-origin/same-origin POST

X-Request-ID / Correlation ID

X-Request-ID: 7d3f8a2b-4c5e-6f7g-8h9i-0jklmnopqrst

Client-generated unique ID for request tracking across distributed systems. Servers typically echo it back and include it in logs, enabling end-to-end tracing.

Essential Response Headers

Response headers communicate server decisions, content metadata, and instructions back to the client.

Complete Response Headers Example
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
HTTP/1.1 200 OK
Date: Sat, 18 Jan 2025 12:00:00 GMT
Server: nginx/1.24.0
Content-Type: application/json; charset=utf-8
Content-Length: 1234
Content-Encoding: gzip
ETag: "abc123def456"
Last-Modified: Fri, 17 Jan 2025 18:30:00 GMT
Cache-Control: public, max-age=3600, stale-while-revalidate=86400
Vary: Accept-Encoding, Accept-Language
Set-Cookie: sessionId=xyz789; HttpOnly; Secure; SameSite=Strict; Path=/
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline'
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Request-ID: 7d3f8a2b-uuid
Access-Control-Allow-Origin: https://app.example.com
Connection: keep-alive

Content-Type Header

The most important response header—tells the client how to interpret the body:

Content-Type: application/json; charset=utf-8
Content-Type: text/html; charset=utf-8
Content-Type: image/png
Content-Type: application/pdf
Content-Type: text/plain

Structure: type/subtype; parameter=value

Common MIME Types:

MIME Type	Use Case
`text/html`	HTML documents
`text/css`	CSS stylesheets
`text/javascript`	JavaScript code
`application/json`	JSON data (API responses)
`application/xml`	XML documents
`image/jpeg`, `image/png`, `image/webp`	Images
`application/pdf`	PDF documents
`application/octet-stream`	Binary data (generic)

Content-Length Header

Content-Length: 15234

Exact size of the response body in bytes. Enables:

Progress indicators for downloads
Connection reuse (client knows when body ends)
Validation that complete response was received

Content-Encoding Header

Content-Encoding: gzip
Content-Encoding: br
Content-Encoding: deflate

Indicates compression applied to the body. The client must decompress before use.

Brotli (br) typically achieves 20-25% better compression than gzip for text content.

Cache Control Headers

Caching headers control how responses are stored and reused:

Cache-Control (Primary):

Cache-Control: public, max-age=3600, stale-while-revalidate=86400

Directive	Meaning
`public`	Any cache (CDN, browser) may store
`private`	Only browser cache, not CDNs
`max-age=N`	Cache valid for N seconds
`s-maxage=N`	Override max-age for shared caches
`no-cache`	Must revalidate before using
`no-store`	Never cache (sensitive data)
`must-revalidate`	After expiry, must revalidate
`stale-while-revalidate=N`	Serve stale while fetching fresh (background update)
`stale-if-error=N`	Serve stale if origin errors
`immutable`	Content will never change

ETag (Validation):

ETag: "abc123def456"

Unique identifier for this specific version. Client sends back in If-None-Match to check freshness.

Last-Modified (Validation):

Last-Modified: Fri, 17 Jan 2025 18:30:00 GMT

Date of last change. Client sends back in If-Modified-Since.

Optimal Caching Strategy

For static assets (JS, CSS, images with hashed filenames):

Cache-Control: public, max-age=31536000, immutable

(Cache forever—filename changes when content changes)

For API responses:

Cache-Control: private, max-age=60, stale-while-revalidate=300
ETag: "version-hash"

(Short max-age, serve stale while revalidating, personal data private)

For sensitive data:

Cache-Control: no-store

(Never cache—banking, health data, etc.)

Set-Cookie Header

Sends cookies to the client for storage:

Set-Cookie: sessionId=abc123; HttpOnly; Secure; SameSite=Strict; Path=/; Max-Age=86400

Attribute	Meaning
`HttpOnly`	Inaccessible to JavaScript (XSS protection)
`Secure`	Only sent over HTTPS
`SameSite=Strict`	Only sent for same-site requests (CSRF protection)
`SameSite=Lax`	Sent for same-site + top-level navigations
`SameSite=None`	Sent for cross-site (requires Secure)
`Path=/`	Cookie scope (path prefix)
`Domain=.example.com`	Cookie scope (subdomains)
`Max-Age=86400`	Expires in N seconds
`Expires=date`	Absolute expiration date

Vary Header

Vary: Accept-Encoding, Accept-Language

Informs caches that the response varies based on these request headers. Without Vary, a cache might serve a gzip response to a client that doesn't support gzip.

Location Header

Location: /api/users/12345
Location: https://new-domain.com/resource

Used with 3xx redirects and 201 Created to indicate where to go next.

Security Headers: Protecting Users

Security headers provide defense-in-depth against common web attacks. Every production web application should implement these headers.

Essential Security Headers
Header	Protection Against	Recommended Value
`Strict-Transport-Security`	SSL stripping, MITM	`max-age=31536000; includeSubDomains; preload`
`Content-Security-Policy`	XSS, injection attacks	`default-src 'self'; script-src 'self'`
`X-Content-Type-Options`	MIME sniffing attacks	`nosniff`
`X-Frame-Options`	Clickjacking	`DENY` or `SAMEORIGIN`
`X-XSS-Protection`	Reflected XSS (legacy)	`0` (modern CSP is better)
`Referrer-Policy`	Privacy leakage via referer	`strict-origin-when-cross-origin`
`Permissions-Policy`	Feature abuse (camera, mic)	`geolocation=(), camera=()`

Strict-Transport-Security (HSTS)

Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

Instructs browsers to only connect via HTTPS for the specified duration.

Attack prevented: SSL stripping (attacker downgrades HTTPS to HTTP to intercept traffic).

How it works:

User visits https://example.com
Server sends HSTS header
Browser remembers: "Always use HTTPS for example.com"
Future requests to http://example.com auto-redirect to HTTPS before any network request

Preload: Submit to the HSTS Preload List to have your domain hardcoded into browsers.

Content-Security-Policy (CSP)

Content-Security-Policy: default-src 'self'; script-src 'self' https://trusted-cdn.com; style-src 'self' 'unsafe-inline'; img-src *; connect-src 'self' https://api.example.com

CSP is the most powerful security header—it controls what resources the page can load.

Directive	Controls
`default-src`	Fallback for unspecified types
`script-src`	JavaScript sources
`style-src`	CSS sources
`img-src`	Image sources
`connect-src`	AJAX, WebSocket, fetch
`font-src`	Web fonts
`frame-src`	Embedded frames
`form-action`	Form submission targets

Values:

'self' — Same origin only
'none' — Block completely
https://domain.com — Specific origin
'unsafe-inline' — Allow inline scripts/styles (weaker security)
'nonce-abc123' — Allow specific inline with matching nonce

Comprehensive Security Headers Configuration
Nginx
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# Nginx configuration for security headers
 
server {
    listen 443 ssl http2;
    server_name example.com;
 
    # HTTPS only for 1 year, including subdomains
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
    
    # Strict CSP - adjust based on your needs
    add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'nonce-$request_id'; style-src 'self' 'unsafe-inline'; img-src 'self' data: https:; font-src 'self' https://fonts.gstatic.com; connect-src 'self' https://api.example.com; frame-ancestors 'none'; base-uri 'self'; form-action 'self';" always;
    
    # Prevent MIME type sniffing
    add_header X-Content-Type-Options "nosniff" always;
    
    # Prevent embedding in frames (clickjacking)
    add_header X-Frame-Options "DENY" always;
    
    # Control referrer information
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;
    
    # Disable dangerous features
    add_header Permissions-Policy "geolocation=(), camera=(), microphone=(), payment=(), usb=()" always;
    
    # Cross-Origin policies
    add_header Cross-Origin-Opener-Policy "same-origin" always;
    add_header Cross-Origin-Embedder-Policy "require-corp" always;
    add_header Cross-Origin-Resource-Policy "same-origin" always;
}

CSP Implementation Challenges

Implementing CSP on existing sites is notoriously difficult:

• Inline scripts/styles require refactoring or nonce/hash • Third-party widgets (ads, analytics) need explicit allowlisting • Legacy code may break unexpectedly

Recommended approach:

Start with Content-Security-Policy-Report-Only (logs violations, doesn't block)
Set up a reporting endpoint to collect violations
Gradually tighten policy based on reports
Switch to enforcing Content-Security-Policy

CORS Headers: Cross-Origin Communication

Cross-Origin Resource Sharing (CORS) headers control which external domains can access your resources. This is fundamental for modern web architectures where APIs are on different domains than frontends.

The Same-Origin Policy

Browsers enforce the Same-Origin Policy: scripts on one origin cannot read responses from another origin. Two URLs are same-origin if they share:

Protocol (http vs https)
Domain (api.example.com vs example.com)
Port (80 vs 8080)

CORS headers explicitly relax this policy for trusted origins.

Key CORS Headers

CORS Response Headers
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Simple request response
HTTP/1.1 200 OK
Access-Control-Allow-Origin: https://app.example.com
Vary: Origin
 
# Response allowing credentials (cookies)
HTTP/1.1 200 OK
Access-Control-Allow-Origin: https://app.example.com
Access-Control-Allow-Credentials: true
Vary: Origin
 
# Preflight response (OPTIONS request)
HTTP/1.1 204 No Content
Access-Control-Allow-Origin: https://app.example.com
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization, X-Request-ID
Access-Control-Expose-Headers: X-Total-Count, X-Request-ID
Access-Control-Max-Age: 86400
Vary: Origin

CORS Response Headers Explained
Header	Purpose	Example Value
`Access-Control-Allow-Origin`	Which origins may access	`https://app.example.com` or `*`
`Access-Control-Allow-Methods`	Allowed HTTP methods	`GET, POST, PUT, DELETE`
`Access-Control-Allow-Headers`	Allowed request headers	`Content-Type, Authorization`
`Access-Control-Expose-Headers`	Headers JS can read	`X-Total-Count, X-Request-ID`
`Access-Control-Allow-Credentials`	Allow cookies/auth	`true` (requires specific origin)
`Access-Control-Max-Age`	Preflight cache duration	`86400` (seconds)

Simple vs. Preflighted Requests

Simple requests (no preflight needed):

Methods: GET, HEAD, POST
Headers: Only Accept, Accept-Language, Content-Language, Content-Type (with limited values)
Content-Type: only text/plain, multipart/form-data, application/x-www-form-urlencoded

Preflighted requests (browser sends OPTIONS first):

Custom headers (Authorization, X-Custom-Header)
Non-simple methods (PUT, DELETE, PATCH)
Content-Type: application/json

Preflight Flow

1. Browser: "I want to PUT to api.example.com with JSON body and Authorization header"
2. Browser sends OPTIONS request first:
   OPTIONS /resource
   Origin: https://app.example.com
   Access-Control-Request-Method: PUT
   Access-Control-Request-Headers: Authorization, Content-Type

3. Server responds with permissions:
   Access-Control-Allow-Origin: https://app.example.com
   Access-Control-Allow-Methods: PUT
   Access-Control-Allow-Headers: Authorization, Content-Type
   Access-Control-Max-Age: 86400

4. Browser caches preflight result (86400 seconds)
5. Browser sends actual PUT request
6. Server processes request and includes CORS headers in response

Common CORS Mistakes

1. Using Access-Control-Allow-Origin: * with credentials This is FORBIDDEN. When Allow-Credentials: true, you MUST specify exact origin.

2. Forgetting Vary: Origin Without it, CDNs may cache the CORS headers for one origin and serve them to others.

3. Not handling OPTIONS method Many frameworks don't auto-respond to OPTIONS preflight requests.

4. Exposing sensitive headers accidentally Access-Control-Expose-Headers makes headers readable by JS—be selective.

Custom Headers and Conventions

Beyond standard headers, applications commonly define custom headers for specific needs.

X-Prefix Convention (Deprecated)

Historically, custom headers used the X- prefix:

X-Request-ID: abc123
X-Rate-Limit-Remaining: 4999

RFC 6648 deprecated this convention in 2012 because:

Transitioning from experimental to standard is awkward
Too many production systems use X-prefixed headers
The prefix provides no real benefit

Current recommendation: Just name your header descriptively. If it becomes a standard, it can keep its name.

Common Custom Headers

Header	Purpose	Example
`X-Request-ID` / `Request-Id`	Request tracing	`7a3f8b2c-uuid`
`X-RateLimit-Limit`	Rate limit ceiling	`1000`
`X-RateLimit-Remaining`	Remaining requests	`999`
`X-RateLimit-Reset`	When limit resets	`1705582800` (Unix timestamp)
`X-Total-Count`	Total items (pagination)	`1234`
`X-Forwarded-For`	Original client IP (proxies)	`203.0.113.1, 198.51.100.2`
`X-Forwarded-Proto`	Original protocol	`https`
`X-Correlation-ID`	Distributed tracing	`trace-abc123`
`Idempotency-Key`	Request deduplication	`unique-uuid`

Rate Limiting Headers Example
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
HTTP/1.1 200 OK
Content-Type: application/json
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 995
X-RateLimit-Reset: 1705586400
X-RateLimit-Window: 3600
Retry-After: 3600
 
{
  "data": [...],
  "meta": {
    "rateLimit": {
      "limit": 1000,
      "remaining": 995,
      "reset": "2025-01-18T14:00:00Z"
    }
  }
}
 
# When rate limited:
HTTP/1.1 429 Too Many Requests
Retry-After: 1800
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1705588200
 
{
  "error": "Rate limit exceeded",
  "retryAfter": 1800
}

Forwarding Headers (Proxy Context)

When requests pass through load balancers or reverse proxies, these headers preserve original request information:

X-Forwarded-For: 203.0.113.1, 198.51.100.22, 10.0.0.5
X-Forwarded-Proto: https
X-Forwarded-Host: original-domain.com
X-Forwarded-Port: 443

X-Forwarded-For chain: Each proxy appends its view of the client IP. The leftmost is (supposedly) the original client, but can be spoofed. The rightmost is the immediate upstream proxy, which is trustworthy.

Security consideration: Only trust XFF headers from known proxy IPs. Clients can forge these headers; your proxy should overwrite/append, not blindly trust.

The Forwarded Header (Standard)

RFC 7239 standardized proxy forwarding into a single header:

Forwarded: for=203.0.113.1; proto=https; host=example.com; by=10.0.0.5

This is cleaner but not as widely supported as the X-Forwarded-* family.

Summary: Mastering HTTP Headers

HTTP headers form the invisible infrastructure that enables the modern web. Let's consolidate the essential knowledge:

Key Takeaways

•Headers are metadata — They control how requests and responses are processed, not what content is transferred
•Content negotiation enables flexibility — Accept headers let clients and servers agree on formats, languages, and encodings
•Caching headers are performance multipliers — Proper Cache-Control, ETag, and Vary headers can reduce server load by orders of magnitude
•Security headers provide defense-in-depth — HSTS, CSP, and X-Content-Type-Options should be on every production site
•CORS enables controlled cross-origin access — Essential for modern API-driven architectures with separate frontend/backend domains
•Custom headers extend HTTP — Rate limiting, request tracing, and application-specific metadata all use custom headers

Page Complete

You now have a comprehensive understanding of HTTP headers—from basic syntax to security implications. In the next page, we'll explore HTTP status codes: the server's vocabulary for communicating success, errors, and everything in between.

2 / 5

Loading learning content...

Computer NetworksHTTP & Web

HTTP Methods and Status

LevelIntermediate

Duration90 mins

TopicHTTP & Web

2 / 5

HTTP Headers: The Metadata Layer

The Hidden Conversation Behind Every Request

Headers answer questions like:

What content formats can the client accept?
Is this response cacheable, and for how long?
What credentials does the client present?
What character encoding does the content use?
Should this content be downloaded or displayed?
Is this request coming from a mobile or desktop browser?

Learning Objectives

Header Anatomy and Syntax

HTTP headers follow a simple, consistent syntax defined in RFC 9110. Understanding this structure is fundamental to working with HTTP at any level.

Basic Syntax

Header-Name: header-value

Rules:

Name: Case-insensitive (though conventionally Title-Case or lowercase)
Colon: Immediately follows the name (no space before)
Value: May contain spaces; leading/trailing whitespace is ignored
Line ending: CRLF (\r\n)

Header Structure in Context

GET /api/users/123 HTTP/1.1\r\n
Host: api.example.com\r\n
Accept: application/json\r\n
Authorization: Bearer token123\r\n
\r\n

The blank line (CRLF only) separates headers from the body.

Header Classification
Category	Description	Examples
General Headers	Apply to both requests and responses	Date, Connection, Cache-Control
Request Headers	Provide context about the request or client	Accept, User-Agent, Authorization, Host
Response Headers	Provide context about the response or server	Server, Set-Cookie, WWW-Authenticate
Representation Headers	Describe the body content	Content-Type, Content-Length, Content-Encoding
Payload Headers	About the payload data itself	Content-Range, Trailer

Multiple Values in Headers

Some headers accept multiple values. Two approaches exist:

1. Comma-Separated (Single Header)

Accept-Encoding: gzip, deflate, br
Cache-Control: no-cache, no-store, must-revalidate

2. Repeated Headers (Multiple Lines)

Set-Cookie: sessionId=abc123; HttpOnly
Set-Cookie: theme=dark; Path=/

Important: Not all headers support repetition. Set-Cookie specifically requires separate headers because its values contain commas for expiration dates.

Header Size Limits

While HTTP doesn't specify maximum header sizes, practical limits exist:

Component	Typical Limits
Single header line	8,192 bytes (Apache default)
Total header size	32KB - 64KB (varies by server)
Number of headers	100 headers (common limit)
Cookie header	4KB per cookie; 50 cookies per domain

Exceeding limits typically results in 431 Request Header Fields Too Large.

Header Injection Vulnerabilities

Never include untrusted user input directly in headers. CRLF characters (\r\n) in header values can inject additional headers:

User input: "malicious\r\nSet-Cookie: admin=true"

This creates an injected Set-Cookie header, potentially granting admin access. Always sanitize header values!

Essential Request Headers

Request headers communicate client capabilities, preferences, and context to the server. Understanding these headers is essential for building and debugging web applications.

Complete Request Headers Example
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
GET /api/products?category=electronics HTTP/1.1
Host: api.example.com
Accept: application/json, text/html;q=0.9, */*;q=0.1
Accept-Language: en-US,en;q=0.9,es;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Charset: UTF-8
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...
Cookie: sessionId=abc123; preferences=dark-mode
Referer: https://www.example.com/products
Origin: https://www.example.com
Cache-Control: no-cache
If-None-Match: "etag-abc123"
If-Modified-Since: Wed, 15 Jan 2025 12:00:00 GMT
Connection: keep-alive
X-Request-ID: 7d3f8a2b-uuid

Host Header (Required in HTTP/1.1)

The Host header is the only required header in HTTP/1.1. It specifies the domain name of the server:

Host: api.example.com
Host: api.example.com:8080  # With non-standard port

Why required? A single IP address often hosts multiple domains (virtual hosting). Without Host, the server wouldn't know which site you're accessing.

Accept Headers (Content Negotiation)

These headers enable content negotiation—the client describing what it can accept:

Header	Purpose	Example
`Accept`	Acceptable media types	`application/json, text/html`
`Accept-Language`	Preferred languages	`en-US,en;q=0.9,fr;q=0.8`
`Accept-Encoding`	Supported compression	`gzip, deflate, br`
`Accept-Charset`	Character encodings	`UTF-8, ISO-8859-1`

Quality Values (q=): Lower values indicate lower preference:

Accept: application/json;q=1.0, text/html;q=0.9, */*;q=0.1

Prefer JSON (q=1.0, maximum)
HTML acceptable (q=0.9)
Anything else as last resort (q=0.1)

User-Agent Header

Identifies the client software making the request:

User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36

Uses:

Serve optimized content for specific browsers
Analytics and traffic analysis
Block known bots or scrapers

Caution: User-Agent can be easily spoofed. Don't rely on it for security decisions.

Authorization Header

Carries authentication credentials:

# Bearer Token (OAuth 2.0, JWT)
Authorization: Bearer eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9...

# Basic Authentication (Base64 encoded username:password)
Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=

# API Key (custom scheme)
Authorization: ApiKey sk-live-abc123

Security Note: Always use HTTPS when sending Authorization headers. Without TLS, credentials are transmitted in plain text.

Cookie Header

Sends stored cookies back to the server:

Cookie: sessionId=abc123; userId=42; theme=dark

Multiple cookies are semicolon-separated. The Cookie header provides state persistence for the stateless HTTP protocol.

Conditional Request Headers

Conditional headers enable efficient caching:

These headers can reduce bandwidth by 90%+ for cacheable resources.

Origin and Referer Headers

These headers indicate where the request came from:

Referer (yes, misspelled historically):

Referer: https://www.example.com/products/123

Full URL of the page that linked to this request. Used for analytics, logging, CSRF protection.

Origin:

Origin: https://www.example.com

Scheme + domain only (no path). Sent with cross-origin requests; fundamental for CORS.

Key Difference:

Referer: Full URL, stripped in many privacy-focused browsers
Origin: Domain only, always sent for cross-origin/same-origin POST

X-Request-ID / Correlation ID

X-Request-ID: 7d3f8a2b-4c5e-6f7g-8h9i-0jklmnopqrst

Client-generated unique ID for request tracking across distributed systems. Servers typically echo it back and include it in logs, enabling end-to-end tracing.

Essential Response Headers

Response headers communicate server decisions, content metadata, and instructions back to the client.

Complete Response Headers Example
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
HTTP/1.1 200 OK
Date: Sat, 18 Jan 2025 12:00:00 GMT
Server: nginx/1.24.0
Content-Type: application/json; charset=utf-8
Content-Length: 1234
Content-Encoding: gzip
ETag: "abc123def456"
Last-Modified: Fri, 17 Jan 2025 18:30:00 GMT
Cache-Control: public, max-age=3600, stale-while-revalidate=86400
Vary: Accept-Encoding, Accept-Language
Set-Cookie: sessionId=xyz789; HttpOnly; Secure; SameSite=Strict; Path=/
Strict-Transport-Security: max-age=31536000; includeSubDomains; preload
Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline'
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-Request-ID: 7d3f8a2b-uuid
Access-Control-Allow-Origin: https://app.example.com
Connection: keep-alive

Content-Type Header

The most important response header—tells the client how to interpret the body:

Content-Type: application/json; charset=utf-8
Content-Type: text/html; charset=utf-8
Content-Type: image/png
Content-Type: application/pdf
Content-Type: text/plain

Structure: type/subtype; parameter=value

Common MIME Types:

MIME Type	Use Case
`text/html`	HTML documents
`text/css`	CSS stylesheets
`text/javascript`	JavaScript code
`application/json`	JSON data (API responses)
`application/xml`	XML documents
`image/jpeg`, `image/png`, `image/webp`	Images
`application/pdf`	PDF documents
`application/octet-stream`	Binary data (generic)

Content-Length Header

Content-Length: 15234

Exact size of the response body in bytes. Enables:

Progress indicators for downloads
Connection reuse (client knows when body ends)
Validation that complete response was received

Content-Encoding Header

Content-Encoding: gzip
Content-Encoding: br
Content-Encoding: deflate

Indicates compression applied to the body. The client must decompress before use.

Brotli (br) typically achieves 20-25% better compression than gzip for text content.

Cache Control Headers

Caching headers control how responses are stored and reused:

Cache-Control (Primary):

Cache-Control: public, max-age=3600, stale-while-revalidate=86400

Directive	Meaning
`public`	Any cache (CDN, browser) may store
`private`	Only browser cache, not CDNs
`max-age=N`	Cache valid for N seconds
`s-maxage=N`	Override max-age for shared caches
`no-cache`	Must revalidate before using
`no-store`	Never cache (sensitive data)
`must-revalidate`	After expiry, must revalidate
`stale-while-revalidate=N`	Serve stale while fetching fresh (background update)
`stale-if-error=N`	Serve stale if origin errors
`immutable`	Content will never change

ETag (Validation):

ETag: "abc123def456"

Unique identifier for this specific version. Client sends back in If-None-Match to check freshness.

Last-Modified (Validation):

Last-Modified: Fri, 17 Jan 2025 18:30:00 GMT

Date of last change. Client sends back in If-Modified-Since.

Optimal Caching Strategy

For static assets (JS, CSS, images with hashed filenames):

Cache-Control: public, max-age=31536000, immutable

(Cache forever—filename changes when content changes)

For API responses:

Cache-Control: private, max-age=60, stale-while-revalidate=300
ETag: "version-hash"

(Short max-age, serve stale while revalidating, personal data private)

For sensitive data:

Cache-Control: no-store

(Never cache—banking, health data, etc.)

Set-Cookie Header

Sends cookies to the client for storage:

Set-Cookie: sessionId=abc123; HttpOnly; Secure; SameSite=Strict; Path=/; Max-Age=86400

Attribute	Meaning
`HttpOnly`	Inaccessible to JavaScript (XSS protection)
`Secure`	Only sent over HTTPS
`SameSite=Strict`	Only sent for same-site requests (CSRF protection)
`SameSite=Lax`	Sent for same-site + top-level navigations
`SameSite=None`	Sent for cross-site (requires Secure)
`Path=/`	Cookie scope (path prefix)
`Domain=.example.com`	Cookie scope (subdomains)
`Max-Age=86400`	Expires in N seconds
`Expires=date`	Absolute expiration date

Vary Header

Vary: Accept-Encoding, Accept-Language

Informs caches that the response varies based on these request headers. Without Vary, a cache might serve a gzip response to a client that doesn't support gzip.

Location Header

Location: /api/users/12345
Location: https://new-domain.com/resource

Used with 3xx redirects and 201 Created to indicate where to go next.

Security Headers: Protecting Users

Security headers provide defense-in-depth against common web attacks. Every production web application should implement these headers.

Essential Security Headers
Header	Protection Against	Recommended Value
`Strict-Transport-Security`	SSL stripping, MITM	`max-age=31536000; includeSubDomains; preload`
`Content-Security-Policy`	XSS, injection attacks	`default-src 'self'; script-src 'self'`
`X-Content-Type-Options`	MIME sniffing attacks	`nosniff`
`X-Frame-Options`	Clickjacking	`DENY` or `SAMEORIGIN`
`X-XSS-Protection`	Reflected XSS (legacy)	`0` (modern CSP is better)
`Referrer-Policy`	Privacy leakage via referer	`strict-origin-when-cross-origin`
`Permissions-Policy`	Feature abuse (camera, mic)	`geolocation=(), camera=()`

Strict-Transport-Security (HSTS)

Strict-Transport-Security: max-age=31536000; includeSubDomains; preload

Instructs browsers to only connect via HTTPS for the specified duration.

Attack prevented: SSL stripping (attacker downgrades HTTPS to HTTP to intercept traffic).

How it works:

User visits https://example.com
Server sends HSTS header
Browser remembers: "Always use HTTPS for example.com"
Future requests to http://example.com auto-redirect to HTTPS before any network request

Preload: Submit to the HSTS Preload List to have your domain hardcoded into browsers.

Content-Security-Policy (CSP)

Content-Security-Policy: default-src 'self'; script-src 'self' https://trusted-cdn.com; style-src 'self' 'unsafe-inline'; img-src *; connect-src 'self' https://api.example.com

CSP is the most powerful security header—it controls what resources the page can load.

Directive	Controls
`default-src`	Fallback for unspecified types
`script-src`	JavaScript sources
`style-src`	CSS sources
`img-src`	Image sources
`connect-src`	AJAX, WebSocket, fetch
`font-src`	Web fonts
`frame-src`	Embedded frames
`form-action`	Form submission targets

Values:

'self' — Same origin only
'none' — Block completely
https://domain.com — Specific origin
'unsafe-inline' — Allow inline scripts/styles (weaker security)
'nonce-abc123' — Allow specific inline with matching nonce

Comprehensive Security Headers Configuration
Nginx
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
# Nginx configuration for security headers
 
server {
    listen 443 ssl http2;
    server_name example.com;
 
    # HTTPS only for 1 year, including subdomains
    add_header Strict-Transport-Security "max-age=31536000; includeSubDomains; preload" always;
    
    # Strict CSP - adjust based on your needs
    add_header Content-Security-Policy "default-src 'self'; script-src 'self' 'nonce-$request_id'; style-src 'self' 'unsafe-inline'; img-src 'self' data: https:; font-src 'self' https://fonts.gstatic.com; connect-src 'self' https://api.example.com; frame-ancestors 'none'; base-uri 'self'; form-action 'self';" always;
    
    # Prevent MIME type sniffing
    add_header X-Content-Type-Options "nosniff" always;
    
    # Prevent embedding in frames (clickjacking)
    add_header X-Frame-Options "DENY" always;
    
    # Control referrer information
    add_header Referrer-Policy "strict-origin-when-cross-origin" always;
    
    # Disable dangerous features
    add_header Permissions-Policy "geolocation=(), camera=(), microphone=(), payment=(), usb=()" always;
    
    # Cross-Origin policies
    add_header Cross-Origin-Opener-Policy "same-origin" always;
    add_header Cross-Origin-Embedder-Policy "require-corp" always;
    add_header Cross-Origin-Resource-Policy "same-origin" always;
}

CSP Implementation Challenges

Implementing CSP on existing sites is notoriously difficult:

• Inline scripts/styles require refactoring or nonce/hash • Third-party widgets (ads, analytics) need explicit allowlisting • Legacy code may break unexpectedly

Recommended approach:

Start with Content-Security-Policy-Report-Only (logs violations, doesn't block)
Set up a reporting endpoint to collect violations
Gradually tighten policy based on reports
Switch to enforcing Content-Security-Policy

CORS Headers: Cross-Origin Communication

The Same-Origin Policy

Browsers enforce the Same-Origin Policy: scripts on one origin cannot read responses from another origin. Two URLs are same-origin if they share:

Protocol (http vs https)
Domain (api.example.com vs example.com)
Port (80 vs 8080)

CORS headers explicitly relax this policy for trusted origins.

Key CORS Headers

CORS Response Headers
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
# Simple request response
HTTP/1.1 200 OK
Access-Control-Allow-Origin: https://app.example.com
Vary: Origin
 
# Response allowing credentials (cookies)
HTTP/1.1 200 OK
Access-Control-Allow-Origin: https://app.example.com
Access-Control-Allow-Credentials: true
Vary: Origin
 
# Preflight response (OPTIONS request)
HTTP/1.1 204 No Content
Access-Control-Allow-Origin: https://app.example.com
Access-Control-Allow-Methods: GET, POST, PUT, DELETE, OPTIONS
Access-Control-Allow-Headers: Content-Type, Authorization, X-Request-ID
Access-Control-Expose-Headers: X-Total-Count, X-Request-ID
Access-Control-Max-Age: 86400
Vary: Origin

CORS Response Headers Explained
Header	Purpose	Example Value
`Access-Control-Allow-Origin`	Which origins may access	`https://app.example.com` or `*`
`Access-Control-Allow-Methods`	Allowed HTTP methods	`GET, POST, PUT, DELETE`
`Access-Control-Allow-Headers`	Allowed request headers	`Content-Type, Authorization`
`Access-Control-Expose-Headers`	Headers JS can read	`X-Total-Count, X-Request-ID`
`Access-Control-Allow-Credentials`	Allow cookies/auth	`true` (requires specific origin)
`Access-Control-Max-Age`	Preflight cache duration	`86400` (seconds)

Simple vs. Preflighted Requests

Simple requests (no preflight needed):

Methods: GET, HEAD, POST
Headers: Only Accept, Accept-Language, Content-Language, Content-Type (with limited values)
Content-Type: only text/plain, multipart/form-data, application/x-www-form-urlencoded

Preflighted requests (browser sends OPTIONS first):

Custom headers (Authorization, X-Custom-Header)
Non-simple methods (PUT, DELETE, PATCH)
Content-Type: application/json

Preflight Flow

1. Browser: "I want to PUT to api.example.com with JSON body and Authorization header"
2. Browser sends OPTIONS request first:
   OPTIONS /resource
   Origin: https://app.example.com
   Access-Control-Request-Method: PUT
   Access-Control-Request-Headers: Authorization, Content-Type

3. Server responds with permissions:
   Access-Control-Allow-Origin: https://app.example.com
   Access-Control-Allow-Methods: PUT
   Access-Control-Allow-Headers: Authorization, Content-Type
   Access-Control-Max-Age: 86400

4. Browser caches preflight result (86400 seconds)
5. Browser sends actual PUT request
6. Server processes request and includes CORS headers in response

Common CORS Mistakes

1. Using Access-Control-Allow-Origin: * with credentials This is FORBIDDEN. When Allow-Credentials: true, you MUST specify exact origin.

2. Forgetting Vary: Origin Without it, CDNs may cache the CORS headers for one origin and serve them to others.

3. Not handling OPTIONS method Many frameworks don't auto-respond to OPTIONS preflight requests.

4. Exposing sensitive headers accidentally Access-Control-Expose-Headers makes headers readable by JS—be selective.

Custom Headers and Conventions

Beyond standard headers, applications commonly define custom headers for specific needs.

X-Prefix Convention (Deprecated)

Historically, custom headers used the X- prefix:

X-Request-ID: abc123
X-Rate-Limit-Remaining: 4999

RFC 6648 deprecated this convention in 2012 because:

Transitioning from experimental to standard is awkward
Too many production systems use X-prefixed headers
The prefix provides no real benefit

Current recommendation: Just name your header descriptively. If it becomes a standard, it can keep its name.

Common Custom Headers

Header	Purpose	Example
`X-Request-ID` / `Request-Id`	Request tracing	`7a3f8b2c-uuid`
`X-RateLimit-Limit`	Rate limit ceiling	`1000`
`X-RateLimit-Remaining`	Remaining requests	`999`
`X-RateLimit-Reset`	When limit resets	`1705582800` (Unix timestamp)
`X-Total-Count`	Total items (pagination)	`1234`
`X-Forwarded-For`	Original client IP (proxies)	`203.0.113.1, 198.51.100.2`
`X-Forwarded-Proto`	Original protocol	`https`
`X-Correlation-ID`	Distributed tracing	`trace-abc123`
`Idempotency-Key`	Request deduplication	`unique-uuid`

Rate Limiting Headers Example
HTTP
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
HTTP/1.1 200 OK
Content-Type: application/json
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 995
X-RateLimit-Reset: 1705586400
X-RateLimit-Window: 3600
Retry-After: 3600
 
{
  "data": [...],
  "meta": {
    "rateLimit": {
      "limit": 1000,
      "remaining": 995,
      "reset": "2025-01-18T14:00:00Z"
    }
  }
}
 
# When rate limited:
HTTP/1.1 429 Too Many Requests
Retry-After: 1800
X-RateLimit-Limit: 1000
X-RateLimit-Remaining: 0
X-RateLimit-Reset: 1705588200
 
{
  "error": "Rate limit exceeded",
  "retryAfter": 1800
}

Forwarding Headers (Proxy Context)

When requests pass through load balancers or reverse proxies, these headers preserve original request information:

X-Forwarded-For: 203.0.113.1, 198.51.100.22, 10.0.0.5
X-Forwarded-Proto: https
X-Forwarded-Host: original-domain.com
X-Forwarded-Port: 443

Security consideration: Only trust XFF headers from known proxy IPs. Clients can forge these headers; your proxy should overwrite/append, not blindly trust.

The Forwarded Header (Standard)

RFC 7239 standardized proxy forwarding into a single header:

Forwarded: for=203.0.113.1; proto=https; host=example.com; by=10.0.0.5

This is cleaner but not as widely supported as the X-Forwarded-* family.

Summary: Mastering HTTP Headers

HTTP headers form the invisible infrastructure that enables the modern web. Let's consolidate the essential knowledge:

Key Takeaways

•Headers are metadata — They control how requests and responses are processed, not what content is transferred
•Content negotiation enables flexibility — Accept headers let clients and servers agree on formats, languages, and encodings
•Caching headers are performance multipliers — Proper Cache-Control, ETag, and Vary headers can reduce server load by orders of magnitude
•Security headers provide defense-in-depth — HSTS, CSP, and X-Content-Type-Options should be on every production site
•CORS enables controlled cross-origin access — Essential for modern API-driven architectures with separate frontend/backend domains
•Custom headers extend HTTP — Rate limiting, request tracing, and application-specific metadata all use custom headers

Page Complete

2 / 5