HTTP Headers: The Hidden Metadata Layer

Headers are the unsung heroes of HTTP. While the URL and body get all the attention, headers carry critical metadata that controls authentication, caching, content format, security, and more. Most API issues come down to missing or incorrect headers, so mastering them is essential.

---

1. What Are HTTP Headers?

Headers are key-value pairs that provide metadata about the request or response. They are text strings in the format Key: Value.

Content-Type: application/json
Authorization: Bearer eyJhbGci...
Accept: application/json
Cache-Control: no-cache

Headers are case-insensitive for the key (but convention is Title-Case), and they can appear in both requests and responses.

---

2. Request Headers You Must Know

Content-Type

Tells the server what format your request body is in:

# Sending JSON
curl -H "Content-Type: application/json" -d '{"key":"value"}' ...

# Sending form data
curl -H "Content-Type: application/x-www-form-urlencoded" -d "key=value" ...

# Uploading a file
curl -H "Content-Type: multipart/form-data" -F "file=@photo.jpg" ...

If you forget Content-Type, the server may reject your request with a 400 or 415 (Unsupported Media Type) error.

Accept

Tells the server what format you want the response in:

curl -H "Accept: application/json" https://api.example.com/data

Authorization

Sends authentication credentials:

# Bearer Token (JWT)
curl -H "Authorization: Bearer eyJhbGciOiJIUzI1NiIs..." ...

# Basic Auth (Base64 encoded username:password)
curl -H "Authorization: Basic dXNlcm5hbWU6cGFzc3dvcmQ=" ...

# API Key
curl -H "X-API-Key: your-key-here" ...

User-Agent

Identifies the client making the request:

User-Agent: curl/8.4.0
User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X)
User-Agent: MyApp/1.0

Some APIs block requests without a proper User-Agent.

Custom Headers

APIs can define their own headers, often prefixed with X-:

curl -H "X-Request-Id: unique-123" \
     -H "X-Correlation-Id: session-456" \
     -H "X-Client-Version: 2.1.0" \
     https://api.example.com/data

---

3. Response Headers You Must Know

Content-Type (Response)

Tells you the format of the response body:

Content-Type: application/json; charset=utf-8

Cache-Control

Instructs clients and proxies about caching:

Cache-Control: max-age=3600          # Cache for 1 hour
Cache-Control: no-cache              # Must revalidate before using
Cache-Control: no-store              # Never cache this
Cache-Control: public, max-age=86400 # Anyone can cache for 24 hours
Cache-Control: private, max-age=600  # Only the browser can cache for 10 min

Rate Limit Headers

X-RateLimit-Limit: 1000             # Max requests per window
X-RateLimit-Remaining: 950           # Requests left
X-RateLimit-Reset: 1708640000        # When the window resets (Unix timestamp)
Retry-After: 60                      # Seconds to wait before retrying (with 429)

Location

Returned with 201 Created to tell you where the new resource is:

HTTP/1.1 201 Created
Location: /api/v1/users/43

ETag

A version identifier for cache validation:

ETag: "33a64df5"

On subsequent requests, the client can send:

If-None-Match: "33a64df5"

If the resource has not changed, the server returns 304 Not Modified with no body, saving bandwidth.

---

4. CORS Headers (Cross-Origin Resource Sharing)

When a website at myapp.com makes an API call to api.example.com, the browser blocks it by default for security. CORS headers explicitly allow this.

How CORS Works

sequenceDiagram
    participant Browser as Browser (myapp.com)
    participant API as API Server (api.example.com)

    Note over Browser,API: Preflight Request (OPTIONS)
    Browser->>API: OPTIONS /data<br/>Origin: https://myapp.com
    API-->>Browser: Access-Control-Allow-Origin: https://myapp.com<br/>Access-Control-Allow-Methods: GET, POST

    Note over Browser,API: Actual Request
    Browser->>API: GET /data<br/>Origin: https://myapp.com
    API-->>Browser: 200 OK + data<br/>Access-Control-Allow-Origin: https://myapp.com

Important CORS Headers

Header	Purpose	Example
Access-Control-Allow-Origin	Which domains can call this API	https://myapp.com or *
Access-Control-Allow-Methods	Which HTTP methods are allowed	GET, POST, PUT, DELETE
Access-Control-Allow-Headers	Which request headers are allowed	Content-Type, Authorization
Access-Control-Max-Age	How long to cache the preflight	86400

CORS only affects browsers, not cURL, Postman, or server-to-server requests.

---

5. Security Headers

Modern APIs use headers for security:

Header	Purpose	Example
Strict-Transport-Security	Force HTTPS	max-age=31536000
X-Content-Type-Options	Prevent MIME sniffing	nosniff
X-Frame-Options	Prevent clickjacking	DENY
Content-Security-Policy	Control resource loading	default-src 'self'
X-XSS-Protection	XSS filter	1; mode=block

You will see these in response headers but rarely need to set them as a client.

---

6. Conditional Requests

Conditional headers allow efficient caching by avoiding re-downloading unchanged data.

If-Modified-Since

curl -H "If-Modified-Since: Sat, 22 Feb 2026 12:00:00 GMT" \
  https://api.example.com/data

If the data has not changed since that date, the server returns 304 Not Modified with no body.

If-None-Match (ETag)

# First request — get the data and ETag
curl -i https://api.example.com/data
# Response includes: ETag: "abc123"

# Second request — only download if changed
curl -H 'If-None-Match: "abc123"' https://api.example.com/data
# If unchanged: 304 Not Modified (no body, saves bandwidth)
# If changed: 200 OK (new data and new ETag)

---

7. Practical Header Exercises

# See all headers the GitHub API returns
curl -I https://api.github.com/users/octocat

# Check rate limit headers
curl -I https://api.github.com/rate_limit

# Send multiple custom headers
curl -H "Accept: application/json" \
     -H "X-Custom-Header: test" \
     -H "User-Agent: RestAPICourse/1.0" \
     https://jsonplaceholder.typicode.com/posts/1

# See exactly what headers cURL sends
curl -v https://jsonplaceholder.typicode.com/posts/1 2>&1 | grep ">"

---

Summary and Key Takeaways

1. Content-Type tells the server what you are sending; Accept tells it what you want back.
2. Authorization carries authentication credentials — always use HTTPS.
3. Cache-Control and ETag enable efficient caching and conditional requests.
4. CORS headers allow browsers to make cross-origin API calls.
5. Rate limit headers tell you how many requests you have left.
6. Missing or incorrect headers are the #1 cause of API errors for beginners.

---

Lesson Review Quiz