Certificate Validation: Authenticating Server Identity

When you connect to https://bank.example.com, how do you know you're actually communicating with your bank and not an attacker? The answer lies in certificate validation—a sophisticated process where cryptographic proofs chain back to trusted authorities.

Certificates are the identity documents of the internet. Just as a passport proves your identity through a chain of governmental trust, an X.509 certificate proves a server's identity through a chain of Certificate Authority (CA) trust. Understanding this system is essential for any engineer working with web security.

In this comprehensive exploration, we'll dissect the complete certificate validation process—from certificate structure and chain building through validation algorithms and failure handling. You'll learn not just how validation works, but why each component exists and what attacks it prevents.

An X.509 certificate is a standardized data structure (defined in RFC 5280) that binds a public key to an identity. Every TLS certificate follows this format precisely, enabling universal interoperability across implementations.

The Three Core Components:

1. TBSCertificate (To Be Signed): The actual certificate data
2. Signature Algorithm: How the certificate was signed
3. Signature Value: The CA's signature over the TBSCertificate

Certificate  ::=  SEQUENCE  {
    tbsCertificate       TBSCertificate,
    signatureAlgorithm   AlgorithmIdentifier,
    signatureValue       BIT STRING
}

Let's examine each field in detail:

Critical Extensions for TLS Certificates:

X.509 version 3 introduced extensions that are essential for modern security:

1. Subject Alternative Name (SAN)

DNS: www.example.com
DNS: example.com
DNS: api.example.com
IP: 192.0.2.1

The SAN extension lists all valid identities for this certificate. This is what browsers actually check when validating hostname matching—not the Subject CN (which is deprecated for hostname validation).

2. Key Usage

Digital Signature: YES
Key Encipherment: YES
Key Agreement: NO

Restricts what operations the key can perform. A TLS server certificate must have Digital Signature to sign handshake data.

3. Extended Key Usage

TLS Web Server Authentication: YES
TLS Web Client Authentication: NO

Further restricts certificate purpose. This prevents, for example, an email signing certificate from being used for TLS.

4. Basic Constraints

CA: FALSE
Path Length: N/A

Indicates whether this certificate can sign other certificates. Leaf certificates (servers) have CA:FALSE. Intermediate CAs have CA:TRUE.

Certificate Encoding Formats:

Certificates are encoded in various formats, which can cause confusion:

Viewing a Certificate (OpenSSL):

# View certificate details
openssl x509 -in cert.pem -text -noout

# Output includes:
Certificate:
    Data:
        Version: 3 (0x2)
        Serial Number: 04:00:00:00:00:01:23:45:67:89
        Signature Algorithm: sha256WithRSAEncryption
        Issuer: CN = Example CA, O = Example Corp
        Validity
            Not Before: Jan  1 00:00:00 2024 GMT
            Not After : Jan  1 00:00:00 2025 GMT
        Subject: CN = www.example.com
        Subject Public Key Info:
            Public Key Algorithm: rsaEncryption
                RSA Public-Key: (2048 bit)
        X509v3 extensions:
            X509v3 Subject Alternative Name:
                DNS:www.example.com, DNS:example.com
            X509v3 Key Usage: critical
                Digital Signature, Key Encipherment
            X509v3 Extended Key Usage:
                TLS Web Server Authentication

A single certificate is meaningless without context. The power of PKI comes from certificate chains—a sequence of certificates linking a server's identity back to a trusted root.

The Three-Level Hierarchy:

┌─────────────────────────────────────────────────────┐
│               Root CA Certificate                    │
│  (Self-signed, distributed with OS/browser)         │
│  Validity: 10-25 years                              │
│  Stored offline in HSMs                             │
└─────────────────────────────────────────────────────┘
                          │
                          │ Signs
                          ↓
┌─────────────────────────────────────────────────────┐
│            Intermediate CA Certificate              │
│  (Signed by Root, signs end-entity certs)          │
│  Validity: 3-10 years                              │
│  Operated online, more exposed                      │
└─────────────────────────────────────────────────────┘
                          │
                          │ Signs
                          ↓
┌─────────────────────────────────────────────────────┐
│           End-Entity (Leaf) Certificate             │
│  (The actual server certificate)                    │
│  Validity: 90 days - 2 years                       │
│  Installed on web servers                           │
└─────────────────────────────────────────────────────┘

Why Use Intermediates?

1. Security Isolation: Root keys are kept offline in Hardware Security Modules (HSMs). If an intermediate is compromised, the root isn't exposed.

2. Revocation Scope: Revoking an intermediate affects only certificates it signed, not the entire root.

3. Operational Flexibility: Multiple intermediates can exist for different purposes (OV, EV, specific regions).

4. Key Rotation: Intermediates can be rotated more frequently than roots.

Chain Building:

During TLS handshake, the server sends its certificate chain (excluding the root). The client must:

1. Receive Leaf Certificate from the server
2. Receive Intermediate(s) from the server (or fetch via AIA)
3. Match Issuer to Subject up the chain
4. Find Trust Anchor (root) in local trust store

Issuer-Subject Linking:

Each certificate's Issuer field must exactly match the next certificate's Subject field:

Leaf Certificate:
    Subject: CN=www.example.com
    Issuer:  CN=Example Intermediate CA
            ↓ Must match
Intermediate Certificate:
    Subject: CN=Example Intermediate CA
    Issuer:  CN=Example Root CA
            ↓ Must match
Root Certificate (in trust store):
    Subject: CN=Example Root CA
    Issuer:  CN=Example Root CA (self-signed)

Trust Stores:

Operating systems and browsers maintain curated lists of trusted root CAs:

How CAs Get Into Trust Stores:

CA inclusion is a rigorous process:

1. CA must meet technical requirements (key sizes, validity, security audit)
2. CA must pass WebTrust or equivalent audit annually
3. CA must publish CP/CPS (Certificate Policy/Certification Practice Statement)
4. CA must participate in Certificate Transparency
5. Trust store operator reviews and votes on inclusion

This gatekeeping is crucial—every root CA can issue certificates for any domain.

Certificate validation is a multi-step process defined in RFC 5280. A certificate is valid only if every step passes. Let's trace through the complete algorithm:

Step 1: Hostname Verification

The presented hostname must match the certificate:

Requested URL: https://www.example.com/api

Certificate SAN:
    DNS: www.example.com     ← Match!
    DNS: example.com
    DNS: api.example.com

Result: PASS

Wildcard Matching Rules:

• *.example.com matches www.example.com, api.example.com
• *.example.com does NOT match example.com (no subdomain)
• *.example.com does NOT match test.www.example.com (only one level)
• * must be leftmost label only

Step 2: Validity Period Check

Current Time:    2024-06-15 10:30:00 UTC
Not Before:      2024-01-01 00:00:00 UTC  ← Before current time: OK
Not After:       2025-01-01 00:00:00 UTC  ← After current time: OK

Result: PASS (certificate is within validity period)

This check applies to every certificate in the chain. If any certificate is expired or not yet valid, the entire chain fails.

Step 3: Signature Verification

For each certificate in the chain, verify the signature using the issuer's public key:

Leaf Certificate:
    Signature: 3082010a....
    Signed by: Intermediate CA
    
    Verification: Decrypt signature with Intermediate's public key
                  Compare to hash of TBSCertificate
                  Result: MATCH → PASS

This proves the certificate hasn't been modified and was genuinely issued by the claimed issuer.

Step 4: Chain Building and Root Trust

Build the chain from leaf to root:

1. Start with leaf certificate
2. Find certificate whose Subject == Leaf's Issuer
3. Repeat until reaching self-signed certificate
4. Verify self-signed certificate is in trust store

Step 5: Basic Constraints Validation

For each certificate acting as a CA:

• CA: TRUE must be set
• pathLenConstraint (if present) must not be exceeded

Root CA:        CA=TRUE, pathLen=1  (can have 1 intermediate below)
Intermediate:   CA=TRUE, pathLen=0  (can sign only leaf certs)
Leaf:           CA=FALSE            (cannot sign any certs)

Step 6: Key Usage Validation

Leaf certificate must have appropriate Key Usage:

• digitalSignature — For ECDHE key exchange signatures
• keyEncipherment — For RSA key exchange (legacy)

Step 7: Extended Key Usage

serverAuth (1.3.6.1.5.5.7.3.1) must be present for TLS server certs.

Certificates can become invalid before their expiration date—private keys get compromised, companies change ownership, or certificates are mis-issued. Revocation is the mechanism to invalidate such certificates.

Two Primary Revocation Methods:

1. Certificate Revocation Lists (CRLs)

A CRL is a signed list of revoked certificate serial numbers published by the CA:

CRL Distribution Point: http://crl.example.com/ca.crl

CRL Contents:
    Issuer: CN=Example CA
    This Update: 2024-06-15 00:00:00 UTC
    Next Update: 2024-06-16 00:00:00 UTC
    Revoked Certificates:
        Serial: 12345, Revocation Date: 2024-05-01, Reason: keyCompromise
        Serial: 67890, Revocation Date: 2024-05-15, Reason: cessationOfOperation
        ...
    Signature: ...

2. Online Certificate Status Protocol (OCSP)

OCSP provides real-time revocation status for individual certificates:

Client Request:
    POST http://ocsp.example.com
    Content-Type: application/ocsp-request
    
    Serial: 12345
    Issuer: CN=Example CA

OCSP Response:
    Response Status: successful
    Certificate Status: good|revoked|unknown
    This Update: 2024-06-15 10:00:00 UTC
    Next Update: 2024-06-15 22:00:00 UTC
    Signature: ...

OCSP Stapling: The Best of Both Worlds

OCSP has a privacy problem—the CA sees every certificate the client verifies. OCSP Stapling solves this:

1. Server periodically fetches its own OCSP response from CA
2. Server includes ("staples") this response in TLS handshake
3. Client validates the stapled response
4. No client-CA communication needed

Soft-Fail vs Hard-Fail:

A critical question: what happens if revocation information is unavailable?

Most browsers use soft-fail by default, which means revocation is easily bypassed. This is why OCSP Stapling with Must-Staple is important for high-security applications.

Server Configuration (Nginx OCSP Stapling):

# Enable OCSP stapling
ssl_stapling on;
ssl_stapling_verify on;

# File with CA chain for OCSP response verification
resolver 8.8.8.8 8.8.4.4 valid=300s;
resolver_timeout 5s;

# Trusted CA certificates for OCSP response verification
ssl_trusted_certificate /etc/ssl/certs/ca-chain.pem;

Certificate Transparency (CT) is a framework for publicly logging all issued certificates, enabling detection of rogue or mis-issued certificates.

The Problem CT Solves:

Historically, CAs could issue any certificate for any domain, and unless the victim actively monitored, mis-issuance went undetected. Notable incidents:

• 2011: DigiNotar — Compromised CA issued Google certificates
• 2015: CNNIC — Issued unauthorized MCS Holdings intermediate
• 2017: Symantec — Thousands of mis-issued certificates

CT Architecture:

┌──────────┐    Issue     ┌──────────────┐   Submit   ┌─────────────┐
│   CA     │──────────────▶│  Certificate │───────────▶│  CT Logs    │
└──────────┘              └──────────────┘            │  (Merkle    │
                                                       │   Trees)    │
                                                       └─────────────┘
                                                              │
                                          Returns SCT         │
                                              ▼                │
┌──────────┐    Validate   ┌──────────────┐   Monitor  ┌─────────────┐
│  Browser │◀──────────────│  Certificate │◀───────────│  Monitors   │
│          │               │  + SCT       │            │  (Domain    │
│          │               │              │            │   owners,   │
└──────────┘               └──────────────┘            │   auditors) │
                                                       └─────────────┘

Signed Certificate Timestamp (SCT):

When a CA submits a certificate to a CT log, the log returns a Signed Certificate Timestamp (SCT)—a cryptographic promise to include the certificate in the log within a specified time.

SCT Delivery Methods:

1. Embedded in Certificate — CA gets SCT before issuance, includes in certificate
2. TLS Extension — Server sends SCTs in signed_certificate_timestamp extension
3. OCSP Response — SCT included in stapled OCSP response

Browser Requirements:

Chrome, Safari, and other browsers require SCTs from multiple independent logs for certificates issued after 2018. Without valid SCTs, certificates are rejected.

Monitoring for Your Domains:

Domain owners can monitor CT logs to detect unauthorized certificate issuance:

# Using crt.sh (public CT log aggregator)
curl "https://crt.sh/?q=example.com&output=json" | jq '.'

# Shows all certificates ever issued for example.com

Services like Facebook's Certificate Transparency Monitoring, crt.sh, and commercial solutions provide alerting when new certificates appear for your domains.

Understanding why certificate validation fails is crucial for both security and troubleshooting. Let's examine the most common failure scenarios:

Diagnosing Certificate Issues (OpenSSL):

# Fetch and display server certificate
openssl s_client -connect example.com:443 -servername example.com

# Verify certificate chain
openssl verify -CAfile ca-bundle.crt -untrusted intermediate.crt server.crt

# Check certificate dates
openssl x509 -in cert.pem -noout -dates

# Check certificate SAN
openssl x509 -in cert.pem -noout -text | grep -A1 "Subject Alternative Name"

# Check OCSP status
openssl ocsp -url http://ocsp.example.com -issuer issuer.pem -cert cert.pem

Common Configuration Mistakes:

Standard certificate validation trusts hundreds of CAs—any one of which could issue a certificate for any domain. Certificate pinning restricts which certificates or CAs are valid for a specific domain, providing defense-in-depth against compromised CAs.

Pinning Strategies:

1. Public Key Pinning Pin the hash of the public key (most flexible—survives certificate renewal):

Pin SHA-256: base64(SHA256(SubjectPublicKeyInfo))

Example: pin-sha256="K87oWBWM9UZfyddvDfoxL+8lpNyoUB2ptGtn0fv6G2Q="

2. Certificate Pinning Pin the hash of the entire certificate (must update pin on renewal):

Pin SHA-256: base64(SHA256(Certificate))

3. CA Pinning Pin specific intermediate or root CA (allows any cert from that CA):

Accept certificates with chain containing:
  Issuer: CN=DigiCert SHA2 Extended Validation Server CA

HTTP Public Key Pinning (HPKP) — Deprecated:

HPKP was a browser feature that let sites declare pins via HTTP header:

Public-Key-Pins: 
  pin-sha256="base64==";
  pin-sha256="backup-base64==";
  max-age=5184000;
  includeSubDomains

Why HPKP Failed:

• Configuration errors caused permanent lockout
• Attackers could use it for ransom (set malicious pins)
• No recovery mechanism once pins were cached
• Chrome removed support in 2018

Modern Alternatives:

1. Expect-CT Header

Expect-CT: max-age=86400, enforce, report-uri="https://example.com/ct-report"

Requires Certificate Transparency (SCTs) for all connections.

2. Mobile App Pinning For mobile apps, pinning is implemented in the app code and highly recommended:

// Android OkHttp pinning
CertificatePinner pinner = new CertificatePinner.Builder()
    .add("api.example.com", "sha256/AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=")
    .add("api.example.com", "sha256/BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB=") // Backup
    .build();

3. CAA Records DNS CAA records specify which CAs can issue for a domain:

example.com. CAA 0 issue "letsencrypt.org"
example.com. CAA 0 issue "digicert.com"
example.com. CAA 0 iodef "mailto:security@example.com"

CAs must check CAA before issuance. This prevents rogue CAs from issuing, though it doesn't affect validation.