Server Types

Not all servers are created equal. The simple concept of 'a program that responds to requests' encompasses an enormous variety of architectures, each optimized for different workloads, scale requirements, and operational constraints. From a tiny embedded server handling a few requests per hour to a global-scale service processing millions per second, the design decisions differ dramatically.

Understanding the taxonomy of server types enables architects and engineers to select the right architecture for their needs, understand the tradeoffs involved, and design systems that meet real-world requirements.

The most fundamental distinction in server design is how multiple clients are handled: one at a time (iterative) or simultaneously (concurrent).

Another critical distinction is whether the server maintains information about clients between requests. This decision has profound implications for scalability, reliability, and complexity.

Stateless Server:

A stateless server treats each request as independent—no memory of previous interactions. Every request contains all information needed to process it.

Client: GET /user/123 (with auth token)
Server: [Validates token, queries database, returns user]
        [Immediately forgets this client]

Client: GET /user/123/orders (with auth token)  
Server: [Validates token again, queries database, returns orders]
        [No memory of previous request]

Stateful Server:

A stateful server maintains session information between requests. The client is 'remembered' across a series of interactions.

Client: CONNECT user/password
Server: [Validates, creates session 'abc123']
        [Stores: session 'abc123' = {user: 'john', authenticated: true}]

Client: REQUEST data (session: abc123)
Server: [Finds session 'abc123', knows user is 'john']
        [Returns data for 'john']

Client: QUIT (session: abc123)  
Server: [Destroys session 'abc123']

Making Stateful Systems Scalable:

When statefulness is required but scalability is also needed, several patterns help:

1. Session Affinity (Sticky Sessions) — Load balancer routes all requests from a user to the same server. Simple but creates uneven load and failover issues.

2. External Session Store — Move session state to a shared store (Redis, Memcached, database). Any server can access any session.

3. Client-Stored Sessions — Encode session in encrypted token (like JWT) sent with each request. Server is stateless but client carries state.

4. Sharded State — Partition sessions across servers deterministically (e.g., by user ID hash). Predictable routing without sticky sessions.

Most modern large-scale systems prefer stateless where possible, using pattern #2 or #3 when sessions are unavoidable.

Servers can be categorized by whether they use connection-oriented protocols (like TCP) or connectionless protocols (like UDP). This choice affects reliability, latency, and server design.

As applications grow in complexity, servers are organized into multiple tiers, each handling different responsibilities. This separation enables specialized optimization, independent scaling, and clearer system organization.

Two-Tier Architecture (Client-Server):

The classic client-server model: clients connect directly to a server that handles everything.

┌──────────┐         ┌───────────────────────────────────┐
│  Client  │ ───────▶│  Server                           │
│  (UI)    │ ◀─────── │  - Presentation Logic             │
└──────────┘         │  - Business Logic                 │
                     │  - Data Access                    │
                     │  - Database                       │
                     └───────────────────────────────────┘

• Advantages: Simple deployment, low latency within server
• Disadvantages: Monolithic, hard to scale, single point of failure

Three-Tier Architecture:

Separates presentation, business logic, and data storage.

┌──────────┐         ┌─────────────┐         ┌─────────────┐         ┌──────────┐
│  Client  │────────▶│ Presentation│────────▶│  Business   │────────▶│ Database │
│          │◀────────│    Tier     │◀────────│  Logic Tier │◀────────│   Tier   │
└──────────┘         │ (Web Server)│         │ (App Server)│         │  (DBMS)  │
                     └─────────────┘         └─────────────┘         └──────────┘

Tier Responsibilities:

N-Tier / Modern Service Architecture:

Modern systems often go beyond three tiers, with specialized components:

Benefits of Multi-Tier:

• Independent Scaling: Scale each tier to its bottleneck
• Technology Freedom: Use best tool for each tier
• Isolation: Failure in one tier can be contained
• Team Organization: Teams can own specific tiers
• Security Boundaries: Database tier not directly exposed

Costs of Multi-Tier:

• Network Latency: Each tier hop adds latency
• Operational Complexity: More components to deploy, monitor, debug
• Distributed System Issues: Network partitions, consistency challenges

Beyond the fundamental categorizations, many specialized server types exist to address specific use cases.

Proxy Servers in Detail:

Proxy servers are particularly important as they sit between clients and origin servers, providing various benefits:

Forward Proxy:

[Client] ──▶ [Forward Proxy] ──▶ [Internet] ──▶ [Origin Server]

• Client explicitly configured to use proxy
• Used for: Access control, anonymity, caching, filtering content
• Examples: Corporate proxy, Tor network

Reverse Proxy:

[Client] ──▶ [Internet] ──▶ [Reverse Proxy] ──▶ [Origin Server(s)]

• Client unaware of proxy
• Used for: Load balancing, SSL termination, caching, security
• Examples: nginx in front of application servers

Transparent Proxy:

• Intercepts traffic without client configuration
• Used by ISPs for caching, enterprises for filtering
• May raise privacy/security concerns

Beyond simple concurrent vs. iterative, modern servers employ sophisticated process and threading models to maximize performance and reliability.

Single-Process, Single-Threaded (Event Loop):

┌─────────────────────────────────────────┐
│  Single Process / Single Thread          │
│  ┌─────────────────────────────────────┐ │
│  │  Event Loop                          │ │
│  │  - poll() for events                 │ │
│  │  - Process ready I/O                 │ │
│  │  - Run callbacks                     │ │
│  │  - Back to poll()                    │ │
│  └─────────────────────────────────────┘ │
└─────────────────────────────────────────┘

• Model: All I/O is non-blocking; single event loop handles thousands of connections
• Examples: Node.js (single process), Redis (single-threaded core)
• Pros: No synchronization needed; simple mental model for I/O-bound work
• Cons: Can't use multiple CPU cores; CPU-intensive work blocks everything

Multi-Process, Single-Threaded Each (Pre-fork):

┌─────────────────────────────────────────┐
│  Master Process (coordination)           │
│      │           │           │           │
│      ▼           ▼           ▼           │
│  ┌───────┐   ┌───────┐   ┌───────┐      │
│  │Worker │   │Worker │   │Worker │      │
│  │Process│   │Process│   │Process│      │
│  │(event │   │(event │   │(event │      │
│  │ loop) │   │ loop) │   │ loop) │      │
│  └───────┘   └───────┘   └───────┘      │
└─────────────────────────────────────────┘

• Model: Master spawns worker processes; each worker has its own event loop
• Examples: nginx, Gunicorn, PM2 cluster mode
• Pros: Uses all CPU cores; worker crash doesn't kill server; isolation
• Cons: Inter-process communication complex; memory not shared (unless explicit)

Multi-Process, Multi-Threaded:

┌─────────────────────────────────────────┐
│  Process Pool                            │
│  ┌─────────────────┐  ┌────────────────┐│
│  │  Process 1      │  │  Process 2     ││
│  │  ┌────┐ ┌────┐  │  │  ┌────┐ ┌────┐ ││
│  │  │ T1 │ │ T2 │  │  │  │ T1 │ │ T2 │ ││
│  │  └────┘ └────┘  │  │  └────┘ └────┘ ││
│  │  ┌────┐ ┌────┐  │  │  ┌────┐ ┌────┐ ││
│  │  │ T3 │ │ T4 │  │  │  │ T3 │ │ T4 │ ││
│  │  └────┘ └────┘  │  │  └────┘ └────┘ ││
│  └─────────────────┘  └────────────────┘│
└─────────────────────────────────────────┘

• Model: Multiple processes, each with multiple threads
• Examples: Apache with worker MPM, Java application servers
• Pros: Maximum parallelism; shared memory within process
• Cons: Most complex; debugging challenging; thread safety required

Selecting the appropriate server architecture depends on workload characteristics, scale requirements, reliability needs, and operational constraints.