Servers: Application,… — HLD: Client-Server Model | OneNoughtOne

0/86

System DesignClient-Server Model

Client-Server Model

LevelBeginner

Duration60 mins

TopicClient-Server Model

4 / 4

Servers: Application, Database, Cache

The Layered Ecosystem That Powers Every Application

When we speak of 'the server' in client-server architecture, we often imagine a single entity—a box that receives requests and returns responses. But in reality, modern server-side infrastructure is a layered ecosystem of specialized components, each optimized for specific responsibilities. A single API request might traverse: A load balancer distributing traffic A CDN edge node serving cached content An application server executing business logic A cache server providing frequently-accessed data A database server for persistent storage A message queue for asynchronous processing Understanding this ecosystem—the role of each layer, how they interact, and when to use each—is fundamental to designing systems that are fast, reliable, and scalable. This page explores the three core server types that form the backbone of virtually every system: application servers, database servers, and cache servers .

What You Will Learn

By the end of this page, you will understand the role and responsibilities of application servers, the architecture and trade-offs of database servers, and why cache servers are essential for performance. You'll see how these layers work together in modern three-tier and multi-tier architectures.

Application Servers: The Business Logic Layer

Application servers are the core of your system—the component that receives client requests, executes business logic, orchestrates data access, and returns responses. They embody the 'what your application does' aspect of your system. Core Responsibilities of Application Servers:

Application Server Responsibilities

•Request Handling — Accepting incoming HTTP/gRPC/WebSocket connections, parsing requests, and routing them to appropriate handlers based on URL, method, and headers.
•Authentication & Authorization — Verifying client identity, validating credentials, and enforcing access control policies before processing requests.
•Business Logic Execution — Implementing domain-specific rules, calculations, validations, and workflows that define your application's behavior.
•Data Orchestration — Coordinating data access across databases, caches, external APIs, and other services to fulfill requests.
•Response Generation — Formatting responses, serializing data to appropriate formats (JSON, Protobuf, HTML), and sending them back to clients.
•Cross-Cutting Concerns — Logging, metrics collection, distributed tracing, rate limiting, and error handling that span all requests.

Types of Application Servers: Web Servers (HTTP-focused): NGINX, Apache, Caddy—primarily handle HTTP traffic, serve static files, and proxy requests to application backends. Often sit in front of application servers as reverse proxies. Language-Specific Application Servers: Node.js : Express, Fastify, NestJS Python : Gunicorn, uWSGI, Uvicorn (for Django, Flask, FastAPI) Java : Tomcat, Jetty, Undertow (for Spring, Jakarta EE) Go : net/http, Gin, Echo (compiled into the application) Ruby : Puma, Unicorn (for Rails) API Gateways: Kong, AWS API Gateway, Apigee—specialized application servers that handle API-specific concerns: authentication, rate limiting, transformation, and routing to backend services.

Application Server Architectures
Architecture	Description	Pros	Cons	Use Cases
Single-threaded event loop	One thread handles many connections via async I/O	Efficient for I/O-bound work, simple	CPU-bound work blocks	Node.js, NGINX worker
Multi-threaded	Thread pool handles concurrent requests	CPU parallelism, isolation	Context switching overhead	Java servlets, traditional
Process per request	Fork new process for each request	Isolation, stability	High overhead	Legacy CGI, some PHP
Worker pool	Pool of workers (processes/threads)	Load distribution, resource limits	Pool sizing complexity	Gunicorn, PM2, Puma
Actor-based	Actors process messages asynchronously	Scalable, fault-tolerant	Complexity	Akka, Erlang/OTP

application-server-layers.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
// Modern Application Server Architecture (Express/Node.js Example)
 
import express, { Request, Response, NextFunction } from 'express';
import { createLogger, transports, format } from 'winston';
import { rateLimit } from 'express-rate-limit';
import { authenticateRequest } from './auth';
import { metricsMiddleware } from './observability';
import { errorHandler } from './errors';
 
const app = express();
 
// Layer 1: Request Parsing
app.use(express.json({ limit: '10mb' }));
app.use(express.urlencoded({ extended: true }));
 
// Layer 2: Observability
app.use(metricsMiddleware());
app.use((req, res, next) => {
  req.correlationId = req.headers['x-correlation-id'] || generateUUID();
  res.setHeader('X-Correlation-ID', req.correlationId);
  next();
});
 
// Layer 3: Security
app.use(rateLimit({ windowMs: 60000, max: 100 })); // 100 req/min
app.use('/api', authenticateRequest);
 
// Layer 4: Routing to Business Logic
app.use('/api/v1/users', userRouter);      // User domain
app.use('/api/v1/orders', orderRouter);    // Order domain
app.use('/api/v1/products', productRouter);// Product domain
 
// Layer 5: Error Handling
app.use(errorHandler);
 
// Business Logic Handler Example
async function createOrder(req: Request, res: Response) {
  const { customerId, items } = req.body;
  
  // Validate input
  validateOrderInput(customerId, items);
  
  // Business logic: check inventory, calculate total
  const inventoryCheck = await inventoryService.check(items);
  if (!inventoryCheck.available) {
    throw new InsufficientInventoryError(inventoryCheck.unavailable);
  }
  
  // Data orchestration: save to database
  const order = await orderRepository.create({
    customerId,
    items,
    total: calculateTotal(items),
    status: 'pending'
  });
  
  // Async side effects: emit event for downstream processing
  await eventBus.publish('order.created', { orderId: order.id });
  
  // Response
  res.status(201).json(order);
}

Stateless Application Servers

Modern application servers are designed to be stateless—they don't store session data in memory. This enables horizontal scaling (add more servers), load balancer freedom (any server can handle any request), and zero-downtime deployments (replace servers without losing state). State is externalized to databases, caches, and session stores.

Database Servers: The Persistence Layer

Database servers are specialized systems optimized for storing, querying, and managing data. They provide durability (data survives restarts), consistency (transactions and constraints), and efficient access (indexes and query optimization). Unlike application servers that are often stateless, database servers are inherently stateful —they own and persist your application's most valuable asset: its data. Core Responsibilities of Database Servers:

Database Server Responsibilities

•Data Persistence — Writing data to durable storage (disk, SSDs) that survives process restarts, server reboots, and hardware failures.
•Query Processing — Parsing, optimizing, and executing queries. The query optimizer is one of the most complex components, choosing efficient execution plans.
•Transaction Management — Ensuring ACID properties (Atomicity, Consistency, Isolation, Durability) for operations that must succeed or fail as a unit.
•Concurrency Control — Managing simultaneous access from multiple clients without data corruption. Locking, MVCC, and isolation levels balance consistency and performance.
•Data Integrity — Enforcing constraints (primary keys, foreign keys, unique, not null, check) that maintain data validity.
•Index Management — Building and maintaining indexes that accelerate query performance, balancing read speed against write overhead.

Categories of Database Servers: Relational Databases (SQL): PostgreSQL, MySQL, Oracle, SQL Server—organize data in tables with defined schemas and relationships. Excel at complex queries, joins, and transactions. Document Databases: MongoDB, CouchDB—store semi-structured documents (JSON/BSON). Flexible schemas, good for hierarchical data and rapid iteration. Key-Value Stores: Redis, DynamoDB, etcd—simple get/set operations, extremely fast. Used for caching, session storage, and high-throughput simple lookups. Wide-Column Stores: Cassandra, HBase, ScyllaDB—designed for massive scale, high write throughput, and geographical distribution. Trade query flexibility for scalability. Graph Databases: Neo4j, Amazon Neptune—optimized for relationship-heavy data and traversing connections. Ideal for social networks, recommendation engines, fraud detection. Time-Series Databases: InfluxDB, TimescaleDB, Prometheus—optimized for time-stamped data with high ingest rates and time-based queries. Used for metrics, IoT, and financial data.

Database Categories and Trade-offs
Category	Data Model	Query Power	Scaling	Best For
Relational	Tables, rows, columns	Very high (SQL)	Challenging to scale writes	Complex queries, transactions, structured data
Document	Nested documents	Moderate (queries)	Good horizontal scaling	Content, catalogs, schema evolution
Key-Value	Key → Value	Limited (by key only)	Excellent scaling	Cache, session, simple lookups
Wide-Column	Column families	Moderate	Excellent, distributed	Time-series at scale, logs, analytics
Graph	Nodes and edges	Relationship queries	Moderate	Connections, networks, recommendations
Time-Series	Time-indexed data	Time-based queries	Good for append	Metrics, IoT, financial ticks

Application-Database Communication: Connection Pooling: Opening database connections is expensive. Connection pools maintain a set of pre-established connections that application servers reuse, dramatically reducing connection overhead. Query Patterns: OLTP (Online Transaction Processing) : Many small, fast queries—typical for web applications OLAP (Online Analytical Processing) : Complex analytical queries over large datasets—typical for data warehouses ORMs and Query Builders: Application code often uses Object-Relational Mappers (Prisma, TypeORM, SQLAlchemy, Hibernate) or query builders that abstract database interaction behind programming language constructs.

database-interaction-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
// Database Interaction Patterns
 
// Pattern 1: Connection Pool Configuration
import { Pool } from 'pg';
 
const pool = new Pool({
  host: 'db.example.com',
  database: 'production',
  user: 'app_user',
  password: process.env.DB_PASSWORD,
  port: 5432,
  max: 20,                  // Maximum connections in pool
  idleTimeoutMillis: 30000, // Close idle connections after 30s
  connectionTimeoutMillis: 10000, // Fail if can't connect in 10s
});
 
// Pattern 2: Transaction Management
async function transferFunds(fromId: string, toId: string, amount: number) {
  const client = await pool.connect();
  
  try {
    await client.query('BEGIN');
    
    // Debit source account
    await client.query(
      'UPDATE accounts SET balance = balance - $1 WHERE id = $2',
      [amount, fromId]
    );
    
    // Credit destination account
    await client.query(
      'UPDATE accounts SET balance = balance + $1 WHERE id = $2',
      [amount, toId]
    );
    
    await client.query('COMMIT');
  } catch (error) {
    await client.query('ROLLBACK');
    throw error;
  } finally {
    client.release(); // Return connection to pool
  }
}
 
// Pattern 3: ORM Usage (Prisma example)
const order = await prisma.order.create({
  data: {
    customerId: 'cust_123',
    status: 'pending',
    items: {
      create: [
        { productId: 'prod_456', quantity: 2 },
        { productId: 'prod_789', quantity: 1 },
      ],
    },
  },
  include: {
    items: { include: { product: true } },
    customer: true,
  },
});

Databases Are Often the Bottleneck

While application servers scale horizontally with ease, databases are harder to scale. They hold state, require consistency, and face write contention. Many performance issues trace to the database: missing indexes, N+1 queries, lock contention, or insufficient read replicas. Master database optimization.

Cache Servers: The Performance Accelerator

Cache servers store frequently-accessed data in memory for ultra-fast retrieval. By reducing repeated trips to slower backends (databases, external APIs, expensive computations), caches dramatically improve response times and reduce load on primary data stores. Why Caching is Essential: Consider the performance difference: Memory access : ~100 nanoseconds SSD read : ~150 microseconds (1500x slower) Database query : ~10 milliseconds (100,000x slower than memory) External API call : ~100 milliseconds (1,000,000x slower than memory) By keeping hot data in memory, caches can reduce response times from hundreds of milliseconds to single-digit milliseconds.

Cache Server Capabilities

•In-Memory Storage — Data lives in RAM for nanosecond-level access. No disk I/O for reads (though some caches persist to disk for durability).
•Key-Value Operations — Simple get/set/delete by key. O(1) operations enable consistent performance regardless of data volume.
•TTL (Time-To-Live) — Automatic expiration of cached data after a configured duration, ensuring data freshness.
•Eviction Policies — When cache is full, decide what to remove: LRU (Least Recently Used), LFU (Least Frequently Used), or random.
•Data Structures — Advanced caches like Redis support lists, sets, sorted sets, hashes, and streams—enabling complex operations beyond simple key-value.
•Pub/Sub Messaging — Some caches provide publish/subscribe capabilities for real-time messaging between services.

Major Cache Server Technologies: Redis: The Swiss Army knife of caching. Supports rich data structures (strings, lists, sets, sorted sets, hashes), persistence options (RDB snapshots, AOF logging), replication, and clustering. Used for caching, session storage, rate limiting, leaderboards, and messaging. Memcached: Simpler, focused on pure key-value caching. Highly performant for basic caching needs, multi-threaded, and easy to scale horizontally. Less feature-rich than Redis but sometimes faster for simple use cases. Comparison:

Redis vs Memcached Comparison
Aspect	Redis	Memcached
Data structures	Rich (lists, sets, sorted sets, hashes)	Simple key-value only
Persistence	Optional (RDB, AOF)	None (pure cache)
Replication	Built-in master-replica	Via external tools
Clustering	Native (Redis Cluster)	Via client-side sharding
Memory efficiency	Moderate	Higher for simple values
Threading	Single-threaded (I/O threads in 6.0+)	Multi-threaded
Use cases	Caching, sessions, queues, pub/sub	Pure caching

Caching Patterns: Cache-Aside (Lazy Loading): Application checks cache first On cache miss, read from database Store result in cache for future requests Write-Through: Application writes to cache Cache writes to database synchronously Ensures cache consistency but adds write latency Write-Behind (Write-Back): Application writes to cache Cache asynchronously writes to database later Fast writes but risk of data loss if cache fails Read-Through: Application reads from cache Cache fetches from database on miss (transparently) Simplifies application code but cache must understand data source

caching-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
// Caching Pattern Implementations
 
import Redis from 'ioredis';
const redis = new Redis({ host: 'cache.example.com', port: 6379 });
 
// Cache-Aside Pattern (most common)
async function getUserById(userId: string): Promise<User> {
  const cacheKey = `user:${userId}`;
  
  // 1. Check cache first
  const cached = await redis.get(cacheKey);
  if (cached) {
    return JSON.parse(cached);
  }
  
  // 2. Cache miss: fetch from database
  const user = await database.users.findById(userId);
  if (!user) {
    throw new NotFoundError('User not found');
  }
  
  // 3. Store in cache for next time (5 minute TTL)
  await redis.setex(cacheKey, 300, JSON.stringify(user));
  
  return user;
}
 
// Cache Invalidation on Update
async function updateUser(userId: string, updates: Partial<User>): Promise<User> {
  // 1. Update database
  const user = await database.users.update(userId, updates);
  
  // 2. Invalidate cache (next read will refresh)
  await redis.del(`user:${userId}`);
  
  // Or: update cache with new value
  // await redis.setex(`user:${userId}`, 300, JSON.stringify(user));
  
  return user;
}
 
// Using Redis Data Structures for Rate Limiting
async function checkRateLimit(userId: string, limit: number = 100): Promise<boolean> {
  const key = `ratelimit:${userId}:${getCurrentMinute()}`;
  
  // Use Redis INCR + EXPIRE atomically
  const count = await redis.incr(key);
  if (count === 1) {
    await redis.expire(key, 60); // Expire after 1 minute
  }
  
  return count <= limit;
}
 
// Using Redis Sorted Sets for Leaderboards
async function updateLeaderboard(playerId: string, score: number): Promise<void> {
  await redis.zadd('game:leaderboard', score, playerId);
}
 
async function getTopPlayers(count: number = 10): Promise<Array<{id: string, score: number}>> {
  // Get top players with scores (highest first)
  const results = await redis.zrevrange('game:leaderboard', 0, count - 1, 'WITHSCORES');
  
  // Parse alternating [id, score, id, score...] format
  const players: Array<{id: string, score: number}> = [];
  for (let i = 0; i < results.length; i += 2) {
    players.push({ id: results[i], score: parseInt(results[i + 1]) });
  }
  return players;
}

Cache Invalidation is Hard

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton. Cached data can become stale, leading to users seeing outdated information. Design your invalidation strategy carefully: time-based expiration (TTL), event-driven invalidation, or versioned cache keys.

The Three-Tier Architecture: Putting It Together

Application servers, database servers, and cache servers combine to form the classic three-tier architecture —a foundational pattern for web applications that separates concerns into distinct layers. Tier 1: Presentation Layer The user interface—web browsers, mobile apps, or API consumers. Clients that interact with end users and communicate with the application tier. Tier 2: Application Layer (Logic Layer) Application servers executing business logic, authentication, request processing, and orchestration. Stateless and horizontally scalable. Tier 3: Data Layer Database servers for persistence, cache servers for performance, and any other data stores. Stateful, often the most operationally complex tier.

three-tier-architecture.md
┌─────────────────────────────────────────────────────────────────────────┐
│                      TIER 1: PRESENTATION LAYER                         │
│                                                                         │
│   ┌─────────────┐   ┌─────────────┐   ┌─────────────┐                  │
│   │  Web Browser│   │  Mobile App │   │  API Client │                  │
│   │   (React)   │   │ (iOS/Android│   │  (External) │                  │
│   └──────┬──────┘   └──────┬──────┘   └──────┬──────┘                  │
│          │                 │                 │                          │
└──────────┼─────────────────┼─────────────────┼──────────────────────────┘
           │                 │                 │
           └────────┬────────┴─────────────────┘
                    │ HTTPS/REST/GraphQL
                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                LOAD BALANCER / API GATEWAY                               │
│              (NGINX, AWS ALB, Kong, etc.)                               │
└─────────────────────────────────┬───────────────────────────────────────┘
                                  │
                    ┌─────────────┴─────────────┐
                    ▼                           ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                    TIER 2: APPLICATION LAYER                             │
│                                                                         │
│   ┌─────────────────┐   ┌─────────────────┐   ┌─────────────────┐      │
│   │ App Server 1    │   │ App Server 2    │   │ App Server N    │      │
│   │ (Node/Express)  │   │ (Node/Express)  │   │ (Node/Express)  │      │
│   │ • Business Logic│   │ • Business Logic│   │ • Business Logic│      │
│   │ • Auth/Authz    │   │ • Auth/Authz    │   │ • Auth/Authz    │      │
│   │ • API Handlers  │   │ • API Handlers  │   │ • API Handlers  │      │
│   └────────┬────────┘   └────────┬────────┘   └────────┬────────┘      │
│            │                     │                     │                │
└────────────┼─────────────────────┼─────────────────────┼────────────────┘
             │                     │                     │
             └──────────┬──────────┴─────────────────────┘
                        │
           ┌────────────┴────────────┐
           ▼                         ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                      TIER 3: DATA LAYER                                  │
│                                                                         │
│   ┌─────────────────────────────┐   ┌─────────────────────────────┐    │
│   │        CACHE CLUSTER        │   │      DATABASE CLUSTER       │    │
│   │  ┌───────┐     ┌───────┐    │   │  ┌───────┐     ┌───────┐   │    │
│   │  │ Redis │     │ Redis │    │   │  │Primary│     │Replica│   │    │
│   │  │Primary│────▶│Replica│    │   │  │(Write)│────▶│(Read) │   │    │
│   │  └───────┘     └───────┘    │   │  └───────┘     └───────┘   │    │
│   │                             │   │                             │    │
│   │  • Hot data caching         │   │  • Persistent storage       │    │
│   │  • Session storage          │   │  • Transactions             │    │
│   │  • Rate limiting            │   │  • Complex queries          │    │
│   └─────────────────────────────┘   └─────────────────────────────┘    │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

Request Flow Through Three-Tier: Client Request : Browser sends GET /api/products/123 Load Balancer : Routes request to an available app server Application Server : Receives request, authenticates user Cache Check : App server queries cache for product:123 Cache Hit : Return cached data (fast path) Cache Miss : Continue to database Database Query : App server queries database for product Cache Update : Store result in cache for future requests Response : App server formats and returns response to client Benefits of Three-Tier: Separation of Concerns : Each tier specializes in specific responsibilities Independent Scaling : Scale app servers without touching database Technology Flexibility : Change database vendor without affecting app servers Security Boundaries : Database isn't directly accessible from internet Team Organization : Different teams can own different tiers

Three-Tier is a Starting Point

Three-tier works well for many applications but isn't the only pattern. As systems grow, you might add CDNs (presentation layer caching), message queues (async processing), search services (Elasticsearch), or decompose into microservices. Start with three-tier; evolve based on need.

Server Infrastructure: Scaling and Reliability

Running production servers requires more than just deploying code. You must address infrastructure concerns that determine whether your system stays up under load, recovers from failures, and remains secure. Horizontal vs. Vertical Scaling: Vertical Scaling (Scale Up) : Add more resources to existing servers—more CPU, RAM, faster disks. Simple but limited: eventually you hit the largest available machine, and you have a single point of failure. Horizontal Scaling (Scale Out) : Add more server instances. Requires stateless design (for app servers) or distributed data systems (for databases). More complex but provides better availability and can scale nearly infinitely.

Scaling Strategy by Server Type
Server Type	Primary Scaling Strategy	Challenges	Common Solutions
Application Servers	Horizontal (add instances)	Session state, warm-up	Stateless design, load balancers
Database (Reads)	Horizontal (read replicas)	Replication lag, routing	Replica sets, read-write splitting
Database (Writes)	Vertical first, then sharding	Sharding complexity, joins	Sharding, distributed databases
Cache Servers	Horizontal (cluster)	Key distribution, hot keys	Consistent hashing, clustering

High Availability Patterns: Redundancy: Run multiple instances of every component. No single point of failure. If one app server fails, others continue serving requests. Health Checks: Load balancers continuously check server health (HTTP endpoint, TCP connection). Unhealthy servers are removed from rotation. Failover: Automatic switching from failed primary to healthy standby. Database primary fails? Replica is promoted. Cache fails? App falls back to database. Distribution: Spread servers across multiple availability zones (data centers). If one zone fails, others continue operating.

Production Infrastructure Checklist

•Monitoring & Alerting — Metrics (CPU, memory, latency), logs, and traces. Alert on anomalies before users notice.
•Auto-Scaling — Automatically add/remove instances based on CPU, memory, queue depth, or request rate.
•Deployment Strategy — Blue-green, canary, or rolling deployments to minimize downtime and enable rollbacks.
•Configuration Management — Environment variables, secrets management (Vault, AWS Secrets Manager), feature flags.
•Backup & Recovery — Regular database backups, tested restore procedures, point-in-time recovery.
•Security Hardening — Network segmentation, minimal permissions, encrypted data at rest and in transit.
•Capacity Planning — Understand limits, plan for peak load, and test capacity before you need it.

Cloud vs. Self-Hosted

Cloud platforms (AWS, GCP, Azure) provide managed versions of all server types: RDS for databases, ElastiCache for Redis, ECS/EKS for application servers. Managed services reduce operational burden but increase cost and may limit flexibility. Many organizations use a mix: managed databases for reliability, self-managed app servers for control.

Connection Management Between Tiers

Communication between servers—app to database, app to cache, service to service—requires careful connection management. Poorly managed connections are a common source of outages, resource exhaustion, and performance degradation. Connection Pools: Opening network connections involves TCP handshakes, TLS negotiation, and protocol initialization—operations that take milliseconds. Connection pools maintain a set of pre-established connections that are reused across requests. Key Pool Parameters: Maximum Size : Upper limit on connections. Too low = contention; too high = resource exhaustion on server. Minimum/Idle Size : Keep some connections warm to avoid cold-start latency. Connection Timeout : How long to wait for a connection from the pool. Idle Timeout : How long unused connections stay open before closing. Max Lifetime : Maximum age of a connection (handles server-side connection limits).

connection-pool-config.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
// Connection Pool Configuration Examples
 
// PostgreSQL Connection Pool
import { Pool } from 'pg';
 
const databasePool = new Pool({
  host: 'db.example.com',
  database: 'production',
  user: 'app_user',
  password: process.env.DB_PASSWORD,
  
  max: 20,                  // Max connections (tune based on load testing)
  min: 5,                   // Keep at least 5 connections warm
  idleTimeoutMillis: 30000, // Close idle connections after 30s
  connectionTimeoutMillis: 10000, // Fail if can't get connection in 10s
  
  // Health checking
  allowExitOnIdle: false,
  keepAlive: true,
  keepAliveInitialDelayMillis: 10000,
});
 
// Monitor pool health
databasePool.on('connect', () => {
  metrics.increment('db.pool.connection_opened');
});
databasePool.on('remove', () => {
  metrics.increment('db.pool.connection_closed');
});
databasePool.on('error', (err) => {
  logger.error('Database pool error', { error: err.message });
  metrics.increment('db.pool.errors');
});
 
// Redis Connection Pool (via ioredis)
import Redis from 'ioredis';
 
const redisCluster = new Redis.Cluster([
  { host: 'redis-1.example.com', port: 6379 },
  { host: 'redis-2.example.com', port: 6379 },
  { host: 'redis-3.example.com', port: 6379 },
], {
  redisOptions: {
    password: process.env.REDIS_PASSWORD,
    connectTimeout: 10000,
    commandTimeout: 5000,
    retryStrategy(times) {
      return Math.min(times * 50, 2000); // Exponential backoff
    },
  },
  clusterRetryStrategy(times) {
    return Math.min(times * 100, 3000);
  },
  enableReadyCheck: true,
  maxRedirections: 16,
  scaleReads: 'slave', // Read from replicas
});
 
// HTTP Connection Pool for External APIs
import axios from 'axios';
import https from 'https';
 
const externalApiClient = axios.create({
  baseURL: 'https://api.external-service.com',
  timeout: 5000,
  
  // Connection pool via Node.js agent
  httpsAgent: new https.Agent({
    keepAlive: true,
    maxSockets: 50,       // Max connections to this host
    maxFreeSockets: 10,   // Keep 10 idle connections
    timeout: 60000,       // Socket timeout
  }),
});

Connection Leaks: A connection leak occurs when code acquires a connection but never releases it. Over time, the pool is exhausted and new requests fail. Prevent leaks with: Try/finally blocks : Always release in finally Using statements : Language features that auto-release (C#'s using, Python's with) Timeouts : Automatically reclaim connections held too long Monitoring : Alert when pool utilization is consistently high Circuit Breakers: If a downstream server is failing, continuing to attempt connections wastes resources and increases latency. Circuit breakers 'trip' after a threshold of failures, failing fast without attempting connections until the circuit 'resets' (after a timeout or when health checks pass).

Size Your Pools Carefully

If you have 10 app servers, each with 20 database connections, your database sees 200 connections. Many databases have connection limits (e.g., RDS default is 150-500 depending on instance size). More connections aren't always better—context switching between many connections reduces performance. Start conservative and tune based on metrics.

Choosing the Right Server Technologies

Each tier offers multiple technology options. Selection depends on your requirements, team expertise, operational constraints, and expected scale. Here are key decision factors for each server type. Application Server Selection: Language/Framework Expertise : Use what your team knows. Node.js, Python/Django, Java/Spring, Go, Ruby/Rails are all proven at scale. Performance Profile : CPU-bound work? Consider Go or Java. I/O-bound? Node.js or async Python excel. Ecosystem : Libraries, frameworks, and community resources matter for development velocity. Operational Maturity : How mature are deployment, monitoring, and debugging tools in your stack?

Application Server Technology Trade-offs
Technology	Strengths	Weaknesses	Best For
Node.js (Express)	I/O efficiency, JS ecosystem, developer pool	CPU-bound work, type safety (without TS)	APIs, real-time apps, BFFs
Python (FastAPI)	Rapid development, ML/data ecosystem	Performance vs compiled langs	Data-heavy APIs, ML backends
Java (Spring)	Enterprise features, JVM performance, tooling	Verbosity, memory usage, cold start	Enterprise, high-throughput systems
Go	Performance, concurrency, small binaries	Smaller ecosystem, error handling verbosity	Microservices, infrastructure
Ruby (Rails)	Developer productivity, conventions	Performance, scaling challenges	Startups, MVPs, content sites

Database Selection: Data Model : Relational data with complex queries? PostgreSQL. Documents with flexible schema? MongoDB. Simple key lookups at scale? DynamoDB. Consistency Requirements : Need ACID transactions? Relational. Can tolerate eventual consistency? NoSQL options open up. Scale Pattern : Read-heavy? Any database with replicas. Write-heavy at massive scale? Consider Cassandra, DynamoDB. Operational Complexity : Managed (RDS, Cosmos DB) vs. self-managed. Managed reduces burden but limits control. Cache Selection: Feature Needs : Just caching? Memcached is simple. Need data structures, pub/sub, persistence? Redis. Cluster Support : Redis Cluster is mature. Memcached sharding is client-side. Memory Efficiency : For pure string caching, Memcached may use less memory. Managed Options : ElastiCache (AWS), MemoryStore (GCP), Azure Cache reduce operational burden.

Technology Selection Principles

•Boring Technology: Prefer proven, widely-used technologies over new shiny ones. PostgreSQL, Redis, and mature frameworks have battle-tested reliability.
•Minimize Diversity: Each technology is operational overhead. Don't use three different databases if one suffices. Reduce cognitive load.
•Consider the Future: Will you need full-text search? Consider Postgres vs adding Elasticsearch later. Think ahead but don't over-engineer.
•Team Capability: The best technology is one your team can operate well. A MongoDB expert team may outperform a PostgreSQL novice team with PostgreSQL.
•Migration Path: If you choose wrong, how hard is migration? Early choices are easier to reverse than late ones.

Start with Defaults

When in doubt: PostgreSQL for persistent data, Redis for caching/sessions, and your language's most popular web framework. These defaults work for the vast majority of applications. Only deviate when you have clear requirements that justify complexity.

System DesignClient-Server Model

Client-Server Model

LevelBeginner

Duration60 mins

TopicClient-Server Model

4 / 4

Servers: Application, Database, Cache

The Layered Ecosystem That Powers Every Application

When we speak of 'the server' in client-server architecture, we often imagine a single entity—a box that receives requests and returns responses. But in reality, modern server-side infrastructure is a layered ecosystem of specialized components, each optimized for specific responsibilities. A single API request might traverse: A load balancer distributing traffic A CDN edge node serving cached content An application server executing business logic A cache server providing frequently-accessed data A database server for persistent storage A message queue for asynchronous processing Understanding this ecosystem—the role of each layer, how they interact, and when to use each—is fundamental to designing systems that are fast, reliable, and scalable. This page explores the three core server types that form the backbone of virtually every system: application servers, database servers, and cache servers .

What You Will Learn

By the end of this page, you will understand the role and responsibilities of application servers, the architecture and trade-offs of database servers, and why cache servers are essential for performance. You'll see how these layers work together in modern three-tier and multi-tier architectures.

Application Servers: The Business Logic Layer

Application servers are the core of your system—the component that receives client requests, executes business logic, orchestrates data access, and returns responses. They embody the 'what your application does' aspect of your system. Core Responsibilities of Application Servers:

Application Server Responsibilities

•Request Handling — Accepting incoming HTTP/gRPC/WebSocket connections, parsing requests, and routing them to appropriate handlers based on URL, method, and headers.
•Authentication & Authorization — Verifying client identity, validating credentials, and enforcing access control policies before processing requests.
•Business Logic Execution — Implementing domain-specific rules, calculations, validations, and workflows that define your application's behavior.
•Data Orchestration — Coordinating data access across databases, caches, external APIs, and other services to fulfill requests.
•Response Generation — Formatting responses, serializing data to appropriate formats (JSON, Protobuf, HTML), and sending them back to clients.
•Cross-Cutting Concerns — Logging, metrics collection, distributed tracing, rate limiting, and error handling that span all requests.

Types of Application Servers: Web Servers (HTTP-focused): NGINX, Apache, Caddy—primarily handle HTTP traffic, serve static files, and proxy requests to application backends. Often sit in front of application servers as reverse proxies. Language-Specific Application Servers: Node.js : Express, Fastify, NestJS Python : Gunicorn, uWSGI, Uvicorn (for Django, Flask, FastAPI) Java : Tomcat, Jetty, Undertow (for Spring, Jakarta EE) Go : net/http, Gin, Echo (compiled into the application) Ruby : Puma, Unicorn (for Rails) API Gateways: Kong, AWS API Gateway, Apigee—specialized application servers that handle API-specific concerns: authentication, rate limiting, transformation, and routing to backend services.

Application Server Architectures
Architecture	Description	Pros	Cons	Use Cases
Single-threaded event loop	One thread handles many connections via async I/O	Efficient for I/O-bound work, simple	CPU-bound work blocks	Node.js, NGINX worker
Multi-threaded	Thread pool handles concurrent requests	CPU parallelism, isolation	Context switching overhead	Java servlets, traditional
Process per request	Fork new process for each request	Isolation, stability	High overhead	Legacy CGI, some PHP
Worker pool	Pool of workers (processes/threads)	Load distribution, resource limits	Pool sizing complexity	Gunicorn, PM2, Puma
Actor-based	Actors process messages asynchronously	Scalable, fault-tolerant	Complexity	Akka, Erlang/OTP

application-server-layers.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
// Modern Application Server Architecture (Express/Node.js Example)
 
import express, { Request, Response, NextFunction } from 'express';
import { createLogger, transports, format } from 'winston';
import { rateLimit } from 'express-rate-limit';
import { authenticateRequest } from './auth';
import { metricsMiddleware } from './observability';
import { errorHandler } from './errors';
 
const app = express();
 
// Layer 1: Request Parsing
app.use(express.json({ limit: '10mb' }));
app.use(express.urlencoded({ extended: true }));
 
// Layer 2: Observability
app.use(metricsMiddleware());
app.use((req, res, next) => {
  req.correlationId = req.headers['x-correlation-id'] || generateUUID();
  res.setHeader('X-Correlation-ID', req.correlationId);
  next();
});
 
// Layer 3: Security
app.use(rateLimit({ windowMs: 60000, max: 100 })); // 100 req/min
app.use('/api', authenticateRequest);
 
// Layer 4: Routing to Business Logic
app.use('/api/v1/users', userRouter);      // User domain
app.use('/api/v1/orders', orderRouter);    // Order domain
app.use('/api/v1/products', productRouter);// Product domain
 
// Layer 5: Error Handling
app.use(errorHandler);
 
// Business Logic Handler Example
async function createOrder(req: Request, res: Response) {
  const { customerId, items } = req.body;
  
  // Validate input
  validateOrderInput(customerId, items);
  
  // Business logic: check inventory, calculate total
  const inventoryCheck = await inventoryService.check(items);
  if (!inventoryCheck.available) {
    throw new InsufficientInventoryError(inventoryCheck.unavailable);
  }
  
  // Data orchestration: save to database
  const order = await orderRepository.create({
    customerId,
    items,
    total: calculateTotal(items),
    status: 'pending'
  });
  
  // Async side effects: emit event for downstream processing
  await eventBus.publish('order.created', { orderId: order.id });
  
  // Response
  res.status(201).json(order);
}

Stateless Application Servers

Modern application servers are designed to be stateless—they don't store session data in memory. This enables horizontal scaling (add more servers), load balancer freedom (any server can handle any request), and zero-downtime deployments (replace servers without losing state). State is externalized to databases, caches, and session stores.

Database Servers: The Persistence Layer

Database servers are specialized systems optimized for storing, querying, and managing data. They provide durability (data survives restarts), consistency (transactions and constraints), and efficient access (indexes and query optimization). Unlike application servers that are often stateless, database servers are inherently stateful —they own and persist your application's most valuable asset: its data. Core Responsibilities of Database Servers:

Database Server Responsibilities

•Data Persistence — Writing data to durable storage (disk, SSDs) that survives process restarts, server reboots, and hardware failures.
•Query Processing — Parsing, optimizing, and executing queries. The query optimizer is one of the most complex components, choosing efficient execution plans.
•Transaction Management — Ensuring ACID properties (Atomicity, Consistency, Isolation, Durability) for operations that must succeed or fail as a unit.
•Concurrency Control — Managing simultaneous access from multiple clients without data corruption. Locking, MVCC, and isolation levels balance consistency and performance.
•Data Integrity — Enforcing constraints (primary keys, foreign keys, unique, not null, check) that maintain data validity.
•Index Management — Building and maintaining indexes that accelerate query performance, balancing read speed against write overhead.

Categories of Database Servers: Relational Databases (SQL): PostgreSQL, MySQL, Oracle, SQL Server—organize data in tables with defined schemas and relationships. Excel at complex queries, joins, and transactions. Document Databases: MongoDB, CouchDB—store semi-structured documents (JSON/BSON). Flexible schemas, good for hierarchical data and rapid iteration. Key-Value Stores: Redis, DynamoDB, etcd—simple get/set operations, extremely fast. Used for caching, session storage, and high-throughput simple lookups. Wide-Column Stores: Cassandra, HBase, ScyllaDB—designed for massive scale, high write throughput, and geographical distribution. Trade query flexibility for scalability. Graph Databases: Neo4j, Amazon Neptune—optimized for relationship-heavy data and traversing connections. Ideal for social networks, recommendation engines, fraud detection. Time-Series Databases: InfluxDB, TimescaleDB, Prometheus—optimized for time-stamped data with high ingest rates and time-based queries. Used for metrics, IoT, and financial data.

Database Categories and Trade-offs
Category	Data Model	Query Power	Scaling	Best For
Relational	Tables, rows, columns	Very high (SQL)	Challenging to scale writes	Complex queries, transactions, structured data
Document	Nested documents	Moderate (queries)	Good horizontal scaling	Content, catalogs, schema evolution
Key-Value	Key → Value	Limited (by key only)	Excellent scaling	Cache, session, simple lookups
Wide-Column	Column families	Moderate	Excellent, distributed	Time-series at scale, logs, analytics
Graph	Nodes and edges	Relationship queries	Moderate	Connections, networks, recommendations
Time-Series	Time-indexed data	Time-based queries	Good for append	Metrics, IoT, financial ticks

Application-Database Communication: Connection Pooling: Opening database connections is expensive. Connection pools maintain a set of pre-established connections that application servers reuse, dramatically reducing connection overhead. Query Patterns: OLTP (Online Transaction Processing) : Many small, fast queries—typical for web applications OLAP (Online Analytical Processing) : Complex analytical queries over large datasets—typical for data warehouses ORMs and Query Builders: Application code often uses Object-Relational Mappers (Prisma, TypeORM, SQLAlchemy, Hibernate) or query builders that abstract database interaction behind programming language constructs.

database-interaction-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
// Database Interaction Patterns
 
// Pattern 1: Connection Pool Configuration
import { Pool } from 'pg';
 
const pool = new Pool({
  host: 'db.example.com',
  database: 'production',
  user: 'app_user',
  password: process.env.DB_PASSWORD,
  port: 5432,
  max: 20,                  // Maximum connections in pool
  idleTimeoutMillis: 30000, // Close idle connections after 30s
  connectionTimeoutMillis: 10000, // Fail if can't connect in 10s
});
 
// Pattern 2: Transaction Management
async function transferFunds(fromId: string, toId: string, amount: number) {
  const client = await pool.connect();
  
  try {
    await client.query('BEGIN');
    
    // Debit source account
    await client.query(
      'UPDATE accounts SET balance = balance - $1 WHERE id = $2',
      [amount, fromId]
    );
    
    // Credit destination account
    await client.query(
      'UPDATE accounts SET balance = balance + $1 WHERE id = $2',
      [amount, toId]
    );
    
    await client.query('COMMIT');
  } catch (error) {
    await client.query('ROLLBACK');
    throw error;
  } finally {
    client.release(); // Return connection to pool
  }
}
 
// Pattern 3: ORM Usage (Prisma example)
const order = await prisma.order.create({
  data: {
    customerId: 'cust_123',
    status: 'pending',
    items: {
      create: [
        { productId: 'prod_456', quantity: 2 },
        { productId: 'prod_789', quantity: 1 },
      ],
    },
  },
  include: {
    items: { include: { product: true } },
    customer: true,
  },
});

Databases Are Often the Bottleneck

While application servers scale horizontally with ease, databases are harder to scale. They hold state, require consistency, and face write contention. Many performance issues trace to the database: missing indexes, N+1 queries, lock contention, or insufficient read replicas. Master database optimization.

Cache Servers: The Performance Accelerator

Cache servers store frequently-accessed data in memory for ultra-fast retrieval. By reducing repeated trips to slower backends (databases, external APIs, expensive computations), caches dramatically improve response times and reduce load on primary data stores. Why Caching is Essential: Consider the performance difference: Memory access : ~100 nanoseconds SSD read : ~150 microseconds (1500x slower) Database query : ~10 milliseconds (100,000x slower than memory) External API call : ~100 milliseconds (1,000,000x slower than memory) By keeping hot data in memory, caches can reduce response times from hundreds of milliseconds to single-digit milliseconds.

Cache Server Capabilities

•In-Memory Storage — Data lives in RAM for nanosecond-level access. No disk I/O for reads (though some caches persist to disk for durability).
•Key-Value Operations — Simple get/set/delete by key. O(1) operations enable consistent performance regardless of data volume.
•TTL (Time-To-Live) — Automatic expiration of cached data after a configured duration, ensuring data freshness.
•Eviction Policies — When cache is full, decide what to remove: LRU (Least Recently Used), LFU (Least Frequently Used), or random.
•Data Structures — Advanced caches like Redis support lists, sets, sorted sets, hashes, and streams—enabling complex operations beyond simple key-value.
•Pub/Sub Messaging — Some caches provide publish/subscribe capabilities for real-time messaging between services.

Major Cache Server Technologies: Redis: The Swiss Army knife of caching. Supports rich data structures (strings, lists, sets, sorted sets, hashes), persistence options (RDB snapshots, AOF logging), replication, and clustering. Used for caching, session storage, rate limiting, leaderboards, and messaging. Memcached: Simpler, focused on pure key-value caching. Highly performant for basic caching needs, multi-threaded, and easy to scale horizontally. Less feature-rich than Redis but sometimes faster for simple use cases. Comparison:

Redis vs Memcached Comparison
Aspect	Redis	Memcached
Data structures	Rich (lists, sets, sorted sets, hashes)	Simple key-value only
Persistence	Optional (RDB, AOF)	None (pure cache)
Replication	Built-in master-replica	Via external tools
Clustering	Native (Redis Cluster)	Via client-side sharding
Memory efficiency	Moderate	Higher for simple values
Threading	Single-threaded (I/O threads in 6.0+)	Multi-threaded
Use cases	Caching, sessions, queues, pub/sub	Pure caching

Caching Patterns: Cache-Aside (Lazy Loading): Application checks cache first On cache miss, read from database Store result in cache for future requests Write-Through: Application writes to cache Cache writes to database synchronously Ensures cache consistency but adds write latency Write-Behind (Write-Back): Application writes to cache Cache asynchronously writes to database later Fast writes but risk of data loss if cache fails Read-Through: Application reads from cache Cache fetches from database on miss (transparently) Simplifies application code but cache must understand data source

caching-patterns.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
// Caching Pattern Implementations
 
import Redis from 'ioredis';
const redis = new Redis({ host: 'cache.example.com', port: 6379 });
 
// Cache-Aside Pattern (most common)
async function getUserById(userId: string): Promise<User> {
  const cacheKey = `user:${userId}`;
  
  // 1. Check cache first
  const cached = await redis.get(cacheKey);
  if (cached) {
    return JSON.parse(cached);
  }
  
  // 2. Cache miss: fetch from database
  const user = await database.users.findById(userId);
  if (!user) {
    throw new NotFoundError('User not found');
  }
  
  // 3. Store in cache for next time (5 minute TTL)
  await redis.setex(cacheKey, 300, JSON.stringify(user));
  
  return user;
}
 
// Cache Invalidation on Update
async function updateUser(userId: string, updates: Partial<User>): Promise<User> {
  // 1. Update database
  const user = await database.users.update(userId, updates);
  
  // 2. Invalidate cache (next read will refresh)
  await redis.del(`user:${userId}`);
  
  // Or: update cache with new value
  // await redis.setex(`user:${userId}`, 300, JSON.stringify(user));
  
  return user;
}
 
// Using Redis Data Structures for Rate Limiting
async function checkRateLimit(userId: string, limit: number = 100): Promise<boolean> {
  const key = `ratelimit:${userId}:${getCurrentMinute()}`;
  
  // Use Redis INCR + EXPIRE atomically
  const count = await redis.incr(key);
  if (count === 1) {
    await redis.expire(key, 60); // Expire after 1 minute
  }
  
  return count <= limit;
}
 
// Using Redis Sorted Sets for Leaderboards
async function updateLeaderboard(playerId: string, score: number): Promise<void> {
  await redis.zadd('game:leaderboard', score, playerId);
}
 
async function getTopPlayers(count: number = 10): Promise<Array<{id: string, score: number}>> {
  // Get top players with scores (highest first)
  const results = await redis.zrevrange('game:leaderboard', 0, count - 1, 'WITHSCORES');
  
  // Parse alternating [id, score, id, score...] format
  const players: Array<{id: string, score: number}> = [];
  for (let i = 0; i < results.length; i += 2) {
    players.push({ id: results[i], score: parseInt(results[i + 1]) });
  }
  return players;
}

Cache Invalidation is Hard

"There are only two hard things in Computer Science: cache invalidation and naming things." — Phil Karlton. Cached data can become stale, leading to users seeing outdated information. Design your invalidation strategy carefully: time-based expiration (TTL), event-driven invalidation, or versioned cache keys.

The Three-Tier Architecture: Putting It Together

Application servers, database servers, and cache servers combine to form the classic three-tier architecture —a foundational pattern for web applications that separates concerns into distinct layers. Tier 1: Presentation Layer The user interface—web browsers, mobile apps, or API consumers. Clients that interact with end users and communicate with the application tier. Tier 2: Application Layer (Logic Layer) Application servers executing business logic, authentication, request processing, and orchestration. Stateless and horizontally scalable. Tier 3: Data Layer Database servers for persistence, cache servers for performance, and any other data stores. Stateful, often the most operationally complex tier.

three-tier-architecture.md
┌─────────────────────────────────────────────────────────────────────────┐
│                      TIER 1: PRESENTATION LAYER                         │
│                                                                         │
│   ┌─────────────┐   ┌─────────────┐   ┌─────────────┐                  │
│   │  Web Browser│   │  Mobile App │   │  API Client │                  │
│   │   (React)   │   │ (iOS/Android│   │  (External) │                  │
│   └──────┬──────┘   └──────┬──────┘   └──────┬──────┘                  │
│          │                 │                 │                          │
└──────────┼─────────────────┼─────────────────┼──────────────────────────┘
           │                 │                 │
           └────────┬────────┴─────────────────┘
                    │ HTTPS/REST/GraphQL
                    ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                LOAD BALANCER / API GATEWAY                               │
│              (NGINX, AWS ALB, Kong, etc.)                               │
└─────────────────────────────────┬───────────────────────────────────────┘
                                  │
                    ┌─────────────┴─────────────┐
                    ▼                           ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                    TIER 2: APPLICATION LAYER                             │
│                                                                         │
│   ┌─────────────────┐   ┌─────────────────┐   ┌─────────────────┐      │
│   │ App Server 1    │   │ App Server 2    │   │ App Server N    │      │
│   │ (Node/Express)  │   │ (Node/Express)  │   │ (Node/Express)  │      │
│   │ • Business Logic│   │ • Business Logic│   │ • Business Logic│      │
│   │ • Auth/Authz    │   │ • Auth/Authz    │   │ • Auth/Authz    │      │
│   │ • API Handlers  │   │ • API Handlers  │   │ • API Handlers  │      │
│   └────────┬────────┘   └────────┬────────┘   └────────┬────────┘      │
│            │                     │                     │                │
└────────────┼─────────────────────┼─────────────────────┼────────────────┘
             │                     │                     │
             └──────────┬──────────┴─────────────────────┘
                        │
           ┌────────────┴────────────┐
           ▼                         ▼
┌─────────────────────────────────────────────────────────────────────────┐
│                      TIER 3: DATA LAYER                                  │
│                                                                         │
│   ┌─────────────────────────────┐   ┌─────────────────────────────┐    │
│   │        CACHE CLUSTER        │   │      DATABASE CLUSTER       │    │
│   │  ┌───────┐     ┌───────┐    │   │  ┌───────┐     ┌───────┐   │    │
│   │  │ Redis │     │ Redis │    │   │  │Primary│     │Replica│   │    │
│   │  │Primary│────▶│Replica│    │   │  │(Write)│────▶│(Read) │   │    │
│   │  └───────┘     └───────┘    │   │  └───────┘     └───────┘   │    │
│   │                             │   │                             │    │
│   │  • Hot data caching         │   │  • Persistent storage       │    │
│   │  • Session storage          │   │  • Transactions             │    │
│   │  • Rate limiting            │   │  • Complex queries          │    │
│   └─────────────────────────────┘   └─────────────────────────────┘    │
│                                                                         │
└─────────────────────────────────────────────────────────────────────────┘

Request Flow Through Three-Tier: Client Request : Browser sends GET /api/products/123 Load Balancer : Routes request to an available app server Application Server : Receives request, authenticates user Cache Check : App server queries cache for product:123 Cache Hit : Return cached data (fast path) Cache Miss : Continue to database Database Query : App server queries database for product Cache Update : Store result in cache for future requests Response : App server formats and returns response to client Benefits of Three-Tier: Separation of Concerns : Each tier specializes in specific responsibilities Independent Scaling : Scale app servers without touching database Technology Flexibility : Change database vendor without affecting app servers Security Boundaries : Database isn't directly accessible from internet Team Organization : Different teams can own different tiers

Three-Tier is a Starting Point

Three-tier works well for many applications but isn't the only pattern. As systems grow, you might add CDNs (presentation layer caching), message queues (async processing), search services (Elasticsearch), or decompose into microservices. Start with three-tier; evolve based on need.

Server Infrastructure: Scaling and Reliability

Running production servers requires more than just deploying code. You must address infrastructure concerns that determine whether your system stays up under load, recovers from failures, and remains secure. Horizontal vs. Vertical Scaling: Vertical Scaling (Scale Up) : Add more resources to existing servers—more CPU, RAM, faster disks. Simple but limited: eventually you hit the largest available machine, and you have a single point of failure. Horizontal Scaling (Scale Out) : Add more server instances. Requires stateless design (for app servers) or distributed data systems (for databases). More complex but provides better availability and can scale nearly infinitely.

Scaling Strategy by Server Type
Server Type	Primary Scaling Strategy	Challenges	Common Solutions
Application Servers	Horizontal (add instances)	Session state, warm-up	Stateless design, load balancers
Database (Reads)	Horizontal (read replicas)	Replication lag, routing	Replica sets, read-write splitting
Database (Writes)	Vertical first, then sharding	Sharding complexity, joins	Sharding, distributed databases
Cache Servers	Horizontal (cluster)	Key distribution, hot keys	Consistent hashing, clustering

High Availability Patterns: Redundancy: Run multiple instances of every component. No single point of failure. If one app server fails, others continue serving requests. Health Checks: Load balancers continuously check server health (HTTP endpoint, TCP connection). Unhealthy servers are removed from rotation. Failover: Automatic switching from failed primary to healthy standby. Database primary fails? Replica is promoted. Cache fails? App falls back to database. Distribution: Spread servers across multiple availability zones (data centers). If one zone fails, others continue operating.

Production Infrastructure Checklist

•Monitoring & Alerting — Metrics (CPU, memory, latency), logs, and traces. Alert on anomalies before users notice.
•Auto-Scaling — Automatically add/remove instances based on CPU, memory, queue depth, or request rate.
•Deployment Strategy — Blue-green, canary, or rolling deployments to minimize downtime and enable rollbacks.
•Configuration Management — Environment variables, secrets management (Vault, AWS Secrets Manager), feature flags.
•Backup & Recovery — Regular database backups, tested restore procedures, point-in-time recovery.
•Security Hardening — Network segmentation, minimal permissions, encrypted data at rest and in transit.
•Capacity Planning — Understand limits, plan for peak load, and test capacity before you need it.

Cloud vs. Self-Hosted

Cloud platforms (AWS, GCP, Azure) provide managed versions of all server types: RDS for databases, ElastiCache for Redis, ECS/EKS for application servers. Managed services reduce operational burden but increase cost and may limit flexibility. Many organizations use a mix: managed databases for reliability, self-managed app servers for control.

Connection Management Between Tiers

Communication between servers—app to database, app to cache, service to service—requires careful connection management. Poorly managed connections are a common source of outages, resource exhaustion, and performance degradation. Connection Pools: Opening network connections involves TCP handshakes, TLS negotiation, and protocol initialization—operations that take milliseconds. Connection pools maintain a set of pre-established connections that are reused across requests. Key Pool Parameters: Maximum Size : Upper limit on connections. Too low = contention; too high = resource exhaustion on server. Minimum/Idle Size : Keep some connections warm to avoid cold-start latency. Connection Timeout : How long to wait for a connection from the pool. Idle Timeout : How long unused connections stay open before closing. Max Lifetime : Maximum age of a connection (handles server-side connection limits).

connection-pool-config.ts
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
// Connection Pool Configuration Examples
 
// PostgreSQL Connection Pool
import { Pool } from 'pg';
 
const databasePool = new Pool({
  host: 'db.example.com',
  database: 'production',
  user: 'app_user',
  password: process.env.DB_PASSWORD,
  
  max: 20,                  // Max connections (tune based on load testing)
  min: 5,                   // Keep at least 5 connections warm
  idleTimeoutMillis: 30000, // Close idle connections after 30s
  connectionTimeoutMillis: 10000, // Fail if can't get connection in 10s
  
  // Health checking
  allowExitOnIdle: false,
  keepAlive: true,
  keepAliveInitialDelayMillis: 10000,
});
 
// Monitor pool health
databasePool.on('connect', () => {
  metrics.increment('db.pool.connection_opened');
});
databasePool.on('remove', () => {
  metrics.increment('db.pool.connection_closed');
});
databasePool.on('error', (err) => {
  logger.error('Database pool error', { error: err.message });
  metrics.increment('db.pool.errors');
});
 
// Redis Connection Pool (via ioredis)
import Redis from 'ioredis';
 
const redisCluster = new Redis.Cluster([
  { host: 'redis-1.example.com', port: 6379 },
  { host: 'redis-2.example.com', port: 6379 },
  { host: 'redis-3.example.com', port: 6379 },
], {
  redisOptions: {
    password: process.env.REDIS_PASSWORD,
    connectTimeout: 10000,
    commandTimeout: 5000,
    retryStrategy(times) {
      return Math.min(times * 50, 2000); // Exponential backoff
    },
  },
  clusterRetryStrategy(times) {
    return Math.min(times * 100, 3000);
  },
  enableReadyCheck: true,
  maxRedirections: 16,
  scaleReads: 'slave', // Read from replicas
});
 
// HTTP Connection Pool for External APIs
import axios from 'axios';
import https from 'https';
 
const externalApiClient = axios.create({
  baseURL: 'https://api.external-service.com',
  timeout: 5000,
  
  // Connection pool via Node.js agent
  httpsAgent: new https.Agent({
    keepAlive: true,
    maxSockets: 50,       // Max connections to this host
    maxFreeSockets: 10,   // Keep 10 idle connections
    timeout: 60000,       // Socket timeout
  }),
});

Connection Leaks: A connection leak occurs when code acquires a connection but never releases it. Over time, the pool is exhausted and new requests fail. Prevent leaks with: Try/finally blocks : Always release in finally Using statements : Language features that auto-release (C#'s using, Python's with) Timeouts : Automatically reclaim connections held too long Monitoring : Alert when pool utilization is consistently high Circuit Breakers: If a downstream server is failing, continuing to attempt connections wastes resources and increases latency. Circuit breakers 'trip' after a threshold of failures, failing fast without attempting connections until the circuit 'resets' (after a timeout or when health checks pass).

Size Your Pools Carefully

If you have 10 app servers, each with 20 database connections, your database sees 200 connections. Many databases have connection limits (e.g., RDS default is 150-500 depending on instance size). More connections aren't always better—context switching between many connections reduces performance. Start conservative and tune based on metrics.

Choosing the Right Server Technologies

Each tier offers multiple technology options. Selection depends on your requirements, team expertise, operational constraints, and expected scale. Here are key decision factors for each server type. Application Server Selection: Language/Framework Expertise : Use what your team knows. Node.js, Python/Django, Java/Spring, Go, Ruby/Rails are all proven at scale. Performance Profile : CPU-bound work? Consider Go or Java. I/O-bound? Node.js or async Python excel. Ecosystem : Libraries, frameworks, and community resources matter for development velocity. Operational Maturity : How mature are deployment, monitoring, and debugging tools in your stack?

Application Server Technology Trade-offs
Technology	Strengths	Weaknesses	Best For
Node.js (Express)	I/O efficiency, JS ecosystem, developer pool	CPU-bound work, type safety (without TS)	APIs, real-time apps, BFFs
Python (FastAPI)	Rapid development, ML/data ecosystem	Performance vs compiled langs	Data-heavy APIs, ML backends
Java (Spring)	Enterprise features, JVM performance, tooling	Verbosity, memory usage, cold start	Enterprise, high-throughput systems
Go	Performance, concurrency, small binaries	Smaller ecosystem, error handling verbosity	Microservices, infrastructure
Ruby (Rails)	Developer productivity, conventions	Performance, scaling challenges	Startups, MVPs, content sites

Database Selection: Data Model : Relational data with complex queries? PostgreSQL. Documents with flexible schema? MongoDB. Simple key lookups at scale? DynamoDB. Consistency Requirements : Need ACID transactions? Relational. Can tolerate eventual consistency? NoSQL options open up. Scale Pattern : Read-heavy? Any database with replicas. Write-heavy at massive scale? Consider Cassandra, DynamoDB. Operational Complexity : Managed (RDS, Cosmos DB) vs. self-managed. Managed reduces burden but limits control. Cache Selection: Feature Needs : Just caching? Memcached is simple. Need data structures, pub/sub, persistence? Redis. Cluster Support : Redis Cluster is mature. Memcached sharding is client-side. Memory Efficiency : For pure string caching, Memcached may use less memory. Managed Options : ElastiCache (AWS), MemoryStore (GCP), Azure Cache reduce operational burden.

Technology Selection Principles

•Boring Technology: Prefer proven, widely-used technologies over new shiny ones. PostgreSQL, Redis, and mature frameworks have battle-tested reliability.
•Minimize Diversity: Each technology is operational overhead. Don't use three different databases if one suffices. Reduce cognitive load.
•Consider the Future: Will you need full-text search? Consider Postgres vs adding Elasticsearch later. Think ahead but don't over-engineer.
•Team Capability: The best technology is one your team can operate well. A MongoDB expert team may outperform a PostgreSQL novice team with PostgreSQL.
•Migration Path: If you choose wrong, how hard is migration? Early choices are easier to reverse than late ones.

Start with Defaults

When in doubt: PostgreSQL for persistent data, Redis for caching/sessions, and your language's most popular web framework. These defaults work for the vast majority of applications. Only deviate when you have clear requirements that justify complexity.

Servers: Application, Database, Cache

The Layered Ecosystem That Powers Every Application

Application Servers: The Business Logic Layer

Database Servers: The Persistence Layer

Cache Servers: The Performance Accelerator

The Three-Tier Architecture: Putting It Together

Server Infrastructure: Scaling and Reliability

Connection Management Between Tiers

Choosing the Right Server Technologies

There’s more to understand.

Servers: Application, Database, Cache

The Layered Ecosystem That Powers Every Application

Application Servers: The Business Logic Layer

Database Servers: The Persistence Layer

Cache Servers: The Performance Accelerator

The Three-Tier Architecture: Putting It Together

Server Infrastructure: Scaling and Reliability

Connection Management Between Tiers

Choosing the Right Server Technologies

There’s more to understand.