Loading learning content...
When the term "document database" is mentioned, MongoDB is invariably the first name that comes to mind. Since its release in 2009, MongoDB has grown from a developer-friendly alternative to relational databases into a enterprise-grade platform powering some of the world's most demanding applications.
MongoDB's journey illustrates how meeting developer needs—flexible schemas, horizontal scalability, and a query language that feels natural—can disrupt a market dominated by 40-year-old relational incumbents.
In this page, we'll explore MongoDB comprehensively: its architectural decisions, core abstractions, scaling mechanisms, and the operational wisdom accumulated from deployment at massive scale.
By the end of this page, you will understand: MongoDB's architecture and core components; the replica set model for high availability; sharding for horizontal scalability; the aggregation framework; write and read concerns; transactions and consistency models; and best practices for production deployments.
MongoDB's architecture has evolved significantly since its inception. Understanding these components is essential for effective deployment and troubleshooting.
1. mongod (MongoDB Daemon)
The primary database process that:
2. mongos (MongoDB Shard Router)
In sharded deployments, mongos:
3. Config Servers
Maintain cluster metadata:
MongoDB uses a pluggable storage engine model. The default WiredTiger engine provides:
┌─────────────────────────────────────────────────────────────┐
│ MongoDB Server (mongod) │
├─────────────────────────────────────────────────────────────┤
│ Query Engine │
│ ┌─────────────────────────────────────────────────────────┐ │
│ │ Query Planning → Optimization → Execution │ │
│ └─────────────────────────────────────────────────────────┘ │
├─────────────────────────────────────────────────────────────┤
│ Storage Engine API │
├─────────────────────────────────────────────────────────────┤
│ WiredTiger │
│ ┌───────────────┬───────────────┬───────────────────────┐ │
│ │ Cache │ B-trees │ Write-Ahead Log │ │
│ │ (Configurable)│ (Data/Index) │ (Journaling) │ │
│ └───────────────┴───────────────┴───────────────────────┘ │
├─────────────────────────────────────────────────────────────┤
│ File System │
│ Data Files (.wt) │ Journal Files │ Logs │
└─────────────────────────────────────────────────────────────┘
WiredTiger Features:
| Relational Concept | MongoDB Equivalent | Notes |
|---|---|---|
| Database | Database | Logical container for collections |
| Table | Collection | Container for documents |
| Row | Document | BSON document (up to 16MB) |
| Column | Field | Key-value pair within document |
| Index | Index | B-tree indexes on fields |
| Primary Key | _id field | Unique identifier, auto-generated if not provided |
| Foreign Key | Reference / DBRef | Manual references or embedded documents |
| JOIN | $lookup / Embedding | Aggregation pipeline or denormalization |
A replica set is a group of MongoDB instances that maintain the same data set, providing redundancy and high availability. Replica sets are the foundation of MongoDB's fault tolerance.
┌─────────────────────────────────────┐
│ Clients │
│ (Drivers with connection URI) │
└───────────────┬─────────────────────┘
│
┌──────────────────┼──────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ PRIMARY │ │ SECONDARY │ │ SECONDARY │
│ │───▶│ │ │ │
│ Writes + │ │ Reads │ │ Reads │
│ Reads │ │ (optional) │ │ (optional) │
│ │ │ │ │ │
│ ┌───────┐ │ │ ┌───────┐ │ │ ┌───────┐ │
│ │ Oplog │──┼───▶│ │ Oplog │ │ │ │ Oplog │ │
│ └───────┘ │ │ └───────┘ │ │ └───────┘ │
└─────────────┘ └─────────────┘ └─────────────┘
▲ ▲
│ │
└──────────────────┘
Replication
Primary
Secondaries
Oplog (Operations Log)
When the primary becomes unavailable:
Election Configuration:
// Replica set configuration
{
"_id": "myReplicaSet",
"members": [
{ "_id": 0, "host": "mongo1:27017", "priority": 2 }, // Preferred primary
{ "_id": 1, "host": "mongo2:27017", "priority": 1 }, // Normal
{ "_id": 2, "host": "mongo3:27017", "priority": 0 }, // Never primary
{ "_id": 3, "host": "arbiter:27017", "arbiterOnly": true } // Voting only
]
}
Reading from secondaries introduces complexity:
• primary: Only read from primary (default, strongly consistent) • primaryPreferred: Primary if available, else secondary • secondary: Only from secondaries (may read stale data) • secondaryPreferred: Secondary if available, else primary • nearest: Lowest latency member (any role)
Reading from secondaries risks reading stale data. Only use when your application tolerates eventual consistency.
When data exceeds a single server's capacity, MongoDB sharding distributes data across multiple machines. This enables horizontal scaling for both storage and throughput.
┌───────────────────────────────────────┐
│ Application │
└───────────────────┬───────────────────┘
│
┌────────────────────────────┼────────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────┐ ┌─────────────┐ ┌─────────────┐
│ mongos │ │ mongos │ │ mongos │
│ (Router) │ │ (Router) │ │ (Router) │
└──────┬──────┘ └──────┬──────┘ └──────┬──────┘
│ │ │
└────────────────────────────┼────────────────────────────┘
│
┌───────────────────┼───────────────────┐
│ │ │
▼ ▼ ▼
┌───────────┐ ┌───────────┐ ┌───────────┐
│ Config │ │ Config │ │ Config │
│ Server │ │ Server │ │ Server │
│ (Primary) │◀─────▶│(Secondary)│◀─────▶│(Secondary)│
└───────────┘ └───────────┘ └───────────┘
│
┌────────────────────────────────┼────────────────────────────┐
│ │ │
▼ ▼ ▼
┌─────────────────┐ ┌─────────────────┐ ┌─────────────────┐
│ Shard 1 │ │ Shard 2 │ │ Shard 3 │
│ (Replica Set) │ │ (Replica Set) │ │ (Replica Set) │
│ Users: A-H │ │ Users: I-P │ │ Users: Q-Z │
└─────────────────┘ └─────────────────┘ └─────────────────┘
The shard key determines how documents are distributed. This is the most critical sharding decision:
Range-Based Sharding:
// Shard by user_id: contiguous ranges on each shard
sh.shardCollection("mydb.users", { "user_id": 1 });
// Shard 1: user_id 1-100000
// Shard 2: user_id 100001-200000
// Shard 3: user_id 200001-300000
Hashed Sharding:
// Shard by hash of user_id: random distribution
sh.shardCollection("mydb.users", { "user_id": "hashed" });
// Documents randomly distributed based on hash value
Compound Shard Keys:
// Shard by tenant + timestamp: isolation + distribution
sh.shardCollection("mydb.events", { "tenant_id": 1, "timestamp": 1 });
An ideal shard key has:
• High cardinality — Many distinct values for even distribution • Low frequency — No single value dominates (avoids jumbo chunks) • Non-monotonic — Avoids sequential inserts hitting one shard • Query coverage — Included in common queries for targeted routing • Immutability — Cannot be changed after document creation
MongoDB provides tunable consistency through write concern and read concern settings. Understanding these is critical for balancing performance against durability and consistency.
Write concern specifies the acknowledgment level for write operations:
| Write Concern | Meaning | Durability | Performance |
|---|---|---|---|
w: 0 | No acknowledgment | None | Fastest |
w: 1 | Primary acknowledges | Volatile | Fast |
w: 1, j: true | Primary + journal | Durable on crash | Medium |
w: "majority" | Majority of replicas | Durable + HA | Slower |
w: 3 | Specific count | Custom | Varies |
Code Examples:
// Fire-and-forget (dangerous, but fastest)
db.orders.insertOne(doc, { writeConcern: { w: 0 } });
// Primary acknowledgment (default)
db.orders.insertOne(doc, { writeConcern: { w: 1 } });
// Journaled write (survives primary crash)
db.orders.insertOne(doc, { writeConcern: { w: 1, j: true } });
// Majority (survives any single node failure)
db.orders.insertOne(doc, { writeConcern: { w: "majority" } });
// With timeout to prevent indefinite blocking
db.orders.insertOne(doc, {
writeConcern: { w: "majority", wtimeout: 5000 }
});
Read concern specifies the consistency guarantees for read operations:
| Read Concern | Guarantees | Use Case |
|---|---|---|
local | Returns local data, may be rolled back | Default, low-latency reads |
available | Like local, ignores orphaned docs in sharding | Sharded low-latency |
majority | Data acknowledged by majority | Consistent reads, won't roll back |
linearizable | Reflects all successful majority writes | Strongest consistency |
snapshot | Point-in-time snapshot of majority data | Multi-document transactions |
Causal Consistency:
MongoDB supports causal consistency to ensure: If operation A happened before operation B, any session that sees B will also see A.
// Start a causally consistent session
const session = client.startSession({ causalConsistency: true });
// All operations in this session are causally ordered
await session.withTransaction(async () => {
await db.orders.insertOne({ ... }, { session });
await db.inventory.updateOne({ ... }, { session });
});
w: majority + readConcern: majorityw: 1 + readConcern: localSince MongoDB 4.0+, multi-document ACID transactions are supported across replica sets (and sharded clusters in 4.2+). This closes a significant gap with relational databases while preserving document model benefits.
MongoDB transactions provide:
Transaction Example:
const session = client.startSession();
try {
session.startTransaction({
readConcern: { level: "snapshot" },
writeConcern: { w: "majority" },
readPreference: "primary"
});
// All operations are part of the transaction
await db.accounts.updateOne(
{ _id: "account_from" },
{ $inc: { balance: -100 } },
{ session }
);
await db.accounts.updateOne(
{ _id: "account_to" },
{ $inc: { balance: 100 } },
{ session }
);
await db.transfers.insertOne(
{
from: "account_from",
to: "account_to",
amount: 100,
timestamp: new Date()
},
{ session }
);
// Atomically commit all changes
await session.commitTransaction();
} catch (error) {
// Roll back all changes
await session.abortTransaction();
throw error;
} finally {
session.endSession();
}
Transactions are powerful but have constraints:
• Duration: Default 60-second limit (tunable) • Size: 16MB oplog entry limit per transaction • Performance: Transactions are slower than single-document operations • Design: Prefer document embedding over multi-document transactions when possible • Retries: Implement retry logic for transient errors (network, elections)
Rule of thumb: If data always changes together, it should be in one document.
Good use cases:
When embedding is better:
MongoDB's Aggregation Framework is a declarative pipeline for data transformation and analysis. It's far more powerful than simple queries, enabling complex analytics within the database.
Documents flow through a sequence of stages, each transforming the data:
┌──────────────┐ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐
│ Collection │───▶│ $match │───▶│ $group │───▶│ $sort │
│ │ │ (filter) │ │ (aggregate) │ │ (order) │
└──────────────┘ └──────────────┘ └──────────────┘ └──────────────┘
│
▼
┌──────────────┐
│ Results │
└──────────────┘
| Stage | Purpose | SQL Equivalent |
|---|---|---|
| $match | Filter documents | WHERE clause |
| $project | Shape documents, include/exclude fields | SELECT columns |
| $group | Group and aggregate | GROUP BY + aggregates |
| $sort | Order results | ORDER BY |
| $limit / $skip | Pagination | LIMIT / OFFSET |
| $lookup | Join with another collection | LEFT OUTER JOIN |
| $unwind | Deconstruct array into multiple docs | Flatten arrays |
| $addFields | Add computed fields | Computed columns |
| $bucket | Categorize into buckets | CASE with ranges |
| $facet | Multiple parallel pipelines | Multiple queries in one |
Scenario: Calculate monthly revenue by product category, showing only categories with > $10,000 revenue, sorted descending.
db.orders.aggregate([
// Stage 1: Filter to last year's orders
{
$match: {
orderDate: { $gte: ISODate("2024-01-01") },
status: "completed"
}
},
// Stage 2: Unwind order items to process individually
{ $unwind: "$items" },
// Stage 3: Group by category and month
{
$group: {
_id: {
category: "$items.category",
month: { $month: "$orderDate" },
year: { $year: "$orderDate" }
},
revenue: { $sum: { $multiply: ["$items.price", "$items.quantity"] } },
orderCount: { $sum: 1 },
avgOrderValue: { $avg: { $multiply: ["$items.price", "$items.quantity"] } }
}
},
// Stage 4: Filter high-revenue categories
{ $match: { revenue: { $gt: 10000 } } },
// Stage 5: Sort by revenue descending
{ $sort: { revenue: -1 } },
// Stage 6: Shape final output
{
$project: {
_id: 0,
category: "$_id.category",
period: {
$concat: [
{ $toString: "$_id.year" },
"-",
{ $toString: "$_id.month" }
]
},
revenue: { $round: ["$revenue", 2] },
orderCount: 1,
avgOrderValue: { $round: ["$avgOrderValue", 2] }
}
}
]);
• Place $match early — Uses indexes, reduces documents flowing through pipeline
• Combine $match stages — MongoDB optimizes adjacent matches
• Use $project to drop unneeded fields early — Reduces memory usage
• Consider $group with $first instead of $lookup for lookups — Often faster
• Use $allowDiskUse: true for large datasets exceeding 100MB memory limit
• explain() your pipelines to understand execution plans
Effective indexing is critical for MongoDB performance. Unlike relational databases where the query optimizer has decades of refinement, MongoDB relies heavily on well-designed indexes.
Single Field Index:
db.users.createIndex({ email: 1 }); // Ascending
db.users.createIndex({ lastLogin: -1 }); // Descending
Compound Index:
// Supports queries on: status, status+category, status+category+date
db.orders.createIndex({ status: 1, category: 1, orderDate: -1 });
Multikey Index (on arrays):
// Indexes each element in the tags array
db.products.createIndex({ tags: 1 });
Text Index:
// Full-text search on multiple fields
db.articles.createIndex({
title: "text",
content: "text",
tags: "text"
}, {
weights: { title: 10, tags: 5, content: 1 }
});
// Search query
db.articles.find({ $text: { $search: "mongodb performance" } });
Geospatial Index:
db.places.createIndex({ location: "2dsphere" });
// Find nearby locations
db.places.find({
location: {
$near: {
$geometry: { type: "Point", coordinates: [-73.99, 40.73] },
$maxDistance: 5000 // meters
}
}
});
When designing compound indexes, follow the E-S-R (Equality-Sort-Range) order:
field: value)$sort$gt, $lt, $in)Example:
// Query pattern:
db.orders.find({
status: "pending", // Equality
amount: { $gt: 100 } // Range
}).sort({ createdAt: -1 }); // Sort
// Optimal index (ESR order):
db.orders.createIndex({ status: 1, createdAt: -1, amount: 1 });
Why ESR works:
• Index overhead: Each index adds write latency and storage
• Memory: Indexes should fit in RAM; check db.stats().indexSize
• Covered queries: Best performance when all fields are in the index
• Selectivity: Low-cardinality fields (e.g., boolean) benefit less from indexes
• Compound order: {a: 1, b: 1} ≠ {b: 1, a: 1} — Order matters!
• Explain: Always use explain("executionStats") to verify index usage
We've explored MongoDB as the canonical document database. Let's consolidate the essential knowledge:
What's Next:
With MongoDB's architecture and operations understood, we'll dive into querying documents—exploring the full query language, advanced operators, text search, and geospatial queries that make MongoDB a powerful data platform.
You now have comprehensive knowledge of MongoDB's architecture, replication, sharding, consistency controls, transactions, aggregation, and indexing. This foundation prepares you to design, deploy, and operate MongoDB in production environments.