Loading learning content...
When data exists in multiple places—replicas, shards, caches—a fundamental question arises: How do we ensure all readers see a consistent view of the data? The answer has profound implications for system design, user experience, and operational complexity.
In system design interviews, consistency is often where candidates stumble. Many memorize the CAP theorem without understanding what consistency actually means operationally. This page provides the deep understanding needed to reason about consistency in real systems and make informed trade-offs.
By the end of this page, you will understand the full spectrum of consistency models from linearizability to eventual consistency. You'll learn how CAP and PACELC theorems guide design decisions, how different databases implement various consistency levels, and how to choose the right consistency model for different parts of your system.
"Consistency" is an overloaded term in database literature. Let's disambiguate:
| Context | Type | Definition |
|---|---|---|
| ACID | Consistency | Transaction takes database from one valid state to another (integrity constraints) |
| CAP Theorem | Consistency | All nodes see the same data at the same time (linearizability) |
| Replication | Consistency | Agreement between replicas about current data state |
| Cache | Consistency | Cache reflects source-of-truth accurately |
In distributed systems, we focus on replication consistency: when data is copied across multiple nodes, what guarantees do we have about read operations?
The Core Challenge:
Consider a replicated database with the following timeline:
Time →
Writer : [Write X=2]─────────────────────────────────────────►
Primary : ────────[X=2]───────────────────────────────────────►
Replica 1 : ────────────────[X=2]───────────────────────────────►
Replica 2 : ────────────────────────────[X=2]───────────────────►
Reader A (Replica 1) at T1: Reads X=?
Reader B (Replica 2) at T1: Reads X=?
During the replication window:
This is the consistency problem. Different consistency models provide different guarantees about what readers can observe.
Network propagation isn't instant. Light travels ~186 miles in 1 millisecond. A cross-continental round trip (3000 miles) takes at minimum ~16ms, cross-Atlantic ~50ms. This physical reality makes perfect synchrony impossible and is why consistency trade-offs exist.
Consistency models form a spectrum from strongest (linearizability) to weakest (eventual consistency). Stronger guarantees require more coordination, increasing latency and reducing availability.
┌─────────────────────────────────────────────────────────────────────┐
│ CONSISTENCY SPECTRUM │
├─────────────────────────────────────────────────────────────────────┤
│ │
│ STRONGER GUARANTEES WEAKER GUARANTEES │
│ (More coordination) (Less coordination) │
│ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │Linearizable │ → │ Sequential │ → │ Causal │ │
│ │(Strongest) │ │ Consistency │ │ Consistency │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ ↓ │
│ ┌──────────────┐ ┌──────────────┐ ┌──────────────┐ │
│ │Read-Your- │ → │ Monotonic │ → │ Eventual │ │
│ │Writes │ │ Reads │ │ (Weakest) │ │
│ └──────────────┘ └──────────────┘ └──────────────┘ │
│ │
│ ← Higher Latency, Lower Availability │
│ Higher Availability, Lower Latency → │
│ │
└─────────────────────────────────────────────────────────────────────┘
Linearizability (Strong Consistency)
The strongest consistency model. Every operation appears to take effect instantaneously at some point between its invocation and response. The system behaves as if there's only one copy of the data.
Guarantees:
Implementation:
1. Consensus protocols (Paxos, Raft)
2. Quorum reads/writes: R + W > N
3. Single-leader replication with synchronous followers
Example Scenario:
Time →
Client A: [Write X=5]──────────────────────[Write Complete]
Client B: [Read X]───────[Returns 5 or blocks until complete]
Linearizability guarantees: If B's read starts after A's write completes,
B MUST see X=5 (or a later value).
Use Cases:
Cost:
Between strong and eventual consistency lie session guarantees—consistency models that provide useful guarantees within a single client session without requiring global coordination.
| Guarantee | Definition | Example Violation |
|---|---|---|
| Read Your Writes | A client always sees its own writes | Update profile, refresh page, see old photo |
| Monotonic Reads | Once you see a value, you never see older values | Refresh feed, see old posts that were hidden before |
| Monotonic Writes | A client's writes take effect in order | Send A then B, recipient gets B then A |
| Writes Follow Reads | Writes are ordered after the reads that influenced them | Reply to comment, reply appears before comment |
Implementing Read-Your-Writes
This is the most commonly needed session guarantee. Several implementation approaches:
1. Sticky Sessions
# Route user to same replica they wrote to
def get_replica_for_read(user_id, wrote_recently):
if wrote_recently:
return get_user_primary_replica(user_id)
return get_any_replica()
2. Version Tracking
# Track version user has seen
def read_with_version(user_id, min_version):
replica = get_replica()
while replica.current_version < min_version:
wait(10ms)
# Or try another replica
return replica.read(user_id)
def write(user_id, data):
version = primary.write(data)
session.set_min_version(user_id, version)
return version
3. Time-Based Routing
# After write, use primary for N seconds
def route_read(user_id):
last_write_time = get_last_write_time(user_id)
if now() - last_write_time < READ_YOUR_WRITES_WINDOW:
return primary
return replica
When discussing eventual consistency in interviews, immediately follow with 'but we'd implement read-your-writes for user-facing operations.' This shows you understand the practical implications and user experience concerns.
The CAP theorem is the most famous result in distributed systems theory. Understanding it properly—beyond the oversimplified 'pick two' interpretation—is essential for system design discussions.
CAP Theorem (Brewer, 2000)
In a distributed system, during a network partition, you must choose between:
┌─────────────────────────────────────┐
│ CAP Theorem │
├─────────────────────────────────────┤
│ │
│ C ─────────────── A │
│ (Consistency) (Availability) │
│ \ / │
│ \ / │
│ \ / │
│ \ / │
│ \ / │
│ P │
│ (Partition Tolerance) │
│ │
│ During partition, choose C or A │
│ P is not optional in real networks │
│ │
└─────────────────────────────────────┘
Key Definitions:
Critical Insight: P is Not Optional
Networks partition. Cables get cut, switches fail, packets drop. A "CA" system is theoretically a single-node database (no partitions because no network). In practice, distributed systems choose between CP and AP.
PACELC Theorem (Abadi, 2010)
CAP only describes behavior during partitions. PACELC extends this to normal operation:
if (Partition) {
choose: Availability or Consistency
} else {
choose: Latency or Consistency
}
Or: PA/EL, PA/EC, PC/EL, PC/EC
| System | Partition Choice | Normal Choice | Classification |
|---|---|---|---|
| Dynamo/Cassandra | Availability | Latency | PA/EL |
| MongoDB | Consistency | Latency | PC/EL |
| PNUTS (Yahoo) | Availability | Consistency | PA/EC |
| Traditional RDBMS | N/A (single node) | Consistency | — |
| CockroachDB | Consistency | Consistency | PC/EC |
| Spanner | Consistency | Consistency | PC/EC |
Interview Tip: Mentioning PACELC shows depth beyond basic CAP knowledge. It acknowledges that even during normal operation, there's a consistency-latency trade-off.
CAP is not 'pick any two.' It's: 'during a partition, pick C or A.' Most of the time, there's no partition and you have both. Also, CAP's C is specifically linearizability—weaker consistency models can coexist with availability.
Different databases offer different consistency levels, often configurable per-operation. Understanding these options is crucial for system design.
Cassandra Consistency Levels
Cassandra uses quorum-based consistency, configurable per query.
-- Strong read (quorum)
SELECT * FROM users WHERE id = ?
USING CONSISTENCY QUORUM;
-- Fast read (any replica)
SELECT * FROM users WHERE id = ?
USING CONSISTENCY ONE;
-- Strongest (all replicas must respond)
SELECT * FROM users WHERE id = ?
USING CONSISTENCY ALL;
Write Consistency:
| Level | Description | Latency | Durability |
|---|---|---|---|
| ANY | Any node (including hints) | Lowest | Weakest |
| ONE | One replica confirms | Low | Weak |
| QUORUM | Majority confirms | Medium | Strong |
| ALL | All replicas confirm | Highest | Strongest |
| LOCAL_QUORUM | Quorum in local DC | Medium | Strong locally |
Strong Consistency Formula:
For RF (replication factor) = 3:
Strong: R + W > RF
- QUORUM read + QUORUM write: 2 + 2 > 3 ✓
- ONE read + ALL write: 1 + 3 > 3 ✓
- ONE read + ONE write: 1 + 1 > 3 ✗ (eventual)
Per-Query Tuning:
// High-value transaction: strong consistency
session.execute("INSERT INTO payments ..."
.setConsistencyLevel(ConsistencyLevel.QUORUM));
// View counter: eventual is fine
session.execute("UPDATE views SET count = count + 1"
.setConsistencyLevel(ConsistencyLevel.ONE));
Different operations within the same system often require different consistency levels. Matching consistency to requirements is a key system design skill.
| Operation Type | Typical Requirement | Consistency Level | Rationale |
|---|---|---|---|
| Payment processing | Must not lose or duplicate | Strong (linearizable) | Financial correctness is non-negotiable |
| Inventory decrement | Must not oversell | Strong | Negative inventory is unacceptable |
| User authentication | Must reflect current state | Read-your-writes | Login/logout must be immediate |
| User profile view | Can tolerate staleness | Eventual | Seeing 5-second-old data is fine |
| Like count display | Approximate is acceptable | Eventual | 24,532 vs 24,531 doesn't matter |
| Feed generation | Bounded staleness OK | Causal / Eventual | New posts appearing 2s late is acceptable |
| Comments on post | Must follow post order | Causal | Reply must not appear before the post |
| Shopping cart | Per-user consistency | Session / Eventual | Can merge on checkout |
| Leaderboard | Near-real-time | Eventual with refresh | ~1 second delay is acceptable |
| Bank balance | Must be accurate | Strong | Legal requirement for accuracy |
Decision Framework for Consistency Selection:
┌──────────────────────────────────────────────────────────┐
│ CONSISTENCY DECISION TREE │
└──────────────────────────────────────────────────────────┘
1. Is data loss financially or legally unacceptable?
YES → Strong consistency (linearizable)
NO → Continue...
2. Can users tolerate seeing stale data?
NO → Strong or Read-your-writes
YES → Continue...
3. Do operations have causal dependencies?
(Comments after posts, replies after messages)
YES → Causal consistency
NO → Continue...
4. How stale is acceptable?
Seconds → Eventual with bounded staleness
Minutes/Hours → Eventual, async refresh
5. What happens if two users modify simultaneously?
Must prevent conflicts → Strong with locking
Can merge conflicts → Eventual with CRDTs
Last write wins → Eventual with LWW
Most production systems use multiple consistency levels. Strong for payments and inventory, eventual for analytics and reactions, read-your-writes for user-facing state. Explain this hybrid approach in interviews.
Understanding common mistakes helps avoid them in designs and demonstrates production experience in interviews.
The Read-After-Write Problem:
PROBLEM
User writes to Primary → Cache/Replica not updated → User reads from Replica
↓
User sees old data!
SOLUTIONS
1. Read from Primary after write (for N seconds)
2. Include write version, wait for replica to catch up
3. Sticky sessions to same node
4. Invalidate cache synchronously during write
The Stale Read After Failover:
PROBLEM
Async replication: Primary ahead of Replica by 10 seconds
Primary fails → Replica promoted
↓
10 seconds of transactions LOST
SOLUTIONS
1. Synchronous replication (higher latency)
2. Semi-synchronous: At least one replica must confirm
3. Accept the risk for non-critical data
4. Detect and alert on replication lag
Two-phase commit (2PC) provides strong consistency across databases but is slow and blocks on coordinator failure. In interviews, acknowledge the cost: 'We could use 2PC for cross-service consistency, but it adds latency and availability concerns. Let's consider sagas or eventual consistency if acceptable.'
Consistency is the heart of distributed systems trade-offs. Let's consolidate the key takeaways:
What's Next:
With consistency models mastered, the final page covers Trade-off Discussions—how to articulate database trade-offs in interviews, structure your reasoning, and communicate decisions effectively. This synthesizes everything into the practical skill of trade-off articulation.
You now have deep understanding of consistency models in distributed systems. Remember: there's no 'best' consistency level—only the right one for each specific use case. The mark of a senior engineer is knowing when to use which.