When Eventual Consistency Works

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48

// The problem with EC for unique constraints:
 
// Timeline with eventual consistency:
// 
// Time 0: Replica A and Replica B both have no user "alice"
// Time 1: User 1 creates "alice" on Replica A
// Time 2: User 2 creates "alice" on Replica B
//         (Replica B hasn't received User 1's write yet)
// Time 3: Both replicas independently create "alice"
// Time 4: Replication propagates...
//         Now we have TWO users named "alice"!
 
// This is why username registration needs strong consistency:
 
// ❌ WRONG: Eventually consistent check
async function registerUserBad(username: string, userId: string) {
    // Read may be stale!
    const existing = await db.read(`username:${username}`, { consistency: 'ONE' });
    if (existing) {
        throw new Error('Username taken');
    }
    // Another replica might have just registered this username
    await db.write(`username:${username}`, userId, { consistency: 'ONE' });
}
 
// ✅ CORRECT: Linearizable check-and-set
async function registerUserGood(username: string, userId: string) {
    // Atomic conditional write - only succeeds if key doesn't exist
    const result = await db.putIfAbsent(`username:${username}`, userId);
    
    if (!result.success) {
        throw new Error('Username taken');
    }
    // Globally unique - guaranteed by linearizable operation
}
 
// Alternative: Use a strongly consistent service for just usernames
async function registerUserWithStrongService(username: string, userId: string) {
    // Dedicated username service with strong consistency
    const reserved = await strongConsistentNameService.reserve(username, userId);
    
    if (!reserved.success) {
        throw new Error('Username taken');
    }
    
    // Rest of user data can use eventual consistency
    await eventualDb.write(`user:${userId}`, userData);
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63

// Configuration-driven consistency selection
 
const consistencyPolicy = {
    entities: {
        'User': {
            read: 'EVENTUAL',
            write: 'QUORUM',
            fields: {
                'passwordHash': { read: 'QUORUM', write: 'ALL' },
                'email': { write: 'LINEARIZABLE' },  // Unique constraint
                'preferences': { read: 'EVENTUAL', write: 'ONE' }
            }
        },
        'Product': {
            read: 'EVENTUAL',  // Cached reads
            write: 'QUORUM',
            fields: {
                'inventory': { 
                    read: 'EVENTUAL',  // Display purposes
                    decrement: 'SERIALIZABLE'  // Checkout
                }
            }
        },
        'Order': {
            read: 'QUORUM',
            write: 'QUORUM',
            create: 'LINEARIZABLE'  // Ensure unique order IDs
        },
        'Analytics': {
            read: 'ONE',
            write: 'ANY'  // Fire and forget
        }
    }
};
 
class DataAccessLayer {
    async read<T>(entity: string, id: string, options?: ReadOptions): Promise<T> {
        const policy = this.getPolicy(entity, options?.field);
        const consistency = options?.consistency || policy.read;
        
        return await this.db.read(`${entity}:${id}`, { consistency });
    }
    
    async write<T>(
        entity: string, 
        id: string, 
        data: T, 
        options?: WriteOptions
    ): Promise<void> {
        const policy = this.getPolicy(entity, options?.field);
        const consistency = options?.consistency || policy.write;
        
        // Log consistency choice for debugging
        this.metrics.track('data_access', {
            entity,
            operation: 'write',
            consistency,
            explicit: !!options?.consistency
        });
        
        await this.db.write(`${entity}:${id}`, data, { consistency });
    }
}

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36

STEP 1: CATEGORIZE THE DATA
□ What type of data is this? (user data, financial, analytical, configuration)
□ Who modifies this data? (single user, multiple users, system only)
□ How often is it modified? (rarely, occasionally, frequently)
□ What's the expected conflict rate? (near-zero, low, high)
 
STEP 2: ANALYZE REQUIREMENTS
□ What's the cost of a stale read? (negligible, inconvenient, damaging, catastrophic)
□ What's the cost of unavailability? (minor, significant, critical)
□ Is there a latency budget? (>=10ms, ~100ms, ~1s, doesn't matter)
□ Are there legal/compliance requirements? (none, audit trails, real-time accuracy)
 
STEP 3: CHECK FOR EC DEAL-BREAKERS
□ Are there unique constraints that must be globally enforced? → Need strong
□ Is this financial/monetary data? → Likely need strong for mutations
□ Do operations have sequential dependencies? → Consider causal or strong
□ Would stale reads cause irreversible harm? → Need strong
□ Is this security/access control related? → Strongly prefer strong
 
STEP 4: CHECK FOR EC ENABLERS
□ Can the application handle stale reads gracefully? → EC is viable
□ Are operations idempotent or can they be made so? → EC is viable
□ Can conflicts be auto-merged (CRDTs) or user-resolved? → EC is viable
□ Is there a viable compensation strategy? → EC is viable
□ Does the business prefer availability over correctness? → EC preferred
 
STEP 5: DECIDE
If mostly deal-breakers → Strong Consistency
If mostly enablers → Eventual Consistency
If mixed → Hybrid approach (different levels for different operations)
 
STEP 6: DOCUMENT
□ Record the decision and rationale
□ Define monitoring for consistency health
□ Plan for compensating actions if using EC
□ Set up alerts for anomalies