System Design (HLD)Choreography vs Orchestration

Choreography vs Orchestration: Coordination Patterns in Event-Driven Systems

LevelAdvanced

Duration90 mins

TopicChoreography vs Orchestration

5 / 5

Hybrid Approaches — Blending Choreography and Orchestration

Beyond the False Dichotomy

The debate between choreography and orchestration often presents a false dichotomy: you must choose one or the other. In practice, the most successful distributed systems blend both patterns, leveraging orchestration's visibility where it matters most and choreography's decoupling where autonomy is essential.

Hybrid approaches recognize a fundamental truth: different parts of your system have different requirements. Complex order fulfillment might need orchestration's explicit control, while downstream analytics and notifications benefit from choreography's loose coupling. Cross-domain communication might be event-driven (choreography), while within-domain workflows use orchestration.

This page explores proven hybrid patterns, showing you how to combine both approaches effectively. You'll learn to design architectures that capture the benefits of each pattern while minimizing their drawbacks.

What You Will Learn

By the end of this page, you will understand hybrid coordination patterns: orchestration within domains with choreography across domains, event-carried state transfer, the orchestrator-as-publisher pattern, and saga coordination. You'll be able to design architectures that blend both paradigms thoughtfully.

Why Hybrid Approaches

Before diving into patterns, let's understand why pure choreography or pure orchestration often falls short in complex systems.

The Problem with Pure Choreography:

As systems grow, pure choreography creates challenges:

Event chains become hard to follow across many services
Complex branching logic distributes across multiple services
No single view of business process health
Changes to workflow require coordinated changes across teams

The Problem with Pure Orchestration:

Pure orchestration creates its own challenges:

Every service change requires orchestrator coordination
Teams lose autonomy; blocked on orchestrator team
Orchestrator becomes a coupling point for the entire system
Difficult to add independent consumers of events

Hybrid Approaches Solve Both:

By combining patterns, you can:

Use orchestration for complex, frequently-changing core workflows
Use choreography for loose coupling between domains
Maintain team autonomy while preserving workflow visibility
Enable independent consumers without central orchestrator changes

Hybrid Pattern Benefits
Challenge	Pure Choreography	Pure Orchestration	Hybrid Approach
Complex workflow visibility	Poor — distributed	Excellent — centralized	Good — orchestrate core, choreograph periphery
Team autonomy	Excellent	Poor	Good — teams own domains, orchestrator owns flow
Adding new consumers	Excellent	Poor — requires orchestrator change	Good — emit events from orchestrator
Cross-domain decoupling	Excellent	Poor	Good — choreography across domains
Debugging complexity	High	Low	Medium — limited scope for each

No Silver Bullet

Hybrid approaches add complexity by requiring teams to understand both patterns. Use them when the benefits outweigh this cognitive overhead. For simple systems, stick with one pattern. For complex enterprise systems, hybrid approaches often provide the best balance.

Pattern 1 — Orchestration Within, Choreography Across

The most common hybrid pattern: use orchestration within a bounded context (domain), but choreography to communicate across bounded contexts.

How It Works:

Each domain has its own orchestrator for internal workflows
Domains communicate through published events
Other domains react to events independently
No domain tells another what to do; events express what happened

Example: E-commerce Platform

Order Domain: Orchestrates order creation, validation, and state transitions
Payment Domain: Orchestrates payment processing, retry, and refund logic
Fulfillment Domain: Orchestrates picking, packing, shipping

Cross-domain communication is via events:

Order Domain emits OrderConfirmed → Payment Domain starts payment workflow
Payment Domain emits PaymentCompleted → Fulfillment Domain starts fulfillment workflow
Fulfillment Domain emits ShipmentDispatched → Order Domain updates order status

Orchestration Within, Choreography Across
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
// ORDER DOMAIN - Internal orchestration
class OrderDomainOrchestrator {
  async processOrder(input: CreateOrderInput): Promise<Order> {
    // Internal orchestration: Order domain owns this flow
    const result = await this.orderWorkflow.execute({
      steps: [
        validateOrder,
        checkCustomerCredit,
        applyDiscounts,
        calculateTax,
        createOrder,
        // Final step: emit event for other domains
        emitOrderConfirmedEvent,
      ],
      input,
    });
    
    return result.order;
  }
  
  // Event published at end of orchestrated flow
  private async emitOrderConfirmedEvent(data: OrderData): Promise<void> {
    await this.eventBus.publish('OrderConfirmed', {
      eventId: uuid(),
      orderId: data.order.id,
      customerId: data.order.customerId,
      items: data.order.items,
      totalAmount: data.order.total,
      shippingAddress: data.order.shippingAddress,
      timestamp: new Date().toISOString(),
    });
  }
  
  // React to events from other domains (choreography)
  async handlePaymentCompleted(event: PaymentCompletedEvent): Promise<void> {
    // Update order state based on external event
    await this.orderService.markAsPaid(event.orderId);
  }
  
  async handleShipmentDispatched(event: ShipmentDispatchedEvent): Promise<void> {
    await this.orderService.markAsShipped(event.orderId, event.trackingNumber);
  }
}
 
// PAYMENT DOMAIN - Internal orchestration
class PaymentDomainOrchestrator {
  // React to OrderConfirmed event (choreography)
  async handleOrderConfirmed(event: OrderConfirmedEvent): Promise<void> {
    // Start internal payment orchestration
    await this.paymentWorkflow.execute({
      steps: [
        retrievePaymentMethod,
        validatePaymentMethod,
        processCharge,
        handlePaymentResult,
        emitPaymentEvent, // Emit result to other domains
      ],
      input: {
        orderId: event.orderId,
        customerId: event.customerId,
        amount: event.totalAmount,
      },
    });
  }
  
  private async emitPaymentEvent(data: PaymentData): Promise<void> {
    if (data.result.success) {
      await this.eventBus.publish('PaymentCompleted', {
        eventId: uuid(),
        orderId: data.orderId,
        paymentId: data.result.paymentId,
        amount: data.amount,
        timestamp: new Date().toISOString(),
      });
    } else {
      await this.eventBus.publish('PaymentFailed', {
        eventId: uuid(),
        orderId: data.orderId,
        reason: data.result.failureReason,
        retryable: data.result.retryable,
        timestamp: new Date().toISOString(),
      });
    }
  }
}
 
// FULFILLMENT DOMAIN - Internal orchestration
class FulfillmentDomainOrchestrator {
  // React to PaymentCompleted event (choreography)
  async handlePaymentCompleted(event: PaymentCompletedEvent): Promise<void> {
    // Start fulfillment orchestration
    await this.fulfillmentWorkflow.execute({
      steps: [
        selectWarehouse,
        allocateInventory,
        createPickList,
        schedulePacking,
        arrangeShipping,
        emitFulfillmentEvents,
      ],
      input: {
        orderId: event.orderId,
        paymentId: event.paymentId,
      },
    });
  }
}

Benefits of This Pattern

•Domain Autonomy — Each domain team controls their internal workflow without external coordination.
•Clear Boundaries — Domain events are the public interface; internal workflows are private.
•Visible Internal Flows — Within each domain, orchestration provides clear workflow visibility.
•Loose Cross-Domain Coupling — Domains don't call each other directly; they react to events.
•Independent Scaling — Each domain scales its orchestrator independently.

Align with Bounded Contexts

This pattern aligns naturally with Domain-Driven Design. Each bounded context has its own orchestrator for internal complexity. Context boundaries become event boundaries. This prevents domain logic from leaking across boundaries while maintaining workflow clarity within each domain.

Pattern 2 — Orchestrator as Publisher

In this pattern, the orchestrator publishes events at key workflow milestones, enabling independent consumers without requiring orchestrator changes.

How It Works:

Orchestrator controls the core workflow
At significant state changes, orchestrator publishes events
Any service can subscribe to these events
Orchestrator doesn't know or care who subscribes

This gives you orchestration's visibility for the core flow while enabling choreography's extensibility for peripheral concerns.

Orchestrator as Publisher Pattern
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
// Order Orchestrator publishes events for extension
class OrderOrchestrator {
  async executeOrderWorkflow(input: OrderInput): Promise<OrderResult> {
    const ctx = { orderId: uuid(), correlationId: uuid() };
    
    // Step 1: Create Order
    const order = await this.orderService.create(input, ctx);
    await this.publishEvent('OrderCreated', { 
      orderId: ctx.orderId,
      items: order.items,
    });
    
    // Step 2: Process Payment (core workflow continues)
    const payment = await this.paymentService.process({
      orderId: ctx.orderId,
      amount: order.total,
    }, ctx);
    await this.publishEvent('OrderPaymentCompleted', {
      orderId: ctx.orderId,
      paymentId: payment.id,
    });
    
    // Step 3: Reserve Inventory
    const inventory = await this.inventoryService.reserve({
      orderId: ctx.orderId,
      items: order.items,
    }, ctx);
    await this.publishEvent('OrderInventoryReserved', {
      orderId: ctx.orderId,
      reservationId: inventory.id,
    });
    
    // Step 4: Schedule Shipping
    const shipment = await this.shippingService.schedule({
      orderId: ctx.orderId,
      reservationId: inventory.id,
      address: input.shippingAddress,
    }, ctx);
    await this.publishEvent('OrderShipmentScheduled', {
      orderId: ctx.orderId,
      shipmentId: shipment.id,
      trackingNumber: shipment.trackingNumber,
    });
    
    // Step 5: Complete
    await this.orderService.complete(ctx.orderId);
    await this.publishEvent('OrderCompleted', {
      orderId: ctx.orderId,
      completedAt: new Date(),
    });
    
    return { orderId: ctx.orderId, status: 'completed' };
  }
}
 
// INDEPENDENT CONSUMERS - Don't need orchestrator changes
 
// Analytics Service - tracks conversion funnels
class AnalyticsConsumer {
  @Subscribe('OrderCreated')
  async onOrderCreated(event: OrderCreatedEvent) {
    await this.analytics.trackFunnelStep('order_created', event.orderId);
  }
  
  @Subscribe('OrderCompleted')
  async onOrderCompleted(event: OrderCompletedEvent) {
    await this.analytics.trackConversion(event.orderId, event.completedAt);
  }
}
 
// Marketing Service - triggers campaigns
class MarketingConsumer {
  @Subscribe('OrderCompleted')
  async onOrderCompleted(event: OrderCompletedEvent) {
    const order = await this.orderClient.getOrder(event.orderId);
    
    if (order.isFirstOrder) {
      await this.campaigns.sendWelcomeSequence(order.customerId);
    }
    
    await this.campaigns.requestReview(order.customerId, order.orderId);
  }
}
 
// Loyalty Service - awards points
class LoyaltyConsumer {
  @Subscribe('OrderPaymentCompleted')
  async onPaymentCompleted(event: OrderPaymentCompletedEvent) {
    const order = await this.orderClient.getOrder(event.orderId);
    const points = this.calculatePoints(order.total);
    await this.loyaltyService.awardPoints(order.customerId, points);
  }
}
 
// Fraud Detection - real-time monitoring
class FraudMonitoringConsumer {
  @Subscribe('OrderCreated')
  async onOrderCreated(event: OrderCreatedEvent) {
    const order = await this.orderClient.getOrder(event.orderId);
    await this.fraudDetection.analyzeOrder(order);
  }
  
  @Subscribe('OrderPaymentCompleted')
  async onPaymentCompleted(event: OrderPaymentCompletedEvent) {
    await this.fraudDetection.recordPaymentSuccess(event.orderId);
  }
}

Key Insight: The orchestrator defines and controls the core business workflow. But it also acts as an event source, enabling an unlimited number of peripheral services to react without modifying the orchestrator.

When new requirements arise:

Need to send SMS notifications? Subscribe a new consumer to OrderShipmentScheduled.
Need to integrate with a new analytics tool? Subscribe to all order events.
Need to trigger warehouse automation? Subscribe to OrderInventoryReserved.

None of these require orchestrator changes. The core workflow remains stable and visible; extensions happen through event subscription.

Design Events for Extended Consumption

When using this pattern, design events with unknown consumers in mind. Include enough context that typical consumers don't need to call back to the orchestrator. Consider what analytics, marketing, or compliance teams might need from each event.

Pattern 3 — Saga Coordinator Pattern

In complex distributed transactions, a Saga Coordinator provides visibility and control over a choreographed saga without fully centralizing it.

How It Works:

Each service still reacts to events (choreography)
But a dedicated Saga Coordinator tracks the overall saga state
The coordinator doesn't invoke services—it observes events
It provides visibility, timeout handling, and intervention capabilities

This gives you choreography's loose coupling with orchestration's observability.

Saga Coordinator Pattern
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
// Saga Coordinator: Observes but doesn't control
 
interface SagaDefinition {
  name: string;
  expectedEvents: EventExpectation[];
  timeout: Duration;
}
 
interface EventExpectation {
  eventType: string;
  afterEvent?: string;
  timeout: Duration;
  optional: boolean;
}
 
const orderSagaDefinition: SagaDefinition = {
  name: 'OrderProcessing',
  expectedEvents: [
    { eventType: 'OrderCreated', timeout: '1m', optional: false },
    { eventType: 'InventoryReserved', afterEvent: 'OrderCreated', timeout: '5m', optional: false },
    { eventType: 'PaymentCompleted', afterEvent: 'InventoryReserved', timeout: '5m', optional: false },
    { eventType: 'ShipmentScheduled', afterEvent: 'PaymentCompleted', timeout: '30m', optional: false },
    { eventType: 'OrderCompleted', afterEvent: 'ShipmentScheduled', timeout: '1m', optional: false },
  ],
  timeout: '1h',
};
 
class SagaCoordinator {
  constructor(
    private readonly sagaStore: SagaStateStore,
    private readonly alerting: AlertingService,
  ) {}
  
  // Called when any saga-related event is observed
  async observeEvent(event: DomainEvent): Promise<void> {
    const saga = await this.findOrCreateSaga(event);
    
    // Record event observation
    await this.sagaStore.recordEvent(saga.id, {
      eventType: event.eventType,
      eventId: event.eventId,
      observedAt: new Date(),
    });
    
    // Check if saga is complete
    if (this.isComplete(saga, event)) {
      await this.completeSaga(saga.id);
      return;
    }
    
    // Update expected next events and timeouts
    await this.updateExpectations(saga, event);
  }
  
  // Background job: Check for stuck sagas
  async checkTimeouts(): Promise<void> {
    const stuckSagas = await this.sagaStore.findSagasWithExpiredTimeouts();
    
    for (const saga of stuckSagas) {
      const missingEvent = this.findMissingEvent(saga);
      
      // Alert operations team
      await this.alerting.send({
        severity: 'high',
        title: 'Saga Timeout',
        message: `Order ${saga.correlationId} stuck waiting for ${missingEvent}`,
        saga: saga,
        suggestedActions: this.getSuggestedActions(saga, missingEvent),
      });
      
      // Optionally: Trigger compensation
      if (saga.autoCompensateOnTimeout) {
        await this.triggerCompensation(saga);
      }
    }
  }
  
  // Compensation: Emit compensating command events
  async triggerCompensation(saga: SagaState): Promise<void> {
    // The coordinator emits compensation REQUEST events
    // Services still decide how to compensate
    
    const completedSteps = saga.observedEvents.map(e => e.eventType);
    
    // Emit compensation requests in reverse order
    if (completedSteps.includes('ShipmentScheduled')) {
      await this.eventBus.publish('CompensationRequested', {
        type: 'CancelShipment',
        sagaId: saga.id,
        correlationId: saga.correlationId,
        reason: saga.compensationReason,
      });
    }
    
    if (completedSteps.includes('PaymentCompleted')) {
      await this.eventBus.publish('CompensationRequested', {
        type: 'RefundPayment',
        sagaId: saga.id,
        correlationId: saga.correlationId,
        reason: saga.compensationReason,
      });
    }
    
    // ... continue for other steps
    
    await this.sagaStore.markCompensating(saga.id);
  }
  
  // Dashboard API: Get saga state for debugging
  async getSagaStatus(correlationId: string): Promise<SagaStatusView> {
    const saga = await this.sagaStore.findByCorrelationId(correlationId);
    
    return {
      id: saga.id,
      correlationId: saga.correlationId,
      status: saga.status,
      observedEvents: saga.observedEvents,
      pendingEvent: this.getNextExpectedEvent(saga),
      timeUntilTimeout: this.getTimeUntilTimeout(saga),
      canCompensate: this.canCompensate(saga),
      compensationActions: this.getAvailableCompensations(saga),
    };
  }
}
 
// Services remain loosely coupled - react to events
class ShippingService {
  @Subscribe('PaymentCompleted')
  async onPaymentCompleted(event: PaymentCompletedEvent) {
    // Still choreography: service reacts to event
    const shipment = await this.scheduleShipment(event.orderId);
    
    // Emit completion event
    await this.eventBus.publish('ShipmentScheduled', {
      orderId: event.orderId,
      shipmentId: shipment.id,
      trackingNumber: shipment.trackingNumber,
    });
  }
  
  @Subscribe('CompensationRequested')
  async onCompensationRequested(event: CompensationRequestedEvent) {
    if (event.type !== 'CancelShipment') return;
    
    await this.cancelShipment(event.correlationId);
    
    await this.eventBus.publish('ShipmentCancelled', {
      correlationId: event.correlationId,
      sagaId: event.sagaId,
    });
  }
}

Saga Coordinator Benefits

•Visibility Without Control — You can see saga state and history without centralizing invocation logic.
•Timeout Handling — Coordinator can detect stuck sagas and alert or compensate.
•Debugging Support — Single place to query saga status for any correlation ID.
•Loose Coupling Maintained — Services still react to events; coordinator only observes.
•Optional Compensation Coordination — Coordinator can trigger compensation while letting services decide how to compensate.

Pattern 4 — Event-Carried State Transfer

Event-Carried State Transfer is a hybrid pattern where events carry enough data that consumers don't need to call back to the source. It enables choreography's loose coupling while avoiding the callback round-trips that often re-introduce coupling.

The Problem:

In naive choreography, events often carry minimal data:

{ eventType: 'OrderCreated', orderId: '123' }

Consumers must then call the Order Service to get order details, creating runtime coupling:

Consumer availability depends on Order Service availability
Consumer needs to know Order Service API
Additional network round-trip adds latency

The Solution:

Include enough state in the event that consumers are self-sufficient:

{ eventType: 'OrderCreated', orderId: '123', items: [...], customer: {...}, total: 150 }

Now consumers have what they need without callbacks.

Event-Carried State Transfer
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
// MINIMAL EVENT (Requires Callbacks)
interface OrderCreatedEventMinimal {
  eventType: 'OrderCreated';
  orderId: string;
  timestamp: string;
}
 
// Consumer must fetch data
class ShippingServiceWithCallbacks {
  async onOrderCreated(event: OrderCreatedEventMinimal) {
    // Must call back to Order Service
    const order = await this.orderClient.getOrder(event.orderId);
    // Now we can work...
    await this.prepareShipping(order);
  }
}
 
// ENRICHED EVENT (Self-Sufficient Consumers)
interface OrderCreatedEventEnriched {
  eventType: 'OrderCreated';
  eventId: string;
  timestamp: string;
  correlationId: string;
  
  // Order data
  orderId: string;
  orderNumber: string;
  createdAt: string;
  
  // Items - enough for inventory, shipping, analytics
  items: Array<{
    productId: string;
    sku: string;
    name: string;
    quantity: number;
    unitPrice: number;
    weight: number;
    dimensions: Dimensions;
  }>;
  
  // Customer info - enough for fraud, marketing, notifications
  customer: {
    id: string;
    email: string;
    name: string;
    tier: 'standard' | 'premium' | 'vip';
    isFirstOrder: boolean;
    totalLifetimeValue: number;
  };
  
  // Shipping - enough for fulfillment
  shippingAddress: {
    street: string;
    city: string;
    state: string;
    country: string;
    postalCode: string;
  };
  
  // Totals - enough for payment, analytics
  subtotal: number;
  tax: number;
  shipping: number;
  total: number;
  currency: string;
  
  // Business context - enough for various decisions
  channel: 'web' | 'mobile' | 'api';
  promotionCodes: string[];
  giftMessage?: string;
}
 
// Consumer is self-sufficient
class ShippingServiceSelfSufficient {
  async onOrderCreated(event: OrderCreatedEventEnriched) {
    // No callbacks needed - everything is in the event
    
    const shippingDetails = {
      destination: event.shippingAddress,
      items: event.items.map(item => ({
        sku: item.sku,
        quantity: item.quantity,
        weight: item.weight,
        dimensions: item.dimensions,
      })),
      priority: event.customer.tier === 'vip' ? 'express' : 'standard',
    };
    
    await this.prepareShipping(shippingDetails);
  }
}
 
// Multiple consumers can work independently
class AnalyticsServiceSelfSufficient {
  async onOrderCreated(event: OrderCreatedEventEnriched) {
    await this.track({
      event: 'order_created',
      orderId: event.orderId,
      customerId: event.customer.id,
      orderValue: event.total,
      itemCount: event.items.length,
      channel: event.channel,
      isFirstOrder: event.customer.isFirstOrder,
      customerTier: event.customer.tier,
    });
  }
}
 
class FraudServiceSelfSufficient {
  async onOrderCreated(event: OrderCreatedEventEnriched) {
    const riskScore = await this.assessRisk({
      orderValue: event.total,
      customerLifetimeValue: event.customer.totalLifetimeValue,
      isNewCustomer: event.customer.isFirstOrder,
      shippingCountry: event.shippingAddress.country,
      itemCategories: event.items.map(i => this.getCategory(i.productId)),
    });
    
    if (riskScore > 0.8) {
      await this.flagForReview(event.orderId, riskScore);
    }
  }
}

Trade-offs of Event-Carried State Transfer:

Pros:

True decoupling: consumers don't depend on source availability
Lower latency: no callback round-trips
Simpler consumer code: data is right there

Cons:

Larger event payloads: more data to serialize and transport
Potential data duplication: same data in events and databases
Consistency window: event data is a snapshot; may be stale

When to Use:

Consumers need event data for decisions, not just IDs
Source service has high latency or availability concerns
You want to minimize cross-service runtime dependencies
Event size is manageable (typically < 1 MB)

Don't Include Sensitive Data

Event-carried state transfer means many consumers see the event data. Never include raw payment credentials, SSNs, or other sensitive PII that shouldn't propagate. Mask or tokenize sensitive fields. Consider what happens if the event is logged, stored in dead letter queues, or accessed by future consumers you haven't anticipated.

Pattern 5 — Layered Coordination

In complex systems, coordination needs vary by layer. Layered Coordination applies different patterns at different architectural levels.

Common Layering:

Layer 1: API Gateway / Edge

Synchronous request-response
Gateway routes to appropriate orchestrator

Layer 2: Domain Orchestrators

Orchestration within each domain
Controls complex domain workflows

Layer 3: Cross-Domain Events

Choreography between domains
Events carry state for consumer independence

Layer 4: Integration / External Systems

Orchestration for external API calls
Handles retries, timeouts, circuit breakers

Layered Coordination Architecture
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
// LAYER 1: API Gateway
class OrderApiController {
  @Post('/orders')
  async createOrder(@Body() req: CreateOrderRequest): Promise<OrderResponse> {
    // Gateway invokes the appropriate domain orchestrator
    const result = await this.orderOrchestrator.createOrder({
      ...req,
      correlationId: req.headers['x-correlation-id'] || uuid(),
    });
    
    return { orderId: result.orderId, status: result.status };
  }
}
 
// LAYER 2: Domain Orchestrator
class OrderDomainOrchestrator {
  async createOrder(input: CreateOrderInput): Promise<OrderCreationResult> {
    // Orchestrate within the Order domain
    return this.workflow.execute({
      input,
      steps: [
        // Internal orchestration
        validateOrderRequest,
        enrichCustomerData,
        calculatePricing,
        applyPromotions,
        persistOrder,
        
        // Emit event for Layer 3 (cross-domain choreography)
        publishOrderCreatedEvent,
      ],
    });
  }
}
 
// LAYER 3: Cross-Domain Event Handlers
class PaymentDomainEventHandler {
  @Subscribe('OrderCreated')
  async onOrderCreated(event: OrderCreatedEvent): Promise<void> {
    // Choreography: react to event from another domain
    // Then orchestrate within Payment domain
    await this.paymentOrchestrator.processPayment({
      orderId: event.orderId,
      amount: event.total,
      customerId: event.customer.id,
    });
  }
}
 
// LAYER 4: External Integration Orchestrator
class PaymentGatewayOrchestrator {
  async processPayment(input: PaymentInput): Promise<PaymentResult> {
    // Orchestrate external payment processing
    return this.workflow.execute({
      input,
      steps: [
        selectPaymentGateway,        // Choose Stripe, Braintree, etc.
        preparePaymentRequest,
        executePaymentWithRetry,      // Retry logic for external API
        handlePaymentResponse,
        persistPaymentRecord,
        publishPaymentResult,
      ],
      
      // External API-specific configuration
      retryPolicy: {
        maxAttempts: 3,
        backoff: 'exponential',
        circuitBreaker: {
          threshold: 5,
          resetAfter: '30s',
        },
      },
      
      timeout: '30s',
    });
  }
}
 
// ARCHITECTURE OVERVIEW
const layeredCoordinationArchitecture = {
  layer1_edge: {
    pattern: 'Request-Response',
    components: ['API Gateway', 'Load Balancer', 'Auth'],
    responsibility: 'Route requests to domain orchestrators',
  },
  
  layer2_domainOrchestration: {
    pattern: 'Orchestration',
    components: ['Order Orchestrator', 'Payment Orchestrator', 'Fulfillment Orchestrator'],
    responsibility: 'Control complex workflows within bounded contexts',
  },
  
  layer3_crossDomainEvents: {
    pattern: 'Choreography',
    components: ['Event Bus', 'Schema Registry', 'Event Store'],
    responsibility: 'Loose coupling between domains via events',
  },
  
  layer4_externalIntegration: {
    pattern: 'Orchestration',
    components: ['Integration Orchestrators', 'Circuit Breakers', 'Retry Handlers'],
    responsibility: 'Reliable communication with external systems',
  },
};

Layered Coordination Benefits

•Right Tool for Each Layer — Apply patterns where they fit best; don't force one pattern everywhere.
•Clear Separation of Concerns — Each layer handles different types of coordination complexity.
•Isolation of External Dependencies — External system complexity contained in integration layer.
•Domain Independence — Domains communicate through events; internal complexity is hidden.
•Scalable Architecture — Each layer can scale independently based on its characteristics.

Match Patterns to Failure Modes

Each layer has different failure characteristics. External APIs fail differently than internal services. Cross-domain events have different reliability than internal orchestration. Choose patterns that handle each layer's specific failure modes appropriately.

Implementing Hybrid Architectures

Implementing hybrid patterns requires careful attention to infrastructure, team practices, and operational considerations.

Infrastructure Requirements:

For Choreography Layers:

Event bus (Kafka, RabbitMQ, AWS SNS/SQS)
Schema registry for event contract management
Dead letter queues for failed event processing
Distributed tracing for event correlation

For Orchestration Layers:

Workflow engine (Temporal, Camunda, Step Functions) or custom orchestrator
State store (PostgreSQL, Redis, DynamoDB)
Activity workers with retry capabilities
Workflow monitoring and alerting

Hybrid Infrastructure Setup
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
// Hybrid Architecture Configuration
 
interface HybridArchitectureConfig {
  orchestration: {
    engine: 'temporal' | 'step-functions' | 'custom';
    stateStore: DatabaseConfig;
    workerConfig: WorkerPoolConfig;
  };
  
  choreography: {
    eventBus: 'kafka' | 'rabbitmq' | 'sns-sqs';
    schemaRegistry: 'confluent' | 'apicurio' | 'custom';
    deadLetterConfig: DLQConfig;
  };
  
  observability: {
    tracing: 'jaeger' | 'zipkin' | 'xray';
    logging: 'elasticsearch' | 'cloudwatch' | 'datadog';
    metrics: 'prometheus' | 'cloudwatch' | 'datadog';
  };
}
 
const productionConfig: HybridArchitectureConfig = {
  orchestration: {
    engine: 'temporal',
    stateStore: {
      type: 'postgresql',
      host: 'temporal-db.internal',
      replication: true,
    },
    workerConfig: {
      minWorkers: 5,
      maxWorkers: 50,
      scaleMetric: 'pending_workflows',
    },
  },
  
  choreography: {
    eventBus: 'kafka',
    schemaRegistry: 'confluent',
    deadLetterConfig: {
      topic: 'dlq.events',
      retentionDays: 30,
      alertThreshold: 100,
    },
  },
  
  observability: {
    tracing: 'jaeger',
    logging: 'elasticsearch',
    metrics: 'prometheus',
  },
};
 
// Team Practices for Hybrid Architecture
 
const teamPractices = {
  eventDesign: {
    // All events must be registered in schema registry
    schemaRegistration: 'required',
    // Events must include correlation ID for tracing
    correlationIdRequired: true,
    // Events must be versioned
    versioningStrategy: 'semantic',
    // Breaking changes require compatibility check
    compatibilityMode: 'backward',
  },
  
  orchestratorOwnership: {
    // Each domain owns its orchestrator
    ownershipModel: 'domain-aligned',
    // Orchestrator changes go through domain team
    changeApproval: 'domain-team',
    // Cross-domain impacts require architecture review
    crossDomainChanges: 'architecture-review',
  },
  
  eventOwnership: {
    // Producer owns the event schema
    schemaOwnership: 'producer',
    // Consumers test against producer contracts
    contractTesting: 'consumer-driven',
    // Schema changes require consumer notification
    changeNotification: 'schema-registry-webhook',
  },
  
  debuggingPractices: {
    // Correlation ID must flow through all layers
    correlationIdPropagation: 'mandatory',
    // All events logged to centralized system
    eventLogging: 'centralized',
    // Saga view service aggregates state
    sagaTracking: 'saga-coordinator',
  },
};

Implementation Best Practices

•Start with Clear Boundaries — Define which workflows use orchestration vs choreography before implementation. Document the reasoning.
•Establish Consistent Correlation IDs — Every request, event, and workflow instance must carry the same correlation ID for end-to-end tracing.
•Use Schema Registry from Day One — Don't let event schemas drift. Enforce registration and compatibility checks.
•Create Saga View Service Early — For choreography layers, build visibility tooling before you need it for debugging.
•Document Event Ownership — Every event has a producing team. Consumers know who to contact for changes.
•Test Across Boundaries — Integration tests should cover orchestrator-to-event and event-to-orchestrator transitions.

Iterate Toward Hybrid

You don't need to implement the full hybrid architecture immediately. Start with one pattern, add the other where needed. Many systems evolve: start with orchestration for visibility, add event publication for extension, discover domains that want autonomy and move them to internal choreography.

Summary: Blending Patterns Effectively

We've explored hybrid approaches that combine choreography and orchestration. Let's consolidate the key insights:

Key Takeaways

•Hybrid approaches are often optimal — Pure choreography or pure orchestration each have drawbacks that hybrid patterns address.
•Orchestration within, choreography across — Use orchestration inside domains for visibility; choreography across domains for loose coupling.
•Orchestrator as publisher — Orchestrators can emit events, enabling independent consumers without orchestrator changes.
•Saga coordinator provides visibility — A coordinator can observe choreographed sagas without controlling them, providing debugging and timeout handling.
•Event-carried state enables independence — Rich events let consumers work without callbacks to source services.
•Layered coordination matches patterns to needs — Different architectural layers have different coordination requirements.
•Implementation requires infrastructure investment — Both event bus and workflow engine capabilities are needed for effective hybrid architectures.

Module Complete:

You've completed the in-depth exploration of Choreography vs Orchestration. You now understand:

Choreography: Decentralized, event-reactive coordination with loose coupling
Orchestration: Centralized, explicit coordination with clear visibility
Sagas: Distributed transaction management through both patterns
Trade-offs: How to evaluate which pattern fits your context
Hybrid approaches: Patterns that combine the strengths of both

These patterns form the foundation for coordinating work in distributed systems. Whether you're building microservices, event-driven architectures, or complex workflows, you now have the knowledge to choose and implement the right coordination approach for your specific needs.

Module Complete

Congratulations! You've mastered the coordination patterns for event-driven architectures. You understand choreography, orchestration, sagas, and hybrid approaches—and most importantly, you know when to use each. Apply these patterns thoughtfully in your systems, always evaluating trade-offs in your specific context.

5 / 5

Loading learning content...

System Design (HLD)Choreography vs Orchestration

Choreography vs Orchestration: Coordination Patterns in Event-Driven Systems

LevelAdvanced

Duration90 mins

TopicChoreography vs Orchestration

5 / 5

Hybrid Approaches — Blending Choreography and Orchestration

Beyond the False Dichotomy

What You Will Learn

Why Hybrid Approaches

Before diving into patterns, let's understand why pure choreography or pure orchestration often falls short in complex systems.

The Problem with Pure Choreography:

As systems grow, pure choreography creates challenges:

Event chains become hard to follow across many services
Complex branching logic distributes across multiple services
No single view of business process health
Changes to workflow require coordinated changes across teams

The Problem with Pure Orchestration:

Pure orchestration creates its own challenges:

Every service change requires orchestrator coordination
Teams lose autonomy; blocked on orchestrator team
Orchestrator becomes a coupling point for the entire system
Difficult to add independent consumers of events

Hybrid Approaches Solve Both:

By combining patterns, you can:

Use orchestration for complex, frequently-changing core workflows
Use choreography for loose coupling between domains
Maintain team autonomy while preserving workflow visibility
Enable independent consumers without central orchestrator changes

Hybrid Pattern Benefits
Challenge	Pure Choreography	Pure Orchestration	Hybrid Approach
Complex workflow visibility	Poor — distributed	Excellent — centralized	Good — orchestrate core, choreograph periphery
Team autonomy	Excellent	Poor	Good — teams own domains, orchestrator owns flow
Adding new consumers	Excellent	Poor — requires orchestrator change	Good — emit events from orchestrator
Cross-domain decoupling	Excellent	Poor	Good — choreography across domains
Debugging complexity	High	Low	Medium — limited scope for each

No Silver Bullet

Pattern 1 — Orchestration Within, Choreography Across

The most common hybrid pattern: use orchestration within a bounded context (domain), but choreography to communicate across bounded contexts.

How It Works:

Each domain has its own orchestrator for internal workflows
Domains communicate through published events
Other domains react to events independently
No domain tells another what to do; events express what happened

Example: E-commerce Platform

Order Domain: Orchestrates order creation, validation, and state transitions
Payment Domain: Orchestrates payment processing, retry, and refund logic
Fulfillment Domain: Orchestrates picking, packing, shipping

Cross-domain communication is via events:

Order Domain emits OrderConfirmed → Payment Domain starts payment workflow
Payment Domain emits PaymentCompleted → Fulfillment Domain starts fulfillment workflow
Fulfillment Domain emits ShipmentDispatched → Order Domain updates order status

Orchestration Within, Choreography Across
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
// ORDER DOMAIN - Internal orchestration
class OrderDomainOrchestrator {
  async processOrder(input: CreateOrderInput): Promise<Order> {
    // Internal orchestration: Order domain owns this flow
    const result = await this.orderWorkflow.execute({
      steps: [
        validateOrder,
        checkCustomerCredit,
        applyDiscounts,
        calculateTax,
        createOrder,
        // Final step: emit event for other domains
        emitOrderConfirmedEvent,
      ],
      input,
    });
    
    return result.order;
  }
  
  // Event published at end of orchestrated flow
  private async emitOrderConfirmedEvent(data: OrderData): Promise<void> {
    await this.eventBus.publish('OrderConfirmed', {
      eventId: uuid(),
      orderId: data.order.id,
      customerId: data.order.customerId,
      items: data.order.items,
      totalAmount: data.order.total,
      shippingAddress: data.order.shippingAddress,
      timestamp: new Date().toISOString(),
    });
  }
  
  // React to events from other domains (choreography)
  async handlePaymentCompleted(event: PaymentCompletedEvent): Promise<void> {
    // Update order state based on external event
    await this.orderService.markAsPaid(event.orderId);
  }
  
  async handleShipmentDispatched(event: ShipmentDispatchedEvent): Promise<void> {
    await this.orderService.markAsShipped(event.orderId, event.trackingNumber);
  }
}
 
// PAYMENT DOMAIN - Internal orchestration
class PaymentDomainOrchestrator {
  // React to OrderConfirmed event (choreography)
  async handleOrderConfirmed(event: OrderConfirmedEvent): Promise<void> {
    // Start internal payment orchestration
    await this.paymentWorkflow.execute({
      steps: [
        retrievePaymentMethod,
        validatePaymentMethod,
        processCharge,
        handlePaymentResult,
        emitPaymentEvent, // Emit result to other domains
      ],
      input: {
        orderId: event.orderId,
        customerId: event.customerId,
        amount: event.totalAmount,
      },
    });
  }
  
  private async emitPaymentEvent(data: PaymentData): Promise<void> {
    if (data.result.success) {
      await this.eventBus.publish('PaymentCompleted', {
        eventId: uuid(),
        orderId: data.orderId,
        paymentId: data.result.paymentId,
        amount: data.amount,
        timestamp: new Date().toISOString(),
      });
    } else {
      await this.eventBus.publish('PaymentFailed', {
        eventId: uuid(),
        orderId: data.orderId,
        reason: data.result.failureReason,
        retryable: data.result.retryable,
        timestamp: new Date().toISOString(),
      });
    }
  }
}
 
// FULFILLMENT DOMAIN - Internal orchestration
class FulfillmentDomainOrchestrator {
  // React to PaymentCompleted event (choreography)
  async handlePaymentCompleted(event: PaymentCompletedEvent): Promise<void> {
    // Start fulfillment orchestration
    await this.fulfillmentWorkflow.execute({
      steps: [
        selectWarehouse,
        allocateInventory,
        createPickList,
        schedulePacking,
        arrangeShipping,
        emitFulfillmentEvents,
      ],
      input: {
        orderId: event.orderId,
        paymentId: event.paymentId,
      },
    });
  }
}

Benefits of This Pattern

•Domain Autonomy — Each domain team controls their internal workflow without external coordination.
•Clear Boundaries — Domain events are the public interface; internal workflows are private.
•Visible Internal Flows — Within each domain, orchestration provides clear workflow visibility.
•Loose Cross-Domain Coupling — Domains don't call each other directly; they react to events.
•Independent Scaling — Each domain scales its orchestrator independently.

Align with Bounded Contexts

Pattern 2 — Orchestrator as Publisher

In this pattern, the orchestrator publishes events at key workflow milestones, enabling independent consumers without requiring orchestrator changes.

How It Works:

Orchestrator controls the core workflow
At significant state changes, orchestrator publishes events
Any service can subscribe to these events
Orchestrator doesn't know or care who subscribes

This gives you orchestration's visibility for the core flow while enabling choreography's extensibility for peripheral concerns.

Orchestrator as Publisher Pattern
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
// Order Orchestrator publishes events for extension
class OrderOrchestrator {
  async executeOrderWorkflow(input: OrderInput): Promise<OrderResult> {
    const ctx = { orderId: uuid(), correlationId: uuid() };
    
    // Step 1: Create Order
    const order = await this.orderService.create(input, ctx);
    await this.publishEvent('OrderCreated', { 
      orderId: ctx.orderId,
      items: order.items,
    });
    
    // Step 2: Process Payment (core workflow continues)
    const payment = await this.paymentService.process({
      orderId: ctx.orderId,
      amount: order.total,
    }, ctx);
    await this.publishEvent('OrderPaymentCompleted', {
      orderId: ctx.orderId,
      paymentId: payment.id,
    });
    
    // Step 3: Reserve Inventory
    const inventory = await this.inventoryService.reserve({
      orderId: ctx.orderId,
      items: order.items,
    }, ctx);
    await this.publishEvent('OrderInventoryReserved', {
      orderId: ctx.orderId,
      reservationId: inventory.id,
    });
    
    // Step 4: Schedule Shipping
    const shipment = await this.shippingService.schedule({
      orderId: ctx.orderId,
      reservationId: inventory.id,
      address: input.shippingAddress,
    }, ctx);
    await this.publishEvent('OrderShipmentScheduled', {
      orderId: ctx.orderId,
      shipmentId: shipment.id,
      trackingNumber: shipment.trackingNumber,
    });
    
    // Step 5: Complete
    await this.orderService.complete(ctx.orderId);
    await this.publishEvent('OrderCompleted', {
      orderId: ctx.orderId,
      completedAt: new Date(),
    });
    
    return { orderId: ctx.orderId, status: 'completed' };
  }
}
 
// INDEPENDENT CONSUMERS - Don't need orchestrator changes
 
// Analytics Service - tracks conversion funnels
class AnalyticsConsumer {
  @Subscribe('OrderCreated')
  async onOrderCreated(event: OrderCreatedEvent) {
    await this.analytics.trackFunnelStep('order_created', event.orderId);
  }
  
  @Subscribe('OrderCompleted')
  async onOrderCompleted(event: OrderCompletedEvent) {
    await this.analytics.trackConversion(event.orderId, event.completedAt);
  }
}
 
// Marketing Service - triggers campaigns
class MarketingConsumer {
  @Subscribe('OrderCompleted')
  async onOrderCompleted(event: OrderCompletedEvent) {
    const order = await this.orderClient.getOrder(event.orderId);
    
    if (order.isFirstOrder) {
      await this.campaigns.sendWelcomeSequence(order.customerId);
    }
    
    await this.campaigns.requestReview(order.customerId, order.orderId);
  }
}
 
// Loyalty Service - awards points
class LoyaltyConsumer {
  @Subscribe('OrderPaymentCompleted')
  async onPaymentCompleted(event: OrderPaymentCompletedEvent) {
    const order = await this.orderClient.getOrder(event.orderId);
    const points = this.calculatePoints(order.total);
    await this.loyaltyService.awardPoints(order.customerId, points);
  }
}
 
// Fraud Detection - real-time monitoring
class FraudMonitoringConsumer {
  @Subscribe('OrderCreated')
  async onOrderCreated(event: OrderCreatedEvent) {
    const order = await this.orderClient.getOrder(event.orderId);
    await this.fraudDetection.analyzeOrder(order);
  }
  
  @Subscribe('OrderPaymentCompleted')
  async onPaymentCompleted(event: OrderPaymentCompletedEvent) {
    await this.fraudDetection.recordPaymentSuccess(event.orderId);
  }
}

When new requirements arise:

Need to send SMS notifications? Subscribe a new consumer to OrderShipmentScheduled.
Need to integrate with a new analytics tool? Subscribe to all order events.
Need to trigger warehouse automation? Subscribe to OrderInventoryReserved.

None of these require orchestrator changes. The core workflow remains stable and visible; extensions happen through event subscription.

Design Events for Extended Consumption

Pattern 3 — Saga Coordinator Pattern

In complex distributed transactions, a Saga Coordinator provides visibility and control over a choreographed saga without fully centralizing it.

How It Works:

Each service still reacts to events (choreography)
But a dedicated Saga Coordinator tracks the overall saga state
The coordinator doesn't invoke services—it observes events
It provides visibility, timeout handling, and intervention capabilities

This gives you choreography's loose coupling with orchestration's observability.

Saga Coordinator Pattern
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
// Saga Coordinator: Observes but doesn't control
 
interface SagaDefinition {
  name: string;
  expectedEvents: EventExpectation[];
  timeout: Duration;
}
 
interface EventExpectation {
  eventType: string;
  afterEvent?: string;
  timeout: Duration;
  optional: boolean;
}
 
const orderSagaDefinition: SagaDefinition = {
  name: 'OrderProcessing',
  expectedEvents: [
    { eventType: 'OrderCreated', timeout: '1m', optional: false },
    { eventType: 'InventoryReserved', afterEvent: 'OrderCreated', timeout: '5m', optional: false },
    { eventType: 'PaymentCompleted', afterEvent: 'InventoryReserved', timeout: '5m', optional: false },
    { eventType: 'ShipmentScheduled', afterEvent: 'PaymentCompleted', timeout: '30m', optional: false },
    { eventType: 'OrderCompleted', afterEvent: 'ShipmentScheduled', timeout: '1m', optional: false },
  ],
  timeout: '1h',
};
 
class SagaCoordinator {
  constructor(
    private readonly sagaStore: SagaStateStore,
    private readonly alerting: AlertingService,
  ) {}
  
  // Called when any saga-related event is observed
  async observeEvent(event: DomainEvent): Promise<void> {
    const saga = await this.findOrCreateSaga(event);
    
    // Record event observation
    await this.sagaStore.recordEvent(saga.id, {
      eventType: event.eventType,
      eventId: event.eventId,
      observedAt: new Date(),
    });
    
    // Check if saga is complete
    if (this.isComplete(saga, event)) {
      await this.completeSaga(saga.id);
      return;
    }
    
    // Update expected next events and timeouts
    await this.updateExpectations(saga, event);
  }
  
  // Background job: Check for stuck sagas
  async checkTimeouts(): Promise<void> {
    const stuckSagas = await this.sagaStore.findSagasWithExpiredTimeouts();
    
    for (const saga of stuckSagas) {
      const missingEvent = this.findMissingEvent(saga);
      
      // Alert operations team
      await this.alerting.send({
        severity: 'high',
        title: 'Saga Timeout',
        message: `Order ${saga.correlationId} stuck waiting for ${missingEvent}`,
        saga: saga,
        suggestedActions: this.getSuggestedActions(saga, missingEvent),
      });
      
      // Optionally: Trigger compensation
      if (saga.autoCompensateOnTimeout) {
        await this.triggerCompensation(saga);
      }
    }
  }
  
  // Compensation: Emit compensating command events
  async triggerCompensation(saga: SagaState): Promise<void> {
    // The coordinator emits compensation REQUEST events
    // Services still decide how to compensate
    
    const completedSteps = saga.observedEvents.map(e => e.eventType);
    
    // Emit compensation requests in reverse order
    if (completedSteps.includes('ShipmentScheduled')) {
      await this.eventBus.publish('CompensationRequested', {
        type: 'CancelShipment',
        sagaId: saga.id,
        correlationId: saga.correlationId,
        reason: saga.compensationReason,
      });
    }
    
    if (completedSteps.includes('PaymentCompleted')) {
      await this.eventBus.publish('CompensationRequested', {
        type: 'RefundPayment',
        sagaId: saga.id,
        correlationId: saga.correlationId,
        reason: saga.compensationReason,
      });
    }
    
    // ... continue for other steps
    
    await this.sagaStore.markCompensating(saga.id);
  }
  
  // Dashboard API: Get saga state for debugging
  async getSagaStatus(correlationId: string): Promise<SagaStatusView> {
    const saga = await this.sagaStore.findByCorrelationId(correlationId);
    
    return {
      id: saga.id,
      correlationId: saga.correlationId,
      status: saga.status,
      observedEvents: saga.observedEvents,
      pendingEvent: this.getNextExpectedEvent(saga),
      timeUntilTimeout: this.getTimeUntilTimeout(saga),
      canCompensate: this.canCompensate(saga),
      compensationActions: this.getAvailableCompensations(saga),
    };
  }
}
 
// Services remain loosely coupled - react to events
class ShippingService {
  @Subscribe('PaymentCompleted')
  async onPaymentCompleted(event: PaymentCompletedEvent) {
    // Still choreography: service reacts to event
    const shipment = await this.scheduleShipment(event.orderId);
    
    // Emit completion event
    await this.eventBus.publish('ShipmentScheduled', {
      orderId: event.orderId,
      shipmentId: shipment.id,
      trackingNumber: shipment.trackingNumber,
    });
  }
  
  @Subscribe('CompensationRequested')
  async onCompensationRequested(event: CompensationRequestedEvent) {
    if (event.type !== 'CancelShipment') return;
    
    await this.cancelShipment(event.correlationId);
    
    await this.eventBus.publish('ShipmentCancelled', {
      correlationId: event.correlationId,
      sagaId: event.sagaId,
    });
  }
}

Saga Coordinator Benefits

•Visibility Without Control — You can see saga state and history without centralizing invocation logic.
•Timeout Handling — Coordinator can detect stuck sagas and alert or compensate.
•Debugging Support — Single place to query saga status for any correlation ID.
•Loose Coupling Maintained — Services still react to events; coordinator only observes.
•Optional Compensation Coordination — Coordinator can trigger compensation while letting services decide how to compensate.

Pattern 4 — Event-Carried State Transfer

The Problem:

In naive choreography, events often carry minimal data:

{ eventType: 'OrderCreated', orderId: '123' }

Consumers must then call the Order Service to get order details, creating runtime coupling:

Consumer availability depends on Order Service availability
Consumer needs to know Order Service API
Additional network round-trip adds latency

The Solution:

Include enough state in the event that consumers are self-sufficient:

{ eventType: 'OrderCreated', orderId: '123', items: [...], customer: {...}, total: 150 }

Now consumers have what they need without callbacks.

Event-Carried State Transfer
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
// MINIMAL EVENT (Requires Callbacks)
interface OrderCreatedEventMinimal {
  eventType: 'OrderCreated';
  orderId: string;
  timestamp: string;
}
 
// Consumer must fetch data
class ShippingServiceWithCallbacks {
  async onOrderCreated(event: OrderCreatedEventMinimal) {
    // Must call back to Order Service
    const order = await this.orderClient.getOrder(event.orderId);
    // Now we can work...
    await this.prepareShipping(order);
  }
}
 
// ENRICHED EVENT (Self-Sufficient Consumers)
interface OrderCreatedEventEnriched {
  eventType: 'OrderCreated';
  eventId: string;
  timestamp: string;
  correlationId: string;
  
  // Order data
  orderId: string;
  orderNumber: string;
  createdAt: string;
  
  // Items - enough for inventory, shipping, analytics
  items: Array<{
    productId: string;
    sku: string;
    name: string;
    quantity: number;
    unitPrice: number;
    weight: number;
    dimensions: Dimensions;
  }>;
  
  // Customer info - enough for fraud, marketing, notifications
  customer: {
    id: string;
    email: string;
    name: string;
    tier: 'standard' | 'premium' | 'vip';
    isFirstOrder: boolean;
    totalLifetimeValue: number;
  };
  
  // Shipping - enough for fulfillment
  shippingAddress: {
    street: string;
    city: string;
    state: string;
    country: string;
    postalCode: string;
  };
  
  // Totals - enough for payment, analytics
  subtotal: number;
  tax: number;
  shipping: number;
  total: number;
  currency: string;
  
  // Business context - enough for various decisions
  channel: 'web' | 'mobile' | 'api';
  promotionCodes: string[];
  giftMessage?: string;
}
 
// Consumer is self-sufficient
class ShippingServiceSelfSufficient {
  async onOrderCreated(event: OrderCreatedEventEnriched) {
    // No callbacks needed - everything is in the event
    
    const shippingDetails = {
      destination: event.shippingAddress,
      items: event.items.map(item => ({
        sku: item.sku,
        quantity: item.quantity,
        weight: item.weight,
        dimensions: item.dimensions,
      })),
      priority: event.customer.tier === 'vip' ? 'express' : 'standard',
    };
    
    await this.prepareShipping(shippingDetails);
  }
}
 
// Multiple consumers can work independently
class AnalyticsServiceSelfSufficient {
  async onOrderCreated(event: OrderCreatedEventEnriched) {
    await this.track({
      event: 'order_created',
      orderId: event.orderId,
      customerId: event.customer.id,
      orderValue: event.total,
      itemCount: event.items.length,
      channel: event.channel,
      isFirstOrder: event.customer.isFirstOrder,
      customerTier: event.customer.tier,
    });
  }
}
 
class FraudServiceSelfSufficient {
  async onOrderCreated(event: OrderCreatedEventEnriched) {
    const riskScore = await this.assessRisk({
      orderValue: event.total,
      customerLifetimeValue: event.customer.totalLifetimeValue,
      isNewCustomer: event.customer.isFirstOrder,
      shippingCountry: event.shippingAddress.country,
      itemCategories: event.items.map(i => this.getCategory(i.productId)),
    });
    
    if (riskScore > 0.8) {
      await this.flagForReview(event.orderId, riskScore);
    }
  }
}

Trade-offs of Event-Carried State Transfer:

Pros:

True decoupling: consumers don't depend on source availability
Lower latency: no callback round-trips
Simpler consumer code: data is right there

Cons:

Larger event payloads: more data to serialize and transport
Potential data duplication: same data in events and databases
Consistency window: event data is a snapshot; may be stale

When to Use:

Consumers need event data for decisions, not just IDs
Source service has high latency or availability concerns
You want to minimize cross-service runtime dependencies
Event size is manageable (typically < 1 MB)

Don't Include Sensitive Data

Pattern 5 — Layered Coordination

In complex systems, coordination needs vary by layer. Layered Coordination applies different patterns at different architectural levels.

Common Layering:

Layer 1: API Gateway / Edge

Synchronous request-response
Gateway routes to appropriate orchestrator

Layer 2: Domain Orchestrators

Orchestration within each domain
Controls complex domain workflows

Layer 3: Cross-Domain Events

Choreography between domains
Events carry state for consumer independence

Layer 4: Integration / External Systems

Orchestration for external API calls
Handles retries, timeouts, circuit breakers

Layered Coordination Architecture
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
// LAYER 1: API Gateway
class OrderApiController {
  @Post('/orders')
  async createOrder(@Body() req: CreateOrderRequest): Promise<OrderResponse> {
    // Gateway invokes the appropriate domain orchestrator
    const result = await this.orderOrchestrator.createOrder({
      ...req,
      correlationId: req.headers['x-correlation-id'] || uuid(),
    });
    
    return { orderId: result.orderId, status: result.status };
  }
}
 
// LAYER 2: Domain Orchestrator
class OrderDomainOrchestrator {
  async createOrder(input: CreateOrderInput): Promise<OrderCreationResult> {
    // Orchestrate within the Order domain
    return this.workflow.execute({
      input,
      steps: [
        // Internal orchestration
        validateOrderRequest,
        enrichCustomerData,
        calculatePricing,
        applyPromotions,
        persistOrder,
        
        // Emit event for Layer 3 (cross-domain choreography)
        publishOrderCreatedEvent,
      ],
    });
  }
}
 
// LAYER 3: Cross-Domain Event Handlers
class PaymentDomainEventHandler {
  @Subscribe('OrderCreated')
  async onOrderCreated(event: OrderCreatedEvent): Promise<void> {
    // Choreography: react to event from another domain
    // Then orchestrate within Payment domain
    await this.paymentOrchestrator.processPayment({
      orderId: event.orderId,
      amount: event.total,
      customerId: event.customer.id,
    });
  }
}
 
// LAYER 4: External Integration Orchestrator
class PaymentGatewayOrchestrator {
  async processPayment(input: PaymentInput): Promise<PaymentResult> {
    // Orchestrate external payment processing
    return this.workflow.execute({
      input,
      steps: [
        selectPaymentGateway,        // Choose Stripe, Braintree, etc.
        preparePaymentRequest,
        executePaymentWithRetry,      // Retry logic for external API
        handlePaymentResponse,
        persistPaymentRecord,
        publishPaymentResult,
      ],
      
      // External API-specific configuration
      retryPolicy: {
        maxAttempts: 3,
        backoff: 'exponential',
        circuitBreaker: {
          threshold: 5,
          resetAfter: '30s',
        },
      },
      
      timeout: '30s',
    });
  }
}
 
// ARCHITECTURE OVERVIEW
const layeredCoordinationArchitecture = {
  layer1_edge: {
    pattern: 'Request-Response',
    components: ['API Gateway', 'Load Balancer', 'Auth'],
    responsibility: 'Route requests to domain orchestrators',
  },
  
  layer2_domainOrchestration: {
    pattern: 'Orchestration',
    components: ['Order Orchestrator', 'Payment Orchestrator', 'Fulfillment Orchestrator'],
    responsibility: 'Control complex workflows within bounded contexts',
  },
  
  layer3_crossDomainEvents: {
    pattern: 'Choreography',
    components: ['Event Bus', 'Schema Registry', 'Event Store'],
    responsibility: 'Loose coupling between domains via events',
  },
  
  layer4_externalIntegration: {
    pattern: 'Orchestration',
    components: ['Integration Orchestrators', 'Circuit Breakers', 'Retry Handlers'],
    responsibility: 'Reliable communication with external systems',
  },
};

Layered Coordination Benefits

•Right Tool for Each Layer — Apply patterns where they fit best; don't force one pattern everywhere.
•Clear Separation of Concerns — Each layer handles different types of coordination complexity.
•Isolation of External Dependencies — External system complexity contained in integration layer.
•Domain Independence — Domains communicate through events; internal complexity is hidden.
•Scalable Architecture — Each layer can scale independently based on its characteristics.

Match Patterns to Failure Modes

Implementing Hybrid Architectures

Implementing hybrid patterns requires careful attention to infrastructure, team practices, and operational considerations.

Infrastructure Requirements:

For Choreography Layers:

Event bus (Kafka, RabbitMQ, AWS SNS/SQS)
Schema registry for event contract management
Dead letter queues for failed event processing
Distributed tracing for event correlation

For Orchestration Layers:

Workflow engine (Temporal, Camunda, Step Functions) or custom orchestrator
State store (PostgreSQL, Redis, DynamoDB)
Activity workers with retry capabilities
Workflow monitoring and alerting

Hybrid Infrastructure Setup
TypeScript
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
// Hybrid Architecture Configuration
 
interface HybridArchitectureConfig {
  orchestration: {
    engine: 'temporal' | 'step-functions' | 'custom';
    stateStore: DatabaseConfig;
    workerConfig: WorkerPoolConfig;
  };
  
  choreography: {
    eventBus: 'kafka' | 'rabbitmq' | 'sns-sqs';
    schemaRegistry: 'confluent' | 'apicurio' | 'custom';
    deadLetterConfig: DLQConfig;
  };
  
  observability: {
    tracing: 'jaeger' | 'zipkin' | 'xray';
    logging: 'elasticsearch' | 'cloudwatch' | 'datadog';
    metrics: 'prometheus' | 'cloudwatch' | 'datadog';
  };
}
 
const productionConfig: HybridArchitectureConfig = {
  orchestration: {
    engine: 'temporal',
    stateStore: {
      type: 'postgresql',
      host: 'temporal-db.internal',
      replication: true,
    },
    workerConfig: {
      minWorkers: 5,
      maxWorkers: 50,
      scaleMetric: 'pending_workflows',
    },
  },
  
  choreography: {
    eventBus: 'kafka',
    schemaRegistry: 'confluent',
    deadLetterConfig: {
      topic: 'dlq.events',
      retentionDays: 30,
      alertThreshold: 100,
    },
  },
  
  observability: {
    tracing: 'jaeger',
    logging: 'elasticsearch',
    metrics: 'prometheus',
  },
};
 
// Team Practices for Hybrid Architecture
 
const teamPractices = {
  eventDesign: {
    // All events must be registered in schema registry
    schemaRegistration: 'required',
    // Events must include correlation ID for tracing
    correlationIdRequired: true,
    // Events must be versioned
    versioningStrategy: 'semantic',
    // Breaking changes require compatibility check
    compatibilityMode: 'backward',
  },
  
  orchestratorOwnership: {
    // Each domain owns its orchestrator
    ownershipModel: 'domain-aligned',
    // Orchestrator changes go through domain team
    changeApproval: 'domain-team',
    // Cross-domain impacts require architecture review
    crossDomainChanges: 'architecture-review',
  },
  
  eventOwnership: {
    // Producer owns the event schema
    schemaOwnership: 'producer',
    // Consumers test against producer contracts
    contractTesting: 'consumer-driven',
    // Schema changes require consumer notification
    changeNotification: 'schema-registry-webhook',
  },
  
  debuggingPractices: {
    // Correlation ID must flow through all layers
    correlationIdPropagation: 'mandatory',
    // All events logged to centralized system
    eventLogging: 'centralized',
    // Saga view service aggregates state
    sagaTracking: 'saga-coordinator',
  },
};

Implementation Best Practices

•Start with Clear Boundaries — Define which workflows use orchestration vs choreography before implementation. Document the reasoning.
•Establish Consistent Correlation IDs — Every request, event, and workflow instance must carry the same correlation ID for end-to-end tracing.
•Use Schema Registry from Day One — Don't let event schemas drift. Enforce registration and compatibility checks.
•Create Saga View Service Early — For choreography layers, build visibility tooling before you need it for debugging.
•Document Event Ownership — Every event has a producing team. Consumers know who to contact for changes.
•Test Across Boundaries — Integration tests should cover orchestrator-to-event and event-to-orchestrator transitions.

Iterate Toward Hybrid

Summary: Blending Patterns Effectively

We've explored hybrid approaches that combine choreography and orchestration. Let's consolidate the key insights:

Key Takeaways

•Hybrid approaches are often optimal — Pure choreography or pure orchestration each have drawbacks that hybrid patterns address.
•Orchestration within, choreography across — Use orchestration inside domains for visibility; choreography across domains for loose coupling.
•Orchestrator as publisher — Orchestrators can emit events, enabling independent consumers without orchestrator changes.
•Saga coordinator provides visibility — A coordinator can observe choreographed sagas without controlling them, providing debugging and timeout handling.
•Event-carried state enables independence — Rich events let consumers work without callbacks to source services.
•Layered coordination matches patterns to needs — Different architectural layers have different coordination requirements.
•Implementation requires infrastructure investment — Both event bus and workflow engine capabilities are needed for effective hybrid architectures.

Module Complete:

You've completed the in-depth exploration of Choreography vs Orchestration. You now understand:

Choreography: Decentralized, event-reactive coordination with loose coupling
Orchestration: Centralized, explicit coordination with clear visibility
Sagas: Distributed transaction management through both patterns
Trade-offs: How to evaluate which pattern fits your context
Hybrid approaches: Patterns that combine the strengths of both

Module Complete

5 / 5