Loading learning content...
Once you've decided to implement the Saga pattern, you face a critical architectural decision: how will the saga steps be coordinated? There are exactly two approaches, each representing a fundamentally different philosophy of distributed system design:\n\nChoreography — Decentralized coordination through events. Each service publishes events; other services react to those events. There is no central authority—coordination emerges from the collective behavior of autonomous services.\n\nOrchestration — Centralized coordination through commands. A central orchestrator explicitly tells each service what to do and when. The orchestrator maintains the saga's state and makes all decisions about flow control.\n\nThese approaches aren't merely implementation details—they reflect deep architectural philosophies about coupling, autonomy, observability, and failure handling. Understanding their trade-offs is essential for making the right choice for your system.
By the end of this page, you will understand both choreography and orchestration in depth, including their execution models, implementation patterns, failure handling approaches, and the specific conditions under which each excels. You'll be equipped to make an informed architectural decision for any saga requirement.
In choreography, there is no central controller. Instead, each participant service knows only about its immediate predecessors and successors in the saga. Services communicate by publishing and subscribing to domain events.\n\nThe Dance Metaphor:\n\nImagine a group of dancers performing a routine. In choreography, there's no dance instructor calling out steps. Each dancer watches the others and knows their cue—when dancer A finishes a spin, dancer B knows to start their leap. The coordination emerges from shared understanding of the sequence.\n\nExecution Model:
Key Characteristics:\n\n1. Event-Driven: Services react to events, not commands\n2. Peer-to-Peer: No hierarchical relationship between services\n3. Emergent Coordination: The saga flow 'emerges' from individual service behaviors\n4. Distributed Knowledge: Each service only knows about events it cares about\n5. Asynchronous: All communication is asynchronous by nature
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179
// Choreography-style Saga Implementation // ============================================// ORDER SERVICE// ============================================class OrderService { constructor( private eventBus: EventBus, private orderRepository: OrderRepository ) { // Subscribe to relevant events this.eventBus.subscribe('PaymentFailed', this.handlePaymentFailed.bind(this)); this.eventBus.subscribe('InventoryReservationFailed', this.handleInventoryFailed.bind(this)); this.eventBus.subscribe('NotificationSent', this.handleSagaComplete.bind(this)); } async createOrder(orderData: CreateOrderRequest): Promise<Order> { // T1: Create order (LOCAL transaction) const order = await this.orderRepository.create({ ...orderData, status: 'PENDING', sagaState: 'IN_PROGRESS' }); // Publish event - Inventory Service is listening await this.eventBus.publish({ type: 'OrderCreated', payload: { orderId: order.id, customerId: order.customerId, items: order.items, timestamp: new Date() } }); return order; } // C1: Compensating transaction for order creation private async handlePaymentFailed(event: PaymentFailedEvent) { await this.orderRepository.update(event.orderId, { status: 'CANCELLED', sagaState: 'COMPENSATED', cancellationReason: 'Payment failed' }); // Notify customer of cancellation await this.eventBus.publish({ type: 'OrderCancelled', payload: { orderId: event.orderId, reason: 'Payment failed' } }); } private async handleInventoryFailed(event: InventoryFailedEvent) { await this.orderRepository.update(event.orderId, { status: 'CANCELLED', sagaState: 'COMPENSATED', cancellationReason: 'Item out of stock' }); } private async handleSagaComplete(event: NotificationSentEvent) { await this.orderRepository.update(event.orderId, { status: 'CONFIRMED', sagaState: 'COMPLETED' }); }} // ============================================// INVENTORY SERVICE// ============================================class InventoryService { constructor( private eventBus: EventBus, private inventoryRepository: InventoryRepository ) { this.eventBus.subscribe('OrderCreated', this.handleOrderCreated.bind(this)); this.eventBus.subscribe('PaymentFailed', this.handlePaymentFailed.bind(this)); this.eventBus.subscribe('OrderCancelled', this.handleOrderCancelled.bind(this)); } // T2: Reserve inventory private async handleOrderCreated(event: OrderCreatedEvent) { try { for (const item of event.items) { await this.inventoryRepository.reserve(item.productId, item.quantity, event.orderId); } await this.eventBus.publish({ type: 'InventoryReserved', payload: { orderId: event.orderId, items: event.items, timestamp: new Date() } }); } catch (error) { // Reservation failed - publish failure event await this.eventBus.publish({ type: 'InventoryReservationFailed', payload: { orderId: event.orderId, reason: error.message } }); } } // C2: Compensating transaction - release reservation private async handlePaymentFailed(event: PaymentFailedEvent) { await this.inventoryRepository.releaseReservation(event.orderId); // Note: We don't publish an event here because the OrderCancelled // event will be published by OrderService } private async handleOrderCancelled(event: OrderCancelledEvent) { await this.inventoryRepository.releaseReservation(event.orderId); }} // ============================================// PAYMENT SERVICE// ============================================class PaymentService { constructor( private eventBus: EventBus, private paymentGateway: PaymentGateway, private paymentRepository: PaymentRepository ) { this.eventBus.subscribe('InventoryReserved', this.handleInventoryReserved.bind(this)); this.eventBus.subscribe('ShipmentFailed', this.handleShipmentFailed.bind(this)); } // T3: Process payment private async handleInventoryReserved(event: InventoryReservedEvent) { try { const payment = await this.paymentGateway.charge({ orderId: event.orderId, amount: this.calculateTotal(event.items) }); await this.paymentRepository.create({ orderId: event.orderId, transactionId: payment.transactionId, amount: payment.amount, status: 'CAPTURED' }); await this.eventBus.publish({ type: 'PaymentProcessed', payload: { orderId: event.orderId, transactionId: payment.transactionId, amount: payment.amount } }); } catch (error) { await this.eventBus.publish({ type: 'PaymentFailed', payload: { orderId: event.orderId, reason: error.message } }); } } // C3: Compensating transaction - refund private async handleShipmentFailed(event: ShipmentFailedEvent) { const payment = await this.paymentRepository.findByOrderId(event.orderId); if (payment) { await this.paymentGateway.refund(payment.transactionId); await this.paymentRepository.update(payment.id, { status: 'REFUNDED' }); } }}Implementing choreography correctly requires careful attention to several critical concerns:
PaymentProcessed arriving before InventoryReserved (if using a partitioned message queue incorrectly).123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778
// Idempotent Event Processing Pattern interface ProcessedEvent { eventId: string; eventType: string; processedAt: Date; result: 'SUCCESS' | 'FAILED' | 'SKIPPED';} class IdempotentEventProcessor { constructor( private processedEventRepo: Repository<ProcessedEvent>, private db: Database ) {} async processEvent<T>( event: DomainEvent<T>, handler: (event: DomainEvent<T>) => Promise<void> ): Promise<void> { // Check if already processed (idempotency check) const existing = await this.processedEventRepo.findById(event.id); if (existing) { console.log(`Event ${event.id} already processed, skipping`); return; } // Use transaction to ensure atomic "check-then-process" await this.db.transaction(async (tx) => { // Double-check within transaction (handles race conditions) const existsInTx = await tx.processedEvents.findById(event.id); if (existsInTx) { return; // Skip if another process handled it } // Record that we're processing this event await tx.processedEvents.create({ eventId: event.id, eventType: event.type, processedAt: new Date(), result: 'PROCESSING' }); try { // Execute the actual handler await handler(event); // Mark as successfully processed await tx.processedEvents.update(event.id, { result: 'SUCCESS' }); } catch (error) { // Mark as failed (will be retried or sent to DLQ) await tx.processedEvents.update(event.id, { result: 'FAILED' }); throw error; } }); }} // Usage in Inventory Service:class InventoryService { constructor( private processor: IdempotentEventProcessor, private inventoryRepo: InventoryRepository ) {} async handleOrderCreated(event: OrderCreatedEvent): Promise<void> { await this.processor.processEvent(event, async (e) => { // This code runs at most once per event.id for (const item of e.items) { await this.inventoryRepo.reserve( item.productId, item.quantity, e.orderId ); } }); }}A common choreography pitfall is the dual-write problem: updating the database AND publishing an event in separate steps. If the service crashes between these operations, you either lose the event (database updated, event not published) or publish a ghost event (event published, database not updated). Solutions include the Transactional Outbox Pattern (covered in Module 4) or Event Sourcing (covered in Module 2).
In orchestration, a dedicated Saga Orchestrator (also called Saga Execution Coordinator or SEC) explicitly controls the saga flow. The orchestrator knows the complete saga definition and tells each participant service exactly what to do.\n\nThe Orchestra Metaphor:\n\nImagine a symphony orchestra. Musicians don't watch each other—they watch the conductor. The conductor has the complete score and signals each section when to play. The coordination is explicit and centralized.\n\nExecution Model:
Key Characteristics:\n\n1. Command-Driven: Orchestrator sends explicit commands, not implicit events\n2. Centralized Control: Single component owns the saga definition\n3. Explicit Flow: The saga sequence is clearly defined in one place\n4. State Machine: Orchestrator maintains saga state explicitly\n5. Request-Reply: Communication follows command/reply pattern (though still async)
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204
// Orchestration-style Saga Implementation // ============================================// SAGA DEFINITION (Declarative)// ============================================interface SagaStep<TInput, TOutput> { name: string; execute: (input: TInput, context: SagaContext) => Promise<TOutput>; compensate: (input: TInput, context: SagaContext) => Promise<void>;} interface OrderSagaData { orderId?: string; customerId: string; items: OrderItem[]; reservationId?: string; paymentId?: string; shipmentId?: string;} // The saga definition - a sequence of stepsconst orderSagaDefinition: SagaStep<OrderSagaData, OrderSagaData>[] = [ { name: 'CreateOrder', execute: async (data, ctx) => { const result = await ctx.services.order.createOrder({ customerId: data.customerId, items: data.items }); return { ...data, orderId: result.orderId }; }, compensate: async (data, ctx) => { if (data.orderId) { await ctx.services.order.cancelOrder(data.orderId); } } }, { name: 'ReserveInventory', execute: async (data, ctx) => { const result = await ctx.services.inventory.reserve({ orderId: data.orderId!, items: data.items }); return { ...data, reservationId: result.reservationId }; }, compensate: async (data, ctx) => { if (data.reservationId) { await ctx.services.inventory.releaseReservation(data.reservationId); } } }, { name: 'ProcessPayment', execute: async (data, ctx) => { const result = await ctx.services.payment.processPayment({ orderId: data.orderId!, customerId: data.customerId, amount: calculateTotal(data.items) }); return { ...data, paymentId: result.paymentId }; }, compensate: async (data, ctx) => { if (data.paymentId) { await ctx.services.payment.refund(data.paymentId); } } }, { name: 'ScheduleShipment', execute: async (data, ctx) => { const result = await ctx.services.shipping.scheduleShipment({ orderId: data.orderId!, items: data.items }); return { ...data, shipmentId: result.shipmentId }; }, compensate: async (data, ctx) => { if (data.shipmentId) { await ctx.services.shipping.cancelShipment(data.shipmentId); } } }]; // ============================================// SAGA ORCHESTRATOR ENGINE// ============================================interface SagaInstance { id: string; definition: string; currentStep: number; state: 'RUNNING' | 'COMPLETED' | 'COMPENSATING' | 'FAILED'; data: Record<string, unknown>; completedSteps: string[]; error?: string;} class SagaOrchestrator { constructor( private sagaRepository: SagaRepository, private serviceContext: SagaContext ) {} async executeSaga<T>( sagaId: string, definition: SagaStep<T, T>[], initialData: T ): Promise<SagaResult<T>> { // Create saga instance let instance = await this.sagaRepository.create({ id: sagaId, definition: 'OrderSaga', currentStep: 0, state: 'RUNNING', data: initialData as Record<string, unknown>, completedSteps: [] }); let data = initialData; try { // Execute each step in sequence for (let i = 0; i < definition.length; i++) { const step = definition[i]; console.log(`Executing step ${i + 1}/${definition.length}: ${step.name}`); // Execute the step data = await step.execute(data, this.serviceContext); // Persist progress after each step instance = await this.sagaRepository.update(instance.id, { currentStep: i + 1, data: data as Record<string, unknown>, completedSteps: [...instance.completedSteps, step.name] }); } // Mark as completed await this.sagaRepository.update(instance.id, { state: 'COMPLETED' }); return { success: true, data }; } catch (error) { console.error(`Saga failed at step ${instance.currentStep}: ${error}`); // Begin compensation await this.compensate(instance, definition, data); return { success: false, error: error.message }; } } private async compensate<T>( instance: SagaInstance, definition: SagaStep<T, T>[], data: T ): Promise<void> { await this.sagaRepository.update(instance.id, { state: 'COMPENSATING' }); // Compensate in REVERSE order, starting from the last completed step for (let i = instance.currentStep - 1; i >= 0; i--) { const step = definition[i]; console.log(`Compensating step ${i + 1}: ${step.name}`); try { await step.compensate(data, this.serviceContext); } catch (compensationError) { // Compensation failure is serious - log and continue console.error(`Compensation failed for ${step.name}: ${compensationError}`); // In production: alert, manual intervention queue, etc. } } await this.sagaRepository.update(instance.id, { state: 'FAILED' }); }} // ============================================// USAGE// ============================================const orchestrator = new SagaOrchestrator(sagaRepo, serviceContext); const result = await orchestrator.executeSaga( 'saga-' + uuid(), orderSagaDefinition, { customerId: 'customer-123', items: [ { productId: 'prod-1', quantity: 2, price: 29.99 }, { productId: 'prod-2', quantity: 1, price: 49.99 } ] }); if (result.success) { console.log('Order saga completed:', result.data);} else { console.log('Order saga failed and compensated:', result.error);}Production-grade orchestration requires sophisticated patterns beyond the basic execution loop:
123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176
// Advanced Orchestrator with State Machine type SagaState = | 'STARTED' | 'ORDER_CREATED' | 'INVENTORY_RESERVED' | 'PAYMENT_PROCESSED' | 'SHIPMENT_SCHEDULED' | 'COMPLETED' | 'COMPENSATING_SHIPMENT' | 'COMPENSATING_PAYMENT' | 'COMPENSATING_INVENTORY' | 'COMPENSATING_ORDER' | 'FAILED'; interface StateTransition { from: SagaState; event: string; to: SagaState; action: (context: SagaContext) => Promise<void>;} // Define saga as explicit state machineconst orderSagaStateMachine: StateTransition[] = [ // Happy path transitions { from: 'STARTED', event: 'CREATE_ORDER', to: 'ORDER_CREATED', action: async (ctx) => { const order = await ctx.services.order.create(ctx.data); ctx.data.orderId = order.id; await ctx.emit('RESERVE_INVENTORY'); } }, { from: 'ORDER_CREATED', event: 'INVENTORY_RESERVED_SUCCESS', to: 'INVENTORY_RESERVED', action: async (ctx) => { ctx.data.reservationId = ctx.event.reservationId; await ctx.emit('PROCESS_PAYMENT'); } }, { from: 'INVENTORY_RESERVED', event: 'PAYMENT_SUCCESS', to: 'PAYMENT_PROCESSED', action: async (ctx) => { ctx.data.paymentId = ctx.event.paymentId; await ctx.emit('SCHEDULE_SHIPMENT'); } }, { from: 'PAYMENT_PROCESSED', event: 'SHIPMENT_SCHEDULED_SUCCESS', to: 'SHIPMENT_SCHEDULED', action: async (ctx) => { ctx.data.shipmentId = ctx.event.shipmentId; await ctx.emit('COMPLETE_SAGA'); } }, { from: 'SHIPMENT_SCHEDULED', event: 'COMPLETE_SAGA', to: 'COMPLETED', action: async (ctx) => { await ctx.services.notification.sendConfirmation(ctx.data.orderId); console.log('Saga completed successfully'); } }, // Compensation transitions (failure handling) { from: 'PAYMENT_PROCESSED', event: 'SHIPMENT_FAILED', to: 'COMPENSATING_PAYMENT', action: async (ctx) => { await ctx.services.payment.refund(ctx.data.paymentId); await ctx.emit('PAYMENT_REFUNDED'); } }, { from: 'COMPENSATING_PAYMENT', event: 'PAYMENT_REFUNDED', to: 'COMPENSATING_INVENTORY', action: async (ctx) => { await ctx.services.inventory.release(ctx.data.reservationId); await ctx.emit('INVENTORY_RELEASED'); } }, { from: 'COMPENSATING_INVENTORY', event: 'INVENTORY_RELEASED', to: 'COMPENSATING_ORDER', action: async (ctx) => { await ctx.services.order.cancel(ctx.data.orderId); await ctx.emit('ORDER_CANCELLED'); } }, { from: 'COMPENSATING_ORDER', event: 'ORDER_CANCELLED', to: 'FAILED', action: async (ctx) => { await ctx.services.notification.sendFailure(ctx.data.orderId, ctx.error); console.log('Saga failed and fully compensated'); } }, // Direct failure transitions from any step { from: 'INVENTORY_RESERVED', event: 'PAYMENT_FAILED', to: 'COMPENSATING_INVENTORY', action: async (ctx) => { await ctx.services.inventory.release(ctx.data.reservationId); await ctx.emit('INVENTORY_RELEASED'); } }, { from: 'ORDER_CREATED', event: 'INVENTORY_RESERVATION_FAILED', to: 'COMPENSATING_ORDER', action: async (ctx) => { await ctx.services.order.cancel(ctx.data.orderId); await ctx.emit('ORDER_CANCELLED'); } }]; // State machine executorclass StateMachineOrchestrator { private transitions: Map<string, StateTransition> = new Map(); constructor(definition: StateTransition[]) { for (const t of definition) { const key = `${t.from}:${t.event}`; this.transitions.set(key, t); } } async processEvent(saga: SagaInstance, event: SagaEvent): Promise<SagaState> { const transitionKey = `${saga.state}:${event.type}`; const transition = this.transitions.get(transitionKey); if (!transition) { throw new Error( `No transition from state '${saga.state}' for event '${event.type}'` ); } const context: SagaContext = { data: saga.data, event: event.payload, services: this.serviceClients, emit: async (eventType: string) => { await this.sagaEventQueue.publish({ sagaId: saga.id, type: eventType, payload: saga.data }); } }; // Execute the transition action await transition.action(context); // Update and persist the new state saga.state = transition.to; saga.data = context.data; await this.sagaRepository.update(saga.id, saga); return transition.to; }}Now that we understand both approaches deeply, let's compare them across all relevant dimensions:
| Dimension | Choreography | Orchestration |
|---|---|---|
| Coupling | Loose - services only know about events | Tighter - orchestrator knows all services |
| Single Point of Failure | None - fully distributed | Orchestrator is SPOF (mitigated by HA) |
| Flow Visibility | Scattered across services | Centralized in orchestrator |
| Debugging Complexity | Hard - must trace events across services | Easier - saga state visible in one place |
| Testing | Complex - requires integration testing | Simpler - can unit test saga logic |
| Adding New Steps | Modify only affected services | Modify orchestrator and add service |
| Cyclic Dependencies | Risk of event loops | Not possible - orchestrator controls flow |
| Saga State | Implicit in service states | Explicit in orchestrator |
| Recovery | Complex - reconstruct from events | Simple - resume from persisted state |
| Parallel Steps | Natural with event fanout | Requires explicit coordination |
| Team Autonomy | High - services evolve independently | Lower - orchestrator changes affect all |
| Coordination Overhead | Low - just event bus | Higher - orchestrator infrastructure |
| Best For | Simple sagas, 3-5 steps | Complex sagas, many steps/branches |
• Saga has 3-5 simple, linear steps • Team autonomy is paramount • Services already use an event-driven architecture • Saga logic rarely changes • You have strong distributed tracing capabilities • Each service team owns their saga participation
• Saga has many steps (5+) with complex branching • Business logic changes frequently • You need clear visibility into saga state • Multiple teams must coordinate closely • Sagas span organizational boundaries • You're using a saga framework (Temporal, Camunda)
Building saga infrastructure from scratch is complex. Production systems typically leverage battle-tested frameworks that handle the difficult aspects: state persistence, retry logic, timeout handling, and observability.
Temporal (formerly Cadence at Uber) is the leading orchestration framework for complex workflows including sagas.\n\nKey Features:\n- Durable Execution: Workflow state survives crashes, restarts, deployments\n- Saga Pattern Built-in: Native support for compensations\n- Language SDKs: Go, Java, TypeScript, Python, PHP\n- Visibility: Full workflow history and real-time state\n- Scalability: Powers Uber's 5,000+ microservices\n\nUsed By: Uber, Netflix, Snap, Coinbase, Box, Stripe
12345678910111213141516171819202122232425262728293031323334353637383940414243444546474849505152535455565758
// Temporal Saga Workflow Exampleimport { proxyActivities, ApplicationFailure } from '@temporalio/workflow';import type * as activities from './activities'; const { createOrder, reserveInventory, processPayment, scheduleShipment, // Compensations cancelOrder, releaseInventory, refundPayment} = proxyActivities<typeof activities>({ startToCloseTimeout: '30 seconds', retry: { maximumAttempts: 3 }}); export async function orderSaga(orderData: OrderData): Promise<OrderResult> { const compensations: Array<() => Promise<void>> = []; try { // Step 1: Create Order const orderId = await createOrder(orderData); compensations.push(() => cancelOrder(orderId)); // Step 2: Reserve Inventory const reservationId = await reserveInventory(orderId, orderData.items); compensations.push(() => releaseInventory(reservationId)); // Step 3: Process Payment const paymentId = await processPayment(orderId, orderData.amount); compensations.push(() => refundPayment(paymentId)); // Step 4: Schedule Shipment const shipmentId = await scheduleShipment(orderId, orderData.address); // No compensation needed - if we got here, we're done return { success: true, orderId, shipmentId }; } catch (error) { // Execute compensations in reverse order for (const compensate of compensations.reverse()) { try { await compensate(); } catch (compensationError) { // Log but continue with other compensations console.error('Compensation failed:', compensationError); } } throw ApplicationFailure.nonRetryable( 'Order saga failed and was compensated', 'SAGA_FAILED', { originalError: error } ); }}In practice, many systems use hybrid approaches that combine choreography and orchestration strategically:\n\nPattern 1: Orchestration Within, Choreography Between\n\nEach bounded context uses internal orchestration for complex workflows, but contexts communicate via events (choreography).
Pattern 2: Choreography for Happy Path, Orchestration for Compensation\n\nThe forward flow uses event choreography for its simplicity and loose coupling. When failures occur, a compensation orchestrator is triggered to coordinate the rollback.\n\nPattern 3: Saga Choreography with Saga Coordinator\n\nEvents drive the saga forward, but a lightweight coordinator service listens to all events for:\n- Monitoring saga progress\n- Detecting stuck sagas (timeouts)\n- Triggering compensation when needed\n- Providing saga visibility\n\nThis preserves choreography's loose coupling while adding centralized observability.
Production systems rarely fit neatly into 'choreography OR orchestration'. Start with the simpler approach (usually choreography for simple sagas), add orchestration infrastructure when complexity demands it. The goal is managing complexity, not architectural purity.
We've deeply explored both saga coordination mechanisms. Here are the essential takeaways:
What's Next:\n\nBoth choreography and orchestration must handle a fundamental challenge: what happens when a saga step fails? The next page explores compensating transactions in depth—how to design, implement, and test rollback logic that maintains data consistency despite partial failures.
You now understand the two fundamental approaches to saga coordination and can make an informed architectural decision. Next, we'll master the art of designing compensating transactions—the mechanism that makes sagas actually work.