Communication During Incidents

Every major incident contains two parallel crises: the technical incident and the communication incident. Solving the technical problem while failing at communication leaves customers frustrated, executives blindsided, and trust eroded. Conversely, excellent communication during a slowly-resolved incident can actually strengthen customer relationships.

Consider two scenarios:

Scenario A: A payment processor experiences 45 minutes of downtime. Customers see generic errors with no explanation. Support overwhelmed with tickets saying 'we're investigating.' Twitter fills with complaints. Post-incident, customers feel abandoned.

Scenario B: The same 45-minute outage. Within 5 minutes, status page updates: 'We're experiencing issues with payment processing. Engineers are actively investigating.' Updates every 10 minutes with specifics. Affected customers receive proactive email. Resolution confirmation with brief explanation. Post-incident, customers feel informed and respected.

Same technical failure. Radically different customer experience. That difference is communication.

Communication during incidents isn't a nice-to-have—it's operationally essential. Poor communication:

Impairs Response: When responders don't know what others are doing, they duplicate effort, step on each other's work, or pursue contradictory theories simultaneously.

Multiplies Disruption: Executives calling engineers for status, support escalating tickets, stakeholders pinging for updates—all of these distract responders from actually solving the problem.

Damages Trust: Silence during outages signals either incompetence (you don't know) or disrespect (you know but won't tell). Neither builds customer loyalty.

Creates Legal Risk: In regulated industries, failure to communicate appropriately can constitute compliance violations. Incidents affecting personal data often have mandated notification requirements.

Effective communication, by contrast, creates space for responders to focus on resolution while keeping stakeholders appropriately informed.

Internal communication during incidents serves coordination, not just information sharing. It enables multiple responders to work effectively in parallel without stepping on each other.

The Incident Channel

A dedicated Slack/Teams channel for each incident is standard practice. This channel is the single source of truth—if it's not in the channel, it didn't happen. Key practices:

1. Auto-Create on Declaration: When an incident is declared, automatically create a named channel (#inc-YYYYMMDD-description)
2. Pin Essential Context: Dashboard links, runbooks, status page, key contacts
3. Structured Updates: Use prefixes ([STATUS], [ACTION], [FINDING]) to categorize messages
4. Regular Heartbeats: IC posts status every 10-15 minutes even if no new information
5. Thread Discipline: Use threads for detailed debugging to keep the main channel navigable
6. Preserve History: Don't delete anything; the channel becomes post-mortem source material

The Bridge Call

For complex or fast-moving incidents, a synchronous video/audio call enables real-time coordination. Bridge call practices:

• Open Bridge, Join Late: Keep the bridge running; responders can join as available
• IC Moderates: Prevent crosstalk; maintain focus on resolution
• Mute When Not Speaking: Reduces background noise and distraction
• Summarize in Channel: Key decisions made on bridge should be posted in Slack immediately
• Screen Share Dashboards: Shared visual context accelerates diagnosis
• Allow Break-Out: If a sub-team needs to deep-dive, they can break out and report back

Stakeholders—executives, cross-functional leaders, account managers—have legitimate needs for incident information but don't need (and shouldn't receive) the full firehose of technical updates. Separate stakeholder communication prevents disruption to responders while keeping leadership informed.

The Stakeholder Channel

For major incidents, maintain a separate channel for stakeholder updates:

• Higher Cadence, Lower Detail: Updates every 15-30 minutes with business-relevant information
• Business Context: Frame impact in business terms (customers affected, revenue impact, brand risk)
• Timeline Expectations: Provide estimated resolution times where possible ("investigating, no ETA yet" is valid)
• Resource Needs: Flag if additional resources or escalation is needed
• Read-Only for Observers: Most stakeholders should observe, not participate in technical discussion

Executive Communication Principles

1. Lead with Impact: Executives care about business outcomes. Start with customer and revenue impact, not technical details.

2. Avoid Jargon: "Database connection pool exhaustion" means nothing to a CEO. "Our system ran out of capacity to handle requests" is clearer.

3. Be Honest About Uncertainty: "We're investigating and don't yet know the cause" is better than speculation or false confidence.

4. Provide Timeline Context: "This is the first payment outage in 6 months" or "Similar to last month's issue" provides perspective.

5. Anticipate Follow-Up Questions: Address obvious questions (What about customer X? Will this happen again?) before they're asked.

6. Own the Narrative: Proactive communication prevents executives hearing about incidents from customers or Twitter first.

Customer communication is where incident response meets brand management. How you communicate during outages shapes customer perception more than almost any other interaction. Done well, transparent incident communication can actually increase customer trust. Done poorly, it accelerates churn.

The Status Page

A public status page is the primary channel for external incident communication:

1. Real-Time Updates: Customers should find current status, not yesterday's information
2. Component-Level Granularity: Show status per service/component, not just overall "degraded"
3. Timeline: Historical incidents help customers understand reliability track record
4. Subscription: Allow customers to subscribe for future updates via email, SMS, or webhook
5. Honest Severity: Don't mark "Partial Outage" when the service is down—customers notice

Crafting Customer Messages

Effective customer incident communication:

1. Acknowledge the Problem: Don't hedge or minimize. "We're experiencing issues with X" is direct.

2. Express Empathy: Brief acknowledgment that this affects their work. "We know this disrupts your operations."

3. State What You Know: Current understanding without speculation. "We've identified an issue with our database infrastructure."

4. Share What You're Doing: Demonstrate active response. "Our engineering team is actively working to resolve."

5. Set Expectations: When will you update again? "We'll provide another update in 30 minutes or sooner if resolved."

6. Avoid Technical Jargon: Customers don't need to know about Kubernetes pod scheduling. They need to know if they can use the product.

When to communicate matters as much as what to communicate. Too early with speculation causes confusion; too late loses the opportunity to control the narrative. The key is establishing a consistent, predictable cadence that stakeholders can rely on.

The Communication Timeline

Managing the 'Silent' Period Problem

The hardest communication challenge is when you're actively investigating but have nothing new to say. Silence breeds anxiety—stakeholders assume the worst. Strategies for the silent period:

1. Scheduled Updates: "We'll update every 15 minutes" sets expectation even for "no change" updates
2. Process Updates: "Still investigating. Sarah is checking database logs, David is reviewing recent deploys" shows activity
3. Hypothesis Sharing: "Current theory is connection pool exhaustion; validating now" shows thinking without commitment
4. Rule-Outs: "We've confirmed this is not a network issue" shows progress by elimination
5. Timeline Resets: "Investigation taking longer than expected; expect next meaningful update in 30 minutes"

Never go silent for more than 30 minutes during an active incident. Even "no update" is an update.

Different audiences need different channels. Sending technical play-by-plays to customers is overwhelming; limiting executives to the status page feels dismissive. Match the channel to the audience and the urgency.

Channel Selection Matrix

Multi-Channel Orchestration

For major incidents, communication spans multiple channels simultaneously:

SEV-1 Payment Outage Channel Usage:

1. Incident Slack Channel (#inc-2024-0142-payment-failures): Technical responders coordinate
2. Stakeholder Channel (#payments-incident-stakeholders): Business updates for executives
3. Support Channel (#support-war-room): CS receives talking points and customer scripts
4. Status Page (status.example.com): Public updates every 15 minutes
5. In-App Banner: "Some payments experiencing issues. Check status for updates."
6. Twitter/X: Acknowledge issue if multiple customers tweeting about it
7. Email (post-resolution): Detailed apology to affected customers

The Communications Lead orchestrates this, ensuring consistent messaging across channels.

Excellent incident communication isn't a series of one-off heroic efforts—it's a cultural norm. Organizations that communicate well during incidents have built systems, expectations, and habits that make effective communication the default.

Building Communication Muscle Memory

The Transparency Spectrum

Organizations vary in how transparent they are about incidents. Consider where you want to be:

Minimal Transparency:

• Status page shows "Operational" unless complete outage
• Customers hear about issues only if they contact support
• Post-incident communication avoided

Standard Transparency:

• Status page updated for significant incidents
• Affected customers notified post-resolution
• Generic explanations ("We experienced an issue")

High Transparency:

• Status page updated within minutes of detection
• Regular updates during incidents
• Honest explanation of what happened and why
• Public post-mortems for major incidents

Radical Transparency (examples: Cloudflare, GitLab):

• Real-time status with detailed technical information
• Published post-mortems with root cause and fixes
• Engineering blog posts about major incidents
• Open acknowledgment of mistakes

Higher transparency builds more trust but requires more communication effort and organizational maturity.