Post-Mortems: Learning from Failure
Post-Mortem Culture
Read aloud is unavailable in this browser.
Culture Is the Operating System
You can have the perfect post-mortem template. You can train everyone in root cause analysis techniques. You can implement sophisticated incident tracking systems. But if your organizational culture punishes people for mistakes, hides failures from leadership, or treats incidents as embarrassments to be minimized—none of it matters.
Culture is the operating system on which all other practices run. A strong post-mortem process on a weak cultural foundation will produce shallow analysis, hidden information, and performative compliance. A strong culture can produce valuable learning even with minimal process.
This isn't to say process doesn't matter—it does. But culture comes first. It determines whether people speak honestly, whether leaders respond constructively, whether action items are actually implemented, and whether the organization genuinely improves or merely performs improvement theater.
Building post-mortem culture is a long-term investment. It requires consistent messaging, modeled behavior, structural incentives, and constant vigilance against backsliding. This page is about how to build that culture—and how to protect it once you have it.
By the end of this page, you will understand the cultural prerequisites for effective post-mortems, how to build psychological safety, the role of leadership in modeling desired behavior, how to evolve culture incrementally, and how to diagnose and address cultural dysfunction.
Cultural Prerequisites
Certain cultural elements must exist—or be deliberately built—before post-mortems can be effective. These are not nice-to-haves; they are foundational requirements:
- •Psychological safety — People must believe they can speak honestly about failures without negative consequences to their careers, reputations, or relationships.
- •Learning orientation — The organization must genuinely value learning and improvement, not just performance and results. Failure must be seen as information, not as moral failing.
- •Systems thinking — People must understand that complex systems fail for systemic reasons, not individual incompetence. The instinct to blame must be counteracted.
- •Long-term focus — The organization must value investing in future reliability, not just shipping features today. Action items must be prioritized, not always deferred.
- •Intellectual humility — Everyone, including senior engineers and leaders, must be willing to admit mistakes and uncertainty. 'I don't know' must be acceptable.
- •Collaboration over competition — Teams must support each other rather than blame-shift during incidents. Cross-team cooperation must be normal.
Assessing your cultural baseline:
Before attempting to improve post-mortem culture, honestly assess where you are. Consider these diagnostic questions:
- When was the last time a senior engineer publicly admitted a mistake?
- Do post-mortems identify systemic issues, or do they stop at individual actions?
- Are action items completed, or do they languish in backlogs?
- Do teams voluntarily share post-mortems with other teams?
- When leadership attends post-mortems, do they ask curious questions or assign blame?
- Do people report near-misses voluntarily?
- Is the word 'fault' commonly used in incident discussions?
Honest answers to these questions reveal the cultural reality beneath any official policies.
You cannot create a blameless culture by writing a policy document that says 'We are blameless.' Culture is formed by repeated behavior, especially by leaders. It is revealed by what actually happens when things go wrong—not by what the handbook says should happen.
Psychological Safety in Depth
Psychological safety—the belief that one will not be punished or humiliated for speaking up with ideas, questions, concerns, or mistakes—is the single most important predictor of team learning and effectiveness. Amy Edmondson's research at Harvard has demonstrated this across industries, from healthcare to technology to manufacturing.
For post-mortems specifically, psychological safety determines:
- Whether people share what they actually did versus a sanitized version
- Whether uncomfortable truths about process failures are raised
- Whether junior engineers feel able to contribute insights
- Whether people admit confusion or gaps in their knowledge
- Whether near-misses are reported at all
| Low Safety Environment | High Safety Environment |
|---|---|
| People hide mistakes until discovered | People proactively report mistakes and near-misses |
| Post-mortems focus on finding someone to blame | Post-mortems focus on finding systems to improve |
| Junior engineers stay silent in meetings | Junior engineers contribute observations and questions |
| 'I don't know' is seen as weakness | 'I don't know' is normal and acceptable |
| Incidents are minimized in reporting | Incidents are honestly described |
| Cross-team blame is common | Cross-team collaboration is the default |
| People say what leaders want to hear | People say what they actually think |
Building psychological safety:
-
Leaders model vulnerability. Senior engineers and managers must publicly admit their own mistakes. 'I once deployed a bad configuration that took down production for an hour.' This normalizes fallibility.
-
Respond well to bad news. When someone brings a problem, thank them. 'Thank you for raising this—catching issues early is exactly what we need.' Never shoot the messenger.
-
Separate learning from evaluation. Post-mortems are for learning, not performance evaluation. Never use post-mortem content as input to performance reviews of individuals involved.
-
Actively solicit dissent. Ask: 'What are we missing?' 'Who disagrees?' 'What's the strongest argument against our conclusion?' Make disagreement expected and valued.
-
Intervene on blame. When blame-oriented language appears in meetings or documents, immediately redirect. 'Let's reframe that as a systems question: what made this outcome possible?'
-
Celebrate learning, not just success. Public recognition for valuable post-mortems, insightful near-miss reports, and significant action item completions.
Psychological safety does not mean avoiding hard conversations or lowering standards. High-performing teams combine high psychological safety with high accountability—they feel safe to discuss failures AND they hold themselves to rigorous standards of improvement. Safety enables candor, which enables genuine accountability.
Leadership's Role
Leadership behavior is the single most powerful lever for cultural change. What leaders say matters less than what they do. A single instance of a VP asking 'Who's responsible for this failure?' can undo months of blameless culture-building.
Constructive leadership behaviors:
- •Attend post-mortems as learners, not judges. Ask curious questions: 'Help me understand the situation at that moment.' Not interrogative questions: 'Why didn't you...'
- •Thank incident responders publicly. 'I want to recognize the team that handled Saturday's incident—they demonstrated exactly the kind of ownership we value.'
- •Share your own failures. 'Earlier in my career, I caused an outage by...' demonstrates that mistakes don't end careers.
- •Protect psychological safety actively. If anyone in the organization blames individuals, intervene and correct. This must be non-negotiable.
- •Prioritize action items. When action items compete with feature work, publicly prioritize reliability. 'We're delaying feature X by a sprint to complete these post-mortem action items.'
- •Ask about learning, not just outcomes. 'What did we learn from last quarter's incidents?' is more valuable than 'Why did we have so many incidents last quarter?'
Destructive leadership behaviors:
- •Asking 'Who did this?' — Even once, this signals that blame is expected.
- •Demanding immediate answers — Pressuring for root cause before proper analysis encourages premature conclusions.
- •Expressing frustration publicly — 'This should never have happened' shuts down honest discussion.
- •Linking incidents to performance reviews — Even implicitly, this destroys psychological safety.
- •Deprioritizing action items — 'We'll get to that after the launch' teaches that reliability work is optional.
- •Skipping post-mortems due to time pressure — 'We don't have time for a post-mortem—we need to move on' teaches that learning is optional.
Leaders set the tone in the first 30 seconds of any incident-related conversation. If those first 30 seconds convey curiosity and support, the conversation goes well. If they convey frustration or blame, people shut down. Leaders should consciously manage those first 30 seconds.
Ceremonies and Rituals
Culture is reinforced through repeated practices—ceremonies and rituals that embed values into everyday work. Effective post-mortem culture is sustained through regular, consistent rituals that normalize learning from failure.
Key cultural rituals:
| Ritual | Frequency | Purpose | Who Attends |
|---|---|---|---|
| Post-mortem meeting | After each incident | Analyze specific failure | Incident participants, stakeholders |
| Post-mortem readout | Weekly/biweekly | Share learnings broadly | All engineering |
| Learning review | Monthly/quarterly | Pattern analysis, metrics review | Team leads, SRE, leadership |
| Near-miss review | Weekly | Discuss reported near-misses | On-call engineers, SRE |
| Failure story sharing | Monthly | Senior engineers share past failures | Open to all |
| New hire incident review | During onboarding | Learn from historical incidents | New engineers |
| Process retrospective | Quarterly | Improve the post-mortem process itself | Post-mortem practitioners |
The 'Failure Story' ritual:
One particularly powerful ritual is the Failure Story session. Periodically (monthly or quarterly), a senior engineer presents a significant failure they were personally involved in—ideally from earlier in their career. They describe:
- What happened
- What they were thinking at the time
- What went wrong
- What they learned
- How it shaped their approach
This ritual accomplishes multiple cultural goals:
- Normalizes that even senior, respected engineers make mistakes
- Demonstrates that failures don't end careers
- Shares hard-won wisdom
- Models vulnerability and humility
Etsy was famous for this practice, with senior engineers presenting 'Three-Finger Salute' talks about times they took down production. The ritual became a point of cultural pride.
Rituals only work if they're consistent. A post-mortem readout that happens 'when we have time' communicates that learning is optional. A readout that happens every Tuesday at 2 PM, regardless of competing priorities, communicates that learning is core to how we work.
Evolving Culture Incrementally
Culture change is slow. Organizations with deeply ingrained blame cultures cannot transform overnight. Attempting radical overnight change often produces backlash or superficial compliance without genuine shift.
The incremental approach:
- •Start with a pilot team. Find a team with receptive leadership and run a blameless pilot. Document results and generate success stories.
- •Train facilitators. Develop a cadre of skilled post-mortem facilitators who can model blameless language and redirect blame when it appears.
- •Introduce the language. Start using systems-oriented language in everyday conversation, not just post-mortems. 'What conditions made this possible?' becomes a normal question.
- •Celebrate early wins. Publicly recognize when blameless analysis produces valuable improvements. Share stories of insights that only emerged because people felt safe to speak.
- •Address resistance directly. When people argue that blamelessness means 'no accountability,' engage the concern seriously. Explain the distinction between blame and accountability.
- •Expand gradually. Once one team is operating well, expand to adjacent teams. Use success stories as proof points.
- •Build leadership coalition. Gradually build a coalition of leaders who model and reinforce the culture. Critical mass of leadership alignment is essential.
- •Codify in policy (last). Once the culture is genuinely established, codify in official policy. Policy should follow culture, not lead it.
A common failure mode: leadership mandates 'blameless post-mortems,' and teams comply superficially. Documents use blameless language, but everyone knows who's 'really' at fault. Hallway conversations assign blame. This performative blamelessness is worse than honest blame because it corrupts the process while maintaining the dysfunction. Real culture change requires genuine belief, not just linguistic compliance.
Timeline expectations:
- Months 1-3: Pilot team, train facilitators, introduce concepts
- Months 4-6: Expand to 2-3 teams, aggregate early success stories, address resistance
- Months 6-12: Broader expansion, leadership alignment, ritual establishment
- Year 2+: Organization-wide adoption, continuous improvement, cultural maintenance
Culture change measured in months, not weeks. Patience and persistence are essential.
Diagnosing Cultural Dysfunction
Even organizations with good intentions can develop cultural dysfunction around incidents. Early diagnosis enables intervention before dysfunction becomes entrenched.
Symptoms of post-mortem culture dysfunction:
| Symptom | Possible Cause | Intervention |
|---|---|---|
| Post-mortems are delayed or skipped | Seen as overhead, not value | Leadership must prioritize; demonstrate value through results |
| Documents are superficial | Fear of honest analysis; time pressure | Improve facilitation; allocate sufficient time; reinforce safety |
| Action items aren't completed | No capacity; low priority; no accountability | Reserve capacity; track completion; leadership sponsorship |
| Same incidents recur | Root causes not addressed; action items insufficient | Review RCA quality; investigate action item effectiveness |
| Only low-severity incidents are post-mortem'd | Fear of blame escalates with severity | Explicitly mandate high-severity post-mortems; senior leadership attendance |
| Post-mortems feel performative | Going through motions; no genuine learning | Introduce new techniques; rotate facilitators; connect to real outcomes |
| Cross-team blame is common | Siloed accountability; competitive dynamics | Mandate cross-team post-mortem participation; shared accountability |
| Near-misses aren't reported | No mechanism; no incentive; fear of judgment | Create channel; publicly thank reporters; address any punishment |
Diagnostic practices:
-
Post-mortem sentiment surveys — Anonymous surveys after post-mortem meetings: 'Did you feel safe to share honestly?' 'Was the analysis thorough?' 'Will action items be implemented?'
-
Action item audits — Random audits of closed action items: Are they actually complete? Do they actually address the root cause?
-
Recurrence analysis — Track whether similar incidents recur. Recurrence indicates either insufficient RCA or incomplete remediation.
-
Language analysis — Review post-mortem documents for blame-oriented language. Patterns indicate cultural issues.
-
Skip-level conversations — Leadership talks directly with individual contributors about their experience with the post-mortem process.
Ask engineers: 'If you made a mistake that almost caused an incident but didn't, would you report it?' Their honest answer reveals the true state of psychological safety. If the answer is 'no' or 'depends,' there's cultural work to do.
Sustaining Culture Long-Term
Building a strong post-mortem culture is hard. Sustaining it is harder. Organizations are constantly changing: people leave, new people join, leadership changes, priorities shift. Without active maintenance, even strong cultures erode.
Threats to sustained culture:
- •Leadership change — New leaders may not understand or value the culture, reverting to blame reflexes.
- •Rapid growth — Many new hires lack acculturation; cultural osmosis is diluted.
- •Crisis pressure — During crises, blame instincts resurface. 'We don't have time for blamelessness—someone needs to be accountable.'
- •Complacency — 'We've been incident-free for 6 months—we don't need to do post-mortems anymore.'
- •Process decay — Rituals become rote; engagement declines; quality degrades.
- •Culture-process gap — Official process remains strong, but informal culture diverges (blame in hallways, not documents).
Sustaining practices:
-
Cultural onboarding — New hires receive explicit training on blameless culture, including history, rationale, and expected behaviors. Senior engineers share failure stories as part of onboarding.
-
Leadership succession planning — When leaders change, explicitly onboard them to the culture. Don't assume they understand.
-
Regular reinforcement — Periodically revisit the 'why' of blameless culture in all-hands meetings, team discussions, and documentation. Repetition builds permanence.
-
Metrics monitoring — Track cultural health metrics (post-mortem completion, action item completion, near-miss reports) as leading indicators of erosion.
-
Cultural champions — Identify and cultivate individuals who embody the culture and can influence others. Distribute champions across teams.
-
Response to violations — When blame behavior occurs, address it promptly and clearly. Violations that go unchallenged become normalized.
-
External benchmarking — Periodically compare your practices against industry leaders. Are you keeping pace with best practices?
Buildings, once constructed, stand on their own. Gardens require constant tending—weeding, watering, pruning. Culture is a garden. It requires ongoing attention and will degrade if neglected. There is no 'done' state; there is only continuous cultivation.
KEEP LEARNING
There’s more to understand.
You’re reading the introduction. Create a free account to read the complete lesson, save notes and keep track of your progress.