← All posts

Blameless postmortems that actually change things

Blameless postmortems that actually change things

A postmortem is not a report. It's a mechanism for turning one team's bad week into the whole organization's improvement — and most postmortems fail at exactly that.

If you've ever watched your team write a thoughtful incident review, nod solemnly in the follow-up meeting, and then suffer the same class of outage six months later, you already know the problem. The document was fine. The process around it was broken.

This guide covers why most incident postmortems fail, the two changes that made postmortems actually work for my teams, and how to implement both without adding process overhead nobody wants.

What Is a Blameless Postmortem?

A blameless postmortem is a structured review of an incident that focuses on systems and conditions rather than individual mistakes. The premise is simple: if an engineer was able to take down production with one command, the interesting question isn't "why did they run that command?" — it's "why was that possible, and why did nothing catch it?"

Blamelessness isn't about being nice. It's about accuracy. People who fear punishment give you sanitized timelines and defensive answers. People who feel safe tell you what actually happened — including the workaround everyone uses, the alert everyone ignores, and the runbook that's been wrong for a year. You cannot fix what nobody will tell you about.

But blamelessness alone doesn't make a postmortem useful. Plenty of organizations run perfectly blameless reviews that change nothing. Which brings us to the real problem.

Why Most Incident Postmortems Fail

The failure mode is remarkably consistent across companies of every size. It looks like this:

  1. An incident happens. It's painful.
  2. Someone writes a postmortem: timeline, impact, root cause, three to ten action items.
  3. The review meeting happens. Everyone agrees the action items are important.
  4. The document is archived in a wiki folder.
  5. The action items go... somewhere. A separate tracker, a spreadsheet, a Slack thread, someone's mental to-do list.
  6. Sprint planning happens. Feature work wins. It always wins.
  7. Six months later, the same class of incident happens again.

Notice what failed here. It wasn't the analysis — the root cause was probably correct. It wasn't the intentions — everyone genuinely wanted the fixes done. What failed was the connection between the postmortem and the actual system of work. The action items lived outside the machinery that decides what engineers do all day, so the machinery ignored them.

A postmortem that produces untracked action items is a ritual, not a mechanism. Here's how to make it a mechanism.

Fix #1: Treat Action Items as Engineering Work

The single most effective change my teams made was structural, not cultural: postmortem action items became indistinguishable from any other engineering work. Four rules make this real.

Every action item gets an owner and a deadline

Not "the platform team will look into alerting." A named person, a specific deliverable, a date. If nobody is willing to own an action item, that's a signal it shouldn't exist — cut it in the meeting rather than let it die slowly afterward.

Action items go in the same backlog as features

This is the rule most organizations get wrong. If reliability work lives in a separate "postmortem tracker," it becomes invisible the moment the incident fades from memory. Product managers don't see it during planning. Engineers don't see it when picking up work. Put action items in the same backlog, with the same ticketing, as everything else — so they compete for priority in the open instead of losing by default in the dark.

Cap action items at three per incident

Ten action items is a wish list. Three is a commitment. The cap forces the hard conversation in the review meeting: of everything we could fix, what actually matters most? That prioritization discussion is often more valuable than the items themselves. And three items that ship beat ten items that rot — every time.

Review overdue items in a monthly reliability meeting leadership actually attends

This rule does most of the heavy lifting. Once a month, put every overdue postmortem action item on a screen in front of engineering leadership. Not to shame anyone — to force a decision. Either the item matters and gets re-prioritized, or it doesn't and gets explicitly closed. What's not allowed is the silent third state where it stays open forever and means nothing.

Leadership attendance is non-negotiable. When directors and VPs see overdue reliability work monthly, deprioritizing it stops being free. When they don't, "we'll get to it next sprint" repeats until the next outage.

Fix #2: Ask Better Questions Than "What Was the Root Cause?"

The second shift happens inside the postmortem discussion itself. Stop asking "what was the root cause?" and start asking three questions:

  • What made this incident hard to detect?
  • What made it hard to diagnose?
  • What made it hard to fix?

Here's why this reframing matters. Root cause analysis converges on a single answer — a bad config, a null pointer, a full disk. That answer is usually correct and usually unhelpful, because the next incident will have a different root cause. Fixing this one bug prevents exactly one recurrence.

The three questions diverge instead. They surface the systemic weaknesses that a single root cause hides:

Hard to detect exposes monitoring gaps: the incident ran for forty minutes before a customer reported it, because no alert covered that failure mode. Fixing detection shortens every future incident, not just this one.

Hard to diagnose exposes tribal knowledge and observability debt: only one engineer knew that service's failure patterns, the logs were spread across three systems, the dashboard everyone opened first was misleading. These are the reasons a 10-minute problem took 2 hours to find.

Hard to fix exposes operational friction: the rollback took 45 minutes because deploys are slow, the fix needed a config change that requires a second approver who was asleep, the runbook was outdated. This is the gap between "we knew the fix" and "the fix was live."

Missing alerts, tribal knowledge, slow deploys — these compound across every incident you'll ever have. A postmortem that produces one alert, one documented runbook, and one deploy-pipeline improvement has done more for reliability than ten postmortems that each correctly identified a root cause and archived it.

A Practical Postmortem Structure You Can Steal

Putting both fixes together, here's the skeleton my teams settled on:

  1. Summary — three sentences: what broke, who was affected, for how long.
  2. Timeline — timestamps from first fault to full resolution. Facts only, no blame, no adjectives.
  3. The three questions — hard to detect, hard to diagnose, hard to fix. This replaces the "root cause" section. (You can still note the trigger, but it's one line, not the centerpiece.)
  4. Action items — maximum three — each with an owner, a deadline, and a ticket link in the main backlog.
  5. What went well — genuinely. Fast escalation, a good runbook, a smart call under pressure. This section is what keeps the process blameless in practice, not just in name.

The review meeting should spend 20% of its time on the timeline and 80% on the three questions and the prioritization of action items. If the meeting is mostly people narrating what happened, you're doing archaeology, not engineering.

How to Know It's Working

You'll know within two quarters. The signals to watch:

  • Action item completion rate goes from "unknown" (the usual honest answer) to a number you track — and that number climbs.
  • Time-to-detect and time-to-resolve trend down across incidents, because the detect/diagnose/fix improvements compound.
  • Repeat incidents drop. Not repeat root causes — repeat classes of incidents. Same service, same failure pattern, same "wait, didn't this happen before?"
  • Engineers stop treating postmortems as paperwork. When people see their action items actually ship, they start writing sharper ones.

The Takeaway

Blameless culture gets you honest postmortems. It does not get you effective ones. Effectiveness comes from two structural changes: action items treated as real engineering work — owned, deadlined, capped at three, living in the main backlog, reviewed monthly in front of leadership — and a discussion built around what made the incident hard to detect, diagnose, and fix rather than a hunt for a single root cause.

A postmortem is not a report. Stop writing reports. Start building the mechanism.

// more like this

Set SLOs before you build dashboards

Terraform state: lessons from three disasters

Comments (2)

Ram

Hey Devesh Nice work

Devesh Saini

Wow Nice Article

Comments are moderated before appearing. Leave your email to be notified when yours is approved or replied to.