← Workflows

阅读中文版本

Root-Cause Analysis Workflow

Trace an incident to contributing causes using causal chains rather than stopping at the first explanation.

intermediate~60 minincidentpostmortemquality

Objective

Identify system-level root causes and contributing factors rather than a single blame target.

Environment

  • The incident timeline
  • Logs and metrics
  • A blameless analysis format

Steps

1. State the observable failure

Describe exactly what broke from the user perspective before explaining why.

Expected state: The failure is stated as a fact.

2. Build causal chains

Trace backward from the failure through each contributing condition.

Expected state: Multiple contributing causes are mapped.

3. Distinguish trigger from cause

Separate the immediate trigger from the underlying conditions that allowed it.

Expected state: Trigger and systemic causes are differentiated.

4. Identify control gaps

Name the missing guard that let the failure reach users.

Expected state: A concrete control gap is documented.

Decision points

Stop at the first cause?

  • Keep asking why
  • Stop at the trigger

Keep asking why until a systemic control gap is found.

Blame or blameless?

  • Blameless
  • Name the person

Blameless analysis surfaces the real causes people would otherwise hide.

Expected outcome

A root-cause analysis that explains the failure, its contributing conditions, and the control gaps to close.

Verification

Confirm the analysis explains the timeline, identifies a control gap, and avoids naming individuals as causes.

Cleanup

Store the analysis with the incident and convert control gaps into tracked actions.

Overview

Identify system-level root causes and contributing factors rather than a single blame target.

This workflow is reusable across teams and environments; adapt the commands to your tooling while keeping the sequence and checkpoints intact.

Environment and prerequisites

  • The incident timeline
  • Logs and metrics
  • A blameless analysis format

Procedure

1. State the observable failure

Describe exactly what broke from the user perspective before explaining why.

Expected state: The failure is stated as a fact.

2. Build causal chains

Trace backward from the failure through each contributing condition.

Expected state: Multiple contributing causes are mapped.

3. Distinguish trigger from cause

Separate the immediate trigger from the underlying conditions that allowed it.

Expected state: Trigger and systemic causes are differentiated.

4. Identify control gaps

Name the missing guard that let the failure reach users.

Expected state: A concrete control gap is documented.

# five whys example
# 1 why: checkout failed -> 2 why: DB pool exhausted -> 3 why: connection leak -> 4 why: missing timeout -> 5 why: no pool budget test

Verification

Confirm the analysis explains the timeline, identifies a control gap, and avoids naming individuals as causes.

Decision points

Stop at the first cause?

  • Keep asking why
  • Stop at the trigger

Recommendation: Keep asking why until a systemic control gap is found.

Blame or blameless?

  • Blameless
  • Name the person

Recommendation: Blameless analysis surfaces the real causes people would otherwise hide.

Common failures

  • Stopping at the trigger hides repeat causes.
  • Blaming individuals suppresses reporting.
  • No control gap means the incident recurs.