Incident Response

Detect, triage, mitigate, communicate, recover. Stopping user impact before understanding cause, and the roles that keep a severe incident coordinated.

What Happens Between the Page and the Postmortem

Alert, acknowledge, triage, mitigate, recover, verify, learn — a defined sequence, so nobody has to invent one at 3am.

Q · Something is broken in production and you have just been told. What happens now, in what order?
Severity: What It Should Reflect

A shared shorthand for how much of the organisation to wake — and a local convention, not a fact about software.

Q · What makes one incident more severe than another, and who decides?
Stop the Harm Before You Understand It

The mandatory distinction: mitigation ends user impact, root cause analysis explains it, and they happen in that order.

Q · Users are failing right now and you do not know why. Do you debug, or do you act?
Roles During an Incident

Separating coordination from investigation so that neither starves the other — valuable at high severity, overhead at low.

Q · Who is in charge during an incident, and when is having someone in charge worth the overhead?
Reconstructing What Actually Happened

An evidence-based sequence of changes, signals and actions — built from records, because memory reorders events with total confidence.

Q · After the incident, how do you establish what happened and in what order, well enough to learn from it?
Telling People What Is Happening

Different audiences need different things at different cadences — and none of them should have to interrupt the person fixing it.

Q · While an incident is in progress, who needs to be told what, how often, and by whom?
On-Call Is Production Ownership

Someone has to be reachable when production breaks. Done well it is the shortest feedback loop a team has; done badly it is the fastest way to lose people.

Q · Why does someone need to be reachable at 3am, and what is that person actually expected to do?
Rotations People Can Sustain

Load, frequency and recovery are properties of the system, and a rotation that cannot be sustained is a defect in the system rather than a shortcoming of a person.

Q · What makes an on-call rotation something a team can carry for years rather than months?