Five Signs Your Runbooks Are Written for a Team That No Longer Exists
Photo: NASA/JPL-Caltech, Public domain, via Wikimedia Commons
Every enterprise operations organization carries documentation debt, and most of it hides inside runbooks — the procedures written during a migration, a launch, or a crisis, then abandoned by their authors. The cost is invisible until 2 a.m., when an on-call engineer opens a runbook, discovers it references a decommissioned system, and starts improvising. Improvisation during incidents is expensive, and the invoice shows up as extended downtime and burned-out staff.
Here are five signs your runbooks describe an organization that no longer exists — and what to do about each.
1. Escalation Chains Featuring Departed Employees
The sign: runbook escalation steps name individuals, and at least one of them hasn't been on payroll since the last reorganization.
The quick test: pick ten runbooks at random and verify every named contact against your current directory. Most enterprises fail this test embarrassingly fast.
The remediation path: replace personal names with role-based distribution lists and team aliases, then tie escalation ownership to an on-call rotation system that updates automatically. Roles persist through turnover; individuals don't.
2. Procedures for Systems That Were Decommissioned Years Ago
The sign: a runbook instructs the reader to SSH into a legacy host, restart a service on a platform migrated to the cloud three years ago, or consult a database that now redirects to a landing page.
The quick test: have a junior engineer execute a runbook end to end in a staging environment. Every step referencing something that no longer responds is a documentation tombstone.
The remediation path: archive rather than delete — decommissioned-system runbooks occasionally hold historical value for audits — but clearly mark them obsolete and remove them from the active library. Nothing corrodes trust in documentation faster than following accurate instructions for a world that's gone.
3. Steps That Assume Institutional Knowledge Nobody Has
The sign: instructions like "apply the usual workaround" or "check with the data team" — steps that made sense to the author and to no one else, ever.
The quick test: hand the runbook to a competent new hire with no context. Any step they can't complete without asking a human is a knowledge gap wearing a documentation costume.
The remediation path: rewrite ambiguous steps with concrete commands, links, and expected outputs. Where genuine human judgment is required, name the owning team and what to ask for. A runbook should compress expertise, not assume it.
4. Mean Time to Repair That Keeps Drifting Upward
The sign: incident metrics show response times lengthening even as the incidents themselves stay routine — a classic symptom of on-call engineers spending their first twenty minutes reconstructing knowledge the runbook was supposed to contain.
The quick test: compare your incident timeline data from two years ago against today for comparable severity levels. Upward drift with stable infrastructure is a documentation problem, not an engineering one.
The remediation path: institute a post-incident rule — every time an on-call engineer deviates from a runbook, the deviation is either folded back into the document or the document is corrected. The runbook should converge toward reality with every incident, not away from it.
5. New Hires Who Stop Reading Documentation Entirely
The sign: onboarding feedback reveals that new engineers learned the environment by asking colleagues, because they learned early that the docs can't be trusted. Once a team collectively stops consulting runbooks, every departure increases the organization's fragility.
The quick test: ask your last three hires how they learned incident procedures. If the answer is "people, not documents," your knowledge base has already become ornamental.
The remediation path: treat documentation maintenance as scheduled operational work with dedicated time, not a stretch goal. Assign each runbook a review cadence and an owner, and prune aggressively. A smaller library of accurate runbooks outperforms a large library of fiction on every metric that matters.
The Bottom Line
Runbooks are infrastructure, and like all infrastructure they depreciate without maintenance. The organizations that weather incidents best aren't the ones with the most documentation — they're the ones whose documentation matches the enterprise that actually exists today.
