A runbook for the colleague who was doing something else

The alert is already in the past

The import alert arrives while a colleague is doing something else. By the time they open it, the system may have changed, and it may not be a service they normally run. A first instruction to restart the worker asks them to accept two untested assumptions: the problem still exists, and the worker caused it. Give them a way to read the current state first, so they can choose an action rather than inherit a diagnosis hidden inside the opening command.

Take a constructed morning import, B17. It accepted 43 records and was expected to finish by 09:15. The alert reports only 40 confirmed outcomes. “Three pending” sounds like one problem, but it is merely a subtraction. The three records might be waiting, rejected, already completed since the alert, or accepted elsewhere without a returned confirmation. Restarting something could help one state and make another harder to understand.

Begin with what the delay means: the import has not accounted for every accepted record by its expected finish time. Then offer a read-only view of the current batch and outcomes. The covering person should not need a private interpretation of the graph legend to know what counts as complete. If the problem has already cleared, that view can spare them an unnecessary intervention. If it remains, it gives them a present-tense question to investigate instead of another copy of the old alert.

Make the first observation hard to misread

For B17, the proposed first view names the batch, accepted count, confirmed count, unresolved record IDs and observation time. It separates an observation made now from a screenshot attached to the original alert. A healthy worker heartbeat is useful supporting evidence, but it does not answer whether these 43 records reached their intended result. Likewise, a quiet error graph cannot distinguish completed work from work that never started.

The downloadable SQLite model supplies forty confirmed rows and three different unresolved ones: J41 not_started, J42 issued_unknown and J43 validation_rejected. Its parameterised query reads B17’s identities and states, then checks them against the expected membership of 43. A different population leads to investigation before record-level advice. That matters because a precise completion percentage is only useful if its denominator belongs to the batch the colleague is actually trying to recover.

Observation also needs an access branch. If the operator cannot read the batch, the instruction is to contact the designated import owner with the batch and alert identifiers, not to guess from whatever dashboard remains available. The fixture returns escalate_access without querying. A deliberately illustrative 90-second freshness rule similarly asks for a renewed observation before classifying stale data. That threshold belongs to the example; a live service must choose one suited to its processing pace and observation source.

The covering colleague needs a working read-only path in the right environment. The model denies an attempted deletion during observation, but the deployed service must grant suitable read access and reject unintended writes at its actual boundary. Include those permissions in the rehearsal. Otherwise the runbook may describe a perfect query that the substitute cannot run, or leave them using a broader account because no appropriate role was prepared. The designated owner should be reachable to resolve that gap before anyone tries to infer the batch state from another screen.

Three records need three different decisions

J41 has no issued attempt, so the example permits an authorised start once the record is still unclaimed. The model supplies that decision, not the start command. A real tool must claim the operation at the write boundary; reading a row and clicking quickly is not a concurrency mechanism. Name the approved operation in the runbook so the covering colleague can use its protection rather than improvise a database edit under the pressure of an outstanding import.

J42 is different. Its request was issued, and the outcome is unknown. The next action is to reconcile that original request with the receiving system before trying again. The runbook needs the original request identity and the receiving lookup, or a clear escalation to someone who has them. “Retry the three failures” is unsafe wording because J42 is not known to be a failure. Its missing confirmation is precisely the fact that prevents the operator from treating it like J41.

J43 has a known validation rejection. Starting it again unchanged is not useful progress. Its owner must correct the input and pass validation before a new attempt is considered. The record should retain the rejection and the relationship to any corrected submission, so that later support work can explain why this import needed attention. This path belongs with the owner of the information, who may be someone other than the engineer carrying the pager.

These branches require more explanation than a restart instruction, but they give the covering person the state distinctions needed for a decision. The first view can remain concise, with links to detailed mechanisms. The issued-unknown warning belongs beside the retry decision, where it can prevent another attempt for J42. Placing it below the command would leave the operator without relevant information at the point they act.

Give the handover a shape

A runbook cannot grant authority by sounding confident. It should state who may start waiting records, who can reconcile receiving outcomes and who owns rejected input. If one person holds several roles, naming the responsibilities still helps a substitute understand what is missing. Google SRE's on-call guidance treats procedures, support and escalation as part of making on-call work sustainable. The relevant lesson for a small service is not to copy a large organisation chart; it is to stop leaving the covering person alone with unspoken expectations.

The escalation packet should be compact enough to assemble from the first view: B17, the expected completion time, the current observation time, 43 accepted, 40 confirmed, the three unresolved identities and states, and any actions already attempted. Include links or authorised identifiers for deeper evidence, rather than copying sensitive payloads into every channel. If J42 has an external request ID, retain it in the appropriate system and give the receiving specialist a way to find it.

A handover needs a next owner as well as a message. Several teams receiving the same escalation does not tell the covering colleague which one has taken responsibility. Let them record the observation, the remaining question and the point at which it moves to someone with the relevant authority. Google’s incident-response guidance separates coordination from technical recovery. B17 needs that distinction on a modest scale too, especially when one person can inspect J42 but another must establish its receiving outcome.

Do not bury the stopping rule. If the observation is stale, access is missing, membership differs or a remote effect remains uncertain, stop the corresponding write path and escalate with evidence. This does not mean stop collecting safe observations. It means the document names the point beyond which guessing would create new facts that the next person must untangle.

Close the import, then improve the page

When all 43 outcomes are confirmed, the operator still verifies the intended receiving result before closing B17. The later constructed snapshot leads to that closure branch rather than another start. Keep the downstream confirmation with the batch’s resolution: a local status cannot establish a real receiving outcome by itself. The old alert then has an explained ending, so the next covering colleague can recognise recovered work without repeating the action it once appeared to need.

Ask someone other than the author to find the actual read path, interpret J41–J43 and identify who can take the next decision, with writes confined to an appropriate test environment. A colleague rehearsal remains to be done. Their point of hesitation may reveal an inaccessible link, a state name that implies the wrong action or an escalation route nobody can answer, even when every query branch behaves as specified. Record the necessary repair and give the page an owner who can maintain those routes when the service or its authority changes.