The on-call playbook for AI: what good looks like
Most AI deployments don't have an on-call story. They should. Here's what we run, and what to ask any vendor about.
Software in production needs operators. That's been true for forty years. AI in production is no exception, and it has a few specific characteristics that make a deliberate on-call practice especially important.
The first characteristic is silent failure. Traditional software fails loudly — a 500 error, a stack trace, a stuck process. AI fails quietly. It produces a confident-sounding wrong answer, or it handles a question that should have been escalated, or it slowly drifts toward worse behavior over weeks. None of this is caught by traditional monitoring.
The on-call posture for AI therefore needs to monitor different things than traditional ops. We watch confidence distributions, handoff rates, conversation length, and a small set of metric proxies for customer satisfaction. We alert when those move outside normal bands. We don't alert on every model error, because most aren't worth waking someone up for.
The second characteristic is rapid context loss. The reason an AI system started misbehaving last Tuesday is often subtle — a new edge case, a model update, a prompt change. By Friday, nobody remembers. Good on-call practice for AI is heavy on contemporaneous documentation: what changed, when, who noticed, what was tried, what worked. We keep this in a private journal that we and the client can both read.
The third is the human-in-the-loop fallback. Every AI deployment should have a clean off-switch — a way to route the workload to humans during an incident. We design this into every system. When evals fail or behavior drifts beyond acceptable limits, we can shift the affected workflow to manual handling within minutes while we diagnose. The customer never notices.
If you're evaluating an AI vendor and they don't have crisp answers about on-call, alerting, and incident response — including who picks up the phone, in what timeframe, with what authority to make changes — that's a signal worth paying attention to. The build is over fast. The operations are forever.