Automation

·

6 min read

·

Agents need runbooks too: operating AI after launch

Portrait of Lena Fischer

Lena Fischer

Partner, Automation

Essay

Scroll ↓

About this essay

Practice

Automation

Reading time

6 min

Published

An agent in production is a system like any other. It needs monitoring, an on-call rota and a written answer to “what do we do when it is wrong?”

What goes in the runbook

How to tell the agent is misbehaving, who is paged, how to switch to the manual path, and how to replay the work once it is fixed. One page per failure mode is enough.

Costs are part of operations

Track cost per task next to accuracy. A workflow that is right ninety-nine times in a hundred and three times more expensive than the manual process is not finished.

“If nobody is on call for it, it is still a demo.”

Key takeaways

  • Write one page per failure mode.

  • Keep a manual path you can switch to.

  • Track cost per task beside accuracy.

Portrait of Lena Fischer

Lena Fischer

Partner, Automation. Takes agents and workflows from prototype to monitored production, with the runbooks to keep them there.

Let’s build what’s next, together

Tell us where you want to be in a year. A partner replies within one working day.

Let’s build what’s next, together

Tell us where you want to be in a year. A partner replies within one working day.

Aleia template

Buy this template for just $39

Create a free website with Framer, the website builder loved by startups, designers and agencies.