Escalation policy

A set of rules that define which decisions an AI agent can make on its own and which must be handed to a person for review before proceeding.

An agent that can take actions needs clear boundaries about when to stop and ask. An escalation policy encodes those boundaries: if the agent encounters an ambiguous instruction, hits an error it cannot resolve, is about to take an irreversible action, or exceeds a cost threshold, the policy tells it to pause, report what it knows, and wait. Without one, the agent either plows ahead into risky territory or halts on everything and provides no speed benefit. A customer-service agent might resolve password resets independently, draft refund responses for review, and immediately escalate legal complaints to a human supervisor.

Builder example

The escalation policy is where your system's safety and usefulness negotiate. Too tight and the agent is a glorified queue that passes every decision to a person. Too loose and the agent makes consequential mistakes autonomously. Writing the policy forces you to classify every action by its reversibility and stakes, which is the design work most teams skip and most incidents trace back to.

Common confusion: An escalation policy is complementary to an autonomy budget, but they address different questions. The autonomy budget defines scope: how many steps, which tools, what data. The escalation policy defines triggers: which conditions cause the agent to stop and defer. An agent can be within its autonomy budget and still hit an escalation trigger.