Orchestrator

The central controller in an AI system that decides which model, tool, agent, or human should handle each piece of work, and in what order.

Every AI system that involves more than one step needs something to manage the flow: deciding which tool to call, which agent to assign a subtask to, when to retry a failure, and when to hand off to a person. That controller is the orchestrator. In simple systems, it is just ordinary code with if/then routing. In more complex systems, a model itself might serve as the orchestrator, reading context and choosing what to do next. For example, a customer-service system might use an orchestrator to route billing questions to one agent, technical issues to another, and escalations to a human supervisor.

Builder example

The orchestrator is where your system's reliability lives. It is where you enforce permissions, set retry logic, define handoff rules, and apply autonomy budgets. If you cannot point to the orchestrator in your system and explain how it makes routing decisions, you have a gap that will show up as unpredictable behavior in production.

Common confusion: Burying orchestration logic inside a prompt makes it feel elegant, but prompts are probabilistic. If a routing decision needs to be reliable (choosing between "charge the customer" and "escalate to a human"), put that logic in deterministic code.