Guardrail
- Standard term
A guardrail is a limit enforced outside the AI, in the tools or systems it works through, that allows, changes, or blocks an action before the action takes effect.
An instruction is text the model weighs along with everything else it reads, so a long, persuasive message can outweigh it. A check written into a tool's code runs every time the tool is used, whatever the model wrote, so it holds when the model misjudges a case or is talked into something.
Where you’ll meet it
You and a coding agent built a support assistant for your online shop. It answers customer emails and can create discount codes. Its instructions cap discounts at 15 percent, and last week it read a long complaint as a special case and sent a 60 percent code.
What to do
Keep the instruction, and ask your coding agent to put the cap inside the discount tool's code, so the tool refuses any code above 15 percent whatever the assistant decides. Then have it ask the assistant for a 20 percent code and confirm the refusal.
Easy to mix up
An approval gate makes the action wait until someone approves it. A guardrail decides on the spot, allowing, changing, or blocking the action with no one asked.
Check yourself
Your agent has a tool that edits files, one that searches the web, and one that types commands into the terminal, the window where typed instructions run. You set the file tool to refuse any delete in your Taxes folder. Is the folder safe from the agent?
Go deeperA guardrail should refuse the action when its own check breaks
A check can break, for example when it can't read an amount or the service it consults is down. Refusing the action then is known as “failing closed.” A check that lets everything through after an error, “failing open,” protects nothing at the moment something unusual happens, so tests should include garbled and missing inputs.
Many products use the word more broadly, for filters that screen what goes into a model and what comes out. These are often classifiers, small models trained to flag harmful content or personal data. They catch things no fixed rule can spell out, such as an insult, and they also miss some cases and block some harmless ones, so products often pair them with limits built into the tools.