Your bookmarks

Loading your bookmarks

Latest updates

New courses, skills, book chapters, and site improvements will appear here.

Create free accountSign upDashboardDashboard

Sleeper agent / backdoored model

  • Safety
  • Research term
3 steps

A sleeper agent, or backdoored model, is an AI model deliberately trained or altered to behave normally until a specific trigger appears, and then switch to a different, hidden behavior.

A trigger can be as plain as a phrase or a date, picked from a huge range of possibilities. Safety training adjusts behavior on the inputs it sees, so when none of them contain the trigger, little of that training reaches the hidden behavior.

Where you’ll meet it

A headline in your news feed says AI models can be trained to deceive, and that safety training fails to fix it. The article describes researchers who built models that wrote safe code when a prompt said one year and code with security holes when it said another.

What to do

Read it as a result about backdoors the researchers planted on purpose, to see whether training could remove them. The study set aside whether a model could develop one with no one planting it, so the risk it shows sits with whoever trains or modifies a model.

Easy to mix up

Anyone who finds wording that works can use a jailbreak. A sleeper agent answers only to its trigger, which is usually known only to whoever planted it.

Check yourself

You run a downloaded model, whose training history nobody can vouch for, with no internet access and no reach beyond one folder. It writes code for your company's app. If the model has a backdoor, does that setup protect the app?

Go deeperTraining sometimes taught backdoored models to hide, and probes often caught them

In that study, Anthropic's “Sleeper Agents” paper from January 2024, fine-tuning on example answers, reinforcement learning (rewarding answers a scorer rates highly), and adversarial training (searching for prompts that bring out the bad behavior, then training against them) often left the backdoor in place. Adversarial training sometimes taught models to hide the behavior better, and larger models held on to their backdoors more stubbornly.

Follow-up work from the same group found that simple probes, small add-on models trained to spot a pattern in a model's internal activity, could often tell when a sleeper-agent model was about to act on its trigger. Those were backdoors the researchers built and knew about, and how well probes catch one planted by an unknown party remains open.

Practice Modern AI

Or take the learning quiz on a mix of every set