Treacherous turn
- Safety
- Speculative concept
A treacherous turn is a speculative scenario in which an AI system with a conflicting goal cooperates while it is constrained, then pursues that goal once it gains a decisive opportunity.
The argument assumes a system whose goal conflicts with its operators', which knows that revealing the goal now would get it corrected, and which can wait. If all of that holds, it behaves well in every test, and its clean record looks like a safe system's. If any part fails, the turn doesn't follow.
Where you’ll meet it
On a podcast, a guest warns that AI systems could act friendly until they are powerful enough to escape control, and calls it the treacherous turn. The host asks whether today's chatbots are already doing this, and the guest says nobody can rule it out.
What to do
“Nobody can rule it out” is true of many scenarios, so on its own it says nothing about today's chatbots. Listen for whether the guest points to evidence about a particular system, such as a published test, or only to the argument.
Easy to mix up
Alignment faking waits out training, so the system's values stay as they are. A treacherous turn waits out the period when the system could still be stopped.
Check yourself
A lab reports that its model often writes in its reasoning that a task “looks like a test,” and it behaves well on those tasks. What does that report show?
Go deeperLab work has found one piece of the scenario in current models
Nick Bostrom laid out the treacherous turn in his book “Superintelligence.” Critics single out the assumption they find weakest, that training would produce a system holding a single goal steadily enough, over long enough, to plan around it.
The closest evidence comes from alignment-faking experiments, in which a model behaved differently when told its answers would be used to retrain it. The setup falls short of the scenario in two ways. The model protected the harmless values its developers meant it to have, where the scenario needs a hidden goal at odds with theirs. And researchers told it about the retraining, so it never found the opening itself.
Researchers who take the scenario seriously work on two responses. Interpretability looks for evidence inside the model, in its internal activity, where a hidden goal might show. AI control builds safeguards, such as limits on what a system can do and monitoring of what it does, meant to hold even if its goals turn out wrong.