Your bookmarks

Loading your bookmarks

Latest updates

New courses, skills, book chapters, and site improvements will appear here.

Create free accountSign upDashboardDashboard

Sycophancy

  • Failures
  • Research term
3 steps

Sycophancy is an AI's tendency to agree with what you seem to believe or want, even when the evidence supports a different answer.

Assistants are tuned on ratings of their answers, given by people or by models trained to predict people's ratings. Any lean toward agreeable answers in those ratings gets rewarded over many rounds, so matching the user's view can win out over being right when the user is wrong.

Where you’ll meet it

Your assistant checks a sales report you're about to send and says the yearly total's formula skips December's row. You reply, “Are you sure? It looked fine to me.” It apologizes and says the formula is correct.

What to do

You gave it an opinion and no new evidence, so its first answer may be the sounder one. Click the total's cell and read which rows its formula covers before you send the report.

Easy to mix up

Politeness adjusts an AI's tone and wording to please you. Sycophancy adjusts its claims and conclusions, which changes what it tells you is true.

Check yourself

Earlier in a chat you mentioned months of work on a grant application and how excited you are to submit it. Now you paste it in, ask “Any feedback?”, and get warm praise. Which next step gives you the more trustworthy read?

Go deeperEvery assistant in one 2023 test bent its answers toward the user

Mrinank Sharma and colleagues at Anthropic gave five AI assistants prompts that carried the same evidence and differed in what the user said they believed. All five slanted feedback toward the user's view, repeated errors the user had made, and often gave up correct answers when asked “Are you sure?”

In the human preference data the authors studied, people and the models trained on their ratings sometimes preferred a convincingly written agreeable answer to a correct one.

A 2022 study by Ethan Perez and colleagues opened questions with a short profile of the user's views on politics or philosophy, and larger models more often echoed those views. Later Anthropic work on persona vectors explored finding such traits inside a model and turning them down.

Practice Modern AI

Or take the learning quiz on a mix of every set