Your bookmarks

Loading your bookmarks

Latest updates

New courses, skills, book chapters, and site improvements will appear here.

Create free accountSign upDashboardDashboard

Probe

  • Control
  • Research term
4 steps

A probe is a small classifier, a program that sorts inputs into groups, trained on a model's internal activity to predict a labeled property, such as the input's topic or whether a statement is true.

A model passes text through a stack of stages called layers, each producing a long list of numbers, its activations. Researchers record them on labeled examples and fit a probe, often a weighted sum of those numbers. If it then scores well on examples it hasn't seen, the property can be read from the activity, for that kind of text.

Where you’ll meet it

A post in your feed says researchers built a lie detector for AI. A probe reads a model's internal activity and flags false statements with 90 percent accuracy, and the post says this could stop chatbots from making things up.

What to do

Look for whether the probe was tested on topics unlike the ones it was trained on, and whether anyone wired the flag to something that blocks or changes the model's answer, since the claim about stopping made-up answers depends on that step.

Easy to mix up

Activation steering changes the model's behavior by pushing its activity toward a trait. A probe only reads that activity, so the model behaves exactly as before.

Check yourself

A probe reads “this claim is false” from a model's activity with high accuracy, yet the model states the false claim anyway. What explains this?

Go deeperHigh probe accuracy needs controls before it describes the model

Guillaume Alain and Yoshua Bengio introduced linear classifier probes in 2016 to study what each layer holds. Simple probes are preferred because a powerful one can learn the task on its own. Hewitt and Liang's 2019 “control tasks” test for this with made-up random labels: a probe that scores well on those is memorizing, so its real score says little about the model.

Intervention tests go further. A weighted-sum probe's weights pick out a direction, a blend of activation numbers that rises with the property, and the test adds or removes that blend while the model runs to see whether the output follows.

Anthropic reported in 2024 on test models deliberately trained to misbehave when a hidden trigger appears, a “backdoor.” Simple probes could flag when those models were about to misbehave, and the authors cautioned that planted backdoors may not behave like ones that arise on their own.

Practice Modern AI

Or take the learning quiz on a mix of every set