Your bookmarks

Loading your bookmarks

Latest updates

New courses, skills, book chapters, and site improvements will appear here.

Create free accountSign upDashboardDashboard

Mechanistic interpretability

  • Control
  • Research term
3 steps

Mechanistic interpretability is research that looks inside a trained AI model, at the numbers it learned and the activity flowing through it, to work out the steps it uses to produce its behavior.

Most AI testing judges a model by its answers. A model reaches each answer by passing numbers through a stack of stages, called “layers,” and this research works through them, looking for patterns that stand for ideas, such as a place name or a mood, and tracing how each pattern passes information toward the answer.

Where you’ll meet it

In a team meeting, a vendor pitches an AI tool that screens job applications. One slide says the tool is fair because “we can see inside the model.” Your manager asks what you think.

What to do

Ask which model and which behavior their inside view studied, since findings so far explain narrow pieces of a model. Then judge the tool as you would without the slide, by running it on pairs of applications that differ in one detail, such as the name, and comparing results.

Easy to mix up

Chain-of-thought is reasoning the model writes out, and chat apps often show it. Mechanistic interpretability studies the numbers inside the model, which takes its files and research tools.

Check yourself

Researchers find a pattern inside a model that lights up whenever its answers flatter the user. What would show that the pattern causes the flattery?

Go deeperThe tools that find features help decide which features turn up

Circuits research by Chris Olah and colleagues set much of the vocabulary. A “neuron” holds one number as the model runs, and a “circuit” is a connected group of them that does one step. Models appear to hold more ideas than neurons, a pattern called “superposition,” so researchers train helper networks, “sparse autoencoders,” to split mixed activity into one-idea features.

Anthropic's 2024 “Scaling Monosemanticity” work used sparse autoencoders to find millions of features in one of its publicly released Claude models, including ones for the Golden Gate Bridge and for deception. Which features turn up depends on how the autoencoder was trained, such as how many features it was given room for, and the features found may not cover everything the model computes.

Practice Modern AI

Or take the learning quiz on a mix of every set