Mechanistic interpretability

Research that tries to identify computational mechanisms inside trained neural networks using their weights, activations, and causal interventions.

Mechanistic interpretability studies features, components, and information flow inside a model rather than relying only on input-output behavior. A proposed mechanism may explain a narrow task or pattern, then fail on another input or model. The field aims for causal and predictive accounts, but current methods do not provide a complete explanation of large production models.

Builder example

Internal analyses can generate hypotheses, locate some behaviors, and support research monitoring. They complement rather than replace behavioral evaluations, security boundaries, and runtime evidence. Product claims should state the model, layer, dataset, intervention, and behavior actually studied.

Researchers identify a feature associated with a behavior such as sycophancy or refusal.

That internal signal can become one input into monitoring, though it still needs external validation.

Common confusion: A model's verbal explanation is generated output. Mechanistic analysis measures internal states, but its interpretation can still be incomplete or mistaken; access to activations is not the same as understanding them.