Superposition
- Control
- Research term
Superposition is the idea, observed clearly in small models, that a network can represent more features than it has dimensions by letting the features' internal directions overlap.
A model works in stages, or layers. Each layer holds one number per neuron, the model's smallest unit, and each number is a dimension. A feature, something learned such as French text, is a direction, a pattern across those numbers. Features sharing space blur when both appear, which training allows where it causes few mistakes.
Where you’ll meet it
A news article about AI research says scientists found "the neuron for deception" inside a chatbot's model, and that turning it down made the model more honest.
What to do
Read "neuron" there loosely. The finding is more likely a direction spread across many neurons, found with a tool built to pull overlapping features apart. Look in the article for whether the effect held on inputs the researchers hadn't used to find it.
Easy to mix up
| Superposition | Polysemanticity |
|---|---|
| No, it concerns how a whole layer's space is shared | Yes, that neuron fires for unrelated inputs |
Check yourself
Which feature is a network more likely to give a direction of its own, overlapping with no other?
Go deeperToy models showed the packing, and sparse autoencoders try to undo it
Elhage and colleagues at Anthropic described it in "Toy Models of Superposition" (2022). In small networks trained on made-up inputs, they set each feature's sparsity, how few inputs it appeared in. Packing switched on abruptly past a threshold, like a phase change, and packed features formed regular shapes, such as five in two dimensions as a pentagon.
Sparse autoencoders follow from the idea. They re-express a layer's activity as far more directions than it has neurons, each meant to be active rarely, so each can match one feature. Evidence from large models fits the framework, though nobody has a complete list of any large model's features, and another method can carve the same activity differently.