Latent capability
- Failures
- Research term
A latent capability is an ability a model shows under some conditions, or is suspected to have, that its default behavior or a particular test does not show.
Training can give a model the pieces of a skill without making that skill its usual reply. A plain prompt draws out the model's most typical response. Help added around the request, such as worked examples, tools, or many attempts with the best kept, can steer it toward rarer behavior the typical response never shows.
Where you’ll meet it
A company's launch video shows its new model solving a hard competition math problem. The footnote says the result used a custom prompt, a code tool, and the best of 64 attempts. You plan to use the same model in its ordinary chat app.
What to do
Read the claim as true for that setup, where the model had those aids. Your chat gives it one attempt and a plain prompt, so before you rely on it, try it on a few of your own problems where you already know the answer.
Easy to mix up
Capability elicitation is the testing itself, trying setups to see what a model can do. A latent capability is what that testing may turn up: an ability the default behavior hides.
Check yourself
Your assistant gets a logic puzzle wrong when it answers right away, and right when you switch on its extended thinking setting. A friend says the extra thinking taught it how to solve the puzzle. Is that right?
Go deeperA test that finds nothing covers only the aids it tried
Safety evaluations looking for a dangerous ability often go as far as fine-tuning, which is extra training on a small set of examples, because it can bring out abilities that prompting misses. A search through prompts alone leaves more room for a hidden ability, which is why a careful report lists what was tried.
The gap can also be hidden on purpose. A model might perform below its ability during an evaluation, which researchers call sandbagging, and that would hide a capability from the very tests meant to find it.