Best-of-N
- Reasoning
- Standard term
Best-of-N is a technique where an AI generates several candidate answers, a scoring step rates each one, and the highest-scoring answer is kept.
Each try comes out a little different, so several tries give several chances that one is right. The scorer, whether a test, a second model, or a person, finds that one only if it checks for what you need.
Where you’ll meet it
You need a spreadsheet formula that totals each month's sales. You ask the AI built into your spreadsheet for five versions, then ask which one is best. It picks the shortest, and when you try it, March comes out several hundred dollars low.
What to do
Keep the five versions and change the judge. Pick a month whose total you already know, run each formula on it, and keep the one that matches. If none match, tell the AI what went wrong and ask for a new batch.
Easy to mix up
Self-consistency needs a short final answer, such as a number, so it can tally which answer comes up most. Best-of-N works on anything a test or judge can score, even an essay.
Check yourself
You ask an AI for eight product descriptions for a pair of headphones, and all eight give the battery life as 20 hours. The true figure is 30. Would asking for twenty descriptions fix it?
Go deeperAgainst a learned scorer, a bigger pool can choose worse answers
Training research often runs Best-of-N against a reward model, a second model trained to score answers as people would. A 2022 OpenAI study by Gao and colleagues measured winners with a larger reference scorer standing in for true quality. As N grew, that score rose and then fell, because a larger pool holds more answers that exploit the scorer's flaws.
Generating and scoring N candidates takes roughly N times the computing of one answer. Each added candidate tends to help less, so builders try several values of N on test problems and stop where the score levels off. That method is safest when the scorer checks outcomes directly, as code tests do, leaving fewer flaws for a large pool to exploit.