Eval-maxxing
- Maxxing
- Practitioner slang
Eval-maxxing is tuning a model, prompt, or product so hard for test scores that the score stops reflecting how well it handles everyday tasks. The number rises while usefulness stalls or drops.
A test covers a limited range of requests. Repeated tuning can reward patterns that raise scores on that set without helping on other tasks. Keeping a separate set of unseen cases helps reveal whether gains extend beyond the questions used for tuning.
Where you’ll meet it
You're deciding whether to switch your AI subscription. The new model's launch post leads with a jump on a popular public leaderboard. A footnote says the score came from a version tuned for that leaderboard, different from the one in the app.
What to do
Read the score as a claim about that tuned version on that test. Before you switch, give both assistants a dozen requests you made last month, the ones whose good answers you remember, and compare the replies side by side.
Easy to mix up
Benchmark contamination means the test questions leaked into the model's training data. Eval-maxxing needs no leak: a model can learn the test's style and scoring habits without ever seeing the questions.
Check yourself
Two prompts for your help bot were scored on the same fifty saved questions. Prompt A was edited against them for a month and scores 96%. Prompt B was written once and scores 88%. Which prompt will answer next week's questions better?
Go deeperPublic scores can be shaped by repeated selection
The economist Charles Goodhart observed in 1975 that a statistic used to steer policy stops behaving as it did. The anthropologist Marilyn Strathern later gave the idea its familiar wording, "When a measure becomes a target, it ceases to be a good measure." Many AI tests publish their questions, so every lab can aim at the same target.
Leaderboards add a route of their own. A 2025 study of one popular chatbot leaderboard, "The Leaderboard Illusion," reported that some providers tested many private versions and published only the best score. Selecting the highest observed score can reward test-specific advantages or measurement noise; it does not establish a general improvement.
Sources