Eval-maxxing

Tuning a model, prompt, or product so aggressively for benchmark scores that the score stops reflecting real-world performance. The metric goes up while actual usefulness stalls or drops.

Eval-maxxing is tuning a model, prompt, or product so aggressively for benchmark scores that the score stops reflecting real-world performance. This is Goodhart's Law ("when a measure becomes a target, it ceases to be a good measure") applied to AI evaluations. A team rewrites their prompt dozens of times to climb a public leaderboard, then discovers the optimized version handles real customer questions worse than the original. The tuning overfitted to the test while drifting away from the capability users actually need. Benchmark contamination, where test questions leak into training data, makes the problem even harder to spot.

Builder example

Public benchmarks can reveal broad capabilities and support shortlisting when their methods match the question. Product-specific cases then test local fit, while held-out and adversarial cases detect overfitting to either set. Treat each score as evidence about its measured distribution, not as a universal ranking.

A model scores well on general reasoning benchmarks, then gives onboarding guidance that does not match the way your actual customers get set up.

Build a small private test set from your real support tickets, setup blockers, and explanation standards.

Common confusion: A high benchmark score can be both legitimate and irrelevant to your workflow. The score measures what the test measures, and that may have little overlap with what your users actually ask the model to do.