Reinforcement Learning with Verifiable Rewards (RLVR)
- Training
- Standard term
Reinforcement Learning with Verifiable Rewards (RLVR) trains a model using rewards a program can check automatically, such as whether a math answer matches, code passes its tests, or a proof verifies.
The model attempts many problems, a program scores each attempt, and training makes the steps behind passing answers more likely. Because no person reads each attempt, training can grade far more attempts than people could. The limit becomes finding enough problems whose answers a program can check, along with the computing power to run them.
Where you’ll meet it
A product announcement says the company's new model was trained with reinforcement learning on math and coding problems whose answers are checked automatically. A chart shows big gains on math tests, and the post calls the model better across the board.
What to do
Read the chart as evidence about tasks like the ones checked in training. For writing, advice, or anything without one right answer, look in the announcement for separate results, or try the model on a few of your own tasks before you switch to it.
Easy to mix up
In RLHF, the scorer is a model trained on people's picks between answers. In RLVR, people write the scorer directly, as tests or an answer key a program can check.
Check yourself
A model is rewarded whenever its code passes a coding problem's tests. One problem asks for a list of numbers sorted from smallest to largest, and its test sorts both the model's output and the expected answer before comparing them. What does a pass show?
Go deeperWhether the training teaches new skills is still debated
The Allen Institute for AI's Tülu 3 report named the method. It drew wide attention when reasoning models trained this way showed that rewarding only correct final answers could produce long step-by-step reasoning.
One study gave the base model, the same model before this training, many tries per problem and found it solved as many problems or more, which suggests the training mostly raises the odds of finding solutions the model could already reach. Others report gains that carry over to tasks outside training.
A checker on final answers also rewards a lucky guess the same as sound reasoning. Some teams add process reward models, which score each step of the work, though those bring back a learned judge and its errors.