Process reward model vs Outcome reward model

An outcome reward model scores a completed answer, while a process reward model scores intermediate steps in a generated solution.

Outcome supervision learns from labels attached to a final result. Process supervision learns from judgments about intermediate steps, such as whether each line in a math solution remains valid. Process labels can provide denser feedback and locate an error, but they are harder to define and can reward a plausible-looking trace that is not faithful to the model's actual computation.

Builder example

Process reward models (PRMs) and outcome reward models (ORMs) are model-training and inference-time selection techniques, not a generic requirement to expose private reasoning in every product. Application workflows can instead verify observable intermediate artifacts: calculations, tool outputs, state transitions, source support, and final behavior. Which signal works best is empirical and task-specific.

Common confusion: Process scoring is not automatically more accurate or safer. Its advantage depends on step labels, the domain, the reward model, and how the score is used; outcome checks can be stronger when the result is exactly verifiable.