All terms
Reward Model
A separate model trained on human preference data, used to score how "good" a candidate AI response is during further training.
Once humans rank a set of model outputs, that ranking data trains a reward model - essentially an automated judge that can score millions of future outputs the way a human would have. The main model is then trained to maximize the reward model's score. This is the mechanism that turns a relatively small amount of human labeling into training signal at scale.
Example
After humans rank 10,000 pairs of responses, a reward model learns the pattern behind those rankings well enough to score a new, unseen response without a human present.