Post-training
- Training
- Standard term
Post-training is the training a model gets after its broad first stage to change how it behaves, teaching it to follow instructions, match people's preferences, handle safety cases, or specialize in a task.
Pretraining gives a model its grasp of language and facts from an enormous amount of text. Post-training starts from that fluent model with far less data, showing it many examples of the behavior wanted, such as answering, declining, or keeping a tone, and rewarding answers that people or other models rate higher until the behavior becomes habit.
Where you’ll meet it
Your writing assistant updates overnight. The release note says it runs on the same underlying model, with new training for “personality and instruction following.” Today it adds cheerful sign-offs you never asked for and turns down a sharply worded product review it helped you with last week.
What to do
Rerun three or four tasks you do often and note what changed. Put the instructions it now needs, such as “no sign-offs,” in the app's saved instructions or custom settings, and if a refusal blocks ordinary work, report it through the app's feedback button so the developer sees it.
Easy to mix up
Fine-tuning is a method, used inside post-training and by customers adapting a model. Post-training is a stage, the training after the broad first one, and it can mix several methods.
Check yourself
You type “What's a good name for a bakery?” into a model that has had only pretraining. Which reply is more likely?
Go deeperLabs chain several methods, and a product's habits have other sources too
Labs often start with supervised fine-tuning, training on example answers people wrote, then train on preferences. In RLHF or RLAIF, a scoring model learns from human or AI rankings, and the model is rewarded for answers that score well. DPO skips that scoring model and learns from pairs of better and worse answers. Newer recipes reward correct answers on tasks a program can grade, such as math or code that passes tests.
Pretraining leaves habits of its own. An app also adds instructions before your message, and its sampling settings decide how much randomness goes into each word choice. Two products built on one base model, the pretrained model before any post-training, can differ in tone and refusals for any of these reasons.
From outside it is often hard to tell which source, or which post-training method, made a given habit. Running the same prompts with and without the app's instructions, or on the base model where one is released, can narrow it down.