03 · How Models Learn
DPO
Direct Preference Optimization
A newer, simpler alternative to RLHF that tunes preferences without building a separate reward model. Recognize it as “the cheaper, more direct way to do what RLHF does.”
Concrete example
Increasingly the default in open-source fine-tuning because it's far less fiddly than full RLHF.