Researchers have identified a problem in preference optimization for generative models, known as manifold drift. This occurs when updates to the model's transport trajectories move terminal samples off the original data distribution. To address this issue, the authors propose two new methods: ThermoDPO and its weighted variant, ThermoDPO-weighted. These methods use a temperature-controlled objective that anchors preference optimization on preferred samples, reducing manifold drift. The authors demonstrate the effectiveness of their methods on several benchmarks, achieving improved results compared to existing approaches. --- Why it matters: Manifold drift has significant implications for the development of generative models, as it can lead to suboptimal performance and decreased reliability. Engineers working with these models need to be aware of this issue and consider using ThermoDPO or its variants to mitigate manifold drift and improve model performance. Source: https://arxiv.org/abs/2608.20011