Researchers conducted an empirical study to investigate how well preference tuning works when applied to different domains. They compared five popular alignment objectives and various adaptation strategies, such as pseudo-labeling and fine-tuning, across three tasks: summarization, question-answering helpfulness, and safety alignment. The study found that while some adaptation strategies can reduce the degradation of performance under domain shift, they also introduce a trade-off between generalization and diversity. --- Why it matters: This study matters to AI researchers because it helps understand how to improve the adaptability of preference tuning across different domains, which is crucial for real-world applications where data distribution may change over time. Source: https://arxiv.org/abs/2601.05882