Researchers have developed a method called Margin Calibration to improve the robustness of large language models against 'relearn attacks'. These attacks occur when an attacker fine-tunes the model on a small set of examples, causing it to recover its original performance. The team found that most existing methods for unlearning converge to a narrow band above the reference margin, which they call the 'margin cliff'. Margin Calibration adds a non-saturating margin hinge to the loss function, restoring pressure on the forget side and making the model more robust. In experiments, a single frozen configuration of the calibrated model outperformed all other methods in head-to-head comparisons, with significant improvements in ROUGE-L scores and reduced retain-side utility. --- Why it matters: This matters because it addresses a critical vulnerability in large language models, which can be exploited by attackers to recover their original performance. Improving the robustness of these models is essential for their deployment in real-world applications, such as natural language processing tasks. Source: https://arxiv.org/abs/2607.27836