Researchers have developed a method to detect when language models are leaking sensitive information. They propose LeakGauge, which adds a suffix to the input that gauges leakage behavior and maps prefill token probabilities to an attack-risk score. The team tested LeakGauge on 11 large language models and found it to be effective in detecting unseen attacks, with an area under the receiver operating characteristic curve (AUROC) ranging from 0.944 to 0.996. They also showed that the risk score is sensitive to internal leakage-related directions and can be used for input detection. --- Why it matters: This matters because large language models are increasingly relying on external contexts, creating an attack surface that can lead to sensitive information disclosure. Detecting these leaks is crucial for ensuring model security and trustworthiness. Source: https://arxiv.org/abs/2608.17829