Researchers have found that post-training quantization, a method used to compress large language models for deployment on resource-constrained devices, can cause behavioral changes in the models even when accuracy and perplexity are preserved. A new metric called Correctness Agreement measures the intersection of correct predictions between the base model and its quantized variant, revealing a shift in behavior at low bit widths. The study found that query and key projections are more sensitive to quantization than value and output projections. --- Why it matters: This matters to AI researchers because it highlights the limitations of traditional evaluation metrics like accuracy and perplexity, which may not capture important behavioral changes caused by quantization. Understanding these effects is crucial for developing effective compression methods that preserve model performance. Source: https://arxiv.org/abs/2607.08734