Researchers have been working on improving language models by allowing them to adapt and learn from their mistakes. However, this process can be flawed if the model doesn't properly revise its criteria for success. A new study, CMB-0.1, was tested on twelve cross-domain cases and found to fail in all five required conditions. The researchers attribute these failures to instrument calibration rather than a lack of capability in the model. They propose a new protocol, CMB-0.4, which is designed to be more discriminating and requires additional steps such as concealed transfer and explicit actions. This study contributes to our understanding of criterion revision and provides a framework for future research. --- Why it matters: This matters because it highlights the importance of proper calibration in language models, particularly when they are adapting to new situations or learning from their mistakes. Improper calibration can lead to flawed decision-making and suboptimal performance. Source: https://arxiv.org/abs/2608.20729