Researchers from the University of Singapore have developed a new method for training critics in reinforcement learning, called Best-Practice Critic Optimization (BPCO). The method aims to stabilize and improve critic-based training recipes, which are often unstable. BPCO combines several techniques, including DPPO, value predictions bounded to the reward range, and length-adaptive generalized advantage estimation. Experiments show that BPCO improves a strong critic-based baseline consistently and matches or exceeds a group-based baseline while sampling one response per prompt. The method also improves learning with rubric-based rewards. --- Why it matters: This matters because reliable critics are essential for training large language models, but current methods often fail to provide stable estimates of token-level advantages. BPCO's improvements can lead to better performance and more efficient training in tasks such as mathematical reasoning. Source: https://arxiv.org/abs/2608.23566