OpenAI has developed a new tool called CoT-Control that allows researchers to study how reasoning models behave. These models often generate long sequences of text, but they can be difficult to control and may produce unwanted or even toxic content. By introducing CoT-Control, OpenAI is highlighting the challenges of controlling these chains of thought, which could have implications for AI safety. According to OpenAI's findings, current reasoning models struggle to control their own chains of thought, suggesting that monitorability could be an important safeguard against potential issues. --- Why it matters: This matters because it highlights the limitations of current reasoning models and underscores the need for better controls over their behavior. Engineers working on AI safety will be interested in understanding how CoT-Control works and what its implications are for the development of more controllable models. Source: https://openai.com/index/reasoning-models-chain-of-thought-controllability