Goodhart's law states that when a measure becomes a target, it stops being a good measure. This concept, originally from economics, is relevant in AI optimization where objectives can be difficult or expensive to quantify. At OpenAI, this issue arises when trying to optimize metrics that are hard to measure directly. The company must balance the need for objective targets with the risk of Goodhart's law, which can lead to unintended consequences. --- Why it matters: Understanding and mitigating Goodhart's law is crucial for AI researchers and engineers who work on optimizing complex objectives, as it can have significant implications for the reliability and effectiveness of their models. Source: https://openai.com/index/measuring-goodharts-law