OpenAI has released a new dataset called SWE-bench Verified, which is designed to evaluate the performance of AI models in solving real-world software engineering problems. Unlike previous versions of SWE-bench, this subset has been manually validated by humans to ensure its accuracy and reliability. --- Why it matters: This matters because it addresses one of the biggest challenges in AI research: ensuring that models can generalize to real-world scenarios. By providing a more reliable benchmark, researchers and developers can better evaluate the effectiveness of their models and make progress towards solving complex software engineering problems. Source: https://openai.com/index/introducing-swe-bench-verified