Researchers have created SWE-bench Science, a benchmark to evaluate coding agents' ability to resolve engineering tasks in science. The benchmark consists of 119 tasks from 98 GitHub repositories across 20 scientific domains and highlights the challenges posed by scientific software engineering. Even the best-performing agent achieves only a 50% success rate, indicating that coding agents struggle with tasks such as repairing scientific software. Four recurring failure mechanisms have been identified: deficits in scientific knowledge or abstraction, misguided exploration or surface-level repair, incomplete repair coverage or system integration, and failures to generalize scientific knowledge beyond observed cases. --- Why it matters: This benchmark matters because it provides a standardized way to evaluate the capabilities of coding agents in scientific software engineering, which is crucial for ensuring the reliability and accuracy of scientific conclusions. By understanding the failure mechanisms of these agents, researchers can develop more effective solutions to improve their performance. Source: https://arxiv.org/abs/2608.19799