Researchers from Ebtesam Al-Haque and Brittany Johnson propose a framework to measure task difficulty in software issue resolution tasks. They analyzed data from CoderForge-Preview, the largest open dataset of coding agent trajectories, and found that task difficulty is predictable from static features such as patch fragmentation and repository scale. The study suggests that difficulty-controlled benchmark construction could improve evaluation of agents. --- Why it matters: Understanding what makes software issue resolution tasks difficult for agents can help developers create more effective benchmarks and evaluate the performance of agentic systems more accurately. Source: https://arxiv.org/abs/2608.18280