Researchers have introduced StartupBench, a benchmark for general-purpose agents that evaluates their ability to complete real-world tasks. Unlike existing benchmarks, which rely on researcher-selected tasks, StartupBench is grounded in market-validated AI startup products and user workflows. The benchmark consists of fine-grained rubrics capturing complex requirements, and results show that even the strongest models struggle with approximately 70% of tasks. Analysis reveals that complex instruction following and domain-specific expertise are major sources of failure. --- Why it matters: This matters to engineers because it highlights the limitations of current general-purpose agents in completing real-world user tasks, and provides a benchmark for evaluating progress toward end-to-end completions. Source: https://arxiv.org/abs/2608.17800