Researchers have developed a benchmark called HarnessRisk to evaluate the safety of agent harnesses in large language models. An agent harness manages tools, extensions, and external actions for these models. Existing benchmarks focus on individual attack mechanisms or limited operational settings, making it hard to compare how safety failures emerge across different harness responsibilities. HarnessRisk organizes agent harness safety into six operational phases and evaluates each trajectory using metrics such as Utility, Attack Success Rate, Persistence, and Detection. The authors tested three harnesses with six language models and 14 model and harness configurations, finding that attack success ranges from 12.6% to 80.9%. They also found that explicit risk recognition does not reliably lead to safe action. --- Why it matters: This benchmark is important for engineers working on large language models because it provides a comprehensive evaluation of agent safety across multiple operational phases and harness configurations, which can help identify vulnerabilities and improve model security. Source: https://arxiv.org/abs/2608.17597