AI

The Price of Thinking: Reasoning Effort as a Model-Specific API Contract

Researchers have found that including a 'reasoning-effort term' in an AI model's contract can lead to a small increase in accuracy, but also a significant increase in cost. The study compared two contracts for the same model, one with and one without the effort term, and found that the high-effort contract resulted in a mean delivered cost of $0.01031 per call higher than the low-effort contract. However, the researchers did not detect a significant difference in accuracy bet
Researchers have found that including a 'reasoning-effort term' in an AI model's contract can lead to a small increase in accuracy, but also a significant increase in cost. The study compared two contracts for the same model, one with and one without the effort term, and found that the high-effort contract resulted in a mean delivered cost of $0.01031 per call higher than the low-effort contract. However, the researchers did not detect a significant difference in accuracy between the two contracts, although they cannot rule out a potential gain of up to 4.67 percentage points. --- Why it matters: This study matters because it highlights the importance of considering the 'reasoning-effort term' when evaluating and purchasing AI models. Understanding how this term affects cost and performance can help engineers and researchers make more informed decisions about which models to use for specific tasks. Source: https://arxiv.org/abs/2608.16956

This article was originally published at: https://arxiv.org/abs/2608.16956