How Much Does Reasoning Effort Affect Score and Cost? (www.vals.ai)

A recent analysis of seven leading AI models has revealed that reducing reasoning effort significantly decreases operation costs while maintaining a high level of performance. Tests conducted on these models, including Claude Opus 5.5, Grok 4.7, and Gemini 4 Argon, showed that at their lowest reasoning settings, four out of the seven models retained at least 80% of their performance score on the Vals Index for a fraction of the cost—some as low as 16%. This insight is important for the AI/ML community as it suggests that significant cost savings can be achieved without a substantial sacrifice in score, which may encourage broader adoption of AI technologies in cost-sensitive applications. The findings indicate that, in many cases, a lower reasoning setting achieved scores equivalent to the higher-cost leaderboard setting, with 22 out of 56 model-benchmark pairs demonstrating this effectiveness. Interestingly, even with hindsight optimizing for high scores, the potential improvement was only 6 points at a considerably higher cost. The study highlights that while some tasks benefit from increased reasoning effort, others do not, implying that a tailored approach to setting model reasoning levels could optimize both performance and efficiency in various applications, aiding developers and organizations in maximizing the utility of AI models.