How do AI agents spend your money? (arxiv.org)

A groundbreaking study has analyzed token consumption patterns of AI agents in coding tasks, revealing critical insights into the economics of deploying large language models (LLMs) in complex workflows. The researchers explored token usage across eight advanced LLMs, finding that agentic tasks consume up to 1000 times more tokens than other coding tasks, primarily due to input tokens. Surprisingly, they discovered high variability in token consumption, with different runs on the same task varying up to 30 times in total token count. Moreover, higher token usage does not necessarily equate to better accuracy, as performance often peaks at intermediate token costs before saturating. This research is significant for the AI/ML community, as it highlights the inefficiencies in current models, such as Kimi-K2 and Claude-Sonnet-4.5, which use significantly more tokens than others like GPT-5. It also exposes a disconnect between human judgments of task difficulty and actual token costs, emphasizing the need for better prediction capabilities among LLMs, which showed only weak correlations in forecasting their token consumption. These findings call for a reevaluation of how AI agents are employed and point to potential avenues for optimizing models to lower costs and improve efficiency.