Anthropic/OpenAI may be spending more than $1000 for every $100 you pay them (ea.rna.nl)

A recent exploration into the economics of coding with large language models (LLMs), particularly Anthropic's Claude and OpenAI's GPT models, reveals that users may be spending significantly more than anticipated for AI assistance. The research highlights that while per-token costs seem low—in the $5 to $25 range per million tokens—the hidden expenses involved in recursive processing and trial-and-error tasks can inflate these costs dramatically. A user, building an application with Claude, discovered that his expenses rapidly exceeded initial estimates, leading him to conclude that paying a flat rate for higher usage might be more economical compared to the potentially spiraling costs of purchasing tokens on an as-needed basis. This analysis is significant for the AI/ML community as it underscores the increasing complexity and cost associated with advanced LLM tasks, especially in coding—where precision is crucial. As LLMs become more capable, their usage demands more tokens per task, particularly for intricate queries that involve deeper cognitive processing. The conversation shifts from merely measuring cost per token to understanding the total expenditure per task resolution, raising concerns about the sustainability of these costs for businesses and developers relying on AI models for day-to-day tasks, especially in high-stakes fields like finance and healthcare.