Your intuition of LLM token usage might be wrong (blog.andreani.in)

A recent exploration of token usage in leveraging GPT-5.4-mini reveals a surprising insight: intuition about how tokens are consumed may be misleading. In a 30-minute testing session involving tasks like optimizing how SQLite databases are loaded, the session recorded a staggering 26,257,024 tokens used for cache reads, dwarfed by only 3,648,340 input and 61,676 output tokens. This stark difference underscores that much of the perceived workload in large language models (LLMs) is related to reading from cache rather than writing or generating new content. This finding is significant for the AI/ML community as it shifts focus towards understanding the dynamics of context management in LLMs. The takeaway is clear: developers should aim to keep context concise to optimize token utilization. As cache reads can considerably inflate apparent usage, practices for managing context length will be crucial, especially given the opaque limits set by various LLM providers. This revelation could foster more efficient coding and deployment strategies in AI-driven applications, encouraging developers to rethink how they structure interactions with LLMs.