What 1.5T tokens cost per month (www.peterp.me)

A recent revelation in the AI community examined the cost implications of processing 1.5 trillion tokens a month, leading to significant discussion about token budgeting. Initially estimated at $132 million annually, the accurate figure is closer to $500,000 monthly—a 20x difference largely due to misunderstanding caching mechanisms. Caching reduces costs by allowing repeated access to previously processed data at a fraction of the original input price. Approximately 97% of tokens used in agent sessions are cached, which dramatically shifts budgeting strategies for companies leveraging AI on a massive scale. This understanding is particularly relevant for engineers and developers operating AI agents, as it illustrates a cost profile where the majority of expenses stem from cache reads rather than fresh input processing. The implications for budgeting, deploying models, and managing token-intensive sessions are profound. The importance of selecting models with favorable cache read pricing becomes evident, suggesting that operational efficiencies can provide significant savings. As AI continues to scale, mastering token economics will be crucial for maintaining competitive productivity and financial sustainability in the field.