State of AI: An Empirical 100T Token Study with OpenRouter (openrouter.ai)

A groundbreaking study by OpenRouter has analyzed over 100 trillion tokens of interactions with large language models (LLMs), marking a significant evolution in the field of AI as it transitions from basic pattern generation to advanced multi-step reasoning. This study, leveraging metadata from billions of user queries, reveals significant trends such as the increasing adoption of open-weight models, a surge in creative roleplay applications, and persistent engagement among early adopters—a phenomenon dubbed the "Cinderella Glass Slipper" effect. These insights underscore the complexities of real-world LLM usage, challenging previous assumptions that largely focused on conventional productivity tasks. The release of OpenAI's o1 reasoning model was a pivotal moment, introducing internal multi-step deliberation and latent planning, which fundamentally enhances the model’s reasoning capabilities. In contrast to older models that relied on single-pass inference, o1's architecture allows for more sophisticated logical consistency and multi-step decision-making. By systematically analyzing the behavior and preferences of users based on a diverse dataset, this study provides model builders, AI developers, and infrastructure providers with data-driven insights to improve the design and deployment of LLM systems, opening new avenues for future applications and research in artificial intelligence.