I traced 3,177 API calls to see what 4 AI coding tools put in the context window (theredbeard.io)

A recent experiment analyzed the performance of four AI coding tools—Claude Opus, Claude Sonnet, Codex (GPT-5.3), and Gemini 2.5 Pro—by tracing 3,177 API calls to investigate how each tool manages the context window when addressing a specific bug in the Express.js framework. Using a context tracer called Context Lens, the study revealed significant variances in token usage, with Gemini consuming an astonishing average of 258,000 tokens compared to Opus's more efficient 27,000 tokens for the same task. The analysis highlighted that while all tools fixed the bug and passed tests effectively, they adopted fundamentally different strategies, affecting their efficiency and operational costs. This exploration is significant for the AI/ML community as it underscores the importance of managing context effectively, especially when tokens directly correlate to usage costs. Tools like Opus demonstrate a strategic advantage through selective information retrieval, relying heavily on git history, while Gemini's aggressive reading and lack of tool definition overhead suggest a more brute-force method that can lead to inefficiencies. Such discussions are crucial for developing better models that not only function correctly but also do so in a resource-effective manner, shaping future design principles and training approaches in AI coding utilities.