What Codex Sends to the Model (www.0xkato.xyz)

In a detailed experiment, a developer analyzed the internal workings of Codex, revealing the extensive data it generates when processing a prompt. For a simple instruction like "Reply with pong.," Codex sent a request of 42,980 bytes, translating to approximately 9,435 tokens. The majority of these tokens were not from the user’s prompt but rather included system instructions, tool definitions, and historical context, highlighting the complexity behind seemingly straightforward AI interactions. Notably, Codex can compact its growing history into manageable summaries, which ensures it can maintain focus without losing critical context over long sessions. This analysis is significant for the AI/ML community as it underscores the intricate mechanisms at play within AI models like Codex. Understanding how requests are constructed and how information accumulates can help developers optimize their interactions and mitigate costs associated with API usage. Additionally, the findings indicate that various factors—such as the size of instruction files or the results produced from command execution—can dramatically influence the request's size and token count, paving the way for more efficient coding practices and better resource management in AI-driven applications.