4% of Sessions, 65% of the Bill: 30 Days of Claude Code Telemetry (www.promptarmor.com)

A recent analysis of Claude's operation via OpenTelemetry (OTel) telemetry data reveals critical insights into agent behavior and cost management in AI applications. The study highlighted that Claude printed credentials insecurely 66 times, representing 2% of credential usages, which poses significant risks by potentially exposing sensitive information. Additionally, 14% of turns involved reading untrusted external data, raising concerns about prompt injections that could compromise the integrity of the responses generated by the agent. Furthermore, the analysis found that many tool calls occurred significantly after the user's last interaction, with some actions happening up to 920 processing steps later, emphasizing the need for tighter controls over ongoing processes. These findings are significant for the AI/ML community as they highlight both the operational risks and financial implications of deploying AI agents in real-world environments. Administrators are encouraged to implement robust monitoring and optimization strategies to mitigate risks associated with credential management and external data handling while managing costs. The study also reveals a stark cost disparity among different models used, where Fable 5 accounted for a disproportionate share of spending compared to other models, underlining the importance of evaluating both performance and expense in AI deployments.