AI cost dashboard is probably wrong – 45 verified token-accounting bugs (github.com)

A recent audit of AI cost estimation tools revealed 45 critical bugs related to token accounting, potentially leading to significant miscalculations in usage costs for AI coding agents. Conducted across 110 repositories within the Claude Code, Codex, and various observability ecosystems, the audit found that many tools systematically overcount costs, attributing discrepancies of 2x to 5x in some cases. The investigation uncovered specific bug classes that most tools have yet to address, despite many tools passing clean evaluations overall. This audit is significant for the AI/ML community as it emphasizes the need for accurate billing dashboards and financial operations (FinOps) in AI projects. Misleading cost metrics can lead to budget overruns and affect project viability. The findings, supported by open data and synthetic test cases, provide a roadmap for developers to rectify these issues and enhance the reliability of cost estimation tools, ensuring future financial assessments of AI services are more precise and grounded in reality.