Show HN: Inferock-bench – per-call billing receipts for OpenAI and Anthropic (github.com)

Inferock-bench has launched a new tool designed to enhance transparency in billing for AI API services, particularly those from OpenAI and Anthropic. This tool allows users to independently track and verify the costs associated with each API call, including detailed receipts that clarify token usage and identify failures. This is particularly significant for the AI/ML community, as it addresses common frustrations around unclear billing practices where providers alone determine the definitions of failures and resulting credits. The tool acts as a local proxy for API calls, measuring usage across various platforms while preserving critical data such as spend, token counts, and instances of failure. It can help users audit their AI usage by exposing discrepancies between what they were billed and what they actually experienced. For instance, it records metrics related to failed or timed-out requests and generates a clear cost analysis, allowing users to identify potential overcharges or inefficiencies in their API consumption. By empowering developers with precise billing information, Inferock-bench aims to foster greater accountability and trust between service providers and users in the rapidly evolving landscape of AI and machine learning services.