Lago Open-source SDK: Bill on top of your LLM token cost with no middleware (github.com)

Lago has announced the release of its open-source SDK designed to streamline billing integration for large language model (LLM) clients without the need for additional middleware. This SDK wraps existing LLM clients, like those connected to AWS Bedrock or Mistral, allowing developers to emit usage events directly to Lago for automated billing. By implementing a normalized usage extraction process, the SDK tracks essential metrics such as input and output token counts while maintaining low overhead—ensuring a p99 wrap-overhead under 5 milliseconds. The integration is resilient against outages, utilizing exponential backoff for reliability. This advancement is significant for the AI/ML community as it simplifies the billing process, reduces vendor lock-in, and allows developers to track their LLM usage seamlessly. The SDK’s compatibility with multiple LLM providers and its straightforward installation through Python's package manager enhances accessibility. Additionally, it offers a customizable experience through context variables and error handling options, providing developers flexibility while ensuring accurate monitoring of their LLM resource consumption—all critical factors for organizations aiming to optimize usage costs and operational efficiency in their AI-driven applications.