Reduce token cost with Metrum AI Router – routing intelligence you can train (github.com)

Metrum AI has announced the launch of the Metrum AI Router, a new intelligent routing solution designed to optimize the cost of token usage when interacting with AI models. This technology addresses a common pain point in machine learning workflows where multiple model calls can quickly escalate costs, as evidenced by a case where one job consumed over 48,000 tokens. The router employs a Learned Routing Policy (LRP) that intelligently selects the least expensive, eligible model based on pre-defined quality and cost contracts, while also implementing safeguards against uncertainty in predictions. This development is significant for AI/ML teams looking to manage their token budgets effectively and maintain high-quality outputs without overspending. By incorporating features such as evidence-based eligibility expiration and abstention under high uncertainty, the Metrum AI Router can substantially reduce costs compared to traditional static routing approaches. The system also provides flexibility in integrating both private and hosted AI models under a unified contract, making it suitable for diverse engineering teams while ensuring that quality standards are continually met. This makes it an appealing option for organizations striving to optimize their AI operations in a cost-effective and efficient manner.