Show HN: ARouter – drop-in OpenAI/Anthropic proxy that cuts cost and fails over (github.com)

ARouter, a new routing proxy for OpenAI and Anthropic, is now available as a drop-in solution that streamlines interactions with both large language models (LLMs). Written in Rust, ARouter allows developers to point their existing SDKs to its server without modifying application code. It intelligently routes requests based on defined policies—reducing costs by directing simpler requests to less expensive models, enabling seamless failover between providers during outages, and ensuring model validation through JSON schema checks. This innovation holds significant implications for the AI/ML community, primarily by addressing cost efficiency, model reliability, and ease of integration. ARouter can potentially save users up to 94% on requests by right-sizing models when conditions are met, thus optimizing resource usage. Additionally, it provides built-in observability features, including metrics and tracing capabilities, facilitating better monitoring and management of API usage across both providers. With these capabilities, ARouter empowers developers to leverage multiple LLMs effectively, enhances application robustness, and ensures cost-effective operation in an increasingly competitive AI landscape.