Show HN: Made a Free LLM Router (www.flexinference.com)

A new tool called FlexInference has been launched, designed to optimize and reduce costs associated with using Large Language Models (LLMs) from various providers. FlexInference allows users to make requests while establishing a budget for time, automatically seeking cheaper inference options across different services. If FlexInference doesn't find a cheaper alternative, it escalates the request to a standard tier at no additional cost, ensuring users only pay for the best available service. This tool aims to streamline access to LLMs by making it more affordable and efficient, signaling a significant innovation for businesses relying on AI. The technical setup involves using Cloudflare's edge computing infrastructure, ensuring minimal latency while maintaining strong encryption (AES-256-GCM) to secure user keys. FlexInference's user-friendly approach allows developers to easily integrate the tool into their existing setups with just a simple adjustment to their API calls. The introduction of strict typing and detailed error handling facilitates debugging and lowers the likelihood of unexpected failures, enhancing the developer experience. As a result, FlexInference not only aims to democratize access to advanced AI capabilities but also empowers users to maximize their existing credits and rates, potentially accelerating innovation in the AI/ML community.