Show HN: Self-hosted AI gateway – MCP, budget, PII, smart router, fallback (github.com)

ILTER has announced a self-hosted AI gateway designed to address key operational challenges in deploying AI models, including cost control, data privacy, and dependency on third-party APIs. This gateway acts as an intermediary between applications and AI providers, allowing users to manage AI traffic on their terms without altering existing application architecture. Notably, it features real-time request scoring for optimal model routing, automatic fallback during outages, and built-in budget controls that immediately cut off access when spending limits are exceeded. This robust solution is significant for the AI/ML community as it offers advanced tools to mask personally identifiable information (PII) before it leaves a network, thereby enhancing compliance with regulations like GDPR and HIPAA. With a compact architecture requiring no complex setups, it integrates seamlessly with existing systems, supporting live metrics tracking and session auditing via an intuitive dashboard. By enabling real-time complexity assessments and smart load balancing among various model providers, ILTER minimizes operational costs while maintaining the flexibility to switch between services without code modifications, essentially redefining how businesses can leverage AI technologies securely and effectively.