Show HN: GreyFox – Free self-hosted AI proxy, token quotas, and local cache (github.com)

GreyFox Community Edition has been announced as a self-hosted AI traffic proxy designed for teams to manage large language model (LLM) token usage effectively. This tool allows users to set per-user token limits, cache responses, and maintain full visibility over their AI traffic within their own infrastructure. The easy-to-install Docker setup eliminates the need for a cloud control plane, empowering organizations to retain control over their data and usage statistics. The significance of GreyFox lies in its ability to facilitate safe AI integrations while enhancing resource management and cost control. Key features include an OpenAI-compatible proxy endpoint, a local admin user interface, support for caching request responses, and token monitoring with manual cost estimation. By directing AI requests through GreyFox, organizations can enforce quotas and logs while utilizing mock modes for demonstrations. The solution supports diverse AI providers, making it adaptable to various team environments and ensuring better operational efficiency in managing AI traffic without compromising data privacy.