Show HN: BurstyRouter: local LLM first then burst to end-to-end encrypted LLM (github.com)

BurstyRouter has been introduced as a local-first proxy tool that enhances the handling of AI model requests while prioritizing privacy and encryption. This tool works by initially sending requests to a user's local machine and, when the local environment is busy, bursting requests to TrustedRouter, an end-to-end encrypted gateway designed to maintain user privacy. The robust architecture ensures that even overflow requests remain confidential, utilizing a Trusted Execution Environment (TEE) that guarantees no logs of prompts or outputs are retained, setting it apart from traditional routers that may log sensitive data. This development is significant for the AI/ML community as it addresses critical concerns around data privacy and control in AI model interactions. By offering a clear mechanism for users to manage whether and when their requests are sent to the cloud, BurstyRouter promotes user autonomy while maintaining high security standards. Technical features include compatibility with OpenAI and Anthropic APIs, the ability to handle model requests locally or pass through to TrustedRouter only when necessary, and the publication of cryptographic attestations to verify code integrity. As organizations increasingly prioritize data privacy, BurstyRouter presents a viable solution for integrating AI capabilities without compromising sensitive information.