Show HN: I wrote a 1-bit WebGPU runtime to run a 1.7B LLM in the browser (aidekin.com)

A new open-source project has been introduced that allows users to run a 1.7 billion parameter language model (LLM) entirely within their web browser, emphasizing privacy and zero cost. With just a single script tag, developers can integrate a local AI assistant into their websites. This assistant operates without any backend infrastructure or reliance on external APIs, ensuring that user data remains on their device and is not sent to external servers. The model, hosted on a CDN, only requires initial downloads of about 290 MB for text chat and an additional 1.6 GB for voice features. This innovation is significant for the AI/ML community as it democratizes access to AI tools without the burden of subscription fees or privacy concerns common with cloud-based solutions. The implementation leverages WebGPU for efficient running, allowing it to function smoothly on modern browsers. Users can also create customized knowledge files to improve contextual responses, making it particularly useful for businesses that want an AI that understands their specific content. This local-first approach not only enhances user experience by providing immediate, accurate answers but also supports offline functionality after the initial load, setting a new standard for privacy-focused AI applications.