Two Years of Building AI in Firefox (blog.ziade.org)

Mozilla has successfully developed and deployed multiple AI features directly within the Firefox browser over the past two years, emphasizing local execution to ensure user privacy. The team created a unique ML inference runtime based on ONNX and Transformers.js, allowing models to run on-device without sending data to external servers. This architecture ensures stability and security, while updates to models can occur independently of the browser's main updates. A notable application of this technology was the development of PDF.js alt text generation, which involved careful attention to data quality and bias, ultimately demonstrating that smaller models trained on cleaner data can outperform larger counterparts. The significance of Mozilla's work lies in its commitment to creating a browser-centric AI ecosystem that prioritizes user privacy, setting a new standard for web-based AI applications. The foundation built allows for easy integration of additional features, such as Smart Tab management, which also operates locally to analyze and suggest tab groupings based on user context. Looking forward, Mozilla aims to address the challenges of server-side AI while maintaining privacy through proposed standards around end-to-end encryption for large language models. This hybrid approach could redefine how AI services are integrated into user experiences, bridging the gap between local and cloud-based computing in a privacy-centric manner.