Show HN: Decaf – rewrites webpage comments using on-device Gemini Nano (github.com)

A new Chrome extension called Decaf has been introduced, utilizing the on-device Gemini Nano model to rewrite webpage comments in real time based on user-defined tones. This innovative tool enables users to modify comments' delivery without altering their core message, accessed simply through a floating coffee cup icon on various platforms, including Hacker News and Reddit. The extension operates entirely within the user's browser, maintaining privacy as no data is sent to external servers, which is particularly significant in today's data-sensitive environment. For the AI/ML community, Decaf's implementation highlights the potential of on-device models like Gemini Nano for practical applications in user interface improvement and content moderation. With its requirements for Chrome 138+, 22 GB of disk space, and a solid RAM and VRAM setup, it demonstrates the growing capability of local AI solutions. The technical approach leverages prompt streaming and a JSON-schema-constrained response system to facilitate rapid rewrites, although the model's performance does have limitations compared to higher-tier models. Overall, Decaf showcases advancements in AI integration into everyday tools, enabling a more refined and user-friendly online experience.