Fine-tuning Gemma 3 for mobile (opensource.googleblog.com)

Cactus, a startup focused on mobile AI, has successfully fine-tuned the open-source Gemma 3 model for mobile applications, leveraging the lightweight Tunix library and Google Colab's Free Tier. This advancement addresses a significant hurdle for app developers who face challenges in running large language models (LLMs) locally due to privacy concerns and device limitations. By enabling users to fine-tune models with minimal technical expertise, Cactus allows app developers to transform generalist models into domain-specific experts, enhancing the functionality of applications such as medical or legal tools. The fine-tuning process uses Tunix, which simplifies supervised fine-tuning (SFT) by stripping away complex dependencies and enabling direct execution on Google Colab. This approach not only democratizes access to powerful AI tools for developers without ML backgrounds but also reduces infrastructure costs. With the ability to export optimized models for easy deployment into mobile applications, Cactus is paving the way for a user-friendly experience in integrating AI into everyday software, thereby accelerating innovation in the mobile space. Looking ahead, Cactus plans to develop a GUI-based portal for fine-tuning and quantization of LLMs, further streamlining mobile AI development.