Samosa Chat: Run Qwen3.6-35B-A3B Locally on a 16 GB Apple Silicon Mac (github.com)

Samosa Chat has introduced a groundbreaking way to run the Qwen3.6-35B-A3B language model locally on a 16 GB Apple Silicon Mac without relying on cloud services or telemetry. This innovative solution leverages a custom inference engine in C and a novel expert cache mechanism that efficiently manages the model's significant 35 billion parameters, using only about 3 billion for any given operation. Users can interact with the model through a command-line interface or a simple web app, making it accessible for scripting and real-time conversation without necessitating a dedicated GPU. This development is particularly significant for the AI/ML community, as it democratizes access to powerful language models, enabling developers and researchers to run complex AI tasks on consumer-grade hardware. The project's open-source nature and emphasis on memory efficiency, stability, and utility ensure that it performs reliably while minimizing wear on hardware components. By successfully demonstrating how to condense advanced AI capabilities into a user-friendly format, Samosa Chat provides a valuable tool for experimentation and innovation in the rapidly evolving field of machine learning.