Running Qwen 3.6 Locally on a Mac Mini M4 with 16GB RAM (maloyan.xyz)

The recent open-sourcing of Qwen 3.6-35B-A3B, a 35-billion parameter Mixture of Experts (MoE) model by Qwen, is making waves in the AI community due to its unprecedented capabilities and efficient use of hardware. Designed to activate only 3 billion parameters per token, it can be run on a modest $599 Mac Mini M4 with just 16GB of RAM, demonstrating an impressive decoding speed of 17 tokens per second without utilizing swap memory. This represents a significant advancement over traditional dense models, offering competitive performance in coding and reasoning tasks typically expected from much larger models. The significance lies in the MoE architecture, which allows the model to remain within the memory constraints while dynamically allocating computational resources at runtime. This architecture, combined with tools like llama.cpp for memory-mapping, makes it possible to achieve usable performance on lower-end hardware, democratizing access to powerful AI tools. Developers can implement various inference setups, with the capability of utilizing Ollama for straightforward installation or LM Studio for enhanced performance optimization. Overall, Qwen 3.6-35B-A3B sets a new standard for running large language models locally, proving that effective AI applications can be both powerful and accessible.