Gemma 4 26B on a 24 GB Mac Mini M4: from 66% CPU to 100% GPU (mac-mini-m4-doc.masterfabric.co)

Recent performance tweaks to the Ollama software on a 24 GB Mac Mini with M4 chip have transformed the utilization of the Gemma 4 26B model, elevating GPU usage from 66% to a full 100%. This change came after a modification of macOS's memory ceiling, allowing more unified memory access for GPU processing. Prior limitations constrained the GPU to about 17.8 GiB, which, when coupled with the model's requirements, forced a significant portion of its operations to spill to the CPU. This spill severely hampered performance due to the overhead associated with CPU-GPU synchronization, particularly for the mixture-of-experts (MoE) model architecture that Gemma 4 employs. The significance of this adjustment extends beyond individual performance gains. It highlights a key insight into optimizing ML models on constrained hardware. With the right configuration — raising the Metal wired-memory limit — the system can dramatically improve processing speeds, achieving up to 23.5 tokens per second for short prompts and retaining consistent performance even with longer contexts. This solution not only enhances the utility of Gemma 4 for developers and researchers using macOS platforms but also offers a practical case study for the broader AI/ML community on memory management and model optimization in resource-constrained environments.