Show HN: Maple – Sub-2-BPW Moe Pruning to Run Gemma 4 26B at 45 TPS on a 6GB GPU (github.com)

A new method called MAPLE has been introduced, utilizing Sub-2-Bits Per Weight Structural Compression to optimize the Gemma 4 26B-A4B model, reducing its size from 9.92 GB to 4.75 GB. This advancement allows the model to run entirely within the VRAM of consumer 6 GB GPUs at an impressive throughput of 45.4 tokens per second. Specifically designed for Python programming, algorithm synthesis, and formal STEM reasoning, the optimized model prunes away 80 dormant multilingual experts while preserving the essential capabilities of the remaining 48 active experts. The significance of MAPLE lies in its potential to enhance localized edge deployments by dramatically improving performance while lowering memory requirements. By applying a composite frequency-energy scoring metric during the pruning process, MAPLE efficiently identifies and retains only the most relevant experts, achieving a 6.57x speedup in generation speed and reducing VRAM usage by 32%. This innovation not only makes advanced AI models more accessible on consumer hardware but also opens pathways for effective applications in various specialized fields, from finance to creative writing, by leveraging a domain-agnostic pruning approach. The pre-built model is publicly available on Hugging Face, facilitating easy access for developers looking to harness its capabilities.