Show HN: Maple – Run Gemma 4 26B on a 6GB Laptop GPU at 45 TPS (Sub-2-BPW) (amsach.github.io)

A new innovation called MAPLE (Manifold-Activated Pruning of Latent Experts) enables the execution of the 26-billion-parameter Gemma model entirely in GPU memory on a consumer-grade 6GB laptop, achieving a remarkable 45.4 transactions per second (TPS). This breakthrough compresses large models from 128-expert architectures down to just 4.75 GB, eliminating bandwidth issues typically caused by PCIe system-RAM paging. By utilizing a high-speed GDDR6 bus, MAPLE resolves the limitations of conventional hardware setups that struggle with high memory demands, effectively preventing what has been termed the "PCIe Paging Death Spiral." From a technical perspective, MAPLE employs advanced techniques like manifold activation analysis and live routing profiling to dynamically prune dormant experts based on their activation thresholds. This selective removal not only maintains the model's performance but also allows for efficient quantization of the preserved expert tensors into a compact binary. With streamlined deployment through popular frameworks like Llama and LM Studio, this development represents a significant step towards making sophisticated AI models more accessible and usable on standard consumer hardware, opening the door for broader applications in AI and machine learning.