Moefit: Fit Qwen 3.8 Flash on your Mac with limited ram (github.com)

Moefit has introduced a groundbreaking method for running the Qwen3.8-Flash-Next mixture-of-experts (MoE) model on Mac systems with limited RAM. This innovative solution allows users with 24 to 64 GB of RAM to execute a ~125 billion-parameter model by keeping essential weights in memory while efficiently paging in expert weights from SSD as needed. This approach effectively overcomes the typical RAM constraints that would otherwise prevent loading such large models, making advanced AI capabilities accessible on a wider range of Apple Silicon devices. The significance of Moefit lies in its technical innovation that optimizes resource usage, which has direct implications for the AI/ML community. By employing a smart paging mechanism and maintaining a "hot-set" of frequently used experts in RAM, the performance of the model is enhanced without demanding the entire model to reside in memory. With measurable token decode rates varying by system configurations, this model can still achieve reasonable processing speeds even on Macs with lower RAM, which is crucial for developers and researchers seeking effective AI solutions on consumer-grade hardware.