Show HN: PulsarForge – run a 744B MoE model on 32GB RAM with zero GPU (pure C) (github.com)

PulsarForge, a groundbreaking CPU-only LLM inference engine developed entirely in C11, has demonstrated the ability to run a massive 744-billion-parameter model (GLM-5.2) on a standard 2018 laptop with just 32GB of RAM and a USB SSD. This achievement stands out in the AI/ML community as it breaks new ground for running ultra-large models in a hardware-constrained environment, delivering impressive performance improvements. The engine validates its performance through rigorous testing against an independent oracle and offers bit-exact outputs across different platforms, making it a valuable tool for researchers and developers interested in LLMs. Key features of PulsarForge include expert streaming directly from disk, a dynamic caching mechanism, and high-performance I/O operations that allow it to fetch experts per token efficiently. With a measured decoding time of approximately 9 seconds per token, it significantly outpaces existing models that require high-end GPUs and extensive memory. This innovation opens up new possibilities for running large-scale AI models on accessible hardware, highlighting the potential of optimization techniques in machine learning frameworks. The detailed engineering retrospective provided thoroughly documents every milestone and measurement, contributing to the broader understanding of LLM efficiency and performance in constrained environments.