From the creator of Redis; run LLM locally with ds4 (dwarfstar.sh)

DwarfStar 4 (ds4), created by the founder of Redis, introduces a groundbreaking local inference engine that enables developers to run large language models (LLMs) locally on high-memory machines, such as those powered by CUDA and ROCm. The engine employs asymmetric 2-bit quantization to effectively manage routed mixture-of-experts (MoE) models, ensuring that critical paths remain precise while compressing other components. This innovative approach allows for practical deployment of models like DeepSeek V4, GLM 5.x, and Qwen3.8 Flash in environments typically reliant on remote servers. The significance of ds4 lies in its ability to democratize access to advanced AI capabilities without needing extensive server infrastructure. By supporting local APIs, command-line interfaces, and a native agent, developers can seamlessly integrate AI into applications and maintain persistent coding sessions. With impressive performance metrics, including fast prompt pre-filling and efficient text generation, ds4 is poised to enhance the efficiency and accessibility of AI applications, fostering further innovation in the AI/ML community.