Coding with DeepSeek 4 on a 128GB MacBook Pro (ronreiter.com)

DeepSeek V4 Flash, a groundbreaking 284-billion-parameter Mixture-of-Experts model, was recently demonstrated running entirely offline on a 128GB Apple MacBook Pro. With a robust setup, the model uses a unique 2-bit quantization approach, dubbed "Dwarf Star," to maintain operational efficiency despite its large size. This model achieves around 21 tokens per second generation speed when utilizing the Metal GPU, making high-performance local AI accessible to users without requiring cloud resources. This announcement is significant for the AI/ML community as it showcases the increasing capability of personal computing hardware to handle advanced machine learning models. The ability to run such a complex model locally could enhance privacy, especially for sensitive applications in areas like software development and experimentation. A unique aspect of the setup is its compatibility with various agent harnesses, notably allowing integration with Claude Code and Pi while leveraging the Sparse Attention mechanism, which drastically reduces the computational load of the key-value cache. Though some limitations persist with context loading at extremely high token counts, the insights gained from this implementation could pave the way for more streamlined local AI implementations in the future.