I ran the full 2.8T-parameter Kimi-K3 on a 64GB Mac Mini (github.com)

An innovative demonstration shows that the massive 2.8 trillion-parameter Kimi-K3 model can be run locally on a 64 GB Mac Mini, utilizing external SSD storage for its significant MXFP4 weights. While the processing time clocks in at roughly 32.4 seconds per token—making it impractical for interactive use—this setup could be invaluable for tasks requiring confidentiality, such as legal or medical analyses that typically take days to weeks for professionals to complete. By leveraging Apple Silicon's MLX and Metal capabilities, the model achieves a unique integration that allows it to operate outside traditional cloud environments while maintaining data security. This achievement is significant for the AI/ML community because it democratizes access to powerful AI models by allowing them to run on consumer-grade hardware, rather than requiring expensive setups. Key technical innovations include a fused Metal MXFP4 kernel, which optimizes processing, and a latent MLA cache that drastically reduces memory usage. As the implications of local inference for sensitive information are explored, the focus shifts from raw speed metrics to time-to-useful-insight, which could revolutionize workflows in fields where data privacy is paramount.