OMLX – Apple Silicon Optimized Local Inference (omlx.ai)

Apple has announced the release of OMLX, a macOS-native MLX server designed to optimize local inference on Apple Silicon. This innovative platform enables coding agents to access previously computed cache blocks stored on SSDs, drastically reducing the time it takes to retrieve information—from minutes to milliseconds. As a result, applications utilizing OMLX, such as Claude Code, OpenClaw, and Cursor, now respond in just five seconds instead of the typical 90 seconds, representing a significant leap in performance for machine learning tasks. The OMLX performance benchmarks were conducted using an M3 Ultra with 512GB of RAM, showcasing impressive token processing speeds across various models and context lengths. For example, the system achieves up to 2,009 tokens per second with 8k context length and demonstrates substantial speedups in batch processing, reaching up to 4.14 times the speed of single requests. This enhancement makes local AI implementations on Macs far more efficient, with users reporting that running Qwen3.5 models on OMLX outperforms existing solutions like LMStudio, making it an attractive option for developers and researchers in the AI/ML community.