Qwen3.8-27B on the iPhone 18 Neural Engine at 15–70 tok/s (1.5 bpw) (twitter.com)

The recent announcement of the Qwen 3.8-27B model running on the iPhone 18's Neural Engine highlights significant advancements in mobile AI capabilities. With this integration, the Qwen model, leveraging a draft configuration known as dflash-2, achieves an impressive decoding speed ranging from 15 to 70 tokens per second (tok/s) at a bit-per-weight (bpw) of 1.5. This translates to a more efficient use of resources while maintaining performance, showcasing the potential for sophisticated AI applications directly on mobile devices. For the AI/ML community, this development represents a leap toward enabling complex machine learning tasks on personal devices, with implications for real-time applications in areas such as natural language processing and image recognition. The model's innovative use of vector quantization (VQ) techniques allows for a mixed approach that enhances efficiency with only a minor drop in acceptance rate, making it a compelling solution for mobile AI deployment. By increasing accessibility to advanced AI functionalities, Apple and Qwen are setting the stage for smarter applications that can leverage powerful AI models seamlessly within everyday mobile contexts.