Apple Neural Engine: Architecture, Programming, and Performance (arxiv.org)

A recent analysis has shed light on the architecture and functionality of Apple's Neural Engine (ANE), the dedicated matrix accelerator incorporated in Apple’s A11 and M1-class chips. This reverse-engineered study delves into the complexities of ANE by measuring its performance and analyzing its underlying systems, including the private runtime and firmware. It documents important technical details such as the engine's throughput limits, energy efficiency, and the data dispatch pathways that lie beneath the Core ML framework. The findings span across multiple chip generations, revealing crucial insights into Apple's architectural advancements from the A11 to A18 and M1 to M5. This research is significant for the AI/ML community as it uncovers the inner workings of a key hardware component that thrives on executing machine learning tasks efficiently. By offering an understanding of ANE's capabilities and limitations, developers and researchers can better harness its potential for real-time AI applications. The study also raises awareness about the undocumented nature of direct access methods, which, while meant for research purposes, may lead to innovative uses in on-device machine learning if properly explored. Overall, this deep dive into the ANE not only enhances the technological discourse around Apple's chips but also paves the way for refining machine learning implementations in consumer products.