ANEForge: Program the Apple Neural Engine Directly, from Python (aneforge.com)

ANEForge has introduced a groundbreaking tool that enables developers to program the Apple Neural Engine (ANE) directly using Python, bypassing CoreML. This innovation allows for comprehensive model training and inference purely on the ANE chip, which was previously limited to inference tasks via CoreML. ANEForge compiles a tensor graph into a single program that operates end-to-end on the engine, facilitating not only neural network training but also supporting a wide array of computations, including FFT, linear algebra, and simulations. Notably, the tool has shown promising results, achieving a 71% accuracy on the CIFAR-10 dataset and enabling the execution of large language models (LLMs) with complex tasks directly on the ANE. This development is significant for the AI/ML community as it opens up new possibilities for utilizing Apple's hardware in machine learning projects without the constraints of CoreML. By utilizing ANEForge, developers can now access lower-layer operations and bridge functions that CoreML does not expose, optimize weight compression, and run diverse workloads more efficiently. Moreover, its ability to cross-compile across various Apple silicons (M1 to M5) further enhances its utility, allowing users to estimate latency and optimize performance effectively. Overall, ANEForge represents a major step forward in leveraging Apple's hardware for comprehensive AI model development.