GitHub – Maderix/ANE: Training Neural Networks on Apple Neural Engine (github.com)

A new project has emerged that demonstrates training neural networks directly on Apple's Neural Engine (ANE), utilizing reverse-engineered private APIs. This groundbreaking approach bypasses Apple's official CoreML training APIs and GPU frameworks, enabling custom compute graphs, including backpropagation, to run exclusively on the ANE. The implementation, focused on transformer training, showcases impressive performance metrics, achieving 9.3 milliseconds per step with 11.2% utilization on a single transformer layer using the M4 chip, which offers 15.8 TFLOPS for inference. This development holds significant implications for the AI/ML community by unlocking the potential of Apple's powerful hardware for model training, traditionally restricted to inference tasks. By utilizing techniques like in-memory compilation and optimizing data layout, the project highlights key advancements in leveraging ANE computing capabilities. Although currently limited to a single transformer layer, it opens avenues for further exploration into more complex multi-layer models and real training data applications, potentially transforming workflows for developers working within the Apple ecosystem.