Axe: A Simple Unified Layout Abstraction for Machine Learning Compilers (arxiv.org)

A recent announcement has unveiled Axe Layout, a novel abstraction designed to enhance the performance of machine learning compilers by addressing the complexities of modern deep learning workloads. Axe provides a unified framework that effectively maps logical tensor coordinates into a multi-axis physical space, accommodating various aspects of data management such as tiling, sharding, and replication. This hardware-aware approach enables more efficient distribution across diverse device meshes and memory hierarchies, fostering consistency in expressing collective operations from device-level constructs down to individual threads. The significance of Axe for the AI/ML community lies in its ability to drastically improve performance metrics, as experiments indicate that its unified methodology can achieve efficiency levels comparable to hand-tuned kernels across modern GPU devices and multi-device setups. This advancement is particularly crucial as the demand for sophisticated deep learning models continues to grow. By integrating thread-local control with collective operators into a single kernel through a multi-granularity distribution-aware domain-specific language (DSL) and compiler, Axe has the potential to simplify the complexities inherent in optimizing machine learning software for diverse hardware configurations, ultimately accelerating the deployment of AI applications.