Training and Inference on Huawei Ascend NPUs (github.com)

Huawei has announced the release of DeepEP-Ascend, a high-performance communication library specifically designed for machine learning training and inference on Huawei's Ascend NPUs. This library enhances expert-parallel (EP) operations, supporting a range of functionalities including MoE dispatch/combine, FP8 and BF16 data handling, as well as pipeline and data parallelism through Bucket collectives. The library is notable for aligning its public buffer APIs with NVIDIA's DeepEP, enabling broader compatibility and flexibility for developers working in AI and ML environments. This development is significant for the AI/ML community as it promises to optimize communication efficiency and throughput in distributed models, which is crucial for scaling machine learning workloads. The performance benchmarks indicate impressive bandwidth capabilities on the Ascend 950DT NPU, reaching up to 375 GB/s for dispatch operations. With continuous improvements and optimizations planned, DeepEP-Ascend aims to support advanced AI applications and provide a robust framework for researchers and developers aiming to harness the power of Huawei's NPU architecture, potentially enhancing the competitiveness of AI deployments at scale.