UBEP: Expert Parallelism Communication Library for Production Superpods (arxiv.org)

Researchers have announced UBEP (Unified-Bus Expert Parallelism), a revolutionary communication library designed specifically for deploying Mixture-of-Experts (MoE) models across high-bandwidth superpods such as NVIDIA’s NVL72/576 and Huawei’s CloudMatrix384. This initiative addresses critical limitations in these advanced computing environments, which traditionally suffer from execution serialization, high synchronization overheads, and load imbalances during interdependent communication phases. By re-engineering the MoE's All-to-All communication primitives, UBEP significantly enhances operational efficiency. This development is significant for the AI/ML community as it streamlines the deployment of sparse MoE models, unlocking the full potential of modern superpod architectures. Large-scale experiments indicate that UBEP can reduce All-to-All latency by up to 52.4% and improve MoE inference Time Per Output Token (TPOT) by as much as 11.1%. These enhancements will not only accelerate the training and inference processes for complex models but also enable researchers and developers to leverage existing high-bandwidth infrastructures more effectively, paving the way for more advanced AI applications.