CUDA-L2: Surpassing cuBLAS Performance for Matrix Multiplication Through RL (github.com)

CUDA-L2 has been introduced as a revolutionary system that optimizes Half-precision General Matrix Multiply (HGEMM) CUDA kernels using large language models and reinforcement learning. This new framework significantly surpasses existing matrix multiplication solutions, including popular options like torch.matmul and advanced NVIDIA libraries such as cuBLAS and cuBLASLt. The release includes optimized HGEMM kernels tailored for the A100 GPU across 1,000 different matrix configurations, aimed at enhancing computational efficiency in deep learning applications. The significance of CUDA-L2 lies in its groundbreaking approach to automating kernel optimization, which could substantially accelerate training and inference in AI/ML workloads. The system supports denser matrix configurations and is compatible with newer GPU architectures like Ada Lovelace and Hopper. With straightforward deployment for open-source large language models, CUDA-L2 stands to transform matrix multiplication tasks by providing users with high-performance, easily accessible solutions. The careful design ensures that kernels are specifically tuned for performance on the A100, although there’s potential for compatibility with other GPUs, suggesting a broader impact on the AI/ML community.