Toward Better Hip Kernel Generation for AMD GPUs (scalingintelligence.stanford.edu)

Recent advancements in hip kernel generation for AMD GPUs have been unveiled, showcasing a novel approach that incorporates synthetic data, multi-agent search, and reinforcement learning (RL). Researchers created a synthetic dataset comprising 500 new PyTorch reference tasks through mutation, composition, and constraint-based generation, aiming to improve the performance of language models in generating high-quality Kernel Intermediate Representation (HIP) kernels. The project employs a multi-agent optimization pipeline involving task generation, HIP translation, hardware evaluation, and evolutionary optimization, effectively overcoming the limitations of traditional single-shot prompting. This work is particularly significant for the AI and machine learning community as it addresses the performance bottleneck caused by the inefficiency of generating high-quality kernels, which relies heavily on specialized knowledge of hardware and optimization techniques that are scarce outside NVIDIA’s CUDA framework. The integration of small, low-cost open-source models, specifically using supervised fine-tuning followed by an RL approach, led to marked improvements in both compilation and correctness rates. However, the authors highlight that achieving substantial speedup over PyTorch necessitates greater hardware awareness in future iterations, which they intend to enhance by incorporating the ROCm profiler to guide optimization. This progression not only expands the capabilities of AMD GPUs in AI workloads but also contributes to a more diverse and accessible ecosystem for kernel generation and optimization.