GPU Forecasters: Language Models as Selective Surrogates for Kernel Optimization (arxiv.org)

Researchers have announced a novel approach using large language models (LLMs) to optimize GPU kernel performance by acting as selective surrogates for kernel evaluation. Traditionally, optimizing GPU kernels involves costly evaluations that require extensive compilation and execution on hardware. This new method leverages LLMs to forecast the performance of proposed kernels, thereby reducing the number of necessary evaluations on the GPU. The study highlights the importance of accuracy and calibration in these forecasts, ultimately allowing more candidates to be tested within the same GPU measurement budget. Significantly, the findings indicate that employing LLMs in this capacity not only improves the efficiency of kernel searches but also enhances the discovery of faster kernels compared to conventional methods. The research further explores the integration of reinforcement learning to refine forecast accuracy and confidence. This advancement suggests that LLMs may play an essential role in future kernel optimization strategies, serving as sophisticated virtual models of GPU performance rather than merely tools for generating kernel code. This shift could lead to substantial efficiency gains in deep learning applications, propelling the AI/ML community forward.