AdaExplore: Search for Efficient Kernel Generation (stiglidu.github.io)

AdaExplore, a novel LLM agent designed for efficient GPU kernel generation, has been introduced with a two-stage pipeline that enhances correctness and performance without any fine-tuning. The first stage, Failure-Driven Adaptation, distills knowledge from past mistakes into a reusable memory of "you cannot..." rules, while the second stage, Diversity-Preserving Search, maintains multiple kernel candidates in parallel. This dual approach addresses significant challenges in kernel generation—specifically, the narrow boundary of correctness and the complex optimization landscape—by allowing the agent to avoid repeating prior errors and efficiently explore structural changes needed for performance improvements. Testing has shown that AdaExplore significantly enhances the performance of kernel generation across various models. The Adapt stage increases the pass@1 correctness for every model tested by creating a compact skill memory that informs future proposals, resulting in fewer errors. Meanwhile, the Explore stage employs a Monte Carlo Tree Search (MCTS) to refine existing candidates and generate new ones, balancing small local edits with larger structural changes. The outcome is a marked improvement in both kernel correctness and speed, validated against KernelBench, solidifying AdaExplore's role as a promising tool for the AI/ML community focused on enhancing automated programming tasks.