Compiling Triton kernels without the Triton compiler (arxiv.org)

A recent investigation has explored the potential of large language models (LLMs) to serve as compilers by converting Triton kernels directly into NVIDIA PTX code, a process termed "AI lowering." This approach addresses the high costs and complexities of developing conventional compiler backends, which face challenges as programming paradigms and hardware evolve. By leveraging LLMs, the researchers successfully achieved performance enhancements of 0.83x to 3.34x over autotuned Triton on various GPU architectures, with significant improvements noted from unique transformations not executed by traditional methods. The findings are significant for the AI/ML community as they suggest a shift towards AI-driven compilers that could streamline software development for both general-purpose and custom chips. Key advancements include efficient memory usage through techniques like directly decoding packed binary weights and optimizing thread assignments—demonstrating the potential of AI to automate and enhance performance in compiler design. By extending existing verification tools like Volta to accommodate modern GPU features, this research opens pathways for less labor-intensive development cycles in future hardware and software ecosystems.