SuperCoder: Assembly Program Superoptimization with Large Language Models (arxiv.org)

Researchers have introduced SuperCoder, a novel approach that utilizes large language models (LLMs) for the superoptimization of assembly programs. Superoptimization aims to enhance the performance of existing code while maintaining its correctness. In a significant advancement for the AI/ML community, this study presents the first large-scale benchmark comprising 8,072 real-world assembly programs, challenging LLMs to generate assembly code that outperforms traditional industry-standard compilers like gcc -O3. The leading LLM, Claude-opus-4, achieved a 51.5% test-passing rate with an average speedup of 1.43x, highlighting the potential of LLMs in this domain. Further improvements were made through fine-tuning, incorporating reinforcement learning to optimize both correctness and performance. The customized model, SuperCoder, demonstrated remarkable results with a 95.0% correctness rate and a 1.46x average speedup. This breakthrough establishes a new frontier for using LLMs as superoptimizers, suggesting a shift in how program performance can be improved beyond the capabilities of existing compiler heuristics. The implications of this research extend to future optimization techniques, opening new avenues for leveraging AI in software development and enhancing computational efficiency.