Let the Barbarians In (www.sigops.org)

In a groundbreaking development, researchers from UC Berkeley's ADRS team have showcased how artificial intelligence can enhance system performance research, coining the term AI-Driven Research for Systems (ADRS). Their latest findings demonstrate the ability of three open-source ADRS frameworks—OpenEvolve, GEPA, and ShinkaEvolve—to generate solutions that outperform human experts across ten real-world problems. Notably, their approach achieved a 13-fold speedup in load balancing for mixture-of-experts inference and a 35% reduction in costs for cloud job scheduling. This signifies a shift from simply tuning existing systems to enabling AI to autonomously rewrite system code, streamlining the research process. The significance of this work lies in its potential to revolutionize how systems performance problems are approached, leveraging AI's capabilities to automate tedious stages of research. By defining clear best practices for problem specification, evaluation, and feedback, the ADRS approach facilitates more effective use of AI tools, enabling researchers to focus on high-level design while AI manages the iterative discovery of efficient algorithms. As the models improve, ADRS could drastically reduce the time and cost involved in system research, potentially transforming the landscape of computer systems research and enabling faster innovations across various domains.