Can AI automate AI R&D yet? (epoch.ai)

AI developers are exploring the potential of automating AI research and development (R&D) through a new evaluation framework called InnovationEval. This initiative seeks to test whether AI can independently discover novel machine learning techniques comparable to human innovations. Initial results indicate that, despite significant GPU investments, current AI models like GPT-5.6 Sol and Claude Fable 5 fell short of producing outcomes on par with the on-policy self-distillation method, a benchmark set by human researchers that they were tasked to surpass. While Sol achieved some minor improvements, they did not represent true innovation but rather refinements of existing techniques. The significance of this research lies in its aim to bridge the gap between current AI capabilities and the demands of complete R&D automation, highlighting the complexities involved in conducting end-to-end research projects. By implementing an end-to-end evaluation that requires AI to generate original algorithms, test them, and iterate, the framework emphasizes the importance of genuine innovation rather than merely reassembling existing methods. The findings shed light on AI's limitations in achieving true autonomy in the research process, an essential goal for advancing AI/ML capabilities in the future. These results will guide future experiments to better assess AI’s potential in automating AI research effectively.