Karpathy's autoresearch, 50 DPO experiments, 300 human judges (huggingface.co)

A recent study explored the effectiveness of autonomous research agents in optimizing machine learning models, utilizing Karpathy's autoresearch framework with a focus on fine-tuning SmolLM2-360M-Instruct. The agent autonomously conducted 50 experiments but struggled to improve its performance metric, ultimately producing results below chance level according to its own assessments. In contrast, human judges rated recipes generated from a brief guided interaction with the same model more favorably, highlighting the disconnect between automated metrics and human preferences. The study found that human involvement, even minimal, enabled the model to propose more effective strategies that significantly outperformed the results of the autonomous experiments. The findings are pivotal for the AI/ML community as they underscore the limitations of relying solely on automated processes to evaluate and optimize models. The research revealed that while autonomous agents are capable of running extensive experiments, they may lack the ability to pivot creatively or meta-cognitively, as seen with the successful introduction of methods like LoRA adapters and more selective data filtering that the agent's loop failed to consider. This emphasizes the importance of human intuition and oversight in research workflows, suggesting that a collaborative approach may yield more robust outcomes in AI development.