Show HN: I Let AI Agents Train Their Own Models. Here's What Happened (hamzamostafa.com)

A developer has created a system named Tinkerer that allows AI agents like Claude Code and OpenAI Codex to autonomously fine-tune language models without human oversight. After running over 100 experiments, the best performance resulted in a 3B model achieving near-perfect arithmetic capabilities in just 18 minutes at minimal cost. However, the experiments also revealed limitations, such as the agents' inability to recognize and address fundamental issues like a broken learning rate scheduler, highlighting a crucial distinction between executing a training process and conducting ML research. Tinkerer's significance lies in its demonstration of how advanced AI agents can run complete training pipelines independently, generating data, writing reward functions, and selecting hyperparameters. While the project validates the potential for AI-to-AI training loops, it also underscores the current gap between procedural execution and the cognitive judgment required for genuine ML research. Consequently, while Tinkerer marks a step toward more autonomous AI research, it emphasizes that training and research require different skill sets, with AI still on the journey to mastering the latter. The open-source nature of Tinkerer invites further exploration and experimentation in this evolving field.