Two Hills: Weights and Harnesses (rlafuente.com)

A recent publication by Anthropic explores the critical distinctions between two approaches to improving language model performance: changing model weights versus modifying the agent harness. The author introduces the concept of "hills," where the "weights hill" deals with optimizing the model’s parameters through established methods like reinforcement learning and supervised fine-tuning, while the "harness hill" focuses on iterative, discrete adjustments to the code that defines how the model operates. The piece emphasizes that evaluating these harness changes is a newer practice, just emerging in the last few years, and requires careful measurement since it lacks the clear feedback mechanisms seen in weight optimization. This discourse highlights a significant challenge in AI/ML: while the community has developed sophisticated techniques for optimizing weights, harness optimization remains largely uncharted territory. The article notes the convergence of both hills, as environments used for training coding agents overlap with those for evaluating harness effectiveness. By bringing attention to the evaluation process's complexity and identifying the need for rigorous metrics, the author sets the stage for a comprehensive series aimed at enhancing the methodologies used in agent development, which could greatly impact the AI/ML landscape moving forward.