Co-Evolving Harnesses and Models (arxiv.org)

A recent study introduced a novel approach to agent harnesses and models in AI, aimed at enabling weaker models to excel at domain-specific tasks. By allowing smaller models to evolve within an automated harness, the study reveals that even lesser-performing AI can match larger models in efficiency while significantly reducing costs. The researchers found that an expert model can utilize the evolved harness more effectively, but they also discovered that training the weaker model using expert trajectories led to performance regressions, as the model could not effectively adopt the expert's planning style. To overcome this challenge, the team developed an on-policy expert-correction pipeline, which allows the stronger model to assist the weaker one by rewriting only those components where performance faltered. This approach not only helps maintain the planning style of the weaker model but also integrates the advancements from the harness evolution, resolving the compatibility issues between model updates and harness modifications. This significant advancement has the potential to reshape the landscape of AI training, enabling more economical and efficient development of specialized AI systems across various enterprise tasks.