What If the Harness Comes Before Pretraining? A Data Flywheel Perspective (hanchenli.github.io)

A recent blog post challenges the conventional sequence of the AI/ML agent life cycle, which typically follows the order of pretraining followed by harness design. The author proposes that these two elements are intertwined, suggesting that harness design should occur before pretraining. This approach recognizes that the harness determines the types of data collected through a data flywheel, which then informs future pretraining efforts. The harness is not only an operational layer for the agent but also crucial for generating valuable training data that can help refine the model. This means that successfully deployed agents create opportunities for better training data, enhancing overall model performance. The implications of this perspective are significant as they highlight the potential biases ingrained during pretraining, such as models trusting their contexts even when they present incorrect information. The harness can serve as a corrective layer, introducing explicit rules to mitigate these biases. The author emphasizes a choice between encoding corrections within the harness or distilling human-like actions into model weights, with the former being more efficient in practice. Ultimately, this discussion calls for a reevaluation of how harnesses and pretraining interact, marking a pivotal shift that could optimize agent training and deployment in future AI/ML applications.