Self Improvement Harness based on pi. pi-harness-rsi (github.com)

A new extension called pi-harness-rsi has been announced, designed to facilitate self-improvement for language models through a methodical loop process. This extension employs what is termed an "enforced context economy," whereby every artifact entering a language model (LLM) context is subject to a predefined budget, ensuring clarity and predictability in training without silent truncation of data. The system is highly documented and includes an executable test suite for rigorous validation, making it a robust tool for improving model outputs efficiently. This development is significant for the AI/ML community as it introduces a structured way to optimize LLMs and enhance their capabilities over time. The workflow consists of several key stages: snapshotting the model’s current state, probing tasks multiple times for accuracy, proposing adjustments based on actionable insights, and adopting successful changes into the model's library. The rigorous tracking of task execution and the provision of evidence-based digests will help developers identify inefficiencies and foster continual improvement in LLM performance, making pi-harness-rsi a potentially transformative addition to AI tooling.