Recursive Self-Improvement via On-Policy Distillation for Reasoning (arxiv.org)

A recent study introduces a novel approach to model training called Recursive Self-Improvement via On-Policy Distillation (OPD) for reasoning tasks. This method involves training a student model to generate predictions that align with those of a teacher model, but with a twist: the teacher remains static, which has been beneficial for stability. However, the research highlights a limitation in this approach, as the teacher fails to adopt improvements made by the student during training. To overcome this, the authors propose a recursive framework that employs Dynamic Co-Evolution (DCE), allowing the teacher to evolve alongside the student, and Self-Refined Concise Learning (SRCL) to enhance the clarity and brevity of outputs. The significance of this research lies in its potential to substantially advance the efficiency and effectiveness of AI reasoning models. Comprehensive evaluations demonstrate that the DCE+SRCL approach outperforms the traditional OPSD methodology across multiple scales, achieving a remarkable 65.97% accuracy on the Qwen3-8B model, a 35.62 percentage point improvement over OPSD, while also reducing average response length by 7.80%. This innovative combination not only enhances performance metrics but also addresses concerns about verbosity in AI outputs, marking a notable step forward in AI/ML model training techniques.