Distilling proprietary model reasoning into open-source search agents (arxiv.org)

A new framework called Multi-Agent Protocol Distillation (MAPD) has been developed to enhance the distillation process of proprietary AI models into open-source search agents. This approach addresses the challenges of conventional knowledge distillation, which often struggles with varied underlying data representations and superficial transfer of skills. MAPD employs a multi-agent system that decomposes tasks, retrieves supporting evidence, and translates exploration traces into a structured JSON protocol. This allows for improved training using a combination of natural language imitation and reinforcement learning, yielding denser supervisory signals essential for effective training. The significance of MAPD lies in its ability to effectively bridge the gap between proprietary model capabilities and open-source applications, addressing the pervasive issue of style drift while enhancing reasoning competence. In extensive evaluations across seven question-answering benchmarks, MAPD has demonstrated superior performance, with average success rates of 39.4% for Qwen3-1.7B and 44.4% for Qwen3-4B, outperforming existing distillation and reinforcement learning methods. This framework not only promises to democratize access to advanced reasoning abilities in AI models but also highlights the potential for collaborative innovation in AI/ML technologies.