MIT's new fine-tuning method lets LLMs learn new skills without losing old ones (venturebeat.com)

Researchers from MIT, the Improbable AI Lab, and ETH Zurich have introduced a groundbreaking technique called self-distillation fine-tuning (SDFT) that allows large language models (LLMs) to learn new skills while preserving their existing knowledge. This advancement addresses a significant pain point for enterprises that currently face the challenge of catastrophic forgetting when fine-tuning models for new tasks, often requiring the maintenance of separate models for each capability. SDFT leverages the models' in-context learning abilities to create a feedback loop, where a "teacher" model, which remains static, assists a "student" model in learning from demonstrations and its own outputs, enabling on-policy learning without the need for explicit reward functions. The implications for the AI/ML community are profound, as SDFT not only enhances the ability of models to adapt to dynamic business environments but also reduces the need for costly retraining cycles. In trials, models using SDFT demonstrated superior performance in learning new tasks without regression on previous skills, proving particularly effective for complex enterprise-grade applications. Although SDFT comes with increased computational demands, its potential to streamline model management and improve adaptive learning positions it as a valuable approach for firms looking to consolidate AI capabilities across diverse departments.