Learning, Fast and Slow: LLMs That Adapt Continually (gepa-ai.github.io)

A recent advancement in LLM training introduces Fast-Slow Training (FST), a novel approach that differentiates between "fast weights" and "slow weights" to enable continual adaptation. FST optimizes parameters in an interleaved manner, with fast weights managing task-specific insights and slow weights maintaining general reasoning capabilities. This methodology enhances data efficiency, improves performance ceilings across various benchmarks, and facilitates continual learning without sacrificing the model's adaptability to new tasks. Significantly, FST outperforms traditional weights-only training by achieving similar accuracy with up to 70% lower KL divergence from the base model, effectively preserving its dynamic learning capabilities. The introduction of context optimization through fast weights allows the model to absorb task-specific information more rapidly, addressing issues like catastrophic forgetting often seen with heavy reliance on reinforcement learning. Overall, FST not only positions itself as a crucial step toward developing general-purpose AI that can swiftly adapt to diverse tasks but also lays the groundwork for a more robust framework in continual learning for LLMs.