Replacing Attention to Phase-Locking to Overcome Catastrophic Forgetting [pdf] (arxiv.org)

Researchers have proposed a novel approach to address the "catastrophic forgetting" phenomenon in AI models, particularly focusing on Transformers. The IEEE paper introduces the Phase-Resonant Intelligent Spectral Model (PRISM), which replaces traditional quadratic self-attention mechanisms with linearithmic Gated Harmonic Convolutions. This shift is crucial as it encodes semantic identities through resonant frequencies, enabling the model to better retain learned information when exposed to new concepts. The significance of this advancement lies in overcoming a limitation known as "Catastrophic Rigidity," where conventional models struggle to learn new data without sacrificing previously acquired knowledge. In rigorous testing, PRISM outperformed the Standard Transformer by maintaining high performance during knowledge acquisition, achieving 96% success in a 5-shot task while experiencing minimal degradation in performance. In contrast, the Standard Transformer experienced a 10.55-point drop in performance when retraining, underscoring PRISM's potential in real-time adaptability. These findings suggest a promising direction for the AI/ML community, offering a structural solution to the ongoing plasticity-stability dilemma and enhancing the capability of models to adapt to new information seamlessly.