BitStack – 1-bit gradient masks for continual learning without replay (github.com)

A groundbreaking approach to continual learning in natural language processing, BitStack utilizes 1-bit gradient masks to significantly reduce the phenomenon of catastrophic forgetting in transformer models. In a five-task benchmark using the GPT-2 model—without relying on data replay or retraining—BitStack achieved a dramatic reduction in average forgetting from 14.5 percentage points (pp) to just 3.8 pp. This 73.8% improvement showcases the method's potential to maintain the performance of models as they learn across sequential tasks. The significance of BitStack lies in its innovative approach to preserving important model parameters for each task while blocking future gradient updates, a technique that can be pivotal for applications requiring ongoing learning in dynamic environments. The method involves using gradient hooks to zero out gradients for masked weights after each task, exemplifying an efficient way to enhance transformer classifiers' longevity and adaptability. As an independent high school research project, these promising early results open the door for further exploration and optimization in the field of AI/ML, especially for applications where training data is scarce or continuously evolving.