Show HN: Sillage a 4 MB memory that lets a frozen language>model remember (github.com)

A new tool called Sillage has been introduced, enabling frozen language models to retain memory in a compact format of just 4 MB, allowing them to learn continuously without the need for gradients or an increasing index size. The name “sillage,” French for the wake left behind by a passing object, highlights the model's ability to remember information from texts it has processed. By implementing four key mechanisms and utilizing a rank-16 adapter, Sillage can effectively lower perplexity and generate text based on previously read materials. The tool can be easily run on a laptop CPU, making it accessible for various applications. This development is significant for the AI/ML community as it offers a way to enhance the performance of language models while addressing memory limitations traditionally associated with machine learning systems. Sillage's fixed memory size supports efficient content recall, achieving up to a 38% reduction in perplexity compared to standard unbounded stores. The framework is versatile, functioning with multiple model architectures, and is designed to work offline with private documents. By optimizing how the model remembers and interacts with text, Sillage could transform the way language models are applied in real-world scenarios, particularly where data privacy and limited computational resources are concerns.