Mini-AGI – dynamic continual learning model trained from scratch on 8GB VRAM (github.com)

The newly announced mini-AGI is a groundbreaking continual learning model designed to function on consumer-grade hardware, specifically a single GPU with 8 GB of VRAM. Unlike traditional models which are trained and then frozen, mini-AGI continuously learns from real-time data without experiencing catastrophic forgetting. This capability is achieved by dynamically assembling its architecture during training, reading data one byte at a time, and managing memory by storing weights as files on disk. As it trains, the model can expand its capacity if necessary, ensuring that users can tailor their training to fit their specific needs and resources. This development is significant as it democratizes access to continual learning techniques in AI, allowing individuals to train and maintain their own models without extensive resources. The model's architecture employs innovative techniques such as adaptive depth and a routing mechanism that adapts to the learning demands of each character it processes. The ability to use existing hardware means that more people can engage with AI/ML projects, potentially leading to a variety of customized applications and further advancements in the field. Despite being a "toy-level" model for now, mini-AGI represents a promising step towards more flexible and user-friendly AI systems.