Training LLMs on 1080 Tis without shadow weights (github.com)

A new proof-of-concept project named PRIMAL has been announced, revolutionizing the training of Large Language Models (LLMs) on consumer GPUs like the GTX 1080 Ti without the typically required shadow weights. This innovation removes the need for dual memory allocation, effectively allowing for training directly on a 4-bit integer grid. By leveraging a custom 13-value Look-Up Table (LUT) based on prime reciprocals, PRIMAL improves precision where it matters most, facilitating larger batch sizes and enhanced throughput, with results showing significant efficiency improvements in VRAM usage. The significance of PRIMAL lies in its ability to streamline LLM training, enabling more accessible and cost-effective model development for researchers using older hardware. The innovative use of a Discrete Optimization Loop and a unique method to manage potential stochastic thrashing makes the approach robust. Additionally, the metrics reveal a training VRAM requirement of just 10.3 GB for a 0.1B parameter model at full saturation and an impressive training throughput of approximately 6,000 transactions per second. This advancement promises to democratize access to LLM training, making it feasible for a broader audience to participate in AI research and development.