Alpie Core: a 32B reasoning model trained and served at 4-bit (huggingface.co)

Alpie Core, a groundbreaking 32 billion parameter reasoning model from India, has been launched as one of the first fine-tuned models operating at 4-bit quantization globally. Utilizing just 8 Hopper GPUs and employing techniques like LoRA for parameter-efficient fine-tuning and QLoRA quantization, Alpie demonstrates that aggressive quantization allows it to not only match but surpass traditional full-precision models in performance. With impressive benchmark scores—81.28% on MMLU and 92.75% on GSM8K—it showcases competitive capabilities against much larger proprietary models, highlighting a significant advancement in memory efficiency and inference speed. This development is crucial for the AI/ML community as it emphasizes the potential of smaller, efficiently trained models to perform at or above the level of larger counterparts, promoting more sustainable AI practices with a reduced carbon footprint. The training methodology also focused on rigorous safety and ethical standards, ensuring that Alpie Core produces reliable, context-aware, and culturally sensitive responses. The model's extensive context length of 65,000 tokens makes it particularly suited for a variety of use cases, including academic research and competitive exam preparation, while remaining open-sourced under Apache 2.0, contributing to the growing trend towards democratized access in AI technologies.