Introducing HRM-Text (sapient.inc)

Sapient has unveiled HRM-Text, an innovative 1.15-billion-parameter language model designed to excel in reasoning tasks while significantly reducing the computational resources typically required. Trained on just 40 billion tokens—up to 1,000 times less than conventional models—HRM-Text demonstrates competitive performance across various benchmarks, achieving notable scores such as 56.2% on MATH and 82.2% on DROP. The model can be pretrained in approximately one day for a cost of around $1,000, making advanced AI capabilities more accessible than ever. The significance of HRM-Text lies in its ability to perform sophisticated reasoning through a unique hierarchical latent recurrent architecture, which allows it to learn from structured instruction-response pairs rather than mere token prediction. This approach not only enhances the model's efficiency and reasoning depth but also enables it to run locally on devices with a minimal footprint of just 0.6 GiB. As a result, HRM-Text opens the door for a wider range of researchers and smaller enterprises to experiment with and deploy advanced AI systems without the barriers posed by traditional large-scale training requirements, while also ensuring data privacy and security for sensitive applications in fields like healthcare and finance.