Falcon-H1R: Hybrid Model for Efficient Test-Time Scaling (falcon-lm.github.io)

The Technology Innovation Institute (TII) in Abu Dhabi has announced the launch of Falcon H1R 7B, a decoder-only large language model that raises the bar for reasoning capabilities while maintaining a compact size of 7 billion parameters. This model not only matches but often surpasses the performance of much larger models (2-7 times its size) across various reasoning-intensive benchmarks, demonstrating its remarkable parameter efficiency. The training strategy combines a tailored two-stage pipeline involving supervised fine-tuning and reinforcement learning (RL) with confidence-aware filtering during test-time scaling, known as Deep Think with Confidence (DeepConf), which enhances its decision-making and reduces token generation. Significantly, Falcon H1R 7B excels on key metrics, leading math benchmarks with results such as 88.1% on AIME-24 and outperforming competitors in code and general tasks. With its efficient inference capabilities—achieving up to 1,800 tokens per second during testing while generating fewer tokens—it positions itself on a new frontier of performance versus computational cost. This model's open-source release under the Falcon LLM license invites collaboration from the AI/ML community, further promoting research and application development in this rapidly evolving field.