We still need small language models – even in the age of frontier AI (www.turing.ac.uk)

Researchers have demonstrated the power of small language models by achieving near-frontier reasoning performance with a 3 billion parameter model. In a focused six-week project, they utilized techniques such as retrieval-augmented generation, reasoning trace fine-tuning, and budget forcing during inference to tackle real-world health queries effectively. This local model can run on a standard laptop, making it accessible for public sector and resource-constrained environments. Their work highlights the potential of lean models to deliver robust performance without the need for expansive resources or reliance on commercial APIs, which often come with significant cost and privacy concerns. The significance of this approach lies in its practicality and scalability. By leveraging test-time scaling techniques like budget forcing, which encourages deeper reasoning without altering the model's weights, the researchers found their model capable of handling complex tasks previously thought suitable only for larger, more resource-intensive models. The team is open-sourcing their methods and model, promoting transparency and enabling others to develop AI applications in sensitive or limited-resource settings. This important step underscores the enduring relevance and capability of smaller language models in the rapidly evolving AI landscape.