Arithmetic Without Numbers – How LLMs Do Math (alvaro-videla.com)

Recent research from Rune showcases a significant advancement in how large language models (LLMs) like Llama can perform arithmetic tasks without relying on traditional programming routes. Instead of routing arithmetic calculations solely through Python or similar tools, the study demonstrated that LLMs can derive necessary inputs directly from their internal states, enabling them to execute operations such as greatest common divisor (gcd), least common multiple (lcm), and division with remainder based solely on prompts. This ability was validated against a vast dataset, showing improved accuracy in providing exact answers compared to a frozen model. The implications for the AI/ML community are substantial, as this mechanism highlights the models' potential for enhanced reasoning capabilities in performing mathematics. Notably, across 11,736 examples, the model accurately answered arithmetic prompts with significant gains, achieving near-perfect accuracy in gcd calculations and impressively correcting what the unassisted model missed. This research not only paves the way for more reliable AI-assisted computations but also opens up further inquiries into the cognitive functions of LLMs, posing intriguing questions about how they interpret and manipulate numerical information without explicit numerical models or programming.