Can LLMs give us AGI if they are bad at arithmetic? (wesmckinney.com)

A recent exploration into the arithmetic capabilities of large language models (LLMs) revealed surprising limitations, raising questions about their potential contribution to achieving artificial general intelligence (AGI). Despite significant advancements in AI, especially with tools like Claude Code, LLMs still struggle with basic tasks like arithmetic operations, particularly when aggregating data from larger datasets. While users can leverage these models for routine coding tasks and project development, they often face challenges related to inconsistency and cognitive deficits—issues that cast doubt on the immediate viability of LLMs as precursors to AGI. The investigation involved rigorous testing of several models, including OpenAI's GPT-4 and Anthropic's Claude, revealing a consistent drop in accuracy as the complexity of arithmetic tasks increased. For instance, many models began to falter when asked to sum over ten numbers or to solve problems with larger datasets, where local models like GPT-OSS exhibited superior performance. This performance gap raises implications for developers and researchers alike, highlighting the need for improved model reliability and cognitive functions. The findings suggest that while LLMs can enhance productivity in software development, their arithmetic deficiencies may hinder the journey toward true AGI.