I asked my local LLM to add 23 numbers and got seven wrong answers (viggy28.dev)

In a recent hands-on test, a user explored the capabilities of a local language model (LLM) while attempting to sum 23 stock transactions for tax filing. Despite using a robust setup with the Qwen 2.5 Coder model running on powerful hardware, the model produced seven incorrect answers over five hours, ranging from absurdly wrong to eventually correct. The author detailed each attempt, highlighting that smaller models often failed to compute arithmetic correctly, relying instead on pattern recognition without actually executing calculations. The final accurate total was only reached after clarifying the data input for the model to ensure it included all relevant transactions. This experiment underscores crucial lessons for the AI/ML community regarding the complexity of using local LLMs. It reveals that effective AI applications consist of multiple layers – including the model, inference engine, orchestrator, and harness – that work together to deliver reliable outputs. The findings stress that without a proper execution framework that can handle code, local LLMs are not reliable for computational tasks. This highlights the importance of integrating robust programming environments with AI models, especially for users who seek to employ these technologies beyond mere dialogue.