Can LLMs Subtract Numbers? (arxiv.org)

Researchers performed a focused study on how well large language models handle subtraction, an understudied, non-commutative arithmetic operation. They evaluated eight pretrained LLMs across four model families on both addition and subtraction tasks and found subtraction performance trails addition by a wide margin. Errors are highly concentrated when the true result is negative (a < b): models often output the correct magnitude but drop the negative sign. Probing analyses reveal that internal model states do encode whether a result should be negative, yet that signal frequently fails to influence the generated token sequence. The work shows this failure is not irrecoverable: few-shot prompting gives modest improvements, while instruction-tuned models nearly eliminate the missing-sign error, producing correct negative outputs with near-perfect accuracy. That pattern implies the deficit is largely a decoding/output-conditioning problem tied to training and instruction alignment rather than a lack of internal numerical representation. For the AI/ML community, the paper highlights subtraction as a diagnostic for model numeracy and exposes how fine-tuning and instruction objectives can repair systematic output biases—important for applications requiring reliable numeric reasoning and for designing benchmarks that probe non-commutative operations.