What animal has a prime number of legs? And why does this confuse LLMs? (tris.sherliker.net)

A recent exploration into the question “What animal has a prime number of legs?” revealed some intriguing challenges faced by large language models (LLMs), particularly Google’s Gemini. While the intuitive answer is typically “humans” or bipedal animals with two legs, Gemini surprisingly attributed incorrect leg counts to various animals, mistakenly labeling 6 (insects) and 8 (arachnids) as primes. This confusion stems from LLMs' reliance on strong associations in their training data, which often prioritizes more common leg counts in taxonomy over the nuanced mathematical definition of prime numbers. The significance of this analysis lies in the insights it offers regarding the capabilities and limitations of LLMs. As they evolve, understanding how these models deal with nuanced queries can illuminate both their potential and their current pitfalls. Notably, while Gemini responded with a series of retracting confirmations, other models like Claude and Kimi provided clearer, more accurate answers, recognizing the distinction between legs and arms in creatures like starfish. This serves as a reminder of the ongoing need to refine LLMs to handle complex reasoning tasks and improve their accuracy in responding to questions that blend factual and interpretive elements.