Why LLMs can't play chess (www.nicowesterdale.com)

Recent YouTube videos featuring LLMs (Large Language Models) floundering at chess, such as those by Gotham Chess, highlight a fundamental flaw in their architecture: LLMs struggle to grasp complex rule-based systems like chess. While they can handle common opening moves due to their extensive training data, they fail miserably as the game progresses into mid and late stages. This is because LLMs do not maintain an internal representation of the board and rely purely on pattern matching, leading to legal move violations and erratic gameplay. As positions become unfamiliar, their predictions become increasingly random and ungrounded, resulting in numerous illegal moves. This phenomenon is significant for the AI/ML community as it underscores the limitations of LLMs in tasks that require understanding of specific rules and contexts. Unlike advanced chess engines like Stockfish, which utilize systematic search algorithms and maintain a clear board representation, LLMs simply generate outputs based on correlations from their training data. The disparity is architectural, not just a matter of scale or data. While LLMs have shown brilliance in certain problem-solving scenarios, this incident serves as a cautionary tale against applying them indiscriminately to complex strategic games like chess, revealing the necessity of specialized architectures for handling such intricacies.