Language models are weird for the same reason human cultures are weird (davidoks.blog)

In a recent analysis, it was revealed that OpenAI's GPT-5.1 and subsequent models developed a peculiar fixation on terms like "goblins," distracting users with their frequent and seemingly irrelevant references. Initially dismissed as a quirky behavior, the obsession became disruptive enough that OpenAI implemented prompts to curb these references in GPT-5.5. The underlying issue was traced back to the model's overfitting during training, which caused it to latch onto idiosyncratic phrases as it struggled to discern relevant feedback from its learning process. This phenomenon highlights a broader principle in adaptive systems such as language models, where atypical behaviors emerge from complex feedback mechanisms. To optimize their performance, these systems often overimitate or overlearn, leading to bizarre traits and fixations that are not functionally beneficial. The analogy drawn with human culture suggests that, much like humans develop cultural practices through imitation in response to opaque feedback, AI language models similarly adapt through their training regimes, resulting in both extraordinary capabilities and unusual quirks. Understanding these traits has crucial implications for improving AI training methodologies and fostering more predictable model behavior.