Position: Stop Anthropomorphizing Intermediate Tokens as Reasoning/Thinking Trac (arxiv.org)

A recent position paper has raised critical concerns about the tendency to anthropomorphize intermediate token generation (ITG) in language models, often referred to as "reasoning traces" or "thinking traces." This practice implies that these intermediate outputs reflect human-like reasoning processes, which misrepresents the actual functionality of AI models. The authors argue that such anthropomorphization can lead to misunderstandings about how these models operate and can negatively impact their application in AI research and practical use. The significance of this paper lies in its call for a shift in how the AI community perceives and interacts with language models. Rather than viewing intermediate tokens as indicative of human reasoning, the authors advocate for a clearer understanding of these outputs as mere artifacts of computation. This perspective is crucial for developing better methodologies for interpreting model behavior and ensuring more responsible AI deployment. By reframing the discussion around ITG, the paper aims to steer future research away from misleading interpretations that could hinder advancements in AI/ML technology.