Stop Thinking of LLMs as Next-Token Predictors (gmcgoldr.github.io)

A new perspective on large language models (LLMs) challenges the conventional view of them as mere "next-token predictors." While this characterization isn't entirely incorrect, it oversimplifies their capabilities. LLMs, particularly those engaged in post-training through reinforcement learning with verifiable rewards (RLVR), gain the ability to generate new sequences and learn from their outcomes, moving beyond merely predicting the next token in existing training data. This distinction is critical: during pre-training, models learn from historical sequences, but through RLVR, they explore new contexts and ideas, adapting based on rewards rather than just mimicking past patterns. This insight is significant for the AI/ML community as it redefines the understanding of LLM functionality and potential. For example, likening an LLM to an advanced chess engine illustrates this shift—while one system predicts moves based solely on historical grandmaster games, the other uses comprehensive explorations of game possibilities to choose optimal moves. Understanding LLMs as exploratory systems that simulate strategic decision-making opens new avenues for their application, encouraging a focus on their ability to innovate rather than simply replicate, thus enhancing their role in AI-driven tasks and problem-solving.