Next-Token Predictor Is an AI's Job, Not Its Species (www.astralcodexten.com)

In a recent discussion, Kelsey Piper argues against reducing AI to mere "next-token predictors" or "stochastic parrots", emphasizing that AI systems, like human brains, operate on multiple levels and use complex learning techniques such as fine-tuning and reinforcement learning from human feedback (RLHF). While this highlights the intricacies of AI functionality, some commenters argue that these methods ultimately rely on next-token prediction as their foundational mechanism. Piper suggests that overemphasizing this perspective overlooks the true nature of AI's learning processes, which resemble the human brain's predictive coding—a system continuously updating its understanding based on sensory input. This discourse is significant for the AI/ML community as it raises essential questions about the nature of intelligence and how we perceive AI's capabilities. Drawing parallels between human evolution and AI development, Piper points out that both operate under optimization principles. For AI, this includes using next-token prediction as a key algorithm, which shapes underlying neural architectures. However, the representation and manipulation processes within AI systems can be highly abstract and complex, involving structures like helical manifolds in high-dimensional spaces. Understanding these layers of optimization—where AIs' thinking diverges from mere token prediction—can deepen our comprehension of what it means for machines to "think" and may influence future research directions in artificial intelligence.