Generalization Dynamics of LM Pre-Training (jiaxin-wen.github.io)

Researchers have unveiled new insights into the pre-training dynamics of language models (LMs), challenging the conventional view that these models steadily transition from basic pattern-matching to more sophisticated generalizable intelligence. Their study introduces the concept of "mode-hopping," where LMs fluctuate between parrot-like behavior—repeating memorized patterns—and intelligence-like behavior that demonstrates contextual understanding. This phenomenon occurs throughout pre-training, implying that models can suddenly revert to shallow processing even after extensive training. The findings suggest that traditional optimization dynamics do not explain these shifts. Instead, mode-hopping is conceptualized as a capacity allocation issue, where different circuits compete depending on the data presented during pre-training. This discovery is significant for the AI/ML community, as it offers a new evaluative framework for understanding and improving model generalization. By employing a toy evaluation suite to track these dynamics, the researchers demonstrated methods to select intermediate pre-training checkpoints that yield better generalization and alignment outcomes than final or mid-training checkpoints. Furthermore, their findings challenge existing assumptions about generalization predictors, indicating that both simple and complex solutions can be effective, thus prompting a reevaluation of how researchers approach model training and evaluation. The study highlights the intricate nature of learning in LMs and the potential for developing more robust and intelligent AI systems.