Thinking Sparks: Emergent Attention Heads in Reasoning Models (arxiv.org)

Researchers used circuit analysis to show that post-training (supervised fine-tuning, distillation, and RL-style policy optimization) causes the spontaneous emergence of functionally specialized attention heads that collectively implement structured reasoning in large reasoning models. Studying Qwen-family models and a DeepSeek-distilled variant, the team finds that SFT and distillation produce a cumulative build-up of stable “reasoning heads,” whereas group-relative policy optimization (a dynamic RL-style regime) repeatedly activates, evaluates, and prunes a small set of candidate heads whose survival tracks reward fluctuations. Controlled “think on/off” models do not form dedicated thinking heads; switching thinking off instead recruits a broader, less efficient set of compensatory heads. Ablations and qualitative analysis link these circuit-level dynamics to a performance trade-off: strengthened specialized heads enable sophisticated strategies on hard problems but also introduce over-thinking failure modes (calculation errors, logical loops) on simpler tasks. The work highlights that different post-training regimes sculpt attention circuits in distinct ways and suggests training-policy design should balance inducing powerful, stable reasoning primitives with guarding against brittle over-computation—pointing to future directions for targeted regularization, head-level monitoring, and reward shaping to align emergent circuits with reliable behavior.