Verbalized Sampling: How to Mitigate Mode Collapse and Unlock LLM Diversity (arxiv.org)

Researchers identify a fundamental, data-level cause of post-training "mode collapse" in aligned LLMs: typicality bias in human preference data, where annotators systematically favor familiar or stereotypical outputs. They formalize this bias theoretically, verify it across preference datasets, and show it drives reduced generative diversity after alignment—shifting the debate from algorithmic limits to a data-centric explanation with practical consequences for model behavior. To counteract this, the authors introduce Verbalized Sampling (VS), a training-free prompting technique that asks a model to generate multiple candidate outputs along with explicit probabilities (e.g., "Generate 5 jokes about coffee and their corresponding probabilities"). VS steers inference toward underrepresented modes without retraining, improving diversity 1.6–2.1x in creative-writing tasks and boosting performance across poems, stories, jokes, dialogue simulation, open-ended QA, and synthetic-data generation. Crucially, gains come without measurable harm to factual accuracy or safety, and larger, more capable models tend to benefit most. The work offers both a theoretical lens on why alignment can homogenize outputs and a simple, deployable mitigation that unlocks pre-trained generative diversity at inference time.