Elias in the Lighthouse, Again? Diagnosing Low Diversity in LLM Stories (arxiv.org)

Recent research highlights a concerning lack of diversity in stories generated by large language models (LLMs), examining 20,000 narratives from four prominent models. Analysts discovered that merely 11 words dominated 88.3% of the outputs, including specific names like Elias and settings such as lighthouses, suggesting a narrow range of thematic exploration. These findings indicate that while LLMs are trained on vast datasets, their outputs often converge around limited vocabulary and concepts, likely stemming from the preference data used during training. This research is significant for the AI/ML community as it raises awareness of the potential biases embedded in LLM outputs, which could hinder creative and innovative applications of these technologies. The "lighthouse" stories—though generated infrequently compared to the broader outputs—reflect the disproportionate influence of smaller datasets combined with alignment algorithms. Understanding these dynamics can drive improvements in model training, encouraging greater variability and richness in AI-generated content, ultimately enhancing user experience and utility in generative applications.