Are there words LLMs won't say? (samermakes.com)

Researchers at GING have unveiled a significant study exploring why Large Language Models (LLMs) might not produce specific words despite having them in their extensive vocabularies. By analyzing the logits assigned to tokens during the inference process, they found that some common English words might be filtered out due to the parameters set in sampling filters like top-k, top-p, and min-p. These filters and the temperature settings that adjust output creativity can drastically alter a model's ability to generate certain words, with findings indicating that between 20% to 45% of sampled words may go unpredicted at default settings. This research is vital for the AI/ML community as it highlights the effects of sampling parameters on lexical diversity in LLM outputs. The introduction of a Word Coverage Score (WCS) allows for a quantitative measure of the impact of these settings, suggesting that as LLMs gain prominence in content creation, they could inadvertently simplify language use by underutilizing a rich vocabulary. The findings also underscore the ongoing influence of LLMs on trends in language within academic publishing, stressing the importance of maintaining linguistic nuance in AI-generated text.