Randomness in large language models: What researchers need to know (and report) (arxiv.org)

Recent research emphasizes the inherent randomness in large language models (LLMs) and its implications for reproducibility in research. As LLMs increasingly generate data for tasks like classification and information extraction, their outputs can vary even under identical prompts due to factors such as sampling methods, numerical rounding, and underlying model updates. Researchers have found that while setting the temperature parameter to zero can minimize deliberate randomness, it does not address other sources of variability. This challenge complicates efforts to replicate results, especially when using proprietary APIs versus locally executed open-weight models. The significance of this finding for the AI/ML community lies in the call for improved transparency and standardization when reporting LLM-generated data. The researchers propose a new reporting standard for scholarly articles and replication packages that treats LLM outputs as probabilistic distributions rather than fixed values. This approach aims to enhance the reliability of research findings and better guide future work, ensuring that the noise inherent in LLM outputs is understood and properly documented. Implementing these standards could streamline replication efforts and foster greater integrity in AI-related research.