GPT Guesses Between 1 and 100 (github.com)

A recent experiment tested GPT-4.1's ability to generate random numbers between 1 and 100, revealing it mirrors common human biases. Researchers ran the model 10,000 times to analyze whether its number choices exhibited a flat distribution typical of uniformly random outputs. In stark contrast, GPT-4.1 showed a pronounced tendency to favor certain numbers—particularly 37, 42, and 73—while avoiding round numbers altogether, similar to patterns observed in human selections. This finding suggests that the model inherits specific biases from human text, indicating that it is not a purely random number generator. This experiment is significant for the AI and machine learning community as it highlights the implications of human-like biases in responses from language models. While GPT-4.1 is informed by human tendencies, it notably under-pick numbers like 69, often favored by humans, suggesting that its training may include guardrails to mitigate less desirable biases. The results challenge assumptions about LLM behavior, emphasizing the need for further investigation into how these models reflect or deviate from human thought processes, especially in tasks requiring randomness.