Competing Biases Underlie Overconfidence and Underconfidence in LLMs (www.nature.com)

Recent research has uncovered that the confidence dynamics of large language models (LLMs) are driven by competing biases, revealing crucial insights for their deployment in high-stakes applications. The study identified a choice-supportive bias, where LLMs display inflated confidence in their initial responses and are less likely to amend them when those responses are visible. This contrasts with their hypersensitivity to contradictory feedback, leading to significant adjustments in confidence when opposing advice is presented. These findings underscore the paradox of LLM behavior where they cling to initial answers while simultaneously overreacting to conflicting information, posing challenges for reliable confidence estimation in AI. The implications for the AI/ML community are substantial, as understanding these confidence patterns is essential for enhancing the robustness and transparency of LLMs. By using a controlled experimental paradigm, the research demonstrated that LLMs' responses can span varying degrees of confidence based on initial choices and the nature of subsequent feedback. Particularly noteworthy is the discovery that LLMs exhibit a nonlinear relationship between confidence levels and the likelihood of changing their answers, highlighting a dependency on the accuracy of advice received. These insights could inform better training and calibration methods for LLMs, paving the way for more effective human-AI interactions and decision-making processes.