How Do LLMs Compute Verbal Confidence (DeepMind) (arxiv.org)

DeepMind's recent research explores how large language models (LLMs) compute verbal confidence, shedding light on the mechanisms behind uncertainty estimation in AI responses. The study investigates whether confidence is calculated in real-time when prompted or generated automatically during answer production. By analyzing models such as Gemma 3 (27B parameters) and Qwen 2.5 (7B parameters), the researchers found that confidence scores are gathered from answer tokens and cached for subsequent retrieval, rather than being constructed post-hoc. This suggests a more nuanced internal evaluation process for LLMs, where confidence is not merely a reflection of fluency but incorporates a deeper appraisal of answer quality. This discovery is significant for the AI/ML community as it promotes a better understanding of metacognition in LLMs, revealing that these models engage in sophisticated self-evaluation. The research indicates that the representations of confidence are influenced by factors beyond simple token probabilities, pointing to the potential for enhancing calibration and reliability in AI outputs. Ultimately, these insights provide a foundation for improving the design of LLMs, offering pathways for more transparent and dependable AI systems in applications that require nuanced decision-making and risk assessment.