Are You Sure Your LLM Is Sure? (arxiv.org)

A new framework called RADAR (Repair, Adopt, and Diagnose After Recalibration) has been introduced to enhance the reliability of confidence scores in large language models (LLMs). These scores play a crucial role in determining if an LLM's answer should be accepted, reviewed, or regenerated. Current methods for recalibrating these scores often fail to address underlying errors. RADAR decouples the processes of confidence repair, decision-making about the repair’s implementation, and the auditing of remaining errors, providing a comprehensive approach to improving LLM reliability. Key to RADAR is the Probe-Anchored Isotonic Calibration (PAIC) method, which creates a monotone mapping based on correctness-probe predictions from unlabeled prompts, offering a more nuanced view of confidence levels. The framework also includes strategies for assessing the potential improvements from these repairs and for identifying systematic errors that may persist post-recalibration. Initial results indicate that richer predictors can enhance confidence estimations, but that translating this information back into a scalar score can lead to losses in predictive accuracy. This research opens the door for better-informed decisions in LLM deployment and monitoring, promising significant advancements in the AI/ML community's ability to evaluate model performance.