The Limits of Test-Time Compute for Self-Evaluation (anirudhb11.github.io)

Researchers have unveiled the limitations of using test-time compute for self-evaluation in language models, particularly through a new approach called Scorpio. While generating multiple solutions can enhance the likelihood of identifying a correct answer, the challenge lies in effectively selecting the right one from these candidates. Notably, the study reveals that although higher test-time compute reduces variance in model judgments, it does not sufficiently mitigate bias, causing accuracy to plateau significantly below optimal levels (pass@N). This finding prompts a reevaluation of prevalent self-verification methods, which often struggle to choose the best solution from generated candidates. Scorpio improves selection accuracy by implementing a bias-variance decomposition to refine the scoring process. By jointly evaluating candidates and optimizing how they are grouped and assessed, Scorpio reduces ranking errors more effectively than existing methods. This new technique not only enhances performance across various models and benchmarks but also underscores a critical insight: despite advancements, inherent bias in rankings continues to hinder the model's self-evaluation capabilities. The researchers advocate for ongoing efforts to address this bias as a crucial step towards leveraging the full potential of multiple candidate samplings in AI/ML applications.