Even (very) noisy LLM evaluators are useful for improving AI agents (www.tensorzero.com)

Recent research highlights that even noisy large language model (LLM) evaluators can be instrumental in enhancing AI agents, despite their limitations in accurately assessing individual outputs. While these evaluators often yield inconsistent results that poorly correlate with desired real-world outcomes—due to issues like biases and sensitivity to surface-level content—they can still effectively rank agent performance on average across multiple evaluations. This finding is significant for the AI/ML community as it suggests that practitioners can leverage even suboptimal evaluators for offline model selection, enabling them to iteratively improve agent quality over time. The study outlines crucial distinctions between output-level and agent-level correlations, emphasizing that while output-level evaluations may falter in production settings, agent-level evaluations benefit from averaging over numerous outputs. This means that a larger sample size can help mitigate the noise inherent in evaluators, allowing them to identify higher-quality agents more consistently. The research underscores the need for sufficient data to distinguish among agents reliably, but reassures developers that they can still make informed decisions about which AI variants to deploy, even in the presence of evaluator noise.