How we evaluate our LLM judge (build.forus.com)

Forus has introduced a novel evaluation framework for their LLM judge, specifically designed to enhance the accuracy of prior authorizations (PAs) in clinical settings. The challenge arises from LLMs frequently over-flagging correct clinical claims, which can hinder timely medication delivery. To address this, Forus developed a rigorous testing approach that generates synthetic perturbations of correct answers to evaluate the LLM’s detection capabilities without incurring the costs of human auditing or relying on existing benchmarks that fall short in complexity. By breaking down PA answers into atomic claims and using diverse query expansions, their evaluation model assesses the judge's performance on both false positive and error detection rates. This dual-focused evaluation not only helps calibrate the judge to minimize unnecessary flags but also enhances its reliability to identify genuine discrepancies in clinical records. The synthetic perturbation model, anchored in real patient data, allows Forus to test the LLM's robustness against plausible yet incorrect claims. As the company continues refining this methodology, they aim to develop a more comprehensive evaluation strategy that integrates insights from synthetic errors, production monitoring, and clinical reviews, ultimately fostering trust in the LLM's decision-making in high-stakes healthcare situations.