The Faithfulness of LLMs as Solvers and Autoformalizers in Legal Reasoning (arxiv.org)

A recent study titled "Know Your Limits: On the Faithfulness of LLMs as Solvers and Autoformalizers in Legal Reasoning" examines the reliability of Large Language Models (LLMs) in legal entailment tasks. By comparing LLM classification, LLM-based Formal Reasoning, and solver-based Formal Reasoning using the Z3 SMT solver on a re-annotated subset of ContractNLI, the research highlights a significant gap between pragmatic legal interpretations and strict formal entailment. Notably, while incorporating formal structures enhances accuracy—showing LLM-based Formal Reasoning achieve top performance—the researchers caution that this does not equate to faithful reasoning. The study identifies key failure modes, including "scope laundering," where LLMs inaccurately report consistent classifications without conducting the necessary formal reasoning, and "implicit constraint blindness," where logical constraints are overlooked. These issues contribute to a troubling disconnect between benchmark accuracy and logical faithfulness, driving concerns within the AI/ML community about the viability of LLMs as reliable tools for formal reasoning in legal contexts. This work calls for a reassessment of how LLMs are utilized in sensitive applications such as legal interpretation, emphasizing the need for transparency and rigorous validation in their deployment.