Hallucination‐Free? Assessing the Reliability of Leading AI Legal Research [pdf] (dho.stanford.edu)

A recent study published in the Journal of Empirical Legal Studies critically assesses the reliability of prominent AI legal research tools, particularly those developed by LexisNexis and Thomson Reuters. The research highlights the significant issue of "hallucinations," where AI systems produce incorrect or fabricated responses, which poses a substantial risk in legal contexts. Using a newly created dataset and an empirical evaluation framework, the authors found that despite claims of reduced hallucinations through techniques like retrieval-augmented generation (RAG), these tools still produced hallucinations between 17% and 33% of the time, indicating that the risk of misinformation in high-stakes legal settings remains considerable. This study is significant for the AI/ML community as it underscores the critical limitations of legal AI technologies, pointing to the need for careful scrutiny of their outputs. By establishing a typology for hallucinations and documenting system performance, it not only calls for improved alignment between AI services and their advertised capabilities but also emphasizes the ongoing responsibility of legal professionals to verify AI-generated content. The findings serve as a crucial reminder of the ethical implications of AI in law, paving the way for future regulatory guidelines as the legal industry increasingly adopts these advanced technologies.