Show HN: Legal RAG Bench (isaacus.com)

Legal RAG Bench has been launched as a comprehensive benchmark and evaluation methodology for assessing the performance of legal retrieval-augmented generation (RAG) systems. This new tool highlights that information retrieval is the critical factor in the effectiveness of legal RAG systems, often outpacing the importance of reasoning capabilities. The study revealed that the Kanon 2 Embedder significantly outperformed other leading models like Gemini 3.1 Pro and GPT-5.2, demonstrating an average accuracy improvement of 17 points. The evaluation also identified that many hallucination errors in legal RAG systems stem from retrieval failures rather than reasoning weaknesses. The significance of Legal RAG Bench lies in its potential to reshape the evaluation landscape for legal AI systems by providing a rigorous, transparent framework informed by legal expertise. It consists of 4,876 carefully curated legal passages linked to 100 intricate, expert-level questions focused on Victorian criminal law. This structure allows for a detailed analysis of how retrieval and generative models contribute to overall performance, using a methodological approach that enhances the reliability of benchmarks in this domain. By emphasizing the necessity for robust information retrieval, Legal RAG Bench aims to advance the development of more effective legal AI systems, thereby addressing existing shortcomings in legal evaluation datasets.