AI Outperforms Law Professors in Stanford Law Study (law.stanford.edu)

A recent study from Stanford Law School, led by Professor Julian Nyarko, found that law professors significantly prefer AI-generated responses to student questions over those written by human peers. In blind assessments involving nearly 3,000 comparisons among 16 law professors from various U.S. law schools, AI answers triumphed in 75% of cases, demonstrating the potential of large language models (LLMs) to serve as effective tutors in complex subjects like contract law, which require nuanced reasoning and judgment. This study is crucial for the AI and legal education communities as it challenges prevailing assumptions about AI's role in pedagogy, particularly in nuanced fields where clear answers are often absent. The research indicates that AI can provide high-quality, on-demand support that enhances legal education while preserving critical thinking skills. Professors deemed AI responses pedagogically harmful only 3.5% of the time, significantly lower than the 12% for peer answers. As law schools explore how to responsibly integrate AI tools, Nyarko emphasizes the need for a shift in discourse from skepticism about AI's capabilities to practical considerations for its deployment in enhancing student learning.