Law professors prefer AI over peer answers (marginalrevolution.com)

A recent blinded evaluation involving sixteen U.S. law professors revealed that large language models (LLMs) are outperforming human peers as educational tutors in law, particularly in contracts courses. Professors assessed nearly 3,000 comparisons between LLM responses and their own, with an impressive 75.33% of evaluations favoring LLMs. This finding is significant for the AI/ML community as it challenges traditional perceptions of educational assessments in judgment-rich fields, suggesting that LLMs may effectively support or even replace human tutors in these contexts. Additionally, LLM responses were deemed less harmful than those from professors, indicating a potential for safer educational tools. The implications extend beyond just law education, positioning LLMs as scalable solutions for evaluating and enhancing learning in various disciplines reliant on judgment. By employing one LLM as a judge for another in evaluations, this method could be generalized across different models, accelerating the integration of AI in educational settings where nuanced reasoning is critical. This study underscores the transformative potential of AI in academia, highlighting a shift towards AI-assisted learning tools that maintain high educational standards while also prompting discussions about the evolving role of human educators in the age of AI.