New AI tutor achieves 0.71-1.30 SD effect size in Dartmouth course [pdf] (intextbooks.science.uu.nl)

Dartmouth College has announced the deployment of Phosphor, a digital learning platform that integrates large language model (LLM)-graded formative assessments directly into instructional content for the Introductory Statistics course. Initial results from a pilot with 151 students indicate that full use of Phosphor correlates with significant improvements in final exam performance, showing effect sizes between 0.71 and 1.30 standard deviations. Remarkably, 90.2% of students engaged with the platform, which is substantially higher than typical textbook reading compliance rates, underscoring the potential of combining AI with traditional educational methods. The significance of this development lies in Phosphor's innovative design, which embeds AI-assisted quizzes throughout the learning materials, effectively turning passive reading into an interactive experience. This "doer effect," where practice questions enhance learning outcomes, reveals that high engagement can coexist with measurable efficacy, addressing the challenges posed by declining textbook reading rates among students. Additionally, the platform's use of constructed-response questions, facilitated by LLM grading, provides valuable feedback, enhancing the learning experience. These findings suggest promising implications for the design of AI-augmented instructional tools, potentially revolutionizing educational methodologies in higher education.