How AI can improve tutor effectiveness

K-12 DIVE

A Stanford University study of its AI tutor assistance tool revealed improved student performance and increased tutor capacity to support learning.

Dive Brief:

  • Using a generative artificial intelligence tool during in-person tutoring sessions increased the tutors’ capacity to guide students through complex math problems and led to improved student math performance, according to a study released Monday from Stanford University researchers.
  • Called Tutor CoPilot, the open-source tool developed at Stanford can be embedded in any tutoring platform and helps live tutors ask guiding questions to students and respond to student needs. However, tutors working with the tool suggested improvements to make the guidance for tutors more grade-appropriate.
  • Researchers said this first-ever randomized controlled trial of a human-AI system in live tutoring situations shows promise in increasing the effectiveness of in-person tutors. 

Dive Insight:

Students whose tutors used Tutor CoPilot were 4 percentage points more likely to progress through math tutoring session assessments successfully compared to students whose tutors did not have AI assistance, the study found.

The approach particularly benefited lower-rated and less-experienced tutors, researchers said. Students of lower-rated tutors who used the AI assistance increased their math proficiency up to 9 percentage points on average compared to students learning from lower-rated tutors without AI assistance.

The study included 900 tutors and 1,800 elementary and secondary school students from a large school district in the South. Stanford partnered with tutoring company FEV Tutor to pilot the tool’s implementation.

Here’s how it works: A tutor presents a subtraction problem to a student. If the student answers incorrectly, the tutor can activate Tutor CoPilot, which will recommend that the tutor ask the student to identify the numbers in the problem or suggest the student draw the items that need to be subtracted. 

“Novices often struggle to remediate student mistakes in real-time, missing critical learning opportunities,” the study said.

For privacy protections, the tool automatically de-identifies student and tutor names and limits the amount of user information sent to external language model services, researchers said. Chat GPT is an example of one such large language model.

A tool like this one provides the potential for school districts to expand tutoring because it can be less expensive than traditional professional development. The study estimates the cost of Tutor CoPilot at $20 per tutor annually, based on the tutors’ usage during the study.

Click here to read full article

Mentioned Publication

Tutor CoPilot: A Human-AI Approach for Scaling Real-Time Expertise

 

Generative AI, particularly Language Models (LMs), has the potential to transform real-world domains with societal impact, particularly where access to experts is limited. For example, in education, training novice educators with expert guidance is important for effectiveness but expensive, creating significant barriers to improving education quality at scale. This challenge disproportionately hurts students from under-served communities, who stand to gain the most from high-quality education and are most likely to be taught by inexperienced educators. We introduce Tutor CoPilot, a novel Human-AI approach that leverages a model of expert thinking to provide expert-like guidance to tutors as they tutor. This study presents the first randomized controlled trial of a Human-AI system in live tutoring, involving 900 tutors and 1,800 K-12 students from historically under-served communities. Following a preregistered analysis plan, we find that students working on mathematics with tutors randomly assigned to have access to Tutor CoPilot are 4 percentage points (p.p.) more likely to master topics (p<0.01). Notably, students of lower-rated tutors experienced the greatest benefit, improving mastery by 9 p.p. relative to the control group. We find that Tutor CoPilot costs only $20 per-tutor annually, based on the tutors’ usage during the study. We analyze 550,000+ messages using classifiers to identify pedagogical strategies, and find that tutors with access to Tutor CoPilot are more likely to use strategies that foster student understanding (e.g., asking guiding questions) and less likely to give away the answer to the student, aligning with high-quality teaching practices. Tutor interviews qualitatively highlight how Tutor CoPilot’s guidance helps them to respond to student needs, though tutors flag common issues in Tutor CoPilot, such as generating suggestions that are not grade-level appropriate. Altogether, our study of Tutor CoPilot demonstrates how Human-AI systems can scale expertise in real-world domains, bridge gaps in skills and create a future where high-quality education is accessible to all students.