Educators' Perceptions of Large Language Models as Tutors: Comparing Human and AI Tutors in a Blind Text-only Setting
Project Overview
The document explores the role of generative AI, specifically large language models (LLMs), as innovative tutors in educational environments, assessing their performance against traditional human tutors. It highlights the creation of intelligent tutoring systems that utilize LLM technology and presents findings from a comparative study measuring key attributes such as engagement, empathy, scaffolding, and conciseness. Results indicate that LLMs are often perceived as superior to human tutors in these metrics, suggesting that AI can significantly reduce the workload for educators while providing effective educational support. However, the document also calls for additional research to refine and improve the efficacy of AI tutoring systems, underscoring the importance of ongoing development in the use of AI in education.
Key Applications
Intelligent Tutoring Systems using LLMs
Context: Educational settings for grade-school math word problems, targeting students and educators.
Implementation: Human tutors annotated and compared the performance of LLM tutors to human tutors in a study, focusing on dialogue generated in a text-based chat interface.
Outcomes: LLM tutors were perceived as more engaging, empathetic, concise, and better at scaffolding compared to human tutors according to experienced annotators.
Challenges: The challenge includes the need for LLMs to improve in alignment with human perceptions of quality tutoring and the cognitive limitations of human tutors leading to fatigue and burnout.
Implementation Barriers
Pedagogical Limitations
Human tutors may experience compassion fatigue and burnout, which can affect their ability to engage empathetically with students.
Proposed Solutions: Delegating repetitive tutoring tasks to AI can help reduce teacher workload and allow them to focus on more complex responsibilities.
Technical Limitations
LLMs have limited capabilities in non-verbal communication, which can affect their effectiveness in tutoring roles that require empathy and engagement.
Proposed Solutions: Further research and development are needed to enhance multimodal communication capabilities in AI to better replicate human tutoring.
Project Team
Sankalan Pal Chowdhury
Researcher
Terry Jingchen Zhang
Researcher
Donya Rooein
Researcher
Dirk Hovy
Researcher
Tanja Käser
Researcher
Mrinmaya Sachan
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Sankalan Pal Chowdhury, Terry Jingchen Zhang, Donya Rooein, Dirk Hovy, Tanja Käser, Mrinmaya Sachan
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai