Training LLM-based Tutors to Improve Student Learning Outcomes in Dialogues
Project Overview
The document examines the use of generative AI, particularly large language models (LLMs), in education to improve personalized tutoring through interactive dialogues. It introduces an innovative method for training LLM-based tutors aimed at enhancing student learning outcomes by generating tutor utterances that encourage correct responses from students, all while following sound pedagogical principles. The findings reveal that this approach leads to better student performance, effectively addressing challenges such as the potential for over-reliance on AI and concerns related to data privacy. Overall, the study underscores the transformative potential of generative AI in educational settings, highlighting its capacity to create more engaging and effective learning experiences.
Key Applications
LLM-based tutoring system using Llama 3.1 8B
Context: Mathematics education, targeting students who require personalized tutoring through interactive dialogues.
Implementation: Training the LLM to generate tutor utterances that maximize student correctness using direct preference optimization.
Outcomes: Significant increase in the probability of correct student responses, while maintaining pedagogical quality comparable to proprietary models.
Challenges: The primary limitation is the lack of experimentation with actual students, relying on simulated models instead.
Implementation Barriers
Technical
Dependence on large proprietary LLMs can limit flexibility, customization, and data privacy.
Proposed Solutions: Development of smaller, open-source LLMs for educational applications.
Practical
Challenges in obtaining access to real students for testing and validation of the AI tutoring system, as well as encouraging collaboration with educational institutions to facilitate real-world testing.
Proposed Solutions: Fostering partnerships with educational institutions for effective real-world testing and validation.
Project Team
Alexander Scarlatos
Researcher
Naiming Liu
Researcher
Jaewook Lee
Researcher
Richard Baraniuk
Researcher
Andrew Lan
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Alexander Scarlatos, Naiming Liu, Jaewook Lee, Richard Baraniuk, Andrew Lan
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai