TutorLLM: Customizing Learning Recommendations with Knowledge Tracing and Retrieval-Augmented Generation
Project Overview
The document explores the application of generative AI in education through the introduction of TutorLLM, a sophisticated personalized learning recommender system that leverages Large Language Models (LLMs) alongside Knowledge Tracing (KT) and Retrieval-Augmented Generation (RAG) techniques. TutorLLM is designed to enhance learning efficiency and personalization by dynamically retrieving context-specific knowledge and offering tailored recommendations based on individual student performance. Evaluation results from a study demonstrate that TutorLLM significantly improves user satisfaction and quiz scores compared to traditional LLMs, highlighting its effectiveness in fostering better educational outcomes. Overall, the findings suggest that integrating generative AI into educational frameworks can lead to more adaptive and responsive learning experiences, ultimately benefiting student engagement and achievement.
Key Applications
TutorLLM - a personalized learning recommender LLM system
Context: Educational context for undergraduate students in an online linear algebra course
Implementation: Developed as a Chrome browser plugin, enabling students to interact with TutorLLM during online learning sessions and receive personalized recommendations post-course.
Outcomes: 10% increase in user satisfaction (measured by System Usability Scale) and a 5% improvement in quiz scores compared to using general LLMs.
Challenges: Challenges include issues with personalization, handling of varying content relevance, and the risk of generating inaccurate information (hallucinations).
Implementation Barriers
Technical
LLMs often generate inaccurate information (hallucinations) and lack the ability to personalize responses effectively.
Proposed Solutions: Integrating Knowledge Tracing and Retrieval-Augmented Generation to provide personalized and context-specific responses.
Implementation
Difficulty in adapting existing educational technologies to work with LLMs.
Proposed Solutions: Further research and development on refining AI-driven educational tools to better fit individual learning needs.
Data Privacy
Concerns over data usage and privacy when collecting student interaction data.
Proposed Solutions: Ensuring compliance with data protection regulations and transparency in data handling practices.
Project Team
Zhaoxing Li
Researcher
Vahid Yazdanpanah
Researcher
Jindi Wang
Researcher
Wen Gu
Researcher
Lei Shi
Researcher
Alexandra I. Cristea
Researcher
Sarah Kiden
Researcher
Sebastian Stein
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Zhaoxing Li, Vahid Yazdanpanah, Jindi Wang, Wen Gu, Lei Shi, Alexandra I. Cristea, Sarah Kiden, Sebastian Stein
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai