How Do Students Interact with an LLM-powered Virtual Teaching Assistant in Different Educational Settings?
Project Overview
The document examines the incorporation of generative AI in education, focusing on the implementation of Jill Watson, a virtual teaching assistant powered by large language models (LLMs). It emphasizes Jill Watson's role in enhancing student engagement and promoting higher-order cognitive questioning, revealing diverse usage patterns across various courses. The findings indicate that such AI tools can substantially enrich the learning experience by customizing interactions to meet students' cognitive requirements. However, it also acknowledges challenges, including inconsistencies in usage frequency and the need for context-specific questioning, which could affect the overall effectiveness of AI in educational settings. By addressing these obstacles, the integration of generative AI like Jill Watson has the potential to transform educational practices, making learning more interactive and responsive to individual student needs.
Key Applications
Jill Watson, a virtual teaching assistant powered by LLMs
Context: Deployed in online and offline classrooms, particularly in Georgia Tech's Knowledge-Based AI and Cognitive Science courses, as well as an English course at Wiregrass Georgia Technical College.
Implementation: Jill Watson was integrated into Learning Management Systems (LMS) using Learning Tools Interoperability (LTI), providing real-time assistance to students.
Outcomes: Increased student engagement and a significant rise in the complexity of questions asked, indicating deeper cognitive engagement with course materials.
Challenges: Variability in usage frequency across different courses and the need for further refinements in design to better suit diverse educational contexts.
Implementation Barriers
Usage variability
The frequency of usage varies significantly across different educational deployments, impacting the overall effectiveness of the AI tool.
Proposed Solutions: Further research is needed to explore the underlying factors influencing usage patterns and to adapt the AI tool to better meet diverse student needs.
Cognitive engagement
The types of questions asked depend significantly on course-specific contexts, which may limit the effectiveness of the AI tool in certain subjects.
Proposed Solutions: Tailoring the AI's capabilities to align with specific course structures and learning objectives could enhance its utility.
Project Team
Pratyusha Maiti
Researcher
Ashok K. Goel
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Pratyusha Maiti, Ashok K. Goel
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai