The StudyChat Dataset: Student Dialogues With ChatGPT in an Artificial Intelligence Course
Project Overview
The document explores the integration of generative AI, particularly large language models (LLMs) like ChatGPT, in university-level education, focusing on an artificial intelligence course. It presents the StudyChat dataset, which records student interactions with LLMs during programming assignments, revealing both the potential advantages and challenges of using these technologies. Key applications of LLMs include personalized learning and enhanced student engagement, with findings indicating that students who engage with LLMs by asking conceptual questions tend to achieve better course performance. However, the study also raises concerns about the risks of misuse and over-reliance on LLMs, which may negatively impact learning outcomes. Overall, the document emphasizes the dual-edged nature of generative AI in education, highlighting the importance of fostering effective engagement strategies while mitigating the risks associated with dependency on AI tools.
Key Applications
StudyChat dataset and LLM-powered tutoring chatbot
Context: Upper-division university-level AI course, targeting Computer Science students
Implementation: Developed a web application mimicking ChatGPT functionalities for students to use during programming assignments without restrictions.
Outcomes: Insights into student interaction patterns with LLMs, correlations between usage patterns and course performance.
Challenges: Concerns about student misuse of LLMs, including over-reliance on AI for assignments.
Implementation Barriers
Ethical concerns
Potential misuse of LLMs by unmotivated students to complete assignments instead of learning.
Proposed Solutions: Understanding student behavior with LLMs and developing Intelligent Tutoring Systems (ITS) that adapt to encourage positive usage.
Reliance on technology
Students may become over-reliant on LLMs, which could undermine their learning and understanding of core concepts.
Proposed Solutions: Monitoring student interactions with LLMs to detect and address over-reliance in real-time.
Project Team
Hunter McNichols
Researcher
Andrew Lan
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Hunter McNichols, Andrew Lan
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai