StuGPTViz: A Visual Analytics Approach to Understand Student-ChatGPT Interactions
Project Overview
The document explores the transformative role of generative AI, particularly Large Language Models (LLMs) like ChatGPT, in education, emphasizing their capacity to enhance learning experiences through innovative teaching methodologies. It underscores the significance of comprehending how students engage with LLMs to inform pedagogical strategies and optimize educational outcomes. To facilitate this understanding, the authors created a visual analytics tool, StuGPTViz, which allows for the analysis of student interactions with ChatGPT. This system categorizes cognitive engagement levels and assesses the quality of AI-generated responses, ultimately offering valuable insights for educators. By leveraging these findings, the document advocates for the integration of generative AI in educational settings, aiming to improve instructional practices and student learning through data-driven approaches.
Key Applications
StuGPTViz - a visual analytics system for analyzing student-ChatGPT interactions
Context: Graduate-level data visualization course for computer science majors
Implementation: Incorporation of ChatGPT into course curriculum with in-class exercises allowing students to interact with ChatGPT
Outcomes: Enhanced understanding of students' cognitive levels, improved instructional strategies, and personalized feedback mechanisms
Challenges: Lack of datasets for structured student-ChatGPT conversations and difficulties in analyzing evolving interaction patterns
Implementation Barriers
Data Availability
Absence of publicly available datasets focused on student-ChatGPT conversations, making it difficult to analyze interaction patterns.
Proposed Solutions: Collect structured data from well-defined learning tasks involving student-ChatGPT interactions.
Cognitive Assessment
Challenges in accurately interpreting students' cognitive levels based on their inquiries to LLMs.
Proposed Solutions: Develop a coding scheme grounded in cognitive levels and thematic analysis to categorize student interactions.
Evaluation Complexity
Difficulty in evaluating the proficiency of students in utilizing LLMs and the varying quality of AI responses.
Proposed Solutions: Implement metrics to assess ChatGPT's response quality and students' adjustments to prompts.
Project Team
Zixin Chen
Researcher
Jiachen Wang
Researcher
Meng Xia
Researcher
Kento Shigyo
Researcher
Dingdong Liu
Researcher
Rong Zhang
Researcher
Huamin Qu
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Zixin Chen, Jiachen Wang, Meng Xia, Kento Shigyo, Dingdong Liu, Rong Zhang, Huamin Qu
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai