Bridging Learnersourcing and AI: Exploring the Dynamics of Student-AI Collaborative Feedback Generation
Project Overview
The document explores the integration of generative AI, specifically GPT-4, in improving feedback mechanisms within data science education, focusing on a collaborative hint-writing exercise conducted with 72 Master's students in a data visualization course. Through a randomized controlled experiment, the study compared the quality of hints written independently by students against those enhanced by AI-generated revisions. Key applications of generative AI in this context included providing tailored feedback to students and fostering collaborative learning experiences. Findings indicated that AI support positively influenced hint quality and student performance, while also highlighting learner preferences for AI-assisted learning. However, the study also addressed challenges associated with large language models (LLMs), such as the potential for over-reliance on AI and issues of transparency in AI-generated content. Overall, the research underscores the promising role of generative AI in educational settings, while advocating for careful consideration of its limitations and ethical implications.
Key Applications
Student-AI collaborative hint writing using GPT-4
Context: Master's level data visualization course for adult learners in an online applied data science program
Implementation: Conducted a randomized controlled experiment where students either wrote hints independently (Hint-writing) or revised hints generated by GPT-4 (AI-hint-revision)
Outcomes: Higher quality hints with increased accuracy and specificity were noted in the AI-hint-revision condition. Students found the exercise helpful for critical thinking about LLM responses.
Challenges: Issues with over-reliance on AI-generated feedback, lack of trust in LLMs, and the complexity added by AI hints were reported.
Implementation Barriers
Trust Barrier
Students expressed concerns about the reliability and accuracy of the hints generated by GPT-4.
Proposed Solutions: Instructors should explain how LLMs are used, provide transparency about the prompting strategies, and address any reliability concerns.
Complexity Barrier
Some students felt that the GPT-4 hints added unnecessary complexity to the hint-writing task.
Proposed Solutions: Consider giving students the AI-generated hints after they attempt to write their own hints to maintain engagement and reduce complexity.
Project Team
Anjali Singh
Researcher
Christopher Brooks
Researcher
Xu Wang
Researcher
Warren Li
Researcher
Juho Kim
Researcher
Deepti Pandey
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Anjali Singh, Christopher Brooks, Xu Wang, Warren Li, Juho Kim, Deepti Pandey
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai