How Students Use AI Feedback Matters: Experimental Evidence on Physics Achievement and Autonomy
Project Overview
The document explores the role of generative AI (GAI) in education, specifically its influence on academic performance and learner autonomy in high school physics. Through two randomized controlled trials, it investigates how GAI-driven personalized recommendations and on-demand assistance affect student outcomes. The findings reveal that GAI has the potential to improve academic achievement, particularly benefiting low-achieving students, while concurrently posing risks to learner autonomy among high-achieving students. This dual impact highlights the necessity of understanding usage patterns in GAI interactions, indicating that different approaches may be required based on students' varying achievement levels. Overall, the research underscores the importance of balancing GAI's advantages in enhancing educational outcomes with the preservation of student independence in learning.
Key Applications
GAI-powered personalized assistance and recommendation system
Context: High school physics students in a classroom setting, where students submit homework errors or request specific types of assistance regarding problem-solving.
Implementation: Students receive tailored recommendations or on-demand AI-generated feedback based on their submissions or requests over a defined period, allowing for personalized learning experiences.
Outcomes: Low-achieving students showed significant improvement in academic performance (d = 0.673), while high-achieving students experienced varying outcomes, including a decline in autonomy and engagement.
Challenges: High-achieving students showed less benefit from personalized recommendations, experiencing a negative impact on autonomy. The shared control system may have reduced engagement among lower-performing students.
Implementation Barriers
Technological Dependence
Extensive reliance on GAI tools may foster dependency, reducing students' independent thinking and self-regulatory skills.
Proposed Solutions: Design GAI interventions that balance support with opportunities for independent learning and self-regulation.
User Engagement
Students' varied engagement levels with GAI feedback can lead to uneven outcomes, particularly among different achievement levels.
Proposed Solutions: Implement tailored usage patterns that align with the specific needs and characteristics of learners.
Project Team
Xusheng Dai
Researcher
Zhaochun Wen
Researcher
Jianxiao Jiang
Researcher
Huiqin Liu
Researcher
Yu Zhang
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Xusheng Dai, Zhaochun Wen, Jianxiao Jiang, Huiqin Liu, Yu Zhang
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai