Using Think-Aloud Data to Understand Relations between Self-Regulation Cycle Characteristics and Student Performance in Intelligent Tutoring Systems
Project Overview
The document explores the integration of generative AI in education, particularly focusing on its application in intelligent tutoring systems (ITS) for teaching stoichiometry in chemistry. By employing AI tools to transcribe think-aloud data, the study investigates the connection between self-regulated learning (SRL) behaviors and student performance. Key findings indicate that students who exhibit higher correctness in problem-solving are more likely to engage effectively in the later stages of the SRL cycle, with immediate actions taken after planning leading to improved performance. These insights underline the importance of designing ITS that better support SRL processes, suggesting that tailored interventions could enhance student learning outcomes. Overall, the findings emphasize the significant role of generative AI in analyzing learning behaviors and improving educational tools to foster better student engagement and achievement in complex subjects.
Key Applications
Intelligent Tutoring Systems (ITS) for stoichiometry chemistry
Context: Undergraduate and graduate students learning stoichiometry chemistry
Implementation: Students worked with two ITS while verbalizing their thought processes (think-aloud method). Data was collected and analyzed to label SRL behaviors based on AI-assisted transcriptions.
Outcomes: Improved understanding of the relationship between SRL behaviors and problem-solving performance; insights into how specific SRL strategies correlate with correctness in problem-solving.
Challenges: Resource-intensive coding of SRL behaviors; potential inaccuracies in AI transcription; limited generalizability to other domains or student populations.
Implementation Barriers
Technical
The process of coding SRL behaviors from think-aloud data is labor-intensive and requires high accuracy.
Proposed Solutions: Utilizing AI transcription tools to assist in generating accurate think-aloud transcripts; developing automated methods for SRL behavior identification.
Generalizability
The findings are based on a specific domain (stoichiometry chemistry) and may not apply broadly to other educational contexts.
Proposed Solutions: Future studies should replicate the methodology across various subjects and educational settings to assess the generalizability of the results.
Project Team
Conrad Borchers
Researcher
Jiayi Zhang
Researcher
Ryan S. Baker
Researcher
Vincent Aleven
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Conrad Borchers, Jiayi Zhang, Ryan S. Baker, Vincent Aleven
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai