The Effects of Generative AI Agents and Scaffolding on Enhancing Students' Comprehension of Visual Learning Analytics
Project Overview
The document explores the integration of generative AI (GenAI) in education, particularly focusing on its application to enhance students' comprehension of visual learning analytics (VLA) through innovative scaffolding techniques. It emphasizes that traditional methods, like data storytelling, while effective, struggle with scalability. A randomized controlled trial involving 117 higher education students revealed that proactive GenAI agents significantly improve comprehension and foster authentic learning experiences compared to passive agents and conventional scaffolding methods. Additionally, the study highlights how these GenAI agents utilize scaffolding prompts and questions alongside various visualizations, such as bar charts and communication networks, to analyze student interactions and task prioritization in simulations. By employing evaluation questions that target different levels of Bloom's taxonomy, the findings demonstrate that generative AI can effectively facilitate understanding and enhance learning outcomes in educational settings. Overall, the document underscores the potential of GenAI agents to transform educational practices by providing scalable, personalized support that aligns with students' learning needs.
Key Applications
Generative AI agents for visual learning analytics and comprehension support
Context: Higher education and nursing education contexts, focusing on helping students comprehend visual learning analytics through various visualizations and guided scaffolding.
Implementation: Utilizing generative AI agents (both passive and proactive) to enhance student understanding of visual learning analytics. The implementation includes visualizations such as bar charts, communication networks, and ward maps, combined with scaffolding prompts that guide student comprehension and task prioritization in educational simulations.
Outcomes: Significant improvements in students' comprehension of visual learning analytics, including enhanced understanding of task prioritization, communication interactions, and stress levels during simulations. Benefits observed persist beyond the intervention.
Challenges: Limited empirical evidence supporting the effectiveness of generative AI; potential difficulties in interpreting visual data; and the risk of developing an illusion of competence without genuine understanding. Effective scaffolding is crucial to support comprehension.
Implementation Barriers
Technical barrier
Generative AI may produce inaccurate content (hallucination) that could mislead students.
Proposed Solutions: Implement retrieval-augmented generation (RAG) techniques to confine content generation to relevant material and reduce inaccuracies.
Literacy and Interpretational barrier
Many students lack data visualisation literacy and may struggle to interpret complex visual data, complicating their ability to understand the implications of the visualizations.
Proposed Solutions: Integrate explanatory and interactive features in educational tools to support comprehension, and implement scaffolding strategies that guide students through the visualizations step-by-step, with clear prompts and feedback.
Project Team
Lixiang Yan
Researcher
Roberto Martinez-Maldonado
Researcher
Yueqiao Jin
Researcher
Vanessa Echeverria
Researcher
Mikaela Milesi
Researcher
Jie Fan
Researcher
Linxuan Zhao
Researcher
Riordan Alfredo
Researcher
Xinyu Li
Researcher
Dragan Gašević
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Lixiang Yan, Roberto Martinez-Maldonado, Yueqiao Jin, Vanessa Echeverria, Mikaela Milesi, Jie Fan, Linxuan Zhao, Riordan Alfredo, Xinyu Li, Dragan Gašević
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai