Using Generative AI and Multi-Agents to Provide Automatic Feedback
Project Overview
The document explores the use of a multi-agent system named AutoFeedback, which leverages generative AI to deliver automatic feedback on student responses in science assessments. By addressing common pitfalls of single-agent systems, such as over-praise and over-inference, AutoFeedback provides more nuanced and effective feedback. The findings indicate that this multi-agent approach significantly enhances the quality of feedback students receive, thus fostering personalized learning experiences. The implementation of AutoFeedback not only proves to be scalable but also leads to improved student learning outcomes, showcasing the potential of generative AI in education to support and enhance the learning process.
Key Applications
AutoFeedback - A multi-agent system for providing automatic feedback
Context: Educational context for middle school science assessments targeted at students constructing scientific models.
Implementation: Developed a multi-agent system with one agent generating feedback and another validating and refining it, tested on a dataset of 240 student responses.
Outcomes: Significantly reduced over-praise and over-inference in feedback, leading to more accurate and pedagogically sound evaluations.
Challenges: Some feedback still exhibited issues of over-praise and over-inference; challenges in ensuring complete accuracy in feedback validation.
Implementation Barriers
Technical Barrier
The quality of generative AI feedback still requires improvement and can exhibit over-praise and over-inference. Implementing multi-agent systems may enhance feedback accuracy and reliability.
Proposed Solutions: Implement multi-agent systems to enhance feedback accuracy and reliability.
Knowledge Barrier
Teachers and educational researchers may lack knowledge of programming or NLP/ML techniques necessary to leverage these systems effectively. Providing training and resources to educators on how to use and implement AI feedback systems is essential.
Proposed Solutions: Provide training and resources to educators on how to use and implement AI feedback systems.
Project Team
Shuchen Guo
Researcher
Ehsan Latif
Researcher
Yifan Zhou
Researcher
Xuan Huang
Researcher
Xiaoming Zhai
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Shuchen Guo, Ehsan Latif, Yifan Zhou, Xuan Huang, Xiaoming Zhai
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai