Synthesizing High-Quality Programming Tasks with LLM-based Expert and Student Agents
Project Overview
Generative AI is making significant strides in enhancing education, particularly in the field of computing, as evidenced by the P YTASKSYN system, which leverages a multi-agent approach to automatically generate and validate personalized programming tasks and feedback. This innovative system simulates the roles of experts, tutors, and students, allowing for the creation of tailored learning experiences that are more engaging and relevant. While the potential of AI-generated tasks is promising, the document highlights ongoing challenges, particularly regarding the quality and comprehensibility of these tasks. Nevertheless, the P YTASKSYN system has demonstrated notable improvements in task quality compared to traditional methods and has effectively reduced the workload for educators, paving the way for a more efficient and personalized learning environment. Overall, generative AI's application in education not only enhances student learning experiences but also supports educators in managing their responsibilities, suggesting a transformative impact on teaching and learning processes in the digital age.
Key Applications
PYTASKSYN
Context: Computing education, targeting students learning programming concepts.
Implementation: A multi-agent system using strong and weaker generative models to generate and validate programming tasks.
Outcomes: High-quality programming tasks comparable to those created by experts, reduced workload for educators, and increased engagement among students.
Challenges: Quality gap between AI-generated and expert-created tasks, incomprehensible tasks, reliance on human validation.
Implementation Barriers
Quality Assurance and Need for Human Intervention
AI-generated tasks may not align with target programming concepts and can be incomprehensible for students. Existing models struggle with self-validation and may require human teachers for validation of AI-generated tasks.
Proposed Solutions: Implement a multi-agent validation system that breaks the task synthesis process into stages to ensure quality. Use a combination of simulated agents to provide comprehensive validation without needing constant human oversight.
Project Team
Manh Hung Nguyen
Researcher
Victor-Alexandru Pădurean
Researcher
Alkis Gotovos
Researcher
Sebastian Tschiatschek
Researcher
Adish Singla
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Manh Hung Nguyen, Victor-Alexandru Pădurean, Alkis Gotovos, Sebastian Tschiatschek, Adish Singla
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai