AI-Tutoring in Software Engineering Education
Project Overview
The document examines the implementation of a generative AI-Tutor, built on OpenAI's GPT-3.5-Turbo model, within an Automated Programming Assessment System (APAS) known as Artemis, focusing on its application in programming education. It highlights how students engage with the AI-Tutor, noting the advantages of providing timely feedback and enhancing scalability in learning environments. However, it also addresses challenges such as the tool's tendency to give generic responses and concerns that it may hinder students' learning progress. The findings suggest a mixed reception among students, indicating a need for more specific feedback and greater interactivity to improve the overall effectiveness of the AI-Tutor. This research underscores the potential of generative AI in educational settings while also pointing out areas for enhancement to better support student learning experiences.
Key Applications
AI-Tutor integrated into the Artemis platform
Context: Programming education for first-year students at the University of Innsbruck in an 'Introduction to Programming' course.
Implementation: Integration of the GPT-3.5-Turbo model into the Artemis platform to provide AI-driven feedback on programming assignments.
Outcomes: Offers timely feedback and scalability; identified two user personas based on interaction patterns (Iterative Ivy and Hybrid Harry).
Challenges: Generic responses, concerns about learning inhibition, and lack of interactive dialog.
Implementation Barriers
Technical
Operational vulnerabilities related to API availability, leading to potential downtimes that could affect the AI-Tutor's functionality.
Proposed Solutions: Ensure robust API management and consider fallback options or local processing when the API is unavailable.
User Experience
Students reported that the AI-Tutor's feedback was often too generic and lacked specificity, leading to dissatisfaction.
Proposed Solutions: Enhance the AI-Tutor's feedback specificity through improved prompt engineering, allow for follow-up questions to increase interactivity, and ensure user engagement in the learning process.
Psychological
Students expressed apprehension about becoming overly reliant on the AI-Tutor, fearing it might inhibit their learning progress.
Proposed Solutions: Emphasize the AI-Tutor's role as a supplementary tool and encourage independent problem-solving skills.
Project Team
Eduard Frankford
Researcher
Clemens Sauerwein
Researcher
Patrick Bassner
Researcher
Stephan Krusche
Researcher
Ruth Breu
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Eduard Frankford, Clemens Sauerwein, Patrick Bassner, Stephan Krusche, Ruth Breu
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai