That's Not the Feedback I Need! -- Student Engagement with GenAI Feedback in the Tutor Kai
Project Overview
The document explores the application of Generative AI (GenAI) in computing education, specifically examining student interactions with AI-generated feedback during programming tasks. It highlights the development of a web application named Tutor Kai, which combines programming tasks, a code editor, and both GenAI and compiler feedback. The study reveals that students, particularly those with less programming experience, often prioritize GenAI feedback over compiler feedback, which can impede their learning by neglecting critical insights from the latter. While GenAI feedback is generally viewed positively by students, the findings underscore significant challenges related to its clarity and effectiveness, especially for novice learners. Overall, the research emphasizes the potential benefits of GenAI in education while calling attention to the need for improvements in feedback comprehensibility to enhance the learning experience.
Key Applications
Tutor Kai - a web application for programming tasks with GenAI feedback
Context: Used in an introductory programming course for undergraduate computer science students
Implementation: A mixed-methods approach using eye-tracking and think-aloud protocols, involved 11 students who engaged with programming tasks while receiving both GenAI and compiler feedback
Outcomes: Inexperienced students spent more time on GenAI feedback, but experienced students utilized it more effectively, leading to improved problem-solving; 60.9% of GenAI feedback was helpful for experienced students compared to 43.0% for inexperienced students
Challenges: Inexperienced students often found GenAI feedback incomprehensible and tended to bypass compiler feedback, raising concerns about their understanding of programming concepts
Implementation Barriers
Cognitive Barrier
Inexperienced students often do not understand GenAI feedback due to its complexity and lack of foundational programming knowledge.
Proposed Solutions: Implement instructional methods that gradually introduce programming concepts and GenAI feedback, ensuring that students develop necessary skills.
Feedback Overreliance
Students, particularly inexperienced ones, may rely solely on GenAI feedback and neglect traditional compiler feedback, which is essential for learning.
Proposed Solutions: Encourage a balanced approach to feedback by designing tasks that require reading both GenAI and compiler feedback before proceeding.
Project Team
Sven Jacobs
Researcher
Maurice Kempf
Researcher
Natalie Kiesler
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Sven Jacobs, Maurice Kempf, Natalie Kiesler
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai