How to Teach Programming in the AI Era? Using LLMs as a Teachable Agent for Debugging
Project Overview
The document explores the application of generative AI in education, focusing on the use of Large Language Models (LLMs) through a system called HypoCompass, which enhances Computer Science education. HypoCompass allows students to engage in debugging practice by serving as Teaching Assistants for LLM-generated buggy code, thereby fostering hypothesis construction—a critical skill in debugging processes. The findings indicate that this innovative approach not only improves student performance but also increases efficiency in generating educational materials compared to traditional teaching methods. Overall, the integration of generative AI, exemplified by HypoCompass, demonstrates significant potential in enhancing learning experiences and outcomes in the field of Computer Science education.
Key Applications
HypoCompass
Context: Computer Science education, specifically for novice programmers (CS1)
Implementation: Students interact with LLM agents simulating novice programmers to help debug code, focusing on hypothesis construction and test suite creation.
Outcomes: Improved student performance by 12% in debugging tests and reduced instructor material generation time by 4.67 times compared to human TAs.
Challenges: Students may struggle with complex debugging tasks and require support to effectively engage with LLMs.
Implementation Barriers
Logistical
Limited instructional time and resources for teaching debugging skills in formal curricula.
Proposed Solutions: Utilization of LLMs like HypoCompass to automate material generation and provide scaffolded practice.
Cognitive
Students often find debugging frustrating due to the cognitive load of simultaneously understanding code and identifying errors.
Proposed Solutions: The teachable agent framework allows students to focus on hypothesis construction while LLMs handle code completion and feedback.
Project Team
Qianou Ma
Researcher
Hua Shen
Researcher
Kenneth Koedinger
Researcher
Tongshuang Wu
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Qianou Ma, Hua Shen, Kenneth Koedinger, Tongshuang Wu
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai