Evaluating Contextually Personalized Programming Exercises Created with Generative AI
Project Overview
The document explores the application of generative AI, particularly large language models (LLMs) like GPT-4, in the realm of education, focusing on their role in developing personalized programming exercises for introductory programming courses. It emphasizes how context personalization can significantly boost student engagement and improve learning outcomes. A user study was conducted to assess the quality of AI-generated exercises, which garnered mostly positive feedback from students, indicating high quality in the tasks provided. However, the study also identified certain challenges, including issues with the depth of personalization and occasional mismatches in the difficulty levels of the exercises. Overall, the findings highlight the potential of generative AI to enhance educational experiences while acknowledging areas for improvement in tailoring content to individual learner needs.
Key Applications
Contextually personalized programming exercises generated with GPT-4
Context: Introductory programming course at Aalto University, targeting both university students and lifelong learners
Implementation: AI-generated exercises embedded in an online IDE, allowing students to select themes, concepts, and difficulty levels
Outcomes: High quality of exercises, positive student engagement, and perceived usefulness for learning
Challenges: Shallow personalization of exercises, some exercises being too easy or too difficult compared to expected levels
Implementation Barriers
Technical Barrier
The difficulty of ensuring that generated exercises match the intended difficulty level.
Proposed Solutions: Use of expert evaluations to assess exercise quality and provide feedback for improvement.
Personalization Barrier
Shallow personalization may limit engagement and learning effectiveness.
Proposed Solutions: Future work to enhance the depth of personalization and provide a wider range of contextual themes.
Engagement Barrier
The exercises may not match the specific interests of all students.
Proposed Solutions: Allowing students to generate exercises on-demand instead of using a predefined list of themes.
Project Team
Evanfiya Logacheva
Researcher
Arto Hellas
Researcher
James Prather
Researcher
Sami Sarsa
Researcher
Juho Leinonen
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Evanfiya Logacheva, Arto Hellas, James Prather, Sami Sarsa, Juho Leinonen
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai