How Novices Use LLM-Based Code Generators to Solve CS1 Coding Tasks in a Self-Paced Learning Environment
Project Overview
The document examines the role of generative AI, particularly Large Language Models (LLMs) like OpenAI Codex, in supporting novice programmers learning Python within a self-paced environment. It explores the context in which these learners utilize AI, the types of prompts they generate, the characteristics of AI-produced code, and their various coding strategies, which are categorized into four distinct approaches: AI Single Prompt, AI Step-by-Step, Hybrid, and Manual. The study also addresses challenges faced by learners, including an over-reliance on AI assistance and potential violations of academic integrity. Overall, it underscores the importance of effectively integrating AI tools into computer science education to improve learning outcomes, suggesting that while LLMs can enhance the learning experience, careful consideration of their use is essential to foster independent problem-solving skills and maintain academic standards.
Key Applications
AI Code Generator (OpenAI Codex)
Context: Self-paced learning environment for novice programmers (ages 10-17) learning Python.
Implementation: Learners completed coding tasks with access to an AI code generator embedded in an online learning platform.
Outcomes: Learners demonstrated varying levels of engagement and understanding, with some achieving high correctness in code-authoring tasks using the AI tools.
Challenges: Challenges included over-reliance on AI for code generation, difficulty in prompt crafting, and issues with the correctness of AI-generated code.
Implementation Barriers
Technical Barrier
Learners struggled with technical jargon and expressing coding intent clearly in prompts.
Proposed Solutions: Improving prompt crafting skills through guided instruction and feedback mechanisms in the tool.
Dependence Barrier
Over-reliance on AI code generators led to diminished coding skills and understanding.
Proposed Solutions: Encouraging a balance between manual coding and AI assistance, promoting self-regulation and critical verification of AI-generated code.
Integrity Barrier
Concerns regarding academic integrity and plagiarism due to reliance on AI-generated outputs.
Proposed Solutions: Implementing educational strategies to reinforce the importance of original coding practices and understanding concepts.
Project Team
Majeed Kazemitabaar
Researcher
Xinying Hou
Researcher
Austin Henley
Researcher
Barbara J. Ericson
Researcher
David Weintrop
Researcher
Tovi Grossman
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Majeed Kazemitabaar, Xinying Hou, Austin Henley, Barbara J. Ericson, David Weintrop, Tovi Grossman
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai