To Google or To ChatGPT? A Comparison of CS2 Students' Information Gathering Approaches and Outcomes
Project Overview
The document explores the implementation of generative AI, notably ChatGPT, in programming education and contrasts its effectiveness with traditional web-based resources such as Google and YouTube. While generative AI provides interactive and personalized learning experiences, findings indicate that students generally achieve better outcomes with conventional methods. The research shows that students using traditional resources tend to engage in keyword-based searches, whereas those interacting with LLMs typically pose more targeted questions. Despite the advantages of AI in promoting self-directed learning, significant challenges persist, including the potential for over-reliance on AI tools, issues surrounding the accuracy of information provided, and the necessity for a well-structured pedagogical approach. Overall, the document underscores the nuanced role of generative AI in education, advocating for a balanced integration that leverages its strengths while addressing its limitations.
Key Applications
ChatGPT as a tool for learning programming concepts
Context: Higher education programming courses, specifically for CS2 students
Implementation: Conducted a mixed-methods study comparing learning outcomes using ChatGPT and traditional methods.
Outcomes: Participants showed better performance in quizzes and debugging tasks when using traditional methods compared to ChatGPT.
Challenges: LLMs can generate inaccurate information, lack understanding of nuanced learner needs, and may foster over-reliance on AI.
Implementation Barriers
Technical
LLMs sometimes provide inaccurate or misleading information.
Proposed Solutions: Implement structured training for students on when and how to effectively use LLMs.
Pedagogical
Incorporating AI tools without sufficient pedagogical support can hinder critical thinking.
Proposed Solutions: Develop structured frameworks for integrating AI tools into the curriculum.
Cognitive
Students may over-rely on AI tools, reducing their problem-solving skills and critical thinking.
Proposed Solutions: Encourage a balanced approach that combines AI assistance with traditional learning methods.
Project Team
Aayush Kumar
Researcher
Daniel Prol
Researcher
Amin Alipour
Researcher
Sruti Srinivasa Ragavan
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Aayush Kumar, Daniel Prol, Amin Alipour, Sruti Srinivasa Ragavan
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai