Statistical investigations into the geometry and homology of random programs
Project Overview
The document explores the integration of generative AI tools, particularly ChatGPT and TinyLlama, in programming education, highlighting their effectiveness in teaching students coding skills. By employing statistical analyses and geometric/topological methods, the authors examine how different input prompts influence the quality and utility of the generated code. The findings reveal that ChatGPT consistently produces more compact and coherent responses, making it a reliable choice for structured learning, while TinyLlama, despite its variability, offers potential advantages for exploratory and creative learning experiences. Overall, the study underscores the promise of generative AI in enhancing educational outcomes in programming, suggesting that these tools can cater to varied learning styles and needs.
Key Applications
ChatGPT and TinyLlama as programming assistants
Context: Programming education for students learning Python
Implementation: Students queried the models to generate code snippets for specific programming tasks.
Outcomes: ChatGPT provided more consistent and compact code responses. TinyLlama had greater variability, which may aid in exploratory learning.
Challenges: Syntax errors in generated code snippets; longer computation times for distance matrix analysis.
Implementation Barriers
Technical Barrier
Generated code snippets often contained syntax errors, which limited their usability in educational contexts.
Proposed Solutions: Improving the models' training datasets to include more syntactically correct examples; implementing robust error-checking mechanisms.
Computational Barrier
Calculating distance matrices for comparing generated code was computationally intensive and time-consuming.
Proposed Solutions: Exploring parallel processing methods to speed up calculations; optimizing the algorithms used for distance computation.
Project Team
Jon Sporring
Researcher
Ken Friis Larsen
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Jon Sporring, Ken Friis Larsen
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai