Junior Software Developers' Perspectives on Adopting LLMs for Software Engineering: a Systematic Literature Review

Large Language Models (LLMs) and AI Code Generators

Context: Used in various educational settings, including introductory programming courses and higher-level programming education. LLMs like ChatGPT and GitHub Copilot are utilized by both novice learners and junior software developers for coding tasks, debugging, and learning programming concepts. Usability studies focus on interactions between students and these AI tools.

Implementation: Integration of LLMs and AI code generators into curricula and systematic studies analyzing the usage and effectiveness of these tools in aiding coding tasks, problem-solving, and learning outcomes. Research also focuses on the interaction between users and AI programming assistants to improve design and functionality.

Outcomes: Enhanced productivity and skill improvement for learners, improved coding practices, and increased engagement in programming education. Positive effects on understanding programming concepts and problem-solving abilities, while also gaining insights into student needs and expectations.

Challenges: Concerns about output quality, overreliance on AI tools leading to diminished problem-solving skills, trust issues regarding AI-generated code quality, and skepticism about LLMs' effectiveness compared to traditional learning methods. Diverse opinions on the ethical implications of AI in education.

Generative AI in Computing Education

Context: Surveys and studies conducted with students and instructors to gather perceptions on the use of generative AI tools in educational settings.

Implementation: Collection of data regarding benefits and drawbacks from multiple perspectives, analyzing the role of generative AI in enhancing education.

Outcomes: Understanding of the diverse views on the integration of generative AI in education, including potential advantages and ethical concerns.

Challenges: Varied opinions on the effectiveness and ethical implications of AI technologies in educational contexts.

Quality of Output

LLMs sometimes generate incorrect or unhelpful code suggestions, leading to doubts in developers' abilities.

Proposed Solutions: Encourage manual validation of AI outputs and provide training on how to effectively evaluate AI-generated content.

Overreliance on AI

Junior developers may become overly dependent on LLM tools, which can hinder their skill development. This overreliance can lead to a lack of foundational programming skills.

Proposed Solutions: Promote a balanced approach to using AI tools, emphasizing the importance of maintaining foundational programming skills.

Integration Challenges

Challenges related to integrating LLM tools into existing software development workflows and educational frameworks.

Proposed Solutions: Develop guidelines and best practices for integrating AI tools into software engineering processes, and establish standardized protocols for implementation.

Perception Barrier

Negative perceptions and skepticism about AI tools among educators and students, leading to reluctance in adoption.

Proposed Solutions: Conduct workshops and training sessions to build understanding and trust in AI.

Ethical Barrier

Concerns regarding academic integrity, the potential for AI misuse, and the need for ethical considerations in AI usage.

Proposed Solutions: Establish clear ethical guidelines for AI usage in educational contexts.

Project Contact: Dr. Jianhua Yang

LLM Model Version: gpt-4o-mini-2024-07-18

Analysis Provider: Openai