Assessing Personalized AI Mentoring with Large Language Models in the Computing Field
Project Overview
This document explores the use of Large Language Models (LLMs) in personalized AI mentoring specifically within computing education, emphasizing the capabilities of models like GPT-4, LLaMA 3, and Palm 2 to create customized mentoring experiences that cater to students' varied social and educational backgrounds. The findings suggest that while these LLMs are effective in providing tailored support, the presence of human mentors remains essential for fostering meaningful educational interactions. Additionally, the document addresses potential challenges associated with the use of AI in mentoring, including concerns about algorithmic bias and the importance of ethical considerations in the deployment of AI technologies. Overall, the integration of generative AI in education holds promise for enhancing personalized learning, though careful attention to ethical implications and the role of human educators is necessary to ensure successful outcomes.
Key Applications
Personalized AI mentoring using LLMs
Context: Computing education for students with diverse social and educational backgrounds, focusing on career planning.
Implementation: Evaluated three LLMs (GPT-4, LLaMA 3, Palm 2) using a zero-shot learning approach with a custom NLP analytics pipeline.
Outcomes: GPT-4 provided the most personalized and useful mentoring responses, particularly in addressing specific student challenges.
Challenges: Limitations included algorithmic bias, privacy concerns, and the need for human involvement to enhance personalized responses.
Implementation Barriers
Ethical
Concerns regarding algorithmic bias and privacy in AI mentoring systems.
Proposed Solutions: Incorporating human mentors and transparent practices to mitigate biases and ensure ethical use of AI.
Project Team
["Xiao Luo", "Sean O"Connell", "Shamima Mithun"]
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: ["Xiao Luo", "Sean O"Connell", "Shamima Mithun"]
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai