Exploring Generative AI assisted feedback writing for students' written responses to a physics conceptual question with prompt engineering and few-shot learning
Project Overview
The document examines the application of generative AI, particularly GPT-3.5, in enhancing the grading and feedback process for student written responses in physics education. It emphasizes the efficiency of generative AI in saving instructors' time while ensuring the quality of feedback remains high. Through techniques like few-shot learning and prompt engineering, the AI generated feedback that students rated as equally accurate to that produced by humans, with many finding it more beneficial. The findings underscore the practicality of integrating generative AI into educational settings, particularly in large enrollment courses, indicating its potential to effectively support instructors and improve the overall learning experience.
Key Applications
GPT-3.5 for generating feedback on student written responses to conceptual questions.
Context: Introductory physics course with large enrollment (99 students).
Implementation: Used few-shot learning to train GPT-3.5 with a small number of example responses and iteratively refined prompts for generating feedback.
Outcomes: Students rated GPT-generated feedback equally correct but more useful than human feedback. Instructors found 70% of the AI-generated feedback satisfactory with minor to no modifications.
Challenges: AI feedback could reflect common student misconceptions, and distinguishing between AI and human feedback was difficult for students.
Implementation Barriers
Technical
The challenge of ensuring accuracy and quality of feedback generated by AI, particularly the possibility of 'hallucinations' or inaccurate outputs.
Proposed Solutions: Using prompt engineering techniques and domain-specific fine-tuning to improve feedback quality.
Perceptual
Students having difficulty distinguishing between AI-generated and human feedback, which could affect their trust in AI-generated responses. This highlights the need for increasing awareness of AI capabilities and limitations, and involving students in discussions about AI-generated feedback.
Proposed Solutions: Increasing awareness of AI capabilities and limitations; involving students in discussions about AI-generated feedback.
Project Team
Tong Wan
Researcher
Zhongzhou Chen
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Tong Wan, Zhongzhou Chen
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai