Can an AI-tool grade assignments in an introductory physics course?
Project Overview
The document examines the application of generative AI, particularly GPT-4, in the educational context of grading physics assignments. It highlights the AI's capability to offer feedback on written problem solutions, emphasizing its effectiveness in both formative and summative assessments. Although GPT-4 demonstrates potential in grading and feedback, the document notes its current limitations, particularly in managing symbolic and numerical calculations, which renders it unreliable for high-stakes assessments. Despite these challenges, the AI can serve as a supportive tool for human graders by pre-sorting student solutions and generating preliminary scores, thereby enhancing the grading process. Overall, the findings suggest that while generative AI holds significant promise for improving educational assessments, further advancements are necessary to fully integrate it into high-stakes evaluation scenarios.
Key Applications
AI-assisted grading workflow using GPT-4
Context: Introductory physics courses targeting students in higher education
Implementation: Assignments are scanned, converted to machine-readable format, and graded by GPT-4, which provides feedback and scores.
Outcomes: AI can provide formative feedback and assist in sorting solutions for human grading.
Challenges: Limited reliability in numerical and symbolic calculations; dependency on access to APIs for functionality.
Implementation Barriers
Technical Barrier
Access to necessary APIs for optical character recognition and AI functionalities is limited.
Proposed Solutions: Future work should include securing API access and developing reliable workflows.
Reliability Barrier
Current AI capabilities in performing accurate symbolic and numerical calculations are insufficient for high-stakes grading.
Proposed Solutions: Use AI for preliminary grading and sorting, while retaining human oversight for final assessments.
Privacy Barrier
Concerns regarding privacy and data security when using cloud-based AI solutions for grading.
Proposed Solutions: Implement secure systems and ensure compliance with data protection regulations.
Project Team
Gerd Kortemeyer
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Gerd Kortemeyer
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai