Grading Assistance for a Handwritten Thermodynamics Exam using Artificial Intelligence: An Exploratory Study
Project Overview
The document examines the use of generative AI in education, with a particular emphasis on AI-assisted grading implemented during a thermodynamics exam at ETH Zurich. It details the exploration of various workflows that incorporate Optical Character Recognition (OCR) and Large Language Models (LLMs) such as GPT-4 to grade handwritten student submissions. Key findings reveal significant challenges in converting handwritten solutions into machine-readable formats and underscore the effectiveness of AI in evaluating nuanced student responses. Furthermore, the study highlights the potential of generative AI to enhance educational research and improve accessibility in learning environments. Overall, the integration of AI technologies in grading processes demonstrates promise for transforming educational assessment and fostering more efficient and equitable evaluation methods.
Key Applications
AI-assisted grading workflow using MathPix and GPT-4V
Context: High-stakes thermodynamics exam for engineering students at ETH Zurich.
Implementation: Student handwritten solutions were scanned, processed with OCR, and then graded using GPT-4's AI capabilities.
Outcomes: Increased scalability of grading with the ability to handle large volumes of data; potential for consistent feedback.
Challenges: Handwritten text conversion, OCR accuracy, and the AI's inability to perform complex bookkeeping tasks.
Implementation Barriers
Technical barrier
Difficulty in converting handwritten answers to machine-readable formats, impacting grading accuracy.
Proposed Solutions: Recommendations include improving exam layout and using clearer handwriting techniques.
Operational barrier
The AI's struggle with complex grading criteria and bookkeeping errors when handling detailed rubrics.
Proposed Solutions: Using a hybrid approach where grading is conducted per part with fewer rubric items to enhance performance.
Project Team
Gerd Kortemeyer
Researcher
Julian Nöhl
Researcher
Daria Onishchuk
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Gerd Kortemeyer, Julian Nöhl, Daria Onishchuk
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai