Marking: Visual Grading with Highlighting Errors and Annotating Missing Bits
Project Overview
The document discusses the innovative use of generative AI in education through a task called 'Marking', which enhances automated grading systems by delivering nuanced feedback on student responses. Unlike conventional binary scoring methods, 'Marking' classifies segments of answers as correct, incorrect, or irrelevant, while also identifying omissions compared to a gold standard answer. This approach utilizes Natural Language Inference (NLI) principles and advanced transformer models such as BERT and RoBERTa to generate comprehensive assessments. To support this initiative, a specialized dataset named BioMarking has been developed for training and evaluating the task, with the objective of improving the quality and depth of feedback provided to students. Overall, the findings indicate that the integration of generative AI in grading not only enhances the precision of assessments but also fosters a more constructive learning environment by offering detailed insights into student performance.
Key Applications
'Marking' - an AI-driven grading framework
Context: Higher education, focusing on biology assessment for university students
Implementation: Developed an AI model trained on a dataset of student responses and gold standards using transformer models.
Outcomes: Provides detailed feedback on correctness and omissions in student responses, enhancing learning outcomes.
Challenges: Complexity in grading nuances and ensuring model accuracy across diverse responses.
Implementation Barriers
Technical barrier
The complexity of accurately grading nuanced student responses and ensuring the model can generalize well.
Proposed Solutions: Utilization of advanced transformer models and preprocessing techniques like Dual Instance Pairing and stopword removal.
Data barrier
Scarcity of high-quality annotated datasets for training AI models in educational contexts.
Proposed Solutions: Creation of the BioMarking dataset specifically curated for the 'Marking' task.
Project Team
Shashank Sonkar
Researcher
Naiming Liu
Researcher
Debshila B. Mallick
Researcher
Richard G. Baraniuk
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Shashank Sonkar, Naiming Liu, Debshila B. Mallick, Richard G. Baraniuk
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai