Automated Assessment of Multimodal Answer Sheets in the STEM domain
Project Overview
The document explores the application of generative AI in education, specifically addressing automated assessment techniques designed for grading multimodal answer sheets in STEM subjects. It identifies the challenges inherent in evaluating assessments that include both textual responses and diagrams, proposing a sophisticated system that leverages Natural Language Processing and Large Language Models to ensure accurate and efficient grading. By minimizing the need for manual intervention, this system aims to enhance the reliability of grading practices, thereby fostering a more equitable educational environment. The findings suggest that the integration of generative AI not only streamlines the assessment process but also improves the overall fairness and effectiveness of educational evaluations, ultimately benefiting both educators and students.
Key Applications
Automated assessment system for grading textual answers and flowchart diagrams in STEM.
Context: STEM education, targeting students in subjects like Data Structures, Computer Networks, and Software Engineering.
Implementation: A two-stage approach involving textual evaluation using LLMs and diagram evaluation utilizing object detection and context-aware analysis.
Outcomes: Improved grading accuracy, efficiency, and fairness; insights into student comprehension; reduced manual grading efforts.
Challenges: Handling the variability in diagrams, maintaining accuracy in OCR for handwritten text, and ensuring contextual understanding in evaluation.
Implementation Barriers
Technological Barrier
Challenges in achieving high accuracy in Optical Character Recognition (OCR) for handwritten text.
Proposed Solutions: Utilizing advanced models like TrOCR and implementing line-by-line segmentation to enhance recognition accuracy.
Evaluation Challenge
Comparing diagrams that may have different representations but convey the same meaning, leading to lower scores based on visual similarity.
Proposed Solutions: Transforming diagrams into textual representations for a nuanced assessment and leveraging LLMs for contextual evaluation.
Project Team
Rajlaxmi Patil
Researcher
Aditya Ashutosh Kulkarni
Researcher
Ruturaj Ghatage
Researcher
Sharvi Endait
Researcher
Geetanjali Kale
Researcher
Raviraj Joshi
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Rajlaxmi Patil, Aditya Ashutosh Kulkarni, Ruturaj Ghatage, Sharvi Endait, Geetanjali Kale, Raviraj Joshi
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai