Autograding Mathematical Induction Proofs with Natural Language Processing
Project Overview
The document explores the use of a natural language processing (NLP) model designed for autograding mathematical induction proofs in education, showcasing the transformative potential of generative AI in providing immediate feedback to enhance student learning. The study reports that students who utilized the autograder demonstrated marked improvements in their proof-writing skills. Despite these advancements, there remains a notable skepticism among students regarding the accuracy and reliability of AI autograders compared to traditional human grading methods. The findings underscore the necessity for ongoing development of these AI models to bolster their effectiveness and address students' trust concerns. Overall, the document highlights both the promising applications of generative AI in educational settings and the challenges that must be navigated to foster acceptance and efficacy in AI-assisted learning environments.
Key Applications
Autograder for mathematical proofs using NLP
Context: Higher education mathematics course, specifically Discrete Mathematics
Implementation: Developed a pipeline of machine learning models to autograde freeform mathematical proofs, trained on proof data collected from students.
Outcomes: Students using the autograder significantly improved their proof scores compared to those who self-evaluated.
Challenges: Students are hesitant to trust AI grading compared to human grading; concerns about AI accuracy and feedback specificity.
Implementation Barriers
Trust and Accuracy Barrier
Students are more likely to trust human graders than AI autograders. AI autograders may produce incorrect grades leading to mistrust, especially for subjective tasks and mathematical nuances.
Proposed Solutions: Provide more detailed feedback and explanations to improve student trust in AI systems. Enhance the models to better handle mathematical language and reasoning, possibly by integrating generative models for feedback.
Project Team
Chenyan Zhao
Researcher
Mariana Silva
Researcher
Seth Poulsen
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Chenyan Zhao, Mariana Silva, Seth Poulsen
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai