Reforming Physics Exams Using Openly Accessible Large Isomorphic Problem Banks created with the assistance of Generative AI: an Explorative Study
Project Overview
The document outlines the transformative role of generative AI, particularly large language models like GPT-3, in reforming assessment practices within STEM education, specifically in physics. It emphasizes the creation of extensive isomorphic problem banks that offer a diverse range of assessment problems aimed at reducing rote memorization while ensuring fairness and appropriate difficulty levels. Exploratory studies indicated that when students were granted open access to these AI-generated problem banks, there was no significant change in exam difficulty or an increase in rote learning behaviors. These findings suggest that the integration of generative AI in education could enhance assessment methods, fostering a more effective learning environment that promotes critical thinking and problem-solving skills among students.
Key Applications
Open Isomorphic Problem Banks
Context: Introductory physics course at the University of Central Florida with 328 registered students, including diverse demographics.
Implementation: Large numbers of isomorphic physics problems were created with the assistance of GPT-3 and other tools. Students had access to these problem banks as practice material prior to mid-term exams.
Outcomes: Results indicated that using open isomorphic problem banks did not significantly change exam difficulty or promote rote memorization. Students performed comparably on open bank and transfer problems.
Challenges: Potential challenges included maintaining the quality and difficulty of generated problems, students may still rely on rote memorization, and ensuring equitable access to problem context.
Implementation Barriers
Technical Barrier
Maintaining the quality and difficulty of generated isomorphic problems. Involvement of human experts in the process to evaluate the clarity and appropriateness of problems.
Proposed Solutions: Human experts should be involved to ensure the quality and appropriateness of generated problems.
Equity Barrier
Certain isomorphic problem variations may not be considered isomorphic by all cultural backgrounds, potentially leading to bias. Future studies should focus on ensuring that the generated problems are culturally inclusive and do not reinforce existing biases.
Proposed Solutions: Future studies should prioritize cultural inclusivity in generated problems to avoid reinforcing existing biases.
Project Team
Zhongzhou Chen
Researcher
Emily Frederick
Researcher
Colleen Cui
Researcher
Munaimah Khan
Researcher
Christopher Klatt
Researcher
Mercedith Huang
Researcher
Shiyang Su
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Zhongzhou Chen, Emily Frederick, Colleen Cui, Munaimah Khan, Christopher Klatt, Mercedith Huang, Shiyang Su
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai