The end of multiple choice tests: using AI to enhance assessment
Project Overview
The document explores the transformative role of generative AI in education, particularly in enhancing the effectiveness of assessments such as multiple-choice tests. It identifies the shortcomings of traditional assessment methods, which often fail to capture the nuances of student understanding. To address these limitations, the document introduces a chatbot named 'Dewey' that leverages AI to analyze student explanations for their selected answers. By doing so, Dewey can uncover misconceptions and deliver personalized, actionable feedback to students, ultimately aiming to bridge gaps in knowledge and improve overall learning outcomes. The findings underscore the potential of generative AI not only to make assessments more insightful but also to foster a deeper understanding among students, paving the way for a more adaptive and responsive educational environment.
Key Applications
Dewey, the AI chatbot for analyzing student explanations of answers in multiple choice assessments
Context: Undergraduate molecular biology courses, specifically targeting students answering multiple-choice questions
Implementation: Students explain their answer choices for multiple-choice questions, and Dewey analyzes these explanations to identify misconceptions and suggest instructional improvements.
Outcomes: Immediate feedback to instructors about student understanding; improved instructional strategies based on actionable insights; potential for better learning outcomes.
Challenges: Dependence on a defined training set for accurate analysis; ensuring all student misconceptions are captured; the need for instructors to act on feedback.
Implementation Barriers
Instructional
Instructors may not embrace actionable formative assessments and may continue traditional assessment methods.
Proposed Solutions: Encouraging a culture shift among instructors and course designers towards valuing and implementing AI-informed feedback.
Technological
The chatbot's training set may not cover all necessary topics or advanced concepts required for comprehensive analysis.
Proposed Solutions: Expanding Dewey's training set to include a broader range of topics and updating it regularly based on curriculum needs.
Project Team
Michael Klymkowsky
Researcher
Melanie M. Cooper
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Michael Klymkowsky, Melanie M. Cooper
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai