Challenges in the Automatic Analysis of Students' Diagnostic Reasoning
Project Overview
The document explores the application of generative AI in education, specifically its potential for automating the analysis of students' diagnostic reasoning skills, which are essential in fields like medicine and teaching. It underscores the necessity for scalable feedback mechanisms to enhance educational outcomes and presents a novel corpus of diagnostic reasoning texts that serve as a foundation for this automation. The authors identify key challenges in accurately recognizing the epistemic activities that underpin diagnostic reasoning and propose an evaluation framework aimed at guiding the development of future AI methodologies. By addressing these challenges, the document highlights the importance of generative AI in improving educational psychology and fostering better learning experiences for students.
Key Applications
Automatic analysis of students' diagnostic reasoning using AI
Context: Educational settings for medical students and pre-service teachers
Implementation: Creation of a corpus of diagnostic reasoning texts annotated with epistemic activities to facilitate AI analysis
Outcomes: Potential for large-scale feedback provision on students' reasoning skills
Challenges: Difficulties in accurately identifying and distinguishing between epistemic activities, especially overlapping ones
Implementation Barriers
Technical barrier
The challenges of accurately identifying epistemic activities, particularly distinguishing similar activities and handling overlapping segments
Proposed Solutions: Development of specific performance metrics for each challenge and enhancement of AI systems to address these issues
Project Team
Claudia Schulz
Researcher
Christian M. Meyer
Researcher
Michael Sailer
Researcher
Jan Kiesewetter
Researcher
Elisabeth Bauer
Researcher
Frank Fischer
Researcher
Martin R. Fischer
Researcher
Iryna Gurevych
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Claudia Schulz, Christian M. Meyer, Michael Sailer, Jan Kiesewetter, Elisabeth Bauer, Frank Fischer, Martin R. Fischer, Iryna Gurevych
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai