GPT-4 can pass the Korean National Licensing Examination for Korean Medicine Doctors
Project Overview
The document examines the application of generative AI, specifically GPT-4, in the field of education, with a focus on its performance in the Korean National Licensing Examination for Korean Medicine Doctors (K-NLEKMD). It highlights GPT-4's potential to enhance medical education by providing assistance in answering exam questions related to Traditional Korean Medicine (TKM). The model achieved an accuracy rate of 66.18%, which exceeds the passing threshold, yet it struggled with questions that demanded specialized TKM knowledge, underscoring the challenges posed by cultural biases and the necessity for models to be tailored to specific educational contexts. This study suggests that while generative AI can be a valuable tool in educational settings, particularly in specialized fields, ongoing development and adaptation are crucial to fully leverage its capabilities and address cultural nuances.
Key Applications
GPT-4 for K-NLEKMD examination
Context: Medical education for Korean Medicine Doctors
Implementation: The model answered 340 questions from the K-NLEKMD with optimized prompts for better accuracy.
Outcomes: Achieved an accuracy of 66.18%, surpassing the required pass mark of 60%.
Challenges: Lower accuracy in questions requiring TKM-specific knowledge and cultural adaptation.
Implementation Barriers
Cultural Bias
LLMs may exhibit biases due to the predominance of English in training data, leading to underrepresentation of Korean TKM knowledge.
Proposed Solutions: Fine-tuning the model with culturally specific datasets and expanding benchmarks for multilingual evaluation.
Knowledge Adaptation
Difficulty in accurately answering questions that require specialized TKM knowledge due to the implicit nature of decision-making processes in TKM.
Proposed Solutions: Develop methods for integrating TKM-specific knowledge into the model and improving dataset quality.
Project Team
Dongyeop Jang
Researcher
Tae-Rim Yun
Researcher
Choong-Yeol Lee
Researcher
Young-Kyu Kwon
Researcher
Chang-Eop Kim
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Dongyeop Jang, Tae-Rim Yun, Choong-Yeol Lee, Young-Kyu Kwon, Chang-Eop Kim
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai