On the robustness of ChatGPT in teaching Korean Mathematics
Project Overview
The document explores the role of generative AI, specifically ChatGPT, in education, particularly its application in solving Korean mathematics problems. It reports an accuracy rate of 66.72% for ChatGPT when tackling these mathematical questions, indicating a noteworthy level of reliability, yet underscores significant challenges in areas such as sequential reasoning and the interpretation of visual information. These findings suggest that while ChatGPT's performance is consistent with established educational theories, which supports its potential use in personalized learning environments, there is a pressing need for enhancements. Specifically, improvements are necessary to bolster the model's effectiveness in non-English educational contexts and to better address complex mathematical tasks. Overall, the study highlights both the promise and the limitations of generative AI in educational applications, advocating for further developments to fully realize its capabilities in supporting diverse learning needs.
Key Applications
ChatGPT for solving Korean mathematics questions
Context: Educational assessment for Korean university entrance exams (CSAT)
Implementation: Evaluation of ChatGPT's responses to a dataset of 586 Korean mathematics questions, including accuracy testing and question rating.
Outcomes: Achieved 66.72% accuracy in solving questions and demonstrated good performance in assessing question difficulty and relevance.
Challenges: Struggled with sequential questions, visual information interpretation, and handling non-English contexts.
Implementation Barriers
Technical Barrier
ChatGPT's performance declines in non-English assessments and complex mathematical problems. Korean is a low-resource language, resulting in limited training data compared to English.
Proposed Solutions: Fine-tuning models on bilingual datasets, enhancing diagram comprehension for better performance, and leveraging translation-based methods to improve accuracy.
Project Team
Phuong-Nam Nguyen
Researcher
Quang Nguyen-The
Researcher
An Vu-Minh
Researcher
Diep-Anh Nguyen
Researcher
Xuan-Lam Pham
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Phuong-Nam Nguyen, Quang Nguyen-The, An Vu-Minh, Diep-Anh Nguyen, Xuan-Lam Pham
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai