Evaluation of ChatGPT and Microsoft Bing AI Chat Performances on Physics Exams of Vietnamese National High School Graduation Examination
Project Overview
The document assesses the effectiveness of generative AI models, specifically ChatGPT and BingChat, in addressing high school physics questions from Vietnamese graduation exams conducted between 2019 and 2023. The evaluation reveals that both AI models underperformed compared to Vietnamese students, especially in complex application-based questions. While the findings indicate that these large language models (LLMs) hold promise for delivering personalized learning experiences and instant feedback, they also highlight significant shortcomings in accuracy and reliability, particularly in specialized subjects such as physics. This suggests that, despite the potential benefits of integrating generative AI into educational contexts, there is a pressing need for enhancements in the models to ensure they can meet the rigorous demands of academic assessments effectively.
Key Applications
AI-Powered Educational Content Generation and Feedback
Context: High school physics education in Vietnam and online learning environments for students and teachers, assisting instructors in presenting lessons and assessing learners.
Implementation: Utilization of AI technologies, including ChatGPT, BingChat, text-to-speech, and speech-driven-face technology, to provide immediate feedback, generate video lectures, and assist in lesson delivery without manual recording.
Outcomes: ['Immediate feedback and individualized learning experiences for students.', 'Reduction of workload for instructors and enhanced engagement for learners.', 'Facilitated easy editing of lessons without video recording.']
Challenges: ['Limited accuracy and inability to answer high-level application questions.', "Dependence on the quality of the virtual assistant's language processing.", 'Potential limitations in the sophistication of AI-generated content.']
Implementation Barriers
Technical
LLMs struggle to answer complex, high-level application questions in physics.
Proposed Solutions: Integrating more domain-specific knowledge into LLMs to improve reasoning capabilities.
Integration
Challenges in incorporating LLMs into existing educational frameworks.
Proposed Solutions: Developing platforms and tools that facilitate the integration of LLMs into classroom settings.
Data Privacy
Concerns regarding privacy and data security in educational contexts.
Proposed Solutions: Establishing policies and infrastructure to address privacy concerns.
Project Team
Dao Xuan-Quy
Researcher
Le Ngoc-Bich
Researcher
Phan Xuan-Dung
Researcher
Ngo Bac-Bien
Researcher
Vo The-Duy
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Dao Xuan-Quy, Le Ngoc-Bich, Phan Xuan-Dung, Ngo Bac-Bien, Vo The-Duy
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai