Knowledge Distillation of LLM for Automatic Scoring of Science Education Assessments
Project Overview
The document explores the use of knowledge distillation (KD) in enhancing the automatic scoring of science assessments through Large Language Models (LLMs) in educational settings. By developing smaller and more efficient student models derived from larger teacher models, the KD method aims to optimize the deployment of AI technologies in resource-constrained environments, thus improving accessibility. The findings indicate that this approach significantly boosts scoring accuracy and efficiency, showcasing the potential of generative AI to transform educational assessment practices. However, the document also addresses challenges related to model performance and the limitations of resources in traditional educational contexts, emphasizing the need for continued innovation to fully leverage AI's capabilities in education. Overall, the study highlights the promising intersection of AI and education, pointing towards a future where advanced assessment tools can be effectively integrated into various learning environments.
Key Applications
Knowledge Distillation of LLMs for Automatic Scoring
Context: Automatic scoring of science assessments for high school students using resource-constrained devices like tablets and laptops.
Implementation: A teacher model (fine-tuned LLM) is used to generate soft labels for training a smaller student model. The student model is trained to replicate the teacher model's predictions while being more efficient.
Outcomes: The student model achieves scoring accuracy comparable to the teacher model while being significantly smaller and faster, making it feasible for use in typical educational settings.
Challenges: The student models often do not reach the performance benchmarks of the teacher models due to their simpler architecture and fewer training parameters.
Implementation Barriers
Technical Barrier
The considerable size and computational requirements of LLMs restrict their deployment in resource-constrained educational environments.
Proposed Solutions: Using knowledge distillation to create smaller, more efficient models that can be deployed on devices with limited processing power.
Project Team
Ehsan Latif
Researcher
Luyang Fang
Researcher
Ping Ma
Researcher
Xiaoming Zhai
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Ehsan Latif, Luyang Fang, Ping Ma, Xiaoming Zhai
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai