Teaching-Assistant-in-the-Loop: Improving Knowledge Distillation from Imperfect Teacher Models in Low-Budget Scenarios
Project Overview
The document explores the innovative application of generative AI in education through a three-component framework that leverages large language models (LLMs) for knowledge distillation. It highlights the challenges associated with the high costs and imperfections of teacher models, proposing the integration of a Teaching Assistant (TA) model designed to filter and evaluate the quality of annotations, which significantly enhances sample efficiency. The findings indicate that this approach leads to notable performance improvements across various reasoning tasks, achieved through a structured two-stage training process. By effectively utilizing generative AI, the framework aims to optimize the learning experience, streamline educational content delivery, and ultimately foster better educational outcomes. This integration of AI not only addresses existing limitations in educational models but also paves the way for more effective teaching strategies that can adapt to diverse learning needs.
Key Applications
Three-component KD framework for LLMs
Context: Educational settings that utilize knowledge distillation for training smaller models from larger language models.
Implementation: A three-component framework that includes a student model, a teacher model, and a TA model to evaluate and filter annotations.
Outcomes: Achieved performance improvements up to 20.79% across datasets when compared to traditional fine-tuning methods.
Challenges: High costs associated with querying large teacher models and the quality of outputs from these models may not always be reliable.
Implementation Barriers
Cost and Quality Barrier
The high cost of querying large language models like GPT-4 for generating sufficient training data, combined with the potential for low-quality outputs from teacher models, which can negatively impact student learning.
Proposed Solutions: Utilizing a Teaching Assistant model to reduce costs by filtering and assessing annotations, ensuring only high-quality examples are included in training.
Project Team
Yuhang Zhou
Researcher
Wei Ai
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Yuhang Zhou, Wei Ai
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai