Using Large Multimodal Models to Extract Knowledge Components for Knowledge Tracing from Multimedia Question Information

Knowledge Component Extraction and Zero-Shot Knowledge Tracing

Context: Educational settings utilizing Intelligent Tutoring Systems and adaptive learning environments where prior student interaction data is not available, applicable to various subjects including AI and general education.

Implementation: Automated extraction of Knowledge Components (KCs) from educational content and generation of KCs using instruction-tuned large language models (specifically GPT-4o API). This includes clustering similar components and facilitating knowledge tracing without prior student interaction records.

Outcomes: Improved performance in Knowledge Tracing models, comparable or superior to human-generated KCs, addressing cold-start issues for users and items, enabling adaptive learning without extensive data histories, and enhancing explainability in assessments.

Challenges: Limitations include potential data loss during preprocessing, the need for more refined KT methodologies to assess detailed KCs, over-reliance on statistical data, biases from content difficulty, and limitations of current KT models.

Technical Barrier

Data loss during preprocessing can hinder the effectiveness of the LMM-generated KCs. Improving data handling protocols and ensuring comprehensive content extraction during preprocessing steps is essential.

Proposed Solutions: Improving data handling protocols and ensuring comprehensive content extraction during preprocessing steps.

Implementation Barrier

Current Knowledge Tracing methodologies are not advanced enough to evaluate the effectiveness of multiple KCs per problem. Refining KT methodologies to better assess detailed knowledge components is necessary to improve model performance.

Proposed Solutions: Refining KT methodologies to better assess detailed knowledge components and improve model performance.

Adaptability Barrier

Existing KT models heavily rely on historical problem-solving data, making them less adaptable to new educational contexts. Developing Zero-Shot KT techniques is crucial to enable models to function effectively without prior records.

Proposed Solutions: Developing Zero-Shot KT techniques to enable models to function effectively without prior records.

Project Contact: Dr. Jianhua Yang

LLM Model Version: gpt-4o-mini-2024-07-18

Analysis Provider: Openai