SimpsonsVQA: Enhancing Inquiry-Based Learning with a Tailored Dataset
Project Overview
The document highlights the innovative use of generative AI in education through the creation of 'SimpsonsVQA', a specialized visual question answering (VQA) dataset derived from The Simpsons TV show. This dataset is tailored to foster inquiry-based learning, particularly benefiting early-age learners and individuals with cognitive impairments. It encompasses three primary tasks: conventional VQA, question relevance assessment, and answer correctness evaluation, effectively addressing diverse educational challenges. With around 23,000 images and 166,000 question-answer pairs, SimpsonsVQA is designed to enhance AI model performance on cartoon images, thereby facilitating a more engaging and interactive learning experience. The findings suggest that such targeted applications of generative AI can significantly improve educational outcomes by providing personalized and contextually relevant support to learners, making it a valuable tool in modern educational practices.
Key Applications
SimpsonsVQA
Context: Educational context focusing on inquiry-based learning for early-age education and individuals with cognitive disabilities.
Implementation: Created using a combination of automated and manual processes, including image collection from The Simpsons, image captioning, and generating question-answer pairs using ChatGPT, followed by manual evaluation.
Outcomes: Enhances interactive learning experiences, improves the ability of AI systems to handle diverse visual questions, and supports assessment of answer correctness.
Challenges: Existing models face difficulties with cartoon images and the dataset may contain irrelevant questions or incorrect answers that do not align with typical human errors.
Implementation Barriers
Technical Challenge
AI models, especially large vision-language models (LVLMs), struggle to process cartoon images effectively due to their training on photorealistic datasets.
Proposed Solutions: Incorporating training on datasets like SimpsonsVQA to improve model adaptability and performance on non-photorealistic images.
Content Quality
The automatically generated irrelevant questions and incorrect answers may introduce a domain gap compared to human learners’ errors.
Proposed Solutions: Conducting human studies to better align the dataset with real learner behaviors.
Project Team
Ngoc Dung Huynh
Researcher
Mohamed Reda Bouadjenek
Researcher
Sunil Aryal
Researcher
Imran Razzak
Researcher
Hakim Hacid
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Ngoc Dung Huynh, Mohamed Reda Bouadjenek, Sunil Aryal, Imran Razzak, Hakim Hacid
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai