Planning with Vision-Language Models and a Use Case in Robot-Assisted Teaching
Project Overview
The document explores the innovative use of generative AI in education through the Image2PDDL framework, which leverages Vision-Language Models (VLMs) to automate the creation of Planning Domain Definition Language (PDDL) problems, thereby streamlining AI planning processes. A notable application of this framework is in robot-assisted teaching aimed at students with Autism Spectrum Disorder (ASD), where it demonstrates promising improvements in learning outcomes through enhanced interactions with robotic tutors. By minimizing the expertise needed for the generation of educational problems, the framework significantly enhances accessibility and scalability for complex educational tasks. Overall, the findings suggest that integrating generative AI into educational contexts can lead to more effective and personalized learning experiences, particularly for students with special needs, indicating a transformative potential for the future of education.
Key Applications
Image2PDDL framework
Context: Robot-assisted teaching for students with Autism Spectrum Disorder (ASD)
Implementation: Utilizes VLMs to convert images and text descriptions into PDDL problems, allowing robots to assist in structured tasks.
Outcomes: Improved accessibility and scalability in AI planning, demonstrated effective performance in educational tasks.
Challenges: Requires further research to verify the transfer of skills acquired during robot-assisted sessions to daily life; struggles with content correctness in complex scenarios.
Implementation Barriers
Technical challenge
Traditional approaches to PDDL generation require domain-specific knowledge and substantial manual effort.
Proposed Solutions: Image2PDDL reduces the need for domain expertise, allowing for automated generation of structured problems from visual and textual inputs.
Research limitation
Need for further research to validate the effectiveness of robot-assisted teaching in improving educational outcomes for students with ASD.
Proposed Solutions: Collaboration with ASD experts to identify effective strategies and conditions for robot-assisted learning.
Project Team
Xuzhe Dang
Researcher
Lada Kudláčková
Researcher
Stefan Edelkamp
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Xuzhe Dang, Lada Kudláčková, Stefan Edelkamp
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai