Training program on sign language: social inclusion through Virtual Reality in ISENSE project
Project Overview
The document outlines the ISENSE project, which leverages Virtual Reality (VR) and Artificial Intelligence (AI) to improve social inclusion for deaf students by facilitating the learning of sign language. This initiative focuses on developing an application capable of recognizing Spanish and Italian sign languages, thereby enhancing communication between deaf individuals and the hearing community. The application provides an immersive VR environment where users can learn, test, and contribute new signs, ultimately aiming to boost retention, memory, and engagement in the learning experience. A significant innovation of the project is the integration of AI for real-time sign recognition within these VR settings, which enhances the interactivity and effectiveness of the educational process. Overall, the ISENSE project exemplifies how generative AI can be applied in educational contexts to foster inclusivity and improve learning outcomes for underrepresented groups.
Key Applications
VR and AI-based recognition system for sign language
Context: Educational context for deaf students and the hearing community, including students, professors, and family members
Implementation: Developed as part of the ISENSE project, utilizing Oculus Meta Quest 2 for hand tracking and real-time sign recognition.
Outcomes: Enhanced learning experience for sign language, improved communication for deaf individuals, and increased accessibility in academic settings.
Challenges: Dependence on the Oculus device for accurate hand tracking; environmental constraints affecting performance.
Implementation Barriers
Technical barrier
Reliance on the Oculus device, which requires a clear view of the user's hands, limiting testing environments.
Proposed Solutions: Future work aims to improve recognition capabilities and explore alternative tracking methods.
Project Team
Alessia Bisio
Researcher
Enrique Yeguas-Bolívar
Researcher
Pilar Aparicio-Martínez
Researcher
María Dolores Redel-Macías
Researcher
Sara Pinzi
Researcher
Stefano Rossi
Researcher
Juri Taborri
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Alessia Bisio, Enrique Yeguas-Bolívar, Pilar Aparicio-Martínez, María Dolores Redel-Macías, Sara Pinzi, Stefano Rossi, Juri Taborri
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai