Enhancing Mathematics Learning for Hard-of-Hearing Students Through Real-Time Palestinian Sign Language Recognition: A New Dataset
Project Overview
The document outlines a study focused on improving mathematics education for hard-of-hearing students through the creation of a Palestinian sign language (PSL) recognition system. A specialized dataset comprising mathematical gestures was developed, and a Vision Transformer model was effectively fine-tuned for gesture classification, achieving high accuracy. This initiative underscores the transformative potential of generative AI applications in education, particularly in enhancing accessibility and learning outcomes for marginalized groups. By leveraging AI-driven solutions, the project demonstrates how technology can bridge communication gaps in specialized learning environments, ultimately fostering inclusivity and supporting diverse educational needs. The findings suggest that such innovations can significantly benefit students with hearing impairments, thereby enriching the educational landscape and promoting equitable access to learning resources.
Key Applications
Palestinian Sign Language Recognition System using Vision Transformer
Context: Enhancing mathematics education accessibility for hard-of-hearing students
Implementation: Developed a custom dataset of 41 gesture classes representing mathematical concepts and trained a Vision Transformer model on it.
Outcomes: Achieved an accuracy of 97.59% in recognizing mathematical signs, improving accessibility and comprehension for hard-of-hearing students.
Challenges: Limited availability of digital resources for Palestinian sign language, high computational power required for real-time deployment.
Implementation Barriers
Resource limitations
Scarcity of digital resources for Palestinian sign language (PSL) and lack of academic attention, particularly in developing a specialized dataset for PSL that focuses on mathematical gestures to address the gap.
Proposed Solutions: Developing a specialized dataset for PSL that focuses on mathematical gestures to address the gap.
Technical limitations
Heavy reliance on hand gesture recognition neglects facial expressions and body movements crucial for accurate PSL interpretation. Future work should explore expanding recognition capabilities to include dynamic sign context and additional gesture features.
Proposed Solutions: Future work should explore expanding recognition capabilities to include dynamic sign context and additional gesture features.
Computational requirements
Deep learning models often require high computational power, making real-time deployment impractical in low-resource environments. Utilizing platforms like Google Colab Pro with advanced GPUs can facilitate model training and experimentation.
Proposed Solutions: Utilizing platforms like Google Colab Pro with advanced GPUs to facilitate model training and experimentation.
Project Team
Fidaa khandaqji
Researcher
Huthaifa I. Ashqar
Researcher
Abdelrahem Atawnih
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Fidaa khandaqji, Huthaifa I. Ashqar, Abdelrahem Atawnih
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai