WhatsAI: Transforming Meta Ray-Bans into an Extensible Generative AI Platform for Accessibility
Project Overview
The document explores the role of generative AI in education, particularly through its application in wearable devices like Meta Ray-Bans, aimed at enhancing accessibility for blind and visually impaired users. It introduces the WhatsAI platform, an extensible framework that empowers users to create tailored visual accessibility technologies, underscoring the significance of community-driven innovation. This platform facilitates real-time visual assistance tasks, such as scene description and object detection, allowing for personalized educational experiences. However, while the technology demonstrates considerable potential in improving accessibility, it also encounters challenges, including a closed ecosystem that hampers broader innovation and the necessity for enhanced computational resources to boost performance. Overall, the integration of generative AI in education through such tools represents a promising frontier, addressing specific needs and fostering inclusivity, albeit with hurdles that must be addressed for optimal efficacy.
Key Applications
WhatsAI - an extensible AI platform for visual accessibility
Context: Designed for blind and visually impaired individuals, aiming to enhance accessibility in daily tasks.
Implementation: Integrates with Meta Ray-Bans and WhatsApp for real-time visual assistance; users initiate calls to access AI assistance.
Outcomes: Provides hands-free operation, real-time scene description, object detection, and OCR; significantly improves task completion rates for users.
Challenges: Closed nature of current platforms limits innovation; reliance on computational resources affects performance.
Implementation Barriers
Technical barrier
The proprietary nature of existing platforms restricts innovation in visual accessibility technologies.
Proposed Solutions: Develop open APIs and extensible frameworks that allow community contributions and customization.
Resource barrier
Performance of AI features relies on available computational resources, which can limit responsiveness and accuracy.
Proposed Solutions: Investigate edge AI deployment to reduce latency and enhance privacy.
Project Team
Nasif Zaman
Researcher
Venkatesh Potluri
Researcher
Brandon Biggs
Researcher
James M. Coughlan
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Nasif Zaman, Venkatesh Potluri, Brandon Biggs, James M. Coughlan
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai