AiGet: Transforming Everyday Moments into Hidden Knowledge Discovery with AI Assistance on Smart Glasses

AiGet - a proactive AI assistant integrated with AR smart glasses

Context: Used in informal learning contexts such as casual walking, shopping, and museum exhibitions by university students, professionals, and staff, facilitating learning during daily activities.

Implementation: AiGet analyzes real-time user gaze patterns and environmental context while leveraging large language models to deliver personalized, relevant knowledge tailored to the user's immediate surroundings.

Outcomes: Enhances enjoyment of primary tasks, stimulates curiosity, improves decision-making, and fosters deeper engagement with the environment. Users reported increased awareness and appreciation of their surroundings while minimizing cognitive overload.

Challenges: Potential for information overload if not designed carefully; challenges in identifying what knowledge is most valuable to users in everyday settings; need for careful design to prevent biases in the information presented.

Cognitive Barrier

Inattentional blindness can prevent users from noticing interesting elements in their environment due to cognitive constraints. Additionally, users may experience cognitive overload from too much information or unexpected knowledge.

Proposed Solutions: AiGet aims to analyze user gaze behavior to predict learning desires and provide relevant knowledge about unseen entities, while also implementing adaptive mechanisms that balance knowledge diversity and depth based on user context.

Motivational Barrier

Time and attention limitations lead users to miss out on learning opportunities as they focus on primary tasks.

Proposed Solutions: AiGet is designed to provide timely knowledge with minimal effort from users, embedding learning opportunities into daily tasks.

Knowledge Relevance Barrier

Users may overestimate their understanding of familiar entities, leading to missed learning opportunities.

Proposed Solutions: AiGet prioritizes delivering knowledge that balances novelty and usefulness based on user interests.

Distraction Barrier

Proactive knowledge delivery could disrupt primary tasks if not designed to minimize cognitive load.

Proposed Solutions: AiGet employs a multimodal output design to present knowledge in concise formats to reduce cognitive load.

AI Biases

AI-generated content risks presenting biased or one-sided information.

Proposed Solutions: Design systems to present multiple perspectives and understand nuanced user preferences.

Trust Issues

Users may distrust unexpected information due to past negative experiences with AI.

Proposed Solutions: Incorporate a scoring system to measure information credibility and allow users to verify sources.

Project Contact: Dr. Jianhua Yang

LLM Model Version: gpt-4o-mini-2024-07-18

Analysis Provider: Openai