Towards Anatomy Education with Generative AI-based Virtual Assistants in Immersive Virtual Reality Environments
Project Overview
The document explores the innovative application of generative AI within an immersive virtual reality (VR) environment designed for anatomy education, emphasizing its potential to transform medical training. By integrating AI-based virtual assistants, the study demonstrates significant advantages over traditional learning methods, particularly in fostering interactive engagement. A pilot study with 16 participants revealed that the use of avatar-based assistants notably improved performance in answering knowledge-based questions compared to screen-based configurations. Generative AI facilitates more natural interactions, offering personalized support that adapts to the unique learning needs of each student. Despite these promising outcomes, the document acknowledges challenges such as cognitive complexity and the necessity for further research into the accuracy of AI responses. Overall, the findings suggest that generative AI can enhance educational experiences in medical training, but further investigation is needed to optimize its effectiveness and address existing limitations.
Key Applications
Generative AI-based virtual assistant in VR for anatomy education
Context: Medical education, specifically for anatomy training among medical students
Implementation: Integration of generative AI services (ChatGPT) into a VR environment to create embodied virtual assistants that support interactive learning.
Outcomes: Participants showed improved performance in answering knowledge-based questions when using the avatar configuration. The environment provided insights into usability, cognitive task load, and sense of presence.
Challenges: The generative AI assistant struggled with analysis-based questions that required higher cognitive engagement and interpretation.
Implementation Barriers
Technical
Generative AI may not effectively handle complex analysis problems that require human-like reasoning.
Proposed Solutions: Future research should focus on improving the generative AI's ability to assist with higher cognitive tasks and ensuring the accuracy of the responses.
User Experience
Participants with varying levels of experience with VR and virtual assistants impacted interaction rates and learning effectiveness.
Proposed Solutions: Future studies should consider the participants' prior experience as a covariate or balance groups based on VR experience.
Project Team
Vuthea Chheang
Researcher
Shayla Sharmin
Researcher
Rommy Marquez-Hernandez
Researcher
Megha Patel
Researcher
Danush Rajasekaran
Researcher
Gavin Caulfield
Researcher
Behdokht Kiafar
Researcher
Jicheng Li
Researcher
Pinar Kullu
Researcher
Roghayeh Leila Barmaki
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Vuthea Chheang, Shayla Sharmin, Rommy Marquez-Hernandez, Megha Patel, Danush Rajasekaran, Gavin Caulfield, Behdokht Kiafar, Jicheng Li, Pinar Kullu, Roghayeh Leila Barmaki
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai