Human-like Nonverbal Behavior with MetaHumans in Real-World Interaction Studies: An Architecture Using Generative Methods and Motion Capture
Project Overview
The document explores the use of generative AI in education through the development of socially interactive agents (SIAs) that leverage motion capture technology to enhance human-machine interactions. By integrating nonverbal behaviors such as facial expressions and gestures, the system aims to facilitate more effective communication and engagement in educational settings. Utilizing Epic Games' MetaHuman, the proposed architecture supports various modules for natural language processing and user management, which are designed to create intuitive interactions between learners and AI. The findings underscore the potential of these technologies to transform educational experiences by making them more engaging and interactive, while also addressing challenges related to implementation and user experience. Overall, the research indicates that generative AI can significantly enhance educational practices, although careful consideration must be given to the practicalities of its application.
Key Applications
Socially Interactive Agents (SIAs) using MetaHuman
Context: Real-world interaction studies in educational settings, specifically in museums
Implementation: A distributed client-server architecture integrating motion capture and generative methods for nonverbal behavior
Outcomes: Enhanced user engagement and improved communication through realistic nonverbal interactions
Challenges: Computational latency, environmental adaptation, and maintaining coherence in interactions
Implementation Barriers
Technical barrier
Insufficient computational power can lead to delays and affect user experience.
Proposed Solutions: Utilizing consumer-grade hardware and optimizing the system architecture for low-latency communication.
Content coherence barrier
Discrepancies between the LLM responses and the knowledge base can confuse users.
Proposed Solutions: Meticulous prompt engineering to align LLM responses with the knowledge base and the agent's personality.
Project Team
Oliver Chojnowski
Researcher
Alexander Eberhard
Researcher
Michael Schiffmann
Researcher
Ana Müller
Researcher
Anja Richert
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Oliver Chojnowski, Alexander Eberhard, Michael Schiffmann, Ana Müller, Anja Richert
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai