Enhancing Higher Education with Generative AI: A Multimodal Approach for Personalised Learning
Project Overview
The document examines the role of Generative AI (GenAI) in higher education, focusing on a multimodal chatbot that enhances personalized learning experiences. This innovative chatbot employs the ChatGPT API for text-based interactions and integrates Google Bard for image analysis and converting diagrams into code, thereby addressing a wide range of educational queries. Additionally, it incorporates sentiment analysis to provide insights into student feedback, highlighting the necessity of responsive and interactive educational tools. The research underscores the significance of multimodal AI in fostering more engaging learning environments and stresses the importance of equipping both students and educators with effective technological support. By demonstrating the capabilities and applications of GenAI in education, the document advocates for its broader adoption to improve learning outcomes and enhance educational experiences.
Key Applications
Multimodal chatbot with diagram-to-code conversion
Context: Higher education, specifically for undergraduate courses in various STEM disciplines, including computer science and engineering.
Implementation: Developed a multimodal chatbot utilizing the ChatGPT API for text analysis and Google Bard for image analysis, with a specific feature for converting diagrams to code. This implementation includes capabilities for handling diverse file inputs and analyzing student feedback.
Outcomes: Enhanced personalized learning experiences, improved student engagement, facilitated better understanding of graphical representations and code implementations, and provided deep insights into student feedback and course evaluations.
Challenges: Encountered technical challenges in the diagram-to-code conversion process, including accurate image recognition and managing various diagram formats, as well as interpreting diagrams and converting them to standardized code.
Implementation Barriers
Technical barrier
Challenges in converting diagrams to code due to the need for precise image recognition and standardization across different diagram formats
Proposed Solutions: Developing advanced image recognition technologies and algorithms capable of processing diverse styles and formats
Implementation barrier
Absence of file input functionality in existing educational tools limits personalized learning and feedback analysis
Proposed Solutions: Integrating file input capabilities in the chatbot to allow students and educators to upload course materials and receive customized analysis
Project Team
Johnny Chan
Researcher
Yuming Li
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Johnny Chan, Yuming Li
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18