An Eye for an AI: Evaluating GPT-4o's Visual Perception Skills and Geometric Reasoning Skills Using Computer Graphics Questions
Project Overview
The document examines the role of generative AI, specifically the GPT-4o model, in enhancing education through its application in solving Computer Graphics (CG) assessment questions. It showcases the model's potential to support learning by improving insights into visual perception and geometric reasoning skills. Although GPT-4o demonstrates improved performance over its predecessor, GPT-4, it still encounters notable challenges, particularly in accurately addressing CG tasks that require visual inputs. The findings underscore the importance of effective prompting techniques and the critical evaluation of AI-generated solutions to maximize the benefits of generative AI in educational contexts. The study advocates for innovative strategies to integrate AI into CG education, aiming to leverage its capabilities while acknowledging its limitations. Overall, the document highlights the transformative potential of generative AI in education while calling for careful implementation and assessment to ensure effective learning outcomes.
Key Applications
GPT-4o (Large Multimodal Model)
Context: Educational context for Computer Graphics students, targeting undergraduate courses.
Implementation: The study involved constructing datasets of CG questions and evaluating GPT-4o's responses to these questions using both text and multimodal inputs.
Outcomes: GPT-4o achieved 50.1% accuracy across CG questions, showing improved performance over its predecessor, particularly in multiple-choice questions.
Challenges: GPT-4o struggled with visual processing and geometric reasoning tasks, showing inconsistency and reliability issues, especially with image-based questions.
Implementation Barriers
Technical Limitation
GPT-4o's performance is inconsistent, particularly in visual and geometric reasoning tasks, limiting its reliability for complex CG questions.
Proposed Solutions: Educators can generate human-written image descriptions to assist the model and encourage students to critically evaluate AI-generated solutions.
Educational Challenge
Students may misuse AI tools like GPT-4o due to its inaccuracies, leading to a false sense of understanding.
Proposed Solutions: Incorporate exercises that involve evaluating incorrect AI solutions to enhance critical thinking and understanding.
Project Team
Tony Haoran Feng
Researcher
Paul Denny
Researcher
Burkhard C. Wünsche
Researcher
Andrew Luxton-Reilly
Researcher
Jacqueline Whalley
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Tony Haoran Feng, Paul Denny, Burkhard C. Wünsche, Andrew Luxton-Reilly, Jacqueline Whalley
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai