Can AI Chatbots Pass the Fundamentals of Engineering (FE) and Principles and Practice of Engineering (PE) Structural Exams?
Project Overview
The document examines the role of generative AI, particularly AI chatbots like ChatGPT-4 and Google Bard, in the context of engineering education, specifically their performance on the Fundamentals of Engineering (FE) and Principles and Practice of Engineering (PE) exams. The study indicates that ChatGPT-4 performed well enough to potentially pass the FE exam, showcasing its capability as a supportive educational tool, while both chatbots faced considerable challenges with the more complex PE exam. These findings highlight the potential of generative AI to function as teaching assistants, providing valuable support to engineering students in their studies. The exploration of AI's applications in education underscores its promise in enhancing learning experiences and outcomes, suggesting that while there are limitations, there is a significant opportunity for AI to augment educational processes in technical fields.
Key Applications
ChatGPT-4 and Google Bard as chatbot technologies for exam preparation and assistance.
Context: Engineering education, targeting students preparing for the FE and PE exams.
Implementation: Chatbots were tested with a range of civil and environmental engineering questions drawn from official practice exams.
Outcomes: ChatGPT-4 scored 70.9% on the FE exam and demonstrated potential to pass; Bard scored lower at 39.2%. For the PE exam, both chatbots scored below 50%.
Challenges: Chatbots faced difficulties with complex engineering concepts and visual-based questions, leading to incorrect answers despite the right processes being followed.
Implementation Barriers
Technical barrier
Chatbots struggle with understanding complex engineering scenarios and visual questions due to limitations in their current capabilities.
Proposed Solutions: Future versions of chatbots are expected to improve their capabilities and understanding of intricate engineering problems.
Content limitation
Chatbots lack access to external resources and data necessary for certain engineering questions, which limits their effectiveness.
Proposed Solutions: Integrating better databases and access to relevant engineering materials in future chatbot designs.
Project Team
M. Z. Naser
Researcher
Brandon Ross
Researcher
Jennier Ogle
Researcher
Venkatesh Kodur
Researcher
Rami Hawileh
Researcher
Jamal Abdalla
Researcher
Huu-Tai Thai
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: M. Z. Naser, Brandon Ross, Jennier Ogle, Venkatesh Kodur, Rami Hawileh, Jamal Abdalla, Huu-Tai Thai
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai