Effects of a Prompt Engineering Intervention on Undergraduate Students' AI Self-Efficacy, AI Knowledge and Prompt Engineering Ability: A Mixed Methods Study
Project Overview
The document explores the integration of generative AI in education, specifically through a prompt engineering intervention implemented in a history course for undergraduate students at a Hong Kong university. It analyzes the intervention's effects on students' AI self-efficacy, knowledge, and proficiency in crafting effective prompts for large language models (LLMs) such as ChatGPT. The study revealed significant enhancements in students' confidence in using AI tools and a deeper comprehension of AI concepts, along with a marked improvement in their ability to create effective prompts. These findings underscore the critical role of structured prompt engineering education in fostering AI literacy, suggesting that such interventions can empower students with the necessary skills to effectively utilize generative AI in their academic pursuits.
Key Applications
Prompt engineering workshop for using ChatGPT to assist with academic writing tasks
Context: Higher education, specifically undergraduate students enrolled in a history course at a university in Hong Kong
Implementation: A 100-minute workshop where students learned prompt engineering strategies and applied them to plan their final essay
Outcomes: Increased AI self-efficacy, enhanced understanding of AI concepts, and improved skills in crafting effective prompts
Challenges: Students' initial low confidence in using AI tools and concerns about ethical implications of AI use in academic writing
Implementation Barriers
Educational
Limited understanding of how to effectively teach prompt engineering to students
Proposed Solutions: Structured interventions based on research-backed strategies and frameworks for teaching prompt engineering
Ethical
Concerns about academic integrity and the appropriate use of AI in writing tasks, along with the need for discussions on ethical AI use
Proposed Solutions: Incorporate discussions on ethical AI use into the curriculum and prompt engineering workshops
Project Team
David James Woo
Researcher
Deliang Wang
Researcher
Tim Yung
Researcher
Kai Guo
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: David James Woo, Deliang Wang, Tim Yung, Kai Guo
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai