Understanding Learner-LLM Chatbot Interactions and the Impact of Prompting Guidelines
Project Overview
The document explores the role of generative AI, particularly Large Language Models (LLMs), in education, highlighting the significance of effective prompting strategies to optimize user interactions with AI chatbots. It identifies the difficulties users encounter in formulating effective prompts and proposes structured prompting guidelines designed to enhance the quality of AI-generated responses and improve overall user experience. The study assesses how these guidelines influence user behavior and aims to bolster AI literacy among users in educational settings. By focusing on these strategies, the document underscores the potential of generative AI to enrich educational practices, facilitate better communication between students and AI tools, and ultimately enhance learning outcomes through improved engagement and understanding of AI capabilities.
Key Applications
Understanding Learner-LLM Chatbot Interactions and the Impact of Prompting Guidelines
Context: An educational experiment involving participants trained to use LLMs for an online learning activity about social media threats.
Implementation: Participants were trained using three different prompting strategies and then engaged in interactions with ChatGPT.
Outcomes: Improvements in crafting successful prompts and enhanced overall quality of conversations with the chatbot.
Challenges: Users struggled with formulating effective prompts; the effectiveness of different prompting strategies did not significantly differ.
Implementation Barriers
Usability Barrier
Users often struggle to create clear and effective prompts for LLM interactions.
Proposed Solutions: Structured prompting guidelines and training to improve user competency in AI interactions.
Research Gap
Limited research on specific educational strategies for improving prompting skills.
Proposed Solutions: Integrating frameworks like scaffolding and authentic learning to develop effective educational strategies.
Project Team
Cansu Koyuturk
Researcher
Emily Theophilou
Researcher
Sabrina Patania
Researcher
Gregor Donabauer
Researcher
Andrea Martinenghi
Researcher
Chiara Antico
Researcher
Alessia Telari
Researcher
Alessia Testa
Researcher
Sathya Bursic
Researcher
Franca Garzotto
Researcher
Davinia Hernandez-Leo
Researcher
Udo Kruschwitz
Researcher
Davide Taibi
Researcher
Simona Amenta
Researcher
Martin Ruskov
Researcher
Dimitri Ognibene
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Cansu Koyuturk, Emily Theophilou, Sabrina Patania, Gregor Donabauer, Andrea Martinenghi, Chiara Antico, Alessia Telari, Alessia Testa, Sathya Bursic, Franca Garzotto, Davinia Hernandez-Leo, Udo Kruschwitz, Davide Taibi, Simona Amenta, Martin Ruskov, Dimitri Ognibene
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai