Large Language Models Meet User Interfaces: The Case of Provisioning Feedback
Project Overview
The document examines the integration of Generative Artificial Intelligence (GenAI), specifically Large Language Models (LLMs), in the educational sector, emphasizing its potential to enhance educator productivity and enrich student learning experiences through conversational user interfaces (CUIs). Key applications include the development of tools like the Feedback Copilot, which aims to automate personalized feedback on student assignments, thereby improving engagement and supporting continuous assessment. However, the use of GenAI in education is accompanied by significant challenges, including the necessity for AI literacy among educators, ethical concerns surrounding bias and oversight, and the limitations of AI in managing complex tasks. The authors advocate for a framework that prioritizes user-centric applications to effectively harness GenAI's capabilities while mitigating these challenges, underscoring the importance of balancing innovation with ethical considerations in the deployment of AI tools in educational contexts.
Key Applications
Feedback and AI Literacy Copilot
Context: Higher education for university instructors in courses with high student enrollment and computer programming courses for undergraduate students, as well as K-12 education aimed at preparing students for AI technology in their future studies and careers.
Implementation: The Feedback and AI Literacy Copilot tool utilizes generative AI to generate personalized feedback for student assignments in computer science courses, as well as to create engaging learning materials for AI literacy curricula. It considers assessment tasks, sample solutions, student responses, and grading standards.
Outcomes: The tool improves the quality of feedback provided to students, enhancing constructiveness, empathy, and actionability. It also increases student awareness and understanding of AI technologies, fostering critical thinking and adaptability, while improving student engagement and understanding by providing tailored feedback.
Challenges: Challenges include the requirement for educators to possess a degree of AI literacy and prompt engineering skills, potential biases in feedback generation, dependence on accurate input data, the need for human oversight to ensure quality, resistance to curriculum changes, and ensuring equitable access to technology.
Implementation Barriers
Technical barrier
Educators may lack the necessary AI literacy and expertise in prompt engineering to effectively use CUIs for generating educational feedback. Additionally, there are challenges related to the integration of AI tools into existing educational systems, including interoperability and infrastructure.
Proposed Solutions: Develop user-centric applications that simplify the interaction with GenAI tools, providing step-by-step guidance and support for educators. Invest in IT infrastructure, ensure compatibility with existing systems, and provide training for educators.
Ethical barrier
Concerns over privacy, data security, and the ethical implications of using AI in high-stakes educational contexts such as grading, as well as concerns regarding data privacy, bias in AI algorithms, and the implications of AI-generated feedback.
Proposed Solutions: Implement guidelines and frameworks that prioritize ethical considerations, ensuring that AI tools are used responsibly and in a manner that aligns with educational values. Implement strict data governance policies, develop transparent AI models, and include diverse datasets in training.
Functional barrier
CUIs are designed for short interactions and struggle with complex, multi-step tasks, limiting their effectiveness in educational settings.
Proposed Solutions: Transition to custom-built applications that allow for more structured interactions and can handle complex educational tasks through improved user interfaces.
Project Team
Stanislav Pozdniakov
Researcher
Jonathan Brazil
Researcher
Solmaz Abdi
Researcher
Aneesha Bakharia
Researcher
Shazia Sadiq
Researcher
Dragan Gasevic
Researcher
Paul Denny
Researcher
Hassan Khosravi
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Stanislav Pozdniakov, Jonathan Brazil, Solmaz Abdi, Aneesha Bakharia, Shazia Sadiq, Dragan Gasevic, Paul Denny, Hassan Khosravi
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai