Leveraging Large Language Models for Actionable Course Evaluation Student Feedback to Lecturers
Project Overview
The document investigates the application of generative AI, particularly large language models (LLMs), in synthesizing student feedback from course evaluations within a Computer Science department. This innovative approach seeks to assist instructors by transforming vast amounts of qualitative feedback into actionable insights, thereby improving teaching practices. The findings indicate that AI can effectively generate useful and relevant feedback, highlighting its potential to enhance educational outcomes. However, challenges remain, including the model's limitations in managing contradictory responses and ensuring the accuracy of the synthesized information. Overall, the study underscores the promise of generative AI in education while acknowledging the need for further refinement to fully address these challenges.
Key Applications
Synthesis of student feedback from course evaluations using Llama2
Context: End-of-semester evaluations in large Computer Science classes, targeting lecturers seeking to improve teaching practices.
Implementation: Used zero-shot inference with Llama2 to summarize evaluations and generate actionable feedback based on the summaries. The model was prompted with concatenated student feedback.
Outcomes: Generated 75 feedback summaries for courses, providing actionable insights for instructors. Findings indicate potential for improving teaching practices and supporting educators' development.
Challenges: Struggled with synthesizing feedback from contradictory comments and ensuring the factual accuracy of generated suggestions. Some outputs included irrelevant information or failed to convey the strength of sentiment.
Implementation Barriers
Technical
The model occasionally generated irrelevant outputs or hallucinated new problems based on limited input data.
Proposed Solutions: Refinement of model training and prompting strategies to improve contextual understanding and output accuracy.
Pedagogical
The model faced difficulties providing specific suggestions relevant to the Computer Science domain due to a lack of pedagogical expertise.
Proposed Solutions: Integrating domain-specific training data into the model or collaborating with educational experts to enhance output relevance.
Project Team
Mike Zhang
Researcher
Euan D Lindsay
Researcher
Frederik Bode Thorbensen
Researcher
Danny Bøgsted Poulsen
Researcher
Johannes Bjerva
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Mike Zhang, Euan D Lindsay, Frederik Bode Thorbensen, Danny Bøgsted Poulsen, Johannes Bjerva
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai