Exploring the Efficacy of ChatGPT in Analyzing Student Teamwork Feedback with an Existing Taxonomy
Project Overview
The document explores the role of generative AI, particularly ChatGPT, in enhancing educational practices by analyzing student feedback on teamwork. It emphasizes the increasing significance of teamwork skills in higher education and the challenges instructors face in effectively managing and interpreting feedback. ChatGPT's ability to categorize and analyze feedback with over 90% accuracy based on a predefined taxonomy positions it as a valuable resource for improving formative assessments and fostering clearer communication among students. Despite its promising applications, the document acknowledges certain limitations, such as ChatGPT's struggles with mathematical reasoning and potential biases stemming from its training data. It also highlights the importance of ethical considerations when implementing AI technologies in educational contexts. Overall, the findings suggest that generative AI can significantly contribute to understanding and enhancing collaborative learning experiences, while also underscoring the need for careful oversight in its application.
Key Applications
ChatGPT for analyzing student comments on teamwork feedback
Context: Higher education, specifically in undergraduate engineering courses
Implementation: ChatGPT was prompted to analyze student feedback comments by utilizing a taxonomy of positive and negative comments.
Outcomes: Achieved over 90% accuracy in labeling comments, providing instructors with a valuable tool for efficient feedback analysis.
Challenges: Struggles with ambiguous comments and potential biases in labeling, particularly with negative sentiments.
Implementation Barriers
Technical Limitations
ChatGPT may not provide meaningful feedback, accurately detect sophisticated plagiarism, and relies on external API calls, posing challenges for handling sensitive student data.
Proposed Solutions: Developing sets of competencies and literacies for teachers and students to understand the technology's limitations, and developing local versions of the model or privacy-preserving techniques to enhance accessibility.
Ethical Concerns
Concerns related to privacy, fairness, and transparency in the use of AI in education.
Proposed Solutions: Institutions need to adopt proactive measures to ensure responsible use of AI tools.
Implementation Challenges
Challenges arise from the reliance on external API calls for ChatGPT, particularly concerning the handling of sensitive student data.
Proposed Solutions: Developing local versions of the model or privacy-preserving techniques to enhance accessibility.
Project Team
Andrew Katz
Researcher
Siqing Wei
Researcher
Gaurav Nanda
Researcher
Christopher Brinton
Researcher
Matthew Ohland
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Andrew Katz, Siqing Wei, Gaurav Nanda, Christopher Brinton, Matthew Ohland
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai