HowkGPT: Investigating the Detection of ChatGPT-generated University Student Homework through Context-Aware Perplexity Analysis
Project Overview
The document explores the implications of generative AI, particularly focusing on HowkGPT, an innovative tool created to detect AI-generated content in university student homework assignments, specifically those produced by ChatGPT. By utilizing a dataset of academic assignments and analyzing perplexity scores, HowkGPT effectively differentiates between texts authored by students and those generated by AI, thereby addressing critical concerns surrounding academic integrity. The findings underscore the challenges that AI-generated content presents in educational environments and highlight the necessity for specialized detection algorithms that cater to the unique characteristics of various academic assignments. Overall, the document emphasizes the growing role of generative AI in education, the importance of maintaining academic honesty, and the need for robust tools to navigate the evolving landscape of student work in the age of AI.
Key Applications
HowkGPT
Context: Detection of AI-generated homework in university settings, targeting educators and academic integrity advocates.
Implementation: The tool computes perplexity scores for submitted homework using a dataset of academic assignments and categorized metadata.
Outcomes: Improved accuracy in distinguishing between student-authored and AI-generated texts, promoting fair grading.
Challenges: Difficulty in detecting AI-generated content due to the structured nature of academic assignments and the need for context-aware algorithms.
Implementation Barriers
Technical barrier
The need for refined algorithms to distinguish AI-generated homework from student work due to the contextual nature of academic assignments.
Proposed Solutions: Developing context-aware detection methods and utilizing metadata for more precise categorization.
Ethical barrier
Concerns regarding academic integrity and the potential misuse of AI-generated content by students. This includes the need to promote understanding of academic integrity and the proper use of AI tools.
Proposed Solutions: Implementing educational policies that address these concerns and promote the understanding of academic integrity.
Project Team
Christoforos Vasilatos
Researcher
Manaar Alam
Researcher
Talal Rahwan
Researcher
Yasir Zaki
Researcher
Michail Maniatakos
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Christoforos Vasilatos, Manaar Alam, Talal Rahwan, Yasir Zaki, Michail Maniatakos
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai