Detecting AI-Generated Text in Educational Content: Leveraging Machine Learning and Explainable AI for Academic Integrity
Project Overview
The document explores the integration of generative AI in education, particularly emphasizing the development of tools aimed at detecting AI-generated content in student submissions to uphold academic integrity. It introduces the CyberHumanAI dataset, which comprises both human-written and AI-generated texts in the field of cybersecurity, and assesses various machine learning algorithms to determine their efficacy in accurately identifying the source of the text. The study reveals that traditional machine learning models, specifically XGBoost and Random Forest, demonstrate significant success in differentiating between human and AI-generated content. These findings underscore the growing need for ethical standards in educational settings as the use of generative AI tools becomes more widespread, highlighting the dual challenge of leveraging AI for educational advancement while also safeguarding against potential misuse in academic work.
Key Applications
Detection of AI-Generated Text in Educational Content
Context: Educational settings, particularly for detecting AI-generated student submissions
Implementation: Utilization of the CyberHumanAI dataset to train and evaluate ML models for classification of text
Outcomes: Achieved high accuracy in detecting AI-generated content (e.g., XGBoost and Random Forest with 83% and 81% accuracy respectively)
Challenges: Classifying shorter texts (e.g., paragraphs) is more challenging than longer articles; potential for misclassification and reliance on generative AI by students
Implementation Barriers
Technological
Challenges in accurately detecting AI-generated content due to the complexities of language models, varying text lengths, and the need for more fine-tuned models.
Proposed Solutions: Developing more fine-tuned and specific ML models, improving datasets for training.
Ethical
Concerns about academic integrity and plagiarism as students may rely heavily on AI tools for assignments.
Proposed Solutions: Implementing detection tools to ensure accountability and promoting original thought in student work.
Project Team
Ayat A. Najjar
Researcher
Huthaifa I. Ashqar
Researcher
Omar A. Darwish
Researcher
Eman Hammad
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Ayat A. Najjar, Huthaifa I. Ashqar, Omar A. Darwish, Eman Hammad
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai