The Imitation Game: Detecting Human and AI-Generated Texts in the Era of ChatGPT and BARD
Project Overview
The document examines the impact of generative AI, particularly large language models (LLMs) such as GPT and BARD, on the education sector, emphasizing both its applications and challenges. It raises significant concerns regarding academic integrity, as students may misuse AI-generated content for their assignments. To address these issues, the research presents a novel dataset comprising both human-written and AI-generated texts across diverse genres, assessing various machine learning models' effectiveness in differentiating between the two. The findings reveal that while LLMs can generate text that closely mimics human writing, there are viable detection methods available to identify AI-produced content. This capability is essential for preserving academic standards and integrity in educational environments. Overall, the document underscores the dual-edged nature of generative AI in education, highlighting its potential benefits while also advocating for strategies to mitigate risks associated with its misuse.
Key Applications
Detection of AI-generated vs human-written texts
Context: Educational institutions and contexts where academic integrity is a concern
Implementation: Utilization of machine learning models to classify texts from a novel dataset comprising human and AI-generated texts
Outcomes: Effective differentiation between human and LLM-generated texts, improving academic integrity checks
Challenges: Difficulty in detecting AI-generated texts, especially from GPT in creative writing tasks
Implementation Barriers
Ethical
Concerns over academic integrity due to students potentially using AI tools to generate assignments.
Proposed Solutions: Development of effective detection tools and discussions on ethical guidelines for AI usage in education.
Technical
Challenges in accurately differentiating between human-generated and AI-generated texts, particularly with sophisticated models.
Proposed Solutions: Continued research and refinement of machine learning models to enhance detection capabilities.
Project Team
Kadhim Hayawi
Researcher
Sakib Shahriar
Researcher
Sujith Samuel Mathew
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Kadhim Hayawi, Sakib Shahriar, Sujith Samuel Mathew
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai