ChatGPT or academic scientist? Distinguishing authorship with over 99% accuracy using off-the-shelf machine learning tools
Project Overview
The document explores the transformative role of generative AI, particularly ChatGPT, in education, highlighting its significant impact on academic writing. It underscores the importance of distinguishing between AI-generated and human-written content in higher education settings, where integrity and originality are crucial. To address this challenge, the authors have developed a robust model employing machine learning techniques that accurately classifies text authorship, achieving an impressive accuracy rate exceeding 99%. This innovative approach not only serves as a tool for detecting AI involvement in academic writing but also offers broader implications for enhancing educational practices and improving plagiarism detection methods. By integrating such technologies, educational institutions can better navigate the complexities introduced by generative AI, ensuring that the integrity of academic standards is maintained while also embracing the potential benefits of AI in educational contexts.
Key Applications
Model for discriminating ChatGPT-generated text from human academic writing
Context: Higher education, targeting academic scientists and educators
Implementation: Developed a supervised classification model using linguistic features to differentiate between human and AI text
Outcomes: Achieved over 99% accuracy in classifying authorship; reduced misclassification rates significantly compared to existing methods
Challenges: Adaptation of the model to various academic writing styles; ensuring the model remains effective across different disciplines
Implementation Barriers
Technical barrier
Challenges in adapting AI detection models to the unique styles of various academic disciplines
Proposed Solutions: Developing tailored models that account for specific linguistic features of different fields; expanding the dataset to include diverse writing styles
Educational barrier
Concerns about students using AI to complete assignments and the implications for academic integrity
Proposed Solutions: Implementing educational frameworks that guide appropriate AI use; enhancing student understanding of authorship and originality
Project Team
Heather Desaire
Researcher
Aleesa E. Chua
Researcher
Madeline Isom
Researcher
Romana Jarosova
Researcher
David Hua
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Heather Desaire, Aleesa E. Chua, Madeline Isom, Romana Jarosova, David Hua
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai