Quantitative Analysis of AI-Generated Texts in Academic Research: A Study of AI Presence in Arxiv Submissions using AI Detection Tool
Project Overview
The document explores the transformative role of generative AI, particularly tools like ChatGPT, in education and academic research. It emphasizes the significant uptick in AI-generated content, especially in disciplines such as computer science, physics, and mathematics, as evidenced by the rising number of submissions to platforms like arXiv. A crucial aspect discussed is the challenge posed by this influx of AI-generated papers, which raises concerns about originality and academic integrity. To address these issues, the document highlights the utility of AI detection tools, specifically Originality.AI, which has demonstrated remarkable effectiveness in identifying non-original content, achieving over 98% accuracy in distinguishing AI-generated texts from human-written submissions. Overall, the findings underscore the need for robust verification mechanisms in academia to ensure the authenticity of scholarly work amidst the growing influence of generative AI technologies.
Key Applications
Originality.AI's AI detection tool
Context: The tool is utilized for analyzing arXiv submissions across various academic fields to detect AI-generated content, leveraging a dataset created from 13,000 arXiv papers.
Implementation: The detection tool was tested to determine its accuracy in differentiating between AI-generated and human-written texts, implementing a consistent methodology across multiple disciplines.
Outcomes: The tool demonstrated a high accuracy of over 98% in distinguishing between AI-generated and human-written texts.
Challenges: A key challenge remains in identifying AI-generated content in papers heavily reliant on numerical data and equations.
Implementation Barriers
Detection Challenges
Challenges in accurately detecting AI-generated content, especially in papers with extensive numerical data and equations.
Proposed Solutions: Continued development and refinement of detection tools to improve accuracy in various contexts.
Academic Integrity Concerns
Concerns about the originality and integrity of academic work due to increasing AI-generated papers.
Proposed Solutions: Implementation of strict guidelines and monitoring to ensure the authenticity of research submissions.
Project Team
Arslan Akram
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Arslan Akram
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai