'Quis custodiet ipsos custodes?' Who will watch the watchmen? On Detecting AI-generated peer-reviews
Project Overview
The document examines the integration of generative AI in education, particularly focusing on its applications in academic publishing, specifically peer reviews. It highlights the challenges posed by the rise of AI-generated content, noting the potential threats to the integrity and quality of the peer-review process. To address these challenges, two models are introduced: the Token Frequency (TF) model and the Review Regeneration (RR) model, both designed to effectively detect AI-generated peer reviews, such as those produced by tools like ChatGPT. The study underscores the necessity of ensuring that the peer-review process remains credible and reliable in the face of increasing AI usage, emphasizing the need for methodologies that can differentiate between human and AI-generated reviews. Overall, the document sheds light on the implications of generative AI in maintaining review quality and the ongoing discourse surrounding its role in academic integrity within educational settings.
Key Applications
Token Frequency (TF) model and Review Regeneration (RR) model for detecting AI-generated peer reviews
Context: Academic publishing, specifically in peer review processes for conferences like ICLR and NeurIPS. Target audience includes editors and reviewers.
Implementation: The TF model uses token frequency analysis to differentiate between AI-generated and human-written reviews, whereas the RR model assesses similarity between embeddings of AI-generated and human-generated reviews.
Outcomes: Both models outperformed existing AI text detectors, with specific improvements noted in the detection of AI-generated peer reviews.
Challenges: The models face challenges from text paraphrasing and synonym replacement attacks that can obscure the AI-generated nature of reviews.
Implementation Barriers
Technical barrier
The AI detection models are vulnerable to attacks such as paraphrasing and token substitution, which can make AI-generated content appear more like human-written text.
Proposed Solutions: The introduction of a paraphrasing defense mechanism that reverts paraphrased reviews to a state resembling their original AI-generated form.
Ethical and policy barrier
There are concerns regarding the integrity and confidentiality of the peer review process when using AI tools.
Proposed Solutions: Establishing clear policies on the acceptable use of AI tools in peer reviews, emphasizing the necessity for human oversight.
Project Team
Sandeep Kumar
Researcher
Mohit Sahu
Researcher
Vardhan Gacche
Researcher
Tirthankar Ghosal
Researcher
Asif Ekbal
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Sandeep Kumar, Mohit Sahu, Vardhan Gacche, Tirthankar Ghosal, Asif Ekbal
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai