Fooling MOSS Detection with Pretrained Language Models
Project Overview
The document examines the implications of generative AI, particularly GPT-J, in educational environments, focusing on its potential misuse in cheating on programming assignments. It reveals that GPT-J can create code that circumvents detection by MOSS, a widely-used plagiarism detection tool, thereby raising ethical concerns about the role of AI in maintaining academic integrity. The findings indicate that AI-generated solutions can be indistinguishable from original work, posing challenges for educators and institutions in ensuring honest assessments. The study highlights the limitations of current plagiarism detection methods and emphasizes the need for reassessing academic integrity in light of advancements in AI technology. Overall, the document underscores the dual-edged nature of generative AI in education, presenting both innovative possibilities and significant ethical dilemmas that must be addressed to foster a fair academic environment.
Key Applications
GPT-J for code generation and assignment completion
Context: College-level programming courses, particularly introductory courses.
Implementation: Students utilize the GPT-J API to generate solutions for programming assignments.
Outcomes: GPT-J successfully generates code that meets assignment requirements with minimal human modification, and the generated code does not trigger plagiarism detection by MOSS.
Challenges: The diversity in code structure may lead to ethical concerns regarding academic integrity and reliance on AI for educational tasks.
Implementation Barriers
Ethical barrier
The potential for AI to enable academic misconduct by making it easy for students to cheat.
Proposed Solutions: Incorporate AI tools into the curriculum for educational purposes, such as teaching students to critique AI-generated code.
Technical barrier
Current plagiarism detection tools may not effectively identify AI-generated code.
Proposed Solutions: Develop advanced detection techniques that can distinguish AI-generated solutions from human-generated ones.
Project Team
Stella Biderman
Researcher
Edward Raff
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Stella Biderman, Edward Raff
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai