Towards Automatic Boundary Detection for Human-AI Collaborative Hybrid Essay in Education
Project Overview
The document explores the implications of generative AI, particularly large language models (LLMs) like ChatGPT, in the educational landscape, emphasizing both the transformative potential and the concerns raised by educators. While generative AI offers innovative tools for enhancing learning and engagement, educators worry that students may rely on AI to complete writing assignments, thereby undermining their critical thinking and writing skills development. To address these concerns, the study presents a novel two-step method for distinguishing between human-written and AI-generated content in hybrid texts, which has been shown to significantly outperform existing detection techniques in accurately identifying AI-generated portions within essays. Overall, the findings underscore the necessity for a balanced approach that integrates generative AI as a supportive educational resource while maintaining academic integrity and fostering essential skills among students.
Key Applications
Two-step boundary detection approach for hybrid essays
Context: Used in educational settings for identifying AI-generated content in student essays, targeting educators and students.
Implementation: Developed a dataset of hybrid essays by combining human-written text and AI-generated content, then applied a detection algorithm to identify boundaries.
Outcomes: The proposed method outperformed baseline models, improving detection accuracy significantly (22% improvement in In-Domain and 18% in Out-of-Domain evaluations).
Challenges: Challenges include potential inaccuracies in boundary detection, especially with texts that have multiple authorship transitions.
Implementation Barriers
Technical Barrier
Existing AI content detection methods often treat text as either fully human-written or fully AI-generated, limiting their effectiveness in real-world scenarios where content is hybrid.
Proposed Solutions: Adopting a more nuanced approach to detection that recognizes the collaborative nature of writing involving both humans and AI, as proposed in the study.
Educational Barrier
Educators are concerned that reliance on LLMs may undermine students' development of writing and critical thinking skills.
Proposed Solutions: Implementing educational practices that incorporate AI use responsibly, focusing on enhancing learning rather than replacing traditional skills.
Project Team
Zijie Zeng
Researcher
Lele Sha
Researcher
Yuheng Li
Researcher
Kaixun Yang
Researcher
Dragan Gašević
Researcher
Guanliang Chen
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Zijie Zeng, Lele Sha, Yuheng Li, Kaixun Yang, Dragan Gašević, Guanliang Chen
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai