Evaluating AI and Human Authorship Quality in Academic Writing through Physics Essays
Project Overview
The document examines the role of generative AI in education, particularly focusing on its application in producing written content, such as physics essays. A study within the document assesses the quality of AI-generated essays against those written by humans, revealing no significant differences in scoring, which indicates that AI can create work that meets educational standards. This finding raises important implications for academic integrity and assessment practices, as the challenge of detecting AI-generated content becomes increasingly pronounced. The study suggests that essays containing up to 50% AI-generated material could still be classified as human-authored, highlighting the necessity for educators to adapt their evaluation methods in light of these advancements. Overall, the document underscores the potential of generative AI to enhance learning while also calling for a reevaluation of traditional assessment approaches to accommodate the integration of AI technologies in educational settings.
Key Applications
AI-generated essays using OpenAI's GPT-4
Context: Physics education, specifically in the 'Physics in Society' module at Durham University; target audience includes undergraduate students
Implementation: Students were assigned essays for a module, and both human and AI-generated essays were evaluated by independent markers using blinded assessments.
Outcomes: AI-generated essays scored similarly to human-authored essays, indicating parity in writing quality; tools like ZeroGPT showed high accuracy in detecting AI authorship.
Challenges: Difficulty in detecting AI-generated content and concerns about academic integrity.
Implementation Barriers
Technical Barrier
Existing detection tools struggle to reliably differentiate between AI-generated and human-authored text, especially when AI content is paraphrased.
Proposed Solutions: Adopting improved AI detection technologies and establishing standards for acceptable levels of AI contribution in academic work.
Ethical Barrier
The ethical implications of using AI in academic writing raise concerns about plagiarism and the integrity of student submissions.
Proposed Solutions: Implementing clear guidelines on the permissible use of AI tools in academic work, such as a limit on the proportion of AI-generated content.
Project Team
Will Yeadon
Researcher
Elise Agra
Researcher
Oto-obong Inyang
Researcher
Paul Mackay
Researcher
Arin Mizouri
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Will Yeadon, Elise Agra, Oto-obong Inyang, Paul Mackay, Arin Mizouri
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai