Experiences with Remote Examination Formats in Light of GPT-4
Project Overview
The document explores the influence of generative AI, specifically GPT-4, on examination methods in software engineering education, particularly examining the effectiveness of open-book and oral exams. It highlights the challenges posed by GPT-4's capabilities, which raise concerns about potential cheating and the integrity of traditional assessment formats. The findings indicate that while GPT-4 can demonstrate proficiency in certain exam types, there is a pressing need to reevaluate conventional examination practices to maintain academic integrity and accurately assess student knowledge. This reassessment is crucial as educational institutions adapt to the evolving landscape shaped by advanced AI technologies.
Key Applications
GPT-4 in exam settings
Context: Third-year BSc Software Engineering program at a Swedish university, focusing on open-book home exams and oral exams.
Implementation: Empirical data comparison of oral exams versus open-book exams, evaluating GPT-4's performance on exam questions.
Outcomes: Higher throughput for open-book exams (73% vs 64%) with similar fail rates, GPT-4 passing both exams with high scores.
Challenges: Concerns regarding academic integrity, difficulty in ensuring the reliability of assessments, and the limitations of GPT-4 in answering specific question types.
Implementation Barriers
Assessment Integrity
The ability of students to use AI tools like GPT-4 for cheating in open-book exams, coupled with logistical difficulties in arranging supervised examinations and ensuring compliance with educational standards.
Proposed Solutions: Redesigning assessment formats to include more challenging questions that require critical thinking and creativity, exploring new assessment formats that can accommodate remote learning while maintaining integrity, and potentially moving towards more oral examinations.
Project Team
Felix Dobslaw
Researcher
Peter Bergh
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Felix Dobslaw, Peter Bergh
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai