Beyond Detection: Designing AI-Resilient Assessments with Automated Feedback Tool to Foster Critical Thinking
Project Overview
The document explores the role of generative AI in education, emphasizing both the opportunities and challenges it presents. While AI tools such as ChatGPT have the potential to aid learning, they also pose risks by undermining critical thinking skills and facilitating academic dishonesty. To counteract these challenges, the research advocates for a proactive strategy involving a web-based Python tool that utilizes Bloom's Taxonomy and advanced natural language processing to develop assessments that are resistant to AI-generated responses. This innovative approach aims to preserve educational integrity and promote originality in student work, while also addressing the shortcomings of existing AI detection methods. Ultimately, the findings underscore the necessity of adapting educational practices to harness the benefits of generative AI while safeguarding the fundamental skills of critical thinking and authenticity in assessments.
Key Applications
Web-based Python tool for AI-solvability analysis of assessments
Context: Higher education, specifically for postgraduate students at the University of Sydney
Implementation: The tool integrates Bloom's Taxonomy with GPT-3.5 Turbo and other NLP techniques to evaluate the cognitive complexity of assignments.
Outcomes: The tool helps educators design assessments that promote higher-order thinking and reduce reliance on AI-generated responses.
Challenges: Initial resistance from educators accustomed to traditional assessment methods and the need for ongoing evaluation of AI integration.
Implementation Barriers
Technical barrier
Current AI detection tools are inaccurate and produce unreliable results, with high rates of false positives and negatives. This impacts the assessment of student work and raises concerns about the integrity of evaluations.
Proposed Solutions: Implementing a new assessment design tool that focuses on cognitive complexity rather than solely detecting AI use.
Pedagogical barrier
There is a risk that reliance on AI tools can undermine critical thinking and creativity among students, as well as a potential decline in independent learning.
Proposed Solutions: Fostering a balance between AI use and independent learning, encouraging educators to design assessments that require higher-order cognitive skills.
Project Team
Muhammad Sajjad Akbar
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Muhammad Sajjad Akbar
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai