Generative AI Takes a Statistics Exam: A Comparison of Performance between ChatGPT3.5, ChatGPT4, and ChatGPT4o-mini
Project Overview
The document explores the transformative role of generative AI, specifically ChatGPT, in education, emphasizing its potential as a personalized tutor that can help address access and achievement disparities among students. It analyzes the performance of various ChatGPT versions in responding to statistics exam questions, uncovering significant differences in accuracy between the free and paid versions. This disparity underscores the critical importance of ensuring equitable access to advanced AI tools, as unequal availability could exacerbate existing educational inequalities. The findings prompt a discussion on the implications for future research and applications of generative AI in educational environments, advocating for strategies that leverage AI to enhance learning outcomes while promoting inclusivity and accessibility across diverse student populations.
Key Applications
ChatGPT as a personalized tutor
Context: Statistics and data science education for first-year graduate students
Implementation: Performance of GPT versions 3.5, 4.0, and 4o-mini was tested on statistics exam questions
Outcomes: GPT4 performed well, while GPT3.5 failed; GPT4o-mini's performance was intermediate. Insights into accuracy and the need for equitable access to AI tools were gained.
Challenges: Free versions of AI may not perform adequately compared to paid versions, potentially widening educational gaps.
Implementation Barriers
Access Barrier
Students from disadvantaged backgrounds may lack access to reliable internet and capable devices needed to utilize generative AI. Additionally, there is a significant difference in the performance of free-tier and paid-tier generative AI platforms, which can exacerbate inequities in educational outcomes.
Proposed Solutions: Institutions could negotiate educational pricing for AI tools, provide free access to advanced platforms for all students, and ensure that all students have access to the same quality of AI tools.
Ethical and Integrity Barrier
Concerns about academic integrity and potential misuse of generative AI in completing exams.
Proposed Solutions: Educators should establish clear guidelines on the appropriate use of AI tools and incorporate their use into teaching methodologies.
Project Team
Monnie McGee
Researcher
Bivin Sadler
Researcher
Contact Information
For information about the paper, please contact the authors.
Authors: Monnie McGee, Bivin Sadler
Source Publication: View Original PaperLink opens in a new window
Project Contact: Dr. Jianhua Yang
LLM Model Version: gpt-4o-mini-2024-07-18
Analysis Provider: Openai