Exploring the Use of AI in Mathematics and Statistics Assessments

About the Project

The mathematical sciences and operational research (MSOR) community in higher education is currently underprepared to address the rapid integration of advanced AI technologies into academic environments. While in-person examinations have traditionally been the primary method of assessment in these disciplines, take-home assignments remain a critical component for evaluating student knowledge and problem-solving skills. However, the increasing sophistication of AI presents significant challenges to the integrity of these assignments.

Purpose of the Project

The primary purpose of this project is to explore and understand the impact of advanced AI technologies, specifically Large Language Models like GPT-4o, on mathematics and statistics assessments in higher education. By thoroughly examining the capabilities and limitations of these AI models, the project aims to provide insights that will inform future assessment strategies within the Mathematical Sciences and Operational Research (MSOR) community.

Research Objectives

1. AI Performance on Current Assignments

How well do AI models like GPT-4o perform in solving university-level mathematics and statistics problems?

To assess this, we ran a series of university-level mathematics and statistics assignments through a default GPT-4o model in a zero-shot setting—meaning the AI received no additional guidance or prompting beyond the standard input. The AI-generated responses were then marked against the established mark scheme used for these assignments. To ensure a comprehensive evaluation, this process was conducted twice: first, the responses were independently marked by our team, and then the second set of generated responses was evaluated by the lecturers who had set the assignments, requesting them to directly compare the AI's performance with that of human students.

2. Student Use of AI

To what extent are students using AI to complete their assignments, and what are their perceptions and understandings of these tools?

We explored this question through a combination of surveys and focus groups involving students from the mathematics and statistics departments. The survey gathered quantitative data on AI usage among students, including their motivations, perceptions, and levels of trust in AI-generated outputs. Additionally, we conducted two focus groups: one with students who actively used AI in their assignments and another with students who did not. These discussions provided qualitative insights into how students interact with AI tools and how they perceive the benefits and challenges of using such technologies in their academic work.

3. Future Assessment Strategies

How can future assessments be designed to incorporate AI as a learning tool while preserving academic integrity?

In addressing this question, we developed a comprehensive set of regulations and recommendations aimed at managing the integration of AI into educational assessment. This involved examining the complexities of current regulatory discussions and crafting a refined narrative that balances the growing use of AI with the ethical concerns surrounding it and the need to maintain academic integrity. Our recommendations are designed to help educators create assessment strategies that incorporate AI as a supportive learning tool without compromising the credibility and reliability of student evaluations.

Significance of the Study

Understanding AI's capabilities and limitations within the context of mathematics and statistics education is of paramount importance. As AI technologies continue to advance rapidly, educators must stay informed and adapt to these changes to provide relevant and effective instruction. This project aligns with broader educational goals by:

• Enhancing Educational Practices: Providing insights that can help educators design assessments and curricula that reflect the current technological landscape.
• Promoting Ethical AI Usage: Encouraging responsible use of AI among students by increasing awareness of its strengths and weaknesses.
• Maintaining Academic Integrity: Developing assessment strategies that uphold the standards of academic honesty while embracing innovative tools.

By addressing these objectives, the project aims to contribute valuable knowledge to the MSOR community and support the ongoing evolution of education in the age of AI.