Exploring the use of AI in mathematics and statistics assessments
About the Project
The mathematical sciences and operational research (MSOR) community in HE is poorly prepared to adapt to the rapid rise of AI. Whilst in-person exams remain an essential assessment mode for our subjects, take-home assignments are also an integral assessment tool. However, our current assignments are not robust against AI, and staff have no time to capture ways in which AI might be used to enhance learning.
Recent AI studies in HE tend to cover a broad range of subjects without the specific needs of mathematical sciences being properly considered. According to the QAA benchmark for MSOR, students “can sometimes be expected to provide an answer which is very close to a model answer”, whilst AI can often provide such model answers.
Project Information
This project aimed to address these issues and make suggestions, specific to MSOR subjects, on how maths assessments could be adapted to work with AI whilst still achieving excellent learning outcomes. The project team investigated assignments in their respective departments, tackling the following questions:
- How well can AI perform in our current assignments?
- To what extent do our students currently use AI to help with assignments?
- Can future assessments incorporate AI as a copilot, whilst still achieving module outcomes?
Based on our findings and further surveys of how maths assessments are changing in other universities, we suggested how AI might be integrated in future assessments. We aimed to develop concrete examples of AI-assisted take-home maths assignments, which are available in the 'results' box, and we will disseminate our findings further in seminar talks and via a journal article.
Project Team
- Siri Chongchitnan (maths, project lead)
- Martyn Parker (statistics, project lead)
- Mani Mahal (research intern)
- Sam Petrie (research intern)
- Home
- 1.Formal Report
- 1.1 Introduction to Project
- 1.2 The Emergence of ChatGPT and Limitations of GPT-3.5
- 1.3 Understanding LLMs and Evolution of AI Models
- 1.4 Extending LLM Capabilities and Introduction of ChatGPT o1
- 1.5 A Step Change in AI Capabilities and Key Findings
- 1.6 Performance of AI Models and Urgency for Institutional Action
- 1.7 Recognising the Problem and Specific Regulations
- 1.8 Recommendations and Conclusion
- 2. Student Conversations
- 3. How ChatGPT Performed on University-Level Work
- 4. Suggested Changes and Future Direction of Regulations
- 4.1.1 Developing Clear Policies on AI Use
- 4.1.2 Enhancing Student Support and Guidance
- 4.1.3 Emphasising Skills That AI Cannot Replicate
- 4.1.4 Adapting Pedagogy and Innovating Assessments
- 4.1.5 Encouraging Collaborative Solutions Among Stakeholders
- 4.1.6 Allocating Resources for Training and Support
- 4.1.7 Adopting Alternative Assessment Methods
- 4.1.8 Relying on Honour Codes and Academic Integrity Pledges
- 4.1.9 Designing AI-Resistant Assignments
- 4.1.10 Using AI Detection Software
- 4.1.11 Implementing Oral Examinations (VIVAs)
- 5 Opportunities AI Presents
- 6 Tips For Markers on Spotting Potential AI Usage
Project Overview
The mathematical sciences and operational research (MSOR) community in higher education is unprepared for the rapid rise of AI. While in-person exams are crucial, take-home assignments, essential to our assessment process, are vulnerable to AI. Current assignments lack robustness against AI, and educators lack the time to explore AI's potential to enhance learning.
Recent AI studies in higher education often generalise across subjects, overlooking the specific needs of mathematical sciences. According to the QAA benchmark for MSOR, students are expected to provide answers that closely resemble model solutions, which AI can readily generate.
Research Areas
1. To what extent do our students currently use AI to help with assignments?
This area investigates the prevalence of AI usage among students and how it influences their approach to coursework.
Student Conversations on AI Usage
2. How well does AI perform in current assignments?
We are evaluating the capabilities of AI tools like ChatGPT in handling university-level assignments to understand their effectiveness and limitations.
How well does ChatGPT perform in university assignments?
3. Can future assignments incorporate AI as a co-pilot while still achieving module outcomes?
We are exploring innovative ways to integrate AI into assignments that enhance learning while maintaining academic integrity.
Further to this, opportunities that generative AI presents in education for module leaders, as well as learning opportunities for students have been present but not explored in depth.
AI in Mathematics and Statistics Education: Recommendations and Future Directions of Regulations