Skip to main content Skip to navigation

Warwick International Higher Education Academy

5. Usage Patterns and Trends

Overview

This section examines the usage patterns and trends of AI tools among Mathematics and Statistics students at Warwick University. Drawing from survey responses of 145 students conducted in June 2024, with 59% reporting they have used AI to help complete assignments, we explore which AI tools are most commonly used, the frequency of use, and changes in usage over the last academic year. By analysing this data and connecting it to prior findings on Ethical Considerations, Academic Integrity, Student Attitudes Towards AI, and Integration of AI into Academic Assignments, we aim to provide a comprehensive understanding of how AI tools are impacting student learning and engagement.

Types of AI Tools Used by Students

Key Findings:

  • ChatGPT by OpenAI: Default (GPT-3.5 Turbo – free) is the most popular AI tool, used by 74 students. However, this is a legacy model and does not fully represent the capabilities of current AI technology.
  • 41 students use ChatGPT by OpenAI: GPT-4o (GPT-4 Omni – free/low use cap), indicating some engagement with more advanced models.
  • Only 12 students reported using ChatGPT Premium (GPT-4/4o – high use cap), suggesting cost barriers and a lack of awareness about newer models.
  • Wolfram Alpha is utilised by 22 students, highlighting its utility for mathematical computations.
  • A substantial 41% of students (60 respondents) have not used any AI tools for completing assignments.

Students' reliance on ChatGPT is evident, but their experiences vary, often influenced by the version they use and their awareness of AI capabilities. Some found it helpful for coding assistance:

"I mainly used them to test code in R as ChatGPT in particular is great at spotting mistakes and optimising code, which is helpful to know in future assignments."

Others, however, were critical of its effectiveness in mathematical tasks:

"I’ve used ChatGPT twice for assignment questions I didn’t know how to complete at the time. I tried for nearly an hour with various prompts but was provided with nothing accurate or usable."

This suggests that limited exposure to more advanced models and a lack of AI literacy may contribute to mixed perceptions of AI tools.

Frequency of AI Tool Usage

Key Findings:

  • Among AI users:
    • Only 6% use AI tools all the time for assignments.
    • The majority use them sometimes (35%) or rarely (35%), indicating supplementary usage.
    • 9% report never using AI tools for assignments, despite having used AI tools to help complete assignments. This suggests varied interpretations of "use."
  • Among non-AI users:
    • A significant 97% report never using AI tools for assignments.
  • Overall, 45% of all respondents never use AI tools, 43% use them sometimes or rarely. While only 3% use them all the time.

This variation highlights that students use AI tools differently. Some use them for learning rather than assignment completion:

"AI is good enough at 'textbook knowledge' to act as an interactive browser for learning. If I don't understand a theorem when going through content, it's possible to ask for specific clarifications."

Others may avoid them due to ethical concerns or a preference for independent work:

"I don’t want to rely on AI, so I decided not to use it at all. Even if the use of AI could’ve made me more confident or secured a few more assignment marks, the exam is weighted highly enough that it didn’t feel like I would be gaining enough exam preparation from assignments by using AI."

Note: Click on the graph labels (e.g., "AI Users", "Non-Users", or "Combined") to view each group's data separately.

Changes in AI Tool Usage Over the Last Academic Year

Key Findings:

  • Among AI users:
    • 19% reported that their use of AI tools has increased a lot over the last academic year.
    • 33% reported an increase a bit, reflecting adaptation to new AI developments.
  • Among non-AI users:
    • A vast majority, 83%, reported that their AI tool usage has not increased at all.
  • This contrast highlights a persistent divide in AI adoption between users and non-users, possibly due to unawareness of AI advancements and ethical considerations.

Some students increased their usage due to improvements in AI tools and their growing capabilities:

"The tools have gotten better and better, especially at maths and statistics."

Others remain sceptical or concerned, often due to ethical reasons:

"To me, the use of AI in assessments is cheating, whether it can be detected or not."

Note: Click on the graph labels (e.g., "AI Users", "Non-Users", or "Combined") to view each group's data separately.

In-Depth Analysis of AI Usage Patterns

1. Diverse Experiences with AI Tools

Students' experiences with AI tools are varied. While some find them helpful for coding and concept clarification, others are critical of their effectiveness in complex mathematical tasks. This variance may stem from differences in AI literacy and awareness of more advanced models:

"If a question has highlighted a clear misunderstanding in my knowledge in a certain area, I use AI to identify where my misunderstanding lies in that topic."
"I asked ChatGPT questions about maths, and it struggled to answer them correctly, so I do not rely on it for complex mathematical questions."

2. General Unawareness of AI Advancements

Many students are unaware of recent developments in AI technology, which impacts their perceptions and usage patterns:

"I discovered how easy it was to use; I didn't really know the free software existed before."

This lack of awareness means students may not be utilising AI tools to their full potential, even though current AI models have significantly improved in handling university-level mathematics and statistics.

3. Ethical Concerns and Academic Integrity

Ethical considerations play a significant role in students' decisions to use or avoid AI tools. Some students express concerns about cheating and the integrity of their work:

"I study for knowledge, not scores, so I will sometimes ask AI questions, especially in programming, but I will never use it if it compromises my understanding of the knowledge."
"I am against using AI as it feels like cheating, so I haven’t used it."

4. Varied Interpretations of AI Tool Usage Among Students

The data reveals that some AI users report "never" using AI tools for assignments, despite indicating that they have used AI tools to help complete assignments. This suggests varied interpretations of what constitutes "use" of AI tools in the context of assignments.

"No use of directly answering questions but can be useful for content clarification when learning. I'm putting this under use for assignments because during term time I'd consider learning content and preparing/working on its associated assignment as one in the same."
"The wording of the question is key here: I don't use AI for assignments because LLMs suck specifically at the kind of things the typical assignment wants. This doesn't mean they aren't useful for general revision!"

5. The Divide Between AI Users and Non-Users

The survey highlights a significant divide between AI users and non-users. Non-users often cite ethical concerns, preference for independent learning, or scepticism about AI's effectiveness:

"Use of AI undermines the integrity of assignments and my degree."
"I prefer to understand the method rather than simply obtaining the answer from AI."

6. Impact of AI Literacy on Usage Patterns

AI literacy appears to influence how students perceive and utilise AI tools. Those who are more knowledgeable about AI advancements tend to use the tools more effectively:

"It's gotten better, and I've gotten more used to using it and understanding how to get the best out of it."

Conversely, limited AI literacy may prevent students from experiencing the full benefits of current AI capabilities.

7. Limitations of AI Tools in Mathematics

Some students find AI tools inadequate for complex mathematical problems, which may reflect limitations in the models they are using or how they are using them:

"I asked ChatGPT to prove the Central Limit Theorem, but it did nothing except confuse me more."

Improving AI literacy could help students navigate these limitations and leverage AI tools more effectively.

8. Uses of AI Tools by Students

Students who use AI tools often employ them for various purposes, such as coding assistance, checking work, and understanding concepts:

"I use AI for long-form written assignments to reword and improve the quality of my writing. I also use it to help with coding, mainly when I know there is an efficient way of doing something but I cannot figure out what it is."
"Wolfram Alpha is very useful for checking answers to calculation questions."

9. Ethical Considerations and Need for Guidelines

Students express a need for clear guidelines and support from the university regarding AI usage to navigate ethical concerns:

"I have no interest in being kicked off my degree by lecturers who don't understand AI... This is to my detriment as AI's use is only growing in industry and my lack of exposure to it is hampering my career prospects."

10. The Role of AI Literacy in Maximising Benefits

Enhancing AI literacy among students could bridge the gap between AI users and non-users, allowing more students to benefit from the advancements in AI technology:

"AI has been improving a lot."
"AI has gotten better at explaining choices in coding."

Conclusion and Recommendations

The usage patterns and trends of AI tools among Mathematics and Statistics students at Warwick University indicate a growing but cautious integration of AI into academic work. While some students leverage AI tools effectively, others are hindered by ethical concerns, scepticism, and a lack of AI literacy. Despite AI's growing capability to correctly answer university-level mathematics and statistics work, many students are unaware of these advancements, impacting their perceptions and usage.

Specific Recommendations

  1. Enhance AI Awareness: Implement programs to inform students about the latest AI developments and tools, emphasising their capabilities and limitations.
  2. Promote AI Literacy: Offer workshops and resources to educate students on effective and ethical use of AI tools, helping them to leverage these tools responsibly.
  3. Address Ethical Concerns: Provide clear guidelines on acceptable AI usage to alleviate fears related to academic integrity and cheating.
  4. Encourage Critical Engagement: Teach students how to critically assess AI-generated content, fostering a thoughtful approach to AI-assisted learning.
  5. Integrate AI Thoughtfully into Curriculum: Encourage educators to incorporate AI tools into teaching strategies that enhance learning without compromising ethical standards.
  6. Facilitate Open Dialogue: Create platforms for students and faculty to discuss experiences with AI tools, addressing concerns and sharing best practices.
  7. Stay Informed on AI Developments: Institutions should keep abreast of rapid AI advancements to adapt policies and support accordingly.

Final Section: Students' Concerns and the Future of AI in Education

The upcoming section explores students' concerns about the future implications of AI in education, including potential impacts on learning outcomes, employment prospects, and the evolution of teaching methods.

Previous: Integration of AI into Academic Assignments

Review the previous section to understand students' perspectives on integrating AI into assignments, focusing on the balance between leveraging AI's benefits and maintaining critical thinking skills.