AI and assessment

Designing assessment for achievement and demonstration of learning outcomes

A key purpose of assessment is to yield evidence of student learning. This evidence depends on submitted work accurately representing students’ efforts and human abilities and/or knowledge.

Therefore use of generative AI technologies to produce text and other media as part of student submissions (or, indeed, as part of the process of developing such submissions) needs to be thoughtfully supported to ensure responsible use and clear demonstration of human achievements.

The proliferation of AI in education provides valuable opportunities to consider why we are assessing our students, what is being evaluated, and how evidence of learning is being gathered. Providing a clear rationale for assessment design and associated integration of AI will help students to understand why they are undertaking such activities and to see the value in responsible uses of AI technologies.

Considerations for assessment design

In considering the role of AI technologies in assessment a key starting point is to determine if you will be:

Collecting evidence to confirm unassisted human abilities or to confirm knowledge and skills that can be collaboratively achieved (with other humans and/or other intelligences).
Integrating AI collaboration into the assessment by designing assessment tasks that require and encourage appropriate use of AI (reflected in assessment criteria and standards).
Designing assessment tasks that make the use of AI less relevant, such as by focussing on embedded continuous assessment, highly contextualised and personal experience, or explicit demonstration of human capabilities.
Designing assessment tasks that demonstrate individual and independent human knowledge, or that bracket AI out with appropriate assessment security measures in place.

The following section provides guidance on how assessment could be modified in relation to the type of evidence required to effectively assess the desired forms of students’ learning (see also ADCs guidance on selecting assessment methods)

Check assessment task design against Generative AI tools

It is important to ‘test’ whether your assessments are designed in a way where they can be easily completed by artificial intelligence e.g. Warwick recommends the use of Copilot to students and staff, so try your assessment with Copilot first. It is also very useful to test the assessment with a number of other AIs. e.g. Claude, ChatGPT, Gemini, Perplexity, to see what output is possible using different tools. GenAI tools are unpredictable, and output will vary for each interaction and this variability makes it important to do multiple tests. Models are frequently updated and improved, so you will need to repeat this regularly.

Note: Creating an account and using these tools will provide data to the companies that own them.

The following activities could be used to assist in reviewing how amenable an assignment is to being answered by a Large Language Model (LLM) AI such as Copilot:

Paste an entire assignment brief into e.g. Copilot and see what it produces.
Paste the assignment brief and add further instructions derived from the assessment rubric, marking guide, and taught material. Try a variety of combinations of instructions and see if this improves the results. As models are able to hold increasing amount of information in their working memory you can also upload the rubric, brief, assessment criteria, guidance that you give to students, even taught material (whilst observing copyright and being mindful of data privacy) etc. Everything that students have access to could be used to formulate a response by the AI.
Sometimes, poor results are due to entering insufficient prompts into Copilot rather than because Copilot cannot produce a satisfactory response to an assignment. Don't give up after one prompt - review the result, and have a conversation with Copilot about the output to fine tune the results.

Notes for testing results:

Current AI models (like Copilot, Claude, etc.) can produce well-structured responses that demonstrate strong comprehension of instructions and context. However, verification of factual claims and citations remains important, as models may still generate plausible-sounding but incorrect information or non-existent references.

When reviewing assignment instructions, these models can now handle complex, multi-part instructions effectively. They generally show good ability to:
- prioritise and balance different requirements without explicit prompting;
- contextualize responses based on the academic level and subject area;
- follow detailed rubrics and assessment criteria;
- maintain consistent focus on key learning outcomes.
Models are increasingly capable of critical thinking and nuanced analysis, rather than just statistical pattern matching. However, they may still:
- need clarification on ambiguous requirements;
- benefit from structured prompting for highly specific tasks;
- occasionally miss subtle aspects of complex assignments;
- generate responses that, while coherent and well-reasoned, might not fully align with your specific aims and intended learning outcomes.

Identifying the strengths and limitations of GenAI tools in relation to your specific assignment will help you refine it - either to encourage students to collaborate with AI effectively, or to use assessment tasks to capitalise on uniquely human abilities. You might also adjust assessment criteria and standards in response to GenAI efficiencies.

Modify current assessment task prompts or processes

Current AI models can:

access and analyse subscription content if provided by users;
generate personalised, context-specific responses that appear authentic - reflection won't save you;
create convincing outlines, research notes, drafts, and revisions that show progression;
Participate effectively in iterative processes, including providing peer review feedback and incorporating suggested changes.

It is therefore unlikely that modifying existing assessment tasks will significantly change the ability of AI to contribute to student work. Changing the mode of assessment, i.e. from written submission to oral assessment, or shifting locus from product to process - or integrating the two might be a more effective way should this be your goal. You might also:

create assignments requiring integration of very recent events, current classroom discussions, or specific local observations that aren't yet well-documented online;
design assessments that integrate live discussion and real-time explanation, where students must verbally defend their work and demonstrate deep understanding of their process and choices;
focus on assignments that require original primary research, such as interviews with specific individuals or original data collection and analysis.

Incorporate AI into assessment tasks

Use of technologies to produce text and other media as part of student submissions (or, indeed, as part of the process of developing such submissions) needs to be thoughtfully supported and/or limited, and clearly and openly acknowledged. Some examples of building AI tools into assessment design are given below:

Allow students to use AI tools but require that they provide the prompt that they used and show via Track Changes (and comments) which changes they made to improve the output and why. (Note: some AIs can suggest improvements and present these as tracked changes - so this isn't AI proof).
Generate AI content and provide it to students for them to evaluate (with predefined criteria) and revise.

Consider alternative assessment formats

Some formats are more difficult for students to achieve with AI tools. They are also more likely to require subscription to access - which potentially introduces issues of equity and fairness.

Ask students to create oral presentations, videos, multimedia resources.
Ask students to incorporate more personal reflections tied to unit concepts - AI can convincingly fake reflection, so you might need to combine this with a multi-modal approach.
Ask students to take part in live interviews (face to face or online). An interview or viva voce can test a student’s understanding, gauging their ability to respond to questions and prompts in conversation. They can also be asked to elaborate or make connections between their responses to different questions. Bear in mind that live conversations are more difficult and anxiety-provoking for some students than others (although this is also true for other formats such as essays and exams).

Assess open or collaborative processes

Rather than only assessing the final output or “product” of students’ work, it is possible to also assess the process through which they produce it. For example:

In mathematics, students may be required to “show their working”, and the way they solve a problem can be more important than whether or not they arrive at a satisfactory answer.
In art or architecture, sketches can be compiled into a folio to show stages of the journey towards producing a finished piece. Each item of evidence of the student’s journey can be annotated or can form the basis of reflective writing, generating further insight into how the student was thinking about their work and making connections between ideas.

In this way, assessors can see a continuity of development that is specific to each student and makes sense as a whole. It may be possible to include the use of AI technologies as part of this process (as long as it is appropriately acknowledged and reflected on) but the primary goal is to engage with the way the student thinks about the unit and their work.

Similarly, process-oriented assessment approaches can also include collaboration with other students, as long as the submissions are structured to clearly indicate each individual’s contribution and insights.

An additional benefit is that these approaches to assessment can promote a culture of valuing the process of learning above the outcome (e.g. grades). It may create space for students to take risks and learn from their mistakes.

Target higher order thinking

In designing an assessment, choices are made (whether explicitly or not) about which forms of knowledge are valued. These choices are informed not only by the goals of the educator and learning outcomes in a unit, but also by practical considerations such as the size of the cohort, how much time there is or what technologies and systems are available, accreditation requirements, etc. Often, it has been simpler and more efficient to focus on simpler assessment formats such as multiple choice exams and standardised essay questions that produce clear and familiar responses.

However, AI technologies are reshaping this landscape because of their capacity to generate satisfactory responses to those same kinds of questions. In response, it may now be necessary to target forms of knowledge and expression that are more difficult for generative AI technologies - critical thinking, evaluation or creativity, for example.

Review assessments to reduce risk

While targeting higher order thinking may reduce the risk of misuse of generative AI tools, it is not sufficient to only consider the kind of thinking required to produce a response to an assessment task (e.g. by targeting higher order categories of Bloom's taxonomy). For example, in some circumstances, it is possible for ChatGPT to represent or imitate each of these categories of learning (e.g. it can create new ideas by synthesising unusual combinations of information sources). However, there are other assessment characteristics that can be associated with higher and lower risk of inappropriate use of generative AI.

Use formative assessment to develop understanding of students

In many higher education models, high stakes, heavily weighted summative assessment is the primary source of data about student performance and progress. Where the evidence used as a basis for judging student performance has been partly or wholly generated by technologies, this can lead to misrepresentation of student capability. Adopting a varied approach in designing an assessment regime, where data about student performance is gathered from formative and diagnostic tasks, can help to create a strong relationship with students, motivate them in their learning and provide information about whether summative assessment performances are consistent with other work.

While generative AI tools pose a significant challenge for some assessment formats when used for high stakes, summative assessment of learning(e.g. end of unit essays, etc.), they may be less problematic for alternative formats and lower stakes, formative assessment for learning.

Selecting a range of tasks with varied weighting can allow students to demonstrate learning in a lower-risk environment, giving you the opportunity to gauge your students' understanding while also lowering the incentive for them to rely on tools to generate responses.

Formative assessment that isn’t weighted also allows for opportunities to check understanding and build a picture of student capability. Where formative assessment helps students prepare for summative assessment, it may also increase their confidence to engage authentically in those summative assessment tasks.

Programmatic, authentic, and future-focussed assessment

Programmatic assessment draws upon the notion of assessment for learning. Programmatic assessment involves a series of interconnected assessment methods that build upon each other. Rather than focus on assessing individual kinds of knowledge or skills using a single assessment method, it focuses on assessing broader attributes using multiple methods.

Programmatic assessment supports a holistic assessment frame that is most likely to assure assessment tasks so that they are diverse, authentic, provide relevant evidence of student learning and minimise risk of inappropriate use of generative AI technologies. Research Link opens in a new windowindicates that programmatic assessment has many benefits for students and educators, including increased confidence in assessment decisions, reduced emotional burden and early identification of at-risk students.

Authentic assessment focuses on assessing student learning against learning outcomes aligned with “real-world” contexts and industry expectations (including, but not limited to. work-integrated learning). Authentic assessment tasks require students to apply knowledge and skills in meaningful ways. Designing authentic assessments involves not just focusing on authentic tasks but also on educator judgement of the relevance of the tasks and how they can contribute to students’ sense of belonging to a profession or society. Examples of authentic assessment can include case studies, exhibitions, reflective portfolios, and problem-based inquiries. While some research Link opens in a new windowappears to support the role of authentic assessment (when done well) in reducing academic misconduct, the evidence is unclear and the likely success of this approach will depend on how it is done in its particular context and conditions Link opens in a new window(Fawns et al., 2024).

Future-focused or “sustainable” assessment Link opens in a new windowadvocates assessing the knowledge, skills, and attitudes that students would need throughout their lives beyond graduation. This kind of assessment focuses on developing the ways in which students learn to contribute to their capacities as lifelong learners. A key principle of future-focused assessment is to develop students’ capabilities to make evaluative judgments about what constitutes good work. Some examples of future-focused assessment tasks across a range of disciplines can be found on the Monash University Be Inspired pages.

Each of these assessment elements has the potential to reduce the risk of breaches of academic integrity in relation to generative AI technologies:

By incorporating multiple assessment formats and targeting a range of different kinds of knowledge, programmatic assessment has the potential to capture learning that cannot be simply replicated by tools like ChatGPT.
Students are less likely to engage in academic misconduct when they find assessment tasks authentic, personally relevant, and more readily applicable to real-world scenarios.
Future-focused assessment can help students understand how they might learn to work with new technologies like ChatGPT in ways that contribute to their personal and professional development over time.

Using AI in learning and teaching < Previous