Patchwork assessment
Using patchwork assessment to measure learning
Introduction
The metaphor of a patchwork refers to the idea that a variety of assessment pieces (patches), which individually are complete in themselves, are stitched together into something new (a patchwork). It is a type of portfolio assessmentLink opens in a new window but is distinct because the patchwork as a whole, which includes a reflective, integrative, summative ‘stitching’ section, is more than the sum of the individual parts. Alternative names for this assessment approach include: on-going narrative, evidenced story or collage of incidents. This assessment approach can include formative and summative elements. It also enables pivotal / critical learning moments, either at the time or retrospectively, to be identified and consolidated.
At first this approach was limited to short periods of learning, across a module or series of sessions, whereas more recently the method has been used across a year and even a whole programme.
Important aspects of patchwork assessment:
- it is personalised; you may set the task but the learner reflects on what they have learned through the process in addition to completing the task
- this constitutes continuous assessment which means that the learner is constantly building on achievement and moving towards autonomous learning
- continuous feedback is essential and this can be from you and / or their fellow learners; peer engagement is a common aspect of this approach as wider dialogue and reflection enriches the learning
- the stitching together (patchwork) cements and enhances learning from the individual tasks (patches) as this deepens learning as reflection on tasks, process, achievement and feedback across time combine.
Winter (2003) reporting the outcome of the Patchwork Text project, provides a neat definition of the approach:
The essence of a patchwork is that it consists of a variety of small sections, each of which is complete in itself, and that the overall unity of these component sections, although planned in advance, is finalised retrospectively, when they are ‘stitched together’. Thus, a ‘patchwork text’ assignment is one that is gradually assembled during the course of a phase of teaching and consists of a sequence of fairly short pieces of writing, which are designed to be as varied as possible and to cover the educational objectives of the teaching....The sequence of tasks within a Patchwork Text… is intended to build into the assessment process a recognition of learning as a gradual 'coming to know'.
What can patchwork assessment assess?
While a diverse range of subjects might adopt patchwork assessments the importance of critical reflection, particularly in the final ‘stitching’ of the individual elements, means that subjects where students need to integrate their practice/experiences with theory might find patchwork assessment particularly useful (e.g. clinical placements, work-based learning, teacher training, etc.). Furthermore, the importance of pivotal learning moments in patchwork assessment means that it is a particularly useful way of capturing learning over time. It may not be appropriate for highly conceptual subjects unless there is a need to reflect on how conceptual learning is being used and applied.
A patchwork assessment is not random; the 'stitching’ together of the patches, and their links and relationships, is crucial and so the tasks/incidents need to be carefully selected and planned. Formative feedback as the patchwork is developed should enable learners to gain confidence in analysis and synthesis of their learning and so reflect on its significance to their knowledge, understanding and practice. Whilst the process might be teacher-led at the outset it should become less so as the learner becomes more self-reliant and able to direct their own development and select the pivotal learning moments.
Design
It will be important to consider the number of assessment pieces that would form the patchwork and consideration of the balance between formative and summative assessment is necessary. One possible model is that each patch is assessed formatively by self and / or peers and is compulsory; the summative element would then be how the individual elements are ‘stitched’, possibly through a critical, reflective and consolidating piece of writing. A further consideration is how the individual elements are identified; one approach would be that students select these themselves from a wider pool of material. In this case the reasons for the selection of the patches might offer an insight into students’ thinking. The modality of the elements would also need to be considered: written, pictorial, video, audio, presentation are all possibilities. The extent to which students have choice about the modality will need to be decided during design, however, this does not need to be consistent across the whole patchwork - for example the students may have choice about the individual elements but are required to submit a written summative reflective text to consolidate the individual elements. The curation of the individual elements needs consideration and perhaps a technological solution such as Moodle or (link to) MyPortfolio would allow the small sections to be built up over time and be available for consideration by tutors or peers.
Can it be used for group assessment?
Individual patchworks might be developed for learning and assessment. However, it would also be possible for a group to work together, perhaps with each person from the group contributing a single patch and then each individual produces a unique and consolidating ‘stitching’ making use of the various group contributions.
Constructive alignment of the learning outcomes and the assessment through the patchwork approach is possible with careful design of the assignment brief. Consideration of how marks will be awarded across the whole patchwork is important at the design stage. Different models for assessing the patchwork against the intended learning outcomes could be adopted including:
- each small section being assessed formatively by self-assessment, peer assessment or tutor assessment with the final reflective analysis accounting for 100% of the summative assessment
- particular learning outcomes could be associated with individual patches, with the reflective analysis, or with the work as a whole
- each of the small patches could contribute to the overall assessment mark in one of the following ways: choosing the best patches (and so the best mark overall), averaging over the submitted patches or providing a mark holistically across the whole patchwork.
Your learning outcomes are likely to include higher order Bloom-type statements such as evaluate or critique. A challenge in designing patchwork assessment is to include tasks that ensure that learners demonstrate the required thinking and reasoning in line with the learning outcomes; it is too easy to be descriptive rather than analytical. In planning for the assessment careful thought will need to be given to how to help students demonstrate this higher order thinking.
Diversity & inclusion
The potential for diversity and inclusivity in the patchwork assessment depends largely on the component parts of the patchwork. Offering students choices about the modality of the individual components could potentially facilitate an approach to assessment that would meet the needs of all students.
Academic integrity
The focus on individual experiences of learning and individual choices means that it would be very difficult for different students to produce similar work. Even when used for group assessment the consolidating summary would be a unique interpretation of these materials by each individual. (Click here for further guidance on academic integrity.)
As patchwork assessment is often grounded in practice (work-based learning, practice-based learning) an entirely AI generated text is less likely (or likely to be less convincing 'out of the box') than other text-based forms of assessment. Integrating creative, or visual methods within the patches may further minimise this. That's not to say that AI can't mimic reflective work, but the synthesis and making sense of experience, and the evaluative choices of what to include and what to exclude privilege human minds. AI may have a valuable role as coach in this sort of learning, performing the role of critical friend (or participant researcher) to help students scaffold reflective thinking, and process their experience through a dialogic interview process.
Student and staff experience
Benefits
It can be hard for students to see a programme or even a module as a whole entity. Their experience might be that the various parts feel separate and lack coherence. Patchwork assessment has the potential to provide an opportunity for student to reflect on the individual parts and consolidate their learning through the process of ‘stitching’ together the different elements. The patchwork approach might be combined with other methods which would allow learning over time to be observed, such as blogs or on line fora posts.
Challenges
Student might become more creative in the presentation than the thinking. For example, it would be possible for students to spend a great deal of time on developing a creative and interesting video or pictorial patch which was mainly descriptive and fail to incorporate the all-important reflective and analytical component.
Staff will need to:
- explain the approach carefully and clarify which skills will be developed through the process
- emphasise both the reporting and analysis/reflective aspects of the patchwork
- support (scaffold) the early stages by
- preparatory workshops on reflective writing
- learning how to use and give peer feedback
- examples of ‘good’ and ‘bad’ patches - and why they are good/bad
- development and discussion of the criteria
- a collection of frequently-asked question and answers - perhaps online
- discussion of the time needed
- how to use the technology if an online tool is being used e.g. Mahara.
Workload
Patchwork assessment, for some departments, might be a significant departure from current practice. The advice would be to start small and then build on the approach. Providing formative feedback and engaging meaningfully with students about their 'patches' may increase workloads - AI may have a role to play in helping us provide individuated feedback, but substituting human encounters and relationships with AI may leave others feeling a little queasy.
Useful resources
HEA Patchwork Assessment Practice Guide, including some case studies
https://www.heacademy.ac.uk/knowledge-hub/patchwork-assessment-practice-guide
Silva-Fletcher, A., May, H., Magnier, K. and May, S. (2014). Teacher development: A patchwork text approach to enhancing critical reflection in veterinary and paraveterinary educators. Journal of Veterinary Medical Education 41 (2): 146-154.
A number of useful links and articles available at http://www.richardwinter.net/node/13