A comparison of assessment criteria and methods for scoring individual contribution to group projects and evaluation of reliability and fairness of peer review when compared with expert observation of group work.

Group work is prevalent in Engineering because it is a required competency for accreditation and for chartership. It can also be a rewarding method for technical learning with students sharing skills and knowledge and achieving more than they would have working alone. Group working also provides an efficiency benefit for marking and supervision. However, we also know that it can be a difficult experience for many students and that unequal perceived effort can result in tension which continues throughout the degree. Students have been found to collude at the expense of others and much effort is spent by academics on justifying final scores.

Ultimately, the School aims to create Engineers who will excel and thrive in team projects once employed in industry. Within the assessment context, the School also wishes to identify and penalise students who make inadequate contributions. The student experience of group work can be improved with continued improvement in clarity of expectation and criteria, and in transparency of marking. We must also justify the hypothesis that students who work well in teams share knowledge, achieve more and therefore have higher learning gain and satisfaction.

Project Activity

In Engineering, we are piloting an alternative peer review system to the standard mean weighting distributed points system (WebPA, WBS, SPARK).

Following a 12-month pilot on selected modules in year 1 and 2 in the School of Engineering (325 students in each), we aim to validate our proposed new peer review scheme via:

1. Investigation of existing methods to evaluate the success of a peer review system including student voice^g, unconscious bias, academic and administrative benefit, learning gain, accuracy and fairness.
2. Comparison of peer review scores with expert observation of group work
3. Consideration of scoring methods (median or mean weighting, knee formulae etc.) and how they affect the final individual score
4. Proposal of a new or updated framework to assess the fairness, accuracy and benefits of peer review

Following the results of this project, we hope to do some follow up projects in the next academic year working across the University on:

1. Learning gain in group projects: design of group exercises and assessment methods to ensure and measure shared technical learning by all students
2. Value driven assessment in group work: exploration of values promoted and rewarded by assessment criteria and impact on diverse student populations. Consideration of discipline specific vs universal values and attributes which signify a University of Warwick graduate.

We have already collected data comparing the new system with the old system, surveying students to understand their point of view and presenting our findings at the 2018 Engineering Education Research Network Symposium and at the University of Warwick Education Forum. We now wish to systematically and rigorously evaluate our system so that we might consider implementation more widely than the School of Engineering.

WIHEA Masterclass

The progress so far was presented at a WIHEA masterclass on 3 April 2019, inviting attendees to use the system. There was discussion of discipline specific values which underpin team work as well as identification of the University of Warwick signature values for our graduates.

The collaborations formed via this masterclass will be the input into the two follow-on projects in the next academic year.

Project Team:

Dr Claire Lucas
Dr Thomas Popham