Outcomes
Review Reports
As outlined in Review delivery, every department/cluster will receive a review report from their review panel. It will be structured around the Evaluation Framework and convey the review panel’s conclusions on each evaluation area, and include an evaluation rating and a risk rating. A report template will be used for consistency. The reports will:
- provide assurance to the University (or highlight risk)
- inform the future enhancement of education and the student experience
- comprise two parts for each evaluation area: an evaluative analysis highlighting strengths and weaknesses at present and an advisory narrative that offers an expert view to the department/cluster on how it can move forward
- specify a ‘condition’ where there is a cause for concern or risk to academic standards and quality
- These should be used sparingly and if they are ratified by AQSC, they will form the focus of regular follow-up activity with the department/team until the Review Sponsor signs off the action as complete.
- Highlight good practice in all four areas of focus to celebrate departmental/cluster strengths, support enhancement or continuous improvement
We reflected on ITLR 2017’s use of the term ‘recommendation’, which was interpreted differently by the review panels and resulted in variable levels of understanding, making it difficult to discern the level of action expected and then met. So that departments can make most effective use of the findings from this ITLR, review panels will make clear that where a risk to quality or the student learning experience is identified, they will determine whether the risk requires a specific action (and specify that action explicitly to departments) or whether the risk is a matter for consideration and forms an advisory suggestion in the report. We believe greater clarity and differentiation will enable the same level of flexibility under the 2017 ITLR approach, but will help departments to effectively manage their resources. As such, ITLR 2023 reports will not use the term ‘recommendation’ and any follow-up activity will focus on addressing any identified conditions.
Consistent Outcomes
The evaluation areas Baseline Assurance and Strategic Improvement will provide assurance to the University (and in turn our regulator, the OfS) that baseline expectations of academic standards and quality are met. To provide assurance about education quality, as noted above, review panels will choose one of a small number of standard outcomes for these evaluation areas and assess whether there is a significant risk to that outcome changing in the future.
The inclusion of an outcome, or ‘grade’ as it was called in ITLR 2017, is not new. However, we're updating this method to reflect the nature of ITLR to support enhancement of the student experience and academic quality and standards. Therefore, the review panels will use one of the following outcomes:
- There is demonstrable commitment of a department's/cluster’s responsibilities with respect to quality and standards, and the enhancement of the students’ learning experience or support.
- There are no issues that require action but there are nevertheless plans to improve further.
- The needs of students are a clear priority in strategy development and a student co-creation or partner approach is often taken to support enhancement or continuous improvement.
- The panel identifies frequent and widespread good practice with some aspects that would be considered sector-leading practice.
Applied as a “best fits” approach where:
- There is clear awareness of a department's/cluster’s responsibilities with respect to quality and standards, and the enhancement of the students’ learning experience/support.
- Department/Cluster has identified where there are issues or gaps, and these are being addressed with clear plans for completion.
- There may be some minor issues that need to be addressed but these do not present a significant risk to quality or standards and do not require major structural or procedural change for remedy.
- The needs of students are a priority in strategy development.
- Student/Stakeholder engagement is widespread and supported with evidence that feedback is acted upon to address concerns or actions identified.
- There are some examples of good practice.
Applied as a “best fits” approach where:
- There is evidence to suggest a limited understanding of the department’s/cluster’s responsibilities for quality and standards and the enhancement of the students’ learning experience or support. Whilst there may be evidence of response to the previous ITLR, other quality reviews or feedback from students/stakeholders, the action does not commonly result in sustained improvement and the same feedback or quality issues are usually repeatedly raised or identified.
- There are conditions that require action which are considered to pose a major or moderate risk either in the immediate future or over time.
- In some rare instances there may be a serious weakness that could undermine quality and standards.
- There is insufficient emphasis or priority given to the needs of students in strategy development.
- Some shortcomings in the way in which quality assurance processes are applied.
- There are no or very few examples of good practice.
Where a department meets expectations, review panels will be asked to consider whether their findings show any significant risk to that continuing to be the case. For example:
We recognise that risk does exist and is, in most cases, managed through careful mitigation to reduce the likelihood of impact. We are particularly interested in identifying where the level of risk is moderate or major, rather than minor, and will develop a framework to help guide review panels’ evaluation of this as part of review panel training.