REF Background

What is the REF?

The Research Excellence Framework (REF) is a system for assessing the quality of UK higher education research across all disciplines. The exercise is held every six to seven used and is used to allocate around £2 billion in annual research funding. It first took place in 2014, replacing the previous Research Assessment Exercise (RAE), and then again in 2021. The next REF exercise will take place in 2028. Submissions will be made in late 2027 and outcomes will be announced in December 2028.

What is the REF's purpose?

To provide accountability for public investment in research and produce evidence of the benefits of this investment.
To provide benchmarking information and establish reputational yardsticks, for use within the HE sector and for public information.
To inform the selective allocation of funding for research.

How is the REF carried out?

The REF is a process of expert review, carried out by expert panels for each of the 34 subject-based units of assessment (UOAs), under the guidance of four main panels. Expert panels are made up of senior academics, international members, and research users.

For each submission, three distinct elements are assessed: the quality of outputs (e.g. publications, performances, and exhibitions), their impact beyond academia, and the environment that supports research.

The Stern Review (2016)

Lord Stern’s independent review in 2016 made recommendations on the future operation of the REF. It examined how university research funding can be allocated more efficiently so that universities can focus on carrying out world-leading research.

The consultation proposals built on the implementation of REF 2014 and incorporated the principles identified in Lord Stern’s Independent Review of the REF.

It included proposals intended to reduce the burden associated with the REF process while maintaining and improving incentives for research excellence.

These include recommendations relating to:

The submission of staff and outputs
The approach to the assessment of impact
The introduction of an institutional level assessment.

Key points from the University’s response to the consultation:

We favoured the existing REF 2014 Unit of Assessment structure and recommended that it should be retained for REF 2021 to give a consistent approach to the assessment.
We supported the inclusion in REF of all research-active staff that independently lead their own research activity. However, we noted that a more precise definition is required in the description of ‘independent researchers’.
We welcomed the opportunity to submit research outputs variably across our staff and the greater flexibility intended by the proposed measures.
We believed non-portability of outputs may be detrimental to Early Career Researchers if implemented without measures taken to mitigate this impact. If the proposals are taken forward for REF2021, we argued that ECRs should be exempt from non-portability rules in order that they can develop their careers without any unintended impediment due to the requirements of REF.
We were strongly in favour of the appointment of interdisciplinary champions on the sub-panel for the assessment of interdisciplinary research.

Read the University’s full response to the consultation here.

The Future Research Assessment Programme (FRAP) (2021 - 2023)

The Future Research Assessment Programme was initiated at the request of UK and devolved government ministers and funding bodies.

It aimed to explore possible approaches to the assessment of UK higher education research performance.

Through dialogue with the higher education sector, the programme sought to understand what a healthy, thriving research system looks like and how an assessment model can best form its foundation.

The work strands included evaluating the REF 2021, understanding international research assessment practice, as well investigating possible evaluation models and approaches, to identify those that can encourage and strengthen the emphasis on delivering excellent research and impact, and support a positive research culture, while simplifying and reducing the administrative burden on the HE sector.

Research Metrics Reports (2022)

On 12 December 2022 UKRI published the findings of three metrics reports initiated as part of the FRAP. These reports were:

Together the reports cautioned against moving to a fully metricised system for the next UK research assessment exercise.

The reports suggested ways in which metrics, artificial intelligence (AI) or machine learning might be used to support or inform low-risk areas of the exercise, from allocating outputs to enabling post-assessment analysis.

The findings of these reports will be considered by the UK's four funding bodies, along with a broader suite of evaluation and consultation activities.

The funding bodies are expected to announce their initial decisions on the next assessment exercise in June 2023.

Research England's Initial Decisions on REF 2028

On 15 June 2023 UKRI published the Initial Decisions of the UK funding bodies on the high-level design of the next assessment exercise: REF 2028. These decisions respond to the key drivers for change that have emerged from the evaluation and consultation activities.

The initial decisions document signals two big shifts in national research assessment:

(i) A shift from the focus on individual performance to the contribution institutions and disciplines make to healthy, dynamic and inclusive research cultures and environments.

(ii) A move away from an overt focus on the public output to a broader assessment of research excellence and related research processes.

Summary of Decisions:

The next UK research assessment exercise will be conducted in 2028, with submissions to be made in late 2027.
REF 2028 will retain the REF 2021 Unit of Assessment (UOA) structure, and will include advisory panels on EDI and Interdisciplinary research.
Assessment will be undertaken by an expert sub-panel for each Unit of Assessment (UOA), working under the guidance of four Main Panels
As in REF 2021, Institutions will be required to submit outputs, impact case studies and a statement relating to each of the disciplinary areas in which they have research activity, alongside an institutional statement.
However, REF 2028 will have an adjusted and expanded definition of research excellence. The three
elements have been renamed and their content adjusted as follows:

1. The Output element is renamed 'Contribution to Knowledge and Understanding' (50% weighting). It will continue to be largely based on the assessment of submitted outputs, but in REF 2028 at
  least 10% of the score will based on a more structured disciplinary-level statement, which will provide evidence and data of the broader contributions to the advancement of the discipline. Further work and sector consultation to identify suitable metrics/data/indicators for this statement is planned for Autumn/Winter 2023.
2. The Impact element is renamed 'Engagement and Impact (25% weighting)'. Submissions will consist of both impact case studies and an accompanying statement, taken from the disciplinary-level statement, to evidence engagement and impact activity beyond case studies. The structured statement will make up at least 20% of the impact quality profile. The funding bodies intend to weight the statement on a sliding scale, proportionate to the number of case studies submitted.
3. The Environment element is renamed 'People, Culture and Environment' (25% weighting). This element replaces the environment element of REF 2014 and 2021 and will be expanded to include an assessment of research culture. Evidence to inform assessment of this element will be collected at both institutional and disciplinary level. The format of the environment statement will move towards a more tightly defined, questionnaire-style template. Further work is required to identify suitable metrics/data/indicators for both until and institutional-level statements.

Key changes and differences from REF2021:

REF 2028 will move away from any assessment of individuals, fully breaking the link between individual staff members and and unit submissions.
There is will be no single staff census date in REF 2028. The volume measure will continue to be determined by the number of staff who have significant responsibility for research and are independent researchers. But the data will be drawn directly from the HESA staff record.
The volume measure will be based on an average staff FTE over multiple years. For REF 2028, this average will be taken across Academic Years 25/26 and 26/27.
Research activity submitted for assessment can relate to any staff where a substantive connection to the institution can be demonstrated, not just research carried out by staff that contribute to the volume measure.
Submitting institutions will still be required to submit 2.5 outputs per FTE of volume-contributing staff in each disciplinary area where they have activity.
In REF 2028 there will not be any minimum or maximum contributions of any individuals. However, units will be required to explain how their submitted output pool is representative of the research undertaken within the disciplinary area.
Given there is no minimum output requirement for volume-contributing staff, there is no longer a need for a process to account for individual equality-related circumstances.
Institutions should continue to follow REF2021 Open Access (OA) requirements until new policies come into force. The funding bodies will consult with the sector in Autumn/Winter 23/24 before developing the full REF 2028 OA policy.
The number of impact case studies required in each submission will be determined by the average FTE of volume-contributing staff in the unit, with a minimum of one case study per unit submission. The funding bodies are proposing to revise the thresholds between case study requirements, and will consult with the sector on this in Autumn 2023.
COVID Impact statements will be retained, and institutions will be required to consider how Covid impacts have been addressed in output selection as part of REF Codes of Practice.

Next Steps and Timeframe:

Although the Initial Decisions document sets the high-level framework for REF 2028, there
are a number of matters on which further consultation and work is needed. Some of these will be pursued through a short, focused consultation, the response deadline for which is noon on 6 October 2023.

Summer 2023: Initial decisions consultation launched; work started on potential people, culture and environment indicators
Autumn 2023: Initial decisions consultation closes (6 October); open access consultation begins; Research England to publish further decisions on REF 2028
Winter 2023-24: Invite nominations for REF panel members; appoint panels
Spring 2024: Publish open access requirements; panels meet to develop criteria
Summer to autumn 2024: Publish draft guidance and consultation on panel criteria
Winter 2024: Publish final guidance and criteria
2025: Complete preparation of submission systems
2027: Submission phase
2028: Assessment phase, with REF results publication day likely to occur in December

Further Information

Read 'Harnessing the metric tide'

Read 'Responsible use of technology in research assessment'

Read 'REF Output Analysis: maximising the use of REF data'

Read Warwick's response to the Consultation on the second Research Excellence Framework (REF)Link opens in a new window

Read Lord Stern's independent report,Link opens in a new window
the Research Excellence Framework (REF) reviewLink opens in a new window