Organisational Behaviour in Response to Incentivised Targets
Leads: Mr Paul Bird (WM-AHSN), Dr Katharine Reeves, Dr Hannah Crothers (Meths), Prof Kamlesh Khunti (ARC EM)
Public Contributors linked to the ARC WM Research Methods theme will work with the project leads to maximise impact of this study.
Dates: May 2020 - May 2021
Monetary incentives have been widely used in healthcare to encourage certain behaviours and practices. Previous evidence has shown they are generally successful, but they can lead to gaming of the system and unintended consequences for other outcomes. Hospitals in England receive a financial incentive to vaccinate frontline staff against influenza each autumn. Additional funds are released to Hospital Trusts if they reach a pre-specified target for the proportion of frontline staff who receive an influenza vaccination.
Policy and Practice Partners:
University Hospitals Birmingham NHS Foundation Trust.
NHS England, others TBC.
Aims and Objectives:
We aim to examine how reported seasonal influenza vaccination rates across NHS England have varied in response to organisational monetary incentives to reach certain proportions of frontline staff vaccinated. From this we will determine whether there is an effect on vaccination rates at the threshold that triggers the payment of the incentive.
For the last few years NHS Trusts have been offered an increasing amount of money the more staff they vaccinate, in most years Trusts received 100% of the cash if they reached 70% of more of their staff, although this has changed in some years. Using publicly reported data on overall vaccination rates for seven years (2012/13 - 2018/19), we investigated how the proportion of staff vaccinated each year has varied in response to changes in the incentive rate. We used a threshold analysis designed to detect any discontinuity in the data at the threshold vaccination rate for a given year.
The analysis showed that there is significant evidence of discontinuity at the target threshold for frontline staff influenza vaccinations in the most recent years: 2016/2017, 2017/2018 and 2018/2019. This discontinuity only becomes apparent over the last two months of the six month vaccination season, and it is found at the threshold level for the particular year.
We found evidence that hospitals react to the existence of the threshold. Threshold specific effects are of interest to policy-makers because they show either that hospitals are predisposed to make a special effort selectively when they are close to meeting the target, or that they are gaming the system. Both of these are sub-optimal. Gaming is bad because it rewards dishonesty. Actions taken selectively around the threshold is bad because more can be gained by shifting the whole distribution than by marginal changes around the threshold. This is a case study that shows that it would be a straightforward matter to incorporate measurements of threshold effects into all quality metrics at the institutional level. If threshold effects are detected, then the incentive needs to be more carefully designed, for example by using an incremental approach.
Implications for Implementation:
This is a case study that shows that incentives are subject to threshold effects. It strongly suggests that examination of threshold effects should be incorporated routinely when monetary incentives are used to try to improve performance in public services. It also strongly suggests that policy-makers have an inflated view of the effectiveness of incentives, which improve performance around the incentive rather than across the entire distribution of performance. These findings should influence practice in the Department of Health and Social Care, and in organisations such as NHS Improvement, and their counter-parts in other parts of the world.