Skip to main content

Publication and related bias in health services and delivery research (HSDR)

What is this project about?

It is important that the people who are responsible for making important decisions about the ways our health services are delivered have the best and most reliable evidence available to them to make sure that they make the right choices. Sometimes, however, the evidence available doesn’t tell the ‘full story’ or the evidence available may be biased. One problem is ‘publication bias’, which happens when only certain studies, usually those which are more exciting or are ‘positive’ are published. There are other forms of bias, too. Sometimes, researchers might not get exciting or positive results, so they use other ways to analyse the data or analyse only parts of the full data set until they get a more interesting result. This is called ‘p-hacking’. There is also ‘outcome reporting bias’, which happens when a study collects lots of data, but only the ‘positive’ or interesting results are published. In this project, we use the term ‘publication bias’ broadly to cover these various forms of bias as well. It is important to study publication bias because if the people making decisions don’t have all the evidence available to them, they might make wrong decisions, which may have a negative effect on our health services and the way the services are delivered.

In clinical research, which focuses on medical treatments for patients, there has been a lot of research on publication bias and examples such as the non-publication of data showing harmful effects from drug trials has hit the headline. It is very important to make sure that there is no publication bias as treatment decision based on biased information might be wasteful or even harmful. Because of the potentially serious consequences, all clinical trials now have to be registered so that people making decisions about treatments have all the information available to them to make the right choices.

In Health Services and Delivery Research (HSDR), which looks at the best ways our services are delivered to patients, there hasn’t been much research on publication bias. This might be because health service delivery research is very complicated. In this study, we will look at publication bias in HSDR to see if it exists, what kinds of publication bias there are, and how we can detect or deal with it. We want to obtain and then share the information about publication bias so that researchers doing HSDR and users of findings from the research can take measures to make sure that the occurrence and impact of publication bias are minimised. Investigation of publication bias in HSDR is very important because its findings are often used to make policy decisions that will influence many people.

Overview of the project

The study has five work packages (WPs), which are all linked as shown in the picture below.

Overview of Project

The work packages include:
Work Package 1 (WP1): a systematic review of empirical and methodological studies concerning the occurrence, potential impact and/or methodology related to publication bias in HSDR and cognate fields to provide a summary of what is known from current literature.
Work Package 2 (WP2): an overview of systematic reviews of intervention and association studies in HSDR to describe current practice and potential challenges in assessing publication bias during evidence synthesis.
Work Package 3 (WP3): in-depth case studies to evaluate the applicability of different methods for detecting and mitigating publication bias in HSDR and to provide guidance for future research and practice.
Work Package 4 (WP4): a retrospective study to follow up the publication status of cohorts of HSDR studies to directly observe publication bias in HSDR
Work Package 5 (WP5): semi-structured interviews with health services researchers and commissioners, journal editors and other stakeholders to explore their perception and experience related to publication bias.

Meet the Team

Core Research and Administrative Team:
Dr Yen-Fu Chen (University of Warwick) - Principal Investigator, evidence synthesis and project management
Dr Iestyn Williams (University of Birmingham) - indepth interviews (WP5)
Dr Abimbola Ayorinde (University of Warwick) - evidence synthesis, data collections, day to day running of the project
Ms Sian Scogings (University of Warwick) - administrative support

Project Management Team:
Prof Richard Lilford (University of Warwick) - HSDR methodology and senior management
Prof Russell Mannion (University of Birmingham) - HSDR methodology and mixed methods
Prof Fujian Song (University of East Anglia) - expertise in publication bias and statistical advice
Dr Magdalena Skrybant (University of Birmingham) - Patient and Public Involvement and Engagement (PPIE) Lead

Study Steering Committee:
Prof Stephen Sutton (University of Cambridge) - Chair of the Study Steering Committee
Revd Barry Clark (PPI representative) - patient and public opinion and input
Dr Kaveh Shojania (University of Toronto) - HSDR methodolody and journal editor perspective
Prof Jeremy Grimshaw (University of Ottawa) - HSDR methodology and evidence synthesis perspective
Prof Timothy Hofer (University of Michigan) - HSDR methodology and health professional perspective
Dr Christopher Chiswell (Birmingham Children's Hospital) - health professional and service provider perspective
Mr Tim Sacks (East Leicestershire and Rutland Clinical Commissioning Group) - service provider and management perspective)
Prof James Thomas (University College London) - evidence synthesis and policy perspective
Prof Richard Lilford (University of Warwick) - HSDR methodology and senior management

NIHR logo

Link to NIHR project website

Project News:

The HSRUK Symposium, which is the 'voice of Health Services Research' was held in Nottingham on 6-7 July 2017. Researchers from our team were represented at the event. In addition to presenting a poster, which included information about our project, Yen-Fu Chen, Principal Investigator, presented in the 'Methods' section. His talk was entitled "Publication and related biases in health services and delivery research: the dog that did not bark?"


Please find the poster here

Contact us: if you would like to know more about this project and/or want to share your experience or idea with us, please contact Dr Yen-Fu Chen or Dr Abimbola Ayorinde