Elsevier

The Lancet

Volume 387, Issue 10027, 9–15 April 2016, Pages 1573-1586
The Lancet

Review
Increasing value and reducing waste in biomedical research: who's listening?

https://doi.org/10.1016/S0140-6736(15)00307-4Get rights and content

Summary

The biomedical research complex has been estimated to consume almost a quarter of a trillion US dollars every year. Unfortunately, evidence suggests that a high proportion of this sum is avoidably wasted. In 2014, The Lancet published a series of five reviews showing how dividends from the investment in research might be increased from the relevance and priorities of the questions being asked, to how the research is designed, conducted, and reported. 17 recommendations were addressed to five main stakeholders—funders, regulators, journals, academic institutions, and researchers. This Review provides some initial observations on the possible effects of the Series, which seems to have provoked several important discussions and is on the agendas of several key players. Some examples of individual initiatives show ways to reduce waste and increase value in biomedical research. This momentum will probably move strongly across stakeholder groups, if collaborative relationships evolve between key players; further important work is needed to increase research value. A forthcoming meeting in Edinburgh, UK, will provide an initial forum within which to foster the collaboration needed.

Introduction

More than 30 years ago, the adverse clinical consequences of biased under-reporting of research were clearly documented1 and non-publication of research remains hugely problematic.2, 3, 4, 5 Non-publication is bad value for funders, who could double research output by ensuring all the funded studies are published, and this situation puts patients and clinicians at a substantial disadvantage in making informed decisions about health care.6 Trial registration, supported by the International Committee of Medical Journal Editors (ICMJE),7 has helped to address this problem8, 9 although this solution is clearly not a panacea.10, 11 Other related initiatives, such as the Alltrials initiative and the Institute of Medicine's report on data sharing12 are working to ensure that the results of all trials are reported and that their data are made available.

Chalmers and Glasziou13 estimated in 2009 that 85% of research funding was being avoidably wasted across the entire biomedical research range (eg, clinical, health services, and basic science). Evidence of the extent and avoidability of waste in research production at each stage of the authors' four stage model has grown, and has confirmed imbalanced research question selection,14 poor study design15, 16 and execution, non-publication,17 and poor reporting18 and some have suggested that a more fundamental reassessment is needed in how research priorities are developed and pursued.19, 20 In addition to 228 citations as of Sept 11, 2015, the 2009 paper13 led National Institute of Health Research (NIHR) in England to establish a working group to monitor and plan actions, with regular meetings and an annual closed conference. The NIHR's Adding Value in Research programme had an additional stage (figure 1) aiming to ensure that NIHR funded research: addresses questions relevant to clinicians, patients, and the public; uses appropriate design and methods; is delivered efficiently; results in accessible full publication; and produces unbiased and useable reports. NIHR developed a quality improvement method21 for these five stages to identify common themes and examples of good practice across their programmes. For example, since 2013, NIHR has required applicants for support of new primary research to reference an existing systematic review “as well as including reference to any relevant literature published subsequent to that systematic review” or when no such systematic review exists, applicants should review the relevant evidence (with a method that systematically identifies, critically appraises, and combines the evidence), which “must also include reference to relevant on-going studies, eg, from trial registries”.22

In 2014 The Lancet published a Series (“Increasing value: reducing waste”)23, 24, 25, 26, 27 extending the 2009 analysis from 4 to 50 journal pages, with more than 40 authors focused on the five NIHR stages. As the Commissioning Editors noted: “Our belief is that research funders, scientific societies, school and university teachers, professional medical associations, and scientific publishers (and their editors) can use this Series as an opportunity to examine more forensically why they are doing what they do…and whether they are getting the most value for the time and money invested in science.”28

The Series, and an accompanying symposium,29 provided a voluminous body of evidence for the issues in biomedical research, along with 17 recommendations (table 1) to help to increase value, covering funders, regulators, journals, academic institutions, and researchers. These issues include (although they are not limited to) whether planned research met the needs of end users.30, 31, 32

Initial media attention included coverage by several newspapers including the leading German paper, Der Spiegel,33 although almost no response has been made from German researchers or organisations (Antes G, German Cochrane Centre, personal communication). Several research funders responded through meetings, working parties, and some changes of processes. In the year since their publication, the five articles have been downloaded 46 596 times from The Lancet and ScienceDirect websites. The five Series papers have already been cited 113 times (Scopus); were all in the top 5% of all articles indexed by Scopus; and their alternative metric scores (used to measure social media) all ranked more than the 98th percentile (of more than 3 million articles scored) including 589 tweets (about 20% of which were by health-care professionals).

This follow-up Review offers an overview of the initial stimulus of the Series. Before the assessment, a protocol was developed outlining the key players and the methods of our investigation, including sampling frames (panel 1). The primary focus was to assess what funders, regulators, journals, academic institutions, and researchers are doing, and plan to do, to address waste in biomedical research.

Section snippets

Funders

A few funders have already responded to the Series. In May, 2014, the French Institute of Health and Medical Research, Institut National de la Santé et de la Recherche Médicale (INSERM), in conjunction with the Enhancing the Quality and Transparency of Health Research (EQUATOR) network, organised a 1-day conference in Paris on “Improving reporting to decrease the waste of research” with the director of the Wellcome Trust and NIHR's Health Technology Assessment programme among the speakers

Regulators

Regulators can help with this goal by not providing ethics approval of protocols that are scientifically poor, which would mean that these protocols are also ethically inadequate. For example, the guidance for researchers issued by the newly established Health Research Authority (HRA)49 in the UK now states: “Any project should build on a review of current knowledge. Replication to check the validity of previous research is justified, but unnecessary duplication is unethical.”

Conversely,

Journals

In view of the fact that more than half of the reports of clinical trials do not set their results in the context of the totality of evidence,27 journals have much work to do to improve this situation. Journals can progress by providing specific guidance on their websites about this crucial feature and by providing similar guidance to peer reviewers. In response to the Series, The Lancet strengthened the journal's requirement to put research into context (table 1).53 From the beginning of this

Academic institutions

We are aware of very little explicit attention by academic institutions to the Lancet Series. An exception has been in Iran, where a group of academics are running a series of workshops on the Lancet series. Two workshops on “Biomedical research: increasing value, reducing waste” were run in February, 2015, for Directors of Clinical Research Centers, research vice chancellors, and Director Generals of Research Affairs of Medical Universities of North West Universities of Iran. A final national

Researchers

Motivated by the principle that initiation of research without a systematic review of already known evidence is unethical, unscientific, and wasteful, particularly when the research involves people or animals, three Scandinavian researchers66 convened and inaugurated an international Evidence-Based Research Network at the end of 2014. This network will urge funders, regulators, researchers, academic institutions, and journals to implement the changes needed to promote evidence-based research.

Looking to the future

The overall response to the 2014 Series might be summed up as some gratifying actions, but much more needs to be done to reduce research waste than at present. From a bibliometric and social media perspective, the Series has gained some traction, which is encouraging. Recognition of the problems described in the Series, and dialogue about the recommendations and possible ways to monitor progress are important first steps. However, if researchers are to avoid the well known issue of not

References (79)

  • C Barnes et al.

    Impact of an online writing aid tool for writing a randomized trial report: the COBWEB (Consort-based WEB tool) randomized controlled trial

    BMC Med

    (2015)
  • E Hemminki

    Study of information submitted by drug companies to licensing authorities

    BMJ

    (1980)
  • RJ Simes

    Publication bias: the case for an international registry of clinical trials

    J Clin Oncol

    (1986)
  • SJ Chapman et al.

    Discontinuation and non-publication of surgical randomised controlled trials: observational study

    BMJ

    (2014)
  • KL Hudson et al.

    Sharing and reporting the results of clinical trials

    JAMA

    (2015)
  • ES Sena et al.

    Publication bias in reports of animal stroke studies leads to major overstatement of efficacy

    PLoS Biol

    (2010)
  • A Liberati

    An unfinished trip through uncertainties

    BMJ

    (2004)
  • C De Angelis et al.

    Clinical trial registration: a statement from the International Committee of Medical Journal Editors

    CMAJ

    (2004)
  • DA Zarin et al.

    Trial Registration at ClinicalTrials.gov between May and October 2005

    N Engl J Med

    (2005)
  • CA Emdin et al.

    Association between randomised trial evidence and global burden of disease: cross sectional study (Epidemiological Study of Randomized Trials–ESORT)

    BMJ

    (2015)
  • S Mathieu et al.

    Comparison of registered and published primary outcomes in randomized controlled trials

    JAMA

    (2009)
  • S Ramagopalan et al.

    Prevalence of primary outcome changes in clinical trials registered on ClinicalTrials.gov: a cross-sectional study

    F1000 Res

    (2014)
  • Discussion framework for clinical trial data sharing: guiding principles, elements, and activities

  • S Crowe et al.

    Patients', clinicians' and the research communities' priorities for treatment research: there is an important mismatch

    Res Involv Engagem

    (2015)
  • Y Yordanov et al.

    Avoidable waste of research related to inadequate methods in clinical trials

    BMJ

    (2015)
  • JA Hirst et al.

    The need for randomization in animal trials: an overview of systematic reviews

    PLoS One

    (2014)
  • A Blümle et al.

    Fate of clinical research studies after ethical approval—follow-up of study protocols until publication

    PLoS One

    (2014)
  • E Mayo-Wilson et al.

    Reporting quality of social and psychological intervention trials: a systematic review of reporting guidelines and trial publications

    PLoS One

    (2013)
  • DW Light et al.

    Good Pharma: the public-health model of the Mario Negri Institute

    (2015)
  • C Leaf

    The truth in small doses: why we're losing the war on cancer—and how to win it

    (2013)
  • Adding value in research quality improvement template

  • Guidance notes for applicants that ensure all primary research is informed by a review of the existing literature

  • R Al-Shahi Salman et al.

    Increasing value and reducing waste in biomedical research regulation and management

    Lancet

    (2014)
  • Symposium on the Lancet Series on Research: Increasing value, reducing waste

  • L Frank et al.

    The PCORI perspective on patient-centered outcomes research

    JAMA

    (2014)
  • P Williamson et al.

    Driving up the quality and relevance of research through the use of agreed core outcomes

    J Health Serv Res Policy

    (2012)
  • PS Tugwell et al.

    Deliberative dialogue on developing consensus on measurement and presentation of patient important outcomes in systematic reviews: a preconference meeting at OMERACT 12

    J Rheumatol

    (2015)
  • N Kuhrt

    Systemkritik: Wissenschaftselite beklagt zu viel Forschungsmüll

  • (AIM or “Core Clinical”) Journal Titles

  • Cited by (292)

    View all citing articles on Scopus
    View full text