Elsevier

The Lancet

Volume 383, Issue 9913, 18–24 January 2014, Pages 257-266
The Lancet

Series
Increasing value and reducing waste: addressing inaccessible research

https://doi.org/10.1016/S0140-6736(13)62296-5Get rights and content

Summary

The methods and results of health research are documented in study protocols, full study reports (detailing all analyses), journal reports, and participant-level datasets. However, protocols, full study reports, and participant-level datasets are rarely available, and journal reports are available for only half of all studies and are plagued by selective reporting of methods and results. Furthermore, information provided in study protocols and reports varies in quality and is often incomplete. When full information about studies is inaccessible, billions of dollars in investment are wasted, bias is introduced, and research and care of patients are detrimentally affected. To help to improve this situation at a systemic level, three main actions are warranted. First, academic institutions and funders should reward investigators who fully disseminate their research protocols, reports, and participant-level datasets. Second, standards for the content of protocols and full study reports and for data sharing practices should be rigorously developed and adopted for all types of health research. Finally, journals, funders, sponsors, research ethics committees, regulators, and legislators should endorse and enforce policies supporting study registration and wide availability of journal reports, full study reports, and participant-level datasets.

Introduction

In 2010, Alessandro Liberati explained the difficulties he encountered when he had to make decisions about his treatment for multiple myeloma: “When I had to decide whether to have a second bone-marrow transplant, I found there were four trials that might have answered my questions, but I was forced to make my decision without knowing the results because, although the trials had been completed some time before, they had not been properly published….I believe that research results must be seen as a public good that belongs to the community—especially patients.”1 The benefits of health research can only be realised when the study methods and results are fully disseminated in a timely and unbiased manner.2 Availability of full information about study methods enables critical appraisal, interpretation of study results, and appropriate replication. Proper reporting of results can improve clinical practice and policy, prevent unnecessary duplication, and help to inform present and future research. Availability of participant-level data enables ancillary research and independent reanalysis of study results.

Despite advances in dissemination of study information, half of health-related studies remain unreported,3 and few study protocols and participant-level datasets are accessible. Inaccessibility of research is detrimental to care of patients and wastes much of the US$240 billion annual worldwide expenditure on health research.4 In this report, we document the extent and effect of non-dissemination and selective reporting of health research, and examine the options to reduce the waste and harms arising from inaccessible study information.

Section snippets

Access to primary reports

A published primary report is traditionally the main way by which research is communicated to the scientific community. Because unreported studies do not contribute to knowledge, they do not provide returns on the investment of research resources or the contributions of participants. For example, only half the health-related studies funded by the European Union between 1998 and 2006—an expenditure of €6 billion—led to identifiable reports.5 In the case of oseltamivir, unreported phase 3

Access to all study methods and results

Although the reporting of all studies has a major role in reductions in bias and improvements in transparency, journal publications alone are insufficient. Reporting of study methods and results is frequently incomplete and selective in journal articles, challenging their traditional role as the sole source of research information.32, 33 Produced by industry sponsors, a clinical study report is the most complete final report of study conduct and results, and contains the study protocol as an

Access to participant-level data

Beyond the compelling rationale for dissemination of primary reports, protocols, and full study reports, sharing of participant-level data has many benefits. First, errors, selective reporting, and fraud can be identified and deterred when others can verify statistical properties and calculations using participant-level data. A substantial proportion of reported studies have statistical errors,55, 56 and willingness to share data has been positively correlated with methodological quality.57

Recommendation 1

We propose three main recommendations to improve accessibility to full information from preclinical and clinical studies. First, institutions and funders should adopt performance metrics that recognise full dissemination of research. Incentives are needed to encourage investigators to complete and submit primary reports. Rather than focusing on total numbers of published reports, reviews of academic performance should explicitly take into account the proportion of a researcher's initiated

References (122)

  • CD De Angelis et al.

    Is this clinical trial fully registered? A statement from the International Committee of Medical Journal Editors

    Lancet

    (2005)
  • A Liberati

    So many questions, so few answers. Interview by Les Olson

    Bull World Health Organ

    (2010)
  • I Chalmers

    Underreporting research is scientific misconduct

    JAMA

    (1990)
  • F Song et al.

    Dissemination and publication of research findings: an updated review of related biases

    Health Technol Assess

    (2010)
  • T Jefferson et al.

    Neuraminidase inhibitors for preventing and treating influenza in healthy adults and children

    Cochrane Database Syst Rev

    (2012)
  • Scherer R, Meerpohl JJ, Schmucker C, Schwarzer G, von Elm E. Full publication of studies presented at biomedical...
  • P Suñé et al.

    Positive outcomes influence the rate and time to publication, but not the impact factor of publications of clinical trial results

    PLoS One

    (2013)
  • G ter Riet et al.

    Publication bias in laboratory animal research: a survey on magnitude, drivers, consequences and potential solutions

    PLoS One

    (2012)
  • ES Sena et al.

    Publication bias in reports of animal stroke studies leads to major overstatement of efficacy

    PLoS Biol

    (2010)
  • JS Ross et al.

    Trial publication after registration in ClinicalTrials.Gov: a cross-sectional analysis

    PLoS Med

    (2009)
  • S Hopewell et al.

    Time to publication for results of clinical trials

    Cochrane Database Syst Rev

    (2007)
  • I Chalmers et al.

    How to increase value and reduce waste when research priorities are set

    Lancet

    (2014)
  • JPA Ioannidis et al.

    Improving value and reducing waste in research design, conduct, and analysis

    Lancet

    (2014)
  • N McGauran et al.

    Reporting bias in medical research—a narrative review

    Trials

    (2010)
  • PC Gøtzsche

    Why we need easy access to all data from all clinical trials and how to accomplish it

    Trials

    (2011)
  • B Hart et al.

    Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses

    BMJ

    (2012)
  • D Eyding et al.

    Reboxetine for acute treatment of major depression: systematic review and meta-analysis of published and unpublished placebo and selective serotonin reuptake inhibitor controlled trials

    BMJ

    (2010)
  • CG Begley et al.

    Drug development: raise standards for preclinical cancer research

    Nature

    (2012)
  • F Prinz et al.

    Believe it or not: how much can we rely on published data on potential drug targets?

    Nat Rev Drug Discov

    (2011)
  • HB van der Worp et al.

    Can animal models of disease reliably inform human studies?

    PLoS Med

    (2010)
  • P Boffetta et al.

    False-positive results in cancer epidemiology: a plea for epistemological modesty

    J Natl Cancer Inst

    (2008)
  • L Chan et al.

    Towards open and equitable access to research and knowledge for development

    PLoS Med

    (2011)
  • I Sample

    Harvard University says it can't afford journal publishers' prices

  • BC Björk et al.

    Open access to the scientific journal literature: situation 2009

    PLoS One

    (2010)
  • J Xia et al.

    Five large Chinese biomedical bibliographic databases: accessibility and coverage

    Health Info Libr J

    (2008)
  • A Morrison et al.

    The effect of English-language restriction on systematic review-based meta-analyses: a systematic review of empirical studies

    Int J Technol Assess Health Care

    (2012)
  • T Wu et al.

    Randomized trials published in some Chinese journals: how many are randomized?

    Trials

    (2009)
  • S Hopewell et al.

    The quality of reports of randomised trials in 2000 and 2006: comparative study of articles indexed in PubMed

    BMJ

    (2010)
  • K Dwan et al.

    Comparison of protocols and registry entries to published reports for randomised controlled trials

    Cochrane Database Syst Rev

    (2011)
  • International Conference on Harmonisation of Technical Requirements for Registration of Pharmaceuticals for Human Use. ICH harmonised tripartite guideline: structure and content of clinical study reports

  • P Doshi et al.

    Clinical study reports of randomised controlled trials: an exploratory review of previously confidential industry reports

    BMJ Open

    (2013)
  • B Wieseler et al.

    Impact of document type on reporting quality of clinical drug trials: a comparison of registry reports, clinical study reports, and journal publications

    BMJ

    (2012)
  • B Wieseler et al.

    Completeness of reporting of patient-relevant clinical trial outcomes: comparison of unpublished clinical study reports with publicly available data

    PLoS Med

    (2013)
  • A Blümle et al.

    Reporting of eligibility criteria of randomised trials: cohort study comparing trial protocols with subsequent articles

    BMJ

    (2011)
  • M Gandhi et al.

    Eligibility criteria for HIV clinical trials and generalizability of results: the gap between published reports and study protocols

    AIDS

    (2005)
  • P Doshi et al.

    The imperative to share clinical study reports: recommendations from the Tamiflu experience

    PLoS Med

    (2012)
  • PC Gøtzsche et al.

    Opening up data at the European Medicines Agency

    BMJ

    (2011)
  • A-W Chan et al.

    Research protocols: waiving confidentiality for the greater good

    BMJ

    (2006)
  • A-W Chan et al.

    Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles

    JAMA

    (2004)
  • JJ Kirkham et al.

    Bias due to changes in specified outcomes during the systematic review process

    PLoS One

    (2010)
  • Cited by (602)

    View all citing articles on Scopus
    View full text