Biotherapies in Solid Tumors: Are Negative Results Still of Low Priority for Publication?

In 2005, we performed the largest survey on clinical trials of biotherapies for all solid tumors and found indirect evidence of a publication bias: editors of medical journals were more prone to publish positive results independently from the quality of the studies. We collected data from 2003 to 2015 in 487 studies, and the publication bias previously described was not found in the years between 2010 and 2015: this could be related to changes and/or innovations in the guidelines and editorial policies of oncology journals occurred over the last years.

In 2005, we performed the largest survey on clinical trials of biologic and immunologic therapies (i.e., Biotherapy: inhibitors of the proliferative signals, antiangiogenic agents, differentiating agents, inhibitors of metastasis/invasiveness, drugs with multiple mechanisms of action; Immunotherapy: vaccines, cytokines, adoptive immunotherapy, antibodies stimulating the immune system, drugs with multiple immunologic mechanisms of action) for all solid tumors in order to review their content and quality (1); we found an indirect evidence of publication bias (2) as an attitude of high-impact factor medical journals to prefer the publication of positive results. We showed that publication was related to positive results independently from study quality; the observation period ranged from 1990 to 2002. This kind of publication bias (related to the reporting of positive outcome) suggested that negative results had a lower chance of being published, particularly in high-impact factor journals.

In non-oncologic research areas, there are direct and indirect pieces of evidence that this type of publication bias is increasing (3–5). This phenomenon might reduce the visibility of negative results and affect the clinical development of a drug (identification of clinical endpoints, selection of patients and therapies, management of toxicities) because this process depends on the interaction between positive and negative findings. Furthermore, oncologic research may be affected by other publication bias previously described (6–8).

Since then, we continued to collect data about biotherapies in solid tumors (general study characteristics, patient and disease factors, study methodology and quality) from selected oncology journals limiting our search to phase II/III clinical trials in colorectal, lung, breast, prostate cancer, pancreas, and melanoma from 2003 to 2015 through Medline searching. Four hundred eighty-seven clinical trials were identified. Quality of studies (quality index: QI) was determined through a modified ad hoc questionnaire (Table 1). Study evaluation methods were similar to the previous study (1); however, the impact factor was arbitrarily divided into three categories: <6, 6–10, >10. A QI<50 points was calculated for 36.7% of phase II and 28.2% for phase III or II/III studies. Results were defined positive when the authors showed significant improvements in activity and/or efficacy of a new treatment, negative when it was inactive or toxic or failed to show significant ameliorations vs the referral drugs.

TABLE 1

Table 1. Quality assessment questionnaire for phase II and III studies.

Here, we report on the association between quality, conclusions, and other explanatory variables of the studies and impact factor of the journals in recent years. In particular, the association between positive outcome and publication in high-impact factor journals was not found in the studies published between 2010 and 2015 (202 studies). In a multiple regression model performed to analyze the correlation between impact factor and some different explanatory variables of studies [quality (<50 vs ≥50), phase of the study (II vs III), academic/company supported (yes vs no), accrual time (≤3 years vs >3 years), study conclusions (positive vs negative)] the only variable found to be predictive of publication in a high-impact factor journal between 2010 and 2015 was the quality of the study (P = 0.012) and not the positive result (P = 0.178) (Table 2).

TABLE 2

Table 2. Multiple regression to analyze the relationship between explanatory variables of studies and publication in high-impact factor journals between 2010 and 2015.

One possible explanation of this specific outcome-related publication bias reduction may be related to changes and/or innovations in the guidelines and editorial policies (reception, evaluation, and publication of studies with negative results) of oncology journals occurred over the last years. Our report has some limitations: (i) it represents an indirect evidence about publication bias since the “true publication bias” is the difference between published and “unpublished” studies (i.e., discordance between trial in official registries and published trial) and (ii) our QI has not been independently validated by other research groups despite being reasonably a useful tool to resume the most important methodology characteristics of a clinical study.

New generation targeted therapies are changing the therapeutic management of advanced tumors and the interest on studying new therapeutic pathways as well as new drugs is increasing. We show an indirect evidence that editors and reviewers have improved in past few years their overall methodology when discussing and evaluating clinical studies on biotherapies.

Author Contributions

AO contributed in planning, analysis, discussing, and writing of this manuscript. AC and MC contributed in discussing and writing of the manuscript. CD and AS contributed in analysis and writing of the manuscript. AA contributed in discussing of the manuscript.

Conflict of Interest Statement

The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.

References

1. Ottaiano A, Mollo E, Di Lorenzo G, Pisano C, Di Maio M, Barletta E, et al. Prospective clinical trials of biotherapies in solid tumors: a 5-year survey. Cancer Immunol Immunother (2005) 54:44–50. doi:10.1007/s00262-004-0567-z

PubMed Abstract | CrossRef Full Text | Google Scholar

2. Ottaiano A, Castello G, Ascierto PA. Evidence of publication bias in clinical trials of biotherapies for solid tumors. Cancer (2005) 103:653. doi:10.1002/cncr.20835

CrossRef Full Text | Google Scholar

3. Easterbrook PJ, Berlin JA, Gopalan R, Matthews DR. Publication bias in clinical research. Lancet (1991) 337:867–72. doi:10.1016/0140-6736(91)90201-Y

PubMed Abstract | CrossRef Full Text | Google Scholar

4. Kjaergard LL, Als-Nielsen B. Association between competing interests and authors’ conclusions: epidemiological study of randomised clinical trials published in the BMJ. BMJ (2002) 325:249. doi:10.1136/bmj.325.7358.249

CrossRef Full Text | Google Scholar

5. Kicinski M. Publication bias in recent meta-analyses. PLoS One (2013) 8:e81823. doi:10.1371/journal.pone.0081823. eCollection 2013. Erratum in: PLoS One (2014) 9. doi:10.1371/annotation/a65c0f61-eb99-42f0-828b-5a8662bce4f7

CrossRef Full Text | Google Scholar

6. Ramsey S, Scoggins J. Commentary: practicing on the tip of an information iceberg? Evidence of underpublication of registered clinical trials in oncology. Oncologist (2008) 13:925–9. doi:10.1634/theoncologist.2008-0133

PubMed Abstract | CrossRef Full Text | Google Scholar

7. Takeda A, Loveman E, Harris P, Hartwell D, Welch K. Time to full publication of studies of anti-cancer medicines for breast cancer and the potential for publication bias: a short systematic review. Health Technol Assess (2008) 12:iii, ix–x, 1–46. doi:10.3310/hta12320

PubMed Abstract | CrossRef Full Text | Google Scholar

8. Peppercorn J, Blood E, Winer E, Partridge A. Association between pharmaceutical involvement and outcomes in breast cancer clinical trials. Cancer (2007) 109:1239–46. doi:10.1002/cncr.22528

PubMed Abstract | CrossRef Full Text | Google Scholar