Skip to main content
SpringerLink
Log in

Quality of evidence matters: is it well reported and interpreted in infertility journals?

  • Commentary
  • Published:
Journal of Assisted Reproduction and Genetics Aims and scope Submit manuscript

Abstract

Purpose

To evaluate if the authors of published systematic reviews (SRs) reported the level of quality of evidence (QoE) in the top 5 impact factor infertility journals and to analyze if they used an appropriate wording to describe it.

Methods

This is a cross-sectional study. We searched in PubMed for SRs published in 2017 in the five infertility journals with the highest impact factor. We analyzed the proportion of SRs published in the top 5 impact factor infertility journals that reported the SRs’ QoE, and the proportion of those SRs in which authors used consistent wording to describe QoE and magnitude of effect.

Results

The QoE was reported in only 21.4% of the 42 included SRs and in less than 10% of the abstracts. Although we did not find important differences in the report of QoE of those that showed statistically significant differences or not, p value was associated with the wording chosen by the authors. We found inconsistent reporting of the size the effect estimate in 54.8% (23/42) and in the level of QoE in 92.9% (39/42). Whereas the effect size was more consistently expressed in studies with statistically significant findings, QoE was better expressed in those cases in which the p value was over 0.05.

Conclusion

We found that in 2017, less than 25% of the authors reported the overall QoE when publishing SRs. Authors focused more on statistical significance as a binary concept than on methodological limitations like study design, imprecision, indirectness, inconsistency, and publication bias. Authors should make efforts to report the QoE and interpret results accordingly.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Murad MH, Asi N, Alsawas M, Alahdab F. New evidence pyramid. Evid Based Med. 2016;21(4):125–7. https://doi.org/10.1136/ebmed-2016-110401.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Hultcrantz M, Rind D, Akl EA, Treweek S, Mustafa RA, Iorio A, et al. The GRADE rWorking Group clarifies the construct of certainty of evidence. J Clin Epidemiol. 2017;87:4–13. https://doi.org/10.1016/j.jclinepi.2017.05.006.

    Article  PubMed  PubMed Central  Google Scholar 

  3. Guyatt G, Oxman AD, Akl EA, Kunz R, Vist G, Brozek J, et al. GRADE guidelines: 1. Introduction-GRADE evidence profiles and summary of findings tables. J Clin Epidemiol. 2011;64(4):383–94. https://doi.org/10.1016/j.jclinepi.2010.04.026.

    Article  PubMed  Google Scholar 

  4. Higgins J, Green S, (editors). Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. 2011.

  5. Glujovsky D, Sueldo CE, Borghi C, Nicotra P, Andreucci S, Ciapponi A. Misleading reporting and interpretation of results in major infertility journals. Fertil Steril. 2016. https://doi.org/10.1016/j.fertnstert.2015.12.134.

    Article  PubMed  Google Scholar 

  6. Schünemann HJ, Vist GE, Higgins JPT, Santesso N, Deeks JJ, Glasziou P et al. Chapter 15: Interpreting results and drawing conclusions. In: Higgins J, Thomas J, Chandler J, Cumpston M, Li T, Page M et al. editors. Cochrane handbook for systematic reviews of interventions version 6 [updated September 2018]: Cochrane. 2018.

  7. Cochrane Effective Practice and Organisation of Care (EPOC). Reporting the effects of an intervention in EPOC reviews. EPOC Resources for review authors (Version: 24 August 2017). 2017. http://epoc.cochrane.org/sites/epoc.cochrane.org/files/public/uploads/Resources-for-authors2017/how_to_report_the_effects_of_an_intervention.pdf. Accessed 19/12/2017.

  8. Reporting results in CKT reviews (using material adapted from EPOC and CCCR). 2017. http://kidneyandtransplant.cochrane.org/sites/kidneyandtransplant.cochrane.org/files/public/uploads/Resources/reporting_results_in_ckt_reviews_2017.pdf. Accessed 19/12/2017.

  9. Amrhein V, Greenland S, McShane B. Scientists rise up against statistical significance. Nature. 2019;567(7748):305–7. https://doi.org/10.1038/d41586-019-00857-9.

    Article  CAS  PubMed  Google Scholar 

  10. Ciapponi A, Glujovsky D, Comande D, Bardach A, editors. Do Cochrane systematic reviews report results integrating certainty of evidence and effect size? 25th Cochrane Colloquium. Scotland: Edinburgh; 2018.

    Google Scholar 

  11. Higgins J, Thomas J, Cumpston M, Chandler J, Li T, Page M, et al. Cochrane handbook for systematic reviews of interventions version 6: DRAFT. 2018.

  12. Glujovsky D, Riestra B, Coscia A, Boggino C, Comande D, Ciapponi A. Assessment of research quality in major infertility journals. Fertil Steril. 2012;98(6):1539–43. https://doi.org/10.1016/j.fertnstert.2012.08.018.

    Article  PubMed  Google Scholar 

  13. Scimago Journal & Country Rank [Portal]. SCImago, (n.d.). SJR. 2019. https://www.scimagojr.com/journalrank.php. Accessed 01/15/2019.

  14. CiteFactor. 2019. https://www.citefactor.org/. Accessed 01/15/2019.

  15. Reveiz L, Cortes-Jofre M, Asenjo Lobos C, Nicita G, Ciapponi A, Garcia-Dieguez M, et al. Influence of trial registration on reporting quality of randomized trials: study from highest ranked journals. J Clin Epidemiol. 2010;63(11):1216–22. https://doi.org/10.1016/j.jclinepi.2010.01.013.

    Article  CAS  PubMed  Google Scholar 

  16. Database of Abstracts of Reviews of Effects (DARE): quality-assessed reviews. York (UK): Centre for Reviews and Dissemination (UK). 2019. https://www.ncbi.nlm.nih.gov/books/NBK285222/. Accessed 01/15/2019.

  17. Covidence systematic review software. Veritas Health Innovation: Melbourne. www.covidence.org.

  18. McGrath TA, McInnes MDF, van Es N, Leeflang MMG, Korevaar DA, Bossuyt PMM. Overinterpretation of research findings: evidence of “spin” in systematic reviews of diagnostic accuracy studies. Clin Chem. 2017;63(8):1353–62. https://doi.org/10.1373/clinchem.2017.271544.

    Article  PubMed  Google Scholar 

  19. Lumbreras B, Parker LA, Porta M, Pollan M, Ioannidis JP, Hernandez-Aguado I. Overinterpretation of clinical applicability in molecular diagnostic research. Clin Chem. 2009;55(4):786–94. https://doi.org/10.1373/clinchem.2008.121517.

    Article  CAS  PubMed  Google Scholar 

  20. Ochodo EA, de Haan MC, Reitsma JB, Hooft L, Bossuyt PM, Leeflang MM. Overinterpretation and misreporting of diagnostic accuracy studies: evidence of “spin”. Radiology. 2013;267(2):581–8. https://doi.org/10.1148/radiol.12120527.

    Article  PubMed  Google Scholar 

  21. Shea BJ, Reeves BC, Wells G, Thuku M, Hamel C, Moran J, et al. AMSTAR 2: a critical appraisal tool for systematic reviews that include randomised or non-randomised studies of healthcare interventions, or both. BMJ. 2017;358. https://doi.org/10.1136/bmj.j4008.

  22. Moher D, Liberati A, Tetzlaff J, Altman DG. Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. PLoS Med. 2009;6(7):e1000097. https://doi.org/10.1371/journal.pmed.1000097.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgments

The authors would like to thank Heike Thiel for the review of the paper.

Author information

Authors and Affiliations

  1. Argentine Cochrane Centre, Institute for Clinical Effectiveness and Health Policy (IECS), Center for Research in Epidemiology and Public Health, National Scientific and Technical Research Council (CONICET), Ravignani 2024, C1414CPV, Buenos Aires, Argentina

    Demian Glujovsky, Ariel Bardach, Daniel Comandé & Agustín Ciapponi

  2. Center for Studies in Genetics and Reproduction (CEGYR), Buenos Aires, Argentina

    Demian Glujovsky & Carlos E. Sueldo

  3. Centro de Estudios en Genética y Reproducción (CEGYR), Viamonte 1432, C1055ABB, Buenos Aires, Argentina

    Demian Glujovsky

  4. Obstetrics and Gynecology Department, University of California San Francisco-Fresno, 155 N Fresno St Suite 233, Fresno, CA, 93701, USA

    Carlos E. Sueldo & Agustín Ciapponi

  5. Aventura, FL, USA

    María del Pilar Valanzasca

Authors
  1. Demian Glujovsky
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Carlos E. Sueldo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ariel Bardach
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. María del Pilar Valanzasca
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Daniel Comandé
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Agustín Ciapponi
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Demian Glujovsky.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Glujovsky, D., Sueldo, C.E., Bardach, A. et al. Quality of evidence matters: is it well reported and interpreted in infertility journals?. J Assist Reprod Genet 37, 263–268 (2020). https://doi.org/10.1007/s10815-019-01663-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10815-019-01663-y

Keywords

Search

Navigation