Skip to main content
Top
Published in: Systematic Reviews 1/2014

Open Access 01-12-2014 | Research

Conflicts of interest and critiques of the use of systematic reviews in policymaking: an analysis of opinion articles

Authors: Susan R Forsyth, Donna H Odierna, David Krauth, Lisa A Bero

Published in: Systematic Reviews | Issue 1/2014

Login to get access

Abstract

Background

Strong opinions for or against the use of systematic reviews to inform policymaking have been published in the medical literature. The purpose of this paper was to examine whether funding sources and author financial conflicts of interest were associated with whether an opinion article was supportive or critical of the use of systematic reviews for policymaking. We examined the nature of the arguments within each article, the types of disclosures present, and whether these articles are being cited in the academic literature.

Methods

We searched for articles that expressed opinions about the use of systematic reviews for policymaking. We included articles that presented opinions about the use of systematic reviews for policymaking and categorized each article as supportive or critical of such use. We extracted all arguments regarding the use of systematic reviews from each article and inductively coded each as internal or external validity argument, categorized disclosed funding sources, conflicts of interest, and article types, and systematically searched for undisclosed financial ties. We counted the number of times each article has been cited in the “Web of Science.” We report descriptive statistics.

Results

Articles that were critical of the use of systematic reviews (n = 25) for policymaking had disclosed or undisclosed industry ties 2.3 times more often than articles that were supportive of the use (n = 34). We found that editorials, comments, letters, and perspectives lacked published disclosures nearly twice as often (60% v. 33%) as other types of articles. We also found that editorials, comments, letters, and perspectives were less frequently cited in the academic literature than other article types (median number of citations = 5 v. 19).

Conclusions

It is important to consider whether an article has industry ties when evaluating the strength of the argument for or against the use of systematic reviews for policymaking. We found that journal conflict of interest disclosures are often inadequate, particularly for editorials, comments, letters, and perspectives and that these articles are being cited as evidence in the academic literature. Our results further suggest the need for more consistent and complete disclosure for all article types.
Appendix
Available only for authorised users
Literature
1.
go back to reference Bero LA, Jadad AR: How consumers and policymakers can use systematic reviews for decision making. Ann Intern Med. 1997, 127: 37-42. 10.7326/0003-4819-127-1-199707010-00007.CrossRefPubMed Bero LA, Jadad AR: How consumers and policymakers can use systematic reviews for decision making. Ann Intern Med. 1997, 127: 37-42. 10.7326/0003-4819-127-1-199707010-00007.CrossRefPubMed
2.
go back to reference Sweet M, Moynihan R, Fund MM: Improving Population Health: The Uses of Systematic Reviews. 2007, New York: Milbank Memorial Fund Sweet M, Moynihan R, Fund MM: Improving Population Health: The Uses of Systematic Reviews. 2007, New York: Milbank Memorial Fund
4.
go back to reference Fox DM: Systematic reviews and health policy: the influence of a project on perinatal care since 1988. Milbank Q. 2011, 89: 425-449. 10.1111/j.1468-0009.2011.00635.x.CrossRefPubMedPubMedCentral Fox DM: Systematic reviews and health policy: the influence of a project on perinatal care since 1988. Milbank Q. 2011, 89: 425-449. 10.1111/j.1468-0009.2011.00635.x.CrossRefPubMedPubMedCentral
5.
go back to reference Bailar JC: Passive smoking, coronary heart disease, and meta-analysis. N Engl J Med. 1999, 340: 958-959. 10.1056/NEJM199903253401211.CrossRefPubMed Bailar JC: Passive smoking, coronary heart disease, and meta-analysis. N Engl J Med. 1999, 340: 958-959. 10.1056/NEJM199903253401211.CrossRefPubMed
6.
go back to reference Chalmers I, Hedges LV, Cooper H: A brief history of research synthesis. Eval Health Prof. 2002, 25: 12-37. 10.1177/0163278702025001003.CrossRefPubMed Chalmers I, Hedges LV, Cooper H: A brief history of research synthesis. Eval Health Prof. 2002, 25: 12-37. 10.1177/0163278702025001003.CrossRefPubMed
7.
go back to reference Meerpohl JJ, Herrle F, Antes G, von Elm E: Scientific value of systematic reviews: survey of editors of core clinical journals. PLoS One. 2012, 7: e35732-10.1371/journal.pone.0035732.CrossRefPubMedPubMedCentral Meerpohl JJ, Herrle F, Antes G, von Elm E: Scientific value of systematic reviews: survey of editors of core clinical journals. PLoS One. 2012, 7: e35732-10.1371/journal.pone.0035732.CrossRefPubMedPubMedCentral
8.
go back to reference Greenhalgh T, Potts HW, Wong G, Bark P, Swinglehurst D: Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method. Milbank Q. 2009, 87: 729-788. 10.1111/j.1468-0009.2009.00578.x.CrossRefPubMedPubMedCentral Greenhalgh T, Potts HW, Wong G, Bark P, Swinglehurst D: Tensions and paradoxes in electronic patient record research: a systematic literature review using the meta-narrative method. Milbank Q. 2009, 87: 729-788. 10.1111/j.1468-0009.2009.00578.x.CrossRefPubMedPubMedCentral
9.
go back to reference Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008, 8: 45-10.1186/1471-2288-8-45.CrossRefPubMedPubMedCentral Thomas J, Harden A: Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol. 2008, 8: 45-10.1186/1471-2288-8-45.CrossRefPubMedPubMedCentral
10.
go back to reference Barnett-Page E, Thomas J: Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009, 9: 59-10.1186/1471-2288-9-59.CrossRefPubMedPubMedCentral Barnett-Page E, Thomas J: Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol. 2009, 9: 59-10.1186/1471-2288-9-59.CrossRefPubMedPubMedCentral
12.
go back to reference Lundh A, Sismondo S, Lexchin J, Busuioc Octavian A, Bero L: Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2012, Issue 12. Art. No.: MR000033. doi:10.1002/14651858.MR000033.pub2. 2012 Lundh A, Sismondo S, Lexchin J, Busuioc Octavian A, Bero L: Industry sponsorship and research outcome. Cochrane Database Syst Rev. 2012, Issue 12. Art. No.: MR000033. doi:10.1002/14651858.MR000033.pub2. 2012
13.
go back to reference Bartels RH, Delye H, Boogaarts J: Financial disclosures of authors involved in spine research: an underestimated source of bias. Eur Spine J. 2012, 21: 1229-1233. 10.1007/s00586-011-2086-x.CrossRefPubMed Bartels RH, Delye H, Boogaarts J: Financial disclosures of authors involved in spine research: an underestimated source of bias. Eur Spine J. 2012, 21: 1229-1233. 10.1007/s00586-011-2086-x.CrossRefPubMed
14.
go back to reference Yank V, Rennie D, Bero LA: Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study. BMJ. 2007, 335: 1202-1205. 10.1136/bmj.39376.447211.BE.CrossRefPubMedPubMedCentral Yank V, Rennie D, Bero LA: Financial ties and concordance between results and conclusions in meta-analyses: retrospective cohort study. BMJ. 2007, 335: 1202-1205. 10.1136/bmj.39376.447211.BE.CrossRefPubMedPubMedCentral
15.
go back to reference Forbes TL: Author disclosure of conflict of interest in vascular surgery journals. J Vasc Surg. 2011, 54: 55S-58S. 10.1016/j.jvs.2011.06.019.CrossRefPubMed Forbes TL: Author disclosure of conflict of interest in vascular surgery journals. J Vasc Surg. 2011, 54: 55S-58S. 10.1016/j.jvs.2011.06.019.CrossRefPubMed
16.
go back to reference Kesselheim AS, Lee JL, Avorn J, Servi A, Shrank WH, Choudhry NK: Conflict of interest in oncology publications. Cancer. 2012, 118: 188-195. 10.1002/cncr.26237.CrossRefPubMed Kesselheim AS, Lee JL, Avorn J, Servi A, Shrank WH, Choudhry NK: Conflict of interest in oncology publications. Cancer. 2012, 118: 188-195. 10.1002/cncr.26237.CrossRefPubMed
17.
go back to reference Blum JA, Freeman K, Dart RC, Cooper RJ: Requirements and definitions in conflict of interest policies of medical journals. JAMA. 2009, 302: 2230-2234. 10.1001/jama.2009.1669.CrossRefPubMed Blum JA, Freeman K, Dart RC, Cooper RJ: Requirements and definitions in conflict of interest policies of medical journals. JAMA. 2009, 302: 2230-2234. 10.1001/jama.2009.1669.CrossRefPubMed
18.
go back to reference Okike K, Kocher MS, Wei EX, Mehlman CT, Bhandari M: Accuracy of conflict-of-interest disclosures reported by physicians. N Engl J Med. 2009, 361: 1466-1474. 10.1056/NEJMsa0807160.CrossRefPubMed Okike K, Kocher MS, Wei EX, Mehlman CT, Bhandari M: Accuracy of conflict-of-interest disclosures reported by physicians. N Engl J Med. 2009, 361: 1466-1474. 10.1056/NEJMsa0807160.CrossRefPubMed
20.
go back to reference Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley A: Applying systematic review methods to studies of people’s views: an example from public health research. J Epidemiol Community Health. 2004, 58: 794-800. 10.1136/jech.2003.014829.CrossRefPubMedPubMedCentral Harden A, Garcia J, Oliver S, Rees R, Shepherd J, Brunton G, Oakley A: Applying systematic review methods to studies of people’s views: an example from public health research. J Epidemiol Community Health. 2004, 58: 794-800. 10.1136/jech.2003.014829.CrossRefPubMedPubMedCentral
21.
go back to reference Woodman J, Harden A, Thomas J, Brunton J, Kavanagh J, Stansfield C: Searching for systematic reviews of the effects of social and environmental interventions: a case study of children and obesity. J Med Libr Assoc. 2010, 98: 140-10.3163/1536-5050.98.2.006.CrossRefPubMedPubMedCentral Woodman J, Harden A, Thomas J, Brunton J, Kavanagh J, Stansfield C: Searching for systematic reviews of the effects of social and environmental interventions: a case study of children and obesity. J Med Libr Assoc. 2010, 98: 140-10.3163/1536-5050.98.2.006.CrossRefPubMedPubMedCentral
22.
go back to reference Schotland MS, Bero LA: Evaluating public commentary and scientific evidence submitted in the development of a risk assessment. Risk Anal. 2002, 22: 131-140. 10.1111/0272-4332.t01-1-00011.CrossRefPubMed Schotland MS, Bero LA: Evaluating public commentary and scientific evidence submitted in the development of a risk assessment. Risk Anal. 2002, 22: 131-140. 10.1111/0272-4332.t01-1-00011.CrossRefPubMed
24.
go back to reference International Committee of Medical Journal Editors: International Committee of Medical Journal Editors (ICMJE): uniform requirements for manuscripts submitted to biomedical journals: writing and editing for biomedical publication. Haematologica. 2004, 89: 264- International Committee of Medical Journal Editors: International Committee of Medical Journal Editors (ICMJE): uniform requirements for manuscripts submitted to biomedical journals: writing and editing for biomedical publication. Haematologica. 2004, 89: 264-
25.
go back to reference Ancker JS, Flanagin A: A comparison of conflict of interest policies at peer-reviewed journals in different scientific disciplines. Sci Eng Ethics. 2007, 13: 147-157. 10.1007/s11948-007-9011-z.CrossRefPubMed Ancker JS, Flanagin A: A comparison of conflict of interest policies at peer-reviewed journals in different scientific disciplines. Sci Eng Ethics. 2007, 13: 147-157. 10.1007/s11948-007-9011-z.CrossRefPubMed
26.
go back to reference Meerpohl JJ, Wolff RF, Niemeyer CM, Antes G, von Elm E: Editorial policies of pediatric journals: survey of instructions for authors. Arch Pediatr Adolesc Med. 2010, 164: 268-272. 10.1001/archpediatrics.2009.287.CrossRefPubMed Meerpohl JJ, Wolff RF, Niemeyer CM, Antes G, von Elm E: Editorial policies of pediatric journals: survey of instructions for authors. Arch Pediatr Adolesc Med. 2010, 164: 268-272. 10.1001/archpediatrics.2009.287.CrossRefPubMed
28.
go back to reference Lavis JN, Posada FB, Haines A, Osei E: Use of research to inform public policymaking. Lancet. 2004, 364: 1615-1621. 10.1016/S0140-6736(04)17317-0.CrossRefPubMed Lavis JN, Posada FB, Haines A, Osei E: Use of research to inform public policymaking. Lancet. 2004, 364: 1615-1621. 10.1016/S0140-6736(04)17317-0.CrossRefPubMed
29.
go back to reference Coyne JC, Thombs BD, Hagedoorn M: Ain’t necessarily so: review and critique of recent meta-analyses of behavioral medicine interventions in health psychology. Health Psychol. 2010, 29: 107-CrossRefPubMed Coyne JC, Thombs BD, Hagedoorn M: Ain’t necessarily so: review and critique of recent meta-analyses of behavioral medicine interventions in health psychology. Health Psychol. 2010, 29: 107-CrossRefPubMed
30.
go back to reference Chan KS, Morton SC, Shekelle PG: Systematic reviews for evidence-based management: how to find them and what to do with them. Am J Manag Care. 2004, 10: 806-812.PubMed Chan KS, Morton SC, Shekelle PG: Systematic reviews for evidence-based management: how to find them and what to do with them. Am J Manag Care. 2004, 10: 806-812.PubMed
31.
go back to reference Coffman JM, HONG MK, Aubry WM, Luft HS, Yelin E: Translating medical effectiveness research into policy: lessons from the California Health Benefits Review Program. Milbank Q. 2009, 87: 863-902. 10.1111/j.1468-0009.2009.00582.x.CrossRefPubMedPubMedCentral Coffman JM, HONG MK, Aubry WM, Luft HS, Yelin E: Translating medical effectiveness research into policy: lessons from the California Health Benefits Review Program. Milbank Q. 2009, 87: 863-902. 10.1111/j.1468-0009.2009.00582.x.CrossRefPubMedPubMedCentral
32.
go back to reference Brownson RC, Fielding JE, Maylahn CM: Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health. 2009, 30: 175-201. 10.1146/annurev.publhealth.031308.100134.CrossRefPubMed Brownson RC, Fielding JE, Maylahn CM: Evidence-based public health: a fundamental concept for public health practice. Annu Rev Public Health. 2009, 30: 175-201. 10.1146/annurev.publhealth.031308.100134.CrossRefPubMed
33.
go back to reference Eysenck H: Meta-analysis or best-evidence synthesis?. J Eval Clin Pract. 1995, 1: 29-36. 10.1111/j.1365-2753.1995.tb00005.x.CrossRefPubMed Eysenck H: Meta-analysis or best-evidence synthesis?. J Eval Clin Pract. 1995, 1: 29-36. 10.1111/j.1365-2753.1995.tb00005.x.CrossRefPubMed
34.
go back to reference Neumann PJ, Drummond MF, Jönsson B, Luce BR, Schwartz JS, Siebert U, Sullivan SD: Are key principles for improved health technology assessment supported and used by health technology assessment organizations?. Int J Technol Assess Health Care. 2010, 26: 71-78.CrossRefPubMed Neumann PJ, Drummond MF, Jönsson B, Luce BR, Schwartz JS, Siebert U, Sullivan SD: Are key principles for improved health technology assessment supported and used by health technology assessment organizations?. Int J Technol Assess Health Care. 2010, 26: 71-78.CrossRefPubMed
35.
go back to reference Laupacis A, Straus S: Systematic reviews: time to address clinical and policy relevance as well as methodological rigor. Ann Intern Med. 2007, 147: 273-274. 10.7326/0003-4819-147-4-200708210-00180.CrossRefPubMed Laupacis A, Straus S: Systematic reviews: time to address clinical and policy relevance as well as methodological rigor. Ann Intern Med. 2007, 147: 273-274. 10.7326/0003-4819-147-4-200708210-00180.CrossRefPubMed
36.
go back to reference Marini JJ: Meta-analysis: convenient assumptions and inconvenient truth. Crit Care Med. 2008, 36: 328-329. 10.1097/01.CCM.0000297959.02114.2B.CrossRefPubMed Marini JJ: Meta-analysis: convenient assumptions and inconvenient truth. Crit Care Med. 2008, 36: 328-329. 10.1097/01.CCM.0000297959.02114.2B.CrossRefPubMed
37.
go back to reference Goodman SN: Have you ever meta-analysis you didn’t like?. Ann Intern Med. 1991, 114: 244-246. 10.7326/0003-4819-114-3-244.CrossRefPubMed Goodman SN: Have you ever meta-analysis you didn’t like?. Ann Intern Med. 1991, 114: 244-246. 10.7326/0003-4819-114-3-244.CrossRefPubMed
38.
go back to reference Ahmad N, Boutron I, Dechartres A, Durieux P, Ravaud P: Applicability and generalisability of the results of systematic reviews to public health practice and policy: a systematic review. Trials. 2010, 11: 20-10.1186/1745-6215-11-20.CrossRefPubMedPubMedCentral Ahmad N, Boutron I, Dechartres A, Durieux P, Ravaud P: Applicability and generalisability of the results of systematic reviews to public health practice and policy: a systematic review. Trials. 2010, 11: 20-10.1186/1745-6215-11-20.CrossRefPubMedPubMedCentral
39.
go back to reference Tugwell P, Robinson V, Grimshaw J, Santesso N: Systematic reviews and knowledge translation. Bull World Health Organ. 2006, 84: 643-651. 10.2471/BLT.05.026658.CrossRefPubMedPubMedCentral Tugwell P, Robinson V, Grimshaw J, Santesso N: Systematic reviews and knowledge translation. Bull World Health Organ. 2006, 84: 643-651. 10.2471/BLT.05.026658.CrossRefPubMedPubMedCentral
40.
go back to reference Rychetnik L, Wise M: Advocating evidence-based health promotion: reflections and a way forward. Health Promot Int. 2004, 19: 247-257. 10.1093/heapro/dah212.CrossRefPubMed Rychetnik L, Wise M: Advocating evidence-based health promotion: reflections and a way forward. Health Promot Int. 2004, 19: 247-257. 10.1093/heapro/dah212.CrossRefPubMed
41.
go back to reference Petticrew M, Tugwell P, Welch V, Ueffing E, Kristjansson E, Armstrong R, Doyle J, Waters E: Better evidence about wicked issues in tackling health inequities. J Public Health. 2009, 31: 453-456. 10.1093/pubmed/fdp076.CrossRef Petticrew M, Tugwell P, Welch V, Ueffing E, Kristjansson E, Armstrong R, Doyle J, Waters E: Better evidence about wicked issues in tackling health inequities. J Public Health. 2009, 31: 453-456. 10.1093/pubmed/fdp076.CrossRef
42.
go back to reference Odierna DH, Bero LA: Systematic reviews reveal unrepresentative evidence for the development of drug formularies for poor and nonwhite populations. J Clin Epidemiol. 2009, 62: 1268-1278. 10.1016/j.jclinepi.2009.01.009.CrossRefPubMed Odierna DH, Bero LA: Systematic reviews reveal unrepresentative evidence for the development of drug formularies for poor and nonwhite populations. J Clin Epidemiol. 2009, 62: 1268-1278. 10.1016/j.jclinepi.2009.01.009.CrossRefPubMed
43.
go back to reference Gruen RL, Morris PS, McDonald EL, Bailie RS: Making systematic reviews more useful for policy-makers. Bull World Health Organ. 2005, 83: 480-PubMedPubMedCentral Gruen RL, Morris PS, McDonald EL, Bailie RS: Making systematic reviews more useful for policy-makers. Bull World Health Organ. 2005, 83: 480-PubMedPubMedCentral
44.
go back to reference Rising K, Bacchetti P, Bero L: Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med. 2008, 5: e217-10.1371/journal.pmed.0050217.CrossRefPubMedPubMedCentral Rising K, Bacchetti P, Bero L: Reporting bias in drug trials submitted to the Food and Drug Administration: review of publication and presentation. PLoS Med. 2008, 5: e217-10.1371/journal.pmed.0050217.CrossRefPubMedPubMedCentral
45.
go back to reference Hart B, Lundh A, Bero L: Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012, 344: d7202-10.1136/bmj.d7202.CrossRefPubMed Hart B, Lundh A, Bero L: Effect of reporting bias on meta-analyses of drug trials: reanalysis of meta-analyses. BMJ. 2012, 344: d7202-10.1136/bmj.d7202.CrossRefPubMed
46.
47.
go back to reference Doshi P, Jefferson T, Del Mar C: The imperative to share clinical study reports: recommendations from the Tamiflu experience. PLoS Med. 2012, 9: e1001201-10.1371/journal.pmed.1001201.CrossRefPubMedPubMedCentral Doshi P, Jefferson T, Del Mar C: The imperative to share clinical study reports: recommendations from the Tamiflu experience. PLoS Med. 2012, 9: e1001201-10.1371/journal.pmed.1001201.CrossRefPubMedPubMedCentral
48.
go back to reference Boutron I, Dutton S, Ravaud P, Altman DG: Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010, 303: 2058-2064. 10.1001/jama.2010.651.CrossRefPubMed Boutron I, Dutton S, Ravaud P, Altman DG: Reporting and interpretation of randomized controlled trials with statistically nonsignificant results for primary outcomes. JAMA. 2010, 303: 2058-2064. 10.1001/jama.2010.651.CrossRefPubMed
49.
go back to reference Pringle P: Eysenck took pounds 800,000 tobacco funds. The Independent. 1996, London: Independent Print Limited Pringle P: Eysenck took pounds 800,000 tobacco funds. The Independent. 1996, London: Independent Print Limited
Metadata
Title
Conflicts of interest and critiques of the use of systematic reviews in policymaking: an analysis of opinion articles
Authors
Susan R Forsyth
Donna H Odierna
David Krauth
Lisa A Bero
Publication date
01-12-2014
Publisher
BioMed Central
Published in
Systematic Reviews / Issue 1/2014
Electronic ISSN: 2046-4053
DOI
https://doi.org/10.1186/2046-4053-3-122

Other articles of this Issue 1/2014

Systematic Reviews 1/2014 Go to the issue