Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2011

Open Access 01-12-2011 | Debate

What counts as reliable evidence for public health policy: the case of circumcision for preventing HIV infection

Authors: Reidar K Lie, Franklin G Miller

Published in: BMC Medical Research Methodology | Issue 1/2011

Login to get access

Abstract

Background

There is an ongoing controversy over the relative merits of randomized controlled trials (RCTs) and non-randomized observational studies in assessing efficacy and guiding policy. In this paper we examine male circumcision to prevent HIV infection as a case study that can illuminate the appropriate role of different types of evidence for public health interventions.

Discussion

Based on an analysis of two Cochrane reviews, one published in 2003 before the results of three RCTs, and one in 2009, we argue that if we rely solely on evidence from RCTs and exclude evidence from well-designed non-randomized studies, we limit our ability to provide sound public health recommendations. Furthermore, the bias in favor of RCT evidence has delayed research on policy relevant issues.

Summary

This case study of circumcision and HIV prevention demonstrates that if we rely solely on evidence from RCTs and exclude evidence from well-designed non-randomized studies, we limit our ability to provide sound public health recommendations.
Literature
1.
go back to reference Glasziou P, Chalmers I, Rawlins M, McCulloch P: When are randomised trials unnecessary? Picking signal from noise. BMJ (Clinical research ed). 2007, 334 (7589): 349-351. 10.1136/bmj.39070.527986.68.CrossRef Glasziou P, Chalmers I, Rawlins M, McCulloch P: When are randomised trials unnecessary? Picking signal from noise. BMJ (Clinical research ed). 2007, 334 (7589): 349-351. 10.1136/bmj.39070.527986.68.CrossRef
2.
go back to reference Kunz R, Oxman AD: The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ (Clinical Research Ed). 1998, 317 (7167): 1185-1190.CrossRef Kunz R, Oxman AD: The unpredictability paradox: review of empirical comparisons of randomised and non-randomised clinical trials. BMJ (Clinical Research Ed). 1998, 317 (7167): 1185-1190.CrossRef
3.
go back to reference Shrier I, Boivin JF, Steele RJ, Platt RW, Furlan A, Kakuma R, Brophy J, Rossignol M: Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. American journal of epidemiology. 2007, 166 (10): 1203-1209. 10.1093/aje/kwm189.CrossRefPubMed Shrier I, Boivin JF, Steele RJ, Platt RW, Furlan A, Kakuma R, Brophy J, Rossignol M: Should meta-analyses of interventions include observational studies in addition to randomized controlled trials? A critical examination of underlying principles. American journal of epidemiology. 2007, 166 (10): 1203-1209. 10.1093/aje/kwm189.CrossRefPubMed
4.
go back to reference Kunz R: Randomized trials and observational studies: still mostly similar results, still crucial differences. Journal of clinical epidemiology. 2008, 61 (3): 207-208.CrossRefPubMed Kunz R: Randomized trials and observational studies: still mostly similar results, still crucial differences. Journal of clinical epidemiology. 2008, 61 (3): 207-208.CrossRefPubMed
5.
go back to reference Benson K, Hartz AJ: A comparison of observational studies and randomized, controlled trials. The New England Journal of Medicine. 2000, 342 (25): 1878-1886. 10.1056/NEJM200006223422506.CrossRefPubMed Benson K, Hartz AJ: A comparison of observational studies and randomized, controlled trials. The New England Journal of Medicine. 2000, 342 (25): 1878-1886. 10.1056/NEJM200006223422506.CrossRefPubMed
6.
go back to reference Concato J, Shah N, Horwitz RI: Randomized, controlled trials, observational studies, and the hierarchy of research designs. The New England Journal of Medicine. 2000, 342 (25): 1887-1892. 10.1056/NEJM200006223422507.CrossRefPubMedPubMedCentral Concato J, Shah N, Horwitz RI: Randomized, controlled trials, observational studies, and the hierarchy of research designs. The New England Journal of Medicine. 2000, 342 (25): 1887-1892. 10.1056/NEJM200006223422507.CrossRefPubMedPubMedCentral
7.
go back to reference Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, Contopoulos-Ioannidis DG, Lau J: Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA: the journal of the American Medical Association. 2001, 286 (7): 821-830. 10.1001/jama.286.7.821.CrossRefPubMed Ioannidis JP, Haidich AB, Pappa M, Pantazis N, Kokori SI, Tektonidou MG, Contopoulos-Ioannidis DG, Lau J: Comparison of evidence of treatment effects in randomized and nonrandomized studies. JAMA: the journal of the American Medical Association. 2001, 286 (7): 821-830. 10.1001/jama.286.7.821.CrossRefPubMed
8.
go back to reference Auvert B, Taljaard D, Lagarde E, Sobngwi-Tambekou J, Sitta R, Puren A: Randomized, controlled intervention trial of male circumcision for reduction of HIV infection risk: the ANRS 1265 Trial. PLoS medicine. 2005, 2 (11): e298-e298. 10.1371/journal.pmed.0020298.CrossRefPubMedPubMedCentral Auvert B, Taljaard D, Lagarde E, Sobngwi-Tambekou J, Sitta R, Puren A: Randomized, controlled intervention trial of male circumcision for reduction of HIV infection risk: the ANRS 1265 Trial. PLoS medicine. 2005, 2 (11): e298-e298. 10.1371/journal.pmed.0020298.CrossRefPubMedPubMedCentral
9.
go back to reference Bailey RC, Moses S, Parker CB, Agot K, Maclean I, Krieger JN, Williams CFM, Campbell RT, Ndinya-Achola JO: Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomised controlled trial. Lancet. 2007, 369 (9562): 643-656. 10.1016/S0140-6736(07)60312-2.CrossRefPubMed Bailey RC, Moses S, Parker CB, Agot K, Maclean I, Krieger JN, Williams CFM, Campbell RT, Ndinya-Achola JO: Male circumcision for HIV prevention in young men in Kisumu, Kenya: a randomised controlled trial. Lancet. 2007, 369 (9562): 643-656. 10.1016/S0140-6736(07)60312-2.CrossRefPubMed
10.
go back to reference Gray RH, Kigozi G, Serwadda D, Makumbi F, Watya S, Nalugoda F, Kiwanuka N, Moulton LH, Chaudhary MA, Chen MZ, et al: Male circumcision for HIV prevention in men in Rakai, Uganda: a randomised trial. Lancet. 2007, 369 (9562): 657-666. 10.1016/S0140-6736(07)60313-4.CrossRefPubMed Gray RH, Kigozi G, Serwadda D, Makumbi F, Watya S, Nalugoda F, Kiwanuka N, Moulton LH, Chaudhary MA, Chen MZ, et al: Male circumcision for HIV prevention in men in Rakai, Uganda: a randomised trial. Lancet. 2007, 369 (9562): 657-666. 10.1016/S0140-6736(07)60313-4.CrossRefPubMed
11.
go back to reference Bailey RC, Plummer FA, Moses S: Male circumcision and HIV prevention: current knowledge and future research directions. Lancet Infectious Diseases. 2001, 1: 223-231. 10.1016/S1473-3099(01)00117-7.CrossRefPubMed Bailey RC, Plummer FA, Moses S: Male circumcision and HIV prevention: current knowledge and future research directions. Lancet Infectious Diseases. 2001, 1: 223-231. 10.1016/S1473-3099(01)00117-7.CrossRefPubMed
12.
go back to reference Halperin DT, Bailey RC: Male circumcision and HIV infection: 10 years and counting. Lancet. 1999, 354: 1813-1815. 10.1016/S0140-6736(99)03421-2.CrossRefPubMed Halperin DT, Bailey RC: Male circumcision and HIV infection: 10 years and counting. Lancet. 1999, 354: 1813-1815. 10.1016/S0140-6736(99)03421-2.CrossRefPubMed
13.
go back to reference Siegfried N, Mueller M, Volmink J, Deeks JJ, Egger M, Low NN, Weiss HH, Walker SA, Williamson PR: Male circumcision for prevention of heterosexual acquisition of HIV in men. Cochrane Database of Systematic Reviews. 2003, Art. No.: CD003362. (Issue 3) Siegfried N, Mueller M, Volmink J, Deeks JJ, Egger M, Low NN, Weiss HH, Walker SA, Williamson PR: Male circumcision for prevention of heterosexual acquisition of HIV in men. Cochrane Database of Systematic Reviews. 2003, Art. No.: CD003362. (Issue 3)
14.
go back to reference Siegfried N, Mueller M, Deeks JJ, Volmink J: Male circumcision for prevention of heterosexual acquisition of HIV in men. Cochrane Database of Systematic Reviews. 2009, Art. No.: CD003362. (Issue 2) Siegfried N, Mueller M, Deeks JJ, Volmink J: Male circumcision for prevention of heterosexual acquisition of HIV in men. Cochrane Database of Systematic Reviews. 2009, Art. No.: CD003362. (Issue 2)
15.
go back to reference Agot KE, Kiarie JN, Nguyen HQ, Odhiambo JO, Onyango TM, Weiss NS: Male circumcision in Siaya and Bondo Districts, Kenya: prospective cohort study to assess behavioral disinhibition following circumcision. Journal of Acquired Immune Deficiency Syndromes (1999). 2007, 44 (1): 66-70. 10.1097/01.qai.0000242455.05274.20.CrossRef Agot KE, Kiarie JN, Nguyen HQ, Odhiambo JO, Onyango TM, Weiss NS: Male circumcision in Siaya and Bondo Districts, Kenya: prospective cohort study to assess behavioral disinhibition following circumcision. Journal of Acquired Immune Deficiency Syndromes (1999). 2007, 44 (1): 66-70. 10.1097/01.qai.0000242455.05274.20.CrossRef
16.
go back to reference Millett GA, Flores SA, Marks G, Reed JB, Herbst JH: Circumcision status and risk of HIV and sexually transmitted infections among men who have sex with men: a meta-analysis. JAMA: The Journal of the American Medical Association. 2008, 300 (14): 1674-1684. 10.1001/jama.300.14.1674.CrossRefPubMed Millett GA, Flores SA, Marks G, Reed JB, Herbst JH: Circumcision status and risk of HIV and sexually transmitted infections among men who have sex with men: a meta-analysis. JAMA: The Journal of the American Medical Association. 2008, 300 (14): 1674-1684. 10.1001/jama.300.14.1674.CrossRefPubMed
17.
go back to reference Kahn JG, Marseille E, Auvert B: Cost-effectiveness of male circumcision for HIV prevention in a South African setting. PLOS Medicine. 2006, 3: e517-10.1371/journal.pmed.0030517.CrossRefPubMedPubMedCentral Kahn JG, Marseille E, Auvert B: Cost-effectiveness of male circumcision for HIV prevention in a South African setting. PLOS Medicine. 2006, 3: e517-10.1371/journal.pmed.0030517.CrossRefPubMedPubMedCentral
18.
go back to reference Montori VM, Devereaux P, Adhikari NK, Burns KE, Eggert CH, Briel M, Lachetti C, Leung TW, Darling E, Bryant DM, et al: Randomized trials stopped early for benefit: a systematic review. JAMA. 2005, 294: 2203-2209. 10.1001/jama.294.17.2203.CrossRefPubMed Montori VM, Devereaux P, Adhikari NK, Burns KE, Eggert CH, Briel M, Lachetti C, Leung TW, Darling E, Bryant DM, et al: Randomized trials stopped early for benefit: a systematic review. JAMA. 2005, 294: 2203-2209. 10.1001/jama.294.17.2203.CrossRefPubMed
19.
go back to reference Bassler D, Briel M, Montori VM, Lane M, Glasziou P, Zhou Q, Heels-Ansdell D, Walter SD, Guyatt GH, STOP-IT 2 Study Group: Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA. 2010, 303: 1180-1187. 10.1001/jama.2010.310.CrossRefPubMed Bassler D, Briel M, Montori VM, Lane M, Glasziou P, Zhou Q, Heels-Ansdell D, Walter SD, Guyatt GH, STOP-IT 2 Study Group: Stopping randomized trials early for benefit and estimation of treatment effects: systematic review and meta-regression analysis. JAMA. 2010, 303: 1180-1187. 10.1001/jama.2010.310.CrossRefPubMed
20.
go back to reference Freedman B: Equipoise and the ethics of clinical research. New England Journal of Medicine. 1987, 317: 141-145. 10.1056/NEJM198707163170304.CrossRefPubMed Freedman B: Equipoise and the ethics of clinical research. New England Journal of Medicine. 1987, 317: 141-145. 10.1056/NEJM198707163170304.CrossRefPubMed
21.
go back to reference Buchanan D, Miller FG: Principles of early stopping of randomized trials for efficacy: a critique of equipoise and an alternative ethical framework. Kennedy Institute of Ethics Journal. 2005, 15: 161-178. 10.1353/ken.2005.0010.CrossRefPubMed Buchanan D, Miller FG: Principles of early stopping of randomized trials for efficacy: a critique of equipoise and an alternative ethical framework. Kennedy Institute of Ethics Journal. 2005, 15: 161-178. 10.1353/ken.2005.0010.CrossRefPubMed
22.
go back to reference Rosen L, Manor O, Engelhard D, Zucker D: In defense of the randomized controlled trial for health promotion research. American Journal of public Health. 2006, 96: 1181-1186. 10.2105/AJPH.2004.061713.CrossRefPubMedPubMedCentral Rosen L, Manor O, Engelhard D, Zucker D: In defense of the randomized controlled trial for health promotion research. American Journal of public Health. 2006, 96: 1181-1186. 10.2105/AJPH.2004.061713.CrossRefPubMedPubMedCentral
23.
go back to reference The World Bank is finally embracing science. Lancet. 2004, 364 (9436): 731-732. 10.1016/S0140-6736(04)16945-6. The World Bank is finally embracing science. Lancet. 2004, 364 (9436): 731-732. 10.1016/S0140-6736(04)16945-6.
24.
go back to reference Victora CG, Habicht JP, Bryce J: Evidence-based public health: moving beyond randomized trials. American Journal of Public Health. 2004, 94 (3): 400-405. 10.2105/AJPH.94.3.400.CrossRefPubMedPubMedCentral Victora CG, Habicht JP, Bryce J: Evidence-based public health: moving beyond randomized trials. American Journal of Public Health. 2004, 94 (3): 400-405. 10.2105/AJPH.94.3.400.CrossRefPubMedPubMedCentral
25.
go back to reference McMichael C, Waters E, Volmink J: Evidence-based public health: what does it offer developing countries?. Journal of Public Health (Oxford, England). 2005, 27 (2): 215-221. 10.1093/pubmed/fdi024.CrossRef McMichael C, Waters E, Volmink J: Evidence-based public health: what does it offer developing countries?. Journal of Public Health (Oxford, England). 2005, 27 (2): 215-221. 10.1093/pubmed/fdi024.CrossRef
Metadata
Title
What counts as reliable evidence for public health policy: the case of circumcision for preventing HIV infection
Authors
Reidar K Lie
Franklin G Miller
Publication date
01-12-2011
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2011
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-11-34

Other articles of this Issue 1/2011

BMC Medical Research Methodology 1/2011 Go to the issue