Skip to main content
Key Specialties
Cardiology Diabetology Endocrinology Gastroenterology Geriatrics and Gerontology Gynecology and Obstetrics Hematology Infectious Disease Internal Medicine Nephrology Neurology Oncology Pediatrics Respiratory Medicine Rheumatology Surgery
Congress Coverage
2024 ACC Congress 2023 ESMO Congress 2023 EASD Congress 2023 ERS Congress 2023 ESC Congress
Clinical Collections
Advances in Alzheimer’s

Keynote webinar | Spotlight on medication adherence

LIVE: Thursday 27th June 2024, 18:00-19:30 (CEST)
Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.
ADVANCED SEARCH
Log in
Top
BMC Medical Research Methodology
Published in: BMC Medical Research Methodology 1/2012

Open Access 01-12-2012 | Research article

Analysis of decisions made in meta-analyses of depression screening and the risk of confirmation bias: A case study

Authors: Felicity A Goodyear-Smith, Mieke L van Driel, Bruce Arroll, Chris Del Mar

Published in: BMC Medical Research Methodology | Issue 1/2012

Login to get access

Abstract

Background

Depression is common in primary care and clinicians are encouraged to screen their patients. Meta-analyses have evaluated the effectiveness of screening, but two author groups consistently reached completely opposite conclusions.

Methods

We identified five systematic reviews on depression screening conducted between 2001 and 2009, three by Gilbody and colleagues and two by the United States Preventive Task Force. The two author groups consistently reached completely opposite conclusions. We analyzed two contemporaneous systematic reviews, applying a stepwise approach to unravel their methods. Decision points were identified, and discrepancies between systematic reviews authors’ justification of choices made were recorded.

Results

Two systematic reviews each addressing three research questions included 26 randomized controlled trials with different combinations in each review. For the outcome depression screening resulting in treatment, both reviews undertook meta-analyses of imperfectly overlapping studies. Two in particular, pooled each by only one of the reviews, influenced the recommendations in opposite directions. Justification for inclusion or exclusion of studies was obtuse.

Conclusion

Systematic reviews may be less objective than assumed. Based on this analysis of two meta-analyses we hypothesise that strongly held prior beliefs (confirmation bias) may have influenced inclusion and exclusion criteria of studies, and their interpretation. Authors should be required to declare a priori any strongly held prior beliefs within their hypotheses, before embarking on systematic reviews.
Appendix
Available only for authorised users
Literature
1.
Systematic Reviews in Health Care. Meta-analysis in Context. Edited by: Egger M, Davey Smith G, Altman D. 2001, BMJ Books, London
2.
Cochrane Handbook for Systematic Reviews of Interventions Version 5.0.2. Edited by: Higgins J, Green S. 2009, The Cochrane Collaboration, , updated September 2009
3.
Guyatt GH, Haynes RB, Jaeschke RZ, Cook DJ, Green L, Naylor CD, Wilson MC, Richardson WS: Users' Guides to the Medical Literature: XXV. Evidence-based Medicine: Principles for Applying the Users' Guides to Patient Care. Evidence-Based Medicine Working Group. JAMA. 2000, 284 (10): 1290-1296. 10.1001/jama.284.10.1290.CrossRefPubMed
4.
5.
6.
Le Fanu J: Misleading meta-analysis. Public policy is based on results of epidemiological meta-analyses that contradict common sense. BMJ. 1995, 310 (6994): 1603-1604.CrossRefPubMedPubMedCentral
7.
8.
Gilbody S, House AO, Sheldon TA: Screening and case finding instruments for depression. Cochrane Database Syst Rev. 2005, CD002792-4
9.
Gilbody S, Sheldon T, House A: Screening and case-finding instruments for depression: a meta-analysis. CMAJ. 2008, 178 (8): 997-1003. 10.1503/cmaj.070281.CrossRefPubMedPubMedCentral
10.
Gilbody SM, House AO, Sheldon TA: Routinely administered questionnaires for depression and anxiety: systematic review. BMJ. 2001, 322 (7283): 406-409. 10.1136/bmj.322.7283.406.CrossRefPubMedPubMedCentral
11.
Jonas E, Schulz-Hardt S, Frey D, Thelen N: Confirmation bias in sequential information search after preliminary decisions: an expansion of dissonance theoretical research on selective exposure to information. J Pers Soc Psychol. 2001, 80: 557-571.CrossRefPubMed
12.
Pignone MP, Gaynes BN, Rushton JL, Burchell CM, Orleans CT, Mulrow CD, Lohr KN: Screening for depression in adults: a summary of the evidence for the U.S. Preventive Services Task Force. Ann Intern Med. 2002, 136 (10): 765-776.CrossRefPubMed
13.
Preventive Services US: Task Force: Screening for depression: recommendations and rationale. [Summary for patients in Ann Intern Med. 2002 May 21;136(10):I56; PMID: 12020161]. Ann Intern Med. 2002, 136 (10): 760-764.CrossRef
14.
O'Connor EA, Whitlock EP, Beil TL, Gaynes BN: Screening for depression in adult patients in primary care settings: a systematic evidence review. Ann Intern Med. 2009, 151 (11): 793-803.CrossRefPubMed
15.
Preventive Services US: Task Force: Screening for depression in adults: U.S. preventive services task force recommendation statement. Ann Intern Med. 2009, 151 (11): 784-792.CrossRef
16.
Johnstone A, Goldberg D: Psychiatric screening in general practice. A controlled trial. Lancet. 1976, 1 (7960): 605-608.CrossRefPubMed
17.
Moore JT, Silimperi DR, Bobula JA: Recognition of depression by family medicine residents: the impact of screening. J Family Practice. 1978, 7 (3): 509-513.
18.
Linn LS, Yager J: Screening of depression in relationship to subsequent patient and physician behavior. Med Care. 1982, 20 (12): 1233-1240. 10.1097/00005650-198212000-00009.CrossRefPubMed
19.
Zung WW, Magill M, Moore JT, George DT: Recognition and treatment of depression in a family medicine practice. J Clin Psychiatry. 1983, 44 (1): 3-6.PubMed
20.
Zung WW, King RE: Identification and treatment of masked depression in a general medical practice. J Clin Psychiatry. 1983, 44 (10): 365-368.PubMed
21.
Hoeper EW, Nycz GR, Kessler LG, Burke JD, Pierce WE: The usefulness of screening for mental illness. Lancet. 1984, 1 (8367): 33-35.CrossRefPubMed
22.
German PS, Shapiro S, Skinner EA, Von Korff M, Klein LE, Turner RW, Teitelbaum ML, Burke J, Burns BJ: Detection and management of mental health problems of older patients by primary care providers. JAMA. 1987, 257 (4): 489-493. 10.1001/jama.1987.03390040105026.CrossRefPubMed
23.
Magruder-Habib K, Zung WW, Feussner JR: Improving physicians' recognition and treatment of depression in general medical care. Results from a randomized clinical trial. Med Care. 1990, 28 (3): 239-250. 10.1097/00005650-199003000-00004.CrossRefPubMed
24.
Callahan CM, Hendrie HC, Dittus RS, Brater DC, Hui SL, Tierney WM: Improving treatment of late life depression in primary care: a randomized clinical trial. J Am Geriatr Soc. 1994, 42 (8): 839-846.CrossRefPubMed
25.
Dowrick C: Does testing for depression influence diagnosis or management by general practitioners?. Fam Pract. 1995, 12 (4): 461-465. 10.1093/fampra/12.4.461.CrossRefPubMed
26.
Callahan CM, Dittus RS, Tierney WM: Primary care physicians' medical decision making for late-life depression. J Gen Intern Med. 1996, 11 (4): 218-225. 10.1007/BF02642478.CrossRefPubMed
27.
Lewis G, Sharp D, Bartholomew J, Pelosi AJ: Computerized assessment of common mental disorders in primary care: effect on clinical outcome. Fam Pract. 1996, 13 (2): 120-126. 10.1093/fampra/13.2.120.CrossRefPubMed
28.
Mazonson PD, Mathias SD, Fifer SK, Buesching DP, Malek P, Patrick DL: The mental health patient profile: does it change primary care physicians' practice patterns?. J Am Board Fam Pract. 1996, 9 (5): 336-345.PubMed
29.
Reifler DR, Kessler HS, Bernhard EJ, Leon AC, Martin GJ: Impact of screening for mental health concerns on health service utilization and functional status in primary care patients. Arch Intern Med. 1996, 156 (22): 2593-2599. 10.1001/archinte.1996.00440210121012.CrossRefPubMed
30.
Williams JW, Mulrow CD, Kroenke K, Dhanda R, Badgett RG, Omori D, Lee S: Case-finding for depression in primary care: a randomized trial. Am J Med. 1999, 106 (1): 36-43. 10.1016/S0002-9343(98)00371-4.CrossRefPubMed
31.
Katzelnick DJ, Simon GE, Pearson SD, Manning WG, Helstad CP, Henk HJ, Cole SM, Lin EH, Taylor LH, Kobak KA: Randomized trial of a depression management program in high utilizers of medical care. Arch Fam Med. 2000, 9 (4): 345-351. 10.1001/archfami.9.4.345.CrossRefPubMed
32.
Weatherall M: A randomized controlled trial of the Geriatric Depression Scale in an inpatient ward for older adults. Clin Rehabil. 2000, 14 (2): 186-191. 10.1191/026921500672596145.CrossRefPubMed
33.
Wells KB, Sherbourne C, Schoenbaum M, Duan N, Meredith L, Unutzer J, Miranda J, Carney MF, Rubenstein LV: Impact of disseminating quality improvement programs for depression in managed primary care: a randomized controlled trial. [Erratum appears in JAMA 2000 Jun 28;283(24):3204]. JAMA. 2000, 283 (2): 212-220. 10.1001/jama.283.2.212.CrossRefPubMed
34.
Whooley MA, Avins AL, Miranda J, Browner WS: Case-finding instruments for depression. Two questions are as good as many. J Gen Int Med. 1997, 12 (7): 439-445. 10.1046/j.1525-1497.1997.00076.x.CrossRef
35.
Rost K, Nutting P, Smith J, Werner J, Duan N: Improving depression outcomes in community primary care practice: a randomized trial of the quEST intervention. Quality Enhancement by Strategic Teaming. J Gen Intern Med. 2001, 16 (3): 143-149. 10.1111/j.1525-1497.2001.00537.x.CrossRefPubMedPubMedCentral
36.
Schriger DL, Gibbons PS, Langone CA, Lee S, Altshuler LL: Enabling the diagnosis of occult psychiatric illness in the emergency department: a randomized, controlled trial of the computerized, self-administered PRIME-MD diagnostic system. Ann Emerg Med. 2001, 37 (2): 132-140. 10.1067/mem.2001.112255.CrossRefPubMed
37.
Christensen KS, Toft T, Frostholm L, Ornbol E, Fink P, Olesen F: The FIP study: a randomised, controlled trial of screening and recognition of psychiatric disorders. Br J Gen Pract. 2003, 53 (495): 758-763.PubMedPubMedCentral
38.
Jarjoura D, Polen A, Baum E, Kropp D, Hetrick S, Rutecki G: Effectiveness of screening and treatment for depression in ambulatory indigent patients. J Gen Intern Med. 2004, 19 (1): 78-84. 10.1111/j.1525-1497.2004.21249.x.CrossRefPubMedPubMedCentral
39.
Bergus GR, Hartz AJ, Noyes R, Ward MM, James PA, Vaughn T, Kelley PL, Sinift SD, Bentler S, Tilman E: The limited effect of screening for depressive symptoms with the PHQ-9 in rural family practices. J Rural Health. 2005, 21 (4): 303-309. 10.1111/j.1748-0361.2005.tb00099.x.CrossRefPubMed
40.
Bosmans J, de Bruijne M, van Hout H, van Marwijk H, Beekman A, Bouter L, Stalman W, van Tulder M: Cost-effectiveness of a disease management program for major depression in elderly primary care patients. J Gen Intern Med. 2006, 21 (10): 1020-1026. 10.1111/j.1525-1497.2006.00555.x.CrossRefPubMedPubMedCentral
41.
Rubenstein LZ, Alessi CA, Josephson KR, Trinidad Hoyl M, Harker JO, Pietruszka FM: A randomized trial of a screening, case finding, and referral system for older veterans in primary care. J Am Geriatr Soc. 2007, 55 (2): 166-174. 10.1111/j.1532-5415.2007.01044.x.CrossRefPubMed
42.
43.
van Driel ML, De Sutter A, De Maeseneer J, Christiaens T: Searching for unpublished trials in Cochrane reviews may not be worth the effort. J Clin Epidemiol. 2009, 62 (8): 838-844. 10.1016/j.jclinepi.2008.09.010. e833CrossRefPubMed
44.
Castellsague X, Albero G, Cleries R, Bosch FX: HPV and circumcision: a biased, inaccurate and misleading meta-analysis. J Infect. 2007, 55 (1): 91-93. 10.1016/j.jinf.2007.02.009. author reply 93–96CrossRefPubMed
45.
Chaturvedi N, Bilous R, Hardy R, Remuzzi G, Ruggenenti P, Viberti GC: Misleading meta-analysis: a need to look beyond the headlines. Diab Med. 2007, 24 (6): 587-591. 10.1111/j.1464-5491.2007.02139.x.CrossRef
46.
Felson DT: Bias in meta-analytic research. J Clin Epidemiol. 1992, 45 (8): 885-892. 10.1016/0895-4356(92)90072-U.CrossRefPubMed
47.
Cook DJ, Reeve BK, Guyatt GH, Heyland DK, Griffith LE, Buckingham L, Tryba M: Stress ulcer prophylaxis in critically ill patients. Resolving discordant meta-analyses. JAMA. 1996, 275 (4): 308-314. 10.1001/jama.1996.03530280060038.CrossRefPubMed
48.
Ford AC, Guyatt GH, Talley NJ, Moayyedi P: Errors in the conduct of systematic reviews of pharmacological interventions for irritable bowel syndrome. Am J Gastroenterol. 2010, 105 (2): 280-288. 10.1038/ajg.2009.658.CrossRefPubMed
49.
Nickerson RS: Confirmation bias: A ubiquitous phenomenon in many guises. Rev Gen Psych. 1998, 2 (2): 175-220.CrossRef
50.
Tilburt JC, Miller FG, Jenkins S, Kaptchuk TJ, Clarridge B, Bolcic-Jankovic D, Emanuel EJ, Curlin FA: Factors that influence practitioners' interpretations of evidence from alternative medicine trials: A factorial vignette experiment embedded in a national survey. Med Care. 2010, 48 (4): 341-348. 10.1097/MLR.0b013e3181ca3ee2.CrossRefPubMedPubMedCentral
Metadata
Title
Analysis of decisions made in meta-analyses of depression screening and the risk of confirmation bias: A case study
Authors
Felicity A Goodyear-Smith
Mieke L van Driel
Bruce Arroll
Chris Del Mar
Publication date
01-12-2012
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2012
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-12-76

Other articles of this Issue 1/2012

BMC Medical Research Methodology 1/2012 Go to the issue

Research article

Investigating linkage rates among probabilistically linked birth and hospitalization records

Research article

Multiple imputation for estimating hazard ratios and predictive abilities in case-cohort surveys

Research article

Evaluation of the Propensity score methods for estimating marginal odds ratios in case of small sample size

Research article

Factor structure and psychometric properties of the trier inventory for chronic stress (TICS) in a representative german sample

Research article

How pragmatic or explanatory is the randomized, controlled trial? The application and enhancement of the PRECIS tool to the evaluation of a smoking cessation trial

Correspondence

The quality of the evidence base for clinical pathway effectiveness: Room for improvement in the design of evaluation trials