Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2014

Open Access 01-12-2014 | Research article

Meta-analysis and meta-modelling for diagnostic problems

Authors: Suphada Charoensawat, Walailuck Böhning, Dankmar Böhning, Heinz Holling

Published in: BMC Medical Research Methodology | Issue 1/2014

Login to get access

Abstract

Background

A proportional hazards measure is suggested in the context of analyzing SROC curves that arise in the meta–analysis of diagnostic studies. The measure can be motivated as a special model: the Lehmann model for ROC curves. The Lehmann model involves study–specific sensitivities and specificities and a diagnostic accuracy parameter which connects the two.

Methods

A study–specific model is estimated for each study, and the resulting study-specific estimate of diagnostic accuracy is taken as an outcome measure for a mixed model with a random study effect and other study-level covariates as fixed effects. The variance component model becomes estimable by deriving within-study variances, depending on the outcome measure of choice. In contrast to existing approaches – usually of bivariate nature for the outcome measures – the suggested approach is univariate and, hence, allows easily the application of conventional mixed modelling.

Results

Some simple modifications in the SAS procedure proc mixed allow the fitting of mixed models for meta-analytic data from diagnostic studies. The methodology is illustrated with several meta–analytic diagnostic data sets, including a meta–analysis of the Mini–Mental State Examination as a diagnostic device for dementia and mild cognitive impairment.

Conclusions

The proposed methodology allows us to embed the meta-analysis of diagnostic studies into the well–developed area of mixed modelling. Different outcome measures, specifically from the perspective of whether a local or a global measure of diagnostic accuracy should be applied, are discussed as well. In particular, variation in cut-off value is discussed together with recommendations on choosing the best cut-off value. We also show how this problem can be addressed with the proposed methodology.
Appendix
Available only for authorised users
Literature
1.
go back to reference Hasselblad V, Hedges LV: Meta-analysis of screening and diagnostic tests. Psychol Bull. 1995, 117: 167-178.CrossRefPubMed Hasselblad V, Hedges LV: Meta-analysis of screening and diagnostic tests. Psychol Bull. 1995, 117: 167-178.CrossRefPubMed
2.
go back to reference Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F: Methods for Meta-Analysis in Medical Research. 2000, New York: Wiley Sutton AJ, Abrams KR, Jones DR, Sheldon TA, Song F: Methods for Meta-Analysis in Medical Research. 2000, New York: Wiley
3.
go back to reference Deeks JJ: Systematic reviews of evaluations of diagnostic and screening tests. Systematic Reviews in Health Care: Meta-Analysis in context, vol.14, pp.248-282. 2007, London: BMJ Books, Deeks JJ: Systematic reviews of evaluations of diagnostic and screening tests. Systematic Reviews in Health Care: Meta-Analysis in context, vol.14, pp.248-282. 2007, London: BMJ Books,
4.
go back to reference Meta-Analysis. New Developments and Applications in Medical and Social Sciences. 2003, Hogrefe & Huber: Göttingen Meta-Analysis. New Developments and Applications in Medical and Social Sciences. 2003, Hogrefe & Huber: Göttingen
5.
go back to reference Pepe MS: Receiver operating characteristic methodology. J Am Stat Assoc. 2000, 95: 308-311. 10.1080/01621459.2000.10473930.CrossRef Pepe MS: Receiver operating characteristic methodology. J Am Stat Assoc. 2000, 95: 308-311. 10.1080/01621459.2000.10473930.CrossRef
6.
go back to reference Pepe MS: The Statistical Evaluation of Medical Tests for Classification and Prediction. 2003, Oxford: Oxford University Press Pepe MS: The Statistical Evaluation of Medical Tests for Classification and Prediction. 2003, Oxford: Oxford University Press
7.
go back to reference Holling H, Böhning W, Böhning D: Meta-analysis of diagnostic studies based upon SROC-curves: a mixed model approach using the Lehmann family. Stat Modelling – Int J. 2012, 12: 347-375. 10.1177/1471082X1201200403.CrossRef Holling H, Böhning W, Böhning D: Meta-analysis of diagnostic studies based upon SROC-curves: a mixed model approach using the Lehmann family. Stat Modelling – Int J. 2012, 12: 347-375. 10.1177/1471082X1201200403.CrossRef
8.
go back to reference Swets JA: Signal Detection Theory and ROC Analysis in Psychology and Diagnostics Collected Papers. 1996, New York, London: Psychology Press Taylor & Francis Group Swets JA: Signal Detection Theory and ROC Analysis in Psychology and Diagnostics Collected Papers. 1996, New York, London: Psychology Press Taylor & Francis Group
9.
go back to reference Doust JA, Glasziou PP, Pietrzak E, Dobson AJ: A system reviews of diagnostic accuracy of natriyretic peptides for heart failure. Arch Intern Med. 2004, 2: 9- Doust JA, Glasziou PP, Pietrzak E, Dobson AJ: A system reviews of diagnostic accuracy of natriyretic peptides for heart failure. Arch Intern Med. 2004, 2: 9-
10.
go back to reference Youden D: Index for rating diagnostic tests. Cancer. 1950, 3: 32-35. 10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3.CrossRefPubMed Youden D: Index for rating diagnostic tests. Cancer. 1950, 3: 32-35. 10.1002/1097-0142(1950)3:1<32::AID-CNCR2820030106>3.0.CO;2-3.CrossRefPubMed
11.
go back to reference Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PMM: The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003, 56: 1129-1135. 10.1016/S0895-4356(03)00177-X.CrossRefPubMed Glas AS, Lijmer JG, Prins MH, Bonsel GJ, Bossuyt PMM: The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003, 56: 1129-1135. 10.1016/S0895-4356(03)00177-X.CrossRefPubMed
12.
go back to reference Moses LE, Shapiro D, Littenberg B: Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993, 12: 1293-316. 10.1002/sim.4780121403.CrossRefPubMed Moses LE, Shapiro D, Littenberg B: Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993, 12: 1293-316. 10.1002/sim.4780121403.CrossRefPubMed
13.
go back to reference Littenberg B, Moses LE: Estimating diagnostic accuracy from multiple conflicting reports: A new meta-analytic method. Med Decis Making. 1993, 13: 313-321. 10.1177/0272989X9301300408.CrossRefPubMed Littenberg B, Moses LE: Estimating diagnostic accuracy from multiple conflicting reports: A new meta-analytic method. Med Decis Making. 1993, 13: 313-321. 10.1177/0272989X9301300408.CrossRefPubMed
14.
go back to reference Rutter CM, Gatsonis CA: A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001, 20: 2865-2884. 10.1002/sim.942.CrossRefPubMed Rutter CM, Gatsonis CA: A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001, 20: 2865-2884. 10.1002/sim.942.CrossRefPubMed
15.
go back to reference Walter SD, Macaskill P: SROC curve. Encyclopedia of Biopharmaceutical statistics. 2004, New York: Marcel Dekker, Walter SD, Macaskill P: SROC curve. Encyclopedia of Biopharmaceutical statistics. 2004, New York: Marcel Dekker,
16.
go back to reference Van Houwelingen HC, Zwinderman KH, Stijnen T: A bivariate approach to meta-analysis. Stat Med. 1993, 12: 2273-2284. 10.1002/sim.4780122405.CrossRefPubMed Van Houwelingen HC, Zwinderman KH, Stijnen T: A bivariate approach to meta-analysis. Stat Med. 1993, 12: 2273-2284. 10.1002/sim.4780122405.CrossRefPubMed
17.
go back to reference Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH: Bivariate analysis of sensitivity and specificity produces informative measures in diagnostic reviews. J Clin Epidemiol. 2005, 58: 982-990. 10.1016/j.jclinepi.2005.02.022.CrossRefPubMed Reitsma JB, Glas AS, Rutjes AWS, Scholten RJPM, Bossuyt PM, Zwinderman AH: Bivariate analysis of sensitivity and specificity produces informative measures in diagnostic reviews. J Clin Epidemiol. 2005, 58: 982-990. 10.1016/j.jclinepi.2005.02.022.CrossRefPubMed
18.
go back to reference Arends LR, Hamza TH, van Houwelingen JC, Heijenbrok-Kal MH, Hunink MGM, Stijnen T: Bivariate random effects meta-analysis of ROC curves. Med Decis Making. 2008, 28: 621-638. 10.1177/0272989X08319957.CrossRefPubMed Arends LR, Hamza TH, van Houwelingen JC, Heijenbrok-Kal MH, Hunink MGM, Stijnen T: Bivariate random effects meta-analysis of ROC curves. Med Decis Making. 2008, 28: 621-638. 10.1177/0272989X08319957.CrossRefPubMed
19.
go back to reference Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JAC: A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2006, 1: 1-21. Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JAC: A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2006, 1: 1-21.
20.
21.
go back to reference Le CT: A solution for the most basic optimization problem associated with an ROC curve. Stat Methods Med Res. 2006, 15: 571-584. 10.1177/0962280206070637.CrossRefPubMed Le CT: A solution for the most basic optimization problem associated with an ROC curve. Stat Methods Med Res. 2006, 15: 571-584. 10.1177/0962280206070637.CrossRefPubMed
23.
go back to reference Aertgeerts FB, Kester A: The value of the CAGE in screening for alcohol abuse and alcohol dependence in general clinical populations: a diagnostic meta-analysis. J Clin Epidemiol. 2003, 57: 30-39.CrossRef Aertgeerts FB, Kester A: The value of the CAGE in screening for alcohol abuse and alcohol dependence in general clinical populations: a diagnostic meta-analysis. J Clin Epidemiol. 2003, 57: 30-39.CrossRef
24.
go back to reference Rücker G, Schumacher M: Letter to the editor. Biostatistics. 2009, 10: 806-807. 10.1093/biostatistics/kxp021.CrossRefPubMed Rücker G, Schumacher M: Letter to the editor. Biostatistics. 2009, 10: 806-807. 10.1093/biostatistics/kxp021.CrossRefPubMed
25.
go back to reference Rücker G, Schumacher M: Summary ROC curve based on a weighted Youden index for selecting an optimal cutpoint in meta-analysis of diagnostic accuracy. Stat Med. 2010, 29: 3069-3078. 10.1002/sim.3937.CrossRefPubMed Rücker G, Schumacher M: Summary ROC curve based on a weighted Youden index for selecting an optimal cutpoint in meta-analysis of diagnostic accuracy. Stat Med. 2010, 29: 3069-3078. 10.1002/sim.3937.CrossRefPubMed
26.
go back to reference Mitchell AJ: A Meta-Analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res. 2009, 43: 411-431. 10.1016/j.jpsychires.2008.04.014.CrossRefPubMed Mitchell AJ: A Meta-Analysis of the accuracy of the mini-mental state examination in the detection of dementia and mild cognitive impairment. J Psychiatr Res. 2009, 43: 411-431. 10.1016/j.jpsychires.2008.04.014.CrossRefPubMed
27.
go back to reference SAS Institute Inc: SAS/STAT(R) 9.2 User’s Guide, Second Edition. 2008, Cary, NC, USA: SAS Insitute Inc. SAS Institute Inc: SAS/STAT(R) 9.2 User’s Guide, Second Edition. 2008, Cary, NC, USA: SAS Insitute Inc.
28.
go back to reference Self SG, Liang K-Y: Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. J Am Stat Assoc. 1987, 82: 605-610. 10.1080/01621459.1987.10478472.CrossRef Self SG, Liang K-Y: Asymptotic properties of maximum likelihood estimators and likelihood ratio tests under nonstandard conditions. J Am Stat Assoc. 1987, 82: 605-610. 10.1080/01621459.1987.10478472.CrossRef
29.
go back to reference Wittkampf KA, Naeije L, Schene AH, Huyser J, van Weet HC: Diagnostic accuracy of the mood module of the patient health questionnaire: a systematic review. Gen Hosp Psychiatry. 2007, 29: 388-395. 10.1016/j.genhosppsych.2007.06.004.CrossRefPubMed Wittkampf KA, Naeije L, Schene AH, Huyser J, van Weet HC: Diagnostic accuracy of the mood module of the patient health questionnaire: a systematic review. Gen Hosp Psychiatry. 2007, 29: 388-395. 10.1016/j.genhosppsych.2007.06.004.CrossRefPubMed
30.
go back to reference Wang P, Guo YM, Qiang YQ, Duan XY, Zhang QJ, Liang W: A meta-analysis of the accuracy of prostate cancer studies which use magnetic resonance spectroscopy (MRS) as a diagnostic tool. Korean J Radiol. 2008, 9: 432-438. 10.3348/kjr.2008.9.5.432.CrossRefPubMedPubMedCentral Wang P, Guo YM, Qiang YQ, Duan XY, Zhang QJ, Liang W: A meta-analysis of the accuracy of prostate cancer studies which use magnetic resonance spectroscopy (MRS) as a diagnostic tool. Korean J Radiol. 2008, 9: 432-438. 10.3348/kjr.2008.9.5.432.CrossRefPubMedPubMedCentral
31.
go back to reference Schuetz GM, Schlattmann P, Dewey M: Use of 3Œ2 tables with an intention to diagnose approach to assess clinical performance of diagnostic tests: meta-analytical evaluation of coronary CT angiography studies. BMJ. 2012, 345: 6717-10.1136/bmj.e6717.CrossRef Schuetz GM, Schlattmann P, Dewey M: Use of 3Œ2 tables with an intention to diagnose approach to assess clinical performance of diagnostic tests: meta-analytical evaluation of coronary CT angiography studies. BMJ. 2012, 345: 6717-10.1136/bmj.e6717.CrossRef
Metadata
Title
Meta-analysis and meta-modelling for diagnostic problems
Authors
Suphada Charoensawat
Walailuck Böhning
Dankmar Böhning
Heinz Holling
Publication date
01-12-2014
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2014
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/1471-2288-14-56

Other articles of this Issue 1/2014

BMC Medical Research Methodology 1/2014 Go to the issue