Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2017

Open Access 01-12-2017 | Research Article

A double SIMEX approach for bivariate random-effects meta-analysis of diagnostic accuracy studies

Author: Annamaria Guolo

Published in: BMC Medical Research Methodology | Issue 1/2017

Login to get access

Abstract

Background

Bivariate random-effects models represent a widely accepted and recommended approach for meta-analysis of test accuracy studies. Standard likelihood methods routinely used for inference are prone to several drawbacks. Small sample size can give rise to unreliable inferential conclusions and convergence issues make the approach unappealing. This paper suggests a different methodology to address such difficulties.

Methods

A SIMEX methodology is proposed. The method is a simulation-based technique originally developed as a correction strategy within the measurement error literature. It suits the meta-analysis framework as the diagnostic accuracy measures provided by each study are prone to measurement error. SIMEX can be straightforwardly adapted to cover different measurement error structures and to deal with covariates. The effortless implementation with standard software is an interesting feature of the method.

Results

Extensive simulation studies highlight the improvement provided by SIMEX over likelihood approach in terms of empirical coverage probabilities of confidence intervals under different scenarios, independently of the sample size and the values of the correlation between sensitivity and specificity. A remarkable amelioration is obtained in case of deviations from the normality assumption for the random-effects distribution. From a computational point of view, the application of SIMEX is shown to be neither involved nor subject to the convergence issues affecting likelihood-based alternatives. Application of the method to a diagnostic review of the performance of transesophageal echocardiography for assessing ascending aorta atherosclerosis enables overcoming limitations of the likelihood procedure.

Conclusions

The SIMEX methodology represents an interesting alternative to likelihood-based procedures for inference in meta-analysis of diagnostic accuracy studies. The approach can provide more accurate inferential conclusions, while avoiding convergence failure and numerical instabilities. The application of the method in the R programming language is possible through the code which is made available and illustrated using the real data example.
Appendix
Available only for authorised users
Literature
1.
go back to reference Honest H, Khan K. Reporting of measures of accuracy in systematic reviews of diagnostic literature. BMC Health Serv Res. 2002; 2:4:266–7.CrossRef Honest H, Khan K. Reporting of measures of accuracy in systematic reviews of diagnostic literature. BMC Health Serv Res. 2002; 2:4:266–7.CrossRef
2.
go back to reference Riley RD, Abrams KR, Sutton AJ, Lambert PC, Thompson JR. Bivariate random-effects meta-analysis and the estimation of between-study correlation. BMC Med Res Methodol. 2007; 7:3:266–7.CrossRef Riley RD, Abrams KR, Sutton AJ, Lambert PC, Thompson JR. Bivariate random-effects meta-analysis and the estimation of between-study correlation. BMC Med Res Methodol. 2007; 7:3:266–7.CrossRef
3.
go back to reference Riley RD, Thompson JR, Abrams KR. An alternative model for bivariate random-effects meta-analysis when the within-study correlations are unknown. Biostatistics. 2008; 9:172–86.CrossRefPubMed Riley RD, Thompson JR, Abrams KR. An alternative model for bivariate random-effects meta-analysis when the within-study correlations are unknown. Biostatistics. 2008; 9:172–86.CrossRefPubMed
4.
go back to reference Hamza TH, van Houwelingen HC, Stijnen T. The binomial distribution of meta-analysis was preferred to model within-study variability. J Clin Epidemiol. 2008; 61:41–51.CrossRefPubMed Hamza TH, van Houwelingen HC, Stijnen T. The binomial distribution of meta-analysis was preferred to model within-study variability. J Clin Epidemiol. 2008; 61:41–51.CrossRefPubMed
5.
go back to reference Littenberg B, Moses LE. Estimating diagnostic-accuracy from multiple conflicting reports: a new meta-analytic method. Med Dec Making. 1993; 13:313–21.CrossRef Littenberg B, Moses LE. Estimating diagnostic-accuracy from multiple conflicting reports: a new meta-analytic method. Med Dec Making. 1993; 13:313–21.CrossRef
6.
go back to reference Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993; 12:1293–316.CrossRefPubMed Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993; 12:1293–316.CrossRefPubMed
7.
go back to reference Reitsma JB, Glas AS, Rutjes AWS, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005; 58:982–90.CrossRefPubMed Reitsma JB, Glas AS, Rutjes AWS, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005; 58:982–90.CrossRefPubMed
8.
go back to reference Arends LR, Hamza TH, van Houwelingen HC, Heijenbrok-Kal MH, Hunink MGM, Stijnen T. Bivariate random effects meta-analysis of ROC curves. Med Dec Making. 2008; 28:621–38.CrossRef Arends LR, Hamza TH, van Houwelingen HC, Heijenbrok-Kal MH, Hunink MGM, Stijnen T. Bivariate random effects meta-analysis of ROC curves. Med Dec Making. 2008; 28:621–38.CrossRef
10.
go back to reference Chu H, Nie L, Chen Y, Huang Y, Sun W. Bivariate random effects models for meta-analysis of comparative studies with binary outcomes: methods for the absolute risk difference and relative risk. Stat Methods Med Res. 2012; 21:621–33.CrossRefPubMed Chu H, Nie L, Chen Y, Huang Y, Sun W. Bivariate random effects models for meta-analysis of comparative studies with binary outcomes: methods for the absolute risk difference and relative risk. Stat Methods Med Res. 2012; 21:621–33.CrossRefPubMed
11.
go back to reference Kuss O, Hoyer A, Solms A. Meta-analysis for diagnostic accuracy studies: a new statistical model using beta-binomial distributions and bivariate copulas. Stat Med. 2014; 33:17–30.CrossRefPubMed Kuss O, Hoyer A, Solms A. Meta-analysis for diagnostic accuracy studies: a new statistical model using beta-binomial distributions and bivariate copulas. Stat Med. 2014; 33:17–30.CrossRefPubMed
12.
go back to reference Nikoloulopoulos AK. A mixed effect model for bivariate meta-analysis of diagnostic test accuracy studies using a copula representation of the random effects distribution. Stat Med. 2015; 34:3842–65.CrossRefPubMed Nikoloulopoulos AK. A mixed effect model for bivariate meta-analysis of diagnostic test accuracy studies using a copula representation of the random effects distribution. Stat Med. 2015; 34:3842–65.CrossRefPubMed
13.
go back to reference Eusebi P, Reitsma JB, Vermunt JK. Latent class bivariate model for the meta-analysis of diagnostic test accuracy studies. BMC Med Res Methodol. 2014; 14:88. Eusebi P, Reitsma JB, Vermunt JK. Latent class bivariate model for the meta-analysis of diagnostic test accuracy studies. BMC Med Res Methodol. 2014; 14:88.
14.
go back to reference Schlattmann P, Verba M, Dewey M, Walther M. Mixture models in diagnostic meta-analyses – clustering summary receiver operating characteristic curves accounted for heterogeneity and correlation. J Clin Epidemiol. 2015; 68:61–72.CrossRefPubMed Schlattmann P, Verba M, Dewey M, Walther M. Mixture models in diagnostic meta-analyses – clustering summary receiver operating characteristic curves accounted for heterogeneity and correlation. J Clin Epidemiol. 2015; 68:61–72.CrossRefPubMed
15.
go back to reference Zapf A, Hoyer A, Kramer K, Kuss O. Nonparametric meta-analysis for diagnostic accuracy studies. Stat Med. 2015; 34:3831–41.CrossRefPubMed Zapf A, Hoyer A, Kramer K, Kuss O. Nonparametric meta-analysis for diagnostic accuracy studies. Stat Med. 2015; 34:3831–41.CrossRefPubMed
16.
go back to reference Hamza TH, Reitsma JB, Stijnen T. Meta-analysis of diagnostic studies: A comparison of random intercept, normal-normal, and binomial-normal bivariate summary ROC approaches. Med Dec Making. 2008; 28:639–49.CrossRef Hamza TH, Reitsma JB, Stijnen T. Meta-analysis of diagnostic studies: A comparison of random intercept, normal-normal, and binomial-normal bivariate summary ROC approaches. Med Dec Making. 2008; 28:639–49.CrossRef
17.
go back to reference Paul M, Riebler A, Bachmann LM, Rue H, Held L. Bayesian bivariate meta-analysis of diagnostic test studies using integrated nested Laplace approximations. Stat Med. 2010; 29:1325–9.CrossRefPubMed Paul M, Riebler A, Bachmann LM, Rue H, Held L. Bayesian bivariate meta-analysis of diagnostic test studies using integrated nested Laplace approximations. Stat Med. 2010; 29:1325–9.CrossRefPubMed
18.
go back to reference Diaz M. Performance measures of the bivariate random effects model for meta-analyses of diagnostic accuracy. Comput Stat Data Anal. 2015; 83:82–90.CrossRef Diaz M. Performance measures of the bivariate random effects model for meta-analyses of diagnostic accuracy. Comput Stat Data Anal. 2015; 83:82–90.CrossRef
20.
go back to reference Cook JR, Stefanski LA. Simulation extrapolation estimation in parametric measurement error models. J Am Stat Assoc. 1994; 89:1314–28.CrossRef Cook JR, Stefanski LA. Simulation extrapolation estimation in parametric measurement error models. J Am Stat Assoc. 1994; 89:1314–28.CrossRef
21.
go back to reference Stefanski LA, Cook JR. Simulation-extrapolation: the measurement error jackknife. J Am Stat Assoc. 1995; 90:1247–56.CrossRef Stefanski LA, Cook JR. Simulation-extrapolation: the measurement error jackknife. J Am Stat Assoc. 1995; 90:1247–56.CrossRef
22.
go back to reference Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001; 20:2865–84.CrossRefPubMed Rutter CM, Gatsonis CA. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001; 20:2865–84.CrossRefPubMed
23.
go back to reference Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JA. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007; 8:239–51.CrossRefPubMed Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JA. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007; 8:239–51.CrossRefPubMed
24.
go back to reference Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006; 59:1331–3.CrossRefPubMed Chu H, Cole SR. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006; 59:1331–3.CrossRefPubMed
25.
go back to reference Van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Stat Med. 2002; 21:589–624.CrossRefPubMed Van Houwelingen HC, Arends LR, Stijnen T. Advanced methods in meta-analysis: multivariate approach and meta-regression. Stat Med. 2002; 21:589–624.CrossRefPubMed
26.
go back to reference McIntosh MW. The population risk as an explanatory variable in research synthesis of clinical trials. Stat Med. 1996; 15:1713–28.CrossRefPubMed McIntosh MW. The population risk as an explanatory variable in research synthesis of clinical trials. Stat Med. 1996; 15:1713–28.CrossRefPubMed
27.
go back to reference Schmid CH, Lau J, McIntosh MW, Cappelleri JC. An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Stat Med. 1998; 17:1923–42.CrossRefPubMed Schmid CH, Lau J, McIntosh MW, Cappelleri JC. An empirical study of the effect of the control rate as a predictor of treatment efficacy in meta-analysis of clinical trials. Stat Med. 1998; 17:1923–42.CrossRefPubMed
28.
go back to reference Carroll RJ, Ruppert D, Stefanski LA, Crainiceanu C. Measurement Error in Nonlinear Models: A Modern Perspective. Boca Raton: Chapman & Hall, CRC Press; 2006.CrossRef Carroll RJ, Ruppert D, Stefanski LA, Crainiceanu C. Measurement Error in Nonlinear Models: A Modern Perspective. Boca Raton: Chapman & Hall, CRC Press; 2006.CrossRef
29.
go back to reference Buonaccorsi JP. Measurement Error: Models, Methods and Applications. Boca Raton: Chapman & Hall, CRC Press; 2010.CrossRef Buonaccorsi JP. Measurement Error: Models, Methods and Applications. Boca Raton: Chapman & Hall, CRC Press; 2010.CrossRef
30.
go back to reference Arends LR, Hoes AW, Lubsen J, Grobbee DE, Stijnen T. Baseline risk as predictor of treatment benefit: three clinical meta-re-analyses. Stat Med. 2000; 19:3497–518.CrossRefPubMed Arends LR, Hoes AW, Lubsen J, Grobbee DE, Stijnen T. Baseline risk as predictor of treatment benefit: three clinical meta-re-analyses. Stat Med. 2000; 19:3497–518.CrossRefPubMed
31.
go back to reference Ghidey W, Stijnen T, van Houwelingen HC. Modelling the effect of baseline risk in meta-analysis: A review from the perspective of errors-in-variables regression. Stat Methods Med Res. 2013; 22:307–23.CrossRefPubMed Ghidey W, Stijnen T, van Houwelingen HC. Modelling the effect of baseline risk in meta-analysis: A review from the perspective of errors-in-variables regression. Stat Methods Med Res. 2013; 22:307–23.CrossRefPubMed
32.
go back to reference Guolo A. The SIMEX approach to measurement error correction in meta-analysis with baseline risk as covariate. Stat Med. 2014; 33:2062–76.CrossRefPubMed Guolo A. The SIMEX approach to measurement error correction in meta-analysis with baseline risk as covariate. Stat Med. 2014; 33:2062–76.CrossRefPubMed
33.
go back to reference Carroll RJ, Küchenhoff H, Lombard F, Stefanski LA. Asymptotics for the SIMEX estimator in nonlinear measurement error models. J Am Stat Assoc. 1996; 91:242–50.CrossRef Carroll RJ, Küchenhoff H, Lombard F, Stefanski LA. Asymptotics for the SIMEX estimator in nonlinear measurement error models. J Am Stat Assoc. 1996; 91:242–50.CrossRef
34.
go back to reference Holcomb J. Regression with covariates and outcome calculated from a common set of variables measured with error: Estimation using the SIMEX method. Stat Med. 1999; 18:2847–62.CrossRefPubMed Holcomb J. Regression with covariates and outcome calculated from a common set of variables measured with error: Estimation using the SIMEX method. Stat Med. 1999; 18:2847–62.CrossRefPubMed
35.
go back to reference Azzalini A. A class of distributions which includes the normal ones. Scand J Stat. 1985; 12:171–8. Azzalini A. A class of distributions which includes the normal ones. Scand J Stat. 1985; 12:171–8.
36.
go back to reference Nelder JA, Mead R. A simplex algorithm for function minimization. Scand J Stat. 1965; 7:308–13. Nelder JA, Mead R. A simplex algorithm for function minimization. Scand J Stat. 1965; 7:308–13.
38.
go back to reference Van Zaane B, Zuithoff NPA, Reitsma JB, Bax L, Nierich AP, Moons KG. Meta-analysis of the diagnostic accuracy of transesophageal echocardiography for assessment of atherosclerosis in the ascending aorta in patients undergoing cardiac surgery. Acta Anaesthesiol Scand. 2008; 52:1179–87.CrossRefPubMed Van Zaane B, Zuithoff NPA, Reitsma JB, Bax L, Nierich AP, Moons KG. Meta-analysis of the diagnostic accuracy of transesophageal echocardiography for assessment of atherosclerosis in the ascending aorta in patients undergoing cardiac surgery. Acta Anaesthesiol Scand. 2008; 52:1179–87.CrossRefPubMed
39.
go back to reference Küchenhoff H, Mwalili SM, Lesaffre E. A general method for dealing with misclassification in regression: The misclassification SIMEX. Biometrics. 2006; 62:85–96.CrossRefPubMed Küchenhoff H, Mwalili SM, Lesaffre E. A general method for dealing with misclassification in regression: The misclassification SIMEX. Biometrics. 2006; 62:85–96.CrossRefPubMed
Metadata
Title
A double SIMEX approach for bivariate random-effects meta-analysis of diagnostic accuracy studies
Author
Annamaria Guolo
Publication date
01-12-2017
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2017
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-016-0284-2

Other articles of this Issue 1/2017

BMC Medical Research Methodology 1/2017 Go to the issue