Skip to main content
Top
Published in: BMC Medical Research Methodology 1/2015

Open Access 01-12-2015 | Research Article

Synthesis of clinical prediction models under different sets of covariates with one individual patient data

Authors: Daisuke Yoneoka, Masayuki Henmi, Norie Sawada, Manami Inoue

Published in: BMC Medical Research Methodology | Issue 1/2015

Login to get access

Abstract

Background

Recently, increased development of clinical prediction models has been reported in the medical literature. However, evidence synthesis methodologies for these prediction models have not been sufficiently studied, especially for practical situations such as a meta-analyses where only aggregated summaries of important predictors are available. Also, in general, the covariate sets involved in the prediction models are not common across studies. As in ordinary model misspecification problems, dropping relevant covariates would raise potentially serious biases to the prediction models, and consequently to the synthesized results.

Methods

We developed synthesizing methods for logistic clinical prediction models with possibly different sets of covariates. In order to aggregate the regression coefficient estimates from different prediction models, we adopted a generalized least squares approach with non-linear terms (a sort of generalization of multivariate meta-analysis). Firstly, we evaluated omitted variable biases in this approach. Then, under an assumption of homogeneity of studies, we developed bias-corrected estimating procedures for regression coefficients of the synthesized prediction models.

Results

Numerical evaluations with simulations showed that our approach resulted in smaller biases and more precise estimates compared with conventional methods, which use only studies with common covariates or which utilize a mean imputation method for omitted coefficients. These methods were also applied to a series of Japanese epidemiologic studies on the incidence of a stroke.

Conclusions

Our proposed methods adequately correct the biases due to different sets of covariates between studies, and would provide precise estimates compared with the conventional approach. If the assumption of homogeneity within studies is plausible, this methodology would be useful for incorporating prior published information into the construction of new prediction models.
Literature
1.
go back to reference Matheny M, McPheeters ML, Glasser A, Mercaldo N, Weaver RB, Jerome RN, et al.Systematic review of cardiovascular disease risk assessment tools. USA: Agency for Healthcare Research and Quality; 2011. Matheny M, McPheeters ML, Glasser A, Mercaldo N, Weaver RB, Jerome RN, et al.Systematic review of cardiovascular disease risk assessment tools. USA: Agency for Healthcare Research and Quality; 2011.
2.
go back to reference Abbasi A, Peelen LM, Corpeleijn E, van der Schouw YT, Stolk RP, Spijkerman AM, et al. Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ: Br Med J. 2012;345. Abbasi A, Peelen LM, Corpeleijn E, van der Schouw YT, Stolk RP, Spijkerman AM, et al. Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ: Br Med J. 2012;345.
3.
go back to reference Steyerberg E, Eijkemans M, Van Houwelingen J, Lee K, Habbema J. Prognostic models based on literature and individual patient data in logistic regression analysis. Stat Med. 2000; 19(2):141–60.CrossRefPubMed Steyerberg E, Eijkemans M, Van Houwelingen J, Lee K, Habbema J. Prognostic models based on literature and individual patient data in logistic regression analysis. Stat Med. 2000; 19(2):141–60.CrossRefPubMed
4.
go back to reference Bleeker S, Moll H, Steyerberg E, Donders A, Derksen-Lubsen G, Grobbee D, et al.External validation is necessary in prediction research:: A clinical example. J Clin Epidemiol. 2003; 56(9):826–32.CrossRefPubMed Bleeker S, Moll H, Steyerberg E, Donders A, Derksen-Lubsen G, Grobbee D, et al.External validation is necessary in prediction research:: A clinical example. J Clin Epidemiol. 2003; 56(9):826–32.CrossRefPubMed
5.
go back to reference Steyerberg EW, Bleeker SE, Moll HA, Grobbee DE, Moons KG. Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol. 2003; 56(5):441–7.CrossRefPubMed Steyerberg EW, Bleeker SE, Moll HA, Grobbee DE, Moons KG. Internal and external validation of predictive models: a simulation study of bias and precision in small samples. J Clin Epidemiol. 2003; 56(5):441–7.CrossRefPubMed
6.
go back to reference Debray T, Koffijberg H, Vergouwe Y, Moons KG, Steyerberg EW. Aggregating published prediction models with individual participant data: a comparison of different approaches. Stat Med. 2012; 31(23):2697–712.CrossRefPubMed Debray T, Koffijberg H, Vergouwe Y, Moons KG, Steyerberg EW. Aggregating published prediction models with individual participant data: a comparison of different approaches. Stat Med. 2012; 31(23):2697–712.CrossRefPubMed
7.
go back to reference Becker BJ, Wu MJ. The synthesis of regression slopes in meta-analysis. Stat Sci. 2007; 22(3):414–29.CrossRef Becker BJ, Wu MJ. The synthesis of regression slopes in meta-analysis. Stat Sci. 2007; 22(3):414–29.CrossRef
8.
go back to reference Balázs K, Hidegkuti I, De Boeck P. Detecting heterogeneity in logistic regression models. Appl Psychol Meas. 2006; 30(4):322–44.CrossRef Balázs K, Hidegkuti I, De Boeck P. Detecting heterogeneity in logistic regression models. Appl Psychol Meas. 2006; 30(4):322–44.CrossRef
9.
go back to reference Higgins J, Thompson S, Deeks J, Altman D. Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice. J Health Serv Res Policy. 2002; 7(1):51–61.CrossRefPubMed Higgins J, Thompson S, Deeks J, Altman D. Statistical heterogeneity in systematic reviews of clinical trials: a critical appraisal of guidelines and practice. J Health Serv Res Policy. 2002; 7(1):51–61.CrossRefPubMed
10.
go back to reference Wu MJ, Becker BJ. Synthesizing regression results: a factored likelihood method. Res Synth Methods. 2013; 4(2):127–43.CrossRefPubMed Wu MJ, Becker BJ. Synthesizing regression results: a factored likelihood method. Res Synth Methods. 2013; 4(2):127–43.CrossRefPubMed
11.
go back to reference The Fibrinogen Studies Collaboration. Systematically missing confounders in individual participant data meta-analysis of observational cohort studies. Stat Med. 2009; 28(8):1218.CrossRefPubMedCentral The Fibrinogen Studies Collaboration. Systematically missing confounders in individual participant data meta-analysis of observational cohort studies. Stat Med. 2009; 28(8):1218.CrossRefPubMedCentral
12.
go back to reference Riley R, Price M, Jackson D, Wardle M, Gueyffier F, Wang J, et al.Multivariate meta-analysis using individual participant data. Res Synth Methods. 2014; 6:157–74.CrossRefPubMedPubMedCentral Riley R, Price M, Jackson D, Wardle M, Gueyffier F, Wang J, et al.Multivariate meta-analysis using individual participant data. Res Synth Methods. 2014; 6:157–74.CrossRefPubMedPubMedCentral
13.
go back to reference Resche-Rigon M, White IR, Bartlett JW, Peters SA, Thompson SG. Multiple imputation for handling systematically missing confounders in meta-analysis of individual participant data. Stat Med. 2013; 32(28):4890–905.CrossRefPubMed Resche-Rigon M, White IR, Bartlett JW, Peters SA, Thompson SG. Multiple imputation for handling systematically missing confounders in meta-analysis of individual participant data. Stat Med. 2013; 32(28):4890–905.CrossRefPubMed
14.
go back to reference Cox DR, Oakes D. Analysis of survival data. Chapman & Hall/CRC Monographs on Statistics & Applied Probability, vol. 21. UK: CRC Press; 1984. Cox DR, Oakes D. Analysis of survival data. Chapman & Hall/CRC Monographs on Statistics & Applied Probability, vol. 21. UK: CRC Press; 1984.
15.
go back to reference Yi GY, Reid N. A note on mis-specified estimating functions. Stat Sinica. 2010; 20:1749–69. Yi GY, Reid N. A note on mis-specified estimating functions. Stat Sinica. 2010; 20:1749–69.
16.
go back to reference White H. Maximum likelihood estimation of misspecified models. Econometrica: J Econ Soc. 1982; 50(1):1–25.CrossRef White H. Maximum likelihood estimation of misspecified models. Econometrica: J Econ Soc. 1982; 50(1):1–25.CrossRef
17.
go back to reference Chao WH, Palta M, Young T. Effect of omitted confounders on the analysis of correlated binary data. Biometrics. 1997; 53(2):678–89.CrossRefPubMed Chao WH, Palta M, Young T. Effect of omitted confounders on the analysis of correlated binary data. Biometrics. 1997; 53(2):678–89.CrossRefPubMed
18.
go back to reference Johnson NL, Kotz S. Distributions in statistics: continuous univariate distributions, 2nd edn. Wiley Series in Probability and Statistics, vol. 2. USA: Houghton Mifflin; 1970. Johnson NL, Kotz S. Distributions in statistics: continuous univariate distributions, 2nd edn. Wiley Series in Probability and Statistics, vol. 2. USA: Houghton Mifflin; 1970.
19.
go back to reference Cramer JS. Logit models from economics and other fields. UK: Cambridge University Press; 2003.CrossRef Cramer JS. Logit models from economics and other fields. UK: Cambridge University Press; 2003.CrossRef
20.
go back to reference Greene WH. Econometric analysis, 7th edn. USA: Pearson Education; 2003. Greene WH. Econometric analysis, 7th edn. USA: Pearson Education; 2003.
21.
go back to reference Liu D, Liu R, Xie M. Multivariate meta-analysis of heterogeneous studies using only summary statistics: efficiency and robustness. J Am Stat Assoc. 2015; 110(509):326–40.CrossRefPubMed Liu D, Liu R, Xie M. Multivariate meta-analysis of heterogeneous studies using only summary statistics: efficiency and robustness. J Am Stat Assoc. 2015; 110(509):326–40.CrossRefPubMed
22.
go back to reference Chen Y, Hong C, Riley RD. An alternative pseudolikelihood method for multivariate random-effects meta-analysis. Stat Med. 2015; 34(3):361–80.CrossRefPubMed Chen Y, Hong C, Riley RD. An alternative pseudolikelihood method for multivariate random-effects meta-analysis. Stat Med. 2015; 34(3):361–80.CrossRefPubMed
23.
go back to reference Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Blaha MJ, et al.Heart disease and stroke statistics–2014 update: a report from the american heart association. Circulation. 2014; 129(3):28.CrossRef Go AS, Mozaffarian D, Roger VL, Benjamin EJ, Berry JD, Blaha MJ, et al.Heart disease and stroke statistics–2014 update: a report from the american heart association. Circulation. 2014; 129(3):28.CrossRef
24.
go back to reference Johnston K, Connors A, Wagner D, Knaus W, Wang XQ, Haley EC, et al.A predictive risk model for outcomes of ischemic stroke. Stroke. 2000; 31(2):448–55.CrossRefPubMed Johnston K, Connors A, Wagner D, Knaus W, Wang XQ, Haley EC, et al.A predictive risk model for outcomes of ischemic stroke. Stroke. 2000; 31(2):448–55.CrossRefPubMed
25.
go back to reference Tsugane S, Sawada N. The jphc study: design and some findings on the typical japanese diet. Jpn J Clin Oncol. 2014; 44(9):777–82.CrossRefPubMed Tsugane S, Sawada N. The jphc study: design and some findings on the typical japanese diet. Jpn J Clin Oncol. 2014; 44(9):777–82.CrossRefPubMed
26.
go back to reference Walker A, Robins M, Weinfeld F. The national survey of stroke. clinical findings. Stroke; J Cerebral Circ. 1981; 12(2 Pt 2 Suppl 1):13. Walker A, Robins M, Weinfeld F. The national survey of stroke. clinical findings. Stroke; J Cerebral Circ. 1981; 12(2 Pt 2 Suppl 1):13.
27.
go back to reference Iso H, Rexrode K, Hennekens CH, Manson JE. Application of computer tomography-oriented criteria for stroke subtype classification in a prospective study. Ann Epidemiol. 2000; 10(2):81–7.CrossRefPubMed Iso H, Rexrode K, Hennekens CH, Manson JE. Application of computer tomography-oriented criteria for stroke subtype classification in a prospective study. Ann Epidemiol. 2000; 10(2):81–7.CrossRefPubMed
28.
go back to reference Saito I, Iso H, Kokubo Y, Inoue M, Tsugane S. Body mass index, weight change and risk of stroke and stroke subtypes: the japan public health center-based prospective (jphc) study. Int J Obes. 2010; 35(2):283–91.CrossRef Saito I, Iso H, Kokubo Y, Inoue M, Tsugane S. Body mass index, weight change and risk of stroke and stroke subtypes: the japan public health center-based prospective (jphc) study. Int J Obes. 2010; 35(2):283–91.CrossRef
29.
go back to reference Yatsuya H, Iso H, Yamagishi K, Kokubo Y, Saito I, Suzuki K, et al.Development of a point-based prediction model for the incidence of total stroke japan public health center study. Stroke. 2013; 44(5):1295–302.CrossRefPubMed Yatsuya H, Iso H, Yamagishi K, Kokubo Y, Saito I, Suzuki K, et al.Development of a point-based prediction model for the incidence of total stroke japan public health center study. Stroke. 2013; 44(5):1295–302.CrossRefPubMed
30.
go back to reference Noda H, Iso H, Saito I, Konishi M, Inoue M, Tsugane S. The impact of the metabolic syndrome and its components on the incidence of ischemic heart disease and stroke: the Japan public health center-based study. Hypertens Res. 2009; 32(4):289–98.CrossRefPubMed Noda H, Iso H, Saito I, Konishi M, Inoue M, Tsugane S. The impact of the metabolic syndrome and its components on the incidence of ischemic heart disease and stroke: the Japan public health center-based study. Hypertens Res. 2009; 32(4):289–98.CrossRefPubMed
31.
go back to reference Pepe MS. The statistical evaluation of medical tests for classification and prediction, 1st edn. Oxford Statistical Science Series. USA: Oxford University Press; 2003. Pepe MS. The statistical evaluation of medical tests for classification and prediction, 1st edn. Oxford Statistical Science Series. USA: Oxford University Press; 2003.
32.
go back to reference Collins GS, de Groot JA, Dutton S, Omar O, Shanyinde M, Tajar A, et al.External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med Res Methodol. 2014; 14(1):40.CrossRefPubMedPubMedCentral Collins GS, de Groot JA, Dutton S, Omar O, Shanyinde M, Tajar A, et al.External validation of multivariable prediction models: a systematic review of methodological conduct and reporting. BMC Med Res Methodol. 2014; 14(1):40.CrossRefPubMedPubMedCentral
33.
go back to reference Riley RD, Abrams K, Lambert P, Sutton A, Thompson J. An evaluation of bivariate random-effects meta-analysis for the joint synthesis of two correlated outcomes. Stat Med. 2007; 26(1):78–97.CrossRefPubMed Riley RD, Abrams K, Lambert P, Sutton A, Thompson J. An evaluation of bivariate random-effects meta-analysis for the joint synthesis of two correlated outcomes. Stat Med. 2007; 26(1):78–97.CrossRefPubMed
34.
go back to reference Riley RD, Abrams KR, Sutton AJ, Lambert PC, Thompson JR. Bivariate random-effects meta-analysis and the estimation of between-study correlation. BMC Med Res Methodol. 2007; 7(1):3.CrossRefPubMedPubMedCentral Riley RD, Abrams KR, Sutton AJ, Lambert PC, Thompson JR. Bivariate random-effects meta-analysis and the estimation of between-study correlation. BMC Med Res Methodol. 2007; 7(1):3.CrossRefPubMedPubMedCentral
Metadata
Title
Synthesis of clinical prediction models under different sets of covariates with one individual patient data
Authors
Daisuke Yoneoka
Masayuki Henmi
Norie Sawada
Manami Inoue
Publication date
01-12-2015
Publisher
BioMed Central
Published in
BMC Medical Research Methodology / Issue 1/2015
Electronic ISSN: 1471-2288
DOI
https://doi.org/10.1186/s12874-015-0087-x

Other articles of this Issue 1/2015

BMC Medical Research Methodology 1/2015 Go to the issue