Top

Published in:

Open Access 01-12-2019 | Prostate Cancer | Research article

Multi-cohort modeling strategies for scalable globally accessible prostate cancer risk tools

Authors: Johanna Tolksdorf, Michael W. Kattan, Stephen A. Boorjian, Stephen J. Freedland, Karim Saba, Cedric Poyet, Lourdes Guerrios, Amanda De Hoedt, Michael A. Liss, Robin J. Leach, Javier Hernandez, Emily Vertosick, Andrew J. Vickers, Donna P. Ankerst

Published in: BMC Medical Research Methodology | Issue 1/2019

Abstract

Background

Online clinical risk prediction tools built on data from multiple cohorts are increasingly being utilized for contemporary doctor-patient decision-making and validation. This report outlines a comprehensive data science strategy for building such tools with application to the Prostate Biopsy Collaborative Group prostate cancer risk prediction tool.

Methods

We created models for high-grade prostate cancer risk using six established risk factors. The data comprised 8492 prostate biopsies collected from ten institutions, 2 in Europe and 8 across North America. We calculated area under the receiver operating characteristic curve (AUC) for discrimination, the Hosmer-Lemeshow test statistic (HLS) for calibration and the clinical net benefit at risk threshold 15%. We implemented several internal cross-validation schemes to assess the influence of modeling method and individual cohort on validation performance.

Results

High-grade disease prevalence ranged from 18% in Zurich (1863 biopsies) to 39% in UT Health San Antonio (899 biopsies). Visualization revealed outliers in terms of risk factors, including San Juan VA (51% abnormal digital rectal exam), Durham VA (63% African American), and Zurich (2.8% family history). Exclusion of any cohort did not significantly affect the AUC or HLS, nor did the choice of prediction model (pooled, random-effects, meta-analysis). Excluding the lowest-prevalence Zurich cohort from training sets did not statistically significantly change the validation metrics for any of the individual cohorts, except for Sunnybrook, where the effect on the AUC was minimal. Therefore the final multivariable logistic model was built by pooling the data from all cohorts using logistic regression. Higher prostate-specific antigen and age, abnormal digital rectal exam, African ancestry and a family history of prostate cancer increased risk of high-grade prostate cancer, while a history of a prior negative prostate biopsy decreased risk (all p-values < 0.004).

Conclusions

We have outlined a multi-cohort model-building internal validation strategy for developing globally accessible and scalable risk prediction tools.

Schröder FH, Hugosson J, Roobol MJ, Tammela TL, Ciatto S, Nelen V, Kwiatkowski M, Lujan M, Lilja H, Zappa M, Denis LJ, Recker F, Berenguer A, Määttänen L, Bangma CH, Aus G, Villers A, Rebillard X, van der Kwast T, Blijenberg BG, Moss SM, de Koning HJ, Auvinen A, ERSPC Investigators. Screening and prostate-cancer mortality in a randomized European study. N Engl J Med. 2009;360(13):1320–8. https://doi.org/10.1056/NEJMoa0810084.CrossRefPubMed

Thompson IM, Ankerst DP, Chi C, Goodman PJ, Tangen CM, Lucia MS, Feng Z, Parnes HL, Coltman CA Jr. Assessing prostate cancer risk: results from the Prostate Cancer Prevention Trial. J Natl Cancer Inst. 2006;98(8):529–34. https://doi.org/10.1093/jnci/djj131.CrossRefPubMed

Chen R, Sjoberg DD, Huang Y, Xie L, Zhou L, He D, Vickers AJ, Sun Y, Chinese Prostate Cancer Consortium, Prostate Biopsy Collaborative Group. Prostate specific antigen and prostate cancer in Chinese men undergoing initial prostate biopsies compared with western cohorts. J Urol. 2017;197(1):90–6. https://doi.org/10.1016/j.juro.2016.08.103.CrossRefPubMed

Ankerst DP, Boeck A, Freedland SJ, Jones JS, Cronin AM, Roobol MJ, Hugosson J, Kattan MW, Klein EA, Hamdy F, Neal D, Donovan J, Parekh DJ, Klocker H, Horninger W, Benchikh A, Salama G, Villers A, Moreira DM, Schröder FH, Lilja H, Vickers AJ, Thompson IM. Evaluating the prostate cancer prevention trial high grade prostate cancer risk calculator in 10 international biopsy cohorts: results from the Prostate Biopsy Collaborative Group. World J Urol. 2014;32(1):185–91. https://doi.org/10.1007/s00345-012-0869-2.CrossRefPubMed

Strobl AN, Thompson IM, Vickers AJ, Ankerst DP. The next generation of clinical decision making tools: development of a real-time prediction tool for outcome of prostate biopsy in response to a continuously evolving prostate cancer landscape. J Urol. 2015;194(1):58–64. https://doi.org/10.1016/j.juro.2015.01.092.CrossRefPubMedPubMedCentral

Vickers AJ, Cronin AM, Roobol MJ, Hugosson J, Jones JS, Kattan MW, Klein E, Hamdy F, Neal D, Donovan J, Parekh DJ, Ankerst D, Bartsch G, Klocker H, Horninger W, Benchikh A, Salama G, Villers A, Freedland SJ, Moreira DM, Schröder FH, Lilja H. The relationship between prostate-specific antigen and prostate cancer risk: the Prostate Biopsy Collaborative Group. Clin Cancer Res. 2010;16(17):4374–81. https://doi.org/10.1158/1078-0432.CCR-10-1328.CrossRefPubMedPubMedCentral

Ankerst DP, Straubinger J, Selig K, Guerrios L, De Hoedt A, Hernandez J, Liss MA, Leach RJ, Freedland SJ, Kattan MW, Nam R, Haese A, Montorsi F, Boorjian SA, Cooperberg MR, Poyet C, Vertosick E, Vickers AJ. A contemporary prostate biopsy risk calculator based on multiple heterogeneous cohorts. Eur Urol. 2018;74(2):197–203. https://doi.org/10.1016/j.eururo.2018.05.003.CrossRefPubMedPubMedCentral

Debray TP, Moons KG, Abo-Zaid GM, Koffijberg H, Riley RD. Individual participant data meta-analysis for a binary outcome: one-stage or two-stage? PLoS One. 2013;8(4):e60650. https://doi.org/10.1371/journal.pone.0060650.CrossRefPubMedPubMedCentral

R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2019. https://www.R-project.org/

10.

Wickham H. ggplot2: elegant graphics for data analysis. New York: Springer-Verlag; 2016.CrossRef

11.

Collins GS, Reitsma JB, Altman DG, Moons KG. Transparent reporting of a multivariable prediction model for individual prognosis or diagnosis (TRIPOD): the TRIPOD statement. BMC Med. 2015;13:1. https://doi.org/10.1186/s12916-014-0241-z.CrossRefPubMedPubMedCentral

12.

Skrondal A, Rabe-Hesketh S. Prediction in multilevel generalized linear models. J R Stat Soc Ser A. 2009;172(3):659–87. https://doi.org/10.1111/j.1467-985X.2009.00587.x.CrossRef

13.

Janssen KJ, Donders AR, Harrell FE Jr, Vergouwe Y, Chen Q, Grobbee DE, Moons KG. Missing covariate data in medical research: to impute is better than to ignore. J Clin Epidemiol. 2010;63(7):721–7. https://doi.org/10.1016/j.jclinepi.2009.12.008.CrossRefPubMed

14.

Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology. 1982;143(1):29–36. https://doi.org/10.1148/radiology.143.1.7063747.CrossRefPubMed

15.

Hosmer DW, Lemeshow S. Applied logistic regression. 3rd ed. New Jersey: Wiley; 2013. https://doi.org/10.1002/9781118548387.CrossRef

16.

Vickers AJ, Elkin EB. Decision curve analysis: a novel method for evaluating prediction models. Med Decis Making. 2006;26(6):565–74. https://doi.org/10.1177/0272989X06295361.CrossRefPubMedPubMedCentral

17.

Debray TPA, Moons KGM, Ahmed I, Koffijberg H, Riley RD. A framework for developing, implementing, and evaluating clinical prediction models in an individual participant data meta-analysis. Stat Med. 2013;32(18):3158–80. https://doi.org/10.1002/sim.5732.CrossRefPubMed

18.

Ernst MD. Permutation methods: a basis for exact inference. Statistical Sci. 2004;19(4):676–85. https://doi.org/10.1214/088342304000000396.CrossRef

19.

Ahmed I, Debray TP, Moons KG, Riley RD. Developing and validating risk prediction models in an individual participant data meta-analysis. BMC Med Res Methodol. 2014;14:3. https://doi.org/10.1186/1471-2288-14-3.CrossRefPubMedPubMedCentral

20.

Debray TPA, Riley RD, Rovers MM, Reitsma JB, Moons KGM, Cochrane IPD Meta-analysis Methods group. Individual participant data (IPD) meta-analyses of diagnostic and prognostic modeling studies: guidance on their use. PLoS Med. 2015;12(10):e1001886. https://doi.org/10.1371/journal.pmed.1001886.CrossRefPubMedPubMedCentral

21.

Poelsterl S, Gupta P, Wang L, Conjeti S, Katouzian A, Navab N. Heterogeneous ensembles for predicting survival of metastatic, castrate-resistant prostate cancer patients. F1000Res. 2016;5:2676. https://doi.org/10.12688/f1000research.8231.3.CrossRef

Title: Multi-cohort modeling strategies for scalable globally accessible prostate cancer risk tools
Authors: Johanna Tolksdorf
Michael W. Kattan
Stephen A. Boorjian
Stephen J. Freedland
Karim Saba
Cedric Poyet
Lourdes Guerrios
Amanda De Hoedt
Michael A. Liss
Robin J. Leach
Javier Hernandez
Emily Vertosick
Andrew J. Vickers
Donna P. Ankerst
Publication date: 01-12-2019
Publisher: BioMed Central
Keywords: Prostate Cancer
Prostate Cancer
Published in: BMC Medical Research Methodology / Issue 1/2019
Electronic ISSN: 1471-2288
DOI: https://doi.org/10.1186/s12874-019-0839-0

At a glance: The STEP trials

Springer Medicine

Multi-cohort modeling strategies for scalable globally accessible prostate cancer risk tools

Abstract

Background

Methods

Results

Conclusions

At a glance: The STEP trials

Springer Medicine

Abstract

Background

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2019

Estimating the loss of lifetime function using flexible parametric relative survival models

Concurrent validity of the Fitbit for assessing sedentary behavior and moderate-to-vigorous physical activity

The Chinese version of attitudes towards guidelines scale: validity and reliability assessment

The utility of multivariate outlier detection techniques for data quality evaluation in large studies: an application within the ONDRI project

Using Bayesian adaptive designs to improve phase III trials: a respiratory care example

Correction to: Feasibility of identifying important changes in care management resulting from cardiovascular magnetic resonance (CMR) using hospital episode data in patients who activate the primary percutaneous coronary intervention (PPCI) pathway