Skip to main content
Top
Published in: Journal of Translational Medicine 1/2023

Open Access 01-12-2023 | Research

A novel deep learning-based algorithm combining histopathological features with tissue areas to predict colorectal cancer survival from whole-slide images

Authors: Yan-Jun Li, Hsin-Hung Chou, Peng-Chan Lin, Meng-Ru Shen, Sun-Yuan Hsieh

Published in: Journal of Translational Medicine | Issue 1/2023

Login to get access

Abstract

Background

Many methodologies for selecting histopathological images, such as sample image patches or segment histology from regions of interest (ROIs) or whole-slide images (WSIs), have been utilized to develop survival models. With gigapixel WSIs exhibiting diverse histological appearances, obtaining clinically prognostic and explainable features remains challenging. Therefore, we propose a novel deep learning-based algorithm combining tissue areas with histopathological features to predict cancer survival.

Methods

The Cancer Genome Atlas Colon Adenocarcinoma (TCGA-COAD) dataset was used in this investigation. A deep convolutional survival model (DeepConvSurv) extracted histopathological information from the image patches of nine different tissue types, including tumors, lymphocytes, stroma, and mucus. The tissue map of the WSIs was segmented using image processing techniques that involved localizing and quantifying the tissue region. Six survival models with the concordance index (C-index) were used as the evaluation metrics.

Results

We extracted 128 histopathological features from four histological types and five tissue area features from WSIs to predict colorectal cancer survival. Our method performed better in six distinct survival models than the Whole Slide Histopathological Images Survival Analysis framework (WSISA), which adaptively sampled patches using K-means from WSIs. The best performance using histopathological features was 0.679 using LASSO-Cox. Compared to histopathological features alone, tissue area features increased the C-index by 2.5%. Based on histopathological features and tissue area features, our approach achieved performance of 0.704 with RIDGE-Cox.

Conclusions

A deep learning-based algorithm combining histopathological features with tissue area proved clinically relevant and effective for predicting cancer survival.
Appendix
Available only for authorised users
Literature
1.
go back to reference Fleming M, Ravula S, Tatishchev SF, Wang HL. Colorectal carcinoma: pathologic aspects. J Gastrointest Oncol. 2012;3(3):153–73.PubMedPubMedCentral Fleming M, Ravula S, Tatishchev SF, Wang HL. Colorectal carcinoma: pathologic aspects. J Gastrointest Oncol. 2012;3(3):153–73.PubMedPubMedCentral
2.
go back to reference Yang J, Ye H, Fan X, Li Y, Wu X, Zhao M, et al. Artificial intelligence for quantifying immune infiltrates interacting with stroma in colorectal cancer. J Transl Med. 2022;20(1):451.CrossRefPubMedPubMedCentral Yang J, Ye H, Fan X, Li Y, Wu X, Zhao M, et al. Artificial intelligence for quantifying immune infiltrates interacting with stroma in colorectal cancer. J Transl Med. 2022;20(1):451.CrossRefPubMedPubMedCentral
3.
go back to reference Chen PC, Yeh YM, Lin BW, Chan RH, Su PF, Liu YC, et al. A prediction model for tumor recurrence in stage II–III colorectal cancer patients: from a machine learning model to genomic profiling. Biomedicines. 2022;10(2):340.CrossRefPubMedPubMedCentral Chen PC, Yeh YM, Lin BW, Chan RH, Su PF, Liu YC, et al. A prediction model for tumor recurrence in stage II–III colorectal cancer patients: from a machine learning model to genomic profiling. Biomedicines. 2022;10(2):340.CrossRefPubMedPubMedCentral
4.
go back to reference Xu H, Cha YJ, Clemenceau JR, Choi J, Lee SH, Kang J, Hwang TH. Spatial analysis of tumor-infiltrating lymphocytes in histological sections using deep learning techniques predicts survival in colorectal carcinoma. J Pathol Clin Res. 2022;8(4):327–39.CrossRefPubMedPubMedCentral Xu H, Cha YJ, Clemenceau JR, Choi J, Lee SH, Kang J, Hwang TH. Spatial analysis of tumor-infiltrating lymphocytes in histological sections using deep learning techniques predicts survival in colorectal carcinoma. J Pathol Clin Res. 2022;8(4):327–39.CrossRefPubMedPubMedCentral
5.
go back to reference Kuo YT, Liao CK, Chen TC, Lai CC, Chiang SF, Chiang JM. A high density of PD-L1-expressing immune cells is significantly correlated with favorable disease free survival in nonmetastatic colorectal cancer. Medicine (Baltimore). 2022;101(3): e28573.CrossRefPubMed Kuo YT, Liao CK, Chen TC, Lai CC, Chiang SF, Chiang JM. A high density of PD-L1-expressing immune cells is significantly correlated with favorable disease free survival in nonmetastatic colorectal cancer. Medicine (Baltimore). 2022;101(3): e28573.CrossRefPubMed
6.
go back to reference Bong JW, Gim JA, Ju Y, Cheong C, Lee SI, Oh SC, et al. Prognosis and sensitivity of adjuvant chemotherapy in mucinous colorectal adenocarcinoma without distant metastasis. Cancers (Basel). 2022;14(5):1297.CrossRefPubMed Bong JW, Gim JA, Ju Y, Cheong C, Lee SI, Oh SC, et al. Prognosis and sensitivity of adjuvant chemotherapy in mucinous colorectal adenocarcinoma without distant metastasis. Cancers (Basel). 2022;14(5):1297.CrossRefPubMed
7.
go back to reference Wang K, Ma W, Wang J, Yu L, Zhang X, Wang Z, et al. Tumor-stroma ratio is an independent predictor for survival in esophageal squamous cell carcinoma. J Thorac Oncol. 2012;7(9):1457–61.CrossRefPubMed Wang K, Ma W, Wang J, Yu L, Zhang X, Wang Z, et al. Tumor-stroma ratio is an independent predictor for survival in esophageal squamous cell carcinoma. J Thorac Oncol. 2012;7(9):1457–61.CrossRefPubMed
8.
go back to reference Idos GE, Kwok J, Bonthala N, Kysh L, Gruber SB, Qu C. The prognostic implications of tumor infiltrating lymphocytes in colorectal cancer: a systematic review and meta-analysis. Sci Rep. 2020;10(1):3360.CrossRefPubMedPubMedCentral Idos GE, Kwok J, Bonthala N, Kysh L, Gruber SB, Qu C. The prognostic implications of tumor infiltrating lymphocytes in colorectal cancer: a systematic review and meta-analysis. Sci Rep. 2020;10(1):3360.CrossRefPubMedPubMedCentral
9.
10.
11.
go back to reference Srinidhi CL, Ciga O, Martel AL. Deep neural network models for computational histopathology: a survey. Med Image Anal. 2021;67: 101813.CrossRefPubMed Srinidhi CL, Ciga O, Martel AL. Deep neural network models for computational histopathology: a survey. Med Image Anal. 2021;67: 101813.CrossRefPubMed
12.
go back to reference Zhu X, Yao J, Huang J. Deep convolutional neural network for survival analysis with pathological images. In: Proceedings of the 2016 IEEE international conference on bioinformatics and biomedicine (BIBM); 2016. p. 544–7. Zhu X, Yao J, Huang J. Deep convolutional neural network for survival analysis with pathological images. In: Proceedings of the 2016 IEEE international conference on bioinformatics and biomedicine (BIBM); 2016. p. 544–7.
13.
go back to reference Yao J, Zhu X, Zhu F, Huang J. Deep correlational learning for survival prediction from multi-modality data. In: Proceedings of the international conference on medical image computing and computer-assisted intervention. Springer; 2017. p. 406–14. Yao J, Zhu X, Zhu F, Huang J. Deep correlational learning for survival prediction from multi-modality data. In: Proceedings of the international conference on medical image computing and computer-assisted intervention. Springer; 2017. p. 406–14.
14.
go back to reference Mobadersany P, Yousefi S, Amgad M, Gutman DA, Barnholtz-Sloan JS, Velázquez Vega JE, et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci U S A. 2018;115(13):E2970–9.CrossRefPubMedPubMedCentral Mobadersany P, Yousefi S, Amgad M, Gutman DA, Barnholtz-Sloan JS, Velázquez Vega JE, et al. Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci U S A. 2018;115(13):E2970–9.CrossRefPubMedPubMedCentral
15.
go back to reference Bhargava HK, Leo P, Elliott R, Janowczyk A, Whitney J, Gupta S, et al. Computationally derived image signature of stromal morphology is prognostic of prostate cancer recurrence following prostatectomy in African American patients. Clin Cancer Res. 2020;26(8):1915–23.CrossRefPubMedPubMedCentral Bhargava HK, Leo P, Elliott R, Janowczyk A, Whitney J, Gupta S, et al. Computationally derived image signature of stromal morphology is prognostic of prostate cancer recurrence following prostatectomy in African American patients. Clin Cancer Res. 2020;26(8):1915–23.CrossRefPubMedPubMedCentral
16.
go back to reference Zhu X, Yao J, Zhu F, Huang J. Wsisa: making survival prediction from whole slide histopathological images. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 7234–42. Zhu X, Yao J, Zhu F, Huang J. Wsisa: making survival prediction from whole slide histopathological images. In: Proceedings of the IEEE conference on computer vision and pattern recognition; 2017. p. 7234–42.
17.
go back to reference Tang B, Li A, Li B, Wang M. Capsurv: capsule network for survival analysis with whole slide pathological images. IEEE Access. 2019;7:26022–30.CrossRef Tang B, Li A, Li B, Wang M. Capsurv: capsule network for survival analysis with whole slide pathological images. IEEE Access. 2019;7:26022–30.CrossRef
18.
go back to reference Courtiol P, Maussion C, Moarii M, Pronier E, Pilcer S, Sefta M, et al. Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nat Med. 2019;25(10):1519–25.CrossRefPubMed Courtiol P, Maussion C, Moarii M, Pronier E, Pilcer S, Sefta M, et al. Deep learning-based classification of mesothelioma improves prediction of patient outcome. Nat Med. 2019;25(10):1519–25.CrossRefPubMed
19.
go back to reference Yamashita R, Long J, Saleem A, Rubin DL, Shen J. Deep learning predicts postsurgical recurrence of hepatocellular carcinoma from digital histopathologic images. Sci Rep. 2021;11(1):1–14.CrossRef Yamashita R, Long J, Saleem A, Rubin DL, Shen J. Deep learning predicts postsurgical recurrence of hepatocellular carcinoma from digital histopathologic images. Sci Rep. 2021;11(1):1–14.CrossRef
21.
go back to reference MacQueen J. Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability; vol. 1. Oakland, CA, USA; 1967. p. 281–97. MacQueen J. Some methods for classification and analysis of multivariate observations. In: Proceedings of the fifth Berkeley symposium on mathematical statistics and probability; vol. 1. Oakland, CA, USA; 1967. p. 281–97.
22.
go back to reference Li R, Yao J, Zhu X, Li Y, Huang J. Graph cnn for survival analysis on whole slide pathological images. In: Proceedings of the international conference on medical image computing and computer-assisted intervention. Springer; 2018. p. 174–82. Li R, Yao J, Zhu X, Li Y, Huang J. Graph cnn for survival analysis on whole slide pathological images. In: Proceedings of the international conference on medical image computing and computer-assisted intervention. Springer; 2018. p. 174–82.
23.
go back to reference Kather JN, Krisam J, Charoentong P, Luedde T, Herpel E, Weis CA, et al. Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. 2019;16(1):1–22.CrossRef Kather JN, Krisam J, Charoentong P, Luedde T, Herpel E, Weis CA, et al. Predicting survival from colorectal cancer histology slides using deep learning: a retrospective multicenter study. PLoS Med. 2019;16(1):1–22.CrossRef
24.
go back to reference Macenko M, Niethammer M, Marron JS, Borland D, Woosley JT, Guan X, et al. A method for normalizing histology slides for quantitative analysis. In: Proceedings of the 2009 IEEE international symposium on biomedical imaging: from nano to macro; 2009. p. 1107–10. Macenko M, Niethammer M, Marron JS, Borland D, Woosley JT, Guan X, et al. A method for normalizing histology slides for quantitative analysis. In: Proceedings of the 2009 IEEE international symposium on biomedical imaging: from nano to macro; 2009. p. 1107–10.
25.
go back to reference He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR); 2016. p. 770–8. He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: Proceedings of the 2016 IEEE conference on computer vision and pattern recognition (CVPR); 2016. p. 770–8.
26.
go back to reference Saha S, Shaik M, Johnston G, Saha SK, Berbiglia L, Hicks M, et al. Tumor size predicts long-term survival in colon cancer: an analysis of the national cancer data base. Am J Surg. 2015;209(3):570–4.CrossRefPubMed Saha S, Shaik M, Johnston G, Saha SK, Berbiglia L, Hicks M, et al. Tumor size predicts long-term survival in colon cancer: an analysis of the national cancer data base. Am J Surg. 2015;209(3):570–4.CrossRefPubMed
27.
go back to reference Haralick RM, Sternberg SR, Zhuang X. Image analysis using mathematical morphology. IEEE Trans Pattern Anal Mach Intell. 1987;9(4):532–50.CrossRefPubMed Haralick RM, Sternberg SR, Zhuang X. Image analysis using mathematical morphology. IEEE Trans Pattern Anal Mach Intell. 1987;9(4):532–50.CrossRefPubMed
28.
go back to reference Rosenfeld A, Pfaltz JL. Sequential operations in digital picture processing. J ACM (JACM). 1966;13(4):471–94.CrossRef Rosenfeld A, Pfaltz JL. Sequential operations in digital picture processing. J ACM (JACM). 1966;13(4):471–94.CrossRef
29.
go back to reference Tibshirani R. The lasso method for variable selection in the cox model. Stat Med. 1997;16(4):385–95.CrossRefPubMed Tibshirani R. The lasso method for variable selection in the cox model. Stat Med. 1997;16(4):385–95.CrossRefPubMed
30.
go back to reference Perperoglou A. Cox models with dynamic ridge penalties on time-varying effects of the covariates. Stat Med. 2014;33(1):170–80.CrossRefPubMed Perperoglou A. Cox models with dynamic ridge penalties on time-varying effects of the covariates. Stat Med. 2014;33(1):170–80.CrossRefPubMed
31.
go back to reference Yang Y, Zou H. A cocktail algorithm for solving the elastic net penalized cox’s regression in high dimensions. Stat Interface. 2013;6(2):167–73.CrossRef Yang Y, Zou H. A cocktail algorithm for solving the elastic net penalized cox’s regression in high dimensions. Stat Interface. 2013;6(2):167–73.CrossRef
32.
go back to reference Pölsterl S, Navab N, Katouzian A. An efficient training algorithm for kernel survival support vector machines; 2016. arXiv preprint. arXiv:1611.07054. Pölsterl S, Navab N, Katouzian A. An efficient training algorithm for kernel survival support vector machines; 2016. arXiv preprint. arXiv:​1611.​07054.
33.
go back to reference Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS. Random survival forests. T Ann Appl Stat. 2008;2(3):841–60. Ishwaran H, Kogalur UB, Blackstone EH, Lauer MS. Random survival forests. T Ann Appl Stat. 2008;2(3):841–60.
34.
go back to reference Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat. 2001;29:1189–232.CrossRef Friedman JH. Greedy function approximation: a gradient boosting machine. Ann Stat. 2001;29:1189–232.CrossRef
35.
go back to reference Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. Imagenet: A large-scale hierarchical image database. In: Proceedings of the 2009 IEEE conference on computer vision and pattern recognition; 2009. p. 248–55. Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L. Imagenet: A large-scale hierarchical image database. In: Proceedings of the 2009 IEEE conference on computer vision and pattern recognition; 2009. p. 248–55.
36.
go back to reference He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision; 2015. p. 1026–34. He K, Zhang X, Ren S, Sun J. Delving deep into rectifiers: Surpassing human-level performance on imagenet classification. In: Proceedings of the IEEE international conference on computer vision; 2015. p. 1026–34.
37.
go back to reference Hothorn T, Lausen B. On the exact distribution of maximally selected rank statistics. Comput Stat Data Anal. 2003;43(2):121–37.CrossRef Hothorn T, Lausen B. On the exact distribution of maximally selected rank statistics. Comput Stat Data Anal. 2003;43(2):121–37.CrossRef
38.
go back to reference Picon AI, Moore HG, Sternberg SS, Minsky BD, Paty PB, Blumberg D, Quan SH, Wong WD, Cohen AM, Guillem JG. Prognostic significance of depth of gross or microscopic perirectal fat invasion in T3 N0 M0 rectal cancers following sharp mesorectal excision and no adjuvant therapy. Int J Colorectal Dis. 2003;18(6):487–92.CrossRefPubMed Picon AI, Moore HG, Sternberg SS, Minsky BD, Paty PB, Blumberg D, Quan SH, Wong WD, Cohen AM, Guillem JG. Prognostic significance of depth of gross or microscopic perirectal fat invasion in T3 N0 M0 rectal cancers following sharp mesorectal excision and no adjuvant therapy. Int J Colorectal Dis. 2003;18(6):487–92.CrossRefPubMed
39.
go back to reference Di Franco S, Stassi G. Adipose stromal cells promote the transition of colorectal cancer cells toward a mesenchymal-like phenotype. Mol Cell Oncol. 2021;8(5):1986343.CrossRefPubMedPubMedCentral Di Franco S, Stassi G. Adipose stromal cells promote the transition of colorectal cancer cells toward a mesenchymal-like phenotype. Mol Cell Oncol. 2021;8(5):1986343.CrossRefPubMedPubMedCentral
40.
go back to reference Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, et al. On the opportunities and risks of foundation models; 2021. arXiv:2108.07258. Bommasani R, Hudson DA, Adeli E, Altman R, Arora S, von Arx S, et al. On the opportunities and risks of foundation models; 2021. arXiv:​2108.​07258.
41.
go back to reference Longato E, Vettoretti M, Di Camillo B. A practical perspective on the concordance index for the evaluation and selection of prognostic time-to-event models. J Biomed Inform. 2020;108: 103496.CrossRefPubMed Longato E, Vettoretti M, Di Camillo B. A practical perspective on the concordance index for the evaluation and selection of prognostic time-to-event models. J Biomed Inform. 2020;108: 103496.CrossRefPubMed
42.
go back to reference Boschman J, Farahani H, Darbandsari A, et al. The utility of color normalization for AI-based diagnosis of hematoxylin and eosin-stained pathology images. J Pathol. 2022;256(1):15–24.CrossRefPubMed Boschman J, Farahani H, Darbandsari A, et al. The utility of color normalization for AI-based diagnosis of hematoxylin and eosin-stained pathology images. J Pathol. 2022;256(1):15–24.CrossRefPubMed
43.
go back to reference Michielli N, Caputo A, Scotto M, et al. Stain normalization in digital pathology: clinical multi-center evaluation of image quality. J Pathol Inform. 2022;13: 100145.CrossRefPubMedPubMedCentral Michielli N, Caputo A, Scotto M, et al. Stain normalization in digital pathology: clinical multi-center evaluation of image quality. J Pathol Inform. 2022;13: 100145.CrossRefPubMedPubMedCentral
Metadata
Title
A novel deep learning-based algorithm combining histopathological features with tissue areas to predict colorectal cancer survival from whole-slide images
Authors
Yan-Jun Li
Hsin-Hung Chou
Peng-Chan Lin
Meng-Ru Shen
Sun-Yuan Hsieh
Publication date
01-12-2023
Publisher
BioMed Central
Published in
Journal of Translational Medicine / Issue 1/2023
Electronic ISSN: 1479-5876
DOI
https://doi.org/10.1186/s12967-023-04530-8

Other articles of this Issue 1/2023

Journal of Translational Medicine 1/2023 Go to the issue
Live Webinar | 27-06-2024 | 18:00 (CEST)

Keynote webinar | Spotlight on medication adherence

Live: Thursday 27th June 2024, 18:00-19:30 (CEST)

WHO estimates that half of all patients worldwide are non-adherent to their prescribed medication. The consequences of poor adherence can be catastrophic, on both the individual and population level.

Join our expert panel to discover why you need to understand the drivers of non-adherence in your patients, and how you can optimize medication adherence in your clinics to drastically improve patient outcomes.

Prof. Kevin Dolgin
Prof. Florian Limbourg
Prof. Anoop Chauhan
Developed by: Springer Medicine
Obesity Clinical Trial Summary

At a glance: The STEP trials

A round-up of the STEP phase 3 clinical trials evaluating semaglutide for weight loss in people with overweight or obesity.

Developed by: Springer Medicine

Highlights from the ACC 2024 Congress

Year in Review: Pediatric cardiology

Watch Dr. Anne Marie Valente present the last year's highlights in pediatric and congenital heart disease in the official ACC.24 Year in Review session.

Year in Review: Pulmonary vascular disease

The last year's highlights in pulmonary vascular disease are presented by Dr. Jane Leopold in this official video from ACC.24.

Year in Review: Valvular heart disease

Watch Prof. William Zoghbi present the last year's highlights in valvular heart disease from the official ACC.24 Year in Review session.

Year in Review: Heart failure and cardiomyopathies

Watch this official video from ACC.24. Dr. Biykem Bozkurt discusses last year's major advances in heart failure and cardiomyopathies.