Top

Published in:

Open Access 01-08-2019 | Breast Cancer | Preclinical study

Breast cancer outcome prediction with tumour tissue images and machine learning

Authors: Riku Turkki, Dmitrii Byckhov, Mikael Lundin, Jorma Isola, Stig Nordling, Panu E. Kovanen, Clare Verrill, Karl von Smitten, Heikki Joensuu, Johan Lundin, Nina Linder

Published in: Breast Cancer Research and Treatment | Issue 1/2019

Abstract

Purpose

Recent advances in machine learning have enabled better understanding of large and complex visual data. Here, we aim to investigate patient outcome prediction with a machine learning method using only an image of tumour sample as an input.

Methods

Utilising tissue microarray (TMA) samples obtained from the primary tumour of patients (N = 1299) within a nationwide breast cancer series with long-term-follow-up, we train and validate a machine learning method for patient outcome prediction. The prediction is performed by classifying samples into low or high digital risk score (DRS) groups. The outcome classifier is trained using sample images of 868 patients and evaluated and compared with human expert classification in a test set of 431 patients.

Results

In univariate survival analysis, the DRS classification resulted in a hazard ratio of 2.10 (95% CI 1.33–3.32, p = 0.001) for breast cancer-specific survival. The DRS classification remained as an independent predictor of breast cancer-specific survival in a multivariate Cox model with a hazard ratio of 2.04 (95% CI 1.20–3.44, p = 0.007). The accuracy (C-index) of the DRS grouping was 0.60 (95% CI 0.55–0.65), as compared to 0.58 (95% CI 0.53–0.63) for human expert predictions based on the same TMA samples.

Conclusions

Our findings demonstrate the feasibility of learning prognostic signals in tumour tissue images without domain knowledge. Although further validation is needed, our study suggests that machine learning algorithms can extract prognostically relevant information from tumour histology complementing the currently used prognostic factors in breast cancer.

Available only for authorised users

http://www.finprog.org.

http://www.image-net.org/.

Obermeyer Z, Emanuel EJ (2016) Predicting the future—big data, machine learning, and clinical medicine. N Engl J Med 375:1216–1219. https://doi.org/10.1056/NEJMp1606181 CrossRefPubMedPubMedCentral

Schmidhuber J (2015) Deep learning in neural networks: an overview. Neural Netw 61:85–117. https://doi.org/10.1016/j.neunet.2014.09.003 CrossRefPubMed

LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521:436–444. https://doi.org/10.1038/nature14539 CrossRefPubMed

Shen D, Wu G, Suk H-I (2017) Deep learning in medical image analysis. Annu Rev Biomed Eng 19:221–248. https://doi.org/10.1146/annurev-bioeng-071516-044442 CrossRefPubMedPubMedCentral

Litjens G, Kooi T, Bejnordi BE et al (2017) A survey on deep learning in medical image analysis. Med Image Anal 42:60–88. https://doi.org/10.1016/j.media.2017.07.005 CrossRefPubMed

Gulshan V, Peng L, Coram M et al (2016) Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA 316:2402. https://doi.org/10.1001/jama.2016.17216 CrossRefPubMed

Esteva A, Kuprel B, Novoa RA et al (2017) Dermatologist-level classification of skin cancer with deep neural networks. Nature 542:115–118. https://doi.org/10.1038/nature21056 CrossRefPubMedPubMedCentral

Madabhushi A, Lee G (2016) Image analysis and machine learning in digital pathology: challenges and opportunities. Med Image Anal 33:170–175. https://doi.org/10.1016/j.media.2016.06.037 CrossRefPubMedPubMedCentral

Cireşan DC, Giusti A, Gambardella LM, Schmidhuber J (2013) Mitosis detection in breast cancer histology images with deep neural networks. Med Image Comput Comput Assist Interv 16:411–418PubMed

10.

Paeng K, Hwang S, Park S, Kim M (2017) A unified framework for tumor proliferation score prediction in breast histopathology. Springer, Cham, pp 231–239

11.

Veta M, van Diest PJ, Willems SM et al (2015) Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal 20:237–248. https://doi.org/10.1016/j.media.2014.11.010 CrossRefPubMed

12.

Turkki R, Linder N, Kovanen PE et al (2016) Antibody-supervised deep learning for quantification of tumor-infiltrating immune cells in hematoxylin and eosin stained breast cancer samples. J Pathol Inform. https://doi.org/10.4103/2153-3539.189703 CrossRefPubMedPubMedCentral

13.

Basavanhally AN, Ganesan S, Agner S et al (2010) Computerized image-based detection and grading of lymphocytic infiltration in HER2 + breast cancer histopathology. IEEE Trans Biomed Eng 57:642–653. https://doi.org/10.1109/TBME.2009.2035305 CrossRefPubMed

14.

Litjens G, Sánchez CI, Timofeeva N et al (2016) Deep learning as a tool for increased accuracy and efficiency of histopathological diagnosis. Sci Rep 6:26286. https://doi.org/10.1038/srep26286 CrossRefPubMedPubMedCentral

15.

Xu J, Luo X, Wang G et al (2016) A deep convolutional neural network for segmenting and classifying epithelial and stromal regions in histopathological images. Neurocomputing 191:214–223. https://doi.org/10.1016/j.neucom.2016.01.034 CrossRefPubMedPubMedCentral

16.

Chen H, Qi X, Yu L et al (2017) DCAN: deep contour-aware networks for object instance segmentation from histology images. Med Image Anal 36:135–146. https://doi.org/10.1016/j.media.2016.11.004 CrossRefPubMed

17.

Turkki R, Linder N, Holopainen T et al (2015) Assessment of tumour viability in human lung cancer xenografts with texture-based image analysis. J Clin Pathol 68:jclinpath-2015. https://doi.org/10.1136/jclinpath-2015-202888 CrossRef

18.

Roxanis I, Colling R, Kartsonaki C et al (2018) The significance of tumour microarchitectural features in breast cancer prognosis: a digital image analysis. Breast Cancer Res 20:11. https://doi.org/10.1186/s13058-018-0934-x CrossRefPubMedPubMedCentral

19.

Robertson S, Azizpour H, Smith K, Hartman J (2017) Digital image analysis in breast pathology-from image processing techniques to artificial intelligence. Transl Res 45:78. https://doi.org/10.1016/j.trsl.2017.10.010 CrossRef

20.

Fukushima K (1980) Neocognitron: a self-organizing neural network model for a mechanism of pattern recognition unaffected by shift in position. Biol Cybern 36:193–202. https://doi.org/10.1007/BF00344251 CrossRefPubMed

21.

Bejnordi BE, Veta M, Johannes van Diest P et al (2017) Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318:2199. https://doi.org/10.1001/jama.2017.14585 CrossRef

22.

Joensuu H, Isola J, Lundin M et al (2003) Amplification of erbB2 and erbB2 expression are superior to estrogen receptor status as risk factors for distant recurrence in pT1N0M0 breast cancer: a nationwide population-based study. Clin Cancer Res 9:923–930PubMed

23.

Kononen J, Bubendorf L, Kallioniemi A et al (1998) Tissue microarrays for high-throughput molecular profiling of tumor specimens. Nat Med 4:844–847CrossRefPubMed

24.

Tavassoéli F, Devilee P (eds) (2003) Pathology and genetics of tumours of the breast and female genital organs. WHO, Geneva

25.

Simonyan K, Zisserman A (2015) Very deep convolutional networks for large-scale image recognition. In: International conference on learning representations

26.

Perronnin F, Sánchez J, Mensink T (2010) Improving the Fisher Kernel for large-scale image classification. Springer, Berlin, pp 143–156

27.

Cimpoi M, Maji S, Kokkinos I, Vedaldi A (2016) Deep filter banks for texture recognition, description, and segmentation. Int J Comput Vis. https://doi.org/10.1007/s11263-015-0872-3 CrossRefPubMedPubMedCentral

28.

Chen Yu, Chen Dian-ren, Li Yang, Chen Lei (2010) Otsu’s thresholding method based on gray level-gradient two-dimensional histogram. In: Proceedings of the IEEE 2010 2nd international asia conference on informatics in control, automation and robotics (CAR 2010). pp 282–285

29.

Fan R-E, Chang K-W, Hsieh C-J et al (2008) LIBLINEAR: a library for large linear classification. J Mach Learn Res 9:1871–1874

30.

Vedaldi A, Fulkerson B (2010) VLFeat: an open and portable library of computer vision algorithms. In: Proceedings of the international conference on multimedia. ACM, pp 1469–1472

31.

Vedaldi A, Lenc K (2015) MatConvNet—convolutional neural networks for MATLAB. In: Proceedings of the 23rd ACM international conference on multimedia, Brisbane, Australia, October 26–30, 2015

32.

Kaplan EL, Meier P (1958) Nonparametric estimation from incomplete observations. J Am Stat Assoc 53:457. https://doi.org/10.2307/2281868 CrossRef

33.

Cox DR (1972) Regression models and life-tables. J R Stat Soc Ser B 34:187–220. https://doi.org/10.2307/2985181 CrossRef

34.

Gönen M, Heller G (2005) Concordance probability and discriminatory power in proportional hazards regression. Biometrika 92:965–970. https://doi.org/10.1093/biomet/92.4.965 CrossRef

35.

Beck AH, Sangoi AR, Leung S et al (2011) Systematic analysis of breast cancer morphology uncovers stromal features associated with survival. Sci Transl Med 3:108ra113. https://doi.org/10.1126/scitranslmed.3002564 CrossRefPubMed

36.

Moor AE, Guevara C, Altermatt HJ et al (2011) PRO_10–a new tissue-based prognostic multigene marker in patients with early estrogen receptor-positive breast cancer. Pathobiology 78:140–148. https://doi.org/10.1159/000323809 CrossRefPubMed

37.

Popovici V, Budinská E, Čápková L et al (2016) Joint analysis of histopathology image features and gene expression in breast cancer. BMC Bioinf 17:209. https://doi.org/10.1186/s12859-016-1072-z CrossRef

38.

Chen J-M, Qu A-P, Wang L-W et al (2015) New breast cancer prognostic factors identified by computer-aided image analysis of HE stained histopathology images. Sci Rep 5:10690. https://doi.org/10.1038/srep10690 CrossRefPubMedPubMedCentral

39.

Bychkov D, Linder N, Turkki R et al (2018) Deep learning based tissue analysis predicts outcome in colorectal cancer. Sci Rep 8:3395. https://doi.org/10.1038/s41598-018-21758-3 CrossRefPubMedPubMedCentral

40.

Mobadersany P, Yousefi S, Amgad M et al (2018) Predicting cancer outcomes from histology and genomics using convolutional networks. Proc Natl Acad Sci USA 115:E2970–E2979. https://doi.org/10.1073/pnas.1717139115 CrossRefPubMedPubMedCentral

41.

Sun C, Shrivastava A, Singh S, Gupta A (2017) Revisiting unreasonable effectiveness of data in deep learning era. In: 2017 IEEE International conference on computer vision (ICCV), Venice, Italy, 22–29 October 2017, pp 843–852

42.

Joulin A, van der Maaten L, Jabri A, Vasilache N (2016) Learning visual features from large weakly supervised data. In: Leibe B, Matas J, Sebe N, Welling M (eds) Computer vision – ECCV 2016. ECCV 2016. Lecture notes in computer Science, vol 9911. Springer, Cham

43.

Szegedy C, Wei Liu, Yangqing Jia, et al (2015) Going deeper with convolutions. In: Proceedings of the 2015 IEEE conference on computer vision and pattern recognition (CVPR). pp 1–9

44.

Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. Commun ACM 60:84–90CrossRef

45.

He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. In: 2016 IEEE conference on computer vision and pattern recognition. pp 770–778

46.

Razavian AS, Azizpour H, Sullivan J, Carlsson S (2014) CNN features off-the-shelf: an astounding baseline for recognition. In: Proceedings 2014 IEEE conference on computer vision and pattern recognition workshops. pp 512–519

47.

Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks? pp 3320–3328

48.

Socher R (2009) ImageNet: a large-scale hierarchical image database. In: Proceedings 2009 IEEE conference on computer vision and pattern recognition. pp 248–255

49.

Voosen P (2017) The AI detectives. Science 357:22–27. https://doi.org/10.1126/science.357.6346.22 CrossRefPubMed

50.

Montavon G, Lapuschkin S, Binder A et al (2017) Explaining nonlinear classification decisions with deep Taylor decomposition. Pattern Recognit 65:211–222. https://doi.org/10.1016/j.patcog.2016.11.008 CrossRef

Title: Breast cancer outcome prediction with tumour tissue images and machine learning
Authors: Riku Turkki
Dmitrii Byckhov
Mikael Lundin
Jorma Isola
Stig Nordling
Panu E. Kovanen
Clare Verrill
Karl von Smitten
Heikki Joensuu
Johan Lundin
Nina Linder
Publication date: 01-08-2019
Publisher: Springer US
Keywords: Breast Cancer
Breast Cancer
Published in: Breast Cancer Research and Treatment / Issue 1/2019
Print ISSN: 0167-6806
Electronic ISSN: 1573-7217
DOI: https://doi.org/10.1007/s10549-019-05281-1

Webinar | 19-02-2024 | 17:30 (CET)

Keynote webinar | Spotlight on antibody–drug conjugates in cancer

Watch now

Antibody–drug conjugates (ADCs) are novel agents that have shown promise across multiple tumor types. Explore the current landscape of ADCs in breast and lung cancer with our experts, and gain insights into the mechanism of action, key clinical trials data, existing challenges, and future directions.

Dr. Véronique Diéras

Prof. Fabrice Barlesi

Developed by: Springer Medicine

At a glance: The STEP trials

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 1/2019

What is the best clinical pathological score to identify high-risk patients with lobular carcinoma of the breast who are likely to benefit from adjuvant chemotherapy?

Potent inhibition of breast cancer by bis-indole-derived nuclear receptor 4A1 (NR4A1) antagonists

The use of contralateral prophylactic mastectomy among elderly patients in the United States

Phase III evaluating the addition of fulvestrant (F) to anastrozole (A) as adjuvant therapy in postmenopausal women with hormone receptor-positive HER2-negative (HR+/HER2−) early breast cancer (EBC): results from the GEICAM/2006–10 study

Assessment of PARP4 as a candidate breast cancer susceptibility gene

Creating a pragmatic trials program for breast cancer patients: Rethinking Clinical Trials (REaCT)

Keynote webinar | Spotlight on antibody–drug conjugates in cancer