Skip to main content
Top
Published in: European Radiology 10/2022

Open Access 19-07-2022 | Bone Tumor | Imaging Informatics and Artificial Intelligence

Applications of machine learning for imaging-driven diagnosis of musculoskeletal malignancies—a scoping review

Authors: Florian Hinterwimmer, Sarah Consalvo, Jan Neumann, Daniel Rueckert, Rüdiger von Eisenhart-Rothe, Rainer Burgkart

Published in: European Radiology | Issue 10/2022

Login to get access

Abstract

Musculoskeletal malignancies are a rare type of cancer. Consequently, sufficient imaging data for machine learning (ML) applications is difficult to obtain. The main purpose of this review was to investigate whether ML is already having an impact on imaging-driven diagnosis of musculoskeletal malignancies and what the respective reasons for this might be. A scoping review was conducted by a radiologist, an orthopaedic surgeon and a data scientist to identify suitable articles based on the PRISMA statement. Studies meeting the following criteria were included: primary malignant musculoskeletal tumours, machine/deep learning application, imaging data or data retrieved from images, human/preclinical, English language and original research. Initially, 480 articles were found and 38 met the eligibility criteria. Several continuous and discrete parameters related to publication, patient distribution, tumour specificities, ML methods, data and metrics were extracted from the final articles. For the synthesis, diagnosis-oriented studies were further examined by retrieving the number of patients and labels and metric scores. No significant correlations between metrics and mean number of samples were found. Several studies presented that ML could support imaging-driven diagnosis of musculoskeletal malignancies in distinct cases. However, data quality and quantity must be increased to achieve clinically relevant results. Compared to the experience of an expert radiologist, the studies used small datasets and mostly included only one type of data. Key to critical advancement of ML models for rare diseases such as musculoskeletal malignancies is a systematic, structured data collection and the establishment of (inter)national networks to obtain substantial datasets in the future.

Key Points

• Machine learning does not yet significantly impact imaging-driven diagnosis for musculoskeletal malignancies compared to other disciplines such as lung, breast or CNS cancer.
• Research in the area of musculoskeletal tumour imaging and machine learning is still very limited.
• Machine learning in musculoskeletal tumour imaging is impeded by insufficient availability of data and rarity of the disease.
Literature
1.
go back to reference Grimer RJ, Briggs TW (2010) Earlier diagnosis of bone and soft-tissue tumours. J Bone Joint Surg Br 92:1489–1492PubMedCrossRef Grimer RJ, Briggs TW (2010) Earlier diagnosis of bone and soft-tissue tumours. J Bone Joint Surg Br 92:1489–1492PubMedCrossRef
2.
go back to reference Grimer RJ, Carter SR, Pynsent PB (1997) The cost-effectiveness of limb salvage for bone tumours. J Bone Joint Surg Br 79:558–561PubMedCrossRef Grimer RJ, Carter SR, Pynsent PB (1997) The cost-effectiveness of limb salvage for bone tumours. J Bone Joint Surg Br 79:558–561PubMedCrossRef
3.
go back to reference Rechl H, Kirchhoff C, Wortler K, Lenze U, Topfer A, von Eisenhart-Rothe R (2011) Diagnosis of malignant bone and soft tissue tumors. Orthopade 40:931–941 quiz 942-933PubMedCrossRef Rechl H, Kirchhoff C, Wortler K, Lenze U, Topfer A, von Eisenhart-Rothe R (2011) Diagnosis of malignant bone and soft tissue tumors. Orthopade 40:931–941 quiz 942-933PubMedCrossRef
4.
go back to reference Clark MA, Thomas JM (2005) Delay in referral to a specialist soft-tissue sarcoma unit. Eur J Surg Oncol 31:443–448PubMedCrossRef Clark MA, Thomas JM (2005) Delay in referral to a specialist soft-tissue sarcoma unit. Eur J Surg Oncol 31:443–448PubMedCrossRef
5.
go back to reference Ayala AG, Zornosa J (1983) Primary bone tumors: percutaneous needle biopsy. Radiologic-pathologic study of 222 biopsies. Radiology 149:675–679PubMedCrossRef Ayala AG, Zornosa J (1983) Primary bone tumors: percutaneous needle biopsy. Radiologic-pathologic study of 222 biopsies. Radiology 149:675–679PubMedCrossRef
6.
go back to reference Mankin HJ, Mankin CJ, Simon MA (1996) The hazards of the biopsy, revisited. Members of the Musculoskeletal Tumor Society. J Bone Joint Surg Am 78:656–663PubMedCrossRef Mankin HJ, Mankin CJ, Simon MA (1996) The hazards of the biopsy, revisited. Members of the Musculoskeletal Tumor Society. J Bone Joint Surg Am 78:656–663PubMedCrossRef
7.
go back to reference Savage N (2020) How AI is improving cancer diagnostics. Nature 579:S14+ Savage N (2020) How AI is improving cancer diagnostics. Nature 579:S14+
9.
go back to reference Zaman A, Park SH, Bang H, Park CW, Park I, Joung S (2020) Generative approach for data augmentation for deep learning-based bone surface segmentation from ultrasound images. Int J Comput Assist Radiol Surg 15:931–941PubMedCrossRef Zaman A, Park SH, Bang H, Park CW, Park I, Joung S (2020) Generative approach for data augmentation for deep learning-based bone surface segmentation from ultrasound images. Int J Comput Assist Radiol Surg 15:931–941PubMedCrossRef
10.
go back to reference Banerjee I, Crawley A, Bhethanabotla M, Daldrup-Link HE, Rubin DL (2018) Transfer learning on fused multiparametric MR images for classifying histopathological subtypes of rhabdomyosarcoma. Comput Med Imaging Graph 65:167–175PubMedCrossRef Banerjee I, Crawley A, Bhethanabotla M, Daldrup-Link HE, Rubin DL (2018) Transfer learning on fused multiparametric MR images for classifying histopathological subtypes of rhabdomyosarcoma. Comput Med Imaging Graph 65:167–175PubMedCrossRef
12.
go back to reference Tricco AC, Lillie E, Zarin W et al (2018) PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 169:467–473PubMedCrossRef Tricco AC, Lillie E, Zarin W et al (2018) PRISMA extension for scoping reviews (PRISMA-ScR): checklist and explanation. Ann Intern Med 169:467–473PubMedCrossRef
13.
go back to reference Bandyopadhyay O, Biswas A, Bhattacharya BB (2019) Bone-cancer assessment and destruction pattern analysis in long-bone X-ray image. J Digit Imaging 32:300–313PubMedCrossRef Bandyopadhyay O, Biswas A, Bhattacharya BB (2019) Bone-cancer assessment and destruction pattern analysis in long-bone X-ray image. J Digit Imaging 32:300–313PubMedCrossRef
14.
go back to reference Chianca V, Albano D, Messina C et al (2021) An update in musculoskeletal tumors: from quantitative imaging to radiomics. Radiol Med 126:1095–1105PubMedCrossRef Chianca V, Albano D, Messina C et al (2021) An update in musculoskeletal tumors: from quantitative imaging to radiomics. Radiol Med 126:1095–1105PubMedCrossRef
15.
go back to reference Chianca V, Cuocolo R, Gitto S et al (2021) Radiomic machine learning classifiers in spine bone tumors: a multi-software, multi-scanner study. Eur J Radiol 137:109586PubMedCrossRef Chianca V, Cuocolo R, Gitto S et al (2021) Radiomic machine learning classifiers in spine bone tumors: a multi-software, multi-scanner study. Eur J Radiol 137:109586PubMedCrossRef
16.
go back to reference Do NT, Jung ST, Yang HJ, Kim SH (2021) Multi-level seg-unet model with global and patch-based X-ray images for knee bone tumor detection. Diagnostics, 11(4):691 Do NT, Jung ST, Yang HJ, Kim SH (2021) Multi-level seg-unet model with global and patch-based X-ray images for knee bone tumor detection. Diagnostics, 11(4):691
17.
go back to reference Dufau J, Bouhamama A, Leporq B et al (2019) Prediction of chemotherapy response in primary osteosarcoma using the machine learning technique on radiomic data. Bull Cancer 106:983–999PubMedCrossRef Dufau J, Bouhamama A, Leporq B et al (2019) Prediction of chemotherapy response in primary osteosarcoma using the machine learning technique on radiomic data. Bull Cancer 106:983–999PubMedCrossRef
19.
go back to reference Fields BKK, Demirjian NL, Hwang DH et al (2021) Whole-tumor 3D volumetric MRI-based radiomics approach for distinguishing between benign and malignant soft tissue tumors. Eur Radiol 31:8522–8535PubMedCrossRef Fields BKK, Demirjian NL, Hwang DH et al (2021) Whole-tumor 3D volumetric MRI-based radiomics approach for distinguishing between benign and malignant soft tissue tumors. Eur Radiol 31:8522–8535PubMedCrossRef
20.
go back to reference Gao Y, Ghodrati V, Kalbasi A et al (2021) Prediction of soft tissue sarcoma response to radiotherapy using longitudinal diffusion MRI and a deep neural network with generative adversarial network-based data augmentation. Med Phys 48:3262–3372PubMedCrossRef Gao Y, Ghodrati V, Kalbasi A et al (2021) Prediction of soft tissue sarcoma response to radiotherapy using longitudinal diffusion MRI and a deep neural network with generative adversarial network-based data augmentation. Med Phys 48:3262–3372PubMedCrossRef
21.
go back to reference Gao Y, Kalbasi A, Hsu W et al (2020) Treatment effect prediction for sarcoma patients treated with preoperative radiotherapy using radiomics features from longitudinal diffusion-weighted MRIs. Phys Med Biol 65:175006PubMedCrossRef Gao Y, Kalbasi A, Hsu W et al (2020) Treatment effect prediction for sarcoma patients treated with preoperative radiotherapy using radiomics features from longitudinal diffusion-weighted MRIs. Phys Med Biol 65:175006PubMedCrossRef
22.
go back to reference García-Gómez JM, Vidal C, Martí-Bonmatí L et al (2004) Benign/malignant classifier of soft tissue tumors using MR imaging. MAGMA 16:194–201PubMedCrossRef García-Gómez JM, Vidal C, Martí-Bonmatí L et al (2004) Benign/malignant classifier of soft tissue tumors using MR imaging. MAGMA 16:194–201PubMedCrossRef
23.
go back to reference Gitto S, Cuocolo R, Albano D et al (2020) MRI radiomics-based machine-learning classification of bone chondrosarcoma. Eur J Radiol 128:109043PubMedCrossRef Gitto S, Cuocolo R, Albano D et al (2020) MRI radiomics-based machine-learning classification of bone chondrosarcoma. Eur J Radiol 128:109043PubMedCrossRef
24.
go back to reference Glass JO, Reddick WE (1998) Hybrid artificial neural network segmentation and classification of dynamic contrast-enhanced MR imaging (DEMRI) of osteosarcoma. Magn Reson Imaging 16:1075–1083PubMedCrossRef Glass JO, Reddick WE (1998) Hybrid artificial neural network segmentation and classification of dynamic contrast-enhanced MR imaging (DEMRI) of osteosarcoma. Magn Reson Imaging 16:1075–1083PubMedCrossRef
25.
go back to reference Gorelik N, Chong J, Lin DJ (2020) Pattern recognition in musculoskeletal imaging using artificial intelligence. Semin Musculoskelet Radiol 24:38–49PubMedCrossRef Gorelik N, Chong J, Lin DJ (2020) Pattern recognition in musculoskeletal imaging using artificial intelligence. Semin Musculoskelet Radiol 24:38–49PubMedCrossRef
26.
27.
28.
go back to reference Hu P, Chen L, Zhou Z (2021) Machine learning in the differentiation of soft tissue neoplasms: comparison of fat-suppressed T2WI and apparent diffusion coefficient (ADC) features-based models. J Digit Imaging 34:1146–1155PubMedPubMedCentralCrossRef Hu P, Chen L, Zhou Z (2021) Machine learning in the differentiation of soft tissue neoplasms: comparison of fat-suppressed T2WI and apparent diffusion coefficient (ADC) features-based models. J Digit Imaging 34:1146–1155PubMedPubMedCentralCrossRef
29.
go back to reference Hu S, Xu C, Guan W, Tang Y, Liu Y (2014) Texture feature extraction based on wavelet transform and gray-level co-occurrence matrices applied to osteosarcoma diagnosis. Biomed Mater Eng 24:129–143PubMed Hu S, Xu C, Guan W, Tang Y, Liu Y (2014) Texture feature extraction based on wavelet transform and gray-level co-occurrence matrices applied to osteosarcoma diagnosis. Biomed Mater Eng 24:129–143PubMed
30.
go back to reference Huang B, Wang J, Sun M et al (2020) Feasibility of multi-parametric magnetic resonance imaging combined with machine learning in the assessment of necrosis of osteosarcoma after neoadjuvant chemotherapy: a preliminary study. BMC Cancer 20:322PubMedPubMedCentralCrossRef Huang B, Wang J, Sun M et al (2020) Feasibility of multi-parametric magnetic resonance imaging combined with machine learning in the assessment of necrosis of osteosarcoma after neoadjuvant chemotherapy: a preliminary study. BMC Cancer 20:322PubMedPubMedCentralCrossRef
31.
go back to reference Huang L, Xia W, Zhang B, Qiu B, Gao X (2017) MSFCN-multiple supervised fully convolutional networks for the osteosarcoma segmentation of CT images. Comput Methods Programs Biomed 143:67–74PubMedCrossRef Huang L, Xia W, Zhang B, Qiu B, Gao X (2017) MSFCN-multiple supervised fully convolutional networks for the osteosarcoma segmentation of CT images. Comput Methods Programs Biomed 143:67–74PubMedCrossRef
32.
go back to reference Juntu J, Sijbers J, De Backer S, Rajan J, Van Dyck D (2010) Machine learning study of several classifiers trained with texture analysis features to differentiate benign from malignant soft-tissue tumors in T1-MRI images. J Magn Reson Imaging 31:680–689PubMedCrossRef Juntu J, Sijbers J, De Backer S, Rajan J, Van Dyck D (2010) Machine learning study of several classifiers trained with texture analysis features to differentiate benign from malignant soft-tissue tumors in T1-MRI images. J Magn Reson Imaging 31:680–689PubMedCrossRef
33.
go back to reference Leporq B, Bouhamama A, Pilleul F et al (2020) MRI-based radiomics to predict lipomatous soft tissue tumors malignancy: a pilot study. Cancer Imaging 20:78PubMedPubMedCentralCrossRef Leporq B, Bouhamama A, Pilleul F et al (2020) MRI-based radiomics to predict lipomatous soft tissue tumors malignancy: a pilot study. Cancer Imaging 20:78PubMedPubMedCentralCrossRef
34.
go back to reference Li L, Wang K, Ma X et al (2019) Radiomic analysis of multiparametric magnetic resonance imaging for differentiating skull base chordoma and chondrosarcoma. Eur J Radiol 118:81–87PubMedCrossRef Li L, Wang K, Ma X et al (2019) Radiomic analysis of multiparametric magnetic resonance imaging for differentiating skull base chordoma and chondrosarcoma. Eur J Radiol 118:81–87PubMedCrossRef
36.
go back to reference Pan D, Liu R, Zheng B et al (2021) Using machine learning to unravel the value of radiographic features for the classification of bone tumors. Biomed Res Int 2021:8811056PubMedPubMedCentralCrossRef Pan D, Liu R, Zheng B et al (2021) Using machine learning to unravel the value of radiographic features for the classification of bone tumors. Biomed Res Int 2021:8811056PubMedPubMedCentralCrossRef
37.
go back to reference Peeken JC, Bernhofer M, Spraker MB et al (2019) CT-based radiomic features predict tumor grading and have prognostic value in patients with soft tissue sarcomas treated with neoadjuvant radiation therapy. Radiother Oncol 135:187–196PubMedCrossRef Peeken JC, Bernhofer M, Spraker MB et al (2019) CT-based radiomic features predict tumor grading and have prognostic value in patients with soft tissue sarcomas treated with neoadjuvant radiation therapy. Radiother Oncol 135:187–196PubMedCrossRef
38.
go back to reference Peeken JC, Goldberg T, Knie C et al (2018) Treatment-related features improve machine learning prediction of prognosis in soft tissue sarcoma patients. Strahlenther Onkol 194:824–834PubMedCrossRef Peeken JC, Goldberg T, Knie C et al (2018) Treatment-related features improve machine learning prediction of prognosis in soft tissue sarcoma patients. Strahlenther Onkol 194:824–834PubMedCrossRef
39.
go back to reference Reinus WR, Wilson AJ, Kalman B, Kwasny S (1994) Diagnosis of focal bone lesions using neural networks. Invest Radiol 29:606–611PubMedCrossRef Reinus WR, Wilson AJ, Kalman B, Kwasny S (1994) Diagnosis of focal bone lesions using neural networks. Invest Radiol 29:606–611PubMedCrossRef
40.
go back to reference Shen R, Li Z, Zhang L et al (2018) Osteosarcoma patients classification using plain X-rays and metabolomic data. Annu Int Conf IEEE Eng Med Biol Soc 2018:690–693PubMed Shen R, Li Z, Zhang L et al (2018) Osteosarcoma patients classification using plain X-rays and metabolomic data. Annu Int Conf IEEE Eng Med Biol Soc 2018:690–693PubMed
41.
go back to reference Terunuma T, Tokui A, Sakae T (2018) Novel real-time tumor-contouring method using deep learning to prevent mistracking in X-ray fluoroscopy. Radiol Phys Technol 11:43–53PubMedCrossRef Terunuma T, Tokui A, Sakae T (2018) Novel real-time tumor-contouring method using deep learning to prevent mistracking in X-ray fluoroscopy. Radiol Phys Technol 11:43–53PubMedCrossRef
42.
go back to reference von Schacky CE, Wilhelm NJ, Schäfer VS et al (2021) Multitask deep learning for segmentation and classification of primary bone tumors on radiographs. Radiology 301:398–406CrossRef von Schacky CE, Wilhelm NJ, Schäfer VS et al (2021) Multitask deep learning for segmentation and classification of primary bone tumors on radiographs. Radiology 301:398–406CrossRef
43.
go back to reference Vos M, Starmans MPA, Timbergen MJM et al (2019) Radiomics approach to distinguish between well differentiated liposarcomas and lipomas on MRI. Br J Surg 106:1800–1809PubMedPubMedCentralCrossRef Vos M, Starmans MPA, Timbergen MJM et al (2019) Radiomics approach to distinguish between well differentiated liposarcomas and lipomas on MRI. Br J Surg 106:1800–1809PubMedPubMedCentralCrossRef
44.
go back to reference Wang B, Perronne L, Burke C, Adler RS (2021) Artificial intelligence for classification of soft-tissue masses at US. Radiol Artif Intell 3:e200125PubMedCrossRef Wang B, Perronne L, Burke C, Adler RS (2021) Artificial intelligence for classification of soft-tissue masses at US. Radiol Artif Intell 3:e200125PubMedCrossRef
45.
go back to reference Wang H, Zhang J, Bao S et al (2020) Preoperative MRI-based radiomic machine-learning nomogram may accurately distinguish between benign and malignant soft-tissue lesions: a two-center study. J Magn Reson Imaging 52:873–882PubMedCrossRef Wang H, Zhang J, Bao S et al (2020) Preoperative MRI-based radiomic machine-learning nomogram may accurately distinguish between benign and malignant soft-tissue lesions: a two-center study. J Magn Reson Imaging 52:873–882PubMedCrossRef
46.
go back to reference Yin P, Mao N, Zhao C, Wu J, Chen L, Hong N (2019) A triple-classification radiomics model for the differentiation of primary chordoma, giant cell tumor, and metastatic tumor of sacrum based on T2-weighted and contrast-enhanced T1-weighted MRI. J Magn Reson Imaging 49:752–759PubMedCrossRef Yin P, Mao N, Zhao C, Wu J, Chen L, Hong N (2019) A triple-classification radiomics model for the differentiation of primary chordoma, giant cell tumor, and metastatic tumor of sacrum based on T2-weighted and contrast-enhanced T1-weighted MRI. J Magn Reson Imaging 49:752–759PubMedCrossRef
47.
go back to reference Yin P, Mao N, Zhao C et al (2019) Comparison of radiomics machine-learning classifiers and feature selection for differentiation of sacral chordoma and sacral giant cell tumour based on 3D computed tomography features. Eur Radiol 29:1841–1847PubMedCrossRef Yin P, Mao N, Zhao C et al (2019) Comparison of radiomics machine-learning classifiers and feature selection for differentiation of sacral chordoma and sacral giant cell tumour based on 3D computed tomography features. Eur Radiol 29:1841–1847PubMedCrossRef
48.
go back to reference Yin P, Zhi X, Sun C et al (2021) Radiomics models for the preoperative prediction of pelvic and sacral tumor types: a single-center retrospective study of 795 cases. Front Oncol 11:709659PubMedPubMedCentralCrossRef Yin P, Zhi X, Sun C et al (2021) Radiomics models for the preoperative prediction of pelvic and sacral tumor types: a single-center retrospective study of 795 cases. Front Oncol 11:709659PubMedPubMedCentralCrossRef
49.
go back to reference Zhang L, Ren Z (2020) Comparison of CT and MRI images for the prediction of soft-tissue sarcoma grading and lung metastasis via a convolutional neural networks model. Clin Radiol 75:64–69PubMedCrossRef Zhang L, Ren Z (2020) Comparison of CT and MRI images for the prediction of soft-tissue sarcoma grading and lung metastasis via a convolutional neural networks model. Clin Radiol 75:64–69PubMedCrossRef
50.
go back to reference Zhang R, Huang L, Xia W, Zhang B, Qiu B, Gao X (2018) Multiple supervised residual network for osteosarcoma segmentation in CT images. Comput Med Imaging Graph 63:1–8PubMedCrossRef Zhang R, Huang L, Xia W, Zhang B, Qiu B, Gao X (2018) Multiple supervised residual network for osteosarcoma segmentation in CT images. Comput Med Imaging Graph 63:1–8PubMedCrossRef
51.
go back to reference Zhang Y, Zhu Y, Shi X et al (2019) Soft tissue sarcomas: preoperative predictive histopathological grading based on radiomics of MRI. Acad Radiol 26:1262–1268PubMedCrossRef Zhang Y, Zhu Y, Shi X et al (2019) Soft tissue sarcomas: preoperative predictive histopathological grading based on radiomics of MRI. Acad Radiol 26:1262–1268PubMedCrossRef
52.
53.
go back to reference Picci P, Manfrini M, Donati D et al (2020) Diagnosis of Musculoskeletal Tumors and Tumor-like Conditions: Clinical, Radiological and Histological Correlations-the Rizzoli Case Archive (pp. 3–11). Cham: Springer Picci P, Manfrini M, Donati D et al (2020) Diagnosis of Musculoskeletal Tumors and Tumor-like Conditions: Clinical, Radiological and Histological Correlations-the Rizzoli Case Archive (pp. 3–11). Cham: Springer
Metadata
Title
Applications of machine learning for imaging-driven diagnosis of musculoskeletal malignancies—a scoping review
Authors
Florian Hinterwimmer
Sarah Consalvo
Jan Neumann
Daniel Rueckert
Rüdiger von Eisenhart-Rothe
Rainer Burgkart
Publication date
19-07-2022
Publisher
Springer Berlin Heidelberg
Keyword
Bone Tumor
Published in
European Radiology / Issue 10/2022
Print ISSN: 0938-7994
Electronic ISSN: 1432-1084
DOI
https://doi.org/10.1007/s00330-022-08981-3

Other articles of this Issue 10/2022

European Radiology 10/2022 Go to the issue