Skip to main content
Top

Open Access 29-01-2025 | Scintigraphy | Original Article

Generative artificial intelligence enables the generation of bone scintigraphy images and improves generalization of deep learning models in data-constrained environments

Authors: David Haberl, Jing Ning, Kilian Kluge, Katarina Kumpf, Josef Yu, Zewen Jiang, Claudia Constantino, Alice Monaci, Maria Starace, Alexander R. Haug, Raffaella Calabretta, Luca Camoni, Francesco Bertagna, Katharina Mascherbauer, Felix Hofer, Domenico Albano, Roberto Sciagra, Francisco Oliveira, Durval Costa, Christian Nitsche, Marcus Hacker, Clemens P. Spielvogel

Published in: European Journal of Nuclear Medicine and Molecular Imaging

Login to get access

Abstract

Purpose

Advancements of deep learning in medical imaging are often constrained by the limited availability of large, annotated datasets, resulting in underperforming models when deployed under real-world conditions. This study investigated a generative artificial intelligence (AI) approach to create synthetic medical images taking the example of bone scintigraphy scans, to increase the data diversity of small-scale datasets for more effective model training and improved generalization.

Methods

We trained a generative model on 99mTc-bone scintigraphy scans from 9,170 patients in one center to generate high-quality and fully anonymized annotated scans of patients representing two distinct disease patterns: abnormal uptake indicative of (i) bone metastases and (ii) cardiac uptake indicative of cardiac amyloidosis. A blinded reader study was performed to assess the clinical validity and quality of the generated data. We investigated the added value of the generated data by augmenting an independent small single-center dataset with synthetic data and by training a deep learning model to detect abnormal uptake in a downstream classification task. We tested this model on 7,472 scans from 6,448 patients across four external sites in a cross-tracer and cross-scanner setting and associated the resulting model predictions with clinical outcomes.

Results

The clinical value and high quality of the synthetic imaging data were confirmed by four readers, who were unable to distinguish synthetic scans from real scans (average accuracy: 0.48% [95% CI 0.46–0.51]), disagreeing in 239 (60%) of 400 cases (Fleiss’ kappa: 0.18). Adding synthetic data to the training set improved model performance by a mean (± SD) of 33(± 10)% AUC (p < 0.0001) for detecting abnormal uptake indicative of bone metastases and by 5(± 4)% AUC (p < 0.0001) for detecting uptake indicative of cardiac amyloidosis across both internal and external testing cohorts, compared to models without synthetic training data. Patients with predicted abnormal uptake had adverse clinical outcomes (log-rank: p < 0.0001).

Conclusions

Generative AI enables the targeted generation of bone scintigraphy images representing different clinical conditions. Our findings point to the potential of synthetic data to overcome challenges in data sharing and in developing reliable and prognostic deep learning models in data-limited environments.

Graphical abstract

Appendix
Available only for authorised users
Literature
1.
go back to reference Häggström I, Leithner D, Alvén J, et al. Deep learning for [18F]fluorodeoxyglucose-PET-CT classification in patients with lymphoma: a dual-centre retrospective analysis. Lancet Digit Health. 2024;6:e114–25.CrossRefPubMed Häggström I, Leithner D, Alvén J, et al. Deep learning for [18F]fluorodeoxyglucose-PET-CT classification in patients with lymphoma: a dual-centre retrospective analysis. Lancet Digit Health. 2024;6:e114–25.CrossRefPubMed
2.
go back to reference Spielvogel CP, Haberl D, Mascherbauer K, et al. Diagnosis and prognosis of abnormal cardiac scintigraphy uptake suggestive of cardiac amyloidosis using artificial intelligence: a retrospective, international, multicentre, cross-tracer development and validation study. Lancet Digit Health. 2024;6:e251–60.CrossRefPubMed Spielvogel CP, Haberl D, Mascherbauer K, et al. Diagnosis and prognosis of abnormal cardiac scintigraphy uptake suggestive of cardiac amyloidosis using artificial intelligence: a retrospective, international, multicentre, cross-tracer development and validation study. Lancet Digit Health. 2024;6:e251–60.CrossRefPubMed
3.
go back to reference Wasserthal J, Breit H-C, Meyer MT, et al. TotalSegmentator: robust segmentation of 104 anatomic structures in CT images. Radiol Artif Intell. 2023;5:e230024.CrossRefPubMedPubMedCentral Wasserthal J, Breit H-C, Meyer MT, et al. TotalSegmentator: robust segmentation of 104 anatomic structures in CT images. Radiol Artif Intell. 2023;5:e230024.CrossRefPubMedPubMedCentral
4.
go back to reference Girum KB, Rebaud L, Cottereau A-S, et al. 18F-FDG PET maximum-intensity projections and Artificial Intelligence: a Win-Win Combination to easily measure prognostic biomarkers in DLBCL patients. J Nucl Med. 2022;63:1925–32.CrossRefPubMedPubMedCentral Girum KB, Rebaud L, Cottereau A-S, et al. 18F-FDG PET maximum-intensity projections and Artificial Intelligence: a Win-Win Combination to easily measure prognostic biomarkers in DLBCL patients. J Nucl Med. 2022;63:1925–32.CrossRefPubMedPubMedCentral
5.
go back to reference Miller RJH, Killekar A, Shanbhag A, et al. Predicting mortality from AI cardiac volumes mass and coronary calcium on chest computed tomography. Nat Commun. 2024;15:2747.CrossRefPubMedPubMedCentral Miller RJH, Killekar A, Shanbhag A, et al. Predicting mortality from AI cardiac volumes mass and coronary calcium on chest computed tomography. Nat Commun. 2024;15:2747.CrossRefPubMedPubMedCentral
6.
go back to reference Gutsche R, Lowis C, Ziemons K, et al. Automated brain tumor detection and segmentation for treatment response assessment using amino acid PET. J Nucl Med. 2023;64:1594–602.CrossRefPubMed Gutsche R, Lowis C, Ziemons K, et al. Automated brain tumor detection and segmentation for treatment response assessment using amino acid PET. J Nucl Med. 2023;64:1594–602.CrossRefPubMed
7.
go back to reference Kevin W, Eric W, Brandon T, et al. Characterizing the Clinical Adoption of Medical AI Devices through U.S. Insurance claims. NEJM AI. 2023;1:AIoa2300030. Kevin W, Eric W, Brandon T, et al. Characterizing the Clinical Adoption of Medical AI Devices through U.S. Insurance claims. NEJM AI. 2023;1:AIoa2300030.
8.
go back to reference Guo R, Xue S, Hu J, et al. Using domain knowledge for robust and generalizable deep learning-based CT-free PET attenuation and scatter correction. Nat Commun. 2022;13:5882.CrossRefPubMedPubMedCentral Guo R, Xue S, Hu J, et al. Using domain knowledge for robust and generalizable deep learning-based CT-free PET attenuation and scatter correction. Nat Commun. 2022;13:5882.CrossRefPubMedPubMedCentral
9.
go back to reference Koetzier LR, Wu J, Mastrodicasa D, et al. Generating synthetic data for medical imaging. Radiology. 2024;312:e232471.CrossRefPubMed Koetzier LR, Wu J, Mastrodicasa D, et al. Generating synthetic data for medical imaging. Radiology. 2024;312:e232471.CrossRefPubMed
10.
go back to reference DuMont Schütte A, Hetzel J, Gatidis S, et al. Overcoming barriers to data sharing with medical image generation: a comprehensive evaluation. NPJ Digit Med. 2021;4:141.CrossRefPubMedPubMedCentral DuMont Schütte A, Hetzel J, Gatidis S, et al. Overcoming barriers to data sharing with medical image generation: a comprehensive evaluation. NPJ Digit Med. 2021;4:141.CrossRefPubMedPubMedCentral
11.
go back to reference Han T, Nebelung S, Haarburger C et al. Breaking medical data sharing boundaries by using synthesized radiographs. Sci Adv. 2020;6. Han T, Nebelung S, Haarburger C et al. Breaking medical data sharing boundaries by using synthesized radiographs. Sci Adv. 2020;6.
12.
go back to reference Ktena I, Wiles O, Albuquerque I et al. April. Generative models improve fairness of medical classifiers under distribution shifts. Nat Med. 2024. Ktena I, Wiles O, Albuquerque I et al. April. Generative models improve fairness of medical classifiers under distribution shifts. Nat Med. 2024.
13.
14.
15.
go back to reference Parker C, Nilsson S, Heinrich D, et al. Alpha emitter radium-223 and survival in metastatic prostate cancer. N Engl J Med. 2013;369:213–23.CrossRefPubMed Parker C, Nilsson S, Heinrich D, et al. Alpha emitter radium-223 and survival in metastatic prostate cancer. N Engl J Med. 2013;369:213–23.CrossRefPubMed
16.
go back to reference Gillmore JD, Maurer MS, Falk RH, et al. Nonbiopsy diagnosis of Cardiac Transthyretin Amyloidosis. Circulation. 2016;133:2404–12.CrossRefPubMed Gillmore JD, Maurer MS, Falk RH, et al. Nonbiopsy diagnosis of Cardiac Transthyretin Amyloidosis. Circulation. 2016;133:2404–12.CrossRefPubMed
17.
go back to reference Maurer MS, Schwartz JH, Gundapaneni B, et al. Tafamidis Treatment for patients with transthyretin amyloid cardiomyopathy. N Engl J Med. 2018;379:1007–16.CrossRefPubMed Maurer MS, Schwartz JH, Gundapaneni B, et al. Tafamidis Treatment for patients with transthyretin amyloid cardiomyopathy. N Engl J Med. 2018;379:1007–16.CrossRefPubMed
18.
go back to reference Elliott P, Gundapaneni B, Sultan MB, Ines M, Garcia-Pavia P. Improved long-term survival with tafamidis treatment in patients with transthyretin amyloid cardiomyopathy and severe heart failure symptoms. Eur J Heart Fail. 2023;25:2060–4.CrossRefPubMed Elliott P, Gundapaneni B, Sultan MB, Ines M, Garcia-Pavia P. Improved long-term survival with tafamidis treatment in patients with transthyretin amyloid cardiomyopathy and severe heart failure symptoms. Eur J Heart Fail. 2023;25:2060–4.CrossRefPubMed
19.
go back to reference Huang Z, Pu X, Tang G, et al. BS-80K: the first large open-access dataset of bone scan images. Comput Biol Med. 2022;151:106221.CrossRefPubMed Huang Z, Pu X, Tang G, et al. BS-80K: the first large open-access dataset of bone scan images. Comput Biol Med. 2022;151:106221.CrossRefPubMed
20.
21.
go back to reference Dorbala S, Ando Y, Bokhari S, et al. ASNC/AHA/ASE/EANM/HFSA/ISA/SCMR/SNMMI expert consensus recommendations for multimodality imaging in cardiac amyloidosis: part 1 of 2-evidence base and standardized methods of imaging. J Nucl Cardiol. 2019;26:2065–123.CrossRefPubMed Dorbala S, Ando Y, Bokhari S, et al. ASNC/AHA/ASE/EANM/HFSA/ISA/SCMR/SNMMI expert consensus recommendations for multimodality imaging in cardiac amyloidosis: part 1 of 2-evidence base and standardized methods of imaging. J Nucl Cardiol. 2019;26:2065–123.CrossRefPubMed
22.
go back to reference Karras T, Aittala M, Hellsten J, Laine S, Lehtinen J, Aila T. June. Training Generative Adversarial Networks with Limited Data. arXiv [csCV]. 2020. Karras T, Aittala M, Hellsten J, Laine S, Lehtinen J, Aila T. June. Training Generative Adversarial Networks with Limited Data. arXiv [csCV]. 2020.
23.
go back to reference McInnes L, Healy J, Melville J. February. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv [statML]. 2018. McInnes L, Healy J, Melville J. February. UMAP: Uniform Manifold Approximation and Projection for Dimension Reduction. arXiv [statML]. 2018.
24.
go back to reference Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009:248–255. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L. ImageNet: A large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition. IEEE; 2009:248–255.
25.
go back to reference Nitsche C, Mascherbauer K, Calabretta R, et al. Prevalence and outcomes of Cardiac Amyloidosis in All-Comer referrals for bone scintigraphy. J Nucl Med. 2022;63:1906–11.CrossRefPubMed Nitsche C, Mascherbauer K, Calabretta R, et al. Prevalence and outcomes of Cardiac Amyloidosis in All-Comer referrals for bone scintigraphy. J Nucl Med. 2022;63:1906–11.CrossRefPubMed
26.
go back to reference Hajianfar G, Sabouri M, Manesh AS et al. Stable diffusion model-based scintigraphy image synthesis: data augmentation toward enhanced multiclass thyroid diagnosis. EUVIP September 2024:1–6. Hajianfar G, Sabouri M, Manesh AS et al. Stable diffusion model-based scintigraphy image synthesis: data augmentation toward enhanced multiclass thyroid diagnosis. EUVIP September 2024:1–6.
Metadata
Title
Generative artificial intelligence enables the generation of bone scintigraphy images and improves generalization of deep learning models in data-constrained environments
Authors
David Haberl
Jing Ning
Kilian Kluge
Katarina Kumpf
Josef Yu
Zewen Jiang
Claudia Constantino
Alice Monaci
Maria Starace
Alexander R. Haug
Raffaella Calabretta
Luca Camoni
Francesco Bertagna
Katharina Mascherbauer
Felix Hofer
Domenico Albano
Roberto Sciagra
Francisco Oliveira
Durval Costa
Christian Nitsche
Marcus Hacker
Clemens P. Spielvogel
Publication date
29-01-2025
Publisher
Springer Berlin Heidelberg
Published in
European Journal of Nuclear Medicine and Molecular Imaging
Print ISSN: 1619-7070
Electronic ISSN: 1619-7089
DOI
https://doi.org/10.1007/s00259-025-07091-8