Skip to main content
Top

03-02-2025 | Scientific Article

Validation of the American College of Radiology Bone Reporting and Data System™ (ACR Bone-RADS™) for classifying osteolytic bone tumors

Authors: Youngjune Kim, Choong Guen Chee, Yusuhn Kang

Published in: Skeletal Radiology

Login to get access

Abstract

Objective

To validate the performance of the American College of Radiology (ACR) Bone-RADS™ in identifying malignant osteolytic bone tumors and to evaluate the inter-observer agreement between musculoskeletal radiologists performing ACR Bone-RADS™ assessments.

Methods

This single-center, retrospective study included patients who visited our orthopedic oncology department on January 2019–December 2020 for bone tumors. Three radiologists evaluated the initial radiographs and assessed whether the tumor was eligible for ACR Bone-RADS™ assessment considering its transverse location and radiodensity. For eligible tumors, the radiologists rated the descriptors of ACR Bone-RADS™ and calculated the Bone-RADS™ categories. Using multi-reader, multi-case analysis, the performance in identifying malignant bone tumors was assessed in terms of sensitivity and area under the receiver-operating-characteristic curve (AUC), while dichotomizing the Bone-RADS™ categories into potentially benign (categories 1–2) and potentially malignant (categories 3–4). Gwet’s AC1 was calculated to evaluate inter-observer agreement on the categories. The proportion of malignancy in each Bone-RADS™ category was calculated.

Results

In total, 278 patients (mean age ± standard deviation, 40 ± 22 years; 161 men; and 93 with malignant tumors) were eligible for assessment. The sensitivity and AUC for discriminating malignancy using ACR Bone-RADS™ were 95.0% and 0.915, respectively. There was moderate inter-observer agreement on the ACR Bone-RADS™ category, with a Gwet’s AC1 of 0.594 (95% confidence interval, 0.536–0.652). The proportions of malignancy were 0.0–1.2% in Bone-RADS™ category 1; 4.0–9.0% in category 2; 18.2–30.3% in category 3; and 77.7–83.3% in category 4.

Conclusion

The ACR Bone-RADS™ showed high diagnostic performance in detecting malignant osteolytic bone tumors, with moderate inter-observer agreement.
Appendix
Available only for authorised users
Literature
2.
go back to reference Expert Panel on Musculoskeletal Imaging, Bestic JM, Wessell DE, Beaman FD, et al. ACR Appropriateness Criteria® primary bone tumors. J Am Coll Radiol. 2020;17(5S):S226–38. Expert Panel on Musculoskeletal Imaging, Bestic JM, Wessell DE, Beaman FD, et al. ACR Appropriateness Criteria® primary bone tumors. J Am Coll Radiol. 2020;17(5S):S226–38.
3.
go back to reference Lalam R, Bloem JL, Noebauer-Huhmann IM, et al. ESSR consensus document for detection, characterization, and referral pathway for tumors and tumorlike lesions of bone. Semin Musculoskelet Radiol. 2017;21(5):630–47.CrossRefPubMed Lalam R, Bloem JL, Noebauer-Huhmann IM, et al. ESSR consensus document for detection, characterization, and referral pathway for tumors and tumorlike lesions of bone. Semin Musculoskelet Radiol. 2017;21(5):630–47.CrossRefPubMed
4.
go back to reference Caracciolo JT, Temple HT, Letson GD, Kransdorf MJ. A modified Lodwick-Madewell grading system for the evaluation of lytic bone lesions. AJR Am J Roentgenol. 2016;207(1):150–6.CrossRefPubMed Caracciolo JT, Temple HT, Letson GD, Kransdorf MJ. A modified Lodwick-Madewell grading system for the evaluation of lytic bone lesions. AJR Am J Roentgenol. 2016;207(1):150–6.CrossRefPubMed
5.
go back to reference Lodwick GS, Wilson AJ, Farrell C, Virtama P, Smeltzer FM, Dittrich F. Estimating rate of growth in bone lesions: observer performance and error. Radiology. 1980;134(3):585–90.CrossRefPubMed Lodwick GS, Wilson AJ, Farrell C, Virtama P, Smeltzer FM, Dittrich F. Estimating rate of growth in bone lesions: observer performance and error. Radiology. 1980;134(3):585–90.CrossRefPubMed
6.
go back to reference Lodwick GS. Radiographic diagnosis and grading of bone tumors, with comments on computer evaluation. Proc Natl Cancer Conf. 1964;5:369–80.PubMed Lodwick GS. Radiographic diagnosis and grading of bone tumors, with comments on computer evaluation. Proc Natl Cancer Conf. 1964;5:369–80.PubMed
7.
go back to reference Madewell JE, Ragsdale BD, Sweet DE. Radiologic and pathologic analysis of solitary bone lesions. Part I: Internal margins. Radiol Clin North Am. 1981;19(4):715–48.CrossRefPubMed Madewell JE, Ragsdale BD, Sweet DE. Radiologic and pathologic analysis of solitary bone lesions. Part I: Internal margins. Radiol Clin North Am. 1981;19(4):715–48.CrossRefPubMed
8.
go back to reference An JY, Unsdorfer KM, Weinreb JC. BI-RADS, C-RADS, CAD-RADS, LI-RADS, Lung-RADS, NI-RADS, O-RADS, PI-RADS, TI-RADS: reporting and data systems. Radiographics. 2019;39(5):1435–6.CrossRefPubMed An JY, Unsdorfer KM, Weinreb JC. BI-RADS, C-RADS, CAD-RADS, LI-RADS, Lung-RADS, NI-RADS, O-RADS, PI-RADS, TI-RADS: reporting and data systems. Radiographics. 2019;39(5):1435–6.CrossRefPubMed
9.
go back to reference Gholamrezanezhad A, Kessler M, Hayeri SM. The need for standardization of musculoskeletal practice reporting: learning from ACR BI-RADS, liver imaging-reporting and data system, and prostate imaging-reporting and data system. J Am Coll Radiol. 2017;14(12):1585–7.CrossRefPubMed Gholamrezanezhad A, Kessler M, Hayeri SM. The need for standardization of musculoskeletal practice reporting: learning from ACR BI-RADS, liver imaging-reporting and data system, and prostate imaging-reporting and data system. J Am Coll Radiol. 2017;14(12):1585–7.CrossRefPubMed
10.
go back to reference Ribeiro GJ, Gillet R, Hossu G, et al. Solitary bone tumor imaging reporting and data system (BTI-RADS): initial assessment of a systematic imaging evaluation and comprehensive reporting method. Eur Radiol. 2021;31(10):7637–52.CrossRefPubMed Ribeiro GJ, Gillet R, Hossu G, et al. Solitary bone tumor imaging reporting and data system (BTI-RADS): initial assessment of a systematic imaging evaluation and comprehensive reporting method. Eur Radiol. 2021;31(10):7637–52.CrossRefPubMed
11.
go back to reference Chang CY, Garner HW, Ahlawat S, et al. Society of Skeletal Radiology- white paper. Guidelines for the diagnostic management of incidental solitary bone lesions on CT and MRI in adults: bone reporting and data system (Bone-RADS). Skeletal Radiol. 2022;51(9):1743–64.CrossRefPubMedPubMedCentral Chang CY, Garner HW, Ahlawat S, et al. Society of Skeletal Radiology- white paper. Guidelines for the diagnostic management of incidental solitary bone lesions on CT and MRI in adults: bone reporting and data system (Bone-RADS). Skeletal Radiol. 2022;51(9):1743–64.CrossRefPubMedPubMedCentral
12.
go back to reference Chhabra A, Gupta A, Thakur U, et al. Osseous Tumor reporting and data system-multireader validation study. J Comput Assist Tomogr. 2021;45(4):571–85.CrossRefPubMed Chhabra A, Gupta A, Thakur U, et al. Osseous Tumor reporting and data system-multireader validation study. J Comput Assist Tomogr. 2021;45(4):571–85.CrossRefPubMed
13.
go back to reference Caracciolo JT, Ali S, Chang CY, et al. Bone tumor risk stratification and management system: a consensus guideline from the ACR Bone Reporting and Data System Committee. J Am Coll Radiol. 2023;20(10):1044–58.CrossRefPubMed Caracciolo JT, Ali S, Chang CY, et al. Bone tumor risk stratification and management system: a consensus guideline from the ACR Bone Reporting and Data System Committee. J Am Coll Radiol. 2023;20(10):1044–58.CrossRefPubMed
14.
go back to reference Jelinek JS, Murphey MD, Welker JA, et al. Diagnosis of primary bone tumors with image-guided percutaneous biopsy: experience with 110 tumors. Radiology. 2002;223(3):731–7.CrossRefPubMed Jelinek JS, Murphey MD, Welker JA, et al. Diagnosis of primary bone tumors with image-guided percutaneous biopsy: experience with 110 tumors. Radiology. 2002;223(3):731–7.CrossRefPubMed
15.
go back to reference Ni Mhuircheartaigh J, McMahon C, Lin YC, Wu J. Diagnostic yield of percutaneous biopsy for sclerotic bone lesions: influence of mean Hounsfield units. Clin Imaging. 2017;46:53–6.CrossRefPubMed Ni Mhuircheartaigh J, McMahon C, Lin YC, Wu J. Diagnostic yield of percutaneous biopsy for sclerotic bone lesions: influence of mean Hounsfield units. Clin Imaging. 2017;46:53–6.CrossRefPubMed
16.
go back to reference Cohen MG, McMahon CJ, Kung JW, Wu JS. Comparison of battery-powered and manual bone biopsy systems for core needle biopsy of sclerotic bone lesions. AJR Am J Roentgenol. 2016;206(5):W83-86.CrossRefPubMed Cohen MG, McMahon CJ, Kung JW, Wu JS. Comparison of battery-powered and manual bone biopsy systems for core needle biopsy of sclerotic bone lesions. AJR Am J Roentgenol. 2016;206(5):W83-86.CrossRefPubMed
17.
go back to reference Li Y, Du Y, Luo TY, et al. Factors influencing diagnostic yield of CT-guided percutaneous core needle biopsy for bone lesions. Clin Radiol. 2014;69(1):e43-47.CrossRefPubMed Li Y, Du Y, Luo TY, et al. Factors influencing diagnostic yield of CT-guided percutaneous core needle biopsy for bone lesions. Clin Radiol. 2014;69(1):e43-47.CrossRefPubMed
18.
go back to reference Gonzalez-Huete A, Salgado-Parente A, Suevos-Ballesteros C, et al. Radiographic evaluation of bone tumors. Radiographics. 2023;43(11):e230048.CrossRefPubMed Gonzalez-Huete A, Salgado-Parente A, Suevos-Ballesteros C, et al. Radiographic evaluation of bone tumors. Radiographics. 2023;43(11):e230048.CrossRefPubMed
19.
go back to reference Ragsdale BD, Madewell JE, Sweet DE. Radiologic and pathologic analysis of solitary bone lesions Part II: Periosteal reactions. Radiol Clin North Am. 1981;19(4):749–83.CrossRefPubMed Ragsdale BD, Madewell JE, Sweet DE. Radiologic and pathologic analysis of solitary bone lesions Part II: Periosteal reactions. Radiol Clin North Am. 1981;19(4):749–83.CrossRefPubMed
20.
go back to reference Cronin CG, Cashell T, Mhuircheartaigh JN, et al. Bone biopsy of new suspicious bone lesions in patients with primary carcinoma: prevalence and probability of an alternative diagnosis. AJR Am J Roentgenol. 2009;193(5):W407–10.CrossRefPubMed Cronin CG, Cashell T, Mhuircheartaigh JN, et al. Bone biopsy of new suspicious bone lesions in patients with primary carcinoma: prevalence and probability of an alternative diagnosis. AJR Am J Roentgenol. 2009;193(5):W407–10.CrossRefPubMed
21.
go back to reference Zhang L, Wang Y, Gu Y, Hou Y, Chen Z. The need for bone biopsies in the diagnosis of new bone lesions in patients with a known primary malignancy: a comparative review of 117 biopsy cases. J Bone Oncol. 2019;14:100213.CrossRefPubMed Zhang L, Wang Y, Gu Y, Hou Y, Chen Z. The need for bone biopsies in the diagnosis of new bone lesions in patients with a known primary malignancy: a comparative review of 117 biopsy cases. J Bone Oncol. 2019;14:100213.CrossRefPubMed
22.
go back to reference Hillis SL, Obuchowski NA, Schartz KM, Berbaum KS. A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data. Stat Med. 2005;24(10):1579–607.CrossRefPubMed Hillis SL, Obuchowski NA, Schartz KM, Berbaum KS. A comparison of the Dorfman-Berbaum-Metz and Obuchowski-Rockette methods for receiver operating characteristic (ROC) data. Stat Med. 2005;24(10):1579–607.CrossRefPubMed
23.
go back to reference DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.CrossRefPubMed DeLong ER, DeLong DM, Clarke-Pearson DL. Comparing the areas under two or more correlated receiver operating characteristic curves: a nonparametric approach. Biometrics. 1988;44(3):837–45.CrossRefPubMed
24.
go back to reference Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008;61(Pt 1):29–48.CrossRefPubMed Gwet KL. Computing inter-rater reliability and its variance in the presence of high agreement. Br J Math Stat Psychol. 2008;61(Pt 1):29–48.CrossRefPubMed
25.
go back to reference Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.CrossRefPubMed Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(1):159–74.CrossRefPubMed
26.
go back to reference D’Orsi CJ, Sickles EA, Mendelson EB, Morris EA. ACR BI-RADS® atlas, breast imaging reporting and data system. Reston, VA: American College of Radiology. 2013. D’Orsi CJ, Sickles EA, Mendelson EB, Morris EA. ACR BI-RADS® atlas, breast imaging reporting and data system. Reston, VA: American College of Radiology. 2013.
27.
go back to reference Geirnaerdt MJ, Hermans J, Bloem JL, et al. Usefulness of radiography in differentiating enchondroma from central grade 1 chondrosarcoma. AJR Am J Roentgenol. 1997;169(4):1097–104.CrossRefPubMed Geirnaerdt MJ, Hermans J, Bloem JL, et al. Usefulness of radiography in differentiating enchondroma from central grade 1 chondrosarcoma. AJR Am J Roentgenol. 1997;169(4):1097–104.CrossRefPubMed
Metadata
Title
Validation of the American College of Radiology Bone Reporting and Data System™ (ACR Bone-RADS™) for classifying osteolytic bone tumors
Authors
Youngjune Kim
Choong Guen Chee
Yusuhn Kang
Publication date
03-02-2025
Publisher
Springer Berlin Heidelberg
Published in
Skeletal Radiology
Print ISSN: 0364-2348
Electronic ISSN: 1432-2161
DOI
https://doi.org/10.1007/s00256-025-04881-x