Skip to main content

Advertisement

SpringerLink
Log in

Ultrasound characterization for thyroid nodules with indeterminate cytology: inter-observer agreement and impact of combining pattern-based and scoring-based classifications in risk stratification

  • Original Article
  • Published:
Endocrine Aims and scope Submit manuscript

Abstract

Background

The American Thyroid Association (ATA) sonographic patterns stratify the risk of malignancy of cytologically indeterminate thyroid nodules (ITNs). This study aimed to (1) assess inter-observer agreement for sonographic features and patterns; (2) identify potential sources of disagreement; and (3) evaluate whether the number of suspicious features risk-stratifies non-ATA and high-suspicion patterns.

Methods

Three observers independently reviewed the ultrasound images of 463 ITNs with histological follow-up consecutively evaluated between October 2008 and June 2015 at an academic cancer center. Each observer evaluated individual sonographic features. ATA sonographic patterns were derived from the interpretation of sonographic features. Nodules not fitting into any of the proposed patterns were clustered into a non-ATA pattern.

Results

The inter-observer agreement for ATA sonographic patterns and echogenicity was fair, moderate for margins, good for composition and echogenic foci, and very good for extrathyroidal extension and lymph node metastasis. The interpretation of each sonographic feature was significantly different between observers, and there was complete disagreement in at least one of the features in 104 (22%) nodules. A total of 169 nodules (37%) were classified into the non-ATA pattern. The number of suspicious features allowed risk stratifying nodules with non-ATA and high-suspicion sonographic patterns. Most Non-invasive Follicular Thyroid Neoplasms with Papillary-like Nuclear Features had 0–1 suspicious features and none had >2.

Conclusions

Echogenicity interpretation was the greatest source of disagreement. The number of suspicious features risk-stratifies ITNs with non-ATA or high-suspicion patterns. Future studies attempting to objectivize the interpretation of echogenicity and heterogeneity are needed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

References

  1. H.S. Ahn, H.J. Kim, H.G. Welch, Korea’s thyroid-cancer “epidemic”-screening and overdiagnosis. N. Engl. J. Med. 371(19), 1765–1767 (2014). https://doi.org/10.1056/NEJMp1409841

    Article  PubMed  Google Scholar 

  2. S. Nagar, B. Aschebrook-Kilfoy, E.L. Kaplan, P. Angelos, R.H. Grogan, Age of diagnosing physician impacts the incidence of thyroid cancer in a population. Cancer Causes Control 25(12), 1627–1634 (2014). https://doi.org/10.1007/s10552-014-0467-2

    Article  PubMed  Google Scholar 

  3. R. Udelsman, Y. Zhang, The epidemic of thyroid cancer in the United States: the role of endocrinologists and ultrasounds. Thyroid 24(3), 472–479 (2014). https://doi.org/10.1089/thy.2013.0257

    Article  PubMed  PubMed Central  Google Scholar 

  4. S. Vaccarella, S. Franceschi, F. Bray, C.P. Wild, M. Plummer, L. Dal Maso, Worldwide thyroid-cancer epidemic? The increasing impact of overdiagnosis. N. Engl. J. Med. 375(7), 614–617 (2016). https://doi.org/10.1056/NEJMp1604412

    Article  PubMed  Google Scholar 

  5. J.P. Zevallos, C.M. Hartman, J.R. Kramer, E.M. Sturgis, E.Y. Chiao, Increased thyroid cancer incidence corresponds to increased use of thyroid ultrasound and fine-needle aspiration: a study of the Veterans Affairs health care system. Cancer 121(5), 741–746 (2015). https://doi.org/10.1002/cncr.29122

    Article  PubMed  Google Scholar 

  6. J.A. Sosa, J.W. Hanna, K.A. Robinson, R.B. Lanman, Increases in thyroid nodule fine-needle aspirations, operations, and diagnoses of thyroid cancer in the United States. Surgery 154(6), 1420–1426 (2013). https://doi.org/10.1016/j.surg.2013.07.006. discussion 1426–1427

    Article  PubMed  Google Scholar 

  7. B.R. Haugen, E.K. Alexander, K.C. Bible, G.M. Doherty, S.J. Mandel, Y.E. Nikiforov, F. Pacini, G.W. Randolph, A.M. Sawka, M. Schlumberger, K.G. Schuff, S.I. Sherman, J.A. Sosa, D.L. Steward, R.M. Tuttle, L. Wartofsky, 2015 American Thyroid Association Management Guidelines for Adult Patients with Thyroid Nodules and Differentiated Thyroid Cancer: The American Thyroid Association Guidelines Task Force on Thyroid Nodules and Differentiated Thyroid Cancer. Thyroid 26(1), 1–133 (2016). https://doi.org/10.1089/thy.2015.0020

    Article  PubMed  PubMed Central  Google Scholar 

  8. P. Campanella, F. Ianni, C.A. Rota, S.M. Corsello, A. Pontecorvi, Quantification of cancer risk of each clinical and ultrasonographic suspicious feature of thyroid nodules: a systematic review and meta-analysis. Eur. J. Endocrinol. / Eur. Fed. Endocr. Soc. 170(5), R203–R211 (2014). https://doi.org/10.1530/eje-13-0995

    Article  CAS  Google Scholar 

  9. L.R. Remonti, C.K. Kramer, C.B. Leitao, L.C. Pinto, J.L. Gross, Thyroid ultrasound features and risk of carcinoma: a systematic review and meta-analysis of observational studies. Thyroid 25(5), 538–550 (2015). https://doi.org/10.1089/thy.2014.0353

    Article  PubMed  PubMed Central  Google Scholar 

  10. J.P. Brito, M.R. Gionfriddo, A. Al Nofal, K.R. Boehmer, A.L. Leppin, C. Reading, M. Callstrom, T.A. Elraiyah, L.J. Prokop, M.N. Stan, M.H. Murad, J.C. Morris, V.M. Montori, The accuracy of thyroid nodule ultrasound to predict thyroid cancer: systematic review and meta-analysis. J. Clin. Endocrinol. Metab. 99(4), 1253–1263 (2014). https://doi.org/10.1210/jc.2013-2928

    Article  CAS  PubMed  Google Scholar 

  11. W.J. Moon, S.L. Jung, J.H. Lee, D.G. Na, J.H. Baek, Y.H. Lee, J. Kim, H.S. Kim, J.S. Byun, D.H. Lee, Benign and malignant thyroid nodules: US differentiation-multicenter retrospective study. Radiology 247(3), 762–770 (2008). https://doi.org/10.1148/radiol.2473070944

    Article  PubMed  Google Scholar 

  12. C.S. Park, S.H. Kim, S.L. Jung, B.J. Kang, J.Y. Kim, J.J. Choi, M.S. Sung, H.W. Yim, S.H. Jeong, Observer variability in the sonographic evaluation of thyroid nodules. J. Clin. Ultrasound 38(6), 287–293 (2010). https://doi.org/10.1002/jcu.20689

    Article  PubMed  Google Scholar 

  13. P. Valderrabano, D.L. Klippenstein, J.B. Tourtelot, Z. Ma, Z.J. Thompson, H.S. Lilienfeld, B. McIver, New American Thyroid Association Sonographic Patterns for Thyroid Nodules Perform Well in Medullary Thyroid Carcinoma: Institutional Experience, Systematic Review, and Meta-Analysis. Thyroid 26(8), 1093–1100 (2016). https://doi.org/10.1089/thy.2016.0196

    Article  CAS  PubMed  Google Scholar 

  14. P. Valderrabano, M.J. McGettigan, C.A. Lam, L. Khazai, Z.J. Thompson, C.H. Chung, B.A. Centeno, B. McIver, Thyroid nodules with indeterminate cytology: utility of the American Thyroid Association Sonographic Patterns for Cancer Risk Stratification. Thyroid 28(8), 1004–1012 (2018). https://doi.org/10.1089/thy.2018.0085

    Article  PubMed  Google Scholar 

  15. E.S. Cibas, S.Z. Ali, The Bethesda system for reporting thyroid cytopathology. Thyroid 19(11), 1159–1165 (2009). https://doi.org/10.1089/thy.2009.0274

    Article  PubMed  Google Scholar 

  16. Randolph, J. Free-marginal multirater kappa: an alternative to Fleiss’ fixed-marginal multirater kappa. Joensuu University Learning and Instruction Symposium, Joensuu, Finland, 2005.

  17. M.J. Warrens, Inequalities between multi-rater kappas. Adv. Data Anal. Classif. 4(4), 271–286 (2010). https://doi.org/10.1007/s11634-010-0073-4

    Article  Google Scholar 

  18. S.H. Choi, E.K. Kim, J.Y. Kwak, M.J. Kim, E.J. Son, Interobserver and intraobserver variations in ultrasound assessment of thyroid nodules. Thyroid 20(2), 167–172 (2010). https://doi.org/10.1089/thy.2008.0354

    Article  PubMed  Google Scholar 

  19. S.H. Park, S.J. Kim, E.K. Kim, M.J. Kim, E.J. Son, J.Y. Kwak, Interobserver agreement in assessing the sonographic and elastographic features of malignant thyroid nodules. AJR Am. J. Roentgenol. 193(5), W416–W423 (2009). https://doi.org/10.2214/AJR.09.2541

    Article  PubMed  Google Scholar 

  20. G. Grani, L. Lamartina, V. Cantisani, M. Maranghi, P. Lucia, C. Durante, Interobserver agreement of various thyroid imaging reporting and data systems. Endocr. Connect 7(1), 1–7 (2018). https://doi.org/10.1530/EC-17-0336

    Article  PubMed  Google Scholar 

  21. W. Phuttharak, A. Boonrod, V. Klungboonkrong, T. Witsawapaisan, Interrater Reliability of Various Thyroid Imaging Reporting and Data System (TIRADS) Classifications for Differentiating Benign from Malignant Thyroid Nodules. Asian Pac. J. Cancer Prev. 20(4), 1283–1288 (2019). https://doi.org/10.31557/APJCP.2019.20.4.1283

    Article  PubMed  Google Scholar 

  22. G. Russ, S.J. Bonnema, M.F. Erdogan, C. Durante, R. Ngu, L. Leenhardt, European Thyroid Association Guidelines for Ultrasound Malignancy Risk Stratification of Thyroid Nodules in Adults: The EU-TIRADS. Eur. Thyroid J. 6(5), 225–237 (2017). https://doi.org/10.1159/000478927

    Article  Google Scholar 

  23. P. Trimboli, M. Deandrea, A. Mormile, L. Ceriani, F. Garino, P.P. Limone, L. Giovanella, American Thyroid Association ultrasound system for the initial assessment of thyroid nodules: use in stratifying the risk of malignancy of indeterminate lesions. Head. Neck 40(4), 722–727 (2018). https://doi.org/10.1002/hed.25038

    Article  PubMed  Google Scholar 

  24. T.G. Rocha, P.W. Rosario, A.L. Silva, M.B. Nunes, T.H. Silva, P.H.L. de Oliveira, M.R. Calsolari, Ultrasonography Classification of the American Thyroid Association for Predicting Malignancy in Thyroid Nodules >1cm with Indeterminate Cytology: A Prospective Study. Horm. Metab. Res 50(8), 597–601 (2018). https://doi.org/10.1055/a-0655-3016

    Article  CAS  PubMed  Google Scholar 

  25. G. Grani, M. D’Alessandri, G. Carbotta, A. Nesca, M. Del Sordo, S. Alessandrini, C. Coccaro, R. Rendina, M. Bianchini, N. Prinzi, A. Fumarola, Grey-scale analysis improves the ultrasonographic evaluation of thyroid nodules. Med. (Baltim.) 94(27), e1129 (2015). https://doi.org/10.1097/MD.0000000000001129

    Article  Google Scholar 

  26. L. Gao, X. Xi, J. Wang, X. Yang, Y. Wang, S. Zhu, X. Lai, X. Zhang, R. Zhao, B. Zhang, Ultrasound risk evaluation of thyroid nodules that are “unspecified” in the 2015 American Thyroid Association management guidelines: a retrospective study. Med. (Baltim.) 97(52), e13914 (2018). https://doi.org/10.1097/MD.0000000000013914

    Article  Google Scholar 

  27. J.E. Lim-Dunham, I. Erdem Toslak, K. Alsabban, A. Aziz, B. Martin, G. Okur, K.C. Longo, Ultrasound risk stratification for malignancy using the 2015 American Thyroid Association Management Guidelines for Children with Thyroid Nodules and Differentiated Thyroid Cancer. Pedia. Radio. 47(4), 429–436 (2017). https://doi.org/10.1007/s00247-017-3780-6

    Article  Google Scholar 

  28. G. Grani, L. Lamartina, V. Ascoli, D. Bosco, F. Nardi, F. D’Ambrosio, A. Rubini, L. Giacomelli, M. Biffoni, S. Filetti, C. Durante, V. Cantisani, Ultrasonography scoring systems can rule out malignancy in cytologically indeterminate thyroid nodules. Endocrine 57(2), 256–261 (2017). https://doi.org/10.1007/s12020-016-1148-6

    Article  CAS  PubMed  Google Scholar 

  29. G. Russ, S. Leboulleux, L. Leenhardt, L. Hegedüs, Thyroid incidentalomas: epidemiology, risk stratification with ultrasound and workup. Eur. Thyroid J. 3(3), 154–163 (2014). https://doi.org/10.1159/000365289

    Article  PubMed  PubMed Central  Google Scholar 

  30. M. Sollini, L. Cozzi, A. Chiti, M. Kirienko, Texture analysis and machine learning to characterize suspected thyroid nodules and differentiated thyroid cancer: Where do we stand? Eur. J. Radio. 99, 1–8 (2018). https://doi.org/10.1016/j.ejrad.2017.12.004

    Article  Google Scholar 

Download references

Author information

Author notes

  1. Pablo Valderrabano

    Present address: Department of Endocrinology and Nutrition, Hospital Universitario Ramón y Cajal, IRYCIS, Ctra. de Colmenar Viejo km. 9,100, 28034, Madrid, Spain

Authors and Affiliations

  1. Department of Diagnostic Imaging, H. Lee Moffitt Cancer Center and Research Institute, 12902 Magnolia Drive, Tampa, FL, 33612, USA

    Cesar A. Lam & Melissa J. McGettigan

  2. Department of Biostatistics and Bioinformatics, H. Lee Moffitt Cancer Center and Research Institute, 12902 Magnolia Drive, Tampa, FL, 33612, USA

    Zachary J. Thompson

  3. Department of Anatomic Pathology, H. Lee Moffitt Cancer Center and Research Institute, 12902 Magnolia Drive, Tampa, FL, 33612, USA

    Laila Khazai & Barbara A. Centeno

  4. Department of Head and Neck-Endocrine Oncology, H. Lee Moffitt Cancer Center and Research Institute, 12902 Magnolia Drive, Tampa, FL, 33612, USA

    Christine H. Chung, Bryan McIver & Pablo Valderrabano

Authors
  1. Cesar A. Lam
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Melissa J. McGettigan
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zachary J. Thompson
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Laila Khazai
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Christine H. Chung
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Barbara A. Centeno
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Bryan McIver
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Pablo Valderrabano
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Cesar A. Lam.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Ethical approval

All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

The study was carried with a waiver of informed consent from our Institutional Review Board.

Additional information

Publisher’s note: Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lam, C.A., McGettigan, M.J., Thompson, Z.J. et al. Ultrasound characterization for thyroid nodules with indeterminate cytology: inter-observer agreement and impact of combining pattern-based and scoring-based classifications in risk stratification. Endocrine 66, 278–287 (2019). https://doi.org/10.1007/s12020-019-02000-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12020-019-02000-0

Keywords

Search

Navigation