Skip to main content
Top
Published in: Diagnostic Pathology 1/2019

Open Access 01-12-2019 | Research

Validation of mitotic cell quantification via microscopy and multiple whole-slide scanners

Authors: Kazuhiro Tabata, Naohiro Uraoka, Jamal Benhamida, Matthew G. Hanna, Sahussapont Joseph Sirintrapun, Brandon D. Gallas, Qi Gong, Rania G. Aly, Katsura Emoto, Kant M. Matsuda, Meera R. Hameed, David S. Klimstra, Yukako Yagi

Published in: Diagnostic Pathology | Issue 1/2019

Login to get access

Abstract

Background

The establishment of whole-slide imaging (WSI) as a medical diagnostic device allows that pathologists may evaluate mitotic activity with this new technology. Furthermore, the image digitalization provides an opportunity to develop algorithms for automatic quantifications, ideally leading to improved reproducibility as compared to the naked eye examination by pathologists. In order to implement them effectively, accuracy of mitotic figure detection using WSI should be investigated. In this study, we aimed to measure pathologist performance in detecting mitotic figures (MFs) using multiple platforms (multiple scanners) and compare the results with those obtained using a brightfield microscope.

Methods

Four slides of canine oral melanoma were prepared and digitized using 4 WSI scanners. In these slides, 40 regions of interest (ROIs) were demarcated, and five observers identified the MFs using different viewing modes: microscopy and WSI. We evaluated the inter- and intra-observer agreements between modes with Cohen’s Kappa and determined “true” MFs with a consensus panel. We then assessed the accuracy (agreement with truth) using the average of sensitivity and specificity.

Results

In the 40 ROIs, 155 candidate MFs were detected by five pathologists; 74 of them were determined to be true MFs. Inter- and intra-observer agreement was mostly “substantial” or greater (Kappa = 0.594–0.939). Accuracy was between 0.632 and 0.843 across all readers and modes. After averaging over readers for each modality, we found that mitosis detection accuracy for 3 of the 4 WSI scanners was significantly less than that of the microscope (p = 0.002, 0.012, and 0.001).

Conclusions

This study is the first to compare WSIs and microscopy in detecting MFs at the level of individual cells. Our results suggest that WSI can be used for mitotic cell detection and offers similar reproducibility to the microscope, with slightly less accuracy.
Literature
1.
go back to reference Pyo JS, Kim NY. Meta-analysis of prognostic role of Ki-67 labeling index in gastric carcinoma. Int J Biol Markers. 2017;32(Suppl 4):e447–53.CrossRef Pyo JS, Kim NY. Meta-analysis of prognostic role of Ki-67 labeling index in gastric carcinoma. Int J Biol Markers. 2017;32(Suppl 4):e447–53.CrossRef
2.
go back to reference Goldhirsch A, Wood WC, Coates AS, Gelber RD, Thürlimann B, Senn HJ, et al. Strategies for subtypes-dealing with the diversity of breast cancer: highlights of the St. Gallen international expert consensus on the primary therapy of early breast Cancer 2011. Ann Oncol. 2011;22(Suppl 8):1736–47.CrossRef Goldhirsch A, Wood WC, Coates AS, Gelber RD, Thürlimann B, Senn HJ, et al. Strategies for subtypes-dealing with the diversity of breast cancer: highlights of the St. Gallen international expert consensus on the primary therapy of early breast Cancer 2011. Ann Oncol. 2011;22(Suppl 8):1736–47.CrossRef
3.
go back to reference Thotakura M, Tirumalasetti N, Krishna R. Role of Ki-67 labeling index as an adjunct to the histopathological diagnosis and grading of astrocytomas. J Cancer Res Ther. 2014;10(Suppl 3):641–5.PubMed Thotakura M, Tirumalasetti N, Krishna R. Role of Ki-67 labeling index as an adjunct to the histopathological diagnosis and grading of astrocytomas. J Cancer Res Ther. 2014;10(Suppl 3):641–5.PubMed
4.
go back to reference Tabata K, Tanaka T, Hayashi T, Hori T, Nunomura S, Yonezawa S, et al. Ki-67 is a strong prognostic marker of non-small cell lung cancer when tissue heterogeneity is considered. BMC Clin Pathol. 2014;14:23.CrossRef Tabata K, Tanaka T, Hayashi T, Hori T, Nunomura S, Yonezawa S, et al. Ki-67 is a strong prognostic marker of non-small cell lung cancer when tissue heterogeneity is considered. BMC Clin Pathol. 2014;14:23.CrossRef
5.
go back to reference Joensuu H, Vehtari A, Riihimaki J, Nishida T, Steigen SE, Brabec P, et al. Risk of recurrence of gastrointestinal stromal tumour after surgery: an analysis of pooled population-based cohorts. Lancet Oncol. 2012;13(Suppl 3):265–74.CrossRef Joensuu H, Vehtari A, Riihimaki J, Nishida T, Steigen SE, Brabec P, et al. Risk of recurrence of gastrointestinal stromal tumour after surgery: an analysis of pooled population-based cohorts. Lancet Oncol. 2012;13(Suppl 3):265–74.CrossRef
6.
go back to reference Klimstra DS, Modlin IR, Coppola D, Lloyd RV, Suster S. The pathologic classification of neuroendocrine tumors: a review of nomenclature, grading, and staging systems. Pancreas. 2010;39(Suppl 6):707–12.CrossRef Klimstra DS, Modlin IR, Coppola D, Lloyd RV, Suster S. The pathologic classification of neuroendocrine tumors: a review of nomenclature, grading, and staging systems. Pancreas. 2010;39(Suppl 6):707–12.CrossRef
7.
go back to reference Chang JM, McCullough AE, Dueck AC, Kosiorek HE, Ocal IT, Lidner TK, et al. Back to basics: traditional Nottingham grade mitotic counts alone are significant in predicting survival in invasive breast carcinoma. Ann Surg Oncol. 2015;22(Suppl 3):S509–15.CrossRef Chang JM, McCullough AE, Dueck AC, Kosiorek HE, Ocal IT, Lidner TK, et al. Back to basics: traditional Nottingham grade mitotic counts alone are significant in predicting survival in invasive breast carcinoma. Ann Surg Oncol. 2015;22(Suppl 3):S509–15.CrossRef
8.
go back to reference Meyer JS, Alvarez C, Milikowski C, Olson N, Russo I, Russo J, et al. Breast carcinoma malignancy grading by bloom-Richardson system vs proliferation index: reproducibility of grade and advantages of proliferation index. Mod Pathol. 2005;18(Suppl 8):1067–78.CrossRef Meyer JS, Alvarez C, Milikowski C, Olson N, Russo I, Russo J, et al. Breast carcinoma malignancy grading by bloom-Richardson system vs proliferation index: reproducibility of grade and advantages of proliferation index. Mod Pathol. 2005;18(Suppl 8):1067–78.CrossRef
9.
go back to reference van Diest PJ, Baak JP, Matze-Cok P, Wisse-Brekelmans EC, Van Galen CM, Kurver PH, et al. Reproducibility of mitosis counting in 2,469 breast cancer specimens: results from the multicenter morphometric mammary carcinoma project. Hum Pathol. 1992;23(Suppl 6):603–7.CrossRef van Diest PJ, Baak JP, Matze-Cok P, Wisse-Brekelmans EC, Van Galen CM, Kurver PH, et al. Reproducibility of mitosis counting in 2,469 breast cancer specimens: results from the multicenter morphometric mammary carcinoma project. Hum Pathol. 1992;23(Suppl 6):603–7.CrossRef
10.
go back to reference Tsuda H, Akiyama F, Kurosumi M, Sakamoto G, Yamashiro K, Oyama T, et al. Evaluation of the interobserver agreement in the number of mitotic figures of breast carcinoma as simulation of quality monitoring in the Japan National Surgical Adjuvant Study of breast Cancer (NSAS-BC) protocol. Jpn J Cancer Res. 2000;91(Suppl 4):451–7.CrossRef Tsuda H, Akiyama F, Kurosumi M, Sakamoto G, Yamashiro K, Oyama T, et al. Evaluation of the interobserver agreement in the number of mitotic figures of breast carcinoma as simulation of quality monitoring in the Japan National Surgical Adjuvant Study of breast Cancer (NSAS-BC) protocol. Jpn J Cancer Res. 2000;91(Suppl 4):451–7.CrossRef
11.
go back to reference Roberts DJ, Spellman RA, Sanok K, Chen H, Chan M, Yurt P, et al. Interlaboratory assessment of mitotic index by flow cytometry confirms superior reproducibility relative to microscopic scoring. Environ Mol Mutagen. 2012;53 Suppl 4:297–303.CrossRef Roberts DJ, Spellman RA, Sanok K, Chen H, Chan M, Yurt P, et al. Interlaboratory assessment of mitotic index by flow cytometry confirms superior reproducibility relative to microscopic scoring. Environ Mol Mutagen. 2012;53 Suppl 4:297–303.CrossRef
12.
go back to reference Montironi R, Collan Y, Scarpelli M, Sisti S, Barbatelli G, Carnevali A, et al. Reproducibility of mitotic counts and identification of mitotic figures in malignant glial tumors. Appl Pathol. 1988;6 Suppl 4:258–65.PubMed Montironi R, Collan Y, Scarpelli M, Sisti S, Barbatelli G, Carnevali A, et al. Reproducibility of mitotic counts and identification of mitotic figures in malignant glial tumors. Appl Pathol. 1988;6 Suppl 4:258–65.PubMed
13.
go back to reference Bauer TW, Schoenfield L, Slaw RJ, Yerian L, Sun Z, Henricks WH. Validation of whole slide imaging for primary diagnosis in surgical pathology. Arch Pathol Lab Med. 2013;137 Suppl 4:518–24.CrossRef Bauer TW, Schoenfield L, Slaw RJ, Yerian L, Sun Z, Henricks WH. Validation of whole slide imaging for primary diagnosis in surgical pathology. Arch Pathol Lab Med. 2013;137 Suppl 4:518–24.CrossRef
14.
go back to reference Bauer TW, Slaw RJ. Validating whole-slide imaging for consultation diagnoses in surgical pathology. Arch Pathol Lab Med. 2014;138(Suppl 11):1459–65.CrossRef Bauer TW, Slaw RJ. Validating whole-slide imaging for consultation diagnoses in surgical pathology. Arch Pathol Lab Med. 2014;138(Suppl 11):1459–65.CrossRef
15.
go back to reference Snead DR, Tsang YW, Meskiri A, Kimani PK, Crossman R, Rajpoot NM, et al. Validation of digital pathology imaging for primary histopathological diagnosis. Histopathology. 2016;68(Suppl 7):1063–72.CrossRef Snead DR, Tsang YW, Meskiri A, Kimani PK, Crossman R, Rajpoot NM, et al. Validation of digital pathology imaging for primary histopathological diagnosis. Histopathology. 2016;68(Suppl 7):1063–72.CrossRef
16.
go back to reference Tabata K, Mori I, Sasaki T, Itoh T, Shiraishi T, Yoshimi N, et al. Whole-slide imaging at primary pathological diagnosis: validation of whole-slide imaging-based primary pathological diagnosis at twelve Japanese academic institutes. Pathol Int. 2017;67(Suppl 11):547–54.CrossRef Tabata K, Mori I, Sasaki T, Itoh T, Shiraishi T, Yoshimi N, et al. Whole-slide imaging at primary pathological diagnosis: validation of whole-slide imaging-based primary pathological diagnosis at twelve Japanese academic institutes. Pathol Int. 2017;67(Suppl 11):547–54.CrossRef
17.
go back to reference Ho J, Ahlers SM, Stratman C, Aridor O, Pantanowitz L, Fine JL, et al. Can digital pathology result in cost savings? A financial projection for digital pathology implementation at a large integrated health care organization. J Pathol Inform. 2014;5(Suppl 1):33.CrossRef Ho J, Ahlers SM, Stratman C, Aridor O, Pantanowitz L, Fine JL, et al. Can digital pathology result in cost savings? A financial projection for digital pathology implementation at a large integrated health care organization. J Pathol Inform. 2014;5(Suppl 1):33.CrossRef
18.
go back to reference Veta M, van Diest PJ, Willems SM, Wang H, Madabhushi A, Cruz-Roa A, et al. Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal. 2015;20(Suppl 1):237–48.CrossRef Veta M, van Diest PJ, Willems SM, Wang H, Madabhushi A, Cruz-Roa A, et al. Assessment of algorithms for mitosis detection in breast cancer histopathology images. Med Image Anal. 2015;20(Suppl 1):237–48.CrossRef
19.
go back to reference Gandomkar Z, Brennan PC, Mello-Thoms C. Determining image processing features describing the appearance of challenging mitotic figures and miscounted nonmitotic objects. J Pathol Inform. 2017;8:34.CrossRef Gandomkar Z, Brennan PC, Mello-Thoms C. Determining image processing features describing the appearance of challenging mitotic figures and miscounted nonmitotic objects. J Pathol Inform. 2017;8:34.CrossRef
20.
go back to reference Veta M, van Diest PJ, Jiwa M, Al-Janabi S, Pluim JP. Mitosis counting in breast cancer: object-level interobserver agreement and comparison to an automatic method. PLoS One. 2016;11(Suppl 8):e0161286.CrossRef Veta M, van Diest PJ, Jiwa M, Al-Janabi S, Pluim JP. Mitosis counting in breast cancer: object-level interobserver agreement and comparison to an automatic method. PLoS One. 2016;11(Suppl 8):e0161286.CrossRef
21.
go back to reference Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J. Mitosis detection in breast cancer histology images with deep neural networks. Med Image Comput Comput Assist Interv. 2013;16(Pt 2):411–8.PubMed Ciresan DC, Giusti A, Gambardella LM, Schmidhuber J. Mitosis detection in breast cancer histology images with deep neural networks. Med Image Comput Comput Assist Interv. 2013;16(Pt 2):411–8.PubMed
22.
go back to reference Gong Q, Berman BP, Gavrielides MA, Gallas BD. Registration accuracy between whole slide images and glass slides in eeDAP workflow. In: Medical imaging 2018: digital pathology. Washington: International Society for Optics and Photonics; 2018. p. 1058118. Gong Q, Berman BP, Gavrielides MA, Gallas BD. Registration accuracy between whole slide images and glass slides in eeDAP workflow. In: Medical imaging 2018: digital pathology. Washington: International Society for Optics and Photonics; 2018. p. 1058118.
23.
go back to reference Gallas BD, Gavrielides MA, Conway CM, et al. Evaluation environment for digital and analog pathology: a platform for validation studies. J Med Imaging (Bellingham). 2014;1(Suppl 3):037501.CrossRef Gallas BD, Gavrielides MA, Conway CM, et al. Evaluation environment for digital and analog pathology: a platform for validation studies. J Med Imaging (Bellingham). 2014;1(Suppl 3):037501.CrossRef
24.
go back to reference Gallas BD, Cheng WC, Gavrielides MA, Ivansky A, Keay TC, Cheng WC, et al. eeDAP: an evaluation environment for digital and analog pathology. Proc SPIE Int Soc Opt Eng. 2014;(903709)1-12. Gallas BD, Cheng WC, Gavrielides MA, Ivansky A, Keay TC, Cheng WC, et al. eeDAP: an evaluation environment for digital and analog pathology. Proc SPIE Int Soc Opt Eng. 2014;(903709)1-12.
25.
go back to reference Wei BR, Halsey C, Hoover S, Puri M, Yang H, Gallas BD, et al. Agreement in histological assessment of mitotic activity between microscopy and digital whole slide images informs conversion to clinical diagnosis. Acad Pathol. 2019-accepted. Wei BR, Halsey C, Hoover S, Puri M, Yang H, Gallas BD, et al. Agreement in histological assessment of mitotic activity between microscopy and digital whole slide images informs conversion to clinical diagnosis. Acad Pathol. 2019-accepted.
26.
go back to reference Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(Suppl 1):159–74.CrossRef Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics. 1977;33(Suppl 1):159–74.CrossRef
27.
go back to reference Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135–60.CrossRef Bland JM, Altman DG. Measuring agreement in method comparison studies. Stat Methods Med Res. 1999;8(2):135–60.CrossRef
28.
go back to reference Gallas BD, Pennello GA, Myers KJ. Multireader multicase variance analysis for binary data. J Opt Soc Am A Opt Image Sci Vis. 2007;24(12):B70–80.CrossRef Gallas BD, Pennello GA, Myers KJ. Multireader multicase variance analysis for binary data. J Opt Soc Am A Opt Image Sci Vis. 2007;24(12):B70–80.CrossRef
29.
go back to reference Hilden J, Glasziou P. Regret graphs, diagnostic uncertainty and Youden's index. Stat Med. 1996;15(10):969–86.CrossRef Hilden J, Glasziou P. Regret graphs, diagnostic uncertainty and Youden's index. Stat Med. 1996;15(10):969–86.CrossRef
30.
go back to reference Gur D, Bandos AI, Rockette HE, Zuley ML, Hakim CM, Chough DM, et al. Is an ROC-type response truly always better than a binary response in observer performance studies? Acad Radiol. 2010;17(5):639–45.CrossRef Gur D, Bandos AI, Rockette HE, Zuley ML, Hakim CM, Chough DM, et al. Is an ROC-type response truly always better than a binary response in observer performance studies? Acad Radiol. 2010;17(5):639–45.CrossRef
31.
go back to reference Feuerman M, Miller AR. Relationships between statistical measures of agreement: sensitivity, specificity and kappa. J Eval Clin Pract. 2008;14(5):930–3.CrossRef Feuerman M, Miller AR. Relationships between statistical measures of agreement: sensitivity, specificity and kappa. J Eval Clin Pract. 2008;14(5):930–3.CrossRef
32.
go back to reference Obuchowski NA, Rockette HE. Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests: an ANOVA approach with dependent observations. Commun Stat B-Simul. 1995;24(2):285–308.CrossRef Obuchowski NA, Rockette HE. Hypothesis testing of diagnostic accuracy for multiple readers and multiple tests: an ANOVA approach with dependent observations. Commun Stat B-Simul. 1995;24(2):285–308.CrossRef
33.
go back to reference Hillis SL. A marginal-mean ANOVA approach for analyzing multireader multicase radiological imaging data. Stat Med. 2014;33(2):330–60.CrossRef Hillis SL. A marginal-mean ANOVA approach for analyzing multireader multicase radiological imaging data. Stat Med. 2014;33(2):330–60.CrossRef
34.
go back to reference Obuchowski NA. Nonparametric analysis of clustered ROC curve data. Biometrics. 1997;53(2):567–78.CrossRef Obuchowski NA. Nonparametric analysis of clustered ROC curve data. Biometrics. 1997;53(2):567–78.CrossRef
36.
go back to reference Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. 1979;6(2):65–70. Holm S. A simple sequentially rejective multiple test procedure. Scand J Stat. 1979;6(2):65–70.
37.
go back to reference Gallas BD. iMRMC v4.0: Application for analyzing and sizing MRMC reader studies. Silver Spring: Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA; 2017. Gallas BD. iMRMC v4.0: Application for analyzing and sizing MRMC reader studies. Silver Spring: Division of Imaging, Diagnostics, and Software Reliability, OSEL/CDRH/FDA; 2017.
38.
go back to reference Matthew GH, Jacqueline C, Sara EM, Juan X, Ishtiaque A, Liron P. Pathology informatics summit 2017. J Pathol Inform. 2017;8:26. Matthew GH, Jacqueline C, Sara EM, Juan X, Ishtiaque A, Liron P. Pathology informatics summit 2017. J Pathol Inform. 2017;8:26.
39.
go back to reference Shibata K, Inagaki M, Ajiro K. Mitosis-specific histone H3 phosphorylation in vitro in nucleosome structures. Eur J Biochem. 1990;192(Suppl 1):87–93.CrossRef Shibata K, Inagaki M, Ajiro K. Mitosis-specific histone H3 phosphorylation in vitro in nucleosome structures. Eur J Biochem. 1990;192(Suppl 1):87–93.CrossRef
40.
go back to reference Villani V, Mahadevan KK, Ligorio M, Fernández-Del Castillo C, Ting DT, Sabbatino F, et al. Phosphorylated histone H3 (PHH3) is a superior proliferation marker for prognosis of pancreatic neuroendocrine tumors. Ann Surg Oncol. 2016;23(Suppl 5):609–17.CrossRef Villani V, Mahadevan KK, Ligorio M, Fernández-Del Castillo C, Ting DT, Sabbatino F, et al. Phosphorylated histone H3 (PHH3) is a superior proliferation marker for prognosis of pancreatic neuroendocrine tumors. Ann Surg Oncol. 2016;23(Suppl 5):609–17.CrossRef
41.
go back to reference Duregon E, Cassenti A, Pittaro A, Ventura L, Senetta R, Rudà R, et al. Better see to better agree: phosphohistone H3 increases interobserver agreement in mitotic count for meningioma grading and imposes new specific thresholds. Neuro-Oncology. 2015;17(Suppl 5):663–9.CrossRef Duregon E, Cassenti A, Pittaro A, Ventura L, Senetta R, Rudà R, et al. Better see to better agree: phosphohistone H3 increases interobserver agreement in mitotic count for meningioma grading and imposes new specific thresholds. Neuro-Oncology. 2015;17(Suppl 5):663–9.CrossRef
42.
go back to reference Ottmann K, Tronnier M, Mitteldorf C. Detection of mitotic figures in thin melanomas-immunohistochemistry does not replace the careful search for mitotic figures in hematoxylin-eosin stain. J Am Acad Dermatol. 2015;73 Suppl 4:637–44.CrossRef Ottmann K, Tronnier M, Mitteldorf C. Detection of mitotic figures in thin melanomas-immunohistochemistry does not replace the careful search for mitotic figures in hematoxylin-eosin stain. J Am Acad Dermatol. 2015;73 Suppl 4:637–44.CrossRef
Metadata
Title
Validation of mitotic cell quantification via microscopy and multiple whole-slide scanners
Authors
Kazuhiro Tabata
Naohiro Uraoka
Jamal Benhamida
Matthew G. Hanna
Sahussapont Joseph Sirintrapun
Brandon D. Gallas
Qi Gong
Rania G. Aly
Katsura Emoto
Kant M. Matsuda
Meera R. Hameed
David S. Klimstra
Yukako Yagi
Publication date
01-12-2019
Publisher
BioMed Central
Published in
Diagnostic Pathology / Issue 1/2019
Electronic ISSN: 1746-1596
DOI
https://doi.org/10.1186/s13000-019-0839-8

Other articles of this Issue 1/2019

Diagnostic Pathology 1/2019 Go to the issue