Top

BMC Medical Informatics and Decision Making

Published in:

Open Access 01-12-2023 | Cataract | Research

Real-time estimation of the remaining surgery duration for cataract surgery using deep convolutional neural networks and long short-term memory

Authors: Bowen Wang, Liangzhi Li, Yuta Nakashima, Ryo Kawasaki, Hajime Nagahara

Published in: BMC Medical Informatics and Decision Making | Issue 1/2023

Abstract

Purpose

Estimating the surgery length has the potential to be utilized as skill assessment, surgical training, or efficient surgical facility utilization especially if it is done in real-time as a remaining surgery duration (RSD). Surgical length reflects a certain level of efficiency and mastery of the surgeon in a well-standardized surgery such as cataract surgery. In this paper, we design and develop a real-time RSD estimation method for cataract surgery that does not require manual labeling and is transferable with minimum fine-tuning.

Methods

A regression method consisting of convolutional neural networks (CNNs) and long short-term memory (LSTM) is designed for RSD estimation. The model is firstly trained and evaluated for the single main surgeon with a large number of surgeries. Then, the fine-tuning strategy is used to transfer the model to the data of the other two surgeons. Mean Absolute Error (MAE in seconds) was used to evaluate the performance of the RSD estimation. The proposed method is compared with the naïve method which is based on the statistic of the historical data. A transferability experiment is also set to demonstrate the generalizability of the method.

Result

The mean surgical time for the sample videos was 318.7 s (s) (standard deviation 83.4 s) for the main surgeon for the initial training. In our experiments, the lowest MAE of 19.4 s (equal to about 6.4% of the mean surgical time) is achieved by our best-trained model for the independent test data of the main target surgeon. It reduces the MAE by 35.5 s (-10.2%) compared to the naïve method. The fine-tuning strategy transfers the model trained for the main target to the data of other surgeons with only a small number of training data (20% of the pre-training). The MAEs for the other two surgeons are 28.3 s and 30.6 s with the fine-tuning model, which decreased by -8.1 s and -7.5 s than the Per-surgeon model (average declining of -7.8 s and 1.3% of video duration). External validation study with Cataract-101 outperformed 3 reported methods of TimeLSTM, RSDNet, and CataNet.

Conclusion

An approach to build a pre-trained model for estimating RSD estimation based on a single surgeon and then transfer to other surgeons demonstrated both low prediction error and good transferability with minimum fine-tuning videos.

Available only for authorised users

Mahmud I, Kelley T, Stowell C, Haripriya A, Boman A, Kossler I, Morlet N, Pershing S, Pesudovs K, Goh PP, Sparrow JM, Lundström M. A proposed minimum standard set of outcome measures for cataract surgery. JAMA Ophthalmol. 2015;133(11):1247–52. https://doi.org/10.1001/jamaophthalmol.2015.2810. (PMID: 26291752).CrossRefPubMed

Lanza M, Koprowski R, Boccia R, Ruggiero A, De Rosa L, Tortori A, Wilczyński S, Melillo P, Sbordone S, Simonelli F. Classification tree to analyze factors connected with post operative complications of cataract surgery in a teaching hospital. J Clin Med. 2021;10(22):5399. https://doi.org/10.3390/jcm10225399.PMID:34830681;PMCID:PMC8625404.CrossRefPubMedPubMedCentral

Gutierrez L, Lim JS, Foo LL, Ng WYY, Yip M, Lim GYS, Wong MHY, Fong A, Rosman M, Mehta JS, Lin H, Ting DSJ, Ting DSW. Application of artificial intelligence in cataract management: current and future directions. Eye Vis (Lond). 2022;9(1):3. https://doi.org/10.1186/s40662-021-00273-z. (PMID: 34996524).CrossRefPubMed

Lanza M, Koprowski R, Boccia R, Krysik K, Sbordone S, Tartaglione A, Ruggiero A, Simonelli F. Application of artificial Intelligence in the analysis of features affecting cataract surgery complications in a teaching hospital. Front Med (Lausanne). 2020;7:607870 PMID: 33363188; PMCID: PMC7759659.CrossRefPubMed

Darzi A, Smith S, Taffinder N. Assessing operative skill. BMJ. 1999;318:887–8.CrossRefPubMedPubMedCentral

Hanna GB, Shimi SM, Cuschieri A. Randomised study of influence of two-dimensional versus three-dimensional imaging on performance of laparoscopic cholecystectomy. Lancet. 1998;351:248–51.CrossRefPubMed

Nderitu P, Ursell P. Factors affecting cataract surgery operating time among trainees and consultants. J Cataract Refract Surg. 2019;45(6):816–22. https://doi.org/10.1016/j.jcrs.2019.01.002. (Epub 2019 Mar 14 PMID: 30879720).CrossRefPubMed

Wai YZ, Fiona Chew LM, Mohamad AS, Ang CL, Chong YY, Adnan TH, Goh PP. The Malaysian cataract surgery registry: incidence and risk factors of postoperative infectious endophthalmitis over a 7-year period. Int J Ophthalmol. 2018;11(10):1685–90. https://doi.org/10.18240/ijo.2018.10.17. PMID: 30364221; PMCID: PMC6192968.CrossRefPubMedPubMedCentral

Gu won A, et al. Real-time estimation of surgical procedure duration. 2015 17th International Conference on E-health Networking. Boston: Application & Services (HealthCom); 2015. p. 6–10. https://doi.org/10.1109/HealthCom.2015.7454464.

10.

Childers CP, Gibbons MM. Understanding costs of care in the operating room. JAMA Surg. 2018;153(4):e176233.CrossRefPubMedPubMedCentral

11.

Kayış E, et al. A robust estimation model for surgery durations with temporal, operational, and surgery team effects. Health Care Manag Sci. 2015;18(3):222–33.PubMed

12.

Twinanda AP, et al. RSDNet: Learning to predict remaining surgery duration from laparoscopic videos without manual annotations. IEEE Trans Med Imaging. 2018;38(4):1069–78.

13.

Aksamentov I, Twinanda AP, Mutter D, Marescaux J, Padoy N. Deep neural networks predict remaining surgery duration from cholecystectomy videos. In: Descoteaux M, Maier-Hein L, Franz A, Jannin P, Collins D, Duchesne S, editors. Medical Image Computing and Computer-Assisted Intervention − MICCAI 2017. Lecture Notes in Computer Science, vol. 10434. Cham: Springer; 2017. https://doi.org/10.1007/978-3-319-66185-8_66.

14.

Bai S, Kolter JZ, Koltun V. An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. 2018. arXiv preprint arXiv:1803.01271.

15.

Macario A, Dexter F. Estimating the duration of a case when the surgeon has not recently scheduled the procedure at the surgical suite. Anesth Analg. 1999;89:1241–5.CrossRefPubMed

16.

Wiegmann DA, ElBardissi AW, Dearani JA, Daly RC, Sundt III TM. Disruptions in surgical flow and their relationship to surgical errors: an exploratory investigation. Surgery. 2007;142(5):658–65.CrossRefPubMed

17.

Li X, Zhang Y, Zhang J, Zhou M, Chen S, Gu Y, et al. Progress Estimation and Phase Detection for Sequential Processes. Proc ACM Interact Mob Wearable Ubiquitous Technol. 2017;1(3):73.CrossRefPubMedPubMedCentral

18.

Rivoir D, et al. Unsupervised temporal video segmentation as an auxiliary task for predicting the remaining surgery duration. https://arxiv.org/abs/2002.11367.

19.

Twinanda AP, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N. Endonet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging. 2017;36(1):86–97.CrossRefPubMed

20.

LeCun Y, Bengio Y, Hinton G. Deep learning. Nature. 2015;521(7553):436–44.CrossRefPubMed

21.

Yamashita R, Nishio M, Do RKG, Togashi K. Convolutional neural networks: an overview and application in radiology. Insights Imaging. 2018;9(4):611–29. https://doi.org/10.1007/s13244-018-0639-9.CrossRefPubMedPubMedCentral

22.

Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, et al. Proceedings of the IEEE conference on Computer Vision and Pattern Recognition (CVPR). 2015. p. 1–9.

23.

Esteva A, Kuprel B, Novoa RA, et al. Dermatologist-level classification of skin cancer with deep neural networks. Nature. 2017;542(7639):115–8.CrossRefPubMedPubMedCentral

24.

Brown JM, Campbell JP, Beers A, et al. Automated diagnosis of plus disease in retinopathy of prematurity using deep convolutional neural networks. JAMA Ophthalmol. 2018;136(7):803–10. https://doi.org/10.1001/jamaophthalmol.2018.1934.CrossRefPubMedPubMedCentral

25.

Kitaguchi D, et al. Development and validation of a 3-dimensional convolutional neural network for automatic surgical skill assessment based on spatiotemporal video analysis. JAMA Netw Open. 2021;4(8):e2120786.CrossRefPubMedPubMedCentral

26.

Li L, Verma M, Nakashima Y, Nagahara H, Kawasaki R. IterNet: Retinal Image Segmentation Utilization Structural Redundancy in Vessel Networks. Proceedings in 2020 Winter Conference on Applications of Computer Vision (WACV 20) https://arxiv.org/abs/1912.05763.

27.

Gulshan V, Peng L, Coram M, et al. Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs. JAMA. 2016;316(22):2402–10.CrossRefPubMed

28.

Ting DSW, Cheung CY-L, Lim G, et al. Development and validation of a deep learning system for diabetic retinopathy and related eye diseases using retinal images from multiethnic populations with diabetes. JAMA. 2017;318(22):2211–23.CrossRefPubMedPubMedCentral

29.

Keel S, Wu J, Lee PY, Scheetz J, He M. Visualizing deep learning models for the detection of referable diabetic retinopathy and glaucoma. JAMA Ophthalmol. 2019;137(3):288–92. https://doi.org/10.1001/jamaophthalmol.2018.6035.CrossRefPubMed

30.

Lee CS, Baughman DM, Lee AY. Deep learning is effective for classifying normal versus age-related macular degeneration OCT images. Ophthalmol Retina. 2017;1(4):322–7. https://doi.org/10.1016/j.oret.2016.12.009.CrossRefPubMedPubMedCentral

31.

Burlina PM, Joshi N, Pekala M, Pacheco KD, Freund DE, Bressler NM. Automated grading of age-related macular degeneration from color fundus images using deep convolutional neural networks. JAMA Ophthalmol. 2017;135(11):1170–6.CrossRefPubMedPubMedCentral

32.

Wang B, Takeda T, Sugimoto K, Zhang J, Wada S, Konishi S, et al. Automatic creation of annotations for chest radiographs based on the positional information extracted from radiographic image reports. Comput Methods Programs Biomed. 2021;209:106331. https://doi.org/10.1016/j.cmpb.2021.106331.

33.

Wang M, Wei Z, Jia M, et al. Deep learning model for multi-classification of infectious diseases from unstructured electronic medical records. BMC Med Inform Decis Mak. 2022;22:41. https://doi.org/10.1186/s12911-022-01776-y.CrossRefPubMedPubMedCentral

34.

Hochreiter S, Schmidhuber J. Long short-term memory. Neural Comput. 1997;9(8):1735–80.CrossRefPubMed

35.

Schoeffmann K, Taschwer M, Sarny S, Münzer B, Jürgen Primus M, Putzgruber D. Cataract-101: video dataset of 101 cataract surgeries. In: Proceedings of the 9th ACM Multimedia Systems Conference (MMSys ’18). New York: Association for Computing Machinery; 2018. p. 421–5. https://doi.org/10.1145/3204949.3208137.

36.

Marafioti A, et al. CataNet: Predicting Remaining Cataract Surgery Duration. In: , et al. Medical Image Computing and Computer Assisted Intervention – MICCAI 2021. MICCAI 2021. Lecture Notes in Computer Science, vol. 12904. Cham: Springer; 2021. https://doi.org/10.1007/978-3-030-87202-1_41.

37.

He K, Zhang X, Ren S, Sun J. Deep residual learning for image recognition. In: 2016 IEEE conference on Computer Vision and Pattern Recognition (CVPR). Las Vegas; 2016. p. 770–8.

Title: Real-time estimation of the remaining surgery duration for cataract surgery using deep convolutional neural networks and long short-term memory
Authors: Bowen Wang
Liangzhi Li
Yuta Nakashima
Ryo Kawasaki
Hajime Nagahara
Publication date: 01-12-2023
Publisher: BioMed Central
Keyword: Cataract
Published in: BMC Medical Informatics and Decision Making / Issue 1/2023
Electronic ISSN: 1472-6947
DOI: https://doi.org/10.1186/s12911-023-02160-0

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Real-time estimation of the remaining surgery duration for cataract surgery using deep convolutional neural networks and long short-term memory

Abstract

Purpose

Methods

Result

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose

Methods

Result

Conclusion

Please log in to get access to this content

Other articles of this Issue 1/2023

An automated detection of epileptic seizures EEG using CNN classifier based on feature fusion with high accuracy

Using an adaptive network-based fuzzy inference system for prediction of successful aging: a comparison with common machine learning algorithms

Prostate cancer detection using e-nose and AI for high probability assessment

CNN-Res: deep learning framework for segmentation of acute ischemic stroke lesions on multimodal MRI images

Machine learning based efficient prediction of positive cases of waterborne diseases

Harmonising electronic health records for reproducible research: challenges, solutions and recommendations from a UK-wide COVID-19 research collaboration