Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

01-08-2018 | Original Article

The intelligent OR: design and validation of a context-aware surgical working environment

Authors: Stefan Franke, Max Rockstroh, Mathias Hofer, Thomas Neumuth

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 8/2018

Abstract

Purpose

Interoperability of medical devices based on standards starts to establish in the operating room (OR). Devices share their data and control functionalities. Yet, the OR technology rarely implements cooperative, intelligent behavior, especially in terms of active cooperation with the OR team. Technical context-awareness will be an essential feature of the next generation of medical devices to address the increasing demands to clinicians in information seeking, decision making, and human–machine interaction in complex surgical working environments.

Methods

The paper describes the technical validation of an intelligent surgical working environment for endoscopic ear–nose–throat surgery. We briefly summarize the design of our framework for context-aware system’s behavior in integrated OR and present example realizations of novel assistance functionalities. In a study on patient phantoms, twenty-four procedures were implemented in the proposed intelligent surgical working environment based on recordings of real interventions. Subsequently, the whole processing pipeline for context-awareness from workflow recognition to the final system’s behavior is analyzed.

Results

Rule-based behavior that considers multiple perspectives on the procedure can partially compensate recognition errors. A considerable robustness could be achieved with a reasonable quality of the recognition. Overall, reliable reactive as well as proactive behavior of the surgical working environment can be implemented in the proposed environment.

Conclusions

The obtained validation results indicate the suitability of the overall approach. The setup is a reliable starting point for a subsequent evaluation of the proposed context-aware assistance. The major challenge for future work will be to implement the complex approach in a cross-vendor setting.

Kasparick M, Schlichting S, Golatowski F, Timmermann D (2015) New IEEE 11073 standards for interoperable, networked point-of-care medical devices. In: Proceedings of the 37th IEEE engineering in medicine and biology society (EMBC). IEEE, pp 1721–1724

Andersen B, Ulrich H, Rehmann D, Kock A-K, Wrage J-H, Ingenerf J (2015) Reporting device observations for semantic interoperability of surgical devices and clinical information systems. In: Proceedings of the 37th IEEE engineering in medicine and biology society (EMBC). IEEE, pp 1725–1728

Rockstroh M, Franke S, Hofer M, Will A, Kasparick M, Andersen B, Neumuth T (2017) ORNET: multi-perspective qualitative evaluation of an integrated operating room based on IEEE 11073 SDC. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-017-1589-2 PubMedCrossRef

Katić D, Julliard C, Wekerle A-L, Kenngott H, Müller-Stich BP, Dillmann R, Speidel S, Jannin P, Gibaud B (2015) LapOntoSPM: an ontology for laparoscopic surgeries and its application to surgical phase recognition. Int J Comput Assist Radiol Surg. https://doi.org/10.1007/s11548-015-1222-1 CrossRefPubMed

Blum T, Padoy N, Feußner H, Navab N (2008) Modeling and online recognition of surgical phases using hidden Markov models. In: Metaxas D, Axel L, Fichtinger G, Székely G (eds) Medical image computing and computer-assisted intervention-MICCAI 2008. Springer, Berlin, pp 627–635CrossRef

Lalys F, Riffaud L, Bouget D, Jannin P (2011) An application-dependent framework for the recognition of high-level surgical tasks in the OR. In: Fichtinger G, Martel A, Peters T (eds) Medical image computing and computer-assisted intervention-MICCAI 2011. Springer, Berlin, pp 331–338CrossRef

Speidel S, Zentek T, Sudra G, Gehrig T, Müller-Stich BP, Gutt C, Dillmann R (2009) Recognition of surgical skills using hidden Markov models. In: Proceedings volume 7261, medical imaging 2009: visualization, image-guided procedures, and modeling, pp 726125-8–726125-8

Kranzfelder M, Schneider A, Fiolka A, Koller S, Reiser S, Vogel T, Wilhelm D, Feussner H (2014) Reliability of sensor-based real-time workflow recognition in laparoscopic cholecystectomy. Int J Comput Assist Radiol Surg 9:941–948. https://doi.org/10.1007/s11548-014-0986-z CrossRefPubMed

Lalys F, Riffaud L, Bouget D, Jannin P (2012) A Framework for the recognition of high-level surgical tasks from video images for cataract surgeries. IEEE Trans Biomed Eng 59:966–976. https://doi.org/10.1109/TBME.2011.2181168 CrossRefPubMed

10.

Bouarfa L, Akman O, Schneider A, Jonker PP, Dankelman J (2012) In-vivo real-time tracking of surgical instruments in endoscopic video. Minim Invasive Therapy Allied Technol 21:129–134. https://doi.org/10.3109/13645706.2011.580764 CrossRef

11.

Kranzfelder M, Schneider A, Fiolka A, Schwan E, Gillen S, Wilhelm D, Schirren R, Reiser S, Jensen B, Feussner H (2013) Real-time instrument detection in minimally invasive surgery using radiofrequency identification technology. J Surg Res 185:704–710. https://doi.org/10.1016/j.jss.2013.06.022 CrossRefPubMed

12.

Glaser B, Dänzer S, Neumuth T (2015) Intra-operative surgical instrument usage detection on a multi-sensor table. Int J Comput Assist Radiol Surg 10:351–362. https://doi.org/10.1007/s11548-014-1066-0 CrossRefPubMed

13.

Neumuth T (2017) Surgical process modeling. Innov Surg Sci. https://doi.org/10.1515/iss-2017-0005 CrossRef

14.

Schmidt A, Beigl M, Gellersen H-W (1999) There is more to context than location. Comput Graph 23:893–901. https://doi.org/10.1016/S0097-8493(99)00120-X CrossRef

15.

Franke S, Meixensberger J, Neumuth T (2015) Multi-perspective workflow modeling for online surgical situation models. J Biomed Inform 54:158–166CrossRefPubMed

16.

Franke S, Neumuth T (2015) Adaptive surgical process models for prediction of surgical work steps from surgical low-level activities. In: Workshop on modeling and monitoring of computer-aided interventions (M2CAI). Medical Image Computing and Computer Assisted Intervention, Munich, Germany

17.

Franke S, Meixensberger J, Neumuth T (2013) Intervention time prediction from surgical low-level tasks. J Biomed Inform 46:152–159. https://doi.org/10.1016/j.jbi.2012.10.002 CrossRefPubMed

18.

Franke S, Neumuth T (2015) Towards structuring contextual information for workflow-driven surgical assistance functionalities. In: 49th annual conference of the German society for biomedical engineering (BMT 2015), Lübeck, Germany

19.

Franke S, Neumuth T (2015) Rule-based medical device adaptation for the digital operating room. In: Proceedings of 37th annual international conference of the IEEE engineering in medicine and biology society. IEEE, Milano, pp 1733–1736

20.

Franke S, Rockstroh M, Schreiber E, Neumann J, Neumuth T (2016) Context-aware medical assistance systems in integrated surgical environments. In: 28th conference of the international society for medical innovation and technology (SMIT), Delft, Netherlands, p 98

21.

Meyer MA, Levine WC, Egan MT, Cohen BJ, Spitz G, Garcia P, Chueh H, Sandberg WS (2007) A computerized perioperative data integration and display system. Int J Comput Assist Radiol Surg 2:191–202. https://doi.org/10.1007/s11548-007-0126-0 CrossRef

22.

Pickering BW, Herasevich V, Ahmed A, Gajic O (2010) Novel representation of clinical information in the ICU: developing user interfaces which reduce information overload. Appl Clin Inform 1:116–131. https://doi.org/10.4338/ACI-2009-12-CR-0027 CrossRefPubMedPubMedCentral

23.

Kranzfelder M, Staub C, Fiolka A, Schneider A, Gillen S, Wilhelm D, Friess H, Knoll A, Feussner H (2013) Toward increased autonomy in the surgical OR: needs, requests, and expectations. Surg Endosc 27:1681–1688. https://doi.org/10.1007/s00464-012-2656-y CrossRefPubMed

24.

Rockstroh M, Franke S, Neumuth T (2013) A workflow-driven surgical information source management. In: Lemke HU (ed) The international journal for computer assisted radiology and surgery. Springer, Heidelberg, pp 189–191

25.

Neumuth T, Jannin P, Strauss G, Meixensberger J, Burgert O (2009) Validation of knowledge acquisition for surgical process models. J Am Med Inf Assoc 16(1):72–80CrossRef

26.

Twinanda A, Shehata S, Mutter D, Marescaux J, de Mathelin M, Padoy N (2017) EndoNet: a deep architecture for recognition tasks on laparoscopic videos. IEEE Trans Med Imaging 36(1):86–97CrossRefPubMed

Title: The intelligent OR: design and validation of a context-aware surgical working environment
Authors: Stefan Franke
Max Rockstroh
Mathias Hofer
Thomas Neumuth
Publication date: 01-08-2018
Publisher: Springer International Publishing
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 8/2018
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-018-1791-x

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

The intelligent OR: design and validation of a context-aware surgical working environment

Abstract

Purpose

Methods

Results

Conclusions

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusions

Please log in to get access to this content

Other articles of this Issue 8/2018

Information architecture for a patient-specific dashboard in head and neck tumor boards

A pre-operative planning framework for global registration of laparoscopic ultrasound to CT images

Prostate segmentation in MRI using a convolutional neural network architecture and training strategy based on statistical shape models

3D/2D model-to-image registration by imitation learning for cardiac procedures

An automated liver tumour segmentation from abdominal CT scans for hepatic surgical planning

Viewpoint planning for quantitative coronary angiography