Top

International Journal of Computer Assisted Radiology and Surgery

Published in:

01-03-2015 | Original Article

Intra-operative surgical instrument usage detection on a multi-sensor table

Authors: Bernhard Glaser, Stefan Dänzer, Thomas Neumuth

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 3/2015

Abstract

Purpose

Automatic detection, classification and recording of operating room (OR) activities in real time during a surgical procedure requires a wide range of sensors to gather information on the activities of the surgeon and staff, the patient, and the OR equipment. The surgical instrument currently being used is an important parameter used to assess the instantaneous operating room status. An automated system was developed that detects unmodified surgical instruments in real time using a sensor-based table.

Methods

A multi-sensor operating room table was designed featuring a 2D camera, digital scale, and infrared camera. Software was developed to detect and record the sequence of changes on the table during a surgical procedure. The detection rates were evaluated under laboratory conditions by recording the observed instrument usage in 27 functional endoscopic sinus surgeries (FESS).

Results

The detection rate achieved using video-based detection alone was 84.9 %. The total detection rate achieved with the combined approach using both video- and weight-based information was 90.3 %.

Conclusion

A multi-sensor table-based automated instrument tracking system was developed that provides a foundation for the intra-operative detection of surgical instruments without modifying the instruments in the surgical tray. This system was tested and found to satisfy clinical FESS requirements with a reasonable accuracy. This system may also be useful to improve patient safety, for example to prevent instruments being left in the patient.

Sutherland J, Van den Heuvel W (2006) Towards an intelligent hospital environment: adaptive workflow in the OR of the future. In: Proceedings of the 39th annual Hawaii international conference on system sciences, 2006, vol 5. HICSS’06, pp 100b–100b

Macario A, Vitez TS, Dunn B, McDonald T (1995) Where are the costs in perioperative care? Analysis of hospital costs and charges for inpatient surgical care. Anesthesiology 83(6):1138–1144CrossRefPubMed

Archer T, Macario A (2006) The drive for operating room efficiency will increase quality of patient care. Curr Opin Anesthesiol 19(2):171–176CrossRef

Neumuth D, Loebe F, Herre H, Neumuth T (2011) Modeling surgical processes: a four-level translational approach. Artif Intell Med 51(3):147–161CrossRefPubMed

Neumuth T, Jannin P, Schlomberg J, Meixensberger J, Wiedemann P, Burgert O (2011) Analysis of surgical intervention populations using generic surgical process models. Int J Comput Assist Radiol Surg 6(1):59–71CrossRefPubMedCentralPubMed

Cleary K, Kinsella A, Mun SK (2005) OR 2020 workshop report: operating room of the future. Int Congr Ser 1281:832–838CrossRef

McKenzie C, Ibbotson J, Cao C (2001) Hierarchical decomposition of laparoscopic surgery: a human factors approach to investigating the operating room environment. Minim Invasive Ther Allied Technol 10(3):121–127CrossRef

Kranzfelder M, Schneider A, Gillen S, Feussner H (2011) New technologies for information retrieval to achieve situational awareness and higher patient safety in the surgical operating room: the MRI institutional approach and review of the literature. Surg Endosc 25(3):696–705CrossRefPubMed

Uecker DR, Wang YF, Lee C, Wang Y (1995) Laboratory investigation: automated instrument tracking in robotically assisted laparoscopic surgery. Comput Aided Surg 1(6):308–325CrossRef

10.

Speidel S, Delles M, Gutt C, Dillmann R (2006) Tracking of instruments in minimally invasive surgery for surgical skill analysis. In: Medical imaging and augmented reality. Springer, pp 148–155

11.

Speidel S, Benzko J, Krappe S, Sudra G, Azad P, Peter B (2009) Automatic classification of minimally invasive instruments based on endoscopic image sequences. Proc SPIE 7261:72610A–1

12.

Voros S, Long J-A, Cinquin P (2006) Automatic localization of laparoscopic instruments for the visual servoing of an endoscopic camera holder. In: Medical image computing and computer-assisted intervention—MICCAI 2006. Springer, pp 535–542

13.

Voros S, Long J-A, Cinquin P (2007) Automatic detection of instruments in laparoscopic images: a first step towards high-level command of robotic endoscopic holders. Int J Robot Res 26(11–12):1173–1190CrossRef

14.

Sznitman R, Richa R, Taylor RH, Jedynak B, Hager GD (2013) Unified detection and tracking of instruments during retinal microsurgery. IEEE Trans Pattern Anal Mach Intell 35(5):1263–1273CrossRefPubMed

15.

Tonet O, Thoranaghatte RU, Megali G, Dario P (2007) Tracking endoscopic instruments without a localizer: a shape-analysis-based approach. Comput Aided Surg 12(1):35–42CrossRefPubMed

16.

Allen BF, Kasper F, Nataneli G, Dutson E, Faloutsos P (2011) Visual tracking of laparoscopic instruments in standard training environments. Stud Health Technol Inf 163:11–17

17.

Bouarfa L, Akman O, Schneider A, Jonker PP, Dankelman J (2012) In-vivo real-time tracking of surgical instruments in endoscopic video. Minim Invasive Ther Allied Technol 21:129–134CrossRefPubMed

18.

Kranzfelder M, Schneider A, Blahusch G, Schaaf H, Feussner H (2009) Feasibility of opto-electronic surgical instrument identification. Minim Invasive Ther Allied Technol 18(5):253–258CrossRefPubMed

19.

Shepherd JP, Brickley MR, Jones ML (1994) Automatic identification of surgical and orthodontic instruments. Ann R Coll Surg Engl 76(2 Suppl):59–62PubMed

20.

Rogers A, Jones E, Oleynikov D (2007) Radio frequency identification (RFID) applied to surgical sponges. Surg Endosc 21(7):1235–1237CrossRefPubMed

21.

Rivera N, Mountain R, Assumpcao L, Williams AA, Cooper AB, Lewis DL, Benson RC, Miragliotta JA, Marohn M, Taylor RH (2008) ASSIST-automated system for surgical instrument and sponge tracking. In: IEEE international conference on RFID, pp 297–302

22.

Kranzfelder M, Zywitza D, Jell T, Schneider A, Gillen S, Friess H, Feussner H (2012) Real-time monitoring for detection of retained surgical sponges and team motion in the surgical operation room using radio-frequency-identification (RFID) technology: a preclinical evaluation. J Surg Res 175(2):191–198CrossRefPubMed

23.

Neumuth T, Meissner C (2012) Online recognition of surgical instruments by information fusion. Int J Comput Assist Radiol Surg 7(2):297–304CrossRefPubMed

24.

Meißner C, Neumuth T (2012) RFID-based surgical instrument detection using Hidden Markov models. Biomed Tech 57:1

25.

Egan MT, Sandberg WS (2007) Auto identification technology and its impact on patient safety in the operating room of the future. Surg Innov 14(1):41–50 CrossRefPubMed

26.

Lemke HU, Berliner L (2011) Systems design and management of the digital operating room. Int J Comput Assist Radiol Surg 6(1):144–158

27.

Beeri Y, Einey D (2003) VISITS—Visual surgical instrument tracking system, 2003. [Online]. http://www.cs.huji.ac.il/course/2004/postPC/projects/fall2003/visits.html. Accessed 22 Apr 2014

28.

Rattner DW, Park AE (2003) Advanced devices for the operating room of the future. Semin Laparosc Surg 10:85–90PubMed

29.

Bobbit R, Connell J, Haas N, Otto C, Pankanti S, Payne J (2011) Visual item verification for fraud prevention in retail self-checkout. In: IEEE workshop on applications of computer vision (WACV), pp 585–590

30.

Volosyak I, Radchenko O, Pape A, Martens C, She H, Wendland E, Gräser A (2003) Smart tray for the support of a wheelchair mounted manipulator. In: Proceedings of the international conference on economic, engineering and manufacturing systems. ICEEMS 2003

31.

Volosyak I, Ivlev O, Graser A (2005) Rehabilitation robot FRIEND II-the general concept and current implementation. In: 9th International conference on rehabilitation robotics, 2005. ICORR 2005, pp 540–544

32.

Canny J (1986) A computational approach to edge detection. IEEE Trans Pattern Anal Mach Intell 6:679–698CrossRef

33.

Pisinger D (2005) Where are the hard knapsack problems? Comput Oper Res 32(9):2271–2284CrossRef

34.

Neumuth T, Durstewitz N, Fischer M, Strauß G, Dietz A, Meixensberger J, Jannin P, Cleary K, Lemke H. U, Burgert O (2006) Structured recording of intraoperative surgical workflows. In: Medical imaging, pp 61450A–61450A

35.

Kruskal WH, Wallis WA (1952) Use of ranks in one-criterion variance analysis. J Am Stat Assoc 47(260):583–621CrossRef

36.

Mann HB, Whitney DR (1947) On a test of whether one of two random variables is stochastically larger than the other. Ann Math Stat 18(1):50–60CrossRef

37.

Liebmann P, Meixensberger J, Wiedemann P, Neumuth T (2013) The impact of missing sensor information on surgical workflow management. Int J Comput Assist Radiol Surg 8(5):867–875CrossRefPubMed

Title: Intra-operative surgical instrument usage detection on a multi-sensor table
Authors: Bernhard Glaser
Stefan Dänzer
Thomas Neumuth
Publication date: 01-03-2015
Publisher: Springer Berlin Heidelberg
Published in: International Journal of Computer Assisted Radiology and Surgery / Issue 3/2015
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI: https://doi.org/10.1007/s11548-014-1066-0

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Intra-operative surgical instrument usage detection on a multi-sensor table

Abstract

Purpose

Methods

Results

Conclusion

Keynote webinar | Spotlight on medication adherence

Springer Medicine

Abstract

Purpose

Methods

Results

Conclusion

Please log in to get access to this content

Other articles of this Issue 3/2015

GPU-based multi-volume ray casting within VTK for medical applications

Target visibility enhancement for C-arm cone beam CT-fluoroscopy-guided hepatic needle placement: implementation and accuracy evaluation

OpenIGTLink interface for state control and visualisation of a robot for image-guided therapy systems

The NifTK software platform for image-guided interventions: platform overview and NiftyLink messaging

Electromagnetic tracking in surgical and interventional environments: usability study

Fluoroscopy-based laser guidance system for linear surgical tool insertion depth control