Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 8/2018

01-08-2018 | Original Article

A deep learning framework for segmentation and pose estimation of pedicle screw implants based on C-arm fluoroscopy

Authors: Hooman Esfandiari, Robyn Newell, Carolyn Anglin, John Street, Antony J. Hodgson

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 8/2018

Login to get access

Abstract

Purpose

Pedicle screw fixation is a challenging procedure with a concerning rates of reoperation. After insertion of the screws is completed, the most common intraoperative verification approach is to acquire anterior–posterior and lateral radiographic images, based on which the surgeons try to visually assess the correctness of insertion. Given the limited accuracy of the existing verification techniques, we identified the need for an accurate and automated pedicle screw assessment system that can verify the screw insertion intraoperatively. For doing so, this paper offers a framework for automatic segmentation and pose estimation of pedicle screws based on deep learning principles.

Methods

Segmentation of pedicle screw X-ray projections was performed by a convolutional neural network. The network could isolate the input X-rays into three classes: screw head, screw shaft and background. Once all the screw shafts were segmented, knowledge about the spatial configuration of the acquired biplanar X-rays was used to identify the correspondence between the projections. Pose estimation was then performed to estimate the 6 degree-of-freedom pose of each screw. The performance of the proposed pose estimation method was tested on a porcine specimen.

Results

The developed machine learning framework was capable of segmenting the screw shafts with 93% and 83% accuracy when tested on synthetic X-rays and on clinically realistic X-rays, respectively. The pose estimation accuracy of this method was shown to be \(1.93^{\circ } \pm 0.64^{\circ }\) and \(1.92 \pm 0.55\,\hbox {mm}\) on clinically realistic X-rays.

Conclusions

The proposed system offers an accurate and fully automatic pedicle screw segmentation and pose assessment framework. Such a system can help to provide an intraoperative pedicle screw insertion assessment protocol with minimal interference with the existing surgical routines.
Appendix
Available only for authorised users
Literature
1.
go back to reference Heim SE (1997) Transpedicle instrumentation in the degenerative spine. Clin Orthop Relat Res 337:97–110CrossRef Heim SE (1997) Transpedicle instrumentation in the degenerative spine. Clin Orthop Relat Res 337:97–110CrossRef
2.
go back to reference Katonis P, Christoforakis J, Aligizakis AC, Papadopoulos C, Sapkas G, Hadjipavlou A (2003) Complications and problems related to pedicle screw fixation of the spine. Clin Orthop Relat Res 411:86–94CrossRef Katonis P, Christoforakis J, Aligizakis AC, Papadopoulos C, Sapkas G, Hadjipavlou A (2003) Complications and problems related to pedicle screw fixation of the spine. Clin Orthop Relat Res 411:86–94CrossRef
3.
go back to reference Ackbas SC, Arslan FY, Tuncer MR (2003) The effect of transpedicular screw misplacement on late spinal stability. Acta Neurochir 145:949–955CrossRef Ackbas SC, Arslan FY, Tuncer MR (2003) The effect of transpedicular screw misplacement on late spinal stability. Acta Neurochir 145:949–955CrossRef
4.
go back to reference Gelalis ID, Paschos NK, Pakos EE, Politis AN, Arnaoutoglou CM, Karageorgos AC, Ploumis A, Xenakis TA (2011) Accuracy of pedicle screw placement: a systematic review of prospective in vivo studies comparing free hand, fluoroscopy guidance and navigation techniques. Eur Spine J 21:247–255CrossRefPubMedPubMedCentral Gelalis ID, Paschos NK, Pakos EE, Politis AN, Arnaoutoglou CM, Karageorgos AC, Ploumis A, Xenakis TA (2011) Accuracy of pedicle screw placement: a systematic review of prospective in vivo studies comparing free hand, fluoroscopy guidance and navigation techniques. Eur Spine J 21:247–255CrossRefPubMedPubMedCentral
5.
go back to reference Amato V, Giannachi L, Irace C, Corona C (2010) Accuracy of pedicle screw placement in the lumbosacral spine using conventional technique: computed tomography postoperative assessment in 102 consecutive patients: Clinical article. J Neurosurg Spine 12:306–313CrossRefPubMed Amato V, Giannachi L, Irace C, Corona C (2010) Accuracy of pedicle screw placement in the lumbosacral spine using conventional technique: computed tomography postoperative assessment in 102 consecutive patients: Clinical article. J Neurosurg Spine 12:306–313CrossRefPubMed
6.
7.
go back to reference Allam Y, Silbermann J, Riese F, Greiner-Perth R (2013) Computer tomography assessment of pedicle screw placement in thoracic spine: comparison between free hand and a generic 3D-based navigation techniques. Eur Spine J 22:648–653CrossRefPubMed Allam Y, Silbermann J, Riese F, Greiner-Perth R (2013) Computer tomography assessment of pedicle screw placement in thoracic spine: comparison between free hand and a generic 3D-based navigation techniques. Eur Spine J 22:648–653CrossRefPubMed
8.
go back to reference Chiang CF, Tsai TT, Chen LH, Lai PL, Fu TS, Niu CC, Chen WJ (2012) Computed tomography-based navigation-assisted pedicle screw insertion for thoracic and lumbar spine fractures. Chang Gung Med J 35:332–338 Chiang CF, Tsai TT, Chen LH, Lai PL, Fu TS, Niu CC, Chen WJ (2012) Computed tomography-based navigation-assisted pedicle screw insertion for thoracic and lumbar spine fractures. Chang Gung Med J 35:332–338
9.
go back to reference Choma TJ, Denis F, Lonstein JE, Perra JH, Schwender JD, Garvey TA, Mullin WJ (2006) Stepwise methodology for plain radiographic assessment of pedicle screw placement: a comparison with computed tomography. J Spinal Disord Tech 19:547–553CrossRefPubMed Choma TJ, Denis F, Lonstein JE, Perra JH, Schwender JD, Garvey TA, Mullin WJ (2006) Stepwise methodology for plain radiographic assessment of pedicle screw placement: a comparison with computed tomography. J Spinal Disord Tech 19:547–553CrossRefPubMed
10.
go back to reference Cordemans V, Kaminski L, Banse X, Francq BG, Cartiaux O (2017) Accuracy of a new intraoperative cone beam CT imaging technique (Artis Zeego II) compared to postoperative CT scan for assessment of pedicle screws placement and breaches detection. Eur Spine J 26:2906–2916CrossRefPubMed Cordemans V, Kaminski L, Banse X, Francq BG, Cartiaux O (2017) Accuracy of a new intraoperative cone beam CT imaging technique (Artis Zeego II) compared to postoperative CT scan for assessment of pedicle screws placement and breaches detection. Eur Spine J 26:2906–2916CrossRefPubMed
12.
go back to reference Markelj P, Tomazevic D, Likar B, Pernus F (2012) A review of 3D/2D registration methods for image-guided interventions. Med Image Anal 16:642–661CrossRefPubMed Markelj P, Tomazevic D, Likar B, Pernus F (2012) A review of 3D/2D registration methods for image-guided interventions. Med Image Anal 16:642–661CrossRefPubMed
13.
go back to reference Otake Y, Schafer S, Stayman JW, Zbijewski W, Kleinszig G, Graumann R, Khanna AJ, Siewerdsen JH (2012) Automatic localization of vertebral levels in x-ray fluoroscopy using 3D–2D registration: a tool to reduce wrong-site surgery. Phys Med Biol 57:5485–5508CrossRefPubMedPubMedCentral Otake Y, Schafer S, Stayman JW, Zbijewski W, Kleinszig G, Graumann R, Khanna AJ, Siewerdsen JH (2012) Automatic localization of vertebral levels in x-ray fluoroscopy using 3D–2D registration: a tool to reduce wrong-site surgery. Phys Med Biol 57:5485–5508CrossRefPubMedPubMedCentral
14.
go back to reference Varnavas A, Carrell T, Penney G (2015) Fully automated 2D–3D registration and verification. Med Image Anal 26:108–119CrossRefPubMed Varnavas A, Carrell T, Penney G (2015) Fully automated 2D–3D registration and verification. Med Image Anal 26:108–119CrossRefPubMed
15.
go back to reference Miao S, Wang ZJ, Liao R (2016) A CNN regression approach for real-time 2D/3D registration. IEEE Trans Med Imaging 35:1352–1363CrossRef Miao S, Wang ZJ, Liao R (2016) A CNN regression approach for real-time 2D/3D registration. IEEE Trans Med Imaging 35:1352–1363CrossRef
16.
go back to reference Popescu D, Amza CG, Laptoiu D, Amza G (2012) Competitive hopfield neural network model for evaluating pedicle Screw placement accuracy. Stroj vestn J Mech Eng 58:509–516CrossRef Popescu D, Amza CG, Laptoiu D, Amza G (2012) Competitive hopfield neural network model for evaluating pedicle Screw placement accuracy. Stroj vestn J Mech Eng 58:509–516CrossRef
17.
go back to reference Uneri A, De Silva T, Goerres J, Jacobson M, Ketcha M, Reaungamornrat S, Kleinszig G, Vogt S, Khanna A, Osgood G, Wolinsky JP, Siewerdsen J (2017) Intraoperative evaluation of device placement in spine surgery using known-component 3D–2D image registration. Phys Med Biol 62:3330–3351CrossRefPubMedPubMedCentral Uneri A, De Silva T, Goerres J, Jacobson M, Ketcha M, Reaungamornrat S, Kleinszig G, Vogt S, Khanna A, Osgood G, Wolinsky JP, Siewerdsen J (2017) Intraoperative evaluation of device placement in spine surgery using known-component 3D–2D image registration. Phys Med Biol 62:3330–3351CrossRefPubMedPubMedCentral
18.
go back to reference Navab N, Bani-Hashemi AR, Mitschke MM, Holdsworth DW, Fahrig R, Fox AJ, Graumann R (1996) Dynamic geometrical calibration for 3D cerebral angiography. In: SPIE—The International Society for Optical Engineering. International Society for Optics and Photonics, pp 361–370 Navab N, Bani-Hashemi AR, Mitschke MM, Holdsworth DW, Fahrig R, Fox AJ, Graumann R (1996) Dynamic geometrical calibration for 3D cerebral angiography. In: SPIE—The International Society for Optical Engineering. International Society for Optics and Photonics, pp 361–370
19.
go back to reference Chintalapani G, Jain AK, Burkhardt DH, PrinceJL, Fichtinger G (2008) CTREC: C-arm tracking and reconstruction using elliptic curves. In: Conference on computer vision and pattern recognition workshops. IEEE, pp 1–7 Chintalapani G, Jain AK, Burkhardt DH, PrinceJL, Fichtinger G (2008) CTREC: C-arm tracking and reconstruction using elliptic curves. In: Conference on computer vision and pattern recognition workshops. IEEE, pp 1–7
20.
go back to reference Schumann S, Thelen B, Ballestra S, Nolte LP, Bchler P, Zheng G (2014) X-ray image calibration and its application to clinical orthopedics. Med Eng Phys 36:968–974CrossRefPubMed Schumann S, Thelen B, Ballestra S, Nolte LP, Bchler P, Zheng G (2014) X-ray image calibration and its application to clinical orthopedics. Med Eng Phys 36:968–974CrossRefPubMed
21.
go back to reference Amiri S, Wilson DR, Masri BA, Anglin C (2014) A low-cost tracked C-arm (TC-arm) upgrade system for versatile quantitative intraoperative imaging. Int J CARS 9:695–711CrossRef Amiri S, Wilson DR, Masri BA, Anglin C (2014) A low-cost tracked C-arm (TC-arm) upgrade system for versatile quantitative intraoperative imaging. Int J CARS 9:695–711CrossRef
22.
go back to reference Esfandiari H, Martinez JF, Gonzlez Ivarez A, Guy P, Street J, Anglin C, Hodgson AJ (2017) An automatic, robust and closed form mini-RSA system for intraoperative C-arm calibration. Int J Comput Assist Radiol Surg 12(Suppl 1):S37–S38 Esfandiari H, Martinez JF, Gonzlez Ivarez A, Guy P, Street J, Anglin C, Hodgson AJ (2017) An automatic, robust and closed form mini-RSA system for intraoperative C-arm calibration. Int J Comput Assist Radiol Surg 12(Suppl 1):S37–S38
23.
go back to reference Abdel-Aziz YI, Karara HM (1971). Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. In: Proceedings of the American society of photogrammetry symposium on close-range photogrammetry, Washington, DC, 1-18. ASP, Falls Church, VA Abdel-Aziz YI, Karara HM (1971). Direct linear transformation from comparator coordinates into object space coordinates in close-range photogrammetry. In: Proceedings of the American society of photogrammetry symposium on close-range photogrammetry, Washington, DC, 1-18. ASP, Falls Church, VA
24.
go back to reference Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440 Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440
25.
go back to reference Vedaldi A, Lenc K (2015) MatConvNet: convolutional neural networks for MATLAB. In: Proceedings of the 23rd ACM international conference on multimedia, ACM Press, Brisbane, Australia, pp 689–692 Vedaldi A, Lenc K (2015) MatConvNet: convolutional neural networks for MATLAB. In: Proceedings of the 23rd ACM international conference on multimedia, ACM Press, Brisbane, Australia, pp 689–692
26.
go back to reference Seroul P, Sarrut D (2008) VV: A viewer for the evaluation of 4D image registration. In: MIDAS Journal (Medical image computing and computer-assisted intervention MICCAI2008, workshop-systems and architectures for computer assisted interventions), p 18 Seroul P, Sarrut D (2008) VV: A viewer for the evaluation of 4D image registration. In: MIDAS Journal (Medical image computing and computer-assisted intervention MICCAI2008, workshop-systems and architectures for computer assisted interventions), p 18
27.
go back to reference Haough Paul VC(1962) Method and means for recognizing complex patterns. Patent number: US3069654 A Haough Paul VC(1962) Method and means for recognizing complex patterns. Patent number: US3069654 A
Metadata
Title
A deep learning framework for segmentation and pose estimation of pedicle screw implants based on C-arm fluoroscopy
Authors
Hooman Esfandiari
Robyn Newell
Carolyn Anglin
John Street
Antony J. Hodgson
Publication date
01-08-2018
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 8/2018
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-018-1776-9

Other articles of this Issue 8/2018

International Journal of Computer Assisted Radiology and Surgery 8/2018 Go to the issue