Skip to main content
Top
Published in: International Journal of Computer Assisted Radiology and Surgery 2/2024

10-10-2023 | Transcranial Magnetic Stimulation | Original Article

A preliminary exploration into top-down and bottom-up deep-learning approaches to localising neuro-interventional point targets in volumetric MRI

Authors: Enora Giffard, Pierre Jannin, John S. H. Baxter

Published in: International Journal of Computer Assisted Radiology and Surgery | Issue 2/2024

Login to get access

Abstract

Purpose

Point localisation is a critical aspect of many interventional planning procedures, specifically representing anatomical regions of interest or landmarks as individual points. This could be seen as analogous to the problem of visual search in cognitive psychology, in which this search is performed either: bottom-up, constructing increasingly abstract and coarse-resolution features over the entire image; or top-down, using contextual cues from the entire image to refine the scope of the region being investigated. Traditional convolutional neural networks use the former, but it is not clear if this is optimal. This article is a preliminary investigation as to how this motivation affects 3D point localisation in neuro-interventional planning.

Methods

Two neuro-imaging datasets were collected: one for cortical point localisation for repetitive transcranial magnetic stimulation and the other for sub-cortical anatomy localisation for deep brain stimulation. Four different frameworks were developed using top-down versus bottom-up paradigms as well as representing points as co-ordinates or heatmaps. These networks were applied to point localisation for transcranial magnetic stimulation and subcortical anatomy localisation. These networks were evaluated using cross-validation and a varying number of training datasets to analyse their sensitivity to quantity of training data.

Results

Each network shows increasing performance as the amount of available training data increases, with the co-ordinate-based top-down network consistently outperforming the others. Specifically, the top-down architectures tend to outperform the bottom-up ones. An analysis of their memory consumption also encourages the top-down co-ordinate based architecture as it requires significantly less memory than either bottom-up architectures or those representing their predictions via heatmaps.

Conclusion

This paper is a preliminary foray into a fundamental aspect of machine learning architectural design: that of the top-down/bottom-up divide from cognitive psychology. Although there are additional considerations within the particular architectures investigated that could affect these results and the number of architectures investigated is limited, our results do indicate that the less commonly used top-down paradigm could lead to more efficient and effective architectures in the future.
Footnotes
1
http://​www.​itksnap.​org/​pmwiki/​pmwiki.​php?​n=​Convert3D.​Convert3D, The specific commands used were swapdim, -resample-mm, and -pad-to.
 
Literature
1.
go back to reference He K, Sun J (2015) Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5353–5360 He K, Sun J (2015) Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5353–5360
2.
go back to reference Qiao S, Lin Z, Zhang J, Yuille A L (2019) Neural rejuvenation: improving deep network training by enhancing computational resource utilization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 61–71 Qiao S, Lin Z, Zhang J, Yuille A L (2019) Neural rejuvenation: improving deep network training by enhancing computational resource utilization. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp. 61–71
3.
go back to reference Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587 Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587
4.
go back to reference Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp. 1440–1448 Girshick R (2015) Fast r-cnn. In: Proceedings of the IEEE international conference on computer vision, pp. 1440–1448
5.
go back to reference Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol. 28, pp. 91–99 Ren S, He K, Girshick R, Sun J (2015) Faster r-cnn: towards real-time object detection with region proposal networks. In: Advances in neural information processing systems, vol. 28, pp. 91–99
6.
go back to reference Sugimori H, Kawakami M (2019) Automatic detection of a standard line for brain magnetic resonance imaging using deep learning. In: Applied Sciences, vol. 9, 3849 Sugimori H, Kawakami M (2019) Automatic detection of a standard line for brain magnetic resonance imaging using deep learning. In: Applied Sciences, vol. 9, 3849
7.
go back to reference Yang X, Tang WT, Tjio G, Yeo SY, Su Y (2020) Automatic detection of anatomical landmarks in brain MR scanning using multi-task deep neural networks. In: Neurocomputing, vol. 396, pp. 514–521 Yang X, Tang WT, Tjio G, Yeo SY, Su Y (2020) Automatic detection of anatomical landmarks in brain MR scanning using multi-task deep neural networks. In: Neurocomputing, vol. 396, pp. 514–521
8.
go back to reference Gohel B, Kumar L, Shah D (2023) Deep learning-based automated localisation of anterior commissure and posterior commissure landmarks in 3d space from three-plane 2d mri localiser slices of the brain. In: Procedia Computer Science, vol. 218, pp. 1027–1032 Gohel B, Kumar L, Shah D (2023) Deep learning-based automated localisation of anterior commissure and posterior commissure landmarks in 3d space from three-plane 2d mri localiser slices of the brain. In: Procedia Computer Science, vol. 218, pp. 1027–1032
9.
go back to reference Baxter JS, Bui QA, Maguet E, Croci S, Delmas A, Lefaucheur J-P, Bredoux L, Jannin P (2021) Automatic cortical target point localisation in MRI for transcranial magnetic stimulation via a multi-resolution convolutional neural network. In: International Journal of Computer Assisted Radiology and Surgery, vol. 16, pp.1077–1087 Baxter JS, Bui QA, Maguet E, Croci S, Delmas A, Lefaucheur J-P, Bredoux L, Jannin P (2021) Automatic cortical target point localisation in MRI for transcranial magnetic stimulation via a multi-resolution convolutional neural network. In: International Journal of Computer Assisted Radiology and Surgery, vol. 16, pp.1077–1087
10.
go back to reference Foulsham T, Chapman C, Nasiopoulos E, Kingstone A (2014) Top-down and bottom-up aspects of active search in a real-world environment. In: Canadian Journal of Experimental Psychology, vol. 68, pp. 8–19 Foulsham T, Chapman C, Nasiopoulos E, Kingstone A (2014) Top-down and bottom-up aspects of active search in a real-world environment. In: Canadian Journal of Experimental Psychology, vol. 68, pp. 8–19
11.
go back to reference Li S, Gong Q, Li H, Chen S, Liu Y, Ruan G, Zhu L, Liu L, Chen H (2022) Automatic location scheme of anatomical landmarks in 3d head MRI based on the scale attention hourglass network. Comput Methods Progr Biomed 214:106564CrossRef Li S, Gong Q, Li H, Chen S, Liu Y, Ruan G, Zhu L, Liu L, Chen H (2022) Automatic location scheme of anatomical landmarks in 3d head MRI based on the scale attention hourglass network. Comput Methods Progr Biomed 214:106564CrossRef
12.
go back to reference Lester H, Arridge SR (1999) A survey of hierarchical non-linear medical image registration. Pattern Recogn 32(1):129–149ADSCrossRef Lester H, Arridge SR (1999) A survey of hierarchical non-linear medical image registration. Pattern Recogn 32(1):129–149ADSCrossRef
13.
14.
go back to reference Balconi M (2013) Dorsolateral prefrontal cortex, working memory and episodic memory processes: insight through transcranial magnetic stimulation techniques. Neurosci Bull 29:381–389CrossRefPubMedPubMedCentral Balconi M (2013) Dorsolateral prefrontal cortex, working memory and episodic memory processes: insight through transcranial magnetic stimulation techniques. Neurosci Bull 29:381–389CrossRefPubMedPubMedCentral
15.
go back to reference Hamid P, Malik BH, Hussain ML (2019) Noninvasive transcranial magnetic stimulation (TMS) in chronic refractory pain: a systematic review. Cureus 11(10):e6019PubMedPubMedCentral Hamid P, Malik BH, Hussain ML (2019) Noninvasive transcranial magnetic stimulation (TMS) in chronic refractory pain: a systematic review. Cureus 11(10):e6019PubMedPubMedCentral
16.
go back to reference Sparing R, Buelte D, Meister IG, Pauš T, Fink GR (2008) Transcranial magnetic stimulation and the challenge of coil placement: a comparison of conventional and stereotaxic neuronavigational strategies. Hum Brain Mapp 29(1):82–96CrossRefPubMed Sparing R, Buelte D, Meister IG, Pauš T, Fink GR (2008) Transcranial magnetic stimulation and the challenge of coil placement: a comparison of conventional and stereotaxic neuronavigational strategies. Hum Brain Mapp 29(1):82–96CrossRefPubMed
17.
go back to reference Siebner HR, Hartwigsen G, Kassuba T, Rothwell JC (2009) How does transcranial magnetic stimulation modify neuronal activity in the brain? Implications for studies of cognition. Cortex 45(9):1035–1042CrossRefPubMedPubMedCentral Siebner HR, Hartwigsen G, Kassuba T, Rothwell JC (2009) How does transcranial magnetic stimulation modify neuronal activity in the brain? Implications for studies of cognition. Cortex 45(9):1035–1042CrossRefPubMedPubMedCentral
18.
go back to reference Middlebrooks E, Domingo R, Vivas-Buitrago T, Okromelidze L, Tsuboi T, Wong J, Eisinger R, Almeida L, Burns M, Horn A et al (2020) Neuroimaging advances in deep brain stimulation: review of indications, anatomy, and brain connectomics. Am J Neuroradiol 41(9):1558–1568 Middlebrooks E, Domingo R, Vivas-Buitrago T, Okromelidze L, Tsuboi T, Wong J, Eisinger R, Almeida L, Burns M, Horn A et al (2020) Neuroimaging advances in deep brain stimulation: review of indications, anatomy, and brain connectomics. Am J Neuroradiol 41(9):1558–1568
19.
go back to reference Baxter JS, Jannin P (2023) Validation in the age of machine learning: a framework for describing validation with examples in transcranial magnetic stimulation and deep brain stimulation. Intell. -Based Med. 7:100090 Baxter JS, Jannin P (2023) Validation in the age of machine learning: a framework for describing validation with examples in transcranial magnetic stimulation and deep brain stimulation. Intell. -Based Med. 7:100090
20.
go back to reference Haegelen C, Coupé P, Fonov V, Guizard N, Jannin P, Morandi X, Collins DL (2013) Automated segmentation of basal ganglia and deep brain structures in MRI of Parkinson’s disease. Int J Comput Assist Radiol Surg 8(1):99–110CrossRefPubMed Haegelen C, Coupé P, Fonov V, Guizard N, Jannin P, Morandi X, Collins DL (2013) Automated segmentation of basal ganglia and deep brain structures in MRI of Parkinson’s disease. Int J Comput Assist Radiol Surg 8(1):99–110CrossRefPubMed
21.
go back to reference Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440 Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 3431–3440
22.
go back to reference Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention-MICCAI, 18th International Conference, Munich, Proceedings, Part III 18. Springer 2015, pp. 234–241 Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: Medical image computing and computer-assisted intervention-MICCAI, 18th International Conference, Munich, Proceedings, Part III 18. Springer 2015, pp. 234–241
23.
go back to reference Ranftl R, Bochkovskiy A, Koltun V (2021) Vision transformers for dense prediction. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 12 179–12 188 Ranftl R, Bochkovskiy A, Koltun V (2021) Vision transformers for dense prediction. In: Proceedings of the IEEE/CVF international conference on computer vision, pp. 12 179–12 188
24.
go back to reference Hatamizadeh A, Nath V, Tang Y, Yang D, Roth HR, Xu D (2021) Swin unetr: swin transformers for semantic segmentation of brain tumors in MRI images. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries: 7th international workshop, BrainLes. Held in Conjunction with MICCAI 2021, Virtual Event, Revised Selected Papers. Part I. Springer 2022:272–284 Hatamizadeh A, Nath V, Tang Y, Yang D, Roth HR, Xu D (2021) Swin unetr: swin transformers for semantic segmentation of brain tumors in MRI images. In: Brainlesion: glioma, multiple sclerosis, stroke and traumatic brain injuries: 7th international workshop, BrainLes. Held in Conjunction with MICCAI 2021, Virtual Event, Revised Selected Papers. Part I. Springer 2022:272–284
25.
go back to reference Baxter JS, Jannin P (2022) Combining simple interactivity and machine learning: a separable deep learning approach to subthalamic nucleus localization and segmentation in MRI for deep brain stimulation surgical planning. J Med Imaging 9(4):045001CrossRef Baxter JS, Jannin P (2022) Combining simple interactivity and machine learning: a separable deep learning approach to subthalamic nucleus localization and segmentation in MRI for deep brain stimulation surgical planning. J Med Imaging 9(4):045001CrossRef
Metadata
Title
A preliminary exploration into top-down and bottom-up deep-learning approaches to localising neuro-interventional point targets in volumetric MRI
Authors
Enora Giffard
Pierre Jannin
John S. H. Baxter
Publication date
10-10-2023
Publisher
Springer International Publishing
Published in
International Journal of Computer Assisted Radiology and Surgery / Issue 2/2024
Print ISSN: 1861-6410
Electronic ISSN: 1861-6429
DOI
https://doi.org/10.1007/s11548-023-03023-9

Other articles of this Issue 2/2024

International Journal of Computer Assisted Radiology and Surgery 2/2024 Go to the issue