Top

Journal of Imaging Informatics in Medicine

Published in:

Open Access 01-06-2018

Towards Portable Large-Scale Image Processing with High-Performance Computing

Authors: Yuankai Huo, Justin Blaber, Stephen M. Damon, Brian D. Boyd, Shunxing Bao, Prasanna Parvathaneni, Camilo Bermudez Noguera, Shikha Chaganti, Vishwesh Nath, Jasmine M. Greer, Ilwoo Lyu, William R. French, Allen T. Newton, Baxter P. Rogers, Bennett A. Landman

Published in: Journal of Imaging Informatics in Medicine | Issue 3/2018

Abstract

High-throughput, large-scale medical image computing demands tight integration of high-performance computing (HPC) infrastructure for data storage, job distribution, and image processing. The Vanderbilt University Institute for Imaging Science (VUIIS) Center for Computational Imaging (CCI) has constructed a large-scale image storage and processing infrastructure that is composed of (1) a large-scale image database using the eXtensible Neuroimaging Archive Toolkit (XNAT), (2) a content-aware job scheduling platform using the Distributed Automation for XNAT pipeline automation tool (DAX), and (3) a wide variety of encapsulated image processing pipelines called “spiders.” The VUIIS CCI medical image data storage and processing infrastructure have housed and processed nearly half-million medical image volumes with Vanderbilt Advanced Computing Center for Research and Education (ACCRE), which is the HPC facility at the Vanderbilt University. The initial deployment was natively deployed (i.e., direct installations on a bare-metal server) within the ACCRE hardware and software environments, which lead to issues of portability and sustainability. First, it could be laborious to deploy the entire VUIIS CCI medical image data storage and processing infrastructure to another HPC center with varying hardware infrastructure, library availability, and software permission policies. Second, the spiders were not developed in an isolated manner, which has led to software dependency issues during system upgrades or remote software installation. To address such issues, herein, we describe recent innovations using containerization techniques with XNAT/DAX which are used to isolate the VUIIS CCI medical image data storage and processing infrastructure from the underlying hardware and software environments. The newly presented XNAT/DAX solution has the following new features: (1) multi-level portability from system level to the application level, (2) flexible and dynamic software development and expansion, and (3) scalable spider deployment compatible with HPC clusters and local workstations.

Available only for authorised users

Izzo M: Biomedical Research and Integrated Biobanking: An Innovative Paradigm for Heterogeneous Data Management. Berlin: Springer, 2016CrossRef

Marcus DS, Olsen TR, Ramaratnam M, Buckner RL: The extensible neuroimaging archive toolkit. Neuroinformatics 5:11–33, 2007CrossRef

Marco F: XTENS—an eXTensible environment for neuroscience. Healthgrid Research, Innovation, and Business Case: Proceedings of HealthGrid 147:127, 2009

Scott A, Courtney W, Wood D, de la Garza R, Lane S, King M, Wang R, Roberts J, Turner JA, Calhoun VD: COINS: an innovative informatics and neuroimaging tool suite built for large heterogeneous datasets. Frontiers in Neuroinformatics 5, 2011

Harrigan RL, Yvernault BC, Boyd BD, Damon SM, Gibney KD, Conrad BN, Phillips NS, Rogers BP, Gao Y, Landman BA: Vanderbilt University Institute of Imaging Science Center for Computational Imaging XNAT: a multimodal data archive and processing environment. NeuroImage 124:1097–1101, 2016CrossRef

Gao Y, et al.: Integration of XNAT/PACS, DICOM, and research software for automated multi-modal image analysis. Proc. Proceedings of SPIE: City

Damon SM, et al.: Versatile big data imaging analytics with containerized high-performance computing, Baltimore, MD, USA, 2017

Damon SM, Boyd BD, Plassard AJ, Taylor W, Landman BA: DAX-the next generation: towards one million processes on commodity hardware. Proc. SPIE Medical Imaging: City

Asman AJ, Landman BA: Non-local statistical label fusion for multi-atlas segmentation. Medical Image Analysis 17:194–208, 2013CrossRef

10.

Tustison NJ, Avants BB, Cook PA, Yuanjie Zheng, Egan A, Yushkevich PA, Gee JC: N4ITK: improved N3 bias correction. IEEE Transactions on Medical Imaging 29:1310–1320, 2010CrossRef

11.

Ourselin S, Roche A, Subsol G, Pennec X, Ayache N: Reconstructing a 3D structure from serial histological sections. Image Vision Comput 19:25–31, 2001CrossRef

12.

Aljabar P, Heckemann RA, Hammers A, Hajnal JV, Rueckert D: Multi-atlas based segmentation of brain images: atlas selection and its effect on accuracy. Neuroimage 46:726–738, 2009CrossRef

13.

Huo Y, Asman AJ, Plassard AJ, Landman BA: Simultaneous total intracranial volume and posterior fossa volume estimation using multi-atlas label fusion. Human Brain Mapping 38:599–616, 2017CrossRef

14.

Xu Z, Burke RP, Lee CP, Baucom RB, Poulose BK, Abramson RG, Landman BA: Efficient multi-atlas abdominal segmentation on clinically acquired CT with SIMPLE context learning. Medical Image Analysis 24:18–27, 2015CrossRef

15.

Merkel D: Docker: lightweight linux containers for consistent development and deployment. Linux Journal 2:2014, 2014

16.

Kurtzer GM, Sochat V, Bauer MW: Singularity: scientific containers for mobility of compute. PloS One 12:e0177459, 2017CrossRef

17.

Schwartz Y, Barbot A, Thyreau B, Frouin V, Varoquaux G, Siram A, Marcus DS, Poline JB: PyXNAT: XNAT in python. Frontiers in Neuroinformatics 6, 2012

18.

Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG: Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. Journal of Biomedical Informatics 42:377–381, 2009CrossRef

19.

Cox RW et al.: A (sort of) new image data format standard: Nifti-1. Neuroimage 22:e1440, 2004

20.

Bao S, Weitendorf FD, Plassard AJ, Huo Y, Gokhale A, Landman BA: Theoretical and empirical comparison of big data image processing with apache hadoop and sun grid engine. Proc. SPIE Medical Imaging: City

21.

Gentzsch W: Sun grid engine: towards creating a compute power grid. Proc. Cluster Computing and the Grid, 2001Proceedings First IEEE/ACM International Symposium on: City

22.

Yoo AB, Jette MA, Grondona M: Slurm: simple linux utility for resource management. Proc. Workshop on Job Scheduling Strategies for Parallel Processing: City

23.

Staples G: TORQUE resource manager. Proc. Proceedings of the 2006ACM/IEEE conference on Supercomputing: City

24.

Fischl B: FreeSurfer. Neuroimage 62:774–781, 2012CrossRef

25.

Mechelli A, Price CJ, Friston KJ, Ashburner J: Voxel-based morphometry of the human brain: methods and applications. Current Medical Imaging Reviews 1:105–113, 2005CrossRef

26.

Smith SM, Jenkinson M, Woolrich MW, Beckmann CF, Behrens TEJ, Johansen-Berg H, Bannister PR, de Luca M, Drobnjak I, Flitney DE, Niazy RK, Saunders J, Vickers J, Zhang Y, de Stefano N, Brady JM, Matthews PM: Advances in functional and structural MR image analysis and implementation as FSL. Neuroimage 23:S208–S219, 2004CrossRef

27.

Jenkinson M, Beckmann CF, Behrens TE, Woolrich MW, Smith SM: Fsl. Neuroimage 62:782–790, 2012CrossRef

28.

Lauzon CB, Landman BA: Correcting power and p-value calculations for bias in diffusion tensor imaging. Magnetic Resonance Imaging 31:857–864, 2013CrossRef

29.

Lauzon CB, Asman AJ, Esparza ML, Burns SS, Fan Q, Gao Y, Anderson AW, Davis N, Cutting LE, Landman BA: Simultaneous analysis and quality assurance for diffusion tensor imaging. PloS one 8:e61737, 2013CrossRef

30.

Behrens TE, Berg HJ, Jbabdi S, Rushworth MF, Woolrich MW: Probabilistic diffusion tractography with multiple fibre orientations: what can we gain? Neuroimage 34:144–155, 2007CrossRef

31.

Smith SM, Jenkinson M, Johansen-Berg H, Rueckert D, Nichols TE, Mackay CE, Watkins KE, Ciccarelli O, Cader MZ, Matthews PM, Behrens TEJ: Tract-based spatial statistics: Voxelwise analysis of multi-subject diffusion data. Neuroimage 31:1487–1505, 2006CrossRef

32.

Yendiki A, et al.: Automated probabilistic reconstruction of white-matter pathways in health and disease using an atlas of the underlying anatomy. Frontiers in Neuroinformatics 5, 2011

33.

Parvathaneni P, et al.: Gray Matter Surface based Spatial Statistics (GS-BSS) in diffusion microstructure. Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention: City

34.

Han X, Pham DL, Tosun D, Rettmann ME, Xu C, Prince JL: CRUISE: cortical reconstruction using implicit surface evolution. NeuroImage 23:997–1012, 2004CrossRef

35.

Huo Y, Plassard AJ, Carass A, Resnick SM, Pham DL, Prince JL, Landman BA: Consistent cortical reconstruction and multi-atlas brain segmentation. NeuroImage 138:197–210, 2016CrossRef

36.

Huo Y, Carass A, Resnick SM, Pham DL, Prince JL, Landman BA: Combining multi-atlas segmentation with brain surface estimation. Proc. Proceedings of SPIE—The International Society for Optical Engineering: City

37.

Huo Y, Bao S, Parvathaneni P, Landman BA: Improved stability of whole brain surface parcellation with multi-atlas segmentation. Proc. SPIE Medical Imaging: City

38.

Lyu I, Kim SH, Bullins J, Gilmore JH, Styner MA: Novel Local Shape-Adaptive Gyrification Index with Application to Brain Development. Proc. International Conference on Medical Image Computing and Computer-Assisted Intervention: City

39.

Lyu I, Kim SH, Styner MA: Cortical surface shape assessment via sulcal/gyral curve-based gyrification index. Proc. Biomedical Imaging (ISBI), 2016 I.E. 13th International Symposium on: City

40.

Shiee N, Bazin P-L, Ozturk A, Reich DS, Calabresi PA, Pham DL: A topology-preserving approach to the segmentation of brain images with multiple sclerosis lesions. NeuroImage 49:1524–1535, 2010CrossRef

41.

Schmidt P, Gaser C, Arsic M, Buck D, Förschler A, Berthele A, Hoshi M, Ilg R, Schmid VJ, Zimmer C, Hemmer B, Mühlau M: An automated tool for detection of FLAIR-hyperintense white-matter lesions in multiple sclerosis. Neuroimage 59:3774–3783, 2012CrossRef

42.

Friedman L, Glover GH: Report on a multicenter fMRI quality assurance protocol. Journal of Magnetic Resonance Imaging 23:827–839, 2006CrossRef

43.

Huo Y, et al.: Splenomegaly segmentation using global convolutional kernels and conditional generative adversarial networks. Proc. SPIE Medical Imaging: City

44.

Smith JE, Nair R: The architecture of virtual machines. Computer 38:32–38, 2005CrossRef

45.

Baoa S, Weitendorfa FD, Plassarda AJ, Huob Y, Gokhalea A, Landmana BA: Theoretical and empirical comparison of big data image processing with apache hadoop and sun grid engine. Proc. SPIE Medical Imaging: City

46.

Bao S, et al.: A data colocation grid framework for big data medical image processing: backend design. Proc. SPIE Medical Imaging: City

47.

Bao S, Plassard AJ, Landman BA, Gokhale A: Cloud engineering principles and technology enablers for medical image processing-as-a-service. Proc. Cloud Engineering (IC2E), 2017IEEE International Conference on: City

Title: Towards Portable Large-Scale Image Processing with High-Performance Computing
Authors: Yuankai Huo
Justin Blaber
Stephen M. Damon
Brian D. Boyd
Shunxing Bao
Prasanna Parvathaneni
Camilo Bermudez Noguera
Shikha Chaganti
Vishwesh Nath
Jasmine M. Greer
Ilwoo Lyu
William R. French
Allen T. Newton
Baxter P. Rogers
Bennett A. Landman
Publication date: 01-06-2018
Publisher: Springer International Publishing
Published in: Journal of Imaging Informatics in Medicine / Issue 3/2018
Print ISSN: 2948-2925
Electronic ISSN: 2948-2933
DOI: https://doi.org/10.1007/s10278-018-0080-0

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Towards Portable Large-Scale Image Processing with High-Performance Computing

Abstract

At a glance: The ONWARDS insulin icodec trials

Springer Medicine

Abstract

Please log in to get access to this content

Other articles of this Issue 3/2018

A Platform for Innovation and Standards Evaluation: a Case Study from the OpenMRS Open-Source Radiology Information System

Learning HL7 FHIR Using the HAPI FHIR Server and Its Use in Medical Imaging with the SIIM Dataset

The Orthanc Ecosystem for Medical Imaging

DICOMweb™: Background and Application of the Web Standard for Medical Imaging

Hello World Deep Learning in Medical Imaging

Patient-Centered Radiology with FHIR: an Introduction to the Use of FHIR to Offer Radiology a Clinically Integrated Platform