Next Article in Journal
Multi-Input Dual-Stream Capsule Network for Improved Lung and Colon Cancer Classification
Previous Article in Journal
A Novel Quantitative Method for Tooth Grinding Surface Assessment Using 3D Scanning
 
 
Font Type:
Arial Georgia Verdana
Font Size:
Aa Aa Aa
Line Spacing:
Column Width:
Background:
Article

Feasibility and Implementation of a Deep Learning MR Reconstruction for TSE Sequences in Musculoskeletal Imaging

1
Department of Diagnostic and Interventional Radiology, Eberhard Karls University Tuebingen, Hoppe-Seyler-Strasse 3, 72076 Tuebingen, Germany
2
MR Applications Predevelopment, Siemens Healthcare GmbH, Allee am Roethelheimpark 2, 91052 Erlangen, Germany
3
Digital Technology & Innovation, Siemens Medical Solutions USA, Inc., Princeton, NJ 08540, USA
4
Department of Neuroradiology, University Medical Center, 55131 Mainz, Germany
*
Author to whom correspondence should be addressed.
Diagnostics 2021, 11(8), 1484; https://doi.org/10.3390/diagnostics11081484
Submission received: 15 June 2021 / Revised: 23 July 2021 / Accepted: 31 July 2021 / Published: 16 August 2021
(This article belongs to the Section Medical Imaging and Theranostics)

Abstract

:
Magnetic Resonance Imaging (MRI) of the musculoskeletal system is one of the most common examinations in clinical routine. The application of Deep Learning (DL) reconstruction for MRI is increasingly gaining attention due to its potential to improve the image quality and reduce the acquisition time simultaneously. However, the technology has not yet been implemented in clinical routine for turbo spin echo (TSE) sequences in musculoskeletal imaging. The aim of this study was therefore to assess the technical feasibility and evaluate the image quality. Sixty examinations of knee, hip, ankle, shoulder, hand, and lumbar spine in healthy volunteers at 3 T were included in this prospective, internal-review-board-approved study. Conventional (TSES) and DL-based TSE sequences (TSEDL) were compared regarding image quality, anatomical structures, and diagnostic confidence. Overall image quality was rated to be excellent, with a significant improvement in edge sharpness and reduced noise compared to TSES (p < 0.001). No difference was found concerning the extent of artifacts, the delineation of anatomical structures, and the diagnostic confidence comparing TSES and TSEDL (p > 0.05). Therefore, DL image reconstruction for TSE sequences in MSK imaging is feasible, enabling a remarkable time saving (up to 75%), whilst maintaining excellent image quality and diagnostic confidence.

1. Introduction

Magnetic Resonance Imaging (MRI) has become an important diagnostic tool for the evaluation of the musculoskeletal (MSK) system. Due to its high clinical impact, MSK imaging is one of the most common performed examinations in clinical routine. One big disadvantage of MRI is the long examination time, which is not tolerated by a substantial proportion of patients and, on the other hand, comes along with other downsides such as decreased image quality due to motion artifacts, increased costs and reduced patient throughput [1].
The acquisition time of MRI is primarily determined by the achievable sampling rate for a given contrast and image quality, which, in turn, is determined by the number of samples needed for the image reconstruction for a given size and resolution. Over the past decades, different acceleration strategies have been proposed and established such as parallel imaging (PI) and Compressed Sensing (CS) [2,3,4,5,6,7,8,9,10,11,12,13,14]. These techniques acquire a reduced amount of k-space data with an array of receiver coils and afterwards reconstruct images from the acquired, under sampled data [2,14]. Artifacts due to residual aliasing, noise, or stair casing and blurring can impair achievable image quality of those acceleration strategies.
Recently, a new acceleration strategy gained attention: Deep Learning (DL) reconstruction, as discussed here and detailed below, may solve nonlinear and ill-posed reconstruction problems efficiently [15,16,17]. Instead of ad hoc regularization that enforces sparsity, the regularization is trained on representative images. This procedure also allows the regularization to generalize for different sampling patterns, acceleration factors, and artifact behavior. With DL reconstruction providing higher signal-to-noise ratio (SNR) and allowing for higher acceleration with conventional sampling patterns, the data acquisition can be tailored. For TSE acquisitions, modifying the original acquisition’s image contrast is usually not a desirable goal for tailored acquisitions. Rather, DL reconstruction in MRI can be used to improve on a combination of image resolution, acquisition time, and SNR while maintaining the original contrast [18,19].
Most prominently, the DL networks are trained in a supervised manner, i.e., representative fully sampled data with known results for the given application—so-called ground truth data—are available, which allow retrospective subsampling and training of the architectures. Testing is performed on a separate set of samples (not seen in training) with subsampled datasets. While training can be computationally intensive and takes rather long, it can be performed offline. The trained architecture can then be used in testing to reconstruct an aliasing and noise-free image within a few seconds and with greatly reduced computational demand [20,21,22].
DL reconstruction has recently been shown to potentially accelerate image acquisition in knee MRI [23]; however, the majority of existing literature on this topic focusses on offline solutions simulating accelerated image acquisition. Realistic performance assessment should be performed on prospectively subsampled datasets. To evaluate the clinical impact of the DL reconstruction for MSK imaging, an implementation in clinical settings using a prospective acquisition of accelerated data, as well as an extension to various joints, is needed.
Therefore, the aim of this study was to first assess the technical feasibility of the DL reconstruction, and to compare the obtainable image quality for accelerated DL-based TSE sequences (TSEDL) in comparison to conventional TSE sequences (TSES) for the most common MSK examinations in healthy volunteers. In addition, technical guidance is provided for prospective implementation of DL image reconstruction into clinical workflow.

2. Materials and Methods

2.1. Study Design

Institutional review board approval was obtained for this prospective, monocentric study (Eberhard Karls University Tuebingen, project identification code: 055/2017BO2, 2 December 2020). All volunteers gave informed consent to participate in this study and all study procedures were conducted in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments.
A total of 60 volunteer-MRI-examinations, ten from each joint (ankle, hand, hip, knee, lumbar spine, and shoulder) were included in this study.
A short protocol comprising PD-weighted TSE sequences with fat suppression (FS) of the two clinically most commonly used orientations for ankle (axial/sagittal), hand (axial/coronal), hip (axial/coronal), knee (coronal/sagittal) and shoulder (axial/coronal), as well as T2-weighted and T1-weighted TSE sequences with FS for the lumbar spine in sagittal orientation were acquired. Accelerated TSEDL sequences were acquired along with standard TSES sequences at clinical 3 T MRI scanner (MAGNETOM Prismafit, Siemens Healthcare, Erlangen, Germany) using clinical surface coils respectively for each joint. An overview of the acquisition parameters is displayed in Table 1.

2.2. Deep Learning Image Reconstruction

A conventional under sampling pattern, as known from parallel imaging, is used. As shown in earlier works [24,25], these provide the same performance when reconstructed with DL-based methods as incoherent sampling patterns favored by CS. They have the important advantages of being clinically established and are highly flexible regarding adaptations of resolution and signal evolution during sampling. Furthermore, their artifact behavior with regard to motion, reduced field of view, and aliasing is well known and potentially even improved by the DL-based reconstruction. In addition, as a part of conventional under sampling patterns, a fraction of the k-space’s periphery is often not acquired in order to reduce acquisition time. This effectively reduces the resolution in the phase-encoding direction and is referred to as phase resolution. It describes the fraction of acquired data in the phase-encoding direction in percent neglecting the regular parallel imaging type of under sampling. For illustration, an exemplary sampling pattern of acceleration factor 2 and a phase resolution <100% is shown in Figure 1.
Besides the data acquisition for the actual image data, calibration data for the coil-sensitivity estimation needs to be acquired. For the TSE sequence, a fully sampled region around the k-space center (typically about 16 phase-encoding lines) is acquired as part of the imaging scan. These data are used for both the image reconstruction and the estimation of coil sensitivity maps.
For all discussed applications, the prototype image reconstruction comprises a fixed iterative reconstruction scheme or variational network [25,26]. The deep neural network model leverages the combined benefits of physical models of MR imaging (SENSE model with coil sensitivity maps [12] with bias field corrections) and data-driven models. The fixed unrolled algorithm for accelerated MR image reconstruction consists of multiple cascades (6 pre-cascades, 12 cascades, and 1 post-cascade), each made up from a data consistency using a trainable Nesterov Momentum followed by a CNN-based regularization. The regularization model’s architecture is based on a novel hierarchical design of an iterative network that repeatedly decreases and increases the resolution of the feature maps, allowing for a more memory-efficient model than conventional CNNs. The CNN module is a “Deep, Iterative, Hierarchical Network” that extends the Down-Up network [27] with a hierarchical block design, facilitating memory efficiency over a standard U-Net [28]. In addition to the input under sampled k-space data, coil-sensitivity maps are also provided, which are estimated from the calibration data as a non-DL based pre-processing step. In addition, a bias field is extracted from a separate adjustment acquisition for image homogenization. The architecture of the reconstruction network is illustrated in Figure 2. During the training phase, the bias field is inserted into the image reconstruction as a final correction step.
For the image reconstruction, the k-space data, the bias field and coil-sensitivity maps are inserted into the variational network. Compared to previous works and the previously described cascades, the variational network also utilizes two additional types of cascades, namely, pre- and post-cascades. Like regular cascades, pre-cascades employ trainable extrapolation; however, no regularization is applied, allowing the network to focus on the parallel imaging part of the reconstruction problem. Such design is motivated by the empirical finding that initial steps in the variational network focus on the signal recovery of missing data near the k-space center. This approach supports acquisitions without integrated calibration and flexible k-space sampling. Finally, post-cascades employing non-trainable extrapolation are also utilized for further guarantees on the data consistency, which minimizes the risk of hallucination when adversarial training is applied. The network is first trained to minimize the combined L1 and a multi-scale version of the structural similarity (SSIM) content losses between network prediction and ground truth images [29].
The reconstruction was trained on volunteer acquisitions using conventional TSE protocols independently of the data acquired in this study. About 10,000 slices were acquired on volunteers using clinical 1.5 T and 3 T scanners (MAGNETOM scanners, Siemens Healthcare, Erlangen, Germany). Fully sampled acquisitions with high resolution were performed in head, pelvis, and knee using representative contrasts for the respective body regions. The training data therefore included a wide range of image contrasts, orientations, body regions, and resolutions. The input to the reconstruction network was retrospectively under sampled to an acceleration factor of 4. The training was implemented in PyTorch and performed on a GPU cluster NVIDIA Tesla V100 (32 GB of memory) GPU.

2.3. Implementation of DL Image Reconstruction in Clinical Workflow

For deployment in the scanner reconstruction pipeline, the obtained network was converted to a C++ implemented inference framework. For the CPU-only reconstruction on a clinical MRI scanner, inference needed about 4 s per slice for the used protocol settings. The reconstruction was triggered after the end of the acquisition, which resulted in a perceived reconstruction time of 2–3 min including additional pre- and post-processing. GPU-based reconstruction can bring the duration down to the order of 20 s for a complete dataset but was not available in the local setting of this study.

2.4. Image Analysis

Image analysis was performed by two radiologists with three and nine years of experience. Both readers independently analyzed both TSES and TSEDL datasets by using a random order. The radiologists assessed qualitative image parameters including overall image quality, diagnostic confidence, edge sharpness, contrast resolution, as well as the extent of noise and artifacts using an ordinal 4-point Likert scale (1 = non-diagnostic, with major streak artifacts; 2 = non-diagnostic, moderate artifacts with low image quality; 3 = minor artifacts with good image quality; 4 = no artifacts with excellent image quality). Complementary, the overall clarity of anatomical structures, the assessment of bone and articular cartilage, as well as the delineation of ligaments and tendons were assessed as well on a 4-point Likert scale (4 = excellent clarity of the anatomical structures/assessment of bone and articular cartilage; 3 = good; 2 = acceptable; 1 = poor). Furthermore, the subjective image impression was evaluated by both readers (1 = very unrealistic, 2 = unrealistic, 3 = realistic, and 4 = very realistic). Reading scores were considered sufficient when reaching ≥3.
Image analysis was performed on a PACS workstation (GE Healthcare Centricity™ PACS RA1000, Milwaukee, WI, USA).

2.5. Statistical Analysis

Statistical analyses were performed using SPSS version 26 (IBM Corp, Armonk, NY, USA). Besides descriptive statistics, compromising median and interquartile range (IQR), reading scores of the qualitative image analysis of the TSE sequences were compared using a Wilcoxon signed-rank test for paired data of ordinal structure. Significance was assumed at a level of p < 0.05.
Inter-rater agreement was calculated through Cohen’s kappa. Kappa values were interpreted as follows: 0–0.20 = poor agreement, 0.21–0.40 = fair agreement, 0.41–0.60 = moderate agreement, 0.61–0.80 = substantial agreement, 0.81–1 = (almost) perfect agreement.

3. Results

The aim of this work was to assess the technical feasibility of the DL image reconstruction for TSE sequences in MSK imaging. In total, 60 examinations in healthy volunteers (37 males, age: 26 ± 7 (20–55) years) were included in this prospective study, ten from each MSK region (knee, ankle, hip, shoulder, hand, lumbar spine), as shown in Table 2.
All TSEDL were successfully acquired for all regions. Fat suppression could be applied successfully for TSEDL. TSEDL allowed a remarkable time saving by at least 21% and up to 75%; for instance, T1-weighted TSE in sagittal orientation for lumbar spine required an acquisition time of 2:45 min compared to T1-weighted TSEDL with an acquisition time of 0:58 min, or TSES of the hand in coronal orientation required 2:23 min compared to 0:36 min for TSEDL (see Table 1).
As inter-reader agreement resulted in a Cohen’s kappa of 0.634 to 0.923, the results of the more experienced reader 1 are described in the following.

3.1. Assessment of Image Quality

Overall image quality was rated to be excellent for TSEDL (median 4, IQR 4−4), significantly higher compared to TSES (median 4, IQR 4−4, p = 0.002) with significantly lower levels of noise (TSES: median 4, IQR 3−4; TSEDL: median 4, IQR 4−4; p < 0.001) and improved edge sharpness (TSES: median 4, IQR 3−4; TSEDL: median 4, IQR 4−4; p = 0.008) in TSEDL. No difference was found for contrast resolution and the extent of artifacts, which were both rated as excellent in TSES (median 4, IQR 4−4) and TSEDL (median 4, IQR 4−4, p > 0.05). Image examples are shown in Figure 3, Figure 4, Figure 5, Figure 6, Figure 7 and Figure 8. To highlight the image quality difference of the DL-reconstruction compared to standard reconstruction, we exported raw data of TSEDL of a single volunteer and reconstructed the accelerated TSEDL raw data with the standard reconstruction method (GRAPPA) as used in TSES (see Figure 9 and Figure 10). To illustrate the effect of the DL reconstruction on the extent of noise, raw data of a single volunteer were exported, and exemplary SNR-maps were determined offline using a pseudo-replica method [30]; see Figure 11.

3.2. Assessment of Anatomical Structures

The overall clarity of anatomic structures was rated excellent for TSEDL (median 4, IQR 4−4) and comparable to TSES (p > 0.05). Furthermore, the assessment of the articular cartilage, the delineation of ligaments and tendons were all rated excellent and to be comparable between the two sequences (median 4, IQR 4−4, p > 0.05). The assessment of the bone was rated to be lower in TSEDL (median 4, IQR 4−4) compared to TSES (median 4, IQR 4−4). There was no significant difference regarding the diagnostic confidence between both sequences (median 4, IQR 4−4, p > 0.05).
Subjective image impression was rated to be more realistic in TSES (median 4, IQR 4−4) compared to TSEDL (median 4, IQR 3−4, p < 0.001). An overview of all results is displayed in Table 3.

4. Discussion

This study investigated the technical feasibility and impact of a DL image reconstruction for TSE sequences in MSK MRI in a prospective volunteer study compared to standard TSE imaging. The results demonstrate that TSE with DL image reconstruction is feasible, providing even higher image quality as well as an improvement in edge sharpness, and concerning the evaluation of MSK-specific items, an equivalent assessment of the articular cartilage, correspondingly equal delineation of ligaments and tendons. Therefore, TSE with DL image reconstruction maintains equal diagnostic confidence compared to conventional TSE.
The fact that DL-based reconstructed images exhibit even lower noise levels than corresponding fully sampled conventional acquisition techniques, and may therefore look artificial to experienced radiologists, was already discussed in prior studies [15,23] and is in line with our findings: TSES is rated to have a more realistic image impression compared to TSEDL. The low extent of noise may also be the cause of the lower ratings of the assessment of the bone, due to almost noise-free image impression, in which very small structures disappear. Nonetheless, this is only an impairment of the image impression and there is no impact on the diagnostic accuracy since TSEDL exhibited excellent diagnostic confidence.
Another aim of the study was to accelerate TSE sequence acquisition by the incorporation of DL image reconstruction. DL allowed for an acquisition time reduction of up to 50% while maintaining excellent image quality and diagnostic confidence. DL therefore seems to allow for higher accelerations than prior acceleration techniques. Prior to DL image reconstruction, high acceleration levels beyond the Nyquist–Shannon sampling limit could be obtained by CS. In fact, if images can be sparsely represented in some transform domain, then a random and incoherent sub-Nyquist sampling together with an appropriate nonlinear iterative image reconstruction allows aliasing-free recovery from incompletely sampled k-space data [14]. CS employs iterative reconstruction algorithms that use a priori fixed sparsity-promoting transformation. Furthermore, the a priori assumption on sparsity with application-specific regularization weighting can, if not chosen appropriately, result in residual aliasing (under-regularized) or stair casing and blurring (over-regularized). Depending on the imaging application and sequence, a sampling trajectory that follows a desired random distribution can be challenging to implement without introducing other artifacts, e.g., by eddy currents due to strong switching gradients. Overall, these factors can impair achievable image quality and/or limit the achievable acceleration. The implementation of DL can overcome this drawback and enables the acceleration of MR acquisition without impairing image quality.
Current Deep Learning-based image reconstruction uses supervised learning techniques with convolutional neural networks (CNNs) [24,31]. DL networks have been proposed that operate on parallel imaging (PI)-accelerated acquisitions [25,26,32] and on CS-accelerated acquisitions [33]. The proposed methods primarily differ in the way the DL network is applied in the image reconstruction and how data consistency is enforced between reconstructed images and acquired data: (i) the network acts as trainable denoiser without explicit data consistency inside the architecture during training, but handling it in outer optimization schemes (plug-and-play denoisers), (ii) physics-based reconstructions that incorporate data consistency during training. Networks furthermore differ in the chosen architectures (VN, UNet, cascaded CNN, etc.), raw k-space or noisy/aliased image input, the input dimensionality (2D, 3D, 2D+time, 3D+time, etc.), single- or multi-parametric input, complex- or real-valued processing of the complex-valued data, and single-coil (coil-combined) or multi-coil processing [20,21,25,26,33,34,35,36,37,38,39,40,41,42,43]. To date, there are only a small number of published studies using DL methods on k-space data for MR image reconstruction in a clinical setting. Although the results are promising, routine application in clinical practice remains rare. As we successfully implemented DL reconstruction in imaging of different body regions, this report is intended to motivate radiologists to establish new AI techniques in everyday clinical practice to further accelerate MR imaging and gain access to MRI for more patients. Prolonged scanning duration can cause patient inconvenience and also limits the availability of MRI, which makes the single examination slots expansive. Drastic acceleration and cut-down of examination times might display one piece of the big puzzle to enhance health care and balance the weight of supply and demand. One of the most challenging tasks in medicine has always been finding a compromise between best medical care and best economical outcome. As most healthcare systems worldwide are insurance-based using different architectures (mandatory versus voluntary; public health insurance versus private sector), all systems face the same problem: shortage of money versus increased demand due to the development of new expensive therapies, increase in expectancy of life, and new diagnostic possibilities. Reduction of TA might allow for a higher number of examinations per day.
This study has limitations. First, we included a small amount of image data acquired in healthy volunteers. Second, we just investigated the TSEDL at 3 T; further studies are needed to evaluate its feasibility and diagnostic confidence at 1.5 T. This impacts the generalizability of our findings. However, this report primarily aims to describe the implementation of DL-based reconstruction and to assess its technical feasibility rather than to systematically and comprehensively evaluate the resulting image quality in routine clinical practice. For image quality assessment, body-region-focused clinical studies at 1.5 and 3 T are still required. A further limitation is the fact that DL-based reconstruction was not applied on 3D MR sequences yet. DL-based reconstruction algorithms for 3D sequences are still being developed by our team.
To conclude, DL image reconstruction can be implemented in clinical workflow and enables accelerated image acquisition, allowing a remarkable time saving while maintaining excellent image quality for TSE sequences.

Author Contributions

J.H.: Report conception and design. Data acquisition, analysis and interpretation. Drafting and revising the manuscript; G.K.: Report conception and design. Revising the manuscript; D.N.: Report design. Data analysis and interpretation. Revising the manuscript; M.M.: Report conception and design. Revising the manuscript; M.N.: Report conception and design. Revising the manuscript; S.G.: Report design. Data interpretation. Revising the manuscript; T.K.: Report design. Drafting and revising the manuscript; A.E.O.: Report conception and design. Data acquisition, analysis and interpretation. Revising the manuscript. All authors have read and agreed to the published version of the manuscript.

Funding

Support by an institutional research grant by Siemens Healthcare GmbH, number: C00235666.

Institutional Review Board Statement

Institutional review board approval was obtained for this prospective, monocentric study. All study procedures were conducted in accordance with the ethical standards as laid down in the 1964 Declaration of Helsinki and its later amendments. The institutional review board of our university (Ethik-Kommission, Karl Jaschonek, Medizinische Fakultät, Universitätsklinikum Tübingen) approved the study under the protocol code: 055/2017B02, 2 December 2020.

Informed Consent Statement

Informed consent was obtained from all subjects involved in the study.

Data Availability Statement

The datasets generated and analyzed during the current study are not publicly available to ensure data privacy protection but are available from the corresponding author on reasonable request.

Acknowledgments

The authors would like to thank Matthias Kündel and Andreas Lingg for their work on the data acquisition.

Conflicts of Interest

The authors declare no conflict of interest.

References

  1. Vanderby, S.; Badea, A.; Sánchez, J.N.P.; Kalra, N.; Babyn, P. Variations in Magnetic Resonance Imaging Provision and Processes among Canadian Academic Centres. Can. Assoc. Radiol. J. 2017, 68, 56–65. [Google Scholar] [CrossRef] [PubMed]
  2. Deshmane, A.; Gulani, V.; Griswold, M.A.; Seiberlich, N. Parallel mr imaging. J. Magn. Reson. Imaging 2012, 36, 55–72. [Google Scholar] [CrossRef] [Green Version]
  3. Gersing, A.S.; Bodden, J.; Neumann, J.; Diefenbach, M.N.; Kronthaler, S.; Pfeiffer, D.; Knebel, C.; Baum, T.; Schwaiger, B.J.; Hock, A.; et al. Accelerating anatomical 2D turbo spin echo imaging of the ankle using compressed sensing. Eur. J. Radiol. 2019, 118, 277–284. [Google Scholar] [CrossRef]
  4. Kreitner, K.-F.; Romaneehsen, B.; Krummenauer, F.; Oberholzer, K.; Müller, L.P.; Düber, C. Fast magnetic resonance imaging of the knee using a parallel acquisition technique (mSENSE): A prospective performance evaluation. Eur. Radiol. 2006, 16, 1659–1666. [Google Scholar] [CrossRef] [PubMed]
  5. Zuo, J.; Li, X.; Be, S.B.; Han, E.; Majumdar, S. Parallel imaging of knee cartilage at 3 Tesla. J. Magn. Reson. Imaging 2007, 26, 1001–1009. [Google Scholar] [CrossRef] [PubMed]
  6. Iuga, A.-I.; Abdullayev, N.; Weiss, K.; Haneder, S.; Brüggemann-Bratke, L.; Maintz, D.; Rau, R.; Bratke, G. Accelerated MRI of the knee. Quality and efficiency of compressed sensing. Eur. J. Radiol. 2020, 132, 109273. [Google Scholar] [CrossRef] [PubMed]
  7. Jaspan, O.N.; Fleysher, R.; Lipton, M.L. Compressed sensing MRI: A review of the clinical literature. Br. J. Radiol. 2015, 88, 20150487. [Google Scholar] [CrossRef]
  8. Matcuk, G.R.; Gross, J.S.; Fields, B.; Cen, S. Compressed Sensing MR Imaging (CS-MRI) of the Knee: Assessment of Quality, Inter-reader Agreement, and Acquisition Time. Magn. Reson. Med. Sci. 2020, 19, 254–258. [Google Scholar] [CrossRef] [Green Version]
  9. Zibetti, M.V.W.; Baboli, R.; Chang, G.; Otazo, R.; Regatte, R. Rapid compositional mapping of knee cartilage with compressed sensing MRI. J. Magn. Reson. Imaging 2018, 48, 1185–1198. [Google Scholar] [CrossRef]
  10. Jang, W.; Song, J.; Kim, S.; Yang, J. Comparison of Compressed Sensing and Gradient and Spin-Echo in Breath-Hold 3D MR Cholangiopancreatography: Qualitative and Quantitative Analysis. Diagnostics 2021, 11, 634. [Google Scholar] [CrossRef]
  11. Sodickson, D.; Manning, W.J. Simultaneous acquisition of spatial harmonics (SMASH): Fast imaging with radiofrequency coil arrays. Magn. Reson. Med. 1997, 38, 591–603. [Google Scholar] [CrossRef]
  12. Pruessmann, K.P.; Weiger, M.; Scheidegger, M.B.; Boesiger, P. Sense: Sensitivity encoding for fast mri. Magn. Reson. Med. Off. J. Int. Soc. Magn. Reson. Med. 1999, 42, 952–962. [Google Scholar] [CrossRef]
  13. Griswold, M.A.; Jakob, P.M.; Heidemann, R.M.; Nittka, M.; Jellus, V.; Wang, J.; Kiefer, B.; Haase, A. Generalized autocalibrating partially parallel acquisitions (grappa). Magn. Reson. Med. Off. J. Int. Soc. Magn. Reson. Med. 2002, 47, 1202–1210. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  14. Lustig, M.; Donoho, D.; Pauly, J.M. Sparse MRI: The application of compressed sensing for rapid MR imaging. Magn. Reson. Med. 2007, 58, 1182–1195. [Google Scholar] [CrossRef] [PubMed]
  15. Knoll, F.; Hammernik, K.; Zhang, C.; Moeller, S.; Pock, T.; Sodickson, D.; Akcakaya, M. Deep-Learning Methods for Parallel Magnetic Resonance Imaging Reconstruction: A Survey of the Current Approaches, Trends, and Issues. IEEE Signal Process. Mag. 2020, 37, 128–140. [Google Scholar] [CrossRef]
  16. Lin, D.; Johnson, P.M.; Knoll, F.; Lui, Y.W. Artificial Intelligence for MR Image Reconstruction: An Overview for Clinicians. J. Magn. Reson. Imaging 2020, 53, 1015–1028. [Google Scholar] [CrossRef]
  17. Hyun, C.M.; Kim, H.P.; Lee, S.M.; Lee, S.; Seo, J.K. Deep learning for undersampled mri reconstruction. Phys. Med. Biol. 2018, 63, 135007. [Google Scholar] [CrossRef] [PubMed]
  18. Arel, I.; Rose, D.C.; Karnowski, T.P. Deep Machine Learning—A New Frontier in Artificial Intelligence Research [Research Frontier]. IEEE Comput. Intell. Mag. 2010, 5, 13–18. [Google Scholar] [CrossRef]
  19. Wang, G. A Perspective on Deep Imaging. IEEE Access 2016, 4, 8914–8924. [Google Scholar] [CrossRef]
  20. Mardani, M.; Gong, E.; Cheng, J.Y.; Vasanawala, S.S.; Zaharchuk, G.; Xing, L.; Pauly, J.M. Deep Generative Adversarial Neural Networks for Compressive Sensing MRI. IEEE Trans. Med. Imaging 2018, 38, 167–179. [Google Scholar] [CrossRef]
  21. Chen, F.; Taviani, V.; Malkiel, I.; Cheng, J.Y.; Tamir, J.I.; Shaikh, J.; Chang, S.T.; Hardy, C.J.; Pauly, J.M.; Vasanawala, S.S. Variable-Density Single-Shot Fast Spin-Echo MRI with Deep Learning Reconstruction by Using Variational Networks. Radiology 2018, 289, 366–373. [Google Scholar] [CrossRef] [Green Version]
  22. Lundervold, A.; Lundervold, A. An overview of deep learning in medical imaging focusing on MRI. Zeitschrift für Medizinische Physik 2019, 29, 102–127. [Google Scholar] [CrossRef]
  23. Recht, M.P.; Zbontar, J.; Sodickson, D.K.; Knoll, F.; Yakubova, N.; Sriram, A.; Murrell, T.; Defazio, A.; Rabbat, M.; Rybak, L.; et al. Using Deep Learning to Accelerate Knee MRI at 3 T: Results of an Interchangeability Study. Am. J. Roentgenol. 2020, 215, 1421–1429. [Google Scholar] [CrossRef] [PubMed]
  24. Knoll, F.; Hammernik, K.; Kobler, E.; Pock, T.; Recht, M.P.; Sodickson, D. Assessment of the generalization of learned image reconstruction and the potential for transfer learning. Magn. Reson. Med. 2018, 81, 116–128. [Google Scholar] [CrossRef] [PubMed]
  25. Hammernik, K.; Klatzer, T.; Kobler, E.; Recht, M.P.; Sodickson, D.; Pock, T.; Knoll, F. Learning a variational network for reconstruction of accelerated MRI data. Magn. Reson. Med. 2017, 79, 3055–3071. [Google Scholar] [CrossRef]
  26. Schlemper, J.; Caballero, J.; Hajnal, J.; Price, A.; Rueckert, D. A Deep Cascade of Convolutional Neural Networks for Dynamic MR Image Reconstruction. IEEE Trans. Med. Imaging 2017, 37, 491–503. [Google Scholar] [CrossRef] [Green Version]
  27. Yu, S.; Park, B.; Jeong, J. Deep iterative down-up cnn for image denoising. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops, Long Beach, CA, USA, 6–20 June 2019. [Google Scholar]
  28. Ronneberger, O.; Fischer, P.; Brox, T. U-net: Convolutional networks for biomedical image segmentation. In Proceedings of the Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, Munich, Germany, 5–9 October 2015; Springer: Berlin/Heidelberg, Germany, 2015; pp. 234–241. [Google Scholar]
  29. Wang, Z.; Simoncelli, E.P.; Bovik, A.C. Multiscale structural similarity for image quality assessment. In Proceedings of the Thirty-Seventh Asilomar Conference on Signals, Systems & Computers, Pacific Grove, CA, USA, 9–12 November 2003; pp. 1398–1402. [Google Scholar]
  30. Robson, P.M.; Grant, A.K.; Madhuranthakam, A.J.; Lattanzi, R.; Sodickson, D.; McKenzie, C. Comprehensive quantification of signal-to-noise ratio andg-factor for image-based andk-space-based parallel imaging reconstructions. Magn. Reson. Med. 2008, 60, 895–907. [Google Scholar] [CrossRef] [Green Version]
  31. LeCun, Y.; Bengio, Y.; Hinton, G. Deep learning. Nat. Cell Biol. 2015, 521, 436–444. [Google Scholar] [CrossRef] [PubMed]
  32. Aggarwal, H.K.; Mani, M.P.; Jacob, M. MoDL: Model-Based Deep Learning Architecture for Inverse Problems. IEEE Trans. Med. Imaging 2018, 38, 394–405. [Google Scholar] [CrossRef] [PubMed]
  33. Küstner, T.; Fuin, N.; Hammernik, K.; Bustin, A.; Qi, H.; Hajhosseiny, R.; Masci, P.G.; Neji, R.; Rueckert, D.; Botnar, R.M.; et al. CINENet: Deep learning-based 3D cardiac CINE MRI reconstruction with multi-coil complex-valued 4D spatio-temporal convolutions. Sci. Rep. 2020, 10, 13710. [Google Scholar] [CrossRef] [PubMed]
  34. Eo, T.; Jun, Y.; Kim, T.; Jang, J.; Lee, H.; Hwang, D. KIKI -net: Cross-domain convolutional neural networks for reconstructing undersampled magnetic resonance images. Magn. Reson. Med. 2018, 80, 2188–2201. [Google Scholar] [CrossRef]
  35. Wang, S.; Su, Z.; Ying, L.; Peng, X.; Zhu, S.; Liang, F.; Feng, D.; Liang, D. Accelerating magnetic resonance imaging via deep learning. In Proceedings of the 2016 IEEE 13th International Symposium on Biomedical Imaging (ISBI), Prague, Czech Republic, 13–16 April 2016; pp. 514–517. [Google Scholar] [CrossRef]
  36. Han, Y.; Sunwoo, L.; Ye, J.C. K-space deep learning for accelerated MRI. IEEE Trans. Med. Imaging 2019, 39, 377–386. [Google Scholar] [CrossRef] [Green Version]
  37. Lee, D.; Yoo, J.; Tak, S.; Ye, J.C. Deep Residual Learning for Accelerated MRI Using Magnitude and Phase Networks. IEEE Trans. Biomed. Eng. 2018, 65, 1985–1995. [Google Scholar] [CrossRef] [Green Version]
  38. Qin, C.; Schlemper, J.; Caballero, J.; Price, A.N.; Hajnal, J.V.; Rueckert, D. Convolutional Recurrent Neural Networks for Dynamic MR Image Reconstruction. IEEE Trans. Med. Imaging 2018, 38, 280–290. [Google Scholar] [CrossRef] [Green Version]
  39. Yang, G.; Yu, S.; Dong, H.; Slabaugh, G.G.; Dragotti, P.L.; Ye, X.; Liu, F.; Arridge, S.R.; Keegan, J.; Guo, Y.; et al. DAGAN: Deep De-Aliasing Generative Adversarial Networks for Fast Compressed Sensing MRI Reconstruction. IEEE Trans. Med. Imaging 2017, 37, 1310–1321. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  40. Zhu, B.; Liu, J.Z.; Cauley, S.F.; Rosen, B.R.; Rosen, M. Image reconstruction by domain-transform manifold learning. Nat. Cell Biol. 2018, 555, 487–492. [Google Scholar] [CrossRef] [PubMed] [Green Version]
  41. Sandino, C.M.; Lai, P.; Vasanawala, S.S.; Cheng, J.Y. Accelerating cardiac cine MRI using a deep learning-based ESPIRiT reconstruction. Magn. Reson. Med. 2020, 85, 152–167. [Google Scholar] [CrossRef] [PubMed]
  42. Hosseini, S.A.H.; Zhang, C.; Weingärtner, S.; Moeller, S.; Stuber, M.; Ugurbil, K.; Akçakaya, M. Accelerated coronary MRI with sRAKI: A database-free self-consistent neural network k-space reconstruction for arbitrary undersampling. PLoS ONE 2020, 15, e0229418. [Google Scholar] [CrossRef] [Green Version]
  43. Fuin, N.; Bustin, A.; Küstner, T.; Oksuz, I.; Clough, J.; King, A.P.; Schnabel, J.A.; Botnar, R.M.; Prieto, C. A multi-scale variational neural network for accelerating motion-compensated whole-heart 3D coronary MR angiography. Magn. Reson. Imaging 2020, 70, 155–167. [Google Scholar] [CrossRef] [PubMed]
Figure 1. Exemplary sampling pattern used for the k-space data acquisition (red arrows point to the acquired date; black arrows point to the missing data). Along the phase encoding direction, data are under sampled by an acceleration factor R. An autocalibration region covering the center of k-space is acquired, either along with the imaging acquisition or separately with a different image contrast. A fraction of the periphery may be skipped, usually referred to as phase resolution.
Figure 1. Exemplary sampling pattern used for the k-space data acquisition (red arrows point to the acquired date; black arrows point to the missing data). Along the phase encoding direction, data are under sampled by an acceleration factor R. An autocalibration region covering the center of k-space is acquired, either along with the imaging acquisition or separately with a different image contrast. A fraction of the periphery may be skipped, usually referred to as phase resolution.
Diagnostics 11 01484 g001
Figure 2. Architecture of the fixed iterative reconstruction scheme. Pre-cascades address data consistency and generate an image similar to conventional parallel imaging, cascades use an alternating scheme between data consistency and regularization, post-cascades reemphasize consistency of a reconstructed image and acquired data. Note that only the regularization is a Convolutional Neural Network (CNN). Nevertheless, the whole architecture can be presented by a general network.
Figure 2. Architecture of the fixed iterative reconstruction scheme. Pre-cascades address data consistency and generate an image similar to conventional parallel imaging, cascades use an alternating scheme between data consistency and regularization, post-cascades reemphasize consistency of a reconstructed image and acquired data. Note that only the regularization is a Convolutional Neural Network (CNN). Nevertheless, the whole architecture can be presented by a general network.
Diagnostics 11 01484 g002
Figure 3. Example of a Deep Learning and standard PD-weighted turbo spin echo (TSE) image of the knee in sagittal orientation. Note that the extent of noise in TSEDL (left) is distinctly reduced compared to TSES (right), which comes with the fact that, at the same time, very small structures such as small bone channels disappear. The visibility of anatomic relevant structures is not impaired.
Figure 3. Example of a Deep Learning and standard PD-weighted turbo spin echo (TSE) image of the knee in sagittal orientation. Note that the extent of noise in TSEDL (left) is distinctly reduced compared to TSES (right), which comes with the fact that, at the same time, very small structures such as small bone channels disappear. The visibility of anatomic relevant structures is not impaired.
Diagnostics 11 01484 g003
Figure 4. Example of a Deep Learning (left) and standard (right) PD-weighted turbo spin echo image of the shoulder in coronal orientation.
Figure 4. Example of a Deep Learning (left) and standard (right) PD-weighted turbo spin echo image of the shoulder in coronal orientation.
Diagnostics 11 01484 g004
Figure 5. Example of a Deep Learning and standard T1- (right) and T2-weighted (left) turbo spin echo image of the lumbar spine in sagittal orientation. Note that TSEDL show lower extents of noise both in T1- and T2-weighted imaging. Nonetheless, some small structures, such as small bone canals, disappear; there is no impact on the delineation and assessment of relevant anatomical structures in both TSES and TSEDL.
Figure 5. Example of a Deep Learning and standard T1- (right) and T2-weighted (left) turbo spin echo image of the lumbar spine in sagittal orientation. Note that TSEDL show lower extents of noise both in T1- and T2-weighted imaging. Nonetheless, some small structures, such as small bone canals, disappear; there is no impact on the delineation and assessment of relevant anatomical structures in both TSES and TSEDL.
Diagnostics 11 01484 g005
Figure 6. Example of a Deep Learning (left) and standard (right) PD-weighted turbo spin echo image of the hip in axial orientation. Note that although the assessment of the bone was rated to be lower, the assessment of anatomical structures and articular cartilage, as well as the delineation of ligaments and tendons, are comparable between TSEDL and TSES.
Figure 6. Example of a Deep Learning (left) and standard (right) PD-weighted turbo spin echo image of the hip in axial orientation. Note that although the assessment of the bone was rated to be lower, the assessment of anatomical structures and articular cartilage, as well as the delineation of ligaments and tendons, are comparable between TSEDL and TSES.
Diagnostics 11 01484 g006
Figure 7. Example of a Deep Learning (left) and standard (right) PD-weighted turbo spin echo image of the ankle in axial orientation.
Figure 7. Example of a Deep Learning (left) and standard (right) PD-weighted turbo spin echo image of the ankle in axial orientation.
Diagnostics 11 01484 g007
Figure 8. Example of a Deep Learning (left) and standard (right) PD-weighted turbo spin echo image of the hand in coronal orientation.
Figure 8. Example of a Deep Learning (left) and standard (right) PD-weighted turbo spin echo image of the hand in coronal orientation.
Diagnostics 11 01484 g008
Figure 9. Comparison of TSES (A), TSEDL raw data reconstructed with a standard (GRAPPA) method (B) and TSEDL (C) of a knee in sagittal orientation. Note that the extent of noise in TSEDL with standard reconstruction (B) is increased compared to TSES (A) and TSEDL.
Figure 9. Comparison of TSES (A), TSEDL raw data reconstructed with a standard (GRAPPA) method (B) and TSEDL (C) of a knee in sagittal orientation. Note that the extent of noise in TSEDL with standard reconstruction (B) is increased compared to TSES (A) and TSEDL.
Diagnostics 11 01484 g009
Figure 10. Comparison of TSES (A), TSEDL raw data reconstructed with a standard (GRAPPA) method (B) and TSEDL (C) of a knee in coronal orientation. Note that the image quality in TSEDL (C) is increased compared to TSES (A) and TSEDL with standard reconstruction (B).
Figure 10. Comparison of TSES (A), TSEDL raw data reconstructed with a standard (GRAPPA) method (B) and TSEDL (C) of a knee in coronal orientation. Note that the image quality in TSEDL (C) is increased compared to TSES (A) and TSEDL with standard reconstruction (B).
Diagnostics 11 01484 g010
Figure 11. Exemplary visualization of signal-to-noise ratio (SNR) as SNR-maps. On the (left), results of the TSEDL dataset, and on the (right), of a TSES dataset of knee in sagittal orientation. The TSEDL dataset shows lower noise levels and an increase of SNR with a more homogeneous distribution compared to the TSES dataset. In addition, SNR levels throughout the whole image are more homogeneous.
Figure 11. Exemplary visualization of signal-to-noise ratio (SNR) as SNR-maps. On the (left), results of the TSEDL dataset, and on the (right), of a TSES dataset of knee in sagittal orientation. The TSEDL dataset shows lower noise levels and an increase of SNR with a more homogeneous distribution compared to the TSES dataset. In addition, SNR levels throughout the whole image are more homogeneous.
Diagnostics 11 01484 g011
Table 1. Acquisition parameters of TSES and TSEDL at 3 T.
Table 1. Acquisition parameters of TSES and TSEDL at 3 T.
SequenceOrientationTA, minFOV, mmVoxel Size, mmACPATTR, msTE, msFA, DegreeBandwith, Hz/PxEcho Spacing, ms
ShoulderTSESTSE PD FSaxial2:141800.6 × 0.6 × 3.012230004415018010.9
coronal2:531800.6 × 0.6 × 3.021233004215019110.6
TSEDLTSE PD FSaxial1:101800.6 × 0.6 × 3.011335204415018010.9
coronal1:091800.6 × 0.6 × 3.011330004215019110.6
KneeTSESTSE PD FScoronal3:111500.2 × 0.2 × 3.021337904415010014.6
sagittal3:111500.2 × 0.2 × 3.021337904415010014.6
TSEDLTSE PD FScoronal1:331500.5 × 0.5 × 3.011335804115012013.7
sagittal1:331500.5 × 0.5 × 3.011335804115012013.7
Lumbar spineTSEST1 TSEsagittal2:563000.9 × 0.9 × 3.01205621015018010.4
T2 TSE FSsagittal2:453000.7 × 0.7 × 3.0212604010215018911.3
TSEDLT1 TSEsagittal1:273000.9 × 0.9 × 3.01234621015018010.4
T2 TSE FSsagittal0:583000.7 × 0.7 × 3.0113447010515018910.5
HipTSESTSE PD FSaxial3:022000.3 × 0.3 × 3.011034104215010014.1
coronal2:012000.3 × 0.3 × 3.011234104215010014.1
TSEDLTSE PD FSaxial1:322000.6 × 0.6 × 3.011330694215012013.1
coronal1:332000.6 × 0.6 × 3.011330004115012013.7
AnkleTSESTSE PD FSaxial2:251500.4 × 0.4 × 3.01123340171509017.1
sagittal1:471600.2 × 0.2 × 3.011330003215010015.9
TSEDLTSE PD FSaxial1:541500.4 × 0.4 × 3.01133000171509016.9
sagittal1:451600.4 × 0.4 × 3.011330003115010015.7
HandTSESTSE PD FScoronal2:232000.5 × 0.5 × 2.021030004115012113.6
axial4:401800.5 × 0.5 × 2.022033104215012113.9
TSEDLTSE PD FScoronal0:362000.5 × 0.5 × 2.011330004415011914.7
axial1:231800.5 × 0.5 × 2.012231904215011914.1
TA indicates time of acquisition; FOV, field of view; A, Averages; C, Concatenations; PAT, Parallel Acquisition Technique; TE/TR, echo time/repetition time; FA, flip angle; TSE, turbo spin echo; PD, Proton Density; FS, fat saturation.
Table 2. Demographics.
Table 2. Demographics.
Variables
Total (male/female), n60 (37/23)
Age, mean ± SD (range), ytotal: 26 ± 7 (20–55)
knee: 25 ± 4 (20–31)
ankle: 26 ± 5 (20–35)
hip: 26 ± 5 (20–35)
shoulder: 27 ± 10 (20–55)
wrist: 27 ± 8 (20–44)
lumbar spine: 29 ± 11 (20–55)
SD indicates standard deviation; y, years; n, number.
Table 3. Image Quality, Inter-reader Agreement and Comparison of TSE and TSEDL.
Table 3. Image Quality, Inter-reader Agreement and Comparison of TSE and TSEDL.
TSESTSEDLTSES vs. TSEDL
R1
m (IQR)
R2
m (IQR)
κR1
m (IQR)
R2
m (IQR)
κR1R2
IQ4 (4−4)4 (3−4)0.6974 (4−4)4 (4−4)0.6340.0020.013
Artifacts4 (4−4)4 (4−4)0.6494 (4−4)4 (4−4)0.7000.1800.157
Edge sharpness4 (3−4)4 (3−4)0.8834 (4−4)4 (4−4)0.792<0.001<0.001
Contrast resolution4 (4−4)4 (4−4)0.7414 (4−4)4 (4−4)0.6490.2570.157
Noise4 (3−4)4 (3−4)0.8974 (4−4)4 (4−4)0.651<0.001<0.001
Clarity of anatomic structures4 (4−4)4 (4−4)0.7474 (4−4)4 (4−4)0.8890.3170.564
Bone4 (4−4)4 (4−4)0.8964 (4−4)4 (4−4)0.7410.0140.025
Articular cartilage4 (4−4)4 (4−4)0.7394 (4−4)4 (4−4)0.7730.1570.705
Ligaments4 (4−4)4 (4−4)0.7324 (4−4)4 (4−4)0.6430.7050.480
Tendons4 (4−4)4 (4−4)0.6404 (4−4)4 (4−4)0.6590.1800.083
Diagnostic confidence4 (4−4)4 (4−4)0.7734 (4−4)4 (4−4)0.6510.1020.096
Image impression4 (4−4)4 (4−4)0.8484 (3−4)4 (3−4)0.923<0.001<0.001
IQ indicates Image Quality; R, Reader; m, median; κ, Cohen’s κ (Inter-rater Reliability); IQR, Interquartile Range.
Publisher’s Note: MDPI stays neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Share and Cite

MDPI and ACS Style

Herrmann, J.; Koerzdoerfer, G.; Nickel, D.; Mostapha, M.; Nadar, M.; Gassenmaier, S.; Kuestner, T.; Othman, A.E. Feasibility and Implementation of a Deep Learning MR Reconstruction for TSE Sequences in Musculoskeletal Imaging. Diagnostics 2021, 11, 1484. https://doi.org/10.3390/diagnostics11081484

AMA Style

Herrmann J, Koerzdoerfer G, Nickel D, Mostapha M, Nadar M, Gassenmaier S, Kuestner T, Othman AE. Feasibility and Implementation of a Deep Learning MR Reconstruction for TSE Sequences in Musculoskeletal Imaging. Diagnostics. 2021; 11(8):1484. https://doi.org/10.3390/diagnostics11081484

Chicago/Turabian Style

Herrmann, Judith, Gregor Koerzdoerfer, Dominik Nickel, Mahmoud Mostapha, Mariappan Nadar, Sebastian Gassenmaier, Thomas Kuestner, and Ahmed E. Othman. 2021. "Feasibility and Implementation of a Deep Learning MR Reconstruction for TSE Sequences in Musculoskeletal Imaging" Diagnostics 11, no. 8: 1484. https://doi.org/10.3390/diagnostics11081484

Note that from the first issue of 2016, this journal uses article numbers instead of page numbers. See further details here.

Article Metrics

Back to TopTop