Shu Liao’s research while affiliated with United Imaging and other places

Purpose Magnetic resonance imaging (MRI) refers to one of the critical image modalities for diagnosis, whereas its long acquisition time limits its application. In this study, the aim was to investigate whether deep learning–based techniques are capable of using the common information in different MRI sequences to reduce the scan time of the most time‐consuming sequences while maintaining the image quality. Method Fully sampled T1‐FLAIR, T2‐FLAIR, and T2WI brain MRI raw data originated from 217 patients and 105 healthy subjects. The T1‐FLAIR and T2‐FLAIR sequences were subsampled using Cartesian masks based on four different acceleration factors. The fully sampled T1/T2‐FLAIR images were predicted from undersampled T1/T2‐FLAIR images and T2WI images through deep learning–based reconstruction. They were qualitatively assessed by two senior radiologists in accordance with the diagnosis decision and a four‐point scale image quality score. Furthermore, the images were quantitatively assessed based on regional signal‐to‐noise ratios (SNRs) and contrast‐to‐noise ratios (CNRs). The chi‐square test was performed, where p < 0.05 indicated a difference with statistical significance. Results The diagnosis decisions from two senior radiologists remained unchanged in accordance with the accelerated and fully sampled images. There were no significant differences in the regional SNRs and CNRs of most assessed regions (p > 0.05) between the accelerated and fully sampled images. Moreover, no significant difference was identified in the image quality assessed by two senior radiologists (p > 0.05). Conclusion Deep learning–based image reconstruction is capable of significantly expediting the brain MR imaging process and producing acceptable image quality without affecting diagnosis decisions.

Fast and low-dose reconstructions of medical images are highly desired in clinical routines. We propose a hybrid deep-learning and iterative reconstruction (hybrid DL-IR) framework and apply it for fast magnetic resonance imaging (MRI), fast positron emission tomography (PET), and low-dose computed tomography (CT) image generation tasks. First, in a retrospective MRI study (6,066 cases), we demonstrate its capability of handling 3- to 10-fold under-sampled MR data, enabling organ-level coverage with only 10- to 100-s scan time; second, a low-dose CT study (142 cases) shows that our framework can successfully alleviate the noise and streak artifacts in scans performed with only 10% radiation dose (0.61 mGy); and last, a fast whole-body PET study (131 cases) allows us to faithfully reconstruct tumor-induced lesions, including small ones (<4 mm), from 2- to 4-fold-accelerated PET acquisition (30-60 s/bp). This study offers a promising avenue for accurate and high-quality image reconstruction with broad clinical value.

Background Stroke is a major disease with high morbidity and mortality worldwide. Currently, there is no quantitative method to evaluate the short-term prognosis and length of hospitalization of patients. Purpose We aimed to develop nomograms as prognosis predictors based on imaging characteristics from non-contrast computed tomography (NCCT) and CT perfusion (CTP) and clinical characteristics for predicting activity of daily living (ADL) and hospitalization time of patients with ischemic stroke. Materials and methods A total of 476 patients were enrolled in the study and divided into the training set (n = 381) and testing set (n = 95). Each of them owned NCCT and CTP images. We propose to extract imaging features representing as the Alberta stroke program early CT score (ASPECTS) values from NCCT, ischemic lesion volumes from CBF, and TMAX maps from CTP. Based on imaging features and clinical characteristics, we addressed two main issues: (1) predicting prognosis according to the Barthel index (BI)–binary logistic regression analysis was employed for feature selection, and the resulting nomogram was assessed in terms of discrimination capability, calibration, and clinical utility and (2) predicting the hospitalization time of patients–the Cox proportional hazard model was used for this purpose. After feature selection, another specific nomogram was established with calibration curves and time-dependent ROC curves for evaluation. Results In the task of predicting binary prognosis outcome, a nomogram was constructed with the area under the curve (AUC) value of 0.883 (95% CI: 0.781–0.985), the accuracy of 0.853, and F1-scores of 0.909 in the testing set. We further tried to predict discharge BI into four classes. Similar performance was achieved as an AUC of 0.890 in the testing set. In the task of predicting hospitalization time, the Cox proportional hazard model was used. The concordance index of the model was 0.700 (SE = 0.019), and AUCs for predicting discharge at a specific week were higher than 0.80, which demonstrated the superior performance of the model. Conclusion The novel non-invasive NCCT- and CTP-based nomograms could predict short-term ADL and hospitalization time of patients with ischemic stroke, thus allowing a personalized clinical outcome prediction and showing great potential in improving clinical efficiency. Summary Combining NCCT- and CTP-based nomograms could accurately predict short-term outcomes of patients with ischemic stroke, including whose discharge BI and the length of hospital stay. Key Results Using a large dataset of 1,310 patients, we show a novel nomogram with a good performance in predicting discharge BI class of patients (AUCs > 0.850). The second nomogram owns an excellent ability to predict the length of hospital stay (AUCs > 0.800).

In clinical practice, multi-modal magnetic resonance imaging (MRI) with different contrasts is usually acquired in a single study to assess different properties of the same region of interest in the human body. The whole acquisition process can be accelerated by having one or more modalities under-sampled in the ${k}$ -space. Recent research has shown that, considering the redundancy between different modalities, a target MRI modality under-sampled in the ${k}$ -space can be more efficiently reconstructed with a fully-sampled reference MRI modality. However, we find that the performance of the aforementioned multi-modal reconstruction can be negatively affected by subtle spatial misalignment between different modalities, which is actually common in clinical practice. In this paper, we improve the quality of multi-modal reconstruction by compensating for such spatial misalignment with a spatial alignment network. First, our spatial alignment network estimates the displacement between the fully-sampled reference and the under-sampled target images, and warps the reference image accordingly. Then, the aligned fully-sampled reference image joins the multi-modal reconstruction of the under-sampled target image. Also, considering the contrast difference between the target and reference images, we have designed a cross-modality-synthesis-based registration loss in combination with the reconstruction loss, to jointly train the spatial alignment network and the reconstruction network. The experiments on both clinical MRI and multi-coil ${k}$ -space raw data demonstrate the superiority and robustness of the multi-modal MRI reconstruction empowered with our spatial alignment network. Our code is publicly available at https://github.com/woxuankai/SpatialAlignmentNetwork .

Background: Multiparametric magnetic resonance imaging (mpMRI) plays an important role in the diagnosis of prostate cancer (PCa) in the current clinical setting. However, the performance of mpMRI usually varies based on the experience of the radiologists at different levels; thus, the demand for MRI interpretation warrants further analysis. In this study, we developed a deep learning (DL) model to improve PCa diagnostic ability using mpMRI and whole-mount histopathology data. Methods: A total of 739 patients, including 466 with PCa and 273 without PCa, were enrolled from January 2017 to December 2019. The mpMRI (T2 weighted imaging, diffusion weighted imaging, and apparent diffusion coefficient sequences) data were randomly divided into training ( n = 659) and validation datasets ( n = 80). According to the whole-mount histopathology, a DL model, including independent segmentation and classification networks, was developed to extract the gland and PCa area for PCa diagnosis. The area under the curve (AUC) were used to evaluate the performance of the prostate classification networks. The proposed DL model was subsequently used in clinical practice (independent test dataset; n = 200), and the PCa detective/diagnostic performance between the DL model and different level radiologists was evaluated based on the sensitivity, specificity, precision, and accuracy. Results: The AUC of the prostate classification network was 0.871 in the validation dataset, and it reached 0.797 using the DL model in the test dataset. Furthermore, the sensitivity, specificity, precision, and accuracy of the DL model for diagnosing PCa in the test dataset were 0.710, 0.690, 0.696, and 0.700, respectively. For the junior radiologist without and with DL model assistance, these values were 0.590, 0.700, 0.663, and 0.645 versus 0.790, 0.720, 0.738, and 0.755, respectively. For the senior radiologist, the values were 0.690, 0.770, 0.750, and 0.730 vs. 0.810, 0.840, 0.835, and 0.825, respectively. The diagnosis made with DL model assistance for radiologists were significantly higher than those without assistance ( P < 0.05). Conclusion: The diagnostic performance of DL model is higher than that of junior radiologists and can improve PCa diagnostic accuracy in both junior and senior radiologists.

Artificial intelligence (AI) as an emerging technology is gaining momentum in medical imaging. Recently, deep learning-based AI techniques have been actively investigated in medical imaging, and its potential applications range from data acquisition and image reconstruction to image analysis and understanding. In this review, we focus on the use of deep learning in image reconstruction for advanced medical imaging modalities including magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET). Particularly, recent deep learning-based methods for image reconstruction will be emphasized, in accordance with their methodology designs and performances in handling volumetric imaging data. It is expected that this review can help relevant researchers understand how to adapt AI for medical imaging and which advantages can be achieved with the assistance of AI.

Purpose Involuntary patient movement results in data discontinuities during computed tomography (CT) scans which lead to a serious degradation in the image quality. In this paper, we specifically address artifacts induced by patient motion during a head scan. Method Instead of trying to solve an inverse problem, we developed a motion simulation algorithm to synthesize images with motion‐induced artifacts. The artifacts induced by rotation, translation, oscillation and any possible combination are considered. Taking advantage of the powerful learning ability of neural networks, we designed a novel 3D network structure with both a large reception field and a high image resolution to map the artifact‐free images from artifact‐contaminated images. Quantitative results of the proposed method were evaluated against the results of U‐Net and proposed networks without dilation structure. Thirty sets of motion contaminated images from two hospitals were selected to do a clinical evaluation. Result Facilitating the training dataset with artifacts induced by variable motion patterns and the neural network, the artifact can be removed with good performance. Validation dataset with simulated random motion pattern showed outperformed image correction, and quantitative results showed the proposed network had the lowest normalized root‐mean‐square error, highest peak signal‐to‐noise ratio and structure similarity, indicating our network gave the best approximation of gold standard. Clinical image processing results further confirmed the effectiveness of our method. Conclusion We proposed a novel deep learning‐based algorithm to eliminate motion artifacts. The convolutional neural networks trained with synthesized image pairs achieved promising results in artifacts reduction. The corrected images increased the diagnostic confidence compared with artifacts contaminated images. We believe that the correction method can restore the ability to successfully diagnose and avoid repeated CT scans in certain clinical circumstances.

Prostate segmentation from magnetic resonance (MR) images plays an important role in prostate cancer diagnosis and treatment. Previous works typically overlooked large variations of prostate shapes, especially on the boundary area. Furthermore, the small glandular areas at ending slices also make the task very challenging. To overcome these problems, this paper presents a two-stage framework that explicitly utilizes prostate morphological representations (e.g., point, boundary) to accurately localize the prostate region with a coarse volumetric segmentation. Based on the 3D coarse outputs of the first stage, a 2D segmentation network with multi-slice association is further introduced to produce more reliable and accurate segmentation, due to large slice thickness in prostate MR images. Besides, several novel loss functions are further designed to enhance the consistency of prostate boundaries. Extensive experiments on large prostate MRI dataset show superior performance of our proposed method compared to several state-of-the-art methods.

In clinical practice, magnetic resonance imaging (MRI) with multiple contrasts is usually acquired in a single study to assess different properties of the same region of interest in human body. The whole acquisition process can be accelerated by having one or more modalities under-sampled in the k-space. Recent researches demonstrate that, considering the redundancy between different contrasts or modalities, a target MRI modality under-sampled in the k-space can be better reconstructed with the helps from a fully-sampled sequence (i.e., the reference modality). It implies that, in the same study of the same subject, multiple sequences can be utilized together toward the purpose of highly efficient multi-modal reconstruction. However, we find that multi-modal reconstruction can be negatively affected by subtle spatial misalignment between different sequences, which is actually common in clinical practice. In this paper, we integrate the spatial alignment network with reconstruction, to improve the quality of the reconstructed target modality. Specifically, the spatial alignment network estimates the spatial misalignment between the fully-sampled reference and the under-sampled target images, and warps the reference image accordingly. Then, the aligned fully-sampled reference image joins the under-sampled target image in the reconstruction network, to produce the high-quality target image. Considering the contrast difference between the target and the reference, we particularly design the cross-modality-synthesis-based registration loss, in combination with the reconstruction loss, to jointly train the spatial alignment network and the reconstruction network. Our experiments on both clinical MRI and multi-coil k-space raw data demonstrate the superiority and robustness of our spatial alignment network. Code is publicly available at https://github.com/woxuankai/SpatialAlignmentNetwork.

Magnetic Resonance Imaging (MRI) is one of the most commonly used modalities in medical imaging, and one of the main limitations is its slow scanning and reconstruction speed. Recently, deep learning used for compressed sensing (CS) methods have been proposed to accelerate the acquisition by undersampling in the K-space and reconstruct images with neural networks. However, there are still some challenges remained: First, directly training networks based on L1/L2 distance to the target fully sampled images may lead to fuzzy reconstruction images because L1/L2 loss only enforces the overall image or patch similarity, but does not consider the local details such as anatomical sharpness. Second, Generative Adversarial Networks (GAN) can partially solve this problem. The undersampling image gets the latent space through the encoder, and the image is reconstructed by the decoder based on GAN loss, but it may generate unrealistic details by lacking of constraints in K-space domain. Third, most of the networks after training are fixed and have limited adaptation capability in the inference time, and the patient-specific information cannot be effectively used. To resolve these challenges, we proposed a new compressed sensing GAN reconstruction method, and there are two main contributions: (1) We proposed a encoder-decoder structure, which guided GAN optimization strategy data-consistency in latent space to improve the reconstruction quality such as preserving more local details and improving the anatomical sharpness while constraining GAN to follow the data distribution in the K-space to prevent the unrealistic details. (2) An online update strategy is used to find the best representation in the latent space for the underlying patient, and the reconstruct result can be further improved by incorporating the patient-specific information. Extensive experimental results show the effectiveness of our method.