Pengyue Zhang’s research while affiliated with Stony Brook University and other places

We propose a segmentation method for nuclei in glioblastoma histopathologic images based on a sparse shape prior guided variational level set framework. By spectral clustering and sparse coding, a set of shape priors is exploited to accommodate complicated shape variations. We automate the object contour initialization by a seed detection algorithm and deform contours by minimizing an energy functional that incorporates a shape term in a sparse shape prior representation, an adaptive contour occlusion penalty term, and a boundary term encouraging contours to converge to strong edges. As a result, our approach is able to deal with mutual occlusions and detect contours of multiple intersected nuclei simultaneously. Our method is applied to several whole-slide histopathologic image datasets for nuclei segmentation. The proposed method is compared with other state-of-the-art methods and demonstrates good accuracy for nuclei detection and segmentation, suggesting its promise to support biomedical image-based investigations.

Accurate vessel centerline tracing greatly benefits vessel centerline geometry assessment and facilitates precise measurements of vessel diameters and lengths. However, cursive and longitudinal geometries of vessels make centerline tracing a challenging task in volumetric images. Treating the problem with traditional feature handcrafting is often ad-hoc and time-consuming, resulting in suboptimal solutions. In this work, we propose a unified end-to-end deep reinforcement learning approach for robust vessel centerline tracing in multi-modality 3D medical volumes. Instead of time-consuming exhaustive search in 3D space, we propose to learn an artificial agent to interact with surrounding environment and collect rewards from the interaction. A deep neural network is integrated to the system to predict stepwise action value for every possible actions. With this mechanism, the agent is able to probe through an optimal navigation path to trace the vessel centerline. Our proposed approach is evaluated on a dataset of over 2,000 3D volumes with diverse imaging modalities, including contrasted CT, non-contrasted CT, C-arm CT and MR images. The experimental results show that the proposed approach can handle large variations from vessel shape to imaging characteristics, with a tracing error as low as 3.28 mm and detection time as fast as 1.71 s per volume.

Magnetic Resonance Imaging (MRI) typically collects data below the Nyquist sampling rate for imaging acceleration. To remove aliasing artifacts, we propose a multi-channel deep generative adversarial network (GAN) model for MRI reconstruction. Because multi-channel GAN matches the parallel data acquisition system architecture on a modern MRI scanner, this model can effectively learn intrinsic data correlation associated with MRI hardware from originally-collected multi-channel complex data. By estimating missing data directly with the trained network, images may be generated from undersampled multi-channel raw data, providing an “end-to-end” approach to parallel MRI reconstruction. By experimentally comparing with other methods, it is demonstrated that multi-channel GAN can perform image reconstruction with an affordable computation cost and an imaging acceleration factor higher than the current clinical standard.

With the rapid advancement in large-throughput scanning technologies, digital pathology has emerged as platform with promise for diagnostic approaches, but also for high-throughput quantitative data extraction and analysis for translational research. Digital pathology and biomarker images are rich sources of information on tissue architecture, cell diversity and morphology, and molecular pathway activation. However, the understanding of disease in three-dimension (3D) has been hampered by their traditional two-dimension (2D) representations on histologic slides. In this paper, we propose a scalable image processing framework to quantitatively investigate 3D phenotypic and cell-specific molecular features from digital pathology and biomarker images in information- lossless 3D tissue space. We also develop a generalized 3D spatial data management framework with multi-level parallelism and provide a sustainable infrastructure for rapid spatial queries through scalable and efficient spatial data processing. The developed framework can facilitate biomedical research by efficiently processing large-scale, 3D pathology and in-situ biomarker imaging data.

High-throughput serial histology imaging provides a new avenue for the routine study of micro-anatomical structures in a 3D space. However, the emergence of serial whole slide imaging poses a new registration challenge, as the gigapixel image size precludes the direct application of conventional registration techniques. In this paper, we develop a three-stage registration with multi-resolution mapping and propagation method to dynamically produce registered subvolumes from serial whole slide images. We validate our algorithm with gigapixel images of serial brain tumor sections and synthetic image volumes. The qualitative and quantitative assessment results demonstrate the efficacy of our approach and suggest its promise for 3D histology reconstruction analysis.

In this paper, we propose a novel segmentation method for cells in histopathologic images based on a sparse shape prior guided variational level set framework. We automate the cell contour initialization by detecting seeds and deform contours by minimizing a new energy functional that incorporates a shape term involving sparse shape priors, an adaptive contour occlusion penalty term, and a boundary term encouraging contours to converge to strong edges. As a result, our approach is able to accommodate mutual occlusions and detect contours of multiple intersected cells. We apply our algorithm to a set of whole-slide histopathologic images of brain tumor sections. The proposed method is compared with other popular methods, and demonstrates good accuracy for cell segmentation by quantitative measures, suggesting its promise to support biomedical image-based investigations.

Glioblastoma (GBM) is a malignant brain tumor with uniformly dismal prognosis. Quantitative analysis of GBM cells is an important avenue to extract latent histologic disease signatures to correlate with molecular underpinnings and clinical outcomes. As a prerequisite, a robust and accurate cell segmentation is required. In this paper, we present an automated cell segmentation method that can satisfactorily address segmentation of overlapped cells commonly seen in GBM histology specimens. This method first detects cells with seed connectivity, distance constraints, image edge map, and a shape-based voting image. Initialized by identified seeds, cell boundaries are deformed with an improved variational level set method that can handle clumped cells. We test our method on 40 histological images of GBM with human annotations. The validation results suggest that our cell segmentation method is promising and represents an advance in quantitative cancer research.