Lecture Notes in Computer Science

Published by Springer Verlag
Online ISSN: 0302-9743
Conference Paper
In the last decade, a revolution has occurred in access to census microdata for social and behavioral research. More than 325 million person records (55 countries, 159 samples) representing two-thirds of the world's population are now readily available to bona fide researchers from the IPUMS-International website: www.ipums.org/international hosted by the Minnesota Population Center. Confidentialized extracts are disseminated on a restricted access basis at no cost to bona fide researchers. Over the next five years, from the microdata already entrusted by National Statistical Office-owners, the database will encompass more than 80 percent of the world's population (85 countries, ~100 additional datasets) with priority given to samples from the 2010 round of censuses. A profile of the most frequently used samples and variables is described from 64,248 requests for microdata extracts. The development of privacy protection standards by National Statistical Offices, international organizations and academic experts is fundamental to eliciting world-wide cooperation and, thus, to the success of the IPUMS initiative. This paper summarizes the legal, administrative and technical underpinnings of the project, including statistical disclosure controls, as well as the conclusions of a lengthy on-site review by the former Australian Statistician, Mr. Dennis Trewin.
Average times for manual and semi-automatic segmentation
The warping function: r /r = 1 − sin(θ) exp −r 2 2σ 2 . The warped radius (r ) is approximately equal to the unwarped radius (r) for angles (θ) close to 0 or π and for large r values. The warped radius (r ) is less than the unwarped radius (r) for angles (θ) close to π/2 and small r values; therefore, causing a stretch. The transition point between small and large r values is set by σ.
Comparison between manual segmentation (dotted line) and the proposed semiautomatic segmentation (solid line)
Conference Paper
This paper presents a new algorithm for the semi-automatic segmentation of the prostate from B-mode trans-rectal ultrasound (TRUS) images. The segmentation algorithm first uses image warping to make the prostate shape elliptical. Measurement points along the prostate boundary, obtained from an edge-detector, are then used to find the best elliptical fit to the warped prostate. The final segmentation result is obtained by applying a reverse warping algorithm to the elliptical fit. This algorithm was validated using manual segmentation by an expert observer on 17 midgland, pre-operative, TRUS images. Distance-based metrics between the manual and semi-automatic contours showed a mean absolute difference of 0.67 +/- 0.18 mm, which is significantly lower than inter-observer variability. Area-based metrics showed an average sensitivity greater than 97% and average accuracy greater than 93%. The proposed algorithm was almost two times faster than manual segmentation and has potential for real-time applications.
Conference Paper
We have developed a novel model-to-image registration technique which aligns a 3-dimensional model of vasculature with two semiorthogonal fluoroscopic projections. Our vascular registration method is used to intra-operatively initialize the alignment of a catheter and a preoperative vascular model in the context of image-guided TIPS (Transjugular, Intrahepatic, Portosystemic Shunt formation) surgery. Registration optimization is driven by the intensity information from the projection pairs at sample points along the centerlines of the model. Our algorithm shows speed, accuracy and consistency given clinical data.
Conference Paper
Accurate 3D/2D vessel registration is complicated by issues of image quality, occlusion, and other problems. This study performs a quantitative comparison of 3D/2D vessel registration in which vessels segmented from preoperative CT or MR are registered with biplane x-ray angiograms by either a) simultaneous two-view registration with advance calculation of the relative pose of the two views, or b) sequential registration with each view. We conclude on the basis of phantom studies that, even in the absence of image errors, simultaneous two-view registration is more accurate than sequential registration. In more complex settings, including clinical conditions, the relative accuracy of simultaneous two-view registration is even greater.
Example of extracting apparent contours (white) and edge pixels (green) 
Reconstruction errors when different number of images were used
Establishing 2D/3D correspondence; detected edge pixels (green), extracted apparent contours (white), and the estabilished correspondences (linked with yellow line segment for visualization purpose) 
Different stages of reconstruction. First: one of the acquired images. Second: the initialization of the mean model of the PDM. Third: after establishing image-to-model correspondence. Forth: after 3D paired point matching. Fifth: after re-establishing correspondence; Sixth: the final reconstruction result after a series of computations 
Reconstruction of patient-specific 3D bone surface from 2D calibrated fluoroscopic images and a point distribution model is discussed. We present a 2D/3D reconstruction scheme combining statistical extrapolation and regularized shape deformation with an iterative image-to-model correspondence establishing algorithm, and show its application to reconstruct the surface of proximal femur. The image-to-model correspondence is established using a non-rigid 2D point matching process, which iteratively uses a symmetric injective nearest-neighbor mapping operator and 2D thin-plate splines based deformation to find a fraction of best matched 2D point pairs between features detected from the fluoroscopic images and those extracted from the 3D model. The obtained 2D point pairs are then used to set up a set of 3D point pairs such that we turn a 2D/3D reconstruction problem to a 3D/3D one. We designed and conducted experiments on 11 cadaveric femurs to validate the present reconstruction scheme. An average mean reconstruction error of 1.2 mm was found when two fluoroscopic images were used for each bone. It decreased to 1.0 mm when three fluoroscopic images were used.
Reference systems in the synthetic x-ray projection environment. The CT scan (CT) and the cup model (Cup) are placed relative to the x-ray film (Radiograph) in order to create a synthetic projection. The anatomic reference plane of the pelvis (APP) is specified in the CT scan.
Registration of the pelvis: (a) real radiograph (b) simulated radiograph corresponding to the solved pose of the CT in the x-ray space (c) real and simulated radiographs overlaid 
Registration of the cup: (a) real radiograph (b) simulated projection for the solved pose of the cup (c) the real and simulated images overlaid
Differences in measured cup orientation between direct CT measurements and the 2D/3D registration method
Conference Paper
Measurements of cup alignment after total hip replacement (THR) surgery are typically performed on postoperative pelvic radiographs. Radiographic measurement of cup orientation depends on the position and orientation of the pelvis on the X-ray table, and its variability could introduce significant measurement errors. We have developed a tool to accurately measure 3D implant orientation from postoperative antero-posterior radiographs by registering to preoperative CT scans. The purpose of this study is to experimentally and clinically validate the automatic CT/X-ray matching algorithm by comparing the X-ray based measurements of cup orientation with direct 3D measurements from postoperative CT scans. The mean measurement errors (+/- stdev) found in this study were 0.4 degrees +/-0.8 degrees for abduction and 0.6 degrees +/- 0.8 degrees for version. In addition, radiographic pelvic orientation measurements demonstrated a wide range of inter-subject variability, with pelvic flexion ranging from -5.9 degrees to 11.2 degrees.
Conference Paper
A novel shape based segmentation approach is proposed by modifying the external energy component of a deformable model. The proposed external energy component depends not only on the gray level of the images but also on the shape information which is obtained from the signed distance maps of objects in a given data set. The gray level distribution and the signed distance map of the points inside and outside the object of interest are accurately estimated by modelling the empirical density function with a linear combination of discrete Gaussians (LCDG) with positive and negative components. Experimental results on the segmentation of the kidneys from low-contrast DCE-MRI and on the segmentation of the ventricles from brain MRI's show how the approach is accurate in segmenting 2-D and 3-D data sets. The 2D results for the kidney segmentation have been validated by a radiologist and the 3D results of the ventricle segmentation have been validated with a geometrical phantom.
Conference Paper
In this paper, we focus on automatic kidneys detection in 2D abdominal computed tomography (CT) images. Identifying abdominal organs is one of the essential steps for visualization and for providing assistance in teaching, clinical training and diagnosis. It is also a key step in medical image retrieval application. However, due to gray levels similarities of adjacent organs, contrast media effect and relatively high variation of organ's positions and shapes, automatically identifying abdominal organs has always been a challenging task. In this paper, we present an original method, in a statistical framework, for fully automatic kidneys detection. It makes use of spatial and gray-levels prior models built using a set of training images. The method is tested on over 400 clinically acquired images and very promising results are obtained.
In this paper we present a novel 3D/2D registration method, where first, a 3D image is reconstructed from a few 2D X-ray images and next, the preoperative 3D image is brought into the best possible spatial correspondence with the reconstructed image by optimizing a similarity measure. Because the quality of the reconstructed image is generally low, we introduce a novel asymmetric mutual information similarity measure, which is able to cope with low image quality as well as with different imaging modalities. The novel 3D/2D registration method has been evaluated using standardized evaluation methodology and publicly available 3D CT, 3DRX, and MR and 2D X-ray images of two spine phantoms, for which gold standard registrations were known. In terms of robustness, reliability and capture range the proposed method outperformed the gradient-based method and the method based on digitally reconstructed radiographs (DRRs).
Intraoperative position measurement and laparoscopic ultrasound image acquisition of the liver 
Analysis of respiratory cycle. A: Inspiration point. B: Start point of deformation. C: Expiration point. D: End point of deformation.
Magnitude of recovered motion of the liver along the body axis (Data set 1). CC: cranio-caudal movement. A-P: anterior-posterior movement. L-R: lateral movement. 
3D US liver vessel model and US image frames of sagittal planes (Data set 1). C-C: the cranio-caudal axis. A-P: the anterior-posterior axis. L-R: the lateral axis. 
Results on accuracy evaluation for recovery of deformation vectors in whole volume
Conference Paper
This paper describes a rapid method for intraoperative recovery of liver motion and deformation due to respiration by using a laparoscopic freehand 3D ultrasound (US) system. Using the proposed method, 3D US images of the liver can be extended to 4D US images by acquiring additional several sequences of 2D US images during a couple of respiration cycles. Time-varying 2D US images are acquired on several sagittal image planes and their 3D positions and orientations are measured using a laparoscopic ultrasound probe to which a miniature magnetic 3D position sensor is attached. During the acquisition, the probe is assumed to move together with the liver surface. In-plane 2D deformation fields and respiratory phase are estimated from the time-varying 2D US images, and then the time-varying 3D deformation fields on the sagittal image planes are obtained by combining 3D positions and orientations of the image planes. The time-varying 3D deformation field of the volume is obtained by interpolating the 3D deformation fields estimated on several planes. The proposed method was evaluated by in vivo experiments using a pig liver.
Conference Paper
The purpose of our project is to develop an image guided system for the medialization laryngoplasty. One of the fundamental challenges in our system is to accurately register the preoperative 3D CT data to the intraoperative 3D surfaces of the patient. In this paper, we will present a combined surface and fiducial based registration method to register the preoperative 3D CT data to the intraoperative surface of larynx. To accurately model the exposed surface area, an active illumination based stereo vision technique is used for the surface reconstruction. To register the point clouds from the intraoperative stage to the preoperative 3D CT data, a shape priori based ICP method is proposed to quickly register the two surfaces. The proposed approach is capable of tracking the fiducial markers and reconstructing the surface of larynx with no damage to the anatomical structure. Although, the proposed method is specifically designed for the image guided laryngoplasty, it can be applied to other image guided surgical areas. We used off-the-shelf digital cameras, LCD projector and rapid 3D prototyper to develop our experimental system. The final RMS error in the registration is less than 1mm.
2D illustration of the NL-means principle. The restored value of voxel i (in red) is a weighted average of all intensities of voxels j in the search volume V i , according to the similarity of their intensities neighborhoods v ( N i ) and v ( N j ). 
Top: details of the Brainweb denoised images obtained via the three compared methods for a noise level of 9%. Bottom: images of the removed noise, i.e. the difference between noisy images and denoised images, centered on 128. From left to right: Anisotropic Diffusion, Total Variation and NL-means.
NL-means restoration of 3T MRI data of 256 3 voxels with d = 1, M = 5 in 
Conference Paper
One critical issue in the context of image restoration is the problem of noise removal while keeping the integrity of relevant image information. Denoising is a crucial step to increase image conspicuity and to improve the performances of all the processings needed for quantitative imaging analysis. The method proposed in this paper is based on an optimized version of the Non Local (NL) Means algorithm. This approach uses the natural redundancy of information in image to remove the noise. Tests were carried out on synthetic datasets and on real 3T MR images. The results show that the NL-means approach outperforms other classical denoising methods, such as Anisotropic Diffusion Filter and Total Variation.
X-ray angiogram, acquired immediately after a stent deployment. The markers can clearly be differentiated due to the high absorption coefficient. 
A projection acquired at motion state T m before (a) and after warping (b) into the optimal cardiac phase (T c 0 ) based on a Thin Plate Spline (TPS) transformation matrix (Φ)
Volume rendered images of a 3D reconstructed stent deployed in patient one, utilizing the original data set (a) and utilizing motion compensated data set (b). The struts of the stent have a diameter of approximately 40µm.C ut planes perpendicular (c) and parallel (d) to the longitudinal direction of the volume rendered images of the reconstructed stent are visualized revealing information about the in-stent lumen.
Illustration of volume rendered images of a deployed stent in patient two. Different cut planes perpendicular to the tangential direction of the stent have been chosen to visualize the inner-stent lumen. 
Conference Paper
A new method is introduce for the three-dimensional (3D) reconstruction of the coronary stents in-vivo utilizing two-dimensional projection images acquired during rotational angiography (RA). The method is based on the application of motion compensated techniques to the acquired angiograms resulting in a temporal snapshot of the stent within the cardiac cycle. For the first time results of 3D reconstructed coronary stents in vivo, with high spatial resolution are presented. The proposed method allows for a comprehensive and unique quantitative 3D assessment of stent expansion that rivals current x-ray and intravascular ultrasound techniques.
Illustration of DW and PT principles. The two orthogonal projections for DW interpolation method and the construction of a "virtual" plane πt containing X for PT method.
B-scans sequences used during evaluation. Left: fan sequence. Right translation sequence. 
Differences between original and reconstructed B-scan for not subsampled fan sequence. From left to right VNN, DW and PT methods. This shows that error between reconstructed B-scan and “ground truth” is visually better with the PT method. 
Zoom on hepatic vessels extracted from liver reconstruction with degree= 1 and subsampling factor= 4. From left to right the VNN, DW and PT methods. The images are extracted from 3D volume along the temporal axis (z) in order to under-light inherent artifacts of the VNN (i.e. discontinuities) and the DW (i.e. blur) methods. The PT method is more efficient at preserving the native texture pattern of US image than the DW method. 
Conference Paper
3D freehand ultrasound imaging is a very attractive technique in medical examinations and intra-operative stage for its cost and field of view capacities. This technique produces a set of non parallel B-scans which are irregularly distributed in the space. Reconstruction amounts to computing a regular lattice volume and is needed to apply conventional computer vision algorithms like registration. In this paper, a new 3D reconstruction method is presented, taking explicitly into account the probe trajectory. Experiments were conducted on different data sets with various probe motion types and indicate that this technique outperforms classical methods, especially on low acquisition frame rate.
Flowchart of an iteration during the needle insertion into the tissue 
Conference Paper
This paper presents a needle-tissue interaction model that is a 3D extension of a prior work based on the finite element method. The model is also adapted to accommodate arbitrary meshes so that the anatomy can effectively be meshed using third-party algorithms. Using this model a prostate brachytherapy simulator is designed to help medical residents acquire needle steering skills. This simulation uses a prostate mesh generated from clinical data segmented as contours on parallel slices. Node repositioning and addition, which are methods for achieving needle-tissue coupling, are discussed. In order to achieve realtime haptic rates, computational approaches to these methods are compared. Specifically, the benefit of using the Woodbury formula (matrix inversion lemma) is studied. Our simulation of needle insertion into a prostate is shown to run faster than 1 kHz.
Conference Paper
In this paper, we present a learning-based method for the detection and segmentation of 3D free-form tubular structures, such as the rectal tubes in CT colonoscopy. This method can be used to reduce the false alarms introduced by rectal tubes in current polyp detection algorithms. The method is hierarchical, detecting parts of the tube in increasing order of complexity, from tube cross sections and tube segments to the whole flexible tube. To increase the speed of the algorithm, candidate parts are generated using a voting strategy. The detected tube segments are combined into a flexible tube using a dynamic programming algorithm. Testing the algorithm on 210 unseen datasets resulted in a tube detection rate of 94.7% and 0.12 false alarms per volume. The method can be easily retrained to detect and segment other tubular 3D structures.
Registration between RT3D US and dynamic 3D MR images. (a) orthogonal slices of the US volume of the beating heart; (b) the MRI volume of the beating heart; (c) the overlay of the two image sets after registration. 
Conference Paper
Real-time three-dimensional ultrasound (RT3D US) is an ideal imaging modality for the diagnosis of cardiac disease. RT3D US is a flexible, inexpensive, non-invasive tool that provides important diagnostic information related to cardiac function. Unfortunately, RT3D US suffers from inherent shortcomings, such as low signal-to-noise ratio and limited field of view, producing images that are difficult to interpret. Multi-modal dynamic cardiac image registration is a well-recognized approach that compensates for these deficiencies while retaining the advantages of RT3D US imaging. The clinical application of multi-modal image registration methods is difficult, and there are a number of implementation issues to be resolved. In this work, we present a method for the rapid registration of RT3D US images of the beating heart to high-resolution magnetic resonance (MR) images. This method was validated using a volunteer image set. Validation results demonstrate that this approach can achieve rapid registration of images of the beating heart with fiducial landmark and registration errors of 1.25 +/- 0.63 and 1.76 mm respectively. This technique can potentially be used to improve the diagnosis of cardiac disease by augmenting RT3D US images with high-resolution MR images and to facilitate intra-operative image fusion for minimally invasive cardio-thoracic surgical navigation.
(a) Subdivision Grid. (b,c) Scaling (d,e) Wavelet Function for Different Nodes
Band Decomposition: various bands B j,i , where j is the resolution and i is the band number, shown in Anterior view (A) and Posterior view (P), see text for color 
Mean Squared Reconstruction Error for various training set sizes where Σ A is the covariance matrix of P (A) and |.| is the determinant 2. If the distributions P (A) and P (B) are independent, then their KL divergence is 0. In practice, we do not accept a partition if D(P (G)||P (A)P (B)) > 0.1. Band Visualization: To visualize the bands, we calculate the influence of all wavelet coefficient in band B j,i on each point x of the surface by setting those coefficients to 1 in (1) and others to 0. If f (x) = 0, then the point is not affected and if f (x) > 0 it is affected according to the magnitude of f (x). Using this function as a colormap (blue= 0, red= 1), Figures 3(a)-3(b) show the first band for the lowest scale. The second band is the complement of the first. As expected each band has a large spatial extent and indicate two uncorrelated shape processes on the prostate data: the variation of the anterior wall of the prostate (typically rounded) and the variation of the posterior wall of the prostate (typically flatter). Figures 3(c)-3(h) show three bands for the scale 3. These bands are more localized. These are uncorrelated variations of the superior and inferior walls of the shape, as well as an uncorrelated variations of the anterior wall at that scale. Bands have compact support, though this is not a constraint of our technique. The symmetry in bands 2 and 3 is also interesting, showing that both the right and left side tend to co-vary similarly. This symmetry of variation is plausible for the prostate, and we plan to investigate this further. Notably a diseased organ could possibly be detected if there is a lack of symmetry. 
Conference Paper
Shape priors attempt to represent biological variations within a population. When variations are global, Principal Component Analysis (PCA) can be used to learn major modes of variation, even from a limited training set. However, when significant local variations exist, PCA typically cannot represent such variations from a small training set. To address this issue, we present a novel algorithm that learns shape variations from data at multiple scales and locations using spherical wavelets and spectral graph partitioning. Our results show that when the training set is small, our algorithm significantly improves the approximation of shapes in a testing set over PCA, which tends to oversmooth data.
Conference Paper
This paper presents an outlier immune 3D active shape models framework for robust volumetric segmentation of the carotid artery required for accurate plaque burden assessment. In the proposed technique, outlier handling is based on a shape metric that is invariant to scaling, rotation and translation by using the ratio of inter-landmark distances as a local shape dissimilarity measure. Tolerance intervals for each descriptor are calculated from the training samples and used to infer the validity of landmarks. The identified outliers are corrected prior to the model fitting using the ratios distributions and appearance information. To improve the feature point search, the method exploits the geometrical knowledge from the outlier analysis at the previous iteration to weight the gray level appearance based fitness measure. A combined intensity-phase feature point search is also introduced which significantly limits the presence of outliers and improves the overall search accuracy. Both numerical and in vivo assessments of the method involving volumetric segmentation of the carotid artery have shown that the outlier handling technique is capable of handling a significant presence of outliers independently of the amplitudes.
Conference Paper
Motivated by the need for methods to aid the deformable registration of brain tumor images, we present a three-dimensional (3D) mechanical model for simulating large non-linear deformations induced by tumors to the surrounding encephalic tissues. The model is initialized with 3D radiological images and is implemented using the finite element (FE) method. To simulate the widely varying behavior of brain tumors, the model is controlled by a number of parameters that are related to variables such as the bulk tumor location, size, mass-effect, and peri-tumor edema extent. Model predictions are compared to real brain tumor-induced deformations observed in serial-time MRI scans of a human subject and 3 canines with surgically transplanted gliomas. Results indicate that the model can reproduce the real deformations with an accuracy that is similar to that of manual placement of landmark points to which the model is compared.
Secondary structure elucidation algorithm for Insulin Receptor Tyrosine Kinase Domain with pdbid: 1IRK. (a) volume rendering of its blurred 3D Map at 8 ˚ A resolution (b) surface rendering of the protein's molecular surface (c) pointset sampling of the molecular surface (d) The red patch inside the transparent surface depicts the β-sheet while the straight lines designate the axes of the cylinders which correspond to the α-helices (e) The secondary structural motifs, documented in the Protein Data Bank, where the helices are shown as ribbon coils and the sheets are sets of ribbon strands. (f) combined display of (d) and (e) 
(a) The molecular surface of 1IRK. (b) The selected Voronoi vertices on U2 and the fitted cylinder. (c) Filtering out subsets of U1 which are small (green) or do not satisfy the width test (magenta). (d) shows the secondary structures obtained from the PDB and its correspondence with the computed structure (b,c). 
(a) σ1 < σ2. (b) The situation when σ has two neighbors σ and σ for both of which σ < σ and σ < σ. (c) 1TIM: Helices surround the sheets to form the tertiary structure called α/β-barrel. (d) Molecular surface of 1TIM at 15Å15Å resolution. (e,f) The initial segmentation and further refinement to bring out the β-fold of the barrel from the surrounding helices (yellow, magenta and blue). 
Performance of the tertiary fold elucidation algorithm. 
Recent advances in three dimensional Electron Microscopy (3D EM) have given an opportunity to look at the structural building blocks of proteins (and nucleic acids) at varying resolutions. In this paper, we provide algorithms to detect the secondary structural motifs (α-helices and β-sheets) from proteins for which the volumetric maps are reconstructed at 5-10Å resolution. Additionally, we show that when the resolution is coarser than 10Å, some of the tertiary structural motifs can be detected from 3D EM. For both these algorithms, we employ the tools from computational geometry and differential topology, specifically the computation of stable/unstable manifolds of certain critical points of the distance function induced by the molecular surface. With the results in this paper, we thus draw a connection between the mathematically well-defined concepts with the bio-chemical structural folds of proteins.
Conference Paper
Preventing complications during hepatic surgery in living-donor transplantation or in oncologic resections requires a careful preoperative analysis of the hepatic venous anatomy. Such an analysis relies on CT hepatic venography data, which enhances the vascular structure due to contrast medium injection. However, a 3D investigation of the enhanced vascular anatomy based on typical computer vision tools is ineffective because of the large amount of occlusive opacities to be removed. This paper proposes an automated 3D approach for the segmentation of the vascular structure in CT hepatic venography, providing the appropriate tools for such an investigation. The developed methodology relies on advanced topological and morphological operators applied in mono- and multiresolution filtering schemes. It allows to discriminate the opacified vessels from the bone structures and liver parenchyma regardless of noise presence or inter-patient variability in contrast medium dispersion. The proposed approach was demonstrated at different phases of hepatic perfusion and is currently under extensive validation in clinical routine.
Conference Paper
We present a fully automatic 3D segmentation method for the left ventricle (LV) in human myocardial perfusion SPECT data. This model-based approach consists of 3 phases: 1. finding the LV in the dataset, 2. extracting its approximate shape and 3. segmenting its exact contour. Finding of the LV is done by flexible pattern matching, whereas segmentation is achieved by registering an anatomical model to the functional data. This model is a new kind of stable 3D mass spring model using direction-weighted 3D contour sensors. Our approach is much faster than manual segmention, which is standard in this application up to now. By testing it on 41 LV SPECT datasets of mostly pathological data, we could show, that it is very robust and its results are comparable with those made by human experts.
Conference Paper
This paper presents a method for the initial detection of ductal structures within 3D ultrasound images using second-order shape measurements. The desire to detect ducts is motivated in a number of way, principally as step in the detection and assessment of ductal carcinoma in-situ. Detection is performed by measuring the variation of the local second-order shape from a prototype shape corresponding to a perfect tube. We believe this work is the first demonstration of the ability to detect sections of duct automatically in ultrasound images. The detection is performed with a view to employing vessel tracking method to delineate the full ductal structure.
The images in the left column (a), (d), and (g) show parts of microtubule assemblies of CAR cells. Next to each image in the same row (b), (e), and (h), respectively, are the segmented microtubules drawn with different colors. The column in the right shows a viewpoint of the 3D tangents of the segmented microtubules at their tips.
Conference Paper
The interaction of the microtubules with the cell cortex plays numerous critical roles in a cell. For instance, it directs vesicle delivery, and modulates membrane adhesions pivotal for cell movement as well as mitosis. Abnormal function of the microtubules is involved in cancer. An effective method to observe microtubule function adjacent to the cortex is TIRFM. To date most analysis of TIRFM images has been done by visual inspection and manual tracing. In this work we have developed a method to automatically process TIRFM images of microtubules so as to enable high throughput quantitative studies. The microtubules are extracted in terms of consecutive segments. The segments are described via Hamilton-Jacobi equations. Subsequently, the algorithm performs a limited reconstruction of the microtubules in 3D. Last, we evaluate our method with phantom as well as real TIRFM images of living cells.
Conference Paper
Robust 3D point registration is difficult for biomedical surfaces, especially for roundish and approximate symmetric soft tissues such as liver, stomach, etc. We present an Iterative Optimization Registration scheme (IOR) based on Hierarchical Vertex Signatures (HVS) between point-sets of medical surfaces. HVSs are distributions of concatenated neighborhood angles relative to the PCA axes of the surfaces which concisely describe global structures and local contexts around vertices in a hierarchical paradigm. The correspondences between point-sets are then established by Chi-Square test statistics. Specifically, to alleviate the sensitivity to axes directions that often affects robustness for other global axes based algorithms, IOR aligns surfaces gradually, and incrementally calibrates the directions of major axes in a multi-resolution manner. The experimental results demonstrate IOR is efficient and robust for liver registration. This method is also promising to other applications such as morphological pathological analysis, 3D model retrieval and object recognition.
Conference Paper
This work studies limits on estimating the width of thin tubular structures in 3D images. Based on nonlinear estimation theory we analyze the minimal stochastic error of estimating the width. Given a 3D analytic model of the image intensities of tubular structures, we derive a closed-form expression for the Cramér-Rao bound of the width estimate under image noise. We use the derived lower bound as a benchmark and compare it with three previously proposed accuracy limits for vessel width estimation. Moreover, by experimental investigations we demonstrate that the derived lower bound can be achieved by fitting a 3D parametric intensity model directly to the image data.
To more accurately separate each pulmonary nodule from its background in a low dose computer tomography (LDCT) chest image, two new adaptive probability models of visual appearance of small 2D and large 3D pulmonary nodules are used to control evolution of deformable boundaries. The appearance prior is modeled with a translation and rotation invariant Markov-Gibbs random field of voxel intensities with pairwise interaction analytically identified from a set of training nodules. Appearance of the nodules and their background in a current multi-modal chest image is also represented with a marginal probability distribution of voxel intensities. The nodule appearance model is isolated from the mixed distribution using its close approximation with a linear combination of discrete Gaussians. Experiments with real LDCT chest images confirm high accuracy of the proposed approach.
Visualization of the variance of the created shape model: The left column shows the variation of the largest eigenmode between ± 3 √ λ 1 , the medium and right column of the second and third largest eigenmode, respectively. 
Histograms showing the displacements from the true landmark positions at different resolutions R 0 to R 5 . From left to right: Intensity, gradient and normalized 
Results of the segmentation using the normalized gradient appearance model (1=ASM with 10 modes, 2=ASM with 30 modes, 3=deformable model with 10 modes). The box connects the 1st and 3rd quartiles of all values with the dot representing the median, the whiskers span the interval between the 0.05 and 0.95 quantiles. 
Conference Paper
This paper presents an evaluation of the performance of a three-dimensional Active Shape Model (ASM) to segment the liver in 48 clinical CT scans. The employed shape model is built from 32 samples using an optimization approach based on the minimum description length (MDL). Three different gray-value appearance models (plain intensity, gradient and normalized gradient profiles) are created to guide the search. The employed segmentation techniques are ASM search with 10 and 30 modes of variation and a deformable model coupled to a shape model with 10 modes of variation. To assess the segmentation performance, the obtained results are compared to manual segmentations with four different measures (overlap, average distance, RMS distance and ratio of deviations larger 5mm). The only appearance model delivering usable results is the normalized gradient profile. The deformable model search achieves the best results, followed by the ASM search with 30 modes. Overall, statistical shape modeling delivers very promising results for a fully automated segmentation of the liver.
Conference Paper
Freehand 3D ultrasound can be acquired without a position sensor by finding the separations of pairs of frames using information in the images themselves. Previous work has not considered how to reconstruct entirely freehand data, which can exhibit irregularly spaced frames, non-monotonic out-of-plane probe motion and significant inplane motion. This paper presents reconstruction methods that overcome these limitations and are able to robustly reconstruct freehand data. The methods are assessed on freehand data sets and compared to reconstructions obtained using a position sensor.
Comparison of methods: (a) shows the reconstructed SPHARM surface of left ventricle, (b) is the first order ellipsoid of surface (a); (c) shows the reconstructed SPHARM surface of right ventricle, (d) is the first order ellipsoid of surface (c). By using the previous method, the first order ellipsoids and parametrizations are rotated to the positions in (f) and (h), and the SPHARM surfaces and parametrizations are rotated as (e) and (g). By using our algorithm, (i) shows the result of poles alignment. North (yellow point) and south (red point) poles are aligned close to the poles of (a). And the parameter mesh is rotated along the north pole. After using the BFGS algorithm [13] in the second step, the last alignment result is shown in (j).
Conference Paper
The spherical harmonic (SPHARM) description is a powerful surface modeling technique that can model arbitrarily shaped but simply connected 3D objects and has been used in many applications in medical imaging. Previous SPHARM techniques use the first order ellipsoid for establishing surface correspondence and aligning objects. However, this first order information may not be sufficient in many cases; a more general method for establishing surface correspondence would be to minimize the mean squared distance between two corresponding surfaces. In this paper, a new surface matching algorithm is proposed for 3D SPHARM models to achieve this goal. This algorithm employs a useful rotational property of spherical harmonic basis functions for a fast implementation. Applications of medical image analysis (e.g., spatio-temporal modeling of heart shape changes) are used to demonstrate this approach. Theoretical proofs and experimental results show that our approach is an accurate and flexible surface correspondence alignment method.
Conference Paper
Intravascular ultrasound (IVUS) produces images of arteries that show the lumen in addition to the layered structure of the vessel wall. A new automatic 3D IVUS fast-marching segmentation model is presented. The method is based on a combination of region and contour information, namely the gray level probability density functions (PDFs) of the vessel structures and the image gradient. Accurate results were obtained on in-vivo and simulated data with average point to point distances between detected vessel wall boundaries and validation contours below 0.105 mm. Moreover, Hausdorff distances (that represent the worst point to point variations) resulted in values below 0.344 mm, which demonstrate the potential of combining region and contour information in a fast-marching scheme for 3D automatic IVUS image processing.
Level set method based segmentation provides an efficient tool for topological and geometrical shape handling, but is slow due to high computational burden. In this work, we provide a framework for streaming computations on large volumetric images on the GPU. A streaming computational model allows processing large amount of data with small memory footprint. Efficient transfer of data to and from the graphics hardware is performed via a memory manager. We show volumetric segmentation using higher order, multi-phase level set method with speedups of the order of 20x.
An illustration of co-helicity between three vectors v 1 , v 2 and v 3 . 
Top Row: A biological phantom created by overlaying two rat cord spinal cords (left), ground truth orientation estimates shown separately for the two cords (middle), and the principal eigenvector orientations of the DTI image in the vicinity of the crossing (right). Bottom Row: The regularized orientation estimates obtained using the technique of [11] (left), those obtained using 3D curve inference on the DTI reconstruction (middle), and the regularized HARDI reconstruction using 3D curve inference (right). 
Left: Table of validation results showing average orientation errors in degrees for the biological phantom data set and the synthetic data set. Right: A snapshot of the noisy synthetic data set, prior to regularization. 
A ROI through the brain DTI data (top left) with the regularization results using curve inference (top right). A slice through the associated fractional anisotropy image (bottom left) with a white rectangle enclosing the ROI. A zoom-in on the result from a different viewpoint in a region of partial volume averaging effects (bottom right). 
Conference Paper
We develop a differential geometric framework for regularizing diffusion MRI data. The key idea is to model white matter fibers as 3D space curves and to then extend Parent and Zucker's 2D curve inference approach [8] by using a notion of co-helicity to indicate compatibility between fibre orientation estimates at each voxel with those in a local neighborhood. We argue that this provides several advantages over earlier regularization methods. We validate the approach quantitatively on a biological phantom and on synthetic data, and qualitatively on data acquired in vivo from a human brain.
(a-b) Recursive Partitioning of an octahedron (c-d) Visualization of wavelet basis functions at various levels. The color corresponds to the value of the functions.  
Surface Evolution using the Ground Truth labelmap as the image force for ASM (top rown) and Mscale (bottom row) algorithms. The ground truth is shown in light gray, the evolving surface in dark grey.  
Surface Evolution using the density estimation as the image force for ASM (top rown) and Mscale (bottom row) algorithms. The ground truth is shown in light gray, the evolving surface in dark grey.  
Conference Paper
This paper presents a novel active surface segmentation algorithm using a multiscale shape representation and prior. We define a parametric model of a surface using spherical wavelet functions and learn a prior probability distribution over the wavelet coefficients to model shape variations at different scales and spatial locations in a training set. Based on this representation, we derive a parametric active surface evolution using the multiscale prior coefficients as parameters for our optimization procedure to naturally include the prior in the segmentation framework. Additionally, the optimization method can be applied in a coarse-to-fine manner. We apply our algorithm to the segmentation of brain caudate nucleus, of interest in the study of schizophrenia. Our validation shows our algorithm is computationally efficient and outperforms the Active Shape Model algorithm by capturing finer shape details.
Conference Paper
Surface-based morphometry (SBM) is widely used in biomedical imaging and other domains to localize shape changes related to different conditions. This paper presents a computational framework that integrates a set of effective surface registration and analysis methods to form a unified SBM processing pipeline. Surface registration includes two parts: surface alignment in the object space by employing the iterative closest point (ICP) method, and surface alignment in the parameter space by using conformal mapping and landmark-based thin-plate spline methods. Statistical group analysis of registered surface data is then conducted by surface-based general linear model and random field theory addressing multiple testing issues. The effectiveness of the proposed framework is demonstrated by applying it to a fetal alcohol syndrome (FAS) study for identifying facial dysmorphology in FAS patients.
(a) Anatomy of the circle of Willis; (b) segmented iso-surface; (c) reconstructed surface. The color code in (b) ranges from blue (0.0mm) to red (3.0mm), (d) reconstructed arterial side 
Conference Paper
In the context of stroke therapy simulation, a method for the segmentation and reconstruction of human vasculature is presented and evaluated. Based on CTA scans, semi-automatic tools have been developed to reduce dataset noise, to segment using active contours, to extract the skeleton, to estimate the vessel radii and to reconstruct the associated surface. The robustness and accuracy of our technique are evaluated on a vascular phantom scanned in different orientations. The reconstructed surface is compared to a surface generated by marching cubes followed by decimation and smoothing. Experiments show that the proposed technique reaches a good balance in terms of smoothness, number of triangles, and distance error. The reconstructed surface is suitable for real-time simulation, interactive navigation and visualization.
Overlap scores between manual and automatic segmentations over 20 brain scans with decreasing levels of difficulty (from set index 1 to 20). Our results compared with seven other algorithms for the task of GM, WM, and CSF Detection.  
WM and GM identification. The upper row presents classification results projected on a 2D T1 slice. The lower row demonstrates a 3D view of the results.  
Conference Paper
This study presents a novel automatic approach for the identification of anatomical brain structures in magnetic resonance images (MRI). The method combines a fast multiscale multi-channel three dimensional (3D) segmentation algorithm providing a rich feature vocabulary together with a support vector machine (SVM) based classifier. The segmentation produces a full hierarchy of segments, expressed by an irregular pyramid with only linear time complexity. The pyramid provides a rich, adaptive representation of the image, enabling detection of various anatomical structures at different scales. A key aspect of the approach is the thorough set of multiscale measures employed throughout the segmentation process which are also provided at its end for clinical analysis. These features include in particular the prior probability knowledge of anatomic structures due to the use of an MRI probabilistic atlas. An SVM classifier is trained based on this set of features to identify the brain structures. We validated the approach using a gold standard real brain MRI data set. Comparison of the results with existing algorithms displays the promise of our approach.
Fibers connecting the primary motor cortex (precentral gyrus) to the pons via the internal capsule and fibers connecting the somatosensory cortex (postcentral gyrus) from the ventral posterio-lateral nucleus (VPL) of the thalamus. The figure shows (a) sagittally reformatted view of the HRP reacted histological volume (bright in this picture), (b) fiber pathways derived from histology DTI tractography superimposed on the corresponding structural MRI data and (c) on the photographic volume.  
Fibers in the primary visual cortex (V1) connecting to the lateral geniculate nucleus (LGN) in the thalamus. The figure shows (a) sagittal view of the HRP reacted histological volume (bright in this picture), (b) fiber pathways derived from histology DTI tractography superimposed on the corresponding structural MRI data and (c) on the photographic volume.  
Fibers connecting the left eye to the lateral geniculate nucleus (LGN) in the thalamus via the chiasma. The figure shows (a) axial view of the HRP reacted histological volume (bright in this picture), (b) fiber pathways derived from histology DTI tractography superimposed on the corresponding structural MRI data and (c) on the photographic volume.
Fibers in the primary motor cortex (precentral gyrus) connecting the hemispheres via the corpus callosum. The figure shows (a) coronal view of the HRP reacted histological volume (bright in this picture), (b) fiber pathways derived from histology DTI tractography superimposed on the corresponding structural MRI data and (c) on the photographic volume.  
Conference Paper
A classical neural tract tracer, WGA-HRP, was injected at multiple sites within the brain of a macaque monkey. Histological sections of the labeled fiber tracts were reconstructed in 3D, and the fibers were segmented and registered with the anatomical post-mortem MRI from the same animal. Fiber tracing along the same pathways was performed on the DTI data using a classical diffusion tracing technique. The fibers derived from the DTI were compared with those segmented from the histology in order to evaluate the performance of DTI fiber tracing. While there was generally good agreement between the two methods, our results reveal certain limitations of DTI tractography, particularly at regions of fiber tract crossing or bifurcation.
Conference Paper
In functional neurosurgery, there is a growing need for accurate localization of the functional targets. Since deep brain stimulation (DBS) of the Vim thalamic nucleus has been proposed for the treatment of Parkinson's disease, the target has evolved toward the globus pallidus and subthalamic nucleus (STN) and the therapeutic indications have enlarged to include psychiatric disorders such as Tourette syndrome or obsessive compulsive disorders. In these pathologies, the target has been restrained to smaller functional subterritories of the basal ganglia, requiring more refined techniques to localize smaller and smallerbrain regions, often invisible in routine clinical MRI. Different strategies have been developed to identify such deep brain targets. Direct methods can identify structures in the MRI itself, but only the larger ones. Indirect methods are based on the use of anatomical atlases. The present strategy comprised a 3D histological atlas and the MRI of the same brain specimen, and deformation methodology developped to fit the atlas toward the brain of any given patient. In this paper, this method is evaluated in the aim of being applied to further studies of anatomo-clinical correlation. The accuracy of the method is first discussed, followed by the study of short series of Parkinsonian patients treated by DBS, allowing to compare the deformed atlas with various per- and post-operative data.
Conference Paper
A 3D Partitioned Active Shape Model (PASM) is proposed in this paper to address the problems of the 3D Active Shape Models (ASM). When training sets are small. It is usually the case in 3D segmentation, 3D ASMs tend to be restrictive. This is because the allowable region spanned by relatively few eigenvectors cannot capture the full range of shape variability. The 3D PASM overcomes this limitation by using a partitioned representation of the ASM. Given a Point Distribution Model (PDM), the mean mesh is partitioned into a group of small tiles. In order to constrain deformation of tiles, the statistical priors of tiles are estimated by applying Principal Component Analysis to each tile. To avoid the inconsistency of shapes between tiles, training samples are projected as curves in one hyperspace instead of point clouds in several hyperspaces. The deformed points are then fitted into the allowable region of the model by using a curve alignment scheme. The experiments on 3D human brain MRIs show that when the numbers of the training samples are limited, the 3D PASMs significantly improve the segmentation results as compared to 3D ASMs and 3D Hierarchical ASMs.
Conference Paper
Registration of 3D segmented cardiac images with tracked electrophysiological data has been previously investigated for use in cardiac mapping and navigation systems. However, dynamic cardiac 4D (3D + time) registration methods do not presently exist. This paper introduces two new 4D registration methods based on the popular iterative closest point (ICP) algorithm that may be applied to dynamic 3D shapes. The first method averages the transformations of the 3D ICP on each phase of the dynamic data, while the second finds the closest point pairs for the data in each phase and performs a least squares fit between all the pairs combined. Experimental results show these methods yield more accurate transformations compared to using a traditional 3D approach (4D errors: Translation 0.4mm, Rotation 0.45 degrees vs. 3D errors: Translation 1.2mm, Rotation 1.3 degrees) while also increasing capture range and success
Comparaison with other method on a synthetic sequence. Six 2D image sequences are represented as 100 × 100 × 100 volumes. First three quadrants correspond slice XY , T Y and XT . The last one is left empty. (a) original sequence used as the ground truth ; (b) artificially noisy sequence ; (c) spatial adaptive estimation [9], i.e. each image of the sequence is processed independently ; (d) 3D anisotropic diffusion [4] when the sequence is considered as a homogeneous 3D volume ; (e) adaptive estimation algorithm applied the sequence considered as a 3D volume ; (f) our 2D + t adaptive estimation (see text).  
Conference Paper
We present a spatio-temporal filtering method for significantly increasing the signal-to-noise ratio (SNR) in noisy fluorescence microscopic image sequences where small particles have to be tracked from frame to frame. Image sequence restoration is achieved using a statistical approach involving an appropriate on-line window geometry specification. We have applied this method to noisy synthetic and real microscopic image sequences where a large number of small fluorescently labeled vesicles are moving in regions close to the Golgi apparatus. The SNR is shown to be drastically improved and the enhanced vesicles can be segmented. This novel approach can be further exploited for biological studies where the dynamics of small objects of interest have to be analyzed in molecular and sub-cellular bio-imaging.
Conference Paper
We present a fully automated method to estimate the location and orientation of the left ventricle (LV) in four-dimensional (4D) cardiac magnetic resonance (CMR) images without any user input. The method is based on low-level image processing techniques incorporating anatomical knowledge and is able to provide rapid, robust feedback for automated scan planning or further processing. The method relies on a novel combination of temporal Fourier analysis of image cines with simple contour detection to achieve a fast localization of the heart. Quantitative validation was performed using 4D CMR datasets from 330 patients (54024 images) with a range of cardiac and vascular disease by comparing manual location with the automatic results. The method failed on one case, and showed average bias and precision of under 5mm in apical, mid-ventricular and basal slices in the remaining 329. The errors in automatic orientation were similar to the errors in scan planning as performed by experienced technicians.
Conference Paper
We propose a novel framework to predict pacing sites in the left ventricle (LV) of a heart and its result can be used to assist pacemaker implantation and programming in cardiac resynchronization therapy (CRT), a widely adopted therapy for heart failure patients. In a traditional CRT device deployment, pacing sites are selected without quantitative prediction. That runs the risk of suboptimal benefits. In this work, the spherical harmonic (SPHARM) description is employed to model the ventricular surfaces and a novel SPHARM-based surface correspondence approach is proposed to capture the ventricular wall motion. A hierarchical agglomerative clustering technique is applied to the time series of regional wall thickness to identify candidate pacing sites. Using clinical MRI data in our experiments, we demonstrate that the proposed framework can not only effectively identify suitable pacing sites, but also distinguish patients from normal subjects perfectly to help medical diagnosis and prognosis.
Conference Paper
We develop a 4D (3D plus time) statistical shape model for implicit level set based shape representations. To this end, we represent hand segmented training sequences of the left ventricle by respective 4-dimensional embedding functions and approximate these by a principal component analysis. In contrast to recent 4D models on explicit shape representations, the implicit shape model developed in this work does not require the computation of point correspondences which is known to be quite challenging, especially in higher dimensions. Experimental results on the segmentation of SPECT sequences of the left myocardium confirm that the 4D shape model outperforms respective 3D models, because it takes into account a statistical model of the temporal shape evolution.
Conference Paper
Assessment of soft tissue in normal and abnormal joint motion today gets feasible by acquiring time series of 3D MRI images. However, slice-by-slice viewing of such 4D kinematic images is cumbersome, and does not allow appreciating the movement in a convenient way. Simply presenting slice data in a cine-loop will be compromised by through-plane displacements of anatomy and "jerks" between frames, both of which hamper visual analysis of the movement. To overcome these limitations, we have implemented a demonstrator for viewing 4D kinematic MRI datasets. It allows to view any user defined anatomical structure from any viewing perspective in real-time. Smoothly displaying the movement in a cine-loop is realized by image post processing, fixing any user defined anatomical structure after image acquisition.
Conference Paper
A 4D image registration method is proposed for consistent estimation of cardiac motion from MR image sequences. Under this 4D registration framework, all 3D cardiac images taken at different time-points are registered simultaneously, and motion estimated is enforced to be spatiotemporally smooth, thereby overcoming potential limitations of some methods that typically estimate cardiac deformation sequentially from one frame to another, instead of treating the entire set of images as a 4D volume. To facilitate our image matching process, an attribute vector is designed for each point in the image to include intensity, boundary and geometric moment invariants (GMIs). Hierarchical registration of two image sequences is achieved by using the most distinctive points for initial registration of two sequences and gradually adding less-distinctive points for refinement of registration. Experimental results on real data demonstrate good performance of the proposed method in registering cardiac images and estimating motions from cardiac image sequences.
Conference Paper
In this paper we present a novel method for building a 4D statistical atlas describing the cardiac anatomy and how the cardiac anatomy changes during the cardiac cycle. The method divides the distribution space of cardiac shapes into two subspaces. One distribution subspace accounts for changes in cardiac shape caused by inter-subject variability. The second distribution subspace accounts for changes in cardiac shape caused by deformation during the cardiac cycle (i.e. intra-subject variability). Principal component analysis (PCA) have been performed in order to calculate the most significant modes of variation of each distribution subspace. During the construction of the statistical atlas we eliminate the need for manual landmarking of the cardiac images by using a non-rigid surface registration algorithm to propagate a set of pseudo-landmarks from an automatically landmarked atlas to each frame of all the image sequences. In order to build the atlas we have used 26 cardiac image sequences from healthy volunteers. We show how the resulting statistical atlas can be used to differentiate between cardiac image sequences from patients with hypertrophic cardiomyopathy and normal subjects.
Conference Paper
Real-time three-dimensional echocardiography (RT3DE) offers an efficient way to obtain complete 3D images of the heart over an entire cardiac cycle in just a few seconds. The complex 3D wall motion and temporal information contained in these 4D data sequences has the potential to enhance and supplement other imaging modalities for clinical diagnoses based on cardiac motion analysis. In our previous work, a 4D optical flow based method was developed to estimate dynamic cardiac metrics, including strains and displacements, from 4D ultrasound. In this study, in order to evaluate the ability of our method in detecting ischemic regions, coronary artery occlusion experiments at various locations were performed on five dogs. 4D ultrasound data acquired during these experiments were analyzed with our proposed method. Ischemic regions predicted by the outcome of strain measurements were compared to predictions from cardiac physiology with strong agreement. This is the first direct validation study of our image analysis method for clinical diagnoses and outcome.
Top-cited authors
Dan Boneh
  • Stanford University
Cynthia Dwork
  • Microsoft
Kobbi Nissim
  • Georgetown University
Wil Van der Aalst
  • RWTH Aachen University
Pascal Paillier
  • CryptoExperts