[Show abstract][Hide abstract] ABSTRACT: In this paper, a novel Markov random field (MRF)-based approach is presented for segmenting medical images while simultaneously registering an atlas nonrigidly. In the literature, both segmentation and registration have been studied extensively. For applications that involve both, such as segmentation via atlas-based registration, earlier studies proposed addressing these problems iteratively by feeding the output of each to initialize the other. This scheme, however, cannot guarantee an optimal solution for the combined task at hand, since these two individual problems are then treated separately. In this paper, we formulate simultaneous registration and segmentation (SRS) as a maximum a-posteriori (MAP) problem. We decompose the resulting probabilities such that the MAP inference can be done using MRFs. An efficient hierarchical implementation is employed, allowing coarse-to-fine registration while estimating segmentation at pixel level. The method is evaluated on two clinical data sets: 1) mandibular bone segmentation in 3D CT and 2) corpus callosum segmentation in 2D midsaggital slices of brain MRI. A video tracking example is also given. Our implementation allows us to directly compare the proposed method with the individual segmentation/registration and the iterative approach using the exact same potential functions. In a leave-one-out evaluation, SRS demonstrated more accurate results in terms of dice overlap and surface distance metrics for both data sets. We also show quantitatively that the SRS method is less sensitive to the errors in the registration as opposed to the iterative approach.
[Show abstract][Hide abstract] ABSTRACT: This paper studies improving joint segmentation and registration by introducing auxiliary labels for anatomy that has similar appearance to the target anatomy while not being part of that target. Such auxiliary labels help avoid false positive labelling of non-target anatomy by resolving ambiguity. A known registration of a segmented atlas can help identify where a target segmentation should lie. Conversely, segmentations of anatomy in two images can help them be better registered. Joint segmentation and registration is then a method that can leverage information from both registration and segmentation to help one another. It has received increasing attention recently in the literature. Often, merely a single organ of interest is labelled in the atlas. In the presense of other anatomical structures with similar appearance, this leads to ambiguity in intensity based segmentation; for example, when segmenting individual bones in CT images where other bones share the same intensity profile. To alleviate this problem, we introduce automatic generation of additional labels in atlas segmentations, by marking similar-appearance non-target anatomy with an auxiliary label. Information from the auxiliary-labeled atlas segmentation is then incorporated by using a novel coherence potential, which penalizes differences between the deformed atlas segmentation and the target segmentation estimate. We validated this on a joint segmentation-registration approach that iteratively alternates between registering an atlas and segmenting the target image to find a final anatomical segmentation. The results show that automatic auxiliary labelling outperforms the same approach using a single label atlasses, for both mandibular bone segmentation in 3D-CT and corpus callosum segmentation in 2D-MRI.
[Show abstract][Hide abstract] ABSTRACT: Atlas-based segmentation is an essential component of computer aided planning for radiotherapy. Commercial products often have access to a large number of candidate images to be used as atlases and thus efficient mechanisms are necessitated to automatically retrieve suitable atlas images. In this study, we have first developed methods to extract global features from thoracic CT images. These include geometrical features based on both voxel intensities and the outlines of automatic approximate bone, lung, and whole-body segmentations that can be calculated in seconds. Our goal is to study image retrieval techniques using these global image features, in particular investigating the feasibility of various supervised learning algorithms. Such retrieved images are then to be used as atlasses for the atlas-based segmentation of anatomy that cannot be segmented automatically such as lymph nodes.
Int Congress on Computer Assisted Radiology and Surgery (CARS), Heidelberg, Germany; 06/2013
[Show abstract][Hide abstract] ABSTRACT: Segmentation via atlas registration is a common technique in medical image analysis. Devising estimates of such segmentation outcome has been of interest in cases with multiple atlases, both for single-atlas selection and for multi-atlas fusion. This paper studies the estimation of expected Dice's similarity metric for registering atlas-target pairs, by employing registration loops with models of such metric (error) accumulation over these loops. In this framework, the use of registration information also from unsegmented images is proposed and is shown to outperform using segmented atlas images alone. We demonstrate a fast, memory-efficient implementation and single-atlas selection results using a CT and an MR dataset.
Biomedical Imaging (ISBI), 2013 IEEE 10th International Symposium on; 01/2013
[Show abstract][Hide abstract] ABSTRACT: Interaction with virtual deformable models is common in several haptic contexts, such as in medical training simulators. This paper presents a methodological procedure for the creation of such virtual models from their real-life counterparts. Both the surface geometry and the elastic parametrization of an object are reconstructed from position/force readings during an operator-assisted exploration of the object. A 3D mesh model is then generated from the surface contact points. The internal elastic modulus is found using the 3D finite element method. This modeling method is compared with two common 1D elastic models, namely Kelvin-Voigt and Hunt-Crossley. Results using three deformable homogeneous silicone samples show successful geometry reconstruction. 1D model parameterizations exhibit high variation dependent on geometry and contact location. In contrast, elastic modulus reconstruction yields a global model parameterization independent of geometry. Elastic moduli estimated in experiments correlated with their known values, and were shown to be reproducible among samples with different geometries.
[Show abstract][Hide abstract] ABSTRACT: A semi-supervised segmentation method using a single atlas is presented in this paper. Traditional atlas-based segmentation suffers from either a strong bias towards the selected atlas or the need for manual effort to create multiple atlas images. Similar to semi-supervised learning in computer vision, we study a method which exploits information contained in a set of unlabelled images by mutually registering them nonrigidly and propagating the single atlas segmentation over multiple such registration paths to each target. These multiple segmentation hypotheses are then fused by local weighting based on registration similarity. Our results on two datasets of different anatomies and image modalities, corpus callosum MR and mandible CT images, show a significant improvement in segmentation accuracy compared to traditional single atlas based segmentation. We also show that the bias towards the selected atlas is minimized using our method. Additionally, we devise a method for the selection of intermediate targets used for propagation, in order to reduce the number of necessary inter-target registrations without loss of final segmentation accuracy.
Medical Computer Vision. Recognition Techniques and Applications in Medical Imaging, Edited by Menze, BjoernH. and Langs, Georg and Lu, Le and Montillo, Albert and Tu, Zhuowen and Criminisi, Antonio, 01/2013: pages 29-37; Springer Berlin Heidelberg., ISBN: 9783642366192
[Show abstract][Hide abstract] ABSTRACT: The finite element method is commonly used to model tissue deformation in order to solve for unknown parameters in the inverse problem of viscoelasticity. Typically, a (regular-grid) structured mesh is used since the internal geometry of the domain to be identified is not known a priori. In this work, the generation of problem-specific meshes is studied and such meshes are shown to significantly improve inverse-problem elastic parameter reconstruction. Improved meshes are generated from axial strain images, which provide an approximation to the underlying structure, using an optimization-based mesh adaptation approach. Such strain-based adapted meshes fit the underlying geometry even at coarse mesh resolutions, therefore improving the effective resolution of the reconstruction at a given mesh size/complexity. Elasticity reconstructions are then performed iteratively using the reflective trust-region method for optimizing the fit between estimated and observed displacements. This approach is studied for Youngs modulus reconstruction at various mesh resolutions through simulations, yielding 40% to 72% decrease in root-mean-square reconstruction error and 4 to 52 times improvement in contrast-to-noise ratio in simulations of a numerical phantom with a circular inclusion. A noise study indicates that conventional structured meshes with no noise perform considerably worse than the proposed adapted meshes with noise levels up to 20% of the compression amplitude. A phantom study and preliminary in-vivo results from a breast tumor case confirm the benefit of the proposed technique. Not only conventional axial strain images but also other elasticity approximations can be used to adapt meshes. This is demonstrated on images generated by combining axial strain and axial-shear strain, which enhances lateral image contrast in particular settings, consequently further improving meshadapted reconstructions.
[Show abstract][Hide abstract] ABSTRACT: In this paper, a novel computer-based virtual training system for prostate brachytherapy is presented. This system incorporates, in a novel way, prior methodologies of ultrasound image synthesis and haptic transrectal ultrasound (TRUS) transducer interaction in a complete simulator that allows a trainee to maneuver the needle and the TRUS, to see the resulting patientspecific images and feel the interaction forces. The simulated TRUS images reflect the volumetric tissue deformation and comprise validated appearance models for the needle and implanted seeds. Rendered haptic forces use validated models for needle shaft flexure and friction, tip cutting, and deflection due to bevel. This paper also presents additional new features that make the simulator complete, in the sense that all aspects of the brachytherapy procedure as practiced at many cancer centers are simulated, including simulations of seed unloading, fluoroscopy imaging, and transversal/sagittal TRUS plane switching. For realtime rendering, methods for fast TRUS-needle-seed image formation are presented. In addition, the simulator computes realtime dosimetry, allowing a trainee to immediately see the consequence of planning changes. The simulation is also patientspecific, as it allows the user to import the treatment plan for a patient together with the imaging data in order for a physician to practice an upcoming procedure or for a medical resident to train using typical implant scenarios or rarely encountered cases.
IEEE transactions on bio-medical engineering 10/2012; · 2.15 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: A new method to model the stress–strain relationship in two dimensions is proposed, which is particularly suited for analyzing nearly incompressible materials, such as soft tissue. In most cases of soft tissue modeling, plane strain is reported to approximate the deformation when an external compression is applied. However, it is subject to limitations when dealing with incompressible materials, e.g., when solving the inverse problem of elasticity. We propose a novel 2D model for the linear stress–strain relationship by describing the out-of-plane strain as a linear combination of the two in-plane strains. As such, the model can be represented in 2D while being able to explain the three-dimensional deformation. We show that in simple cases where the applied force is dominantly in one direction, one can approximate the sum of the three principal strain components in a plane by a scalar multiplied by the out-of-plane strain. 3D finite-element simulations have been performed. The proposed model has been tested under different boundary conditions and material properties. The results show that the model parametrization is affected mostly by the boundary conditions, while being relatively independent of the underlying distribution of Young's modulus. An application to the inverse problem of elasticity is presented where a more accurate estimate is obtained using the proposed dilatation model compared to the plane-stress and plane-strain models.
Physics in Medicine and Biology 06/2012; 57(12):4055-73. · 2.92 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: The fusion of TransRectal Ultrasound (TRUS) and Magnetic Resonance (MR)
images of the prostate can aid diagnosis and treatment planning for
prostate cancer. Surface segmentations of the prostate are available in
both modalities. Our goal is to develop a 3D deformable registration
method based on these segmentations and a biomechanical model. The
segmented source volume is meshed and a linear finite element model is
created for it. This volume is deformed to the target image volume by
applying surface forces computed by assuming a negative relative
pressure between the non-overlapping regions of the volumes and the
overlapping ones. This pressure drives the model to increase the volume
overlap until the surfaces are aligned. We tested our algorithm on
prostate surfaces extracted from post-operative MR and TRUS images for
14 patients, using a model with elasticity parameters in the range
reported in the literature for the prostate. We used three evaluation
metrics for validating our technique: the Dice Similarity Coefficient
(DSC) (ideally equal to 1.0), which is a measure of volume alignment,
the volume change in source surface during registration, which is a
measure of volume preservation, and the distance between the urethras to
assess the anatomical correctness of the method. We obtained a DSC of
0.96+/-0.02 and a mean distance between the urethras of 1.5+/-1.4 mm.
The change in the volume of the source surface was 1.5+/-1.4%. Our
results show that this method is a promising tool for physicallybased
deformable surface registration.
[Show abstract][Hide abstract] ABSTRACT: We study displacement and strain measurement error of dual transducers (two linear arrays, aligned orthogonally and coplanar). Displacements along the beam of each transducer are used to obtain measurements in two-dimensions. Simulations (5MHz) and experiments (10MHz) are compared to measurements with a single linear array, with and without angular compounding. Translation simulations demonstrate factors of 1.07 larger and 8.0 smaller biases in the axial and lateral directions respectively, for dual transducers compared to angular compounding. As the angle between dual transducers decreases from 90° to 40°, for 1% compression simulations, the lateral RMS error ranges from 2.1 to 3.9μm compared to 9μm with angular compounding. Simulation of dual transducer misalignment of 1mm and 2° result in errors of less than 9μm. Experiments demonstrate factors of 3.0 and 5.2 lower biases for dual transducers in the axial and lateral directions respectively compared to angular compounding.
[Show abstract][Hide abstract] ABSTRACT: We have previously presented multi-dimensional sub-sample motion estimation techniques that use multi-dimensional polynomial fitting to the discrete cross-correlation function to jointly estimate the sub-sample motion in all three spatial directions. Previous simulation and experimental results showed that these estimators significantly improve the performance of the motion estimation in 2-D and 3-D. In this short communication, we present additional simulation results and compare these techniques to 2-D tracking using beam steering. The results show that beam steering technique performs better in estimating the motion vector especially the lateral component.
IEEE transactions on ultrasonics, ferroelectrics, and frequency control 08/2011; 58(8):1534-7. · 1.80 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: The characterization of tissue viscoelastic properties requires the measurement of tissue motion over a region of interest at frequencies that significantly exceed the frame rates of conventional ultrasound systems. In this paper, we propose that the bandpass sampling technique be applied to tissue motion sampling. With this approach, high-frequency signals limited to a frequency band can be sampled and reconstructed without aliasing at a sampling frequency that is lower than the Nyquist rate. We first review this approach and discuss the selection of the tissue excitation frequency band and of the feasible sampling frequencies that allow signal reconstruction without aliasing. We then demonstrate the approach using simulations based on the finite element method and ultrasound simulations. Finally, we perform experiments on tissue-mimicking materials and demonstrate accurate motion estimation using a lower sampling rate than that required by the conventional sampling theorem. The estimated displacements were used to measure the elasticity and viscosity in a phantom in which an inclusion has been correctly delineated. Thus, with bandpass sampling, it is feasible to use conventional beamforming on diagnostic ultrasound systems to perform high-frequency dynamic elastography. The method is simple to implement because it does not require beam interleaving, additional hardware, or synchronization.
IEEE transactions on ultrasonics, ferroelectrics, and frequency control 07/2011; 58(7):1332-43. · 1.80 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: This paper presents a haptic simulator for prostate brachytherapy. Both needle insertion and the manipulation of the transrectal ultrasound (TRUS) probe are controlled via haptic devices. Tissue interaction forces that are computed by a deformable tissue model based on the finite element method (FEM) are rendered to the user by these devices. The needle insertion simulation employs 3D models of needle flexibility and asymmetric tip bevel. The needle-tissue simulation allows a trainee to practice needle insertion and targeting. The TRUS-tissue interaction simulation allows a trainee to practice the 3D intraoperative TRUS placement for registration with the preoperative volume study and to practice TRUS axial translation and rotation for imaging needles during insertions. Approaches to computational acceleration for realtime haptic performance are presented. Trade-offs between accuracy and speed are discussed. A graphics-card implementation of the numerically intensive mesh-adaptation operation is also presented. The simulator can be used for training, rehearsal, and treatment planning.
IEEE Transactions on Haptics 05/2011; 4:188-198. · 2.03 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: This paper presents a haptic simulator for prostate brachytherapy. Both needle insertion and the manipulation of the transrectal ultrasound (TRUS) probe are controlled via haptic devices. These are used to render tissue interaction forces computed using a deformable tissue model based on the finite element method (FEM). Needle flexibility and lateral needle bevel forces are also simulated. The TRUS-tissue simulation allows a trainee to practice the 3D intra-operative placement of the TRUS probe for registration with the pre-operative volume study. The needle-tissue simulation allows a trainee to practice needle insertion and targeting. The TRUS probe and the needle can be maneuvered simultaneously. Approaches to computational acceleration for real-time haptic performance are presented. Trade-offs between accuracy and speed are discussed. A graphics-card implementation of the numerically intensive mesh-adaptation operation is also presented.
IEEE World Haptics Conference, WHC 2011, 21-24 June 2011, Istanbul, Turkey; 01/2011
[Show abstract][Hide abstract] ABSTRACT: In medical simulations involving tissue deformation, the finite element method (FEM) is a widely used technique, where the size, shape, and placement of the elements in a model are important factors that affect the interpolation and numerical errors of a solution. Conventional model generation schemes for FEM consist of a segmentation step delineating the anatomy followed by a meshing step generating elements conforming to this segmentation. In this paper, a single-step model generation technique is proposed based on optimization. Starting from an initial mesh covering the domain of interest, the mesh nodes are adjusted to minimize an objective function which penalizes intra-element intensity variations and poor element geometry for the entire mesh. Trade-offs between mesh geometry quality and intra-element variance are achieved by adjusting the relative weights of the geometric and intensity variation components of the cost function. This meshing approach enables a more accurate rendering of shapes with fewer elements and provides more accurate models for deformation simulation, especially when the image intensities represent a mechanical feature of the tissue such as the elastic modulus. The use of the proposed mesh optimization is demonstrated in 2-D and 3-D on synthetic phantoms, MR images of the brain, and CT images of the kidney. A comparison with previous meshing techniques that do not account for image intensity is also provided demonstrating the benefits of our approach.
IEEE transactions on medical imaging. 01/2011; 30(1):11-21.
[Show abstract][Hide abstract] ABSTRACT: A widely used time-domain technique for motion or delay estimation between digitized ultrasound RF signals involves the maximization of a discrete pattern-matching function, usually the cross-correlation. To achieve sub-sample accuracy, the discrete pattern-matching function is interpolated using the values at the discrete maximizer and adjacent samples. In prior work, only 1-D fit, applied separately along the axial, lateral, and elevational axes, has been used to estimate the sub-sample motion in 1-D, 2-D, and 3-D. In this paper, we explore the use of 2-D and 3-D polynomial fitting for this purpose. We quantify the estimation error in noise-free simulations using Field II and experiments with a commercial ultrasound machine. In simulated 2-D translational motions, function fitting with quartic spline polynomials leads to maximum bias of 0.2% of the sample spacing in the axial direction and 0.4% of the sample spacing in the lateral direction, corresponding to 38 nm and 1.31 μm, respectively. The maximum standard deviations were approximately 1% of the sample spacing in both the axial and the lateral directions, corresponding to 193 nm axially and 4.43 μm laterally. In simulated 1% axial strain, the same function fitting leads to mean absolute displacement estimation errors of 255 nm in the axial direction and 4.77 μm in the lateral direction. In experiments with a linear array transducer, 2-D quartic spline fitting leads to maximum bias of 458 nm and 6.27 μm in the axial and the lateral directions, respectively. These results are more than one order of magnitude smaller than those obtained with separate 1-D fit when applied to the same data set. Simulations and experiments in 3-D yield similar results when comparing 3-D polynomial fitting with 1-D fitting along the axial, lateral, and elevational directions.
IEEE transactions on ultrasonics, ferroelectrics, and frequency control 11/2010; 57(11):2403-20. · 1.80 Impact Factor
[Show abstract][Hide abstract] ABSTRACT: This paper presents a medical simulator for prostate brachytherapy procedure. Needles are inserted in deformable tissue models
using a haptic device while the force feedback computed using a needle-tissue interaction model is rendered on the user’s
hand. Transrectal ultrasound images of the region of interest are also displayed in real-time using an interpolation scheme
accounting for the mesh-based tissue deformation. Employing a 3D ultrasound volume data reconstructed a priori, this simulation method achieves realistic ultrasound feedback coupled with immediate tissue deformation. Models for simulating
tissue deformation using the finite element method are obtained by segmented the relevant anatomy on MR slices. These models
are rigidly registered to the ultrasound voxel volume using the prostate surface. The presented simulation system is suitable
for brachytherapy training using haptic control/feedback. It can also be used for treatment planning.
[Show abstract][Hide abstract] ABSTRACT: Soft tissue needle guidance and steering for clinical applications has been an active topic of research in the past decade. Although dynamic feedback control of needle insertion systems is expected to provide more accurate target tracking, it has received little attention due to the fact that most available models for needle-tissue interaction do not incorporate the dynamics of motions. In this paper, we study the controllability of rigid or flexible needles inside soft tissues using mechanical-based dynamic models. The results have significant implications on the design of suitable feedback controllers for different types of needle insertion systems.
Conference proceedings: ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conference 01/2010; 2010:2287-91.