[Show abstract][Hide abstract] ABSTRACT: It is a challenging task to develop robust object tracking methods to overcome dynamic object appearance and background changes. Online learning-based methods have been widely applied to cope with the challenges. However, online methods suffer from the problem of drifting. Sparse appearance representation has recently shown promising object tracking results. However, it lacks of information update to accurately track objects in long sequences or when object appearance changes. In this paper, we propose a novel framework for tracking objects using a semi-supervised appearance dictionary learning method. Firstly, an object appearance dictionary is learned on the initial frame. Secondly, a graph model is employed in the proposed method for learning new bases when detecting object appearance change. The selected bases automatically replace the current rarely used bases. The proposed method is quantitatively compared with state-of-the-art methods on several challenging data sets. Results have shown that our proposed framework outperforms other methods even when drastic appearance variations happen.
[Show abstract][Hide abstract] ABSTRACT: Recently, techniques for the automatic detection or tracking of surgical instruments in X-ray guided computer-assisted interventions have emerged. The purposes of these methods are to facilitate inter-modality registration, motion compensation, enhanced visualization or automatic landmark generation in augmented-reality applications. Most techniques incorporate a model of the device as prior information to evaluate results obtained from a low-level detector. In this paper, we present novel approaches which are able to generate both 2-D and 3-D models of circular and linear catheters from biplane X-ray images with only minimal user input. We apply these methods in the context of Electrophysiology to generate models of ablation and mapping catheters. An evaluation on clinical data sets yielded promising results.
[Show abstract][Hide abstract] ABSTRACT: Medical image processing tools are playing an increasingly important role in assisting the clinicians in diagnosis, therapy planning and image-guided interventions. Accurate, robust and fast tracking of deformable anatomical objects, such as the heart, is a crucial task in medical image analysis. One of the main challenges is to maintain an anatomically consistent representation of target appearance that is robust enough to cope with inherent changes due to target movement, imaging device movement, varying imaging conditions, and is consistent with the domain expert clinical knowledge. To address these challenges, this chapter presents a probabilistic framework that relies on anatomically indexed component-based object models which integrate several sources of information to determine the temporal trajectory of the deformable target. Large annotated imaging databases are exploited to encode the domain knowledge in shape models and motion models and to learn discriminative image classifiers for the target appearance. The framework robustly fuses the prior information with traditional tracking approaches based on template matching and registration. We demonstrate various medical image analysis applications with focus on cardiology such as 2D auto left heart, catheter detection and tracking, 3D cardiac chambers surface tracking, and 4D complex cardiac structure tracking, in multiple modalities including Ultrasound (US), cardiac Computed Tomography (CT), Magnetic Resonance Imaging (MRI), and X-ray fluoroscopy.
[Show abstract][Hide abstract] ABSTRACT: A fully automatic framework is proposed to identify consistent landmarks and wire structures in a rotational X-ray scan. In our application, we localize the balloon marker pair and the guidewire in between the marker pair on each projection angle from a rotational fluoroscopic sequence. We present an effective offline balloon marker tracking algorithm that leverages learning based detectors and employs the Viterbi algorithm to track the balloon markers in a globally optimal manner. Localizing the guidewire in between the tracked markers is formulated as tracking the middle control point of the spline fitting the guidewire. The experimental studies demonstrate that our methods achieve a marker tracking accuracy of 96.33% and a mean guidewire localization error of 0.46 mm, suggesting a great potential of our methods for clinical applications. The proposed offline marker tracking method is also successfully applied to the problem of automatic self-initialization of generic online marker trackers for 2D live fluoroscopy stream, demonstrating a success rate of 95.9% on 318 sequences. Its potential applications also include localization of landmarks in a generic rotational scan.
[Show abstract][Hide abstract] ABSTRACT: Detailed visualization of stents during their positioning and deployment is critical for the success of an interventional procedure. This paper presents a novel method that relies on balloon markers to enable real-time enhanced visualization and assessment of the stent positioning and expansion, together with the blood flow over the lesion area. The key novelty is an automatic tracking framework that includes a self-initialization phase based on the Viterbi algorithm and an online tracking phase implementing the Bayesian fusion of multiple cues. The resulting motion compensation stabilizes the image of the stent and by compounding multiple frames we obtain a much better stent contrast. Robust results are obtained from more than 350 clinical data sets.
[Show abstract][Hide abstract] ABSTRACT: New minimal-invasive interventions such as transcatheter valve procedures exploit multiple imaging modalities to guide tools (fluoroscopy) and visualize soft tissue (transesophageal echocardiography (TEE)). Currently, these complementary modalities are visualized in separate coordinate systems and on separate monitors creating a challenging clinical workflow. This paper proposes a novel framework for fusing TEE and fluoroscopy by detecting the pose of the TEE probe in the fluoroscopic image. Probe pose detection is challenging in fluoroscopy and conventional computer vision techniques are not well suited. Current research requires manual initialization or the addition of fiducials. The main contribution of this paper is autonomous six DoF pose detection by combining discriminative learning techniques with a fast binary template library. The pose estimation problem is reformulated to incrementally detect pose parameters by exploiting natural invariances in the image. The theoretical contribution of this paper is validated on synthetic, phantom and in vivo data. The practical application of this technique is supported by accurate results (< 5 mm in-plane error) and computation time of 0.5s.
[Show abstract][Hide abstract] ABSTRACT: Electrophysiology (EP) procedures are conducted by cardiac specialists
to help diagnose and treat abnormal heart rhythms. Such procedures are
conducted under mono-plane and bi-plane x-ray fluoroscopy guidance to
allow the specialist to target ablation points within the heart.
Ablations lesions are usually set by applying radio-frequency energy to
endocardial tissue using catheters placed inside a patient's heart.
Recently we have developed a system capable of overlaying information
involving the heart and targeted ablation locations from pre-operational
image data for additional assistance. Although useful, such information
offers only approximate guidance due to heart beat and breathing motion.
As a solution to this problem, we propose to make use of a 2D lasso
catheter tracking method. We apply it to bi-plane fluoroscopy images to
dynamically update fluoro overlays. The dynamic overlays are computed at
3.5 frames per second to offer real-time updates matching the heart
motion. During the course of our experiments, we found an average 3-D
error of 1.6 mm on average. We present the workflow and features of the
motion-adjusted, augmented fluoroscopy system and demonstrate the
dramatic improvement in the overlay quality provided by this approach.
[Show abstract][Hide abstract] ABSTRACT: Catheter tracking in X-ray fluoroscopic images has become more important
in interventional applications for atrial fibrillation (AF) ablation
procedures. It provides real-time guidance for the physicians and can be
used as reference for motion compensation applications. In this paper,
we propose a novel approach to track a virtual electrode (VE), which is
a non-existing electrode on the coronary sinus (CS) catheter at a more
proximal location than any real electrodes. Successful tracking of the
VE can provide more accurate motion information than tracking of real
electrodes. To achieve VE tracking, we first model the CS catheter as a
set of electrodes which are detected by our previously published
learning-based approach.1 The tracked electrodes are then
used to generate the hypotheses for tracking the VE. Model-based
hypotheses are fused and evaluated by a Bayesian framework. Evaluation
has been conducted on a database of clinical AF ablation data including
challenging scenarios such as low signal-to-noise ratio (SNR), occlusion
and nonrigid deformation. Our approach obtains 0.54mm median error and
90% of evaluated data have errors less than 1.67mm. The speed of our
tracking algorithm reaches 6 frames-per-second on most data. Our study
on motion compensation shows that using the VE as reference provides a
good point to detect non-physiological catheter motion during the AF
No preview · Article · Feb 2012 · Proceedings of SPIE - The International Society for Optical Engineering
[Show abstract][Hide abstract] ABSTRACT: Catheter ablation is widely accepted as the best remaining option for the treatment of atrial fibrillation if drug therapy fails. Ablation procedures can be guided by 3-D overlay images projected onto live fluoroscopic X-ray images. These overlay images are generated from either MR, CT or C-Arm CT volumes. As the alignment of the overlay is often compromised by cardiac and respiratory motion, motion compensation methods are desirable. The most recent and promising approaches use either a catheter in the coronary sinus vein, or a circumferential mapping catheter placed at the ostium of one of the pulmonary veins. As both methods suffer from different problems, we propose a novel method to achieve motion compensation for fluoroscopy guided cardiac ablation procedures. Our new method localizes the coronary sinus catheter. Based on this information, we estimate the position of the circumferential mapping catheter. As the mapping catheter is placed at the site of ablation, it provides a good surrogate for respiratory and cardiac motion. To correlate the motion of both catheters, our method includes a training phase in which both catheters are tracked together. The training information is then used to estimate the cardiac and respiratory motion of the left atrium by observing the coronary sinus catheter only. The approach yields an average 2-D estimation error of 1.99 ± 1.20 mm.
[Show abstract][Hide abstract] ABSTRACT: The accurate and robust tracking of catheters and transducers employed during image-guided coronary intervention is critical to improve the clinical workflow and procedure outcome. Image-based device detection and tracking methods are preferred due to the straightforward integration into existing medical equipments. In this paper, we present a novel computational framework for image-based device detection and tracking applied to the co-registration of angiography and intravascular ultrasound (IVUS), two modalities commonly used in interventional cardiology. The proposed system includes learning-based detections, model-based tracking, and registration using the geodesic distance. The system receives as input the selection of the coronary branch under investigation in a reference angiography image. During the subsequent pullback of the IVUS transducers, the system automatically tracks the position of the medical devices, including the IVUS transducers and guiding catheter tips, under fluoroscopy imaging. The localization of IVUS transducers and guiding catheter tips is used to continuously associate an IVUS imaging plane to the vessel branch under investigation. We validated the system on a set of 65 clinical cases, with high accuracy (mean errors less than 1.5mm) and robustness (98.46% success rate). To our knowledge, this is the first reported system able to automatically establish a robust correspondence between the angiography and IVUS images, thus providing clinicians with a comprehensive view of the coronaries.
[Show abstract][Hide abstract] ABSTRACT: 2D X-ray fluoroscopy is widely used in computer assisted and image guided interventions because of the real time visual guidance it can provide to the physicians. During cardiac interventions, acquisitions of angiography are often used to assist the physician in visualizing the blood vessel structures, guide wires, or catheters, localizing bifurcations, estimating severity of a lesion, or observing the blood flow. Computational algorithms often need to process differently to frames with or without contrast medium. In order to automate this process and streamline the clinical workflow, a fully automatic contrast inflow detection algorithm is proposed. The robustness of the algorithm is validated by more than 1300 real fluoroscopic scenes. The algorithm is computationally efficient; a sequence with 100 frames can be processed within a second.
[Show abstract][Hide abstract] ABSTRACT: Catheter tracking has become more and more important in recent interventional applications. It provides real time navigation for the physicians and can be used to control a motion compensated fluoro overlay reference image for other means of guidance, e.g. involving a 3D anatomical model. Tracking the coronary sinus (CS) catheter is effective to compensate respiratory and cardiac motion for 3D overlay navigation to assist positioning the ablation catheter in Atrial Fibrillation (Afib) treatments. During interventions, the CS catheter performs rapid motion and non-rigid deformation due to the beating heart and respiration. In this paper, we model the CS catheter as a set of electrodes. Novelly designed hypotheses generated by a number of learning-based detectors are fused. Robust hypothesis matching through a Bayesian framework is then used to select the best hypothesis for each frame. As a result, our tracking method achieves very high robustness against challenging scenarios such as low SNR, occlusion, foreshortening, non-rigid deformation, as well as the catheter moving in and out of ROI. Quantitative evaluation has been conducted on a database of 13221 frames from 1073 sequences. Our approach obtains 0.50mm median error and 0.76mm mean error. 97.8% of evaluated data have errors less than 2.00mm. The speed of our tracking algorithm reaches 5 frames-per-second on most data sets. Our approach is not limited to the catheters inside the CS but can be extended to track other types of catheters, such as ablation catheters or circumferential mapping catheters.
[Show abstract][Hide abstract] ABSTRACT: Learning-based methods have been widely used in detecting landmarks or anatomical structures in various medical imaging applications. The performance of discriminative learning techniques has been demonstrated superior to traditional low-level filtering in robustness and scalability. Nevertheless, some structures and patterns are more difficult to be defined by such methods and complicated and ad-hoc methods still need to be used, e.g. a non-rigid and highly deformable wire structure. In this paper, we propose a novel scheme to train classifiers to detect the markers and guide wire segment anchored by markers. The classifier utilizes the markers as the end point and parameterizes the wire in-between them. The probabilities of the markers and the wire are integrated in a Bayesian framework. As a result, both the marker and the wire detection are improved by such a unified approach. Promising results are demonstrated by quantitative evaluation on 263 fluoroscopic sequences with 12495 frames. Our training scheme can further be generalized to localize longer guidewire with higher degrees of parameterization.
[Show abstract][Hide abstract] ABSTRACT: Catheter ablation of atrial fibrillation has become an accepted treatment option if a patient no longer responds to or tolerates drug therapy. A main goal is the electrical isolation of the pulmonary veins attached to the left atrium. Catheter ablation may be performed under fluoroscopic image guidance. Due to the rather low soft-tissue contrast of X-ray imaging, the heart is not visible in these images. To overcome this problem, overlay images from pre-operative 3-D volumetric data can be used to add anatomical detail. Unfortunately, this overlay is compromised by respiratory and cardiac motion. In the past, two methods have been proposed to perform motion compensation. The first approach involves tracking of a circumferential mapping catheter placed at an ostium of a pulmonary vein. The second method relies on a motion estimate obtained by localizing an electrode of the coronary sinus (CS) catheter. We propose a new motion compensation scheme which combines these two methods. The effectiveness of the proposed method is verified using 19 real clinical data sets. The motion in the fluoroscopic images was estimated with an overall average error of 0.55 mm by tracking the circumferential mapping catheter. By applying an algorithm involving both the CS catheter and the circumferential mapping catheter, we were able to detect motion of the mapping catheter from one pulmonary vein to another with a false positive rate of 5.8 %.
[Show abstract][Hide abstract] ABSTRACT: This chapter presents a framework of using computer vision and machine learning methods to tracking guidewire, a medical device
inserted into vessels during image guided interventions. During interventions, the guidewire exhibits nonrigid deformation
due to patients’ breathing and cardiac motions. Such 3D motions are complicated when being projected onto the 2D fluoroscopy.
Furthermore, fluoroscopic images have severe image artifacts and other wire-like structures. Those factors make robust guidewire
tracking a challenging problem. To address these challenges, this chapter presents a probabilistic framework for the purpose
of robust tracking. We introduce a semantic guidewire model that contains three parts, including a catheter tip, a guidewire
tip and a guidewire body. Measurements of different parts are integrated into a Bayesian framework as measurements of a whole
guidewire for robust guidewire tracking. For each part, two types of measurements, one from learning-based detectors and the
other from appearance models, are combined. Ahierarchical and multi-resolution tracking scheme based on kernel-based measurement
smoothing is then developed to track guidewires effectively and efficiently in a coarse-to-fine manner. The framework has
been validated on a testing set containing 47 sequences acquired under clinical environments, and achieves a mean tracking
error of less than 2 pixels.
[Show abstract][Hide abstract] ABSTRACT: An accurate and robust method to detect curve structures, such as a vessel branch or a guidewire, is essential for many medical imaging applications. A fully automatic method, although highly desired, is prone to detection errors that are caused by image noise and curve-like artifacts. In this paper, we present a novel method to interactively detect a curve structure in a 2D fluoroscopy image with a minimum requirement of human corrections. In this work, a learning based method is used to detect curve segments. Based on the detected segment candidates, a graph is built to search a curve structure as the best path passing through user interactions. Furthermore, our method introduces a novel hyper-graph based optimization method to allow for imposing geometric constraints during the path searching, and to provide a smooth and quickly converged result. With minimum human interactions involved, the method can provide accurate detection results, and has been used in different applications for guidewire and vessel detections.
[Show abstract][Hide abstract] ABSTRACT: In this paper, we present a method of using the needle detection and tracking to compensate breathing motion in 2D fluoroscopic videos. The method can robustly detect and tracking needles, even with the presence of image noises and large needle movements. The method first introduces an offline learned needle segment detector that detects needle segments at individual frames. Based on detected needle segments, a needle is interactively detected at the beginning of an intervention, and then is automatically tracked based on a probabilistic tracking framework. A multi-resolution kernel density estimation is applied to handle large needle movements efficiently and effectively. Experiments on phantom and clinical sequences demonstrate that the method can successfully track needles in fluoroscopy, and can provide motion compensation for abdominal interventions.
[Show abstract][Hide abstract] ABSTRACT: In this paper, we present a novel probabilistic framework for automatic follicle quantification in 3D ultrasound data. The proposed framework robustly estimates size and location of each individual ovarian follicle by fusing the information from both global and local context. Follicle candidates at detected locations are then segmented by a novel database guided segmentation method. To efficiently search hypothesis in a high dimensional space for multiple object detection, a clustered marginal space learning approach is introduced. Extensive evaluations conducted on 501 volumes containing 8108 follicles showed that our method is able to detect and segment ovarian follicles with high robustness and accuracy. It is also much faster than the current ultrasound manual workflow. The proposed method is able to streamline the clinical workflow and improve the accuracy of existing follicular measurements.
[Show abstract][Hide abstract] ABSTRACT: This paper presents a new technique of coronary digital subtraction angiography which separates layers of moving background structures from dynamic fluoroscopic sequences of the heart and obtains moving layers of coronary arteries. A Bayeisan framework combines dense motion estimation, uncertainty propagation and statistical fusion to achieve reliable background layer estimation and motion compensation for coronary sequences. Encouraging results have been achieved on clinically acquired coronary sequences, where the proposed method considerably improves the visibility and perceptibility of coronary arteries undergoing breathing and cardiac movements. Perceptibility improvement is significant especially for very thin vessels. Clinical benefit is expected in the context of obese patients and deep angulation, as well as in the reduction of contrast dose in normal size patients.