No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
FAIR: Flexible Algorithms for Image Registration
Abstract
This book really shows how registration works: the flip-book appearing at the top right corners shows a registration of a human knee from bent to straight position (keeping bones rigid). Of course, the book also provides insight into concepts and practical tools. The presented framework exploits techniques from various fields such as image processing, numerical linear algebra, and optimization. Therefore, a brief overview of some preliminary literature in those fields is presented in the introduction (references [1–51]), and registration-specific literature is assembled at the end of the book (references [52–212]). Examples and results are based on the FAIR software, a package written in MATLAB. The FAIR software, as well as a PDF version of this entire book, can be freely downloaded from www.siam.org/books/fa06.
This book would not have been possible without the help of Bernd Fischer, Eldad Haber, Claudia Kremling, Jim Nagy, and the Safir Research Group from Lübeck: Sven Barendt, Björn Beuthin, Konstantin Ens, Stefan Heldmann, Sven Kabus, Janine Olesch, Nils Papenberg, Hanno Schumacher, and Stefan Wirtz. I'm also indebted to Sahar Alipour, Reza Heydarian, Raya Horesh, Ramin Mafi, and Bahram Marami Dizaji for improving the manuscript.
... The idea to exploit the problem structure for constructing a better approximation of ∇ 2 J (x k ) and use it as a seed matrix in L-BFGS appears in the numerical studies [1,2,17,35,38,40,49,50,54,66] that address a wide range of reallife problems. On the considered large-scale problems, the authors report significant speed ups over all methods that are used for comparison, including standard L-BFGS, Gauss-Newton, and truncated Newton. ...
... The 22 test cases, listed in Table 1, cover many different registration models, estimating small to large deformations. The three-dimensional (3D) lung CT images are from the well-known DIR dataset [20,41], and the rest of the datasets are from [54]; in Fig. 5 we display five of the datasets together with registration results. 2D-Disc images are an academic example with large deformations from [18]. ...
... The image processing operations are carried out matrixfree with the open-source image registration toolbox FAIR [54]. ...
We devise an L-BFGS method for optimization problems in which the objective is the sum of two functions, where the Hessian of the first function is computationally unavailable while the Hessian of the second function has a computationally available approximation that allows for cheap matrix–vector products. This is a prototypical setting for many inverse problems. The proposed L-BFGS method exploits the structure of the objective to construct a more accurate Hessian approximation than in standard L-BFGS. In contrast with existing works on structured L-BFGS, we choose the first part of the seed matrix, which approximates the Hessian of the first function, as a diagonal matrix rather than a multiple of the identity. We derive two suitable formulas for the coefficients of the diagonal matrix and show that this boosts performance on real-life image registration problems, which are highly non-convex inverse problems. The new method converges globally and linearly on non-convex problems under mild assumptions in a general Hilbert space setting, making it applicable to a broad class of inverse problems. An implementation of the method is freely available.
... In order to fully exploit the segmentation information and provide a better alignment of the deformable lung surface at different breathing states, we extend the model of [31] as follows. An additional deformable registration [32] of the lung segmentations is performed using again the SSD distance of lung masks together with curvature regularization [33] and a penalty on local volume change [34]. This mimics our intensity-driven registration framework as described in detail in Section II-D1 and yields an initial transformationŷ together with a prealigned moving imageM. ...
... The employed dense intensity-driven registration algorithm is based on the lung registration method from [9]. Here, a standard variational approach [32], [53], [54] is extended by two terms, one that improves lung boundary alignment [34] and one controlling local volume change, thereby restricting the transformation to meaningful volume change and guaranteeing invertibility, cf. also [55]. We augment the method by three elements: an extended pre-registration using nonlinear transformations (cf. ...
... 3) Numerical Optimization: The numerical optimization of the joint objective (11) is performed within the discretizeoptimize framework [32]. All components of the registration are discretized first, yielding a finite-dimensional optimization problem that can be solved using Newton-type methods. ...
We present a novel algorithm for the registration of pulmonary CT scans. Our method is designed for large respiratory motion by integrating sparse keypoint correspondences into a dense continuous optimization framework. The detection of keypoint correspondences enables robustness against large deformations by jointly optimizing over a large number of potential discrete displacements, whereas the dense continuous registration achieves subvoxel alignment with smooth transformations. Both steps are driven by the same normalized gradient fields data term. We employ curvature regularization and a volume change control mechanism to prevent foldings of the deformation grid and restrict the determinant of the Jacobian to physiologically meaningful values. Keypoint correspondences are integrated into the dense registration by a quadratic penalty with adaptively determined weight. Using a parallel matrix-free derivative calculation scheme, a runtime of about 5 min was realized on a standard PC. The proposed algorithm ranks first in the EMPIRE10 challenge on pulmonary image registration. Moreover, it achieves an average landmark distance of 0.82 mm on the DIR-Lab COPD database, thereby improving upon the state of the art in accuracy by 15%. Our algorithm is the first to reach the inter-observer variability in landmark annotation on this dataset.
... We follow the guidelines in [25], and consider images as continuously differentiable and compactly supported functions on a domain of interest, Ω ⊂ R d (typically, d = 2 or 3). We assume that the image attains values in a field F where F = R corresponds to real-valued and F = C to complex-valued images. ...
... This interpolation can be represented via a sparse matrix T (y(w)) ∈ R n×n determined by the transformation. For the examples in this paper, we use bilinear or trilinear interpolation corresponding to the dimension of the problem, but other alternatives are possible; see, e.g., [25] for alternatives. The transformed version of the discrete image x is then expressed as a matrix-vector product T (y(w))x. ...
... Next, we run numerical experiments using a relatively small two-dimensional super resolution problem. To construct a super resolution problem with known ground truth image and motion parameters, we use the 2D MRI dataset provided in FAIR [25] (original resolution 128 × 128) to generate 32 frames of low-resolution test data (resolution 32 × 32) after applying 2D rigid body transformations with randomly chosen parameters. Gaussian white noise is added using the formula ...
Many inverse problems involve two or more sets of variables that represent different physical quantities but are tightly coupled with each other. For example, image super-resolution requires joint estimation of the image and motion parameters from noisy measurements. Exploiting this structure is key for efficiently solving these large-scale optimization problems, which are often ill-conditioned. In this paper, we present a new method called Linearize And Project (LAP) that offers a flexible framework for solving inverse problems with coupled variables. LAP is most promising for cases when the subproblem corresponding to one of the variables is considerably easier to solve than the other. LAP is based on a Gauss-Newton method, and thus after linearizing the residual, it eliminates one block of variables through projection. Due to the linearization, this block can be chosen freely. Further, LAP supports direct, iterative, and hybrid regularization as well as constraints. Therefore LAP is attractive, e.g., for ill-posed imaging problems. These traits differentiate LAP from common alternatives for this type of problem such as variable projection (VarPro) and block coordinate descent (BCD). Our numerical experiments compare the performance of LAP to BCD and VarPro using three coupled problems whose forward operators are linear with respect to one block and nonlinear for the other set of variables.
... We consider the transport equation (assuming intensities are related at corresponding points) and the continuity equation (assuming that mass is preserved) as constraints. The connection to traditional image registration formulations [53,54] is that a sufficiently smooth velocity field v gives rise to a diffeomorphism y via the method of characteristics. Vice versa, representing diffeomorphisms through velocity fields has been used for efficient statistical analysis; see, e.g., [3]. ...
... • We derive a flexible framework supporting both stationary and non-stationary velocity fields. Our methods are embedded into the FAIR framework [54]. This allows us to consider different regularization norms and distance measures. ...
... We limit this review to work closely related to ours. For a general insight into the area of image registration, its applications, and its formulation we refer to [53,54,63]. Our work builds upon the LDDMM framework described in [23,65,9], which is based on the pioneering work on velocity-based fluid registration described in [20]. ...
We present an efficient solver for diffeomorphic image registration problems in the framework of Large Deformations Diffeomorphic Metric Mappings (LDDMM). We use an optimal control formulation, in which the velocity field of a hyperbolic PDE needs to be found such that the distance between the final state of the system (the transformed/transported template image) and the observation (the reference image) is minimized. Our solver supports both stationary and non-stationary (i.e., transient or time-dependent) velocity fields. As transformation models, we consider both the transport equation (assuming intensities are preserved during the deformation) and the continuity equation (assuming mass-preservation). We consider the reduced form of the optimal control problem and solve the resulting unconstrained optimization problem using a discretize-then-optimize approach. A key contribution is the elimination of the PDE constraint using a Lagrangian hyperbolic PDE solver. Lagrangian methods rely on the concept of characteristic curves that we approximate here using a fourth-order Runge-Kutta method. We also present an efficient algorithm for computing the derivatives of final state of the system with respect to the velocity field. This allows us to use fast Gauss-Newton based methods. We present quickly converging iterative linear solvers using spectral preconditioners that render the overall optimization efficient and scalable. Our method is embedded into the image registration framework FAIR and, thus, supports the most commonly used similarity measures and regularization functionals. We demonstrate the potential of our new approach using several synthetic and real world test problems with up to 14.7 million degrees of freedom.
... 1. Introduction. In the context of medical imaging, image registration is the problem of finding a reasonable transformation ⃗ y to align a template image T with a reference image R. In the optimization framework, we find a transformation by minimizing the sum of a distance function D[T • ⃗ y, R] and a regularization term [12]. Approaches to find optimal transformations in a parameterized set of admissible transformations can be found either by numerical optimization techniques, or by solving a nonlinear PDE derived from the optimality condition [5]. ...
... Approaches to find optimal transformations in a parameterized set of admissible transformations can be found either by numerical optimization techniques, or by solving a nonlinear PDE derived from the optimality condition [5]. Our strategy, following [12], is to parameterize the transformations by a vector and numerically solve for the optimal transformation as a finite-dimensional unconstrained optimization problem. ...
... The prediction step of the predictor-corrector method requires a derivative with respect to a scaling parameter in the interpolation, and a variable projection involves differentiation through an inexact iterative least squares solve. Much of the theory for these problems exists in the literature, but perhaps due to the difficulty of some required derivatives, they do not exist in standard image registration packages like FAIR [12]. The code and data required generate the figures used in this text and to reproduce results can be found at our GitHub repository [19]. ...
We demonstrate that automatic differentiation, which has become commonly available in machine learning frameworks, is an efficient way to explore ideas that lead to algorithmic improvement in multi-scale affine image registration and affine super-resolution problems. In our first experiment on multi-scale registration, we implement an ODE predictor-corrector method involving a derivative with respect to the scale parameter and the Hessian of an image registration objective function, both of which would be difficult to compute without AD. Our findings indicate that exact Hessians are necessary for the method to provide any benefits over a traditional multi-scale method; a Gauss-Newton Hessian approximation fails to provide such benefits. In our second experiment, we implement a variable projected Gauss-Newton method for super-resolution and use AD to differentiate through the iteratively computed projection, a method previously unaddressed in the literature. We show that Jacobians obtained without differentiating through the projection are poor approximations to the true Jacobians of the variable projected forward map and explore the performance of some other approximations. By addressing these problems, this work contributes to the application of AD in image registration and sets a precedent for further use of machine learning tools in this field.
... Many types of \scrS (\bfu ) have been proposed, for instance, total variation (TV) [8], mean curvature [9], Gaussian curvature [26], and fractional-order T V \alpha -L 2 [52]. For the fidelity \scrD (\cdot , \cdot ), one may also have different choices [3,7,21, 22,31,39,40] according to whether the image pair T (\cdot ) and R(\cdot ) are produced by the same imaging technique. Generally, if T (\cdot ) and R(\cdot ) are of the same type (i.e., CT-CT, MRI-MRI), it is called monomodality image registration. ...
... In this case, the intensity difference based SSD no longer works, since the intensity in different modalities is of different physical meanings. To address this problem, some other fidelities are proposed to characterize the similarity of multimodality image pairs, for example, mutual information (MI) [17], the maximum correlation coefficient (MCC) [7], the normalized cross-correlation (NCC) [31], and the normalized gradient fields (NGF) [17]. These fidelities can be classified into two categories: geometric feature based fidelity (GFBF) and intensity based fidelity (IBF). ...
... The pioneering work for IBF comes from Maes et al. [29], who modeled the fidelity using MI between the image pairs. Since then, many other fidelities for multimodality image registration have been introduced, for example, MCC [7] and NCC [31]. Though there are many different fidelities, MI is considered the state-of-the-art fidelity In physical view, det(\nabla \bfitvar (\bfx )) denotes the volume stretching rate for the transformation \bfitvar : \Omega \rightar \Omega . ...
... The idea to exploit the problem structure for constructing a better approximation of ∇ 2 J (x k ) and use it as a seed matrix in L-BFGS appears in the numerical studies [29,26,43,30,1,53,2,38,14,39] that address a wide range of real-life problems. On the considered large-scale problems, the authors report signiĄcant speed ups over all methods that are used for comparison, including standard L-BFGS, GaussŰNewton and truncated Newton. ...
... The image processing operations are carried out matrix free with the open-source image registration toolbox FAIR [43]. For the stopping criteria of the optimization methods we follow [43, p. 78]. ...
... We consistently use ℓ = 5 in ROSE. We employ the Armijo line search routine from FAIR [43] with parameters LSmaxIter = 50 and LSreduction = 10 −4 . We do not consider the WolfeŰPowell line search because it does not work as well on image registration problems [39]. ...
We devise an L-BFGS method for optimization problems in which the objective is the sum of two functions, where the Hessian of the first function is computationally unavailable while the Hessian of the second function has a computationally available approximation that allows for cheap matrix-vector products. This is a prototypical setting for many inverse problems. The proposed L-BFGS method exploits the structure of the objective to construct a more accurate Hessian approximation than in standard L-BFGS. In contrast to existing works on structured L-BFGS, we choose the first part of the seed matrix, which approximates the Hessian of the first function, as a diagonal matrix rather than a multiple of the identity. We derive two suitable formulas for the coefficients of the diagonal matrix and show that this boosts performance on real-life image registration problems, which are highly non-convex inverse problems. The new method converges globally and linearly on non-convex problems under mild assumptions in a general Hilbert space setting, making it applicable to a broad class of inverse problems. An implementation of the method is freely available.
Mathematics Subject Classification 65J22 · 65K05 · 65K10 · 90C06 · 90C26 · 90C30 · 90C48 · 90C53 · 90C90
... The main motivation for this paper arose from the need to improve the performance of L-BFGS on image registration problems [Mod09]. These problems are large-scale, highly nonconvex inverse problems in which the structure (1) is ubiquitous. ...
... These problems are large-scale, highly nonconvex inverse problems in which the structure (1) is ubiquitous. In this context, the results of this paper cover regularizers such as L 2 -norm based Tikhonov regularizers [Mod09,Han10], smooth total-variation norm [Vog02], quadratic forms of derivative based regularization, where S(x) = Bx 2 L2 with a linear differential operator B, and the non-quadratic and highly nonconvex hyperelastic regularizer [BMR13]. However, we stress that the convergence analysis of this paper is very general and not restricted to inverse problems. ...
... Still, in view of the additional linear algebra costs, the question arises whether using (4) can actually lower the run-time of L-BFGS. While the answer to this question is problem-dependent in general, extensive numerical evidence clearly shows that using a better approximation of the Hessian ∇ 2 J (x k ) as seed matrix can be vastly superior to the standard choice B (0) k = τ k I for large-scale realworld problems, see [JBES04,Hel06,Mod09,KŘ13,ACG18,YGJ18,AM21,BDLP21]. In this paper we observe the same effect in the structured setting (1) for choices of the form (4). ...
Many inverse problems are phrased as optimization problems in which the objective function is the sum of a data-fidelity term and a regularization. Often, the Hessian of the fidelity term is computationally unavailable while the Hessian of the regularizer allows for cheap matrix-vector products. In this paper, we study an L-BFGS method that takes advantage of this structure. We show that the method converges globally without convexity assumptions and that the convergence is linear under a Kurdyka–Łojasiewicz-type inequality. In addition, we prove linear convergence to cluster points near which the objective function is strongly convex. To the best of our knowledge, this is the first time that linear convergence of an L-BFGS method is established in a non-convex setting. The convergence analysis is carried out in infinite dimensional Hilbert space, which is appropriate for inverse problems but has not been done before. Numerical results show that the new method outperforms other structured L-BFGS methods and classical L-BFGS on non-convex real-life problems from medical image registration. It also compares favorably with classical L-BFGS on ill-conditioned quadratic model problems. An implementation of the method is freely available.
... For linear forward operators K , these approaches can be often extended to the indirect setting (9) without any complicated modifications. As Lagrangian approaches turned out to be very efficient for indirect image matching, we decided to adapt the methods developed in [23,24], which build upon the FAIR toolbox [30]. Since the problem is non-smooth due to the TV regularization of z, we cannot deploy their proposed Gauss-Newton-Krylov solver, and we use the iPALM algorithm [37] instead. ...
... 3. In order to approximate solutions of (10) numerically, we follow a discretize-then-optimize approach that involves the iPALM algorithm as outlined in Sect. 4. This allows one to easily exchange the regularizer for v and z if desired. Our implementation builds upon the FAIR toolbox [30], which allows for a simple extension to other distances and regularizers that are already implemented as part of the toolbox. Numerical results for the proposed model are provided in Sect. 5. Finally, conclusions are drawn in Sect. ...
... for all k = 0, . . . , N t − 1, see also [30]. Further, we require the proximal mappings of G 1 and G 2 . ...
The reconstruction of images from measured data is an increasing field of research. For highly under-determined problems, template-based image reconstruction provides a way of compensating for the lack of sufficient data. A caveat of this approach is that dealing with different topologies of the template and the target image is challenging. In this paper, we propose a LDDMM-based image-reconstruction model that resolves this issue by adding a source term. On the theoretical side, we show that the model satisfies all criteria for being a well-posed regularization method. For the implementation, we pursue a discretize-then-optimize approach involving the proximal alternating linearized minimization algorithm, which is known to converge under mild assumptions. Our simulations with both artificial and real data confirm the robustness of the method, and its ability to successfully deal with topology changes even if the available amount of data is very limited.
... We compare the two types of image pairs using the same optimization-based image registration method that is based on minimizing an energy functional consisting of a distance measure and a regularizer. 13 This class of optimization-based methods is widely used in medical imaging 14,15 and has also been applied to problems in pathology. 10,[16][17][18][19] These energy-minimizing methods make explicit model assumptions through the choice of distance measure and regularization scheme. ...
... It consists of (1) a robust prealignment, (2) an affine registration computed on coarse resolution images, and (3) a curvature-regularized deformable registration. The method is based on the variational image registration framework first described by Fischer and Modersitzki,13,27 which has been applied to many clinical fields from histology 28 to radiology. 29,30 A more detailed and formal description of the method can be found in Appendix A. ...
... Different resolution levels are commonly used in multilevel image pyramids to align images while avoiding convergence to local minima. 13 In theory, higher image resolution in two images showing the same structures should lead to a more accurate alignment, which is consistent with our observations in restained sections. To our knowledge, no systematic evaluation of the effect of increased image resolution has been done for registration in histopathology. ...
Significance:
Although the registration of restained sections allows nucleus-level alignment that enables a direct analysis of interacting biomarkers, consecutive sections only allow the transfer of region-level annotations. The latter can be achieved at low computational cost using coarser image resolutions.
Purpose:
In digital histopathology, virtual multistaining is important for diagnosis and biomarker research. Additionally, it provides accurate ground truth for various deep-learning tasks. Virtual multistaining can be obtained using different stains for consecutive sections or by restaining the same section. Both approaches require image registration to compensate for tissue deformations, but little attention has been devoted to comparing their accuracy.
Approach:
We compared affine and deformable variational image registration of consecutive and restained sections and analyzed the effect of the image resolution that influences accuracy and required computational resources. The registration was applied to the automatic nonrigid histological image registration (ANHIR) challenge data (230 consecutive slide pairs) and the hyperparameters were determined. Then without changing the parameters, the registration was applied to a newly published hybrid dataset of restained and consecutive sections (HyReCo, 86 slide pairs, 5404 landmarks).
Results:
We obtain a median landmark error after registration of 6.5 μm (HyReCo) and 24.1 μm (ANHIR) between consecutive sections. Between restained sections, the median registration error is 2.2 and 0.9 μm in the two subsets of the HyReCo dataset. We observe that deformable registration leads to lower landmark errors than affine registration in both cases (p<0.001), though the effect is smaller in restained sections.
Conclusion:
Deformable registration of consecutive and restained sections is a valuable tool for the joint analysis of different stains.
... In (bio)medical imaging applications, the automatic matching of k-dimensional deformable shapes across subjects or multi-temporal data of individual patients is a critical step to aid clinical diagnosis [56,78,87,126,128,148,155,168]. From a mathematical point of view, this matching problem constitutes an inverse problem [56]: Given two (or more) shapes s i ∈ S, i = 0, 1, in R d representing an object/anatomy of interest, defined in some shape space S, we seek a plausible spatial transformation y ∈ Y ad , Y ad ⊆ {ϕ | ϕ : R d → R d } with d ∈ {2, 3} that establishes a point-wise correspondence between these objects [126,128]. ...
... In (bio)medical imaging applications, the automatic matching of k-dimensional deformable shapes across subjects or multi-temporal data of individual patients is a critical step to aid clinical diagnosis [56,78,87,126,128,148,155,168]. From a mathematical point of view, this matching problem constitutes an inverse problem [56]: Given two (or more) shapes s i ∈ S, i = 0, 1, in R d representing an object/anatomy of interest, defined in some shape space S, we seek a plausible spatial transformation y ∈ Y ad , Y ad ⊆ {ϕ | ϕ : R d → R d } with d ∈ {2, 3} that establishes a point-wise correspondence between these objects [126,128]. Based on the problem specific notion of plausibility of an admissible transformation y, various continuum mechanical models emerged to restrict the space of admissible transformations Y ad . In a variational setting, this prior knowledge on admissible maps y is typically prescribed through regularization functionals or constraints. ...
... The preferred model typically depends on the application, of which there are many in medical imaging. Excellent reference works giving an overview of developments in this field are [56,78,87,126,128,148,168]. First, one may simply be interested in relating local information observed in two views of the same object. ...
We present formulations and numerical algorithms for solving diffeomorphic shape matching problems. We formulate shape matching as a variational problem governed by a dynamical system that models the flow of diffeomorphism . We overview our contributions in this area, and present an improved, matrix-free implementation of an operator splitting strategy for diffeomorphic shape matching. We showcase results for diffeomorphic shape matching of real clinical cardiac data in to assess the performance of our methodology.
... The differences between the two images can derive from different conditions, and for analyzing them we want to make images more similar each other after transformation, for an overview see e.g. [3,4,5,6,7,8]. In particular, in [8], Zitová and Flusser gave an overview of image registration, and presented different methods to solve this problem. ...
... Some methods for image registration are based on landmarks. The landmark-based image registration process is based on two finite sets of landmarks, i.e. scattered data points located on images, where each landmark of the source image has to be mapped onto the corresponding landmark of the target image (see [4,5,6]). The landmark-based registration problem can thus be formulated in the context of multivariate scattered data interpolation. ...
The purpose of image registration is to determine a transformation such that the transformed version of the source image is similar to the target one. In this paper we focus on landmark-based image registration using radial basis functions (RBFs) transformations, in particular on the topology preservation of compactly supported radial basis functions (CSRBFs) transformations. In [1] the performances of Gneiting's and Wu's functions are compared with the ones of other well known schemes in image registration, as thin plate spline and Wendland's functions. Several numerical experiments and real-life cases with medical images show differences in accuracy and smoothness of the considered interpolation methods, which can be explained taking into account their topology preservation properties. Here we analyze analytically and experimentally the topology preservation performances of Gneiting's functions, comparing results with the ones obtained in [2], where Wendland's and Wu's functions are considered.
... Our idea is based on the continuous interpretation of the forward propagation in which the number of layers in the network corresponds to the number of discretization points. Our idea is closely related to cascadic multigrid methods [8] and ideas in image processing where multi-level strategies are commonly used to decrease the risk of being trapped in local minima; see, e.g., [39]. More details about multi-level methods in learning can be found in [25]. ...
... In our experiments, this approach has been a simple yet effective way to obtain good initializations for the learning problems. Our regularization methods are commonly used in imaging science, e.g., image registration [39], however, to the best of our knowledge not commonly employed in deep learning. Our numerical examples show that approximately solving the regularized learning problem yields works that generalize well even when the number of network parameters exceeds the number of training features. ...
Deep neural networks have become invaluable tools for supervised machine learning, e.g., classification of text or images. While often offering superior results over traditional techniques and successfully expressing complicated patterns in data, deep architectures are known to be challenging to design and train such that they generalize well to new data. Important issues with deep architectures are numerical instabilities in derivative-based learning algorithms commonly called exploding or vanishing gradients. In this paper we propose new forward propagation techniques inspired by systems of Ordinary Differential Equations (ODE) that overcome this challenge and lead to well-posed learning problems for arbitrarily deep networks. The backbone of our approach is our interpretation of deep learning as a parameter estimation problem of nonlinear dynamical systems. Given this formulation, we analyze stability and well-posedness of deep learning and use this new understanding to develop new network architectures. We relate the exploding and vanishing gradient phenomenon to the stability of the discrete ODE and present several strategies for stabilizing deep learning for very deep networks. While our new architectures restrict the solution space, several numerical experiments show their competitiveness with state-of-the-art networks.
... Registration between images with the same resolution observed by the same modality is a standard framework [25,26,34]. The second class of problems is registration between images observed using different modalities [35,36]. ...
... We deal with image analysis methodologies using numerical computation [54][55][56][57][58][59][60] in mathematical image analysis [34,[61][62][63]145]. The method allows us to deal with the algorithms using stencils, matrices and tensors [54-56, 59, 60]. ...
In this paper, we aim to clarify the statistical and geometric properties of linear resolution conversion for registration between different resolutions observed using the same modality. The pyramid transform is achieved by smoothing and downsampling. The dual operation of the pyramid transform is achieved by linear smoothing after upsampling. The rational-order pyramid transform is decomposed into upsampling for smoothing and the conventional integer-order pyramid transform. By controlling the ratio between upsampling for smoothing and downsampling in the pyramid transform, the rational-order pyramid transform is computed. The tensor expression of the multiway pyramid transform implies that the transform yields orthogonal base systems for any ratio of the rational pyramid transform. The numerical evaluation of the transform shows that the rational-order pyramid transform preserves the normalised distribution of greyscale in images.
... These approaches leverage the composite form of optimisation problem (2) to derive easy-to-solve sub-problems that return better local optima of the original problem. An incomplete list of example applications of composite optimisation problems in computer vision include super-resolution from raw time-of-flight data [31], image and point set registration [50], shape matching [6,16,74]. Composite optimisation is classically well studied for continuous problems [22,31,43,58]. ...
There is growing interest in solving computer vision problems such as mesh or point set alignment using Adiabatic Quantum Computing (AQC). Unfortunately, modern experimental AQC devices such as D-Wave only support Quadratic Unconstrained Binary Optimisation (QUBO) problems, which severely limits their applicability. This paper proposes a new way to overcome this limitation and introduces QuCOOP, an optimisation framework extending the scope of AQC to composite and binary-parametrised, possibly non-quadratic problems. The key idea of QuCOOP~is to iteratively approximate the original objective function by a sequel of local (intermediate) QUBO forms, whose binary parameters can be sampled on AQC devices. We experiment with quadratic assignment problems, shape matching and point set registration without knowing the correspondences in advance. Our approach achieves state-of-the-art results across multiple instances of tested problems.
... It is a challenging task but, yet, a useful one in diverse fields of computational sciences and engineering such astronomy, optics, biology, chemistry, medicine and remote sensing and particularly in medical imaging. For an overview of image registration methodology and approaches, we refer to [20,22,33,38,43]. Here, we focus on development of robust variational models for deformable image registration as in the related works of [9,12,15,24,31,32,48]. ...
Registration aligns features of two related images so that information can be compared and/or fused in order to highlight differences and complement information. In real life images where bias field is present, this undesirable artefact causes inhomogeneity of image intensities and hence leads to failure or loss of accuracy of registration models based on minimization of the differences of the two image intensities. Here, we propose a non-linear variational model for joint image intensity correction (illumination and translation) and registration and reformulate it in a game framework. While a non-potential game offers flexible reformulation and can lead to better fitting errors, proving the solution existence for a non-convex model is non-trivial. Here we establish an existence result using the Schauder’s fixed point theorem. To solve the model numerically, we use an alternating minimization algorithm in the discrete setting. Finally numerical results can show that the new model outperforms existing models.
... However, fine kernels make large displacements more expensive than small ones, and such over-parameterization will likely trap the optimization procedure in a local minimum, achieving a reasonable numerical solution that is qualitatively bad. To overcome such problem, hierarchical algorithms have been widely used in the field of image registration [3,20]: after solving the registration problem at coarse scales, the solution is transferred to increasingly fine scales to refine the transformation. These strategies avoid more efficiently trapping the algorithm in local minima related to unrealistic transformations. ...
Estimating accurate high-dimensional transformations remains very challenging, especially in a clinical setting. In this paper, we introduce a multiscale parameterization of deformations to enhance registration and atlas estimation in the large deformation diffeomorphic metric mapping framework. Using the Haar wavelet transform, a multiscale representation of the initial velocity fields is computed to optimize transformations in a coarse-to-fine fashion. This additional layer of spatial regularization does not modify the underlying model of deformations. As such, it preserves the original kernel Hilbert space structure of the velocity fields, enabling the algorithm to perform efficient gradient descent. Numerical experiments on several datasets, including abnormal fetal brain images, show that compared to the original algorithm, the coarse-to-fine strategy reaches higher performance and yields template images that preserve important details while avoiding unrealistic features. This highly versatile strategy can easily be applied to other mathematical frameworks for almost no additional computational cost.
... We also specify a maximal number of Newton iterations (maxit N,T and maxit N,R ) and a lower bound of 1E−6 for the absolute norm of the gradient as a safeguard against a prohibitively high number of iterations. Details for the stopping conditions can be found in (Mang and Biros, 2015;Modersitzki, 2009); see (Gill et al., 1981, 305 ff.) for a discussion. ...
We present the SIBIA (Scalable Integrated Biophysics-based Image Analysis) framework for joint image registration and biophysical inversion and we apply it to analyse MR images of glioblastomas (primary brain tumors). In particular, we consider the following problem. Given the segmentation of a normal brain MRI and the segmentation of a cancer patient MRI, we wish to determine tumor growth parameters and a registration map so that if we "grow a tumor" (using our tumor model) in the normal segmented image and then register it to the patient segmented image, then the registration mismatch is as small as possible. We call this "the coupled problem" because it two-way couples the biophysical inversion and registration problems. In the image registration step we solve a large-deformation diffeomorphic registration problem parameterized by an Eulerian velocity field. In the biophysical inversion step we estimate parameters in a reaction-diffusion tumor growth model that is formulated as a partial differential equation. In this paper, our contributions are the introduction of a PDE-constrained optimization formulation of the coupled problem, the derivation of the optimality conditions, and the derivation of a Picard iterative scheme for the solution of the coupled problem. In addition, we perform several tests to experimentally assess the performance of our method on synthetic and clinical datasets. We demonstrate the convergence of the SIBIA optimization solver in different usage scenarios. We demonstrate that we can accurately solve the coupled problem in three dimensions ( resolution) in a few minutes using 11 dual-x86 nodes. Also, we demonstrate that, with our coupled approach, we can successfully register normal MRI to tumor-bearing MRI while obtaining Dice coefficients that match those achieved when registering of normal-to-normal MRI.
... There exists a rich literature on registration problems for images with values in the Euclidean space, see e.g. [11,16,18,19] and, for an overview, the books of Modersitzki [26,27]. ...
This paper addresses the morphing of manifold-valued images based on the time discrete geodesic paths model of Berkels, Effland and Rumpf 2015. Although for our manifold-valued setting such an interpretation of the energy functional is not available so far, the model is interesting on its own. We prove the existence of a minimizing sequence within the set of images having values in a finite dimensional Hadamard manifold together with a minimizing sequence of admissible diffeomorphisms. To this end, we show that the continuous manifold-valued functions are dense in . We propose a space discrete model based on a finite difference approach on staggered grids, where we focus on the linearized elastic potential in the regularizing term. The numerical minimization alternates between i) the computation of a deformation sequence between given images via the parallel solution of certain registration problems for manifold-valued images, and ii) the computation of an image sequence with fixed first (template) and last (reference) frame based on a given sequence of deformations via the solution of a system of equations arising from the corresponding Euler-Lagrange equation. Numerical examples give a proof of the concept of our ideas.
... This continuous data model has led to solid mathematical theories for classical data processing tasks obtained by leveraging the rich results from PDEs and variational calculus (e.g., [43]). The continuous perspective has also enabled more abstract formulations that are independent of the actual resolution, which has been exploited to obtain efficient multiscale and multilevel algorithms (e.g., [34]). ...
Partial differential equations (PDEs) are indispensable for modeling many physical phenomena and also commonly used for solving image processing tasks. In the latter area, PDE-based approaches interpret image data as discretizations of multivariate functions and the output of image processing algorithms as solutions to certain PDEs. Posing image processing problems in the infinite dimensional setting provides powerful tools for their analysis and solution. Over the last few decades, the reinterpretation of classical image processing problems through the PDE lens has been creating multiple celebrated approaches that benefit a vast area of tasks including image segmentation, denoising, registration, and reconstruction. In this paper, we establish a new PDE-interpretation of a class of deep convolutional neural networks (CNN) that are commonly used to learn from speech, image, and video data. Our interpretation includes convolution residual neural networks (ResNet), which are among the most promising approaches for tasks such as image classification having improved the state-of-the-art performance in prestigious benchmark challenges. Despite their recent successes, deep ResNets still face some critical challenges associated with their design, immense computational costs and memory requirements, and lack of understanding of their reasoning. Guided by well-established PDE theory, we derive three new ResNet architectures that fall into two new classes: parabolic and hyperbolic CNNs. We demonstrate how PDE theory can provide new insights and algorithms for deep learning and demonstrate the competitiveness of three new CNN architectures using numerical experiments.
... Note that the number of layers in the network corresponds to the number of discretization points in the discrete forward propagation. Similar ideas have been used in multigrid [4] and image processing; see, e.g., [15]. ...
In this work we establish the relation between optimal control and training deep Convolution Neural Networks (CNNs). We show that the forward propagation in CNNs can be interpreted as a time-dependent nonlinear differential equation and learning as controlling the parameters of the differential equation such that the network approximates the data-label relation for given training data. Using this continuous interpretation we derive two new methods to scale CNNs with respect to two different dimensions. The first class of multiscale methods connects low-resolution and high-resolution data through prolongation and restriction of CNN parameters. We demonstrate that this enables classifying high-resolution images using CNNs trained with low-resolution images and vice versa and warm-starting the learning process. The second class of multiscale methods connects shallow and deep networks and leads to new training strategies that gradually increase the depths of the CNN while re-using parameters for initializations.
... where I 0 is the first image frame. Note that besides the optical flow, there exist other methods to recover φ related to the general image registration problem (1) capable of larger motion, see [25,26]. Because the displacement is sufficiently small in our application, it is reasonable to work with the optical flow equation (4), which can be interpreted as a linearization of the registration problem, see [13,45]. ...
In this paper, we propose mathematical models for reconstructing the optical flow in time-harmonic elastography. In this image acquisition technique, the object undergoes a special time-harmonic oscillation with known frequency so that only the spatially varying amplitude of the velocity field has to be determined. This allows for a simpler multi-frame optical flow analysis using Fourier analytic tools in time. We propose three variational optical flow models and show how their minimization can be tackled via Fourier transform in time. Numerical examples with synthetic as well as real-world data demonstrate the benefits of our approach.
... ( Mohammadi, Freund, et al., 2013) (Appendix Fig. E1). Unlike the linearized models, the nonlinear least squared (NLLS) method is based on an implementation ( Modersitzki, 2009) of the Gauss-Newton algorithm and operates on the nonlogarithmic data, avoiding the distortion of the noise distribution. ...
Diffusion MRI (dMRI) has become a crucial imaging technique in the field of neuroscience, with a growing number of clinical applications. Although most studies still focus on the brain, there is a growing interest in utilizing dMRI to investigate the healthy or injured spinal cord. The past decade has also seen the development of biophysical models that link MR-based diffusion measures to underlying microscopic tissue characteristics, which necessitates validation through ex vivo dMRI measurements. Building upon 13 years of research and development, we present an open-source, MATLAB-based academic software toolkit dubbed ACID: A Comprehensive Toolbox for Image Processing and Modeling of Brain, Spinal Cord, and Ex Vivo Diffusion MRI Data. ACID is an extension to the Statistical Parametric Mapping (SPM) software, designed to process and model dMRI data of the brain, spinal cord, and ex vivo specimens by incorporating state-of-the-art artifact correction tools, diffusion and kurtosis tensor imaging, and biophysical models that enable the estimation of microstructural properties in white matter. Additionally, the software includes an array of linear and nonlinear fitting algorithms for accurate diffusion parameter estimation. By adhering to the Brain Imaging Data Structure (BIDS) data organization principles, ACID facilitates standardized analysis, ensures compatibility with other BIDS-compliant software, and aligns with the growing availability of large databases utilizing the BIDS format. Furthermore, being integrated into the popular SPM framework, ACID benefits from a wide range of segmentation, spatial processing, and statistical analysis tools as well as a large and growing number of SPM extensions. As such, this comprehensive toolbox covers the entire processing chain from raw DICOM data to group-level statistics, all within a single software package.
... The proposed method was an iterative variational image registration approach based on (Modersitzki, 2009), in which the registration of two volumes can be modeled as the minimization of a discretized objective function. The final solution consists of a parametric and a non-parametric step. ...
Registration of longitudinal brain Magnetic Resonance Imaging (MRI) scans containing pathologies is challenging due to dramatic changes in tissue appearance. Although there has been considerable progress in developing general-purpose medical image registration techniques, they have not yet attained the requisite precision and reliability for this task, highlighting its inherent complexity. Here we describe the Brain Tumor Sequence Registration (BraTS-Reg) challenge, as the first public benchmark environment for deformable registration algorithms focusing on estimating correspondences between pre-operative and follow-up scans of the same patient diagnosed with a diffuse brain glioma. The challenge was conducted in conjunction with both the IEEE International Symposium on Biomedical Imaging (ISBI) 2022 and the International Conference on Medical Image Computing and Computer-Assisted Intervention (MICCAI) 2022. The BraTS-Reg data comprise de-identified multi-institutional multi-parametric MRI (mpMRI) scans, curated for size and resolution according to a canonical anatomical template, and divided into training, validation, and testing sets. Clinical experts annotated ground truth (GT) landmark points of anatomical locations distinct across the temporal domain. The training data with their GT annotations, were publicly released to enable the development of registration algorithms. The validation data, without their GT annotations, were also released to allow for algorithmic evaluation prior to the testing phase, which only allowed submission of containerized algorithms for evaluation on hidden hold-out testing data. Quantitative evaluation and ranking was based on the Median Euclidean Error (MEE), Robustness, and the determinant of the Jacobian of the displacement field. The top-ranked methodologies yielded similar performance across all evaluation metrics and shared several methodological commonalities, including pre-alignment, deep neural networks, inverse consistency analysis, and test-time instance optimization per-case basis as a post-processing step. The top-ranked method attained the MEE at or below that of the inter-rater variability for approximately 60% of the evaluated landmarks, underscoring the scope for further accuracy and robustness improvements, especially relative to human experts. The aim of BraTS-Reg is to continue to serve as an active resource for research, with the data and online evaluation tools accessible at https://bratsreg.github.io/.
... Any positive value for β ensures the intensity modulation constraint is satisfied, but lower values can lead to more ill-conditioned problems. For the purpose of this study, we fix α = 300 and β = 1e − 4. PyHySCO follows the discretize-then-optimize paradigm commonly used in image registration, see, e.g., Modersitzki (2009). PyHySCO discretizes the variational problem (Equation 4) as in Macdonald and Ruthotto (2017) to obtain a finite-dimensional optimization problem almost entirely separable in the phase encoding direction. ...
Over the past decade, reversed gradient polarity (RGP) methods have become a popular approach for correcting susceptibility artifacts in echo-planar imaging (EPI). Although several post-processing tools for RGP are available, their implementations do not fully leverage recent hardware, algorithmic, and computational advances, leading to correction times of several minutes per image volume. To enable 3D RGP correction in seconds, we introduce PyTorch Hyperelastic Susceptibility Correction (PyHySCO), a user-friendly EPI distortion correction tool implemented in PyTorch that enables multi-threading and efficient use of graphics processing units (GPUs). PyHySCO uses a time-tested physical distortion model and mathematical formulation and is, therefore, reliable without training. An algorithmic improvement in PyHySCO is its use of the one-dimensional distortion correction method by Chang and Fitzpatrick to initialize the non-linear optimization. PyHySCO is published under the GNU public license and can be used from the command line or its Python interface. Our extensive numerical validation using 3T and 7T data from the Human Connectome Project suggests that PyHySCO can achieve accuracy comparable to that of leading RGP tools at a fraction of the cost. We also validate the new initialization scheme, compare different optimization algorithms, and test the algorithm on different hardware and arithmetic precisions.
... Coarse-to-fine or hierarchical optimization strategies seek to solve the registration problem at progressively increasing resolutions, with the objective of decreasing the computational cost and finding a more accurate solution [12]. The search area is first restricted to coarse functions and the results are progressively refined, with the parameters estimated at the previous coarser level propagated to the next finer level. ...
Estimating accurate high-dimensional transformations remains very challenging, especially in a clinical setting. In this paper, we introduce a multiscale parameterization of deformations to enhance registration and atlas estimation in the Large Deformation Diffeomorphic Metric Mapping framework. Using the Haar wavelet transform, a multiscale representation of the initial velocity fields is computed to optimize transformations in a coarse-to-fine fashion. This additional layer of spatial regularization does not modify the underlying model of deformations. As such, it preserves the original kernel Hilbert space structure of the velocity fields, enabling the algorithm to perform efficient gradient descent. Numerical experiments on three datasets, including abnormal fetal brain images, show that compared to the original algorithm, the coarse-to-fine strategy reaches higher performance and yields template images that preserve important details while avoiding unrealistic features. This highly versatile strategy can easily be applied to other mathematical frameworks for almost no additional computational cost.
... 2013) (Fig. D1). Unlike the linearized models, the non-linear least squared (NLLS) method is based on an implementation (Modersitzki, 2009) of the Gauss-Newton algorithm and operates on the nonlogarithmic data, avoiding the distortion of noise distribution. ...
Diffusion MRI (dMRI) has become a crucial imaging technique within the field of neuroscience and has an increasing number of clinical applications. Although most studies still focus on the brain, there is a growing interest in utilizing dMRI to investigate the healthy or injured spinal cord. The past decade has also seen the development of biophysical models that link MR-based diffusion measures to underlying microscopic tissue characteristics. Building upon 13 years of research and development, we present an open-source, MATLAB-based academic software toolkit dubbed ACID: A Comprehensive Toolbox for Image Processing and Modeling of Brain, Spinal Cord, and Post-mortem Diffusion MRI Data. ACID is designed to process and model dMRI data of the brain, spinal cord, and post-mortem specimens by incorporating state-of-the-art artifact correction tools, diffusion and kurtosis tensor imaging, and biophysical models that enable the estimation of microstructural properties in white matter. Additionally, the software includes an array of linear and non-linear fitting algorithms for accurate diffusion parameter estimation. By adhering to the Brain Imaging Data Structure (BIDS) data organization principles, ACID facilitates standardized analysis, ensures compatibility with other BIDS-compliant software, and aligns with the growing availability of large databases utilizing the BIDS format. Furthermore, ACID seamlessly integrates into the popular Statistical Parametric Mapping (SPM) framework, benefitting from a wide range of established segmentation, spatial processing, and statistical analysis tools as well as a large and growing number of SPM extensions. As such, this comprehensive toolbox covers the entire processing chain from raw DICOM data to group-level statistics, all within a single software package.
... Popular implementations of this framework include classical methods such as TOPUP from the FMRIB Software Library 9,14 and hyperelastic susceptibility correction of DTI data (HySCO) from the Statistical Parametric Mapping toolbox. 15,16 Since no additional data collection is needed beyond reversed-PE images, classical methods in the unwarping-based framework can offer notable benefits over measured-field-based, registration-based, or point spread function-(PSF) based approaches in the literature. 17,18 Nonetheless, these classical methods are based on iterative optimization techniques that introduce substantial computational burden, rendering them impractical under clinical settings. ...
Purpose
To introduce an unsupervised deep‐learning method for fast and effective correction of susceptibility artifacts in reversed phase‐encode (PE) image pairs acquired with echo planar imaging (EPI).
Methods
Recent learning‐based correction approaches in EPI estimate a displacement field, unwarp the reversed‐PE image pair with the estimated field, and average the unwarped pair to yield a corrected image. Unsupervised learning in these unwarping‐based methods is commonly attained via a similarity constraint between the unwarped images in reversed‐PE directions, neglecting consistency to the acquired EPI images. This work introduces a novel unsupervised deep Forward‐Distortion Network (FD‐Net) that predicts both the susceptibility‐induced displacement field and the underlying anatomically correct image. Unlike previous methods, FD‐Net enforces the forward‐distortions of the correct image in both PE directions to be consistent with the acquired reversed‐PE image pair. FD‐Net further leverages a multiresolution architecture to maintain high local and global performance.
Results
FD‐Net performs competitively with a gold‐standard reference method (TOPUP) in image quality, while enabling a leap in computational efficiency. Furthermore, FD‐Net outperforms recent unwarping‐based methods for unsupervised correction in terms of both image and field quality.
Conclusion
The unsupervised FD‐Net method introduces a deep forward‐distortion approach to enable fast, high‐fidelity correction of susceptibility artifacts in EPI by maintaining consistency to measured data. Therefore, it holds great promise for improving the anatomical accuracy of EPI imaging.
... It has been studied extensively over the past several decades, both for medical and non-medical applications [28][29][30][31][32][33]. This group consists of uniform scaling, rotations, and arbitrary translations along the axes [8,19,[34][35][36]. Let us first present each of these separately. ...
Image registration is the process of approximate matching of the source image to the target so that they resemble each other. In this study, two-dimensional image registration is presented using the rigid group. This group is a finite dimensional group (four-dimensional in this case) under composition. The dimensions of the rigid group are scaling, rotation, and translations along the axes. In this paper, an algorithm for the construction of rigid transformation is presented using the discretized objective function. This objective function is based on SSD (sum of the squares of the distances between the pixels intensities) and calculates the discrepancy between the images. The coarse search and the gradient descent approaches have been used for the optimization. The proposed algorithm is implemented on variety of images. The numerical examples illustrate the ability of the proposed algorithm.
Medical imaging plays a critical role in clinical decision-making and patient care. However, the presence of high levels of noise in medical images can significantly impact the accuracy of diagnosis and subsequent analysis. In recent years, joint segmentation and registration models have emerged as an effective alternative approach for enhancing medical images. Nevertheless, traditional methods, such as the Chan-Vese model, face challenges when dealing with images with high levels of noise. To address this limitation, this paper introduces a different approach that incorporates generalized mean into the joint model. Our joint model combines the generalized mean-based image segmentation which utilizes the fuzzy-membership function, modified normalized gradient fields and linear curvature for registration task. The performance of the proposed model is tested on 2D synthetic and real medical images with and without the presence of the white Gaussian noise. Then it is compared to the existing joint model using three evaluation criterions which are Dice coefficient metric, registration value and computational time. The proposed joint model improved by 60% according to the numerical results when tested on images with high level of noise. The model is useful and beneficial to the radiologists to perform quantitative analysis in assessing disease progression, response to treatment, and overall patient health.
Let be a dataset of smooth 3D surfaces, partitioned into disjoint classes , . We show how optimized diffeomorphic registration applied to large numbers of pairs , can provide descriptive feature vectors to implement automatic classification on and generate classifiers invariant by rigid motions in . To enhance the accuracy of shape classification, we enrich the smallest classes by diffeomorphic interpolation of smooth surfaces between pairs . We also implement small random perturbations of surfaces by random flows of smooth diffeomorphisms . Finally, we test our classification methods on a cardiology database of discretized mitral valve surfaces.
We present our work on scalable, GPU-accelerated algorithms for diffeomorphic image registration. The associated software package is termed CLAIRE. Image registration is a nonlinear inverse problem. It is about computing a spatial mapping from one image of the same object or scene to another. In diffeomorphic image registration, the set of admissible spatial transformations is restricted to maps that are smooth, are one-to-one, and have a smooth inverse. We formulate diffeomorphic image registration as a variational problem governed by transport equations. We use an inexact, globalized (Gauss–)Newton–Krylov method for numerical optimization. We consider semi-Lagrangian methods for numerical time integration. Our solver features mixed-precision, hardware-accelerated computational kernels for optimal computational throughput. We use the message-passing interface for distributed-memory parallelism and deploy our code on modern high-performance computing architectures. Our solver allows us to solve clinically relevant problems in under four seconds on a single GPU. It can also be applied to large-scale 3D imaging applications with data that is discretized on meshes with billions of voxels. We demonstrate that our numerical framework yields high-fidelity results in only a few seconds, even if we search for an optimal regularization parameter.
Recent studies indicate that malignant breast lesions can be predicted from structural changes in prior exams of preventive breast MRI examinations. Due to non-rigid deformation between studies, spatial correspondences between structures in two consecutive studies are lost. Thus, deformable image registration can contribute to predicting individual cancer risks. This study evaluates a registration approach based on a novel breast mask segmentation and non-linear image registration based on data from 5 different sites. The landmark error (mean ± standard deviation [1st quartile, 3rd quartile]), annotated by three radiologists, is 2.9 ± 2.8 [1.3, 3.2] mm when leaving out two outlier cases from the evaluation for which the registration failed completely. We assess the inter-observer variabilities of keypoint errors and find an error of 3.6 ± 4.7 [1.6, 4.0] mm, 4.4 ± 4.9 [1.8, 4.8] mm, and 3.8 ± 4.0 [1.7, 4.1] mm when comparing each radiologist to the mean keypoints of the other two radiologists. Our study shows that the current state of the art in registration is well suited to recover spatial correspondences of structures in cancerous and non-cancerous cases, despite the high level of difficulty of this task.
Artificial intelligence has been used with great success for the segmentation of anatomical structures in medical imaging. We use these achievements to improve classical registration schemes. Particularly, we derive geometrical features such as centroids and principal axes of segments and use those in a combined approach. A smart filtering of the features results in a two phase preregistration, followed in a third phase by an intensity guided registration. We also propose to use a regularization, which enables a coupling of all components of the 3D transformation in a unified framework. Finally, we show how easily our approach can be applied even to challenging 3D medical data.
Multimodal imaging is a prominent strategy for biomedical research. For instance, Mass Spectrometry Imaging (MSI) can reveal the chemical composition of tissues with high specificity, helping to elucidate their metabolic pathways. However, this technique is not necessarily informative about the structural organization of a tissue. Other modalities, such as Magnetic Resonance Imaging (MRI), reveal functional areas in tissue. Images are analyzed jointly using several computational methods. Registration is a pivotal step that estimates a transformation to spatially align two images. Automatic methods usually rely on similarity metrics. Similarity metrics are used as optimization functions to find the transformation parameters. MALDI–MS and MR images have different intensity distributions that cannot be accounted for by traditional similarity metrics. In this article, we propose a novel similarity metric for deformable registration, based on the update of distance transformation values. We show that our method limits the intensity distortions while providing precisely registered images, on both synthetic and mouse brain images.
Deep learning-based cell detectors have shown promise in automating cell detection in histopathology images, which can aid in cancer diagnosis and prognosis. Nevertheless, the color variation in stain appearance across histopathology images obtained from diverse locations might degrade the accuracy of cell detection. The limitations of a basic cell detector on a dissimilar target dataset are attributed to its emphasis on domain-specific features while overlooking domain-invariant features. Thus, a cell detector that is trained on a particular source dataset may not perform optimally on a different target dataset. In this work, we propose a domain generalization method contextual regularization (CR) for cell detection in histopathology images, which is derived from the basic object detector and focuses on domain-invariant features to enhance detection performance across varying datasets. Specifically, we involves a reconstruction task that involves masking the high-level semantic features either stochastically or adaptively. Then, a transformer-based reconstruction head is designed to recover the original features based on partial observations. Additionally, CR can be seamlessly integrated with various deep learning-based cell detectors without further modifications, and it does not request additional cost in the inference time. The proposed method was validated on a publicly available dataset that comprises histopathology images acquired at different sites, and the results show that our method can effectively improve the generalization of cell detectors to unseen domains.
The visual comparison of MRI images obtained before and after neurosurgical procedures is useful to identify tumor recurrences in postoperative images. Image registration algorithms are utilized to establish precise correspondences between subsequent acquisitions. The changes observable in subsequent MRI acquisitions can be tackled by methods combining rigid and non-rigid registration. A rigid step is useful to accommodate global transformations, due, for example, to the different positions and orientations of the patient’s head within the scanning device. Furthermore, brain shift caused by tumor resection can only be tackled by non-rigid approaches. In this work, we propose an automatic iterative method to register pre- and postoperative MRI acquisitions. First, the solution rigidly registers two subsequent images. Then, a deformable registration is computed. The T1-CE and T2 MRI sequences are used to guide the registration process. The method is proposed as a solution to the BraTS-Reg challenge. The method improves the average median absolute error from 7.8 mm to 1.98 mm in the validation set.
In this paper, a theoretical asymptotic result in relation to the nonlinear elasticity-based multiscale registration model [Debroux et al. 2023] is proved. Specifically, it establishes a link between the original minimisation problem comprising high non linearity and non convexity, and the one derived from splitting techniques by means of auxiliary variables and -penalisations which exhibits increased numerical manageability. This latter problem thus appears to be a good compromise between mechanical/physical realism and practical feasibility.KeywordsRegistrationMultiscale decompositionOgden materialsBi-Lipschitz homeomorphismsAsymptotic analysis
This paper constructs metrics on the space of images I defined as orbits under group actions G. The groups studied include the finite dimensional matrix groups and their products, as well as the infinite dimensional diffeomorphisms examined in Trouv (1999, Quaterly of Applied Math.) and Dupuis et al. (1998). Quaterly of Applied Math. Left-invariant metrics are defined on the product G I thus allowing the generation of transformations of the background geometry as well as the image values. Examples of the application of such metrics are presented for rigid object matching with and without signature variation, curves and volume matching, and structural generation in which image values are changed supporting notions such as tissue creation in carrying one image to another.
We present a theoretical and computational framework for nonrigid multimodal registration. We proceed by minimization of statistical similarity criteria (global and local) in a variational framework, and use the corresponding gradients to drive a flow of diffeomorphisms allowing large deformations. This flow is introduced through a new template propagation method, by composition of small displacements. Regularization is performed using fast filtering techniques. This approach yields robust matching algorithms offering a good computational efficiency. We apply this method to compensate distortions between EPI images (fMRI) and anatomical MRI volumes.
Two new consistent image registration algorithms are presented: one is based on matching corresponding landmarks and the other is based on matching both landmark and intensity information. The consistent landmark and intensity registration algorithm produces good correspondences between images near landmark locations by matching corresponding landmarks and away from landmark locations by matching the image intensities. In contrast to similar unidirectional algorithms, these new consistent algorithms jointly estimate the forward and reverse transformation between two images while minimizing the inverse consistency error-the error between the forward (reverse) transformation and the inverse of the the reverse (forward) transformation. This reduces the ambiguous correspondence between the forward and reverse transformations associated with large inverse consistency errors. In both algorithms a thin-plate spline (TPS) model is used to regularize the estimated transformations. Two-dimensional (2-D) examples are presented that show the inverse consistency error produced by the traditional unidirectional landmark TPS algorithm can be relatively large and that this error is minimized using the consistent landmark algorithm. Results using 2-D magnetic resonance imaging data are presented that demonstrate that using landmark and intensity information together produce better correspondence between medical images than using either landmarks or intensity information alone.
This paper presents a robust method to correct for intensity differ- ences across a series of aligned stained histological slices. The method is made up of two steps. First, for each slice, a scale-space analysis of the histogram pro- vides a set of alternative interpretations in terms of tissue classes. Each of these interpretations can lead to a different classification of the related slice. A simple heuristics selects for each slice the most plausible interpretation. Then, an iter- ative procedure refines the interpretation selections across the series in order to maximize a score measuring the spatial consistency of the classifications across contiguous slices. Results are presented for a series of 121 baboon slices.
Voxel-based nonrigid image registration can be formulated as an optimisation problem whose goal is to minimise a cost function,
consisting of a first term that characterises the similarity between both images and a second term that regularises the transformation
and/or penalties improbable or impossible deformations. Within this paper, we extend previous works on nonrigid image registration
by the introduction of a new penalty term that expresses the local rigidity of the deformation. A necessary and sufficient
condition for the transformation to be locally rigid at a particular location is that its Jacobian matrix J
T at this location is orthogonal, satisfying the orthogonality condition JT JTT = 1J_{\rm T} J_{\rm T}^{T} = {\bf 1}. So we define the penalty term as the weighted integral of the Frobenius norm of JT JTT - 1J_{\rm T} J_{\rm T}^{T} - {\bf 1} integrated over the overlap of the images to be registered. We fit the implementation of the penalty term in a multidimensional,
continuous and differentiable B-spline deformation framework and analytically determine the derivative of the similarity criterion
and the penalty term with respect to the deformation parameters. We show results of the impact of the proposed rigidity constraint
on artificial and clinical images demonstrating local shape preservation with the proposed constraint.
This paper examine the Euler-Lagrange equations for the solution of the large deformation diffeomorphic metric mapping problem studied in Dupuis et al. (1998) and Trouvé (1995) in which two images I 0, I 1 are given and connected via the diffeomorphic change of coordinates I 0○ϕ−1=I 1 where ϕ=Φ1 is the end point at t= 1 of curve Φ t , t∈[0, 1] satisfying .Φ t =v t (Φ t ), t∈ [0,1] with Φ0=id. The variational problem takes the form \mathop {\arg {\text{m}}in}\limits_{\upsilon :\dot \phi _t = \upsilon _t \left( {\dot \phi } \right)} \left( {\int_0^1 {\left\| {\upsilon _t } \right\|} ^2 {\text{d}}t + \left\| {I_0 \circ \phi _1^{ - 1} - I_1 } \right\|_{L^2 }^2 } \right)
Already for reasonable sized 3D images, image registration be- comes a computationally intensive task. Here, we introduce and ex- plore the concept of OcTree's for registration which drastically reduces the number of processed data and thus the computational costs. We show how to map the registration problem onto an OcTree and present a suitable optimization technique. Furthermore, we demonstrate the performance of the new approach by academic as well as real life ex- amples. These examples indicate that the computational time can be reduced by a factor of 10 compared with standard approaches.
This article describes a methodology for creating a generic volumetric biomechanical model from different image modalities and segmenting time series of medical images using this model. The construction of such a generic model consists of three stages: geometric meshing, non-rigid deformation of the mesh in images of various modalities, and image-to-mesh information mapping through rasterization. The non-rigid deformation stage, which relies on a combination of global and local deformations, can then be used to segment time series of images, e.g. cine MRI or gated SPECT cardiac images. We believe that this type of deformable biomechanical model can play an important role in the extraction of useful quantitative local parameters of cardiac function. The biomechanical model of the heart will be coupled with an electrical model of another collaborative project in order to simulate and analyze a larger class of pathologies.
Although numerous methods to register brains of different individuals have been proposed, no work has been done, as far as we know, to evaluate and objectively compare the performances of different nonrigid (or elastic) registration methods on the same database of subjects. In this paper, we propose an evaluation framework, based on global and local measures of the relevance of the registration. We have chosen to focus more particularly on the matching of cortical areas, since intersubject registration methods are dedicated to anatomical and functional normalization, and also because other groups have shown the relevance of such registration methods for deep brain structures. Experiments were conducted using 6 methods on a database of 18 subjects. The global measures used show that the quality of the registration is directly related to the transformation's degrees of freedom. More surprisingly, local measures based on the matching of cortical sulci did not show significant differences between rigid and non rigid methods.
A new method for medical image registration is formulated as a minimization problem involving robust estimators. We propose an efficient hierarchical optimization framework which is both multiresolution and multigrid. An anatomical segmentation of the cortex is introduced in the adaptive partitioning of the volume on which the multigrid minimization is based. This allows to limit the estimation to the areas of interest, to accelerate the algorithm, and to refine the estimation in specified areas. At each stage of the hierarchical estimation, we refine current estimate by seeking a piecewise affine model for the incremental deformation field. The performance of this method is numerically evaluated on simulated data and its benefits and robustness are shown on a database of 18 magnetic resonance imaging scans of the head.
This paper presents a novel method for validation of nonrigid medical image registration. This method is based on the simulation of physically plausible, biomechanical tissue deformations using finite-element methods. Applying a range of displacements to finite-element models of different patient anatomies generates model solutions which simulate gold standard deformations. From these solutions, deformed images are generated with a range of deformations typical of those likely to occur in vivo. The registration accuracy with respect to the finite-element simulations is quantified by co-registering the deformed images with the original images and comparing the recovered voxel displacements with the biomechanically simulated ones. The functionality of the validation method is demonstrated for a previously described nonrigid image registration technique based on free-form deformations using B-splines and normalized mutual information as a voxel similarity measure, with an application to contrast-enhanced magnetic resonance mammography image pairs. The exemplar nonrigid registration technique is shown to be of subvoxel accuracy on average for this particular application. The validation method presented here is an important step toward more generic simulations of biomechanically plausible tissue deformations and quantification of tissue motion recovery using nonrigid image registration. It will provide a basis for improving and comparing different nonrigid registration techniques for a diversity of medical applications, such as intrasubject tissue deformation or motion correction in the brain, liver or heart.
In this paper, we investigate the introduction of cortical constraints for non rigid intersubject brain registration. We extract sulcal patterns with the active ribbon method, presented by Le Goualher et al. (1997). An energy based registration method (Hellier et al., 2001), which will be called photometric registration method in this paper, makes it possible to incorporate the matching of cortical sulci. The local sparse similarity and the photometric similarity are, thus, expressed in a unified framework. We show the benefits of cortical constraints on a database of 18 subjects, with global and local assessment of the registration. This new registration scheme has also been evaluated on functional magnetoencephalography data. We show that the anatomically constrained registration leads to a substantial reduction of the intersubject functional variability.
Groupwise nonrigid registrations of medical images define dense correspondences across a set of images, defined by a continuous deformation field that relates each target image in the group to some reference image. These registrations can be automatic, or based on the interpolation of a set of user-defined landmarks, but in both cases, quantifying the normal and abnormal structural variation across the group of imaged structures implies analysis of the set of deformation fields. We contend that the choice of representation of the deformation fields is an integral part of this analysis. This paper presents methods for constructing a general class of multi-dimensional diffeomorphic representations of deformations. We demonstrate, for the particular case of the polyharmonic clamped-plate splines, that these representations are suitable for the description of deformations of medical images in both two and three dimensions, using a set of two-dimensional annotated MRI brain slices and a set of three-dimensional segmented hippocampi with optimized correspondences. The class of diffeomorphic representations also defines a non-Euclidean metric on the space of patterns, and, for the case of compactly supported deformations, on the corresponding diffeomorphism group. In an experimental study, we show that this non-Euclidean metric is superior to the usual ad hoc Euclidean metrics in that it enables more accurate classification of legal and illegal variations.
We propose a non-parametric diffeomorphic image registration algorithm based on Thirion's demons algorithm. The demons algorithm can be seen as an optimization procedure on the entire space of displacement fields. The main idea of our algorithm is to adapt this procedure to a space of diffeomorphic transformations. In contrast to many diffeomorphic registration algorithms, our solution is computationally efficient since in practice it only replaces an addition of free form deformations by a few compositions. Our experiments show that in addition to being diffeomorphic, our algorithm provides results that are similar to the ones from the demons algorithm but with transformations that are much smoother and closer to the true ones in terms of Jacobians.
We present an algorithm for nonlinear multidimensional registration. The correspondence function is represented in a spline space. We also use cubic splines to interpolate the volumetric data to be registered. We use a multiresolution strategy, which gives us robustness and additional speedup. We analyze the computational complexity of the algorithm and its performance using different multidimensional optimization methods. Finally, we present an application of the algorithm for registering technetium ethylene cysteine diethylester (ECD) SPECT images for realignment of corresponding Xe inhalation SPECT images.
Though fluid dynamics offer a good approach to nonrigid registration and give accurate results, even with large-scale deformations, its application is still very time consuming. We introduce and discuss different approaches to solve the core problem of nonrigid registration, the partial differential equation of fluid dynamics. We focus on the solvers, their computational costs and the accuracy of registration. Numerical experiments show that relaxation is currently the best approach, especially when reducing the cost/iteration by focusing the updates on deformation spots.
We propose a simple and efficient method to interpolate landmark matching by a non-ambiguous mapping (a diffeomorphism). This method is based on spline interpolation, and on recent techniques developed for the estimation of ows of diffeomorphisms. Experimental results show interpolations of remarkable quality. Moreover, the method provides a Riemannian distance on sets of landmarks (with fixed cardinality), which can be defined intrinsically, without referring to diffeomorphisms. The numerical implementation is simple and efficient, based on an energy minimization by gradient descent. This opens important perspectives for shape analysis, applications in medical imaging, or computer graphics.
Image registration requires the transformation of one image to another so as to spatially align the two images. This involves interpolation to estimate gray values of one of the images at positions other than the grid points. When registering two images that have equal grid distances in one or more dimensions, the grid points can be aligned in those dimensions for certain geometric transformations. Consequently, the number of times interpolation is required to compute the registration measure of two images is dependent on the image transformation. When an entropy-based registration measure, such as mutual information, is plotted as a function of the transformation, it will show sudden changes in value for grid-aligning transformations. Such patterns of local extrema impede the registration optimization process. More importantly, they rule out subvoxel accuracy. In this paper, two frequently applied interpolation methods in mutual information-based image registration are analyzed, viz. linear interpolation and partial volume interpolation. It is shown how the registration function depends on the interpolation method and how a slight resampling of one of the images may drastically improve the smoothness of this function.
We introduce a new non-linear registration model based on a curvature type regularizer. We show that affine linear transformations belong to the kernel of this regularizer. Consequently, an additional global registration is superfluous. Furthermore, we present an implementation of the new scheme based on the numerical solution of the underlying Euler-Lagrange equations. The real DCT is the backbone of our implementation and leads to a stable and fast algorithm, where n denotes the number of voxels. We demonstrate the advantages of the new technique for synthetic data sets. Moreover, first convincing results for the registration of MR-mammography images are presented.
Flow and transport phenomena occurring within serpentine microchannels are analyzed for both two- and three-dimensional curvilinear configurations
A variational method for nonrigid registration of multimodal image data is presented. A suitable deformation will be determined via the minimization of a morphological, i.e., contrast invariant, matching functional along with an appropriate regularization energy. The aim is to correlate the morphologies of a template and a reference image under the deformation. Mathematically, the morphology of images can be described by the entity of level sets of the image and hence by its Gauss map. A class of morphological matching functionals is presented which measure the defect of the template Gauss map in the deformed state with respect to the deformed Gauss map of the reference image. The problem is regularized by considering a nonlinear elastic regularization energy. Existence of homeomorphic , minimizing deformation is proved under assumptions on the class of admissible deformations. With respect to actual medical applications, suitable generalizations of the matching energies and the boundary conditions are presented. Concerning the robust implementation of the approach, the problem is embedded in a multiscale context. A discretization based on multilinear finite elements is discussed, and the first numerical results are presented.
Mutual information has developed into an accurate measure for rigid and affine monomodality and multimodality image registration. The robustness of the measure is questionable, however. A possible reason for this is the absence of spatial information in the measure. The present paper proposes to include spatial information by combining mutual information with a term based on the image gradient of the images to be registered. The gradient term not only seeks to align locations of high gradient magnitude, but also aims for a similar orientation of the gradients at these locations. Results of combining both standard mutual information as well as a normalized measure are presented for rigid registration of three-dimensional clinical images [magnetic resonance (MR), computed tomography (CT), and positron emission tomography (PET)]. The results indicate that the combined measures yield a better registration function does mutual information or normalized mutual information per se. The registration functions are less sensitive to low sampling resolution, do not contain incorrect global maxima that are sometimes found in the mutual information function, and interpolation-induced local minima can be reduced. These characteristics yield the promise of more robust registration measures. The accuracy of the combined measures is similar to that of mutual information-based methods.
In this paper, we extend a previously reported intensity-based nonrigid registration algorithm by using a novel regularization term to constrain the deformation. Global motion is modeled by a rigid transformation while local motion is described by a free-form deformation based on B-splines. An information theoretic measure, normalized mutual information, is used as an intensity-based image similarity measure. Registration is performed by searching for the deformation that minimizes a cost function consisting of a weighted combination of the image similarity measure and a regularization term. The novel regularization term is a local volume-preservation (incompressibility) constraint, which is motivated by the assumption that soft tissue is incompressible for small deformations and short time periods. The incompressibility constraint is implemented by penalizing deviations of the Jacobian determinant of the deformation from unity. We apply the nonrigid registration algorithm with and without the incompressibility constraint to precontrast and postcontrast magnetic resonance (MR) breast images from 17 patients. Without using a constraint, the volume of contrast-enhancing lesions decreases by 1%--78% (mean 26%). Image improvement (motion artifact reduction) obtained using the new constraint is compared with that obtained using a smoothness constraint based on the bending energy of the coordinate grid by blinded visual assessment of maximum intensity projections of subtraction images. For both constraints, volume preservation improves, and motion artifact correction worsens, as the weight of the constraint penalty term increases. For a given volume change of the contrast-enhancing lesions (2% of the original volume), the incompressibility constraint reduces motion artifacts ...
We formulate and interpret several registration methods in the context of a unified statistical and information theoretic framework. A unified interpretation clarifies the implicit assumptions of each method yielding a better understanding of their relative strengths and weaknesses. Additionally, we discuss a generative statistical model from which we derive a novel analysis tool, the auto-information function, as a means of assessing and exploiting the common spatial dependencies inherent in multi-modal imagery. We analytically derive useful properties of the auto-information as well as verify them empirically on multi-modal imagery. Among the useful aspects of the auto-information function is that it can be computed from imaging modalities independently and it allows one to decompose the search space of registration problems.
Rapidly advancing registration methods increasingly employ warping transforms. High degrees of freedom (DOF) warpings can be specified by manually placing control points or instantiating a regular, dense grid of control points everywhere. The former approach is laborious and prone to operator bias, whereas the latter is computationally expensive. We propose to improve upon the latter approach by adaptively placing control points where they are needed. Local estimates of mutual information (MI) and entropy are used to identify local regions requiring additional DOF.
In this paper we propose a diffeomorphic non-rigid registration algorithm based on free-form deformations (FFDs) which are modelled by B-splines. In contrast to existing non-rigid registration methods based on FFDs the proposed diffeomorphic non-rigid registration algorithm based on free-form deformations (FFDs) which are modelled by B-splines. To construct a diffeomorphic transformation we compose a sequence of free-form deformations while ensuring that individual FFDs are one-to-one transformations. We have evaluated the algorithm on 20 normal brain MR images which have been manually segmented into 67 anatomical structures. Using the agreement between manual segmentation and segmentation propagation as a measure of registration quality we have compared the algorithm to an existing FFD registration algorithm and a modified FFD registration algorithm which penalises non-diffeomorphic transformations. The results show that the proposed algorithm generates diffeomorphic transformations while providing similar levels of performance as the existing FFD registration algorithm in terms of registration accuracy.
This paper describes DARTEL, which is an algorithm for diffeomorphic image registration. It is implemented for both 2D and 3D image registration and has been formulated to include an option for estimating inverse consistent deformations. Nonlinear registration is considered as a local optimisation problem, which is solved using a Levenberg-Marquardt strategy. The necessary matrix solutions are obtained in reasonable time using a multigrid method. A constant Eulerian velocity framework is used, which allows a rapid scaling and squaring method to be used in the computations. DARTEL has been applied to intersubject registration of 471 whole brain images, and the resulting deformations were evaluated in terms of how well they encode the shape information necessary to separate male and female subjects and to predict the ages of the subjects.
We propose a new method for the intermodal registration of images using a criterion known as mutual information. Our main contribution is an optimizer that we specifically designed for this criterion. We show that this new optimizer is well adapted to a multiresolution approach because it typically converges in fewer criterion evaluations than other optimizers. We have built a multiresolution image pyramid, along with an interpolation process, an optimizer, and the criterion itself, around the unifying concept of spline-processing. This ensures coherence in the way we model data and yields good performance. We have tested our approach in a variety of experimental conditions and report excellent results. We claim an accuracy of about a hundredth of a pixel under ideal conditions. We are also robust since the accuracy is still about a tenth of a pixel under very noisy conditions. In addition, a blind evaluation of our results compares very favorably to the work of several other researchers.
Positive denite symmetric matrices (so-called tensors in this article) are nowadays a common source of geometric information. In this paper, we propose to provide the tensor space with an ane-invariant Riemannian metric. We demonstrate that it leads to strong theoretical properties: the cone of positive denite symmetric matrices is replaced by a regular manifold of constant curvature without boundaries (null eigenvalues are at the innity), the geodesic between two tensors and the mean of a set of tensors are uniquely dened, etc.
A fast multiscale and multigrid method for the matching of images in 2D and 3D is presented. Especially in medical imaging this problem - denoted as the registration problem - is of fundamental importance in the handling of images from multiple image modalities or of image time series. The paper restricts to the simplest matching energy to be minimized, i.e., are the intensity maps of the two images to be matched and is a deformation. The focus is on a robust and efficient solution strategy. Matching of images, i.e., finding an optimal deformation which minimizes is known to be an ill-posed problem. Hence, to regularize this problem a regularization of the descent path is considered in a gradient flow method. Thus the initial value problem with some regular initial deformation "!#%$ is solved on a suitable space of deformations &(')& . The gradient grad is measured w.r.t a suitable regularizing metric . Existence and uniqueness of solutions is demonstrated for different types of regularizations. For the implementation a metric based on multigrid cycles on hierarchical grids is proposed, using their superior smoothing properties. This is combined with an effective time-step control in the descent algorithm. Furthermore, to avoid convergence to local minima, multiple scales of the images to be matched are considered. Again, these image scales can be generated applying multigrid operators and we propose to resolve the pyramid of scales on a properly chosen pyramid of hierarchical grids. Examples on 2D and large 3D image matching problems prove the robustness and efficiency of the proposed approach. 1.
98 7.3 T1/T2 weighted MRI's of a head
- Distance Measures For Mon-Modal Images
Distance Measures for mon-modal images............... 98
7.3
T1/T2 weighted MRI's of a head................... 99
7.4
53 5.1 Reference and template with corresponding landmarks
- More.................... Spline Based Transformation
Spline based transformation and more................ 53
5.1
Reference and template with corresponding landmarks....... 60
5.2
Linear and quadratic landmark based registrations......... 62
5.3
18 2.2 Transforming images
- Transformation.................................. Visualizations
Visualizations of an image and a transformation........... 18
2.2
Transforming images.......................... 19
88 6.15 Multilevel representation of data and images 89 6.16 MLPIR iteration history for SSD and rigid transformations
- Pir Regularized
- Spline Transformations.................. Ssd
Regularized PIR for SSD and spline transformations........ 88
6.15
Multilevel representation of data and images............. 89
6.16
MLPIR iteration history for SSD and rigid transformations.... 90
6.17
MLPIR results for SSD and rigid transformations......... 90
6.18
MLPIR iteration history for SSD and affine transformations.... 91
6.19
MLPIR results for SSD and affine transformations......... 92
7.1
105 7.8 NGF for MRI's of a head
- ...................................... Parzen-Window Density Estimators
Parzen-Window density estimators.................. 105
7.8
NGF for MRI's of a head........................ 108
7.9
124 List of Tables 4.1 A more efficient implementation of the affine linear transformation using a persistent variable
- Staggered Grids In
Staggered Grids in 3D......................... 124
List of Tables
4.1
A more efficient implementation of the affine linear transformation
using a persistent variable Q...................... 55
6.1
Implementations of the rotation and translation examples..... 75
6.2
71 6.3 SSD versus rotations, coarse
- Duty............................................... Fine..................... Quadrature On
Quadrature on duty........................... 71
6.3
SSD versus rotations, coarse...................... 73
6.4
SSD versus rotations, fine....................... 73
6.5
SSD versus translations......................... 74
6.6
89 6.16 MLPIR iteration history for SSD and rigid transformations. ... 90 6.17 MLPIR results for SSD and rigid transformations
- Of Multilevel Representation
- Images................... Data
Multilevel representation of data and images............. 89
6.16
MLPIR iteration history for SSD and rigid transformations.... 90
6.17
MLPIR results for SSD and rigid transformations......... 90
6.18
MLPIR iteration history for SSD and affine transformations.... 91
6.19
MLPIR results for SSD and affine transformations......... 92
7.1
80 6.7 PIR for SSD and rotations, m = (32, 16) 84 6.8 PIR for SSD and rotations, m = (256, 128)
- ...................................... Plots From Parametric Image Registration
Plots from Parametric Image Registration.............. 80
6.7
PIR for SSD and rotations, m = (32, 16)............... 84
6.8
PIR for SSD and rotations, m = (256, 128).............. 84
6.9
PIR iteration histories for SSD and rotations............. 85
6.10
PIR for SSD and rigid transformations, m = (32, 16)........ 85
6.11
PIR for SSD and rigid transformations, m = (256, 128)....... 86
6.12
PIR iteration histories for SSD and rigid transformations...... 86
6.13
PIR results for SSD and spline transformation............ 87
6.14