No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Fast diffusion registration
Abstract
Image registration is one of the most challenging tasks within digital imaging, in particular in medical imaging. Typically, the underlying problems are high dimensional and demand for fast and efficient numerical schemes. Here, we propose a novel scheme for automatic image registration by introducing a specific regularizing term. The new scheme is called diffusion registration since its implementation is based on the solution of a diffusion type partial differential equation. The main ingredient for a fast implementation of the diffusion registration is the so-called additive (Operator Splitting (AOS) Scheme. The AOS-scheme is known to be as accurate as a conventional semi-implicit scheme and has a linear complexity with respect to the size of the images. We present a proof of these properties based purely on matrix analysis. The performance of the new scheme is demonstrated for a typical medical registration problem. It is worth noticing that the diffusion registration is extremely well-suited for a parallel implementation. Finally, we also draw a connection to Thirion’s demon based approach.
... In (2), the first term S(u) is a regularization term which controls the smoothness of u and reflects our expectations in penalizing unlikely transformations. Various regularizers have been proposed, such as first-order derivatives-based on total variation [10,23], diffusion [15] and elastic regularizer registration models, higher-order derivatives-based on linear curvature [16], mean curvature [12], Gaussian curvature [24], and fractional order derivatives based models [50]; refer also to [11,31,44,51,52]. ...
... A regularizer controls the smoothness. Our primary choice for smoothness control is the diffusion model [15] which uses first-order derivatives and promotes smoothness. As affine linear transformations are not included in the kernel of the H 1 -regularizer, we desire a regularizer which can penalize such transformation. ...
... To simplify (15), define 3 vectors r(U ), ...
In this Chapter we discuss multi-modality image registration models and efficient algorithms. We propose a simple method to enhance a variational model to generate a diffeomorphic transformation. The idea is illustrated by using a particular model based on reformulated normalized gradients of the images as the fidelity term and higher-order derivatives as the regularizer. By adding a control term motivated by quasi-conformal maps and Beltrami coefficients, the model has the ability to guarantee a diffeomorphic transformation. Without this feature, the model may lead to visually pleasing but invalid results. To solve the model numerically, we present both a Gauss-Newton method and an augmented Lagrangian method to solve the resulting discrete optimization problem. A multilevel technique is employed to speed up the initialization and reduce the possibility of getting local minima of the underlying functional. Finally numerical experiments demonstrate that this new model can deliver good performances for multi-modal image registration and simultaneously generate an accurate diffeomorphic transformation.
... Non-rigid registration is suitable for the non-rigid object deformation, and can achieve local deformation more accurately. The commonly used models of non-rigid registration include: total variation (TV) model [19,20], diffusion model [16], curvature model [17], elastic model [6,18], viscous fluid model [13] and optical flow field model [40] etc. Fischler and Elschlager [18] proposed an embedding elastic metric to constrain relative movement. Bajcsy and Kovacic [2] used affine transformation to correct large global differences before elastic registration. ...
... (Ω) is defined by (3.4). Usually the regularization term S[ϕ] can be elastic modulus [6], diffusion modulus [16] or curvature modulus [17], etc. ...
... so the following discussion will focus on minimizing E u i ,λ i (u). To obtain the optimal solution of (4.27), the first-order variation of E u i ,λ i (u) can be used to deduce the Euler-Lagrange equation as follow (see [16] for more details) ...
Image registration has played an important role in image processing problems, especially in medical imaging applications. It is well known that when the deformation is large, many variational models cannot ensure diffeomorphism. In this paper, we propose a new registration model based on an optimal control relaxation constraint for large deformation images, which can theoretically guarantee that the registration mapping is diffeomorphic. We present an analysis of optimal control relaxation for indirectly seeking the diffeomorphic transformation of Jacobian determinant equation and its registration applications, including the construction of diffeomorphic transformation as a special space. We also provide an existence result for the control increment optimization problem in the proposed diffeomorphic image registration model with an optimal control relaxation. Furthermore, a fast iterative scheme based on the augmented Lagrangian multipliers method (ALMM) is analyzed to solve the control increment optimization problem, and a convergence analysis is followed. Finally, a grid unfolding indicator is given, and a robust solving algorithm for using the deformation correction and backtrack strategy is proposed to guarantee that the solution is diffeomorphic. Numerical experiments show that the registration model we proposed can not only get a diffeomorphic mapping when the deformation is large, but also achieves the state-of-the-art performance in quantitative evaluations in comparing with other classical models.
... Image registration is the process that attempts to seek an optimal transformation between two or more images to establish a geometric correspondence. In the last few decades, a great number of popular registration models have been developed to obtain reasonable transformations, including total variation (TV) model [23,35], modified total variation (MTV) model [12], total fractional variation (TFV) model [58,27], diffusion model [19,37], curvature model [20,21,29], elastic model [22], viscous fluid model [11,15], and optical flow model [32]. Although these registration models can generate smooth transformations for relatively small deformations, not all models are effective for large deformations. ...
... where > 0 is a constant to balance ( ) and ( ). We refer readers to [31,44,14] for many classical similarity measures, and [13,19,20,22,58,59,61] for the popular regularizers in image registration. ...
Diffeomorphic registration has become a powerful approach for seeking a smooth and invertible spatial transformation between two coordinate systems which have been measured via the template and reference images. While the pointwise volume-preserving constraint is effective for some problems, it is too stringent for many other problems especially when the local deformations are relatively large, because it may lead to a poor large-deformation for enforcing local matching.In this paper, we propose a novel bi-variant diffeomorphic image registration model with the soft constraint of Jacobian equation, which allows local deformations to shrink and grow in a flexible range.The Jacobian determinant of the transformation is explicitly controlled by optimizing the relaxation function. To prevent deformation folding and enhance the smoothness of deformation, we not only impose a positivity constraint in optimizing the relaxation function, but also employ a regularizer to ensure the smoothness of the relaxation function.Furthermore, the positivity constraint ensures that is as close to one as possible, which helps to obtain a volume-preserving transformation on average.We further analyze the existence of the minimizer for the variational model and propose a penalty splitting method with a multilevel strategy to solve this model. Numerical experiments show that the proposed algorithm is convergent, and the positivity constraint can control the range of relative volume and not compromise registration accuracy. Moreover, the proposed model produces diffeomorphic maps for large deformation, and achieves better performance compared to the several existing registration models.
... Image registration is a challenging image processing task which can be encountered in diverse fields such as astronomy (Beroiz et al. 2020), optics (Quang et al. 2020), biology (Alavi and Bar-Joseph 2020), chemistry (Zhang et al. 2021), life sciences (Schindelin et al. 2015), remote sensing (Goshtasby 2005) and medical imaging (Scherzer 2006). It consists in constructing a geometric correspondence between two or more images of the same object which were taken at different times or were acquired using different devices (scanner, IRM, etc.), see e.g., Fischer and Modersitzki (2002), Gigengack et al. (2011), Modersitzki (2009), Sotiras et al. (2013), Theljani and Chen (2019) and Zitova and Flusser (2003). The image registration problem can be described as follows: Given a fixed image R, which is called the reference image, and a moving image T , which is called the template image, both images are represented by scalar-valued functions R, T : ⊂ R d −→ R, where d denotes the spatial dimension of the images. ...
... Various regularizers have been proposed in the literature. For example, first-order derivatives that are based on total variation (see e.g., Hu et al. 2014;Papafitsoros et al. 2013) and diffusion (see e.g., Fischer and Modersitzki 2002). For monomodal images, i.e., when the images were acquired using the same modality (MRI T1, T2, X-ray laser, etc.), both images R and T (u) have the same contrast and similar features. ...
In this paper, we are interested in deformable registration models for multi-modality images. We introduce a new similarity term for image registration which is based on the geometric information (edges and thin structures) extracted from the images using the Blake-Zisserman's energy. The latter is well suited for detecting discontinuities at different scales, i.e., of first and second order. We start by giving a theoretical analysis of the proposed model. Then, we use the Gauss-Newton method and multilevel technique to speed up the numerical computations for the solution of this model. Finally, we present some numerical results of the new approach and we compare them with those obtained by some existing methods. The experiments illustrate the efficiency and effectiveness of the proposed model.
... The variational model is among the most successful and accurate approaches to calculate a deformation between two images [3]. Given a specific regularization term, such a model has a clear mathematical structure and it is also well understood which mathematical space the solution lies in, e.g., Hilbert space [4]- [6], bounded variation [7], [8], etc. However, the variational model has limitations: (1) For each image pair, the hyper-parameter λ needs to be tuned carefully to deliver a precise deformation. ...
... This is followed by a deformable transformation which has more degrees of freedom as well as higher capability to describe local deformations. There is a wide range of classical variational methods to account for local deformations such as diffusion models [4], total variation models [7], [8], fluid models [5], [6], elastic models [24]- [26], biharmonic (linear curvature) models [27], [28], mean curvature models [29], [30], optical flow models [3], [31], [32], fractional-order variation models [33], [34], non-local graph models [35]- [37], etc. The free-form deformation (FFD) methods based on B-splines model [38], [39] are able to accurately model global and local deformations with fewer degrees of freedom parameterized by control points. ...
Data-driven deep learning approaches to image registration can be less accurate than conventional iterative approaches, especially when training data is limited. To address this issue and meanwhile retain the fast inference speed of deep learning, we propose VR-Net, a novel cascaded variational network for unsupervised deformable image registration. Using a variable splitting optimization scheme, we first convert the image registration problem, established in a generic variational framework, into two sub-problems, one with a point-wise, closed-form solution and the other one being a denoising problem. We then propose two neural layers (i.e. warping layer and intensity consistency layer) to model the analytical solution and a residual U-Net (termed generalized denoising layer) to formulate the denoising problem. Finally, we cascade the three neural layers multiple times to form our VR-Net. Extensive experiments on three (two 2D and one 3D) cardiac magnetic resonance imaging datasets show that VR-Net outperforms state-of-the-art deep learning methods on registration accuracy, whilst maintaining the fast inference speed of deep learning and the data-efficiency of variational models.
... The variational model is among the most successful and accurate approaches to calculate a deformation between two images [3]. Given a specific regularization term, such a model has a clear mathematical structure and it is also well understood which mathematical space the solution lies in, e.g., Hilbert space [4]- [6], bounded variation [7], [8], etc. However, the variational model has limitations: (1) For each image pair, the hyper-parameter λ needs to be tuned carefully to deliver a precise deformation. ...
... This is followed by a deformable transformation which has more degrees of freedom as well as higher capability to describe local deformations. There is a wide range of classical variational methods to account for local deformations such as diffusion models [4], total variation models [7], [8], fluid models [5], [6], elastic models [24]- [26], biharmonic (linear curvature) models [27], [28], mean curvature models [29], [30], optical flow models [3], [31], [32], fractional-order variation models [33], [34], non-local graph models [35]- [37], etc. The free-form deformation (FFD) methods based on B-splines model [38], [39] are able to accurately model global and local deformations with fewer degrees of freedom parameterized by control points. ...
Data-driven deep learning approaches to image registration can be less accurate than conventional iterative approaches, especially when training data is limited. To address this whilst retaining the fast inference speed of deep learning, we propose VR-Net, a novel cascaded variational network for unsupervised deformable image registration. Using the variable splitting optimization scheme, we first convert the image registration problem, established in a generic variational framework, into two sub-problems, one with a point-wise, closed-form solution while the other one is a denoising problem. We then propose two neural layers (i.e. warping layer and intensity consistency layer) to model the analytical solution and a residual U-Net to formulate the denoising problem (i.e. generalized denoising layer). Finally, we cascade the warping layer, intensity consistency layer, and generalized denoising layer to form the VR-Net. Extensive experiments on three (two 2D and one 3D) cardiac magnetic resonance imaging datasets show that VR-Net outperforms state-of-the-art deep learning methods on registration accuracy, while maintains the fast inference speed of deep learning and the data-efficiency of variational model.
... For the multimodality image registration, mutual information, normalized cross correlation or normalized gradient field may be a good candidate [29,32,39,41]. For the regularization term, there also exist many different choices [5,7,17,18,[21][22][23][24]35,49] and here, we mainly highlight the hyperelastic regularizer [7,21]. The hyperelastic regularizer in image registration was firstly used by Droske and Rumpf [21] in 2004. ...
... . and the proof is complete. (22) Proof Firstly, by Lemma 2 in Sect. 3.3 of [46], we have ...
Image segmentation is to extract meaningful objects from a given image. For degraded images due to occlusions, obscurities or noises, the accuracy of the segmentation result can be severely affected. To alleviate this problem, prior information about the target object is usually introduced. In Chan et al. (J Math Imaging Vis 60(3):401–421, 2018), a topology-preserving registration-based segmentation model was proposed, which is restricted to segment 2D images only. In this paper, we propose a novel 3D topology-preserving registration-based segmentation model with the hyperelastic regularization, which can handle both 2D and 3D images. The existence of the solution of the proposed model is established. We also propose a converging iterative scheme to solve the proposed model. Numerical experiments have been carried out on the synthetic and real images, which demonstrate the effectiveness of our proposed model.
... [5][6][7] In contrast to other traditional regularizers generating globally smooth deformations, e.g. elastic 8 diffusion 9 and curvature regularizers 10,11 the TV regularizer is able to produces locally non-smooth deformations, which are required in matching several moved objects or partially occluded objects in medical applications, particularly at organ boundaries during the breathing induced organ motion. ...
... We propose an iterative algorithm to solve the minimization of L u; w; k 1 ; k 2 ð Þ ; see Algorithm 1. Since u and w are coupled together in the minimization problem (9), it is very difficult to solve all variables simultaneously. We separate the minimization problem into two sub-problems and develop an alternating minimization procedure to approximate the solution. ...
Variational methods for image registration basically involve a regularizer to ensure that the resulting well-posed problem admits a solution. Different choices of regularizers lead to different deformations. On one hand, the conventional regularizers, such as the elastic, diffusion and curvature regularizers, are able to generate globally smooth deformations and generally useful for many applications. On the other hand, these regularizers become poor in some applications where discontinuities or steep gradients in the deformations are required. As is well-known, the total (TV) variation regularizer is more appropriate to preserve discontinuities of the deformations. However, it is difficult in developing an efficient numerical method to ensure that numerical solutions satisfy this requirement because of the non-differentiability and non-linearity of the TV regularizer. In this work we focus on computational challenges arising in approximately solving TV-based image registration model. Motivated by many efficient numerical algorithms in image restoration, we propose to use augmented Lagrangian method (ALM). At each iteration, the computation of our ALM requires to solve two subproblems. On one hand for the first subproblem, it is impossible to obtain exact solution. On the other hand for the second subproblem, it has a closed-form solution. To this end, we propose an efficient nonlinear multigrid (NMG) method to obtain an approximate solution to the first subproblem. Numerical results on real medical images not only confirm that our proposed ALM is more computationally efficient than some existing methods, but also that the proposed ALM delivers the accurate registration results with the desired property of the constructed deformations in a reasonable number of iterations.
... Usually the choice of regularizer can be classified into two main categories: the first type is to limit the displacement field u(x) to the parametric model [10][11][12][13][14], for example, rigid or affine transformations (parameterized by rotation, scaling, and translation) or linear combinations of a set of basis functions (B-splines) [3,5,[15][16][17][18]; the second type is based on the derivative of the displacement field. At present, there are regularizers based on first-order derivatives, such as elastic regularizer [19][20][21], diffusion regularizer [22], total variational regularizer [23,24], modified total variational regularizer [25,26], total fractionalorder regularizer [27], and the ones based on higher-order derivatives, such as linear curvature [28,29], mean curvature [8,30], and Gaussian curvature [31]. It has been proved that, in many cases, the selection method of the first kind of a regularizer is too strict, and the required transformation cannot be guaranteed to be included in the parametric model. ...
... In discrete simulation, the midpoint quadrature formula can be used to approximate the integral. According to (22) and (23), the discrete form of distance measurement D(2) can be written directly as follows: ...
We propose a constrained linear curvature image registration model to explicitly control the deformation according to the transformed Jacobian matrix determinant using point-by-point inequality constraints in this paper. In addition, an effective numerical method is proposed to solve the resulting inequality constrained optimization model. Finally, some numerical examples are given to prove the obvious advantages of the curvature image registration model with inequality constraints.
1. Introduction
In image processing, people are interested not only in analyzing an image but also comparing or combining information from images which take different time, different places, different viewpoints, or different modalities. Thus, image registration is one of the most useful and challenging problems in the field of image processing. Its main idea is to find a geometric transformation which aligns points in one view of one object with corresponding points in another view of the same or similar object. At present, there are a large number of application areas which require image registration, such as computer vision, biological imaging, remote sensing, and medical imaging. For comprehensive surveys of these applications, refer to [1–5].
The basic framework of image registration can be described as follows: given two images of the same object, which are called reference image and template image , respectively, and our purpose is to find a vector value transformation as defined below:or equivalently find the unknown displacement field :so that the transformed template image is as similar to the reference image as possible. Here, denotes spatial dimension of the given images.
Without loss of generality, here we focus on throughout this paper, but it is easy to generalize to with some additional modifications. The variational model is an important tool for studying image registration and has been widely concerned by many researchers [5–9]. This variational model treats the image registration problem as a minimization problem of the joint energy functional in the following form:wherewhere denotes distance measure which quantifies distance or similarity of the transformed template image and reference , for other choices on , refer to [5, 7], is the deformation regularizer which constrains and ensures the well-posedness of the problem, and is a regularization parameter which balances similarity and regularity of displacement.
We all know that different regularizers will produce displacement fields with different degrees of smoothness and the selection of a regularizer is critical to the solution of the problem and its properties; for more details, refer to [5]. Usually the choice of regularizer can be classified into two main categories: the first type is to limit the displacement field to the parametric model [10–14], for example, rigid or affine transformations (parameterized by rotation, scaling, and translation) or linear combinations of a set of basis functions (-splines) [3, 5, 15–18]; the second type is based on the derivative of the displacement field. At present, there are regularizers based on first-order derivatives, such as elastic regularizer [19–21], diffusion regularizer [22], total variational regularizer [23, 24], modified total variational regularizer [25, 26], total fractional-order regularizer [27], and the ones based on higher-order derivatives, such as linear curvature [28, 29], mean curvature [8, 30], and Gaussian curvature [31]. It has been proved that, in many cases, the selection method of the first kind of a regularizer is too strict, and the required transformation cannot be guaranteed to be included in the parametric model. Therefore, the second kind of method is a common method to select a regularizer. For the second method, it is easy to implement for low-order regularizers, while they are less effective than high-order ones in producing smooth displacement fields which are important in some applications including medical imaging. Although the registration models based on a higher-order regularizer can produce more satisfactory registration results visually, they do not take into account mesh folding.
In fact, the regularity of the displacement field is also an important measure in image registration [32]. In many variational models (1) that currently exist, although they can produce satisfactory registration results visually, they cannot ensure that the transformation found is reversible. The irreversibility of the transformation means that the displacement field is not regular. In this case, there will be mesh folding during the registration process, which is not allowed in practical applications. Therefore, it is necessary to avoid mesh folding during the registration process. Currently, a direct idea to avoid mesh folding is to use a larger regularization parameter . However, such a value will cause the similarity between the transformed template image and the reference image to become worse. In order to avoid mesh folding phenomenon, some scholars have proposed to add an additional regular term on the transformed Jacobian matrix determinant in the objective function formula (3) [33–36], i.e.,where represents the determinant of the Jacobian matrix of the transformation. However, this method only penalizes the irregular displacement field as a whole, while the local displacement field cannot be guaranteed to be regular [32]. In addition, this method is only effective for the smaller regularizer parameter , and increasing the value of usually leads to ill-posed optimization problems [37]. To solve this problem, Haber and Modersitzki proposed a new registration model by adding additional explicit volume inequality constraints [32]; however, this constrained method usually leads to solving a large-scale highly nonlinear inequality constrained optimization problem. Other methods to ensure the regularity of the displacement field can be found in the literature [20, 38–43]. However, some of them require more computation time due to the complexity of the regularizer.
There are two purposes for image registration. One is to enhance some similarities between two images by geometrically transforming one of the given two images. The other is to ensure that this transformation is reasonable. In fact, it is equivalent to find geometric transformation and the displacement field in the framework of variational model. If the displacement field is irregular, the transformation is considered unreasonable, and then the mesh folding phenomenon will appear which is not allowed in practical applications. In this paper, we propose a new image registration model by integrating the evaluation criteria to measure the registration results directly into the basic framework of the variational model (3).
The rest of the paper is organized as follows: in Section 2, we propose a new constrained linear curvature image registration model. Then in Section 3, we discuss the numerical method for solving the new model by using a combination of the multiplier method and Gauss–Newton scheme with the Armijos line search and further combine with a multilevel method to achieve fast convergence. Next, some experimental results from syntectic and real images are illustrated in Section 4. Finally, conclusions and future work are summarized in Section 5.
2. Constrained Linear Curvature Image Registration Model
Firstly, we briefly review the Fischer–Modersitzki’s linear curvature image registration model [25, 27]. Choosing in (3) based on an approximation to the curvature of the surface of the displacement field is given by the following form:
There are two major advantages to the particular choice of the regularizer. Firstly, it can penalize oscillations; secondly, without requiring an additional affine linear preregistration step, it can produce visually more satisfactory registration results than a diffusion model and an elastic model for smooth displacement fields. However, a mesh folding phenomenon is not considered in this linear curvature model. In order to avoid this, the evaluation criteria to measure the registration results are directly integrated into the basic framework of the variational model (3), and we propose a constrained linear curvature image registration model in the following form:where
Compared with model (5), our new model can ensure that the displacement field is regular both globally. In addition, the new model prevents mesh folding even for very small regularization parameters . Finally, visually pleasing registration results can be obtained by using our new model with low computing time for smooth registration problems. The numerical solution of the new model (7) is given below.
3. Numerical Solution of the New Model
In general, it is difficult to solve the optimization problem (7) by the analytic method. Thus it is necessary to adopt the numerical method and appropriate discretization. In this paper, we choose the discretize-optimize method which aims to take advantage of efficient optimization techniques. In this section, we first discuss briefly the discretization we use and then describe the details of numerical algorithms.
3.1. Finite Difference Discretization
Assume that given discrete images have pixels. For simplicity, the image region is further assumed to be , and then each side of these cell-centered images has width . Thus the discrete domain can be denoted by
3.1.1. Discretization of Regularizer
The discrete form of the continuous displacement field can be represented by , where and are the discrete grid functions defined on the discrete region . For convenience, let , and . Since the curvature regularizer is expressed based on the Laplacian operator which can be regarded as the product of gradient operator and divergence operator , we introduce the symbols and to represent their discrete forms, respectively. The discrete gradient operator can be defined at each pixel by the following form:where
The displacement field satisfies the homogeneous Neumann boundary conditions on the boundary of the image region :
Through the analysis of continuous setting, we know that the discrete divergence operator is the negative conjugate transposition of the gradient operator, namely, . Thus, it can be defined by the following form:where is a vector. For the convenience of calculation, the grid functions and can be changed into column vectors and according to lexicographical ordering, respectively:
We can get , , and , where . Discrete gradient operator can also be expressed as the product of matrix and the vector in the following form:
Let
By this notation, we can get
Let , , and
Then the discrete form of (18) is as follows:
According to the midpoint quadrature formula, the linear curvature regularizer has the following discrete form:where .
3.1.2. Discretization of Template and Reference
For a given discrete image, if we want to know the gray value at any spatial location other than the grid point, then image interpolation is needed. In order to take full advantage of the fast and effective optimization method, a smooth cubic -spline is used for interpolation. Next, and are used to represent the continuous smooth approximation of template image and reference image , respectively. Letand .
Thus the discrete reference image and transformed template image can be represented by the following form, respectively:and further we can get the Jacobian of :where and the Jacobian of is a block matrix with diagonal blocks.
3.1.3. Discretization of Distance Measure
Although it is in a continuous setting, it is not possible to compute integrals analytically. Thus it is necessary to use numerical integration. In discrete simulation, the midpoint quadrature formula can be used to approximate the integral. According to (22) and (23), the discrete form of distance measurement can be written directly as follows:
In addition, the derivative of the discrete functional on can also be calculated and has the following form:
Furthermore, we can calculate the second derivative of the distance measurement :where . On one hand, it is consuming and numerically unstable to compute higher-order derivatives (27) in registering two images for practical applications. On the other hand, the difference between and will become smaller if the template image is well registered. To have an efficient and stable numerical algorithm as proposed in work [5], can be approximated by the following form:
3.1.4. Discretization of Inequality Constraint Functional
In model (7), the inequality constraint functional is defined by
According to the previous analysis, the discrete form of the partial derivative of the continuous displacement field element can be expressed as follows:
Obviously, , and , where . Letwhere symbol denotes the multiplication of the corresponding elements of two vectors and . For the convenience of calculation, let denote the -th element of , . Therefore, the continuous inequality constraint function has the following discrete form:
Since the first-order variation of the continuous inequality constraint function on continuous displacement field is as follows:
Thus we can get the discrete form of the first-order variation :
Obviously, , , and .
3.2. Solving the Discrete Optimization Problem
According to the above analysis, inequality constrained functional (7) has the following discrete form:
Below we use the multiplier method to numerically solve the inequality constrained optimization problem (35). The basic idea of this method is to transform the original problem into a series of unconstrained optimization problems to solve and simultaneously estimate the Lagrangian multiplier. For more details on multiplier scheme, see [37]. Before solving (35), let us briefly review the multiplier method of inequality constrained optimization.
3.2.1. Multiplier Method for Inequality Constrained Problems
Consider the following inequality constrained optimization problem:
Let , and the above inequality constraint can be transformed into the following equivalent equality constraint problem:
In this case, the augmented Lagrange function can be expressed as
In order to eliminate the auxiliary variables , the minimization of with respect to variable can be considered. According to the first-order necessary condition, let
We can get
Namely,
Therefore, when ,that is to say,
Thus when , we have
And when , we can obtain
According to the above two cases,
Substituting it into formula (38), we can get the corresponding augmented Lagrange function of (36):
Since the multiplier vector needs to be updated to solve the inequality constrained optimization problems (36) by using the multiplier method, next we derive the multiplier iterative formula. Firstly, fix the penalty parameter to some value at its -th iteration, and fix at the current estimate . Secondly, perform minimization with respect to . Using to denote the approximate minimizer of , then we can get by the optimality conditions for unconstrained minimization that
Let satisfy the KKT conditions for (37), then we have
By comparing (48) with (49), we can deduce that
According to (50), to improve the current estimate of the Lagrange multiplier vectors, the multiplier iteration formula can be given by the following form:
Then, taking (43) into the multiplier iteration formula (51), we have
Furthermore, it can be written as
Similarly, take (43) into the termination criterion
We can get
3.2.2. Multiplier Method for Solving Model
Next, we use the multiplier method to solve the model (35). Firstly, we construct the augmented Lagrange function for solving model (35):
The corresponding multiplier iteration formula has the following form:
And the corresponding stopping criterion is
Although the augmented Lagrangian function (57) of the model (35) contains the min function, it is still continuously differentiable; for details, see [37, 44]. The detailed steps of the multiplier method for solving the model (35) can be summarized by Algorithm 1.
Step 1: input the initial value: , the objective function and its gradient , inequality constrained vector , and the transpose of its Jacobian matrix ; let , , , , , , , and .
Step 2: solving the subproblem. With as the initial point, solve the minimum value of the unconstrained subproblem (51) by using the Gauss–Newton scheme with Armijo line search.
Step 3: check the termination condition. If or , where is defined by (57), the iteration is stopped, and is output as the approximate minimum of the original problem; otherwise, go to Step 4.
Step 4: update penalty parameters. If , let ; otherwise, set .
Step 5: update multiplier vector. Calculate
Step 6: set , and go to Step 1.
... Here the first term, the regularizer, S(u) controls the smoothness of u and reflects our expectations in penalizing unlikely transformations. Various regularizers have been proposed, such as first-order derivatives-based total variation [15,16], diffusion [17] and elastic [18] regularizers, and higher-order derivatives-based linear curvature [19], mean curvature [20] and Gaussian curvature [21] regularizers; refer also to [8,13,[22][23][24]. ...
... A regularizer controls the smoothness. Our primary choice for smoothness control is the diffusion regularizer [17] which uses first-order derivatives and promotes smoothness. As affine linear transformations are not included in the kernel of the H 1 -regularizer, we desire a regularizer which can penalize such transformations. ...
In this work, we propose a new variational model for multi-modal image registration and present an efficient numerical implementation. The model minimizes a new functional based on using reformulated normalized gradients of the images as the fidelity term and higher-order derivatives as the regularizer. A key feature of the model is its ability of guaranteeing a diffeo-morphic transformation which is achieved by a control term motivated by the quasi-conformal map and Beltrami coefficient. The existence of the solution of this model is established. To solve the model numerically, we design a Gauss-Newton method to solve the resulting discrete optimization problem and prove its convergence; a multilevel technique is employed to speed up the initialization and avoid likely local minima of the underlying functional. Finally, numerical experiments demonstrate that this new model can deliver good performances for multi-modal image registration and simultaneously generate an accurate diffeomorphic transformation.
... Common choices for L s (·, ·) include voxelwise mean squared difference, mean absolute difference, and normalized cross-correlation (CC), 23 while a common choice for L r (·) includes the ℓ 2 -norm of spatial gradients. 24,16 Although VxM is capable of training a network for each target-source pair separately, its real advantage resides in training the network using multiple targetsource pairs and then using the trained CNN to register images using one forward pass through the network. ...
Purpose: To perform image registration and averaging of multiple free-breathing single-shot cardiac images, where the individual images may have a low signal-to-noise ratio (SNR). Methods: To address low SNR encountered in single-shot imaging, especially at low field strengths, we propose a fast deep learning (DL)-based image registration method, called Averaging Morph with Edge Detection (AiM-ED). AiM-ED jointly registers multiple noisy source images to a noisy target image and utilizes a noise-robust pre-trained edge detector to define the training loss. We validate AiM-ED using synthetic late gadolinium enhanced (LGE) imaging data from the MR extended cardiac-torso (MRXCAT) phantom and retrospectively undersampled single-shot data from healthy subjects (24 slices) and patients (5 slices) under various levels of added noise. Additionally, we demonstrate the clinical feasibility of AiM-ED by applying it to prospectively undersampled data from patients (6 slices) scanned at a 0.55T scanner. Results: Compared to a traditional energy-minimization-based image registration method and DL-based VoxelMorph, images registered using AiM-ED exhibit higher values of recovery SNR and three perceptual image quality metrics. An ablation study shows the benefit of both jointly processing multiple source images and using an edge map in AiM-ED. Conclusion: For single-shot LGE imaging, AiM-ED outperforms existing image registration methods in terms of image quality. With fast inference, minimal training data requirements, and robust performance at various noise levels, AiM-ED has the potential to benefit single-shot CMR applications.
... (i) by analogy with physical models: for instance, elastic models (Broit 1981) in which the shapes to be matched are considered as the observations of the same body before and after being subject to constraints, fluid models (Christensen et al. 1996) in which the shapes to be matched are viewed as fluids evolving in accordance with Navier-Stokes equations, diffusion models (Fischer and Modersitzki 2002), curvature models (Fischer and Modersitzki 2003), flows of diffeomorphisms (Beg et al. 2005), and nonlinear models (Burger et al. 2013, Derfoul and Le Guyader 2014, Droske and Rumpf 2004, Le Guyader and Vese 2011, Rumpf and Wirth 2009, Rabbitt et al. 1995, Pennec et al. 2005 to allow for large deformations. (ii) by interpolation or approximation-driven models: it means that the deformation is described in a parameterizable set. ...
... To aleviate this, variational methods generally introduce a regularization term in the minimization functional to ensure well-posedness and restrict the solution space to transformations that conform with certain desirable properties. The diffusion regularization 6 serves as a straightforward approach to avoid oscillatory and non-smooth solutions. The curvature regularization 7,8 provides similar benefits without penalizing rigid motion while combinations of the two 9 can offer benefits in terms of registration accuracy and deformation field plausibility. ...
Variational image registration methods commonly employ a similarity metric and a regularization term that renders the minimization problem well-posed. However, many frequently used regularizations such as smoothness or curvature do not necessarily reflect the underlying physics that apply to anatomical deformations. This, in turn, can make the accurate estimation of complex deformations particularly challenging. Here, we present a new highly flexible regularization inspired from the physics of fluid dynamics which allows applying independent penalties on the divergence and curl of the deformations and/or their nth order derivative. The complexity of the proposed generalized div-curl regularization renders the problem particularly challenging using conventional optimization techniques. To this end, we develop a transformation model and an optimization scheme that uses the divergence and curl components of the deformation as control parameters for the registration. We demonstrate that the original unconstrained minimization problem reduces to a constrained problem for which we propose the use of the augmented Lagrangian method. Doing this, the equations of motion greatly simplify and become managable. Our experiments indicate that the proposed framework can be applied on a variety of different registration problems and produce highly accurate deformations with the desired physical properties.
... The goal of this regularization is to promote smooth and continuous vector fields, by penalizing abrupt variations and discontinuities. By adding this regularization term to the cost function, the model is encouraged to produce a vector field whose gradient magnitude is small, which avoids sharp jumps and discontinuities [58]. The second regularization term is called "curvature regularization" or "L 2 Hessian norm regularization" [59]. ...
Medical image registration is a crucial step in computer-assisted medical diagnosis, and has seen significant progress with the adoption of deep learning methods like convolutional neural networks (CNN). Creating a deep learning network for image registration is complex because humans can’t easily prepare or supervise the training data unless it’s very basic. This article presents an innovative approach to unsupervised deep learning-based multilevel image registration approach. We propose to develop a CNN to detect the geometric features, such as edges and thin structures, from images using a loss function derived from the Blake-Zisserman energy. This method enables the detection of discontinuities at different scales without relying on labeled data. Subsequently, we use this geometric information extracted from the input images, to define a second loss function and to perform our multimodal image registration process. Furthermore, we introduce a novel deep neural network architecture for multilevel image registration, offering enhanced precision and efficiency compared to traditional methods. Numerical simulations are employed to demonstrate the accuracy and relevance of our approach. We perform some numerical simulations to show the accuracy and the relevance of our approach for multimodal registration and its multilevel implementation.
... From a mathematical point of view, the regularization term turns an ill-posed problem into a well-posed one, i.e., leads to a unique minimizer and sometimes to a convex objective function. There are several regularizers that have been used in previous works, for example first-order derivatives that are based on total variations (see [10]) and diffusion (see [4]). This turns our attention for how to choose the "best" regularization term that gives the more possible plausible transformations. ...
In this work, we use the geometric information, such as edges and thin structures, to build a similarity measure for deformable registration models of multi-modality images. The idea is to extract a geometric information from the images and then use it to build a robust and efficient similarity term. In order to extract this information, we use the Blake-Zisserman’s energy that is well suited for detecting discontinuities at different scales, i.e. of first and second order. In addition, we present a theoretical analysis of the proposed model. For the numerical solution of the model, we use a gradient descent method and iteratively solve corresponding the EulerLagrangian equation. We present some numerical results that demonstrate the efficiency of the proposed model.
... is diffusion regularizer [9] and total variation regularizer [2] respectively. In the next section, we propose to use an alternating method [6,23] in combination with a Split Bregman method to solve (2.4). ...
In this paper, we propose an efficient numerical method for registration and intensity correction of images. We first introduced a variational image registration model in our previous work. The variational minimization of the functional in the model is solved by using an alternating minimization method. In this work, we improve the numerical solution of the minimization problem by applying the Split Bregman method together with the alternating minimization method. As expected, our numerical experiments show that the proposed method converges rapidly and is significantly more accurate than the method used in our previous work. Subject Classification: (2010) 65K10, 65M06, 75G65.
... A diffusive regularizer [13] was included to physically define the deformation model during image registration to ensure the robustness and computation speed. To avoid the local minimal value and optimize the joint objective function, the L-BFGS [14] was applied as an optimizer using a discretized analytic derivative and was initialized with the analytic Hessian of the regularizer, which can be interpreted as a special case of linear elasticity as follows:[ ...
Background:
Accurate and precise alignment of histopathology tissue sections is a key step for the interpretation of the proteome topology and cell level three-dimensional (3D) reconstruction of diseased tissues. However, the realization of an automated and robust method for aligning nonglobally stained immunohistochemical (IHC) sections is still challenging. In this study, we aim to assess the feasibility of multidimensional graph-based image registration on aligning serial-section and whole-slide IHC section images.
Materials and methods:
An automated, patch graph-based registration method was established and applied to align serial, whole-slide IHC sections at ×10 magnification (average 32,947 × 27,054 pixels). The alignment began with the initial alignment of high-resolution reference and translated images (object segmentation and rigid registration) and nonlinear registration of low-resolution reference and translated images, followed by the multidimensional graph-based image registration of the segmented patches, and finally, the fusion of deformed patches for inspection. The performance of the proposed method was formulated and evaluated by the Hausdorff distance between continuous image slices.
Results:
Sets of average 315 patches from five serial whole slide, IHC section images were tested using 21 different IHC antibodies across five different tissue types (skin, breast, stomach, prostate, and soft tissue). The proposed method was successfully automated to align most of the images. The average Hausdorff distance was 48.93 μm with a standard deviation of 14.94 μm, showing a significant improvement from the previously published patch-based nonlinear image registration method (average Hausdorff distance of 93.89 μm with 50.85 μm standard deviation).
Conclusions:
Our method was effective in aligning whole-slide tissue sections at the cell-level resolution. Further advancements in the screening of the proteome topology and 3D tissue reconstruction could be expected.
... Therefore, a well-posed image registration problem can be written as the following framework: u = arg min u∈A S(u) + µR(u), (1.2) where µ > 0, S(u) ≜ ∫ Ω [T (x + u(x)) − D(x)] 2 dx, R(u) is regularization and A is an appropriate function space. Based on framework (1.2), many variational image registration models are proposed [4,[13][14][15][16][17][18][19][20][21]. Some models preserve the discontinuity of deformation [18,19], some [13,14,16,17,20,21] perform better on smooth image registration and the others [4] have advantage on both smooth and non-smooth image registration. ...
In Han and Wang (2020), a 2D diffeomorphic image registration model is proposed to eliminate mesh folding. To solve the 2D diffeomorphic model, a diffeomorphic fractional-order image registration algorithm(DFIRA for short) is proposed in Han and Wang (2020). DFIRA achieves a satisfactory image registration result but it costs too much CPU time. To accelerate DFIRA, we propose a fast multi grid algorithm for 2D diffeomorphic image registration model in this paper. This algorithm achieves a satisfactory image registration result by using only one-tenth CPU time of DFIRA. At the same time, no mesh folding occurs in proposed algorithm. Furthermore, convergence analysis of the proposed algorithm is also presented. Moreover, numerical tests are also performed to show that the proposed algorithm is competitive compared with some other algorithms.
... For the multimodality image registration, mutual information, normalized cross correlation or normalized gradient field may be a good candidate [33,36,46,49]. For the regularization term, there also exist many different choices [5,7,21,22,25,26,27,28,39,62] and here, we mainly highlight the hyperelastic regularizer [7,25]. The hyperelastic regularizer in image registration was firstly used by Droske and Rumpf [25] in 2004. ...
Image segmentation is to extract meaningful objects from a given image. For degraded images due to occlusions, obscurities or noises, the accuracy of the segmentation result can be severely affected. To alleviate this problem, prior information about the target object is usually introduced. In [10], a topology-preserving registration-based segmentation model was proposed, which is restricted to segment 2D images only. In this paper, we propose a novel 3D topology-preserving registration-based segmentation model with the hyperelastic regularization, which can handle both 2D and 3D images. The existence of the solution of the proposed model is established. We also propose a converging iterative scheme to solve the proposed model. Numerical experiments have been carried out on the synthetic and real images, which demonstrate the effectiveness of our proposed model.
... Regularization term in inverse problem is always important and challenging and can make a model more efficient. Some of the well known regularization terms are diffusion based [6], [7], total variation based [8], [9], elastic based [10] (low order models) and curvature based [11], non-linear mean curvature based [12], linear curvature based [13] and Gaussian curvature based [14] (higher order models). ...
Registration of multi-modal images is one of the challenging problems in image processing nowadays. In this paper, two novel non-rigid registration models are proposed for multi-modality images. In model 1, mutual information of the template and reference images is used as data fitting term with Gaussian curvature regularization. This approach may not give satisfactory results in noisy images or images having bias field. To overcome this drawback, model 2 is proposed which is based on normalized gradient of both template and reference images as a data fitting term instead of mutual information. To get best transformations, both the models are minimized by using Augmented Lagrangian Method. The proposed models can register multi-modality images without effecting edges and other important fine details and are also tested on various medical images like (T1-T2 MRI, PD weighted-T2 MRI) noisy and synthetic images. The proposed models are also tested on a well known free available Brainweb dataset, where they produced satisfactory results. From experimental results, it can be observed that normalized gradient field based model gives better results than mutual information based model. Comparison is done qualitatively and quantitatively through Jaccard Similarity Coefficient.
... where \scrS (\bfity ) is the regularizer which can rule out the unwanted solutions and \alpha > 0 is a positive parameter to balance these two terms. There exist many different regularizers which lead to many nonlinear registration models, such as the elastic model [6], fluid model [12], diffusion model [19], total variation (TV) model [22], MTV (modified TV) model [13], linear curvature model [20,21], mean curvature model [14], Gaussian curvature model [28], and total fractional-order variation model [51]. These models can produce different registration transformations since they are inspired by different physical properties [44], each having advantages in its class of problems, though not all of these models have been tested in registration of 3D images. ...
... However, depending on the underlying physical assumptions on how the image is allowed to deform, the estimated deformations are regularized. Typically, physical models are elastic- [Davatzikos, 1997;Pennec, 2005], fluid- [Christensen, 1996] or diffusion-based [Thirion, 1998;Fischer, 2002;Vercauteren, 2007a]. Diffusion-based models are based on the fact that the Gaussian kernel is the Green's function of the diffusion equation. ...
This thesis presents new computational tools for quantifying deformations and motion of anatomical structures from medical images as required by a large variety of clinical applications. Generic deformable registration tools are presented that enable deformation analysis useful for improving diagnosis, prognosis and therapy guidance. These tools were built by combining state-of-the-art medical image analysis methods with cutting-edge machine learning methods.First, we focus on difficult inter-subject registration problems. By learning from given deformation examples, we propose a novel agent-based optimization scheme inspired by deep reinforcement learning where a statistical deformation model is explored in a trial-and-error fashion showing improved registration accuracy.Second, we develop a diffeomorphic deformation model that allows for accurate multi-scale registration and deformation analysis by learning a low-dimensional representation of intra-subject deformations. The unsupervised method uses a latent variable model in form of a conditional variational autoencoder (CVAE) for learning a probabilistic deformation encoding that is useful for the simulation, classification and comparison of deformations.Third, we propose a probabilistic motion model derived from image sequences of moving organs. This generative model embeds motion in a structured latent space, the motion matrix, which enables the consistent tracking of structures and various analysis tasks. For instance, it leads to the simulation and interpolation of realistic motion patterns allowing for faster data acquisition and data augmentation.Finally, we demonstrate the importance of the developed tools in a clinical application where the motion model is used for disease prognosis and therapy planning. It is shown that the survival risk for heart failure patients can be predicted from the discriminative motion matrix with a higher accuracy compared to classical image-derived risk factors.
... which penalizes local spatial variations in φ d to promote smooth local deformations [4]. Initial experiments revealed that, when objects in the background of the target image are present, the affine transformation degenerate towards extreme compression or expansion. ...
We propose an unsupervised deep learning method for atlas-based registration to achieve segmentation and spatial alignment of the embryonic brain in a single framework. Our approach consists of two sequential networks with a specifically designed loss function to address the challenges in 3D first trimester ultrasound. The first part learns the affine transformation and the second part learns the voxelwise nonrigid deformation between the target image and the atlas. We trained this network end-to-end and validated it against a ground truth on synthetic datasets designed to resemble the challenges present in 3D first trimester ultrasound. The method was tested on a dataset of human embryonic ultrasound volumes acquired at 9 weeks gestational age, which showed alignment of the brain in some cases and gave insight in open challenges for the proposed method. We conclude that our method is a promising approach towards fully automated spatial alignment and segmentation of embryonic brains in 3D ultrasound.
... along with the distance measure. First-order methods include diffusive [Fis01] or elastic [Bro81] regularization, a second-order operator is used in curvature [Fis03b] regularization. ...
Large-scale images that exceed a single computer's random-access memory (RAM) represent a great challenge for nonlinear image registration. One area where these images occur is in digital pathology, where differently stained sections are superimposed to combine their information. Here, registration of large-scale images compensates for the deformation that is caused by a cutting and staining procedure.
In the literature, the decomposition of large-scale, nonlinear problems aims at increasing the available RAM by distributing the problem to multiple computing nodes. Two critical building blocks common among many nonlinear decomposition approaches are the formulation of local nonlinear subproblems and the coupling of the local subproblems to the global problem.
In this thesis, we present the novel combined fine local and coarse global (CLG) registration method. Decomposing the large-scale problem into smaller subproblems, CLG adapts the building blocks from domain decomposition for a single computing node. Unlike previously proposed methods, CLG links independent fine local registration problems to a coarse global representation of the global image, which improves the compatibility of the local solutions. By solving local problems independently of each other, only a fraction of the high-resolution image data needs to be loaded into RAM at once. The proposed method can be expressed as a discretization of a global, variational formulation of the registration problem. The downside of the new approach are potential inconsistencies between the solutions of the subproblems, which are compensated by a blending strategy.
To combine the local solutions, we adapt and compare different blending approaches from the literature. The proposed blending scheme computes a globally smooth deformation and preserves the homogeneity of neighboring solutions.
The accuracy and improved compatibility of local deformations are demonstrated in academic examples. We compare the proposed method to a coarse global and to a fine, purely local variant. In these experiments, the new CLG method is always more accurate than a coarse global registration and at least as accurate as a purely local registration in terms of image distance and deformation error. Registering the images sequentially, the CLG method does not compute one but a combination of multiple globally regularized deformations. The new method outperforms a purely local registration in terms of deformation error and irregularity measure in cases where the subdomain boundary region consists of low-contrast image information.
In addition to academic examples, we apply the new method to clinical whole-slide image data---each of multiple gigapixels in size---that comprises four different staining combinations and originates from two independent laboratories. When evaluating a total of 82 manually placed landmarks, the CLG and fine global registration are of similar accuracy, while CLG uses only a fraction of the RAM.
An exemplary image pair of 24 000 × 54 000 pixels is registered on a single workstation using 6 GB of RAM where---otherwise---more than 32 GB would be needed. Independently of the image size, only a fraction of the RAM is required to solve the same registration problem. While a standard registration method would be faster for small problems, the CLG method enables the registration of large-scale images that could otherwise not be computed.
... which penalizes local spatial variations in φ d to promote smooth local deformations [4]. Initial experiments revealed that, when objects in the background of the target image are present, the affine transformation degenerate towards extreme compression or expansion. ...
We propose an unsupervised deep learning method for atlas based registration to achieve segmentation and spatial alignment of the embryonic brain in a single framework. Our approach consists of two sequential networks with a specifically designed loss function to address the challenges in 3D first trimester ultrasound. The first part learns the affine transformation and the second part learns the voxelwise nonrigid deformation between the target image and the atlas. We trained this network end-to-end and validated it against a ground truth on synthetic datasets designed to resemble the challenges present in 3D first trimester ultrasound. The method was tested on a dataset of human embryonic ultrasound volumes acquired at 9 weeks gestational age, which showed alignment of the brain in some cases and gave insight in open challenges for the proposed method. We conclude that our method is a promising approach towards fully automated spatial alignment and segmentation of embryonic brains in 3D ultrasound.
... The second part of criterion describes the regularization of the deformation field. The idea is related to the diffusive regularization ( ) = 1 2 ∑ =1 ∫ ∈ ‖∇ ‖ 2 d (Fischer and Modersitzki, 2002). Due to our sparse representation ∇ is substituted by the partial derivatives in the direction of neighboring sample points and approximated as follows: ...
The registration of two medical images is usually based on the assumption that corresponding regions exist in both images. If this assumption is violated by e. g. pathologies most approaches encounter problems. The registration approach proposed is based on probabilistic correspondences of sparse image representations and enables a robust handling of potentially missing correspondences. A maximum a-posteriori framework is used to derive an optimization criterion with respect to deformation parameters that aim to reduce the shape and appearance differences between the registered images. A multi-resolution approach speeds up the optimization and increases the robustness of the registration. The computed probabilistic correspondences enable the approach to deal with missing correspondences in the images. Furthermore, they provide additional information about the quality of fit and potentially non-corresponding/pathological image regions. The approach is compared to two state-of-the-art registration methods using MR brain and cardiac images: a variational intensity-based registration algorithm and a feature-based registration approach using a discrete optimization scheme. The comprehensive quantitative evaluation using additional simulated stroke lesions shows a significantly higher accuracy and robustness of the proposed approach. Furthermore, the correspondence probability maps were used to characterize pathological regions in the MRI brain data.
... [9][10][11][12][13][14][15] The nonparametric image registration was treated by numerous approaches, including the ones based on regularization criterion. [16][17][18] One of the most effective models used in the nonparametric registration was the elastic registration. [19][20][21][22] Although the elastic problem looks easy to solve, it is quite complicated. ...
In this paper, we present a fast algorithm of the nonparametric elastic image registration using a simple implementation of the Range Restricted GMRES (RRGMRES) method. This approach differs from the others in the fact that it is specified to the tridiagonal block matrix type to resolve a nonsymmetric linear system. In what follows, we prove existence and uniqueness of minimizer of the elastic registration problem and present the corresponding discrete problem by employing a finite difference scheme. The accuracy of the proposed method is demonstrated on different image registration examples; we also show the speedup of the proposed approach by calculating the corresponding CPU time and compared it with the classical elastic registration method.
... More recently, Hadj et al [47], motivated by [73], incorporated the similarity metric with a confidence mask to improve the robustness of the algorithm. Fischer et al [36] proposed a fast diffusion based registration that exploits the underlying PDE for the regularization. Wang et al [125] recommended adaptive force strength adjustment iteratively in the iteration process . ...
The context of this thesis is the image registration for endomicroscopic images. Multiphoton microendoscope provides different scanning trajectories which are considered in this work. First we propose a nonrigid registration method whose motion estimation is cast into a feature matching problem under the Log-Demons framework using Graph Wavelets. We investigate the Spectral Graph Wavelets (SGWs) to capture the shape feature of the images. The data representation on graphs is more adapted to data with complex structures. Our experiments on endomicroscopic images show that this method outperforms the existing nonrigid image registration techniques. We then propose a novel image registration strategy for endomicroscopic images acquired on irregular grids. The Graph Wavelet transform is flexible to apply on different types of data regardless of the data point densities and how complex the data structure is. We also show how the Log-Demons framework can be adapted to the optimization of the objective function defined for images with an irregular sampling.
It is very common to see many terms in a variational model from Imaging and Vision, each aiming to optimize some desirable measure. This is naturally so because we desire several objectives in an objective functional. Among these is data fidelity which in itself is not unique and often one hopes to have both L1 and L2 norms to be small for instance, or even two differing fidelities: one for geometric fitting and the other for statistical closeness. Regularity is another demanding quantity to be settled on. Apart from combination models where one wants both minimizations to be achieved (e.g., total generalized variation or infimal convolution) in some balanced way through an internal parameter, quite often, we demand both gradient and curvature based terms to be minimized; such demand can be conflicted. A conflict is resolved by a suitable choice of parameters which can be a daunting task. Overall, it is fair to state that many variational models for Imaging and Vision try to make multiple decisions through one complicated functional. Game theory deals with situations involving multiple decision makers, each making its optimal strategies. When assigning a decision (objective) by a variational model to a player by associating it with a game framework, many complicated functionals from Imaging and Vision modeling may be simplified and studied by game theory. The decoupling effect resulting from game theory reformulation is often evident when dealing with the choice of competing parameters. However, the existence of solutions and equivalence to the original formulations are emerging issues to be tackled. This chapter first presents a brief review of how game theory works and then focuses on a few typical Imaging and Vision problems, where game theory has been found useful for solving joint problems effectively.
Incorporating prior knowledge into a segmentation process— whether it be geometrical constraints such as landmarks to overcome the issue of weak boundary definition, shape prior knowledge or volume/area penalization, or topological prescriptions in order for the segmented shape to be homeomorphic to the initial one or to preserve the contextual relations between objects— proves to achieve more accurate results, while limiting human intervention. In this contribution, we intend to give an exhaustive overview of these so-called weakly/semi-supervised segmentation methods, following three main angles of inquiry: inclusion of geometrical constraints (landmarks, shape prior knowledge, volume/area penalization, etc.), incorporation of topological constraints (topology preservation enforcement, prescription of the number of connected components/holes, regularity enforcement on the evolving front, etc.), and, lastly, joint treatment of segmentation and registration that can be viewed as a special case of cosegmentation.
In this work, we investigate image registration in a variational framework and focus on regularization generality and solver efficiency. We first propose a variational model combining the state-of-the-art sum of absolute differences (SAD) and a new arbitrary order total variation regularization term. The main advantage is that this variational model preserves discontinuities in the resultant deformation while being robust to outlier noise. It is however non-trivial to optimize the model due to its non-convexity, non-differentiabilities, and generality in the derivative order. To tackle these, we propose to first apply linearization to the problem to formulate a convex objective function and then break down the resultant convex optimization into several point-wise, closed-form subproblems using a fast, over-relaxed alternating direction method of multipliers (ADMM). With our proposed algorithm, we show that solving higher-order variational formulations is similar to solving their lower-order counterparts. Extensive experiments show that our ADMM is significantly more efficient than both the subgradient and primal-dual algorithms particularly when higher-order derivatives are used, and that our new models outperform state-of-the-art methods based on deep learning and free-form deformation. Our code implemented in both Matlab and Pytorch is publicly available at https://github.com/j-duan/AOTV.
In this paper, we propose new variational model for image registration from tomographic data. First, we employ the topological gradient approach for a tomographic reconstruction that uses the first‐ and the second‐order discontinuities in order to detect important objects of a given observed X‐ray tomographic data (sinograms). Second, we use this geometric information furnished by a high‐order operator in order to define an appropriate fidelity measure for the image registration process. A theoretical study of the proposed model is provided; Gauss–Newton method and multilevel technique are used for its numerical implementation. The performed numerical experiments show the efficiency and effectiveness of our model.
In this paper, we propose a vectorial minimized surface regularizer based image registration model which is suitable for smooth and non-smooth registration. In order to avoid the mesh folding phenomenon, inequality constraint on transformed Jacobian matrix determinant is imposed. In addition, we use Lagrange multipliers combining Gauss-Newton method with Armijo line search with the multilevel method to solve the corresponding model. And guided filter is utilized on the displacement field before and after the registration of each level to avoid noise and preserve the edge information of the image. Furthermore, the convergence analysis of the algorithm is given. Finally, numerical experiments using both synthetic and realistic images are carried out to show the robustness of the proposed model and the effectiveness of the algorithm.
Image segmentation aims to extract the target objects or to identify the corresponding boundaries. For corrupted images due to occlusions, obscurities or noises, to get an accurate segmentation result is very difficult. To overcome this issue, the prior information is often introduced and particularly, the convex prior attracts more and more attentions recently. In this paper, we propose a topology- and convexity-preserving registration-based segmentation model, which can be suitable for both 2D and 3D cases. By incorporating the level set representation and imposing the constraints on the suitable regions, we can explicitly force the fully convex segmentation results or the partially convex segmentation results. To solve the proposed model, we employ the alternating direction method of multipliers and numerical experiments on 2/3D synthetic and real images demonstrate that the proposed model can indeed lead to the accurately topology- and convexity-preserving segmentation.
This book constitutes the refereed proceedings of the 9th International Workshop on Biomedical Image Registration, WBIR 2020, which was supposed to be held in Portorož, Slovenia, in June 2020. The conference was postponed until December 2020 due to the COVID-19 pandemic.
The 16 full and poster papers included in this volume were carefully reviewed and selected from 22 submitted papers. The papers are organized in the following topical sections: Registration initialization and acceleration, interventional registration, landmark based registration, multi-channel registration, and sliding motion.
To overcome the weakness of TV based image registration models, we propose a fractional-order decomposition model for image registration. In this model, deformation is decomposed into two components: discontinuous component and smooth component. The fractional-order total variation regularization and higher order regularization are added on these two components, respectively. Furthermore, a numerical algorithm is proposed to solve this model. Numerical tests are also performed to show the efficiency of the proposed model.
Due to the complicated thoracic movements which contain both sliding motion occurring at lung surfaces and smooth motion within individual organs, respiratory estimation is still an intrinsically challenging task. In this paper, we propose a novel regularization term called locally adaptive total p-variation (LaTpV) and embed it into a parametric registration framework to accurately recover lung motion. LaTpV originates from a modified
-norm constraint (
), where a prior distribution of modeled by the Dirac-shaped function is constructed to specifically assign different values to voxels. LaTpV adaptively balances the smoothness and discontinuity of the displacement field to encourage an expected sliding interface. Additionally, we also analytically deduce the gradient of the cost function with respect to transformation parameters. To validate the performance of LaTpV, we not only test it on two mono-modal databases including synthetic images and pulmonary computed tomography (CT) images, but also on a more difficult thoracic CT and positron emission tomography (PET) dataset for the first time. For all experiments, both the quantitative and qualitative results indicate that LaTpV significantly surpasses some existing regularizers such as bending energy and parametric total variation. The proposed LaTpV based registration scheme might be more superior for sliding motion correction and more potential for clinical applications such as the diagnosis of pleural mesothelioma and the adjustment of radiotherapy plans.
In this thesis, we propose a novel computational approach to fast and memory-efficient deformable image registration. We demonstrate the relevance of the proposed method in three real-world medical applications with very different requirements, ranging from processing of large datasets to registration with real-time constraints.
The approach builds on a variational image registration model. In this model, finding a transformation which provides a reasonable image alignment is performed by minimizing an objective function. In the utilized discretize-then-optimize approach, the minimization is realized by using derivative-based optimization methods. Here, the objective function derivatives are typically the computationally most expensive operations, both in terms of runtime and memory requirements. Therefore, we analyze the matrix structure for all derivative components. Based on the analysis, we derive equivalent, fully matrix-free closed-form expressions for gradient computations as well as the Hessian-vector multiplication, enabling the use of matrix-free computations for both L-BFGS and Gauss-Newton optimization schemes.
The matrix-free computations completely eliminate the need for storing intermediate results and the cost of sparse matrix arithmetic. The expressions are fully parallelizable and the memory complexity for the derivative computations is reduced from linear to constant. We show that all important matrix-free derivative computations scale virtually linear, allowing to fully benefit from parallel execution. In comparison with matrix-based algorithms, the proposed approach is several orders of magnitude faster. The generic formulation of the matrix-free approach enables the implementation on different platforms. Besides multi-core CPUs, we present a GPU implementation which achieves a substantial, additional speedup. In order to justify the effort for deriving the matrix-free computations, we additionally implement the registration algorithm using an automatic differentiation framework. This method automatically computes optimized, analytically exact derivatives and allows for seamless execution on GPUs. In comparison with matrix-based methods, the automatic differentiation-based approach achieves comparable runtimes, making it a well-suited alternative for rapid prototyping and algorithm development.
In the second part of the thesis, we utilize the matrix-free registration in three clinical applications. First, in an application from oncology, we present an automatic pipeline for registration of follow-up thorax-abdomen CT scans. We evaluate the algorithm on a large number of datasets, achieving clinically feasible runtimes. Second, we consider registration of CT and cone-beam CT images in radiotherapy. To achieve physically plausible deformations, we introduce an additional local rigidity constraint. In comparison to a commercial registration method, we achieve comparable accuracy, while obtaining physically more plausible deformations within a clinically suitable runtime. Third, we use the registration in real-time liver ultrasound tracking in order to determine respiratory motion. For this, we integrate the matrix-free registration in a tracking scheme with a moving-window strategy. In a public benchmark, we achieve real-time performance with the lowest mean tracking error of all participants. In each of these applications, the matrix-free methods allow the use of registration in scenarios where it would not be possible otherwise, due to run-time or memory constraints. Thus, the matrix-free registration can contribute to further increasing the use of image registration in clinical practice.
Registration aligns features of two related images so that information can be compared and/or fused in order to highlight differences and complement information. In real life images where bias field is present, this undesirable artefact causes inhomogeneity of image intensities and hence leads to failure or loss of accuracy of registration models based on minimization of the differences of the two image intensities. Here, we propose a non-linear variational model for joint image intensity correction (illumination and translation) and registration and reformulate it in a game framework. While a non-potential game offers flexible reformulation and can lead to better fitting errors, proving the solution existence for a non-convex model is non-trivial. Here we establish an existence result using the Schauder's fixed point theorem. To solve the model numerically, we use an alternating minimization algorithm in the discrete setting. Finally numerical results can show that the new model outperforms existing models.
In this paper, we propose an alternating direction implicit (ADI) scheme of a fractional-order diffusion tensor image (DTI) registration model. Furthermore, compatibility, stability and convergence of ADI scheme are proved. Moreover, five numerical tests are also performed to show the efficiency of this scheme.
ResearchGate has not been able to resolve any references for this publication.