A fast inverse consistent deformable image registration method based on symmetric optical
This content has been downloaded from IOPscience. Please scroll down to see the full text.
IP Address: 22.214.171.124
This content was downloaded on 04/10/2013 at 19:47
Please note that terms and conditions apply.
2008 Phys. Med. Biol. 53 6143
View the table of contents for this issue, or go to the journal homepage for more
Home Search CollectionsJournalsAbout Contact us My IOPscience
PHYSICS IN MEDICINE AND BIOLOGY
Phys. Med. Biol. 53 (2008) 6143–6165
A fast inverse consistent deformable image registration
method based on symmetric optical flow computation
Deshan Yang1, Hua Li, Daniel A Low, Joseph O Deasy
and Issam El Naqa
Department of Radiation Oncology, School of Medicine, Washington University in St. Louis,
4921 Parkview Place, LL, St. Louis, MO 63110, USA
Received 10 June 2008, in final form 16 September 2008
Published 14 October 2008
Online at stacks.iop.org/PMB/53/6143
Deformable image registration is widely used in various radiation therapy
applications including daily treatment planning adaptation to map planned
tissue or dose to changing anatomy. In this work, a simple and efficient inverse
consistency deformable registration method is proposed with aims of higher
I to a second image J, the two images are symmetrically deformed toward one
by minimizing a symmetric optical flow system cost function using modified
optical flow algorithms. The images are then further deformed with the delta
motion field in the positive and negative directions respectively, and then used
for the next pass. The magnitude of the delta motion field is forced to be less
than 0.4 voxel for every pass in order to guarantee smoothness and invertibility
for the two overall motion fields that are accumulating the delta motion fields
in both positive and negative directions, respectively. The final motion fields
to register the original images I and J, in either direction, are calculated by
inverting one overall motion field and combining the inversion result with the
other overall motion field. The final motion fields are inversely consistent
and this is ensured by the symmetric way that registration is carried out. The
proposed method is demonstrated with phantom images, artificially deformed
patient images and 4D-CT images. Our results suggest that the proposed
method is able to improve the overall accuracy (reducing registration error by
30% or more, compared to the original and inversely inconsistent optical flow
the convergence rate (by 100% or more). The overall computation speed may
slightly decrease, or increase in most cases because the new method converges
1Author to whom any correspondence should be addressed.
0031-9155/08/216143+23$30.00© 2008 Institute of Physics and Engineering in MedicinePrinted in the UK6143
6144 D Yang et al
faster. Compared to previously reported inverse consistency algorithms, the
proposed method is simpler, easier to implement and more efficient.
(Some figures in this article are in colour only in the electronic version)
It has been witnessed in the recent years that anatomical image information (kVCT, daily
fMRI, etc) were increasingly adopted into patient radiation treatment management. Image
registration is a procedure to transform different image datasets into a common coordinate
system so that corresponding points of the images are matched and the complementary
information from the different images can be analyzed for different diagnostic and therapeutic
purposes. Kessler provided a comprehensive review of image registration for radiation
therapy (Kessler 2006). Image registration algorithms can be generally grouped into rigid
registration and deformable (non-rigid) registration according to the type of transformation
with limited number of free parameters (up to 12), deformable registration uses much larger
number of free parameters (up to three times the total number of voxels in an image) in order
to describe non-rigid tissue deformation in 3D space.
Deformable image registration can be computed based on features extracted from the
images, e.g. points (Kessler et al 1991), lines (Balter et al 1992) and surfaces (van Herk
and Kooy 1994) or based on metrics directly derived from the image intensity values,
e.g. mean square error (MSE) (Thirion 1998) for images from the same modality, mutual
information (MI) (Viola and Wells 1995) and cross-correlation (Kim and Fessler 2004) for
images from different modalities. Mean-squared-error (MSE) based CT to CT deformable
image registration is especially important for radiation therapy applications, including patient
response monitoring, treatment adaptation, dose tracking and patient motion modeling, etc
(Lu et al 2004, Wang et al 2005, Sarrut et al 2007). This paper focuses on such algorithms.
Regardless of the image registration algorithm, registration accuracy is always one of the
images, registration results often cannot be validated on a voxel-by-voxel basis because there
is no such ground truth available. While landmark matching and structure volume matching
are often used for results validation, they are not voxel by voxel based and the overall accuracy
of such a validation is quite limited because landmarks or structures can only cover limited
regions of the entire image.
Inverse consistency, which means that the registration results are consistent from
registering the images in the forward direction (from image 1 to image 2) or in the reverse
direction (from image 2 to image 1), is often considered as one of the more feasible ways
for measuring image registration accuracy (Christensen et al 2006) for any registration
algorithms. This is based on the fact that results by an accurate registration algorithm must
be inversely consistent. Therefore, inverse consistency is always desirable for any deformable
registration algorithm in addition to its accuracy. For image-guided and adaptive radiation
therapy (ART) applications, such inverse consistency is not only desirable but also practically
useful. Information such as treatment planning contours, etc is defined on the treatment
planning CT, while daily doses, contours, etc are referenced to the daily images. Inverse
consistency registrations can provide voxel mapping in both directions so that information can
be consistently mapped from one image to the other image.
A fast inverse consistent deformable image registration method 6145
Table 1. Notations used in this paper.
I ◦ V
The moving image, or the first image
The fixed image, or the second image
The difference image, Id= J − I
The image domain
Coordinates of image positions in ?
The ‘pull-back’ displacement motion vector fields
The delta motion field
=I(x−V(x)), the image I deformed by V
=V1(x−V2(x)) + V2(x), the composition of two motion fields
The inverted vector field of V
(ART). For example, registration needs to be computed quickly and accurately between the
treatment fractionations. In the future, such a computation may need to be completed online
while a patient is on the treatment table. Computation speed is also demanded by 4D-CT
based respiratory motion estimation because of the large amount of 4D image data.
Because of these reasons, we propose a new and efficient inversely consistent deformable
registration method in this paper. The new method uses a simplified system cost function
and solves registration in a symmetric way.
intensity gradients) from both images are symmetrically used in the computation, both
asymmetrical inverse-inconsistency algorithms. Because the system cost function is simpler,
the overall computation speed is improved compared to other inverse consistency algorithms.
Because image information (intensity and
1.1. Optical flow deformable image registration and inverse consistency
1.1.1. Optical flow.
modality deformable image registration. They are based on image intensity and gradient
information. For two images I and J to be registered, let I be the moving image and J be the
fixed image, a displacement motion vector field V registers I to J so that
Optical flow algorithms are among the most used algorithms for single
J = I ◦ V ≡ I(x − V(x)),
where ◦ is the composition operator, V is the motion field and x is the spatial coordinate. The
motion field V is the displacement vector field instead of the transformation vector field. V
is often referred to as the deformation field or the optical flow field. Description of notations
used in this paper is in table 1.
V normally cannot be resolved by only using equation (1) because the system is
underdetermined. Other constraints, such as global smoothness, are often enforced in order
to successfully compute V. With the additional global smoothness constraints, the system cost
function could be written as
where R is the smoothness constraint (also known as regularization constraint) function,
? is the image domain and α is a constant. Many optical flow algorithms use R(V) =
tr((∇V)T(∇V)), where tr( ) is the matrix trace operator. If |V| is small, equation (2) could
((J − I ◦ V)2+ α2R(V))d?,
6146 D Yang et al
Figure 1. Illustration of asymmetric registration and inverse consistency error. Point A (in
image I) and B (in image J) are matching points. V is computed by registering I to J. U is computed
by registering image J to image I. (a) After imperfect asymmetric registrations, point A moves to
point A?and point B moves to point B?. (b) Using U, A?will be moved to A??. Similarly, B??is
B?moved by using V. The distance from A to A??, and from B to B??, are the inverse consistency
be expressed using a Taylor expansion of the first-order terms in the following differential
where Id= J − I, ∇ is the gradient operator, · is the vector inner product operator.
V could be solved by minimizing E1with many numerical methods, either iteratively or
techniques, region-based matching, energy-based methods and phase-based techniques. This
paper uses the Horn–Schunck (HS) algorithm (Horn and Schunck 1981) and the demons
where the differential form of the system cost equation is solved using the image intensity and
gradient. Such differential optical flow algorithms are often referred to as small-motion-model
algorithms because they only work if |V| is sufficiently small so that the Taylor expansion
series can be applied.
((Id+ ∇I · V)2+ α2R(V))d?,
1.1.2. Registration in the inverse direction.
to image I in the backward direction, the second motion field U needs to be computed so
Traditionally, if image J needs to be registered
I = J ◦ U.
A similar system equation could be written as
Even if V has already been computed, U has to be computed independently because there
is unfortunately no direct dependence among the solutions of V and U. This is illustrated in
((I − J + ∇J · U)2+ α2R(U))d?.
1.1.3. Inverse consistency.
consistent so that registration could start with either image and the results are consistent.
Inverse consistency could be written as
It is desirable for many applications that V and U are inversely
V ◦ U = 0 = U ◦ V,
where the composition operator ◦ between two motion fields is defined in table 1.
A fast inverse consistent deformable image registration method 6147
The inverse consistency error (ICE) could be then defined as
ICE1= |V ◦ U|
ICE2= |U ◦ V|.
If V and U are inversely consistent, ICE1and ICE2will be both 0. Otherwise, ICE1and
ICE2will not be 0 and may not be the same, as illustrated in figure 1(b). A combined inverse
consistency error term ICE can be defined as
ICE = (ICE1+ ICE2)/2.
1.2. Previous inverse consistency methods
It is generally difficult to have V and U consistent if the image registration computations for
Therefore, most inverse consistency registration algorithms perform computations for both
directions simultaneously and explicitly constrain V and U to be, or closely to be, inversely
Christensen and Johnson (2001) seem to be among the earliest groups to consider inverse
computed by minimizing the symmetric system cost equation (10), which contained the
Diffeomorphism refers to the continuous differentiability of the motion field as discussed
+(V − U−1)2+ (V−1− U)2+ |L(V)|2+ |L(U)|2)d?,
where the linear elastic operator L = −a2∇2− b∇ + c, a, b and c are constants. U and V
were parameterized with Fourier sequences and solved iteratively. Both V and U needed to
be inverted to obtain V−1and U−1in every iteration. An inversion procedure, to be further
discussed later, was performed iteratively or analytically, for a displacement motion vector
field V, by minimizing |V−1◦ V| or |V−1◦ V|.
Alvarez et al (2007) proposed an algorithm based on the system cost equation (11). The
algorithm does not explicitly invert the forward and reverse motion fields during the iterations.
Instead, the inverse consistency error is computed and minimized per iteration.
+αER(J,U) + βES(V,U) + βES(U,V))d?,
?((I ◦ V − J)2+ (J ◦ U − V)2
?((I ◦ V − J)2+ (J ◦ U − V)2+ αER(I,V)
where the regularization constraint ER(I,V) = tr((∇V)TD(∇I)∇V), α and β are constants,
theinverseconsistencyconstraintES(V,U) = |U◦V|2andD (∇I)isaregularizedprojection
matrix in the direction perpendicular to ∇I.
Cachier and Rey (2000) analyzed the reasons why results of unidirectional registrations
are asymmetric and pointed out that inversely inconsistant approaches penalized the image
expansion more than the shrinkage. They proposed an inverse-invariant type system cost
equation given in equation (12) and two finite element implementations to solve the new cost
function, depending on where motion field inversion is being computed or not. Registration
6148 D Yang et al
does not need to be performed simultaneously for both forward and reverse directions in this
Leow et al (2005) reported an approach to model the backward motion field by a function
computing the inversion of the motion fields. They used a symmetric system cost function,
similar to equation (10), with the V−1and U−1replaced by the functions of V and U.
Diffeomorphism algorithms (Dupuis et al 1998, Christensen et al 1996, Trouve 1998) are
closely related to inverse consistency. Diffeomorphism means continuous, differentiable and
invertible. These algorithms are often referred to as large-motion-model algorithms because
the regularization term in the system cost function is different and the algorithms can compute
smooth and continuous large motions. Dupuis et al (1998) showed theoretically that the
solution for the diffeomorphism system cost equation is unique, smooth, differentiable and
invertible. It should be understood that the invertibility is not equal to inverse consistency and
diffeomorphism algorithms are not inversely consistent by default.
There are a few inverse consistency algorithms proposed under the diffeomorphism
framework. Joshi et al (2004) proposed a method to construct a template image from multiple
images for brain mapping. The major computation of this algorithm is done in the Fourier
frequency domain. The system cost equation is given by
where N is the total number of images,ˆI is the shape average image, which is updated during
the iterations, Viis the motion field to deform image i toˆI,viis the velocity vector field for
image i and Vi(x) =?1
and the system cost equation reduces to
Similar algorithms have also been proposed by Avants and Gee (2004) and by Beg and
Khan (2007). These algorithms are all based on the idea that both images are deformed toward
the ‘mean shape’ image in order to achieve better registration. Such an idea is quite similar
to the basic concept of the method investigated in this paper. We will further compare our
method to these algorithms in the later sections.
The goal of computing image registration with inverse consistency is to improve the
registration accuracy and to provide consistent motion fields for both registration directions.
Better accuracy has been achieved by adding additional inverse consistency constraints and
using symmetric system cost functions. However, solving the more complicated registration
problem is usually much slower.
(1 + |dV|)(I ◦ V − J)2+ α
(Ii◦ Vi−ˆI)2d? +
If the number of images is 2, then this method becomes an inverse consistency method
(I ◦ V − J ◦ U)2d? +
2. Methods and materials
Our method is similar to the inverse consistency diffeomorphism algorithms (Joshi et al 2004,
Avants and Gee 2004, Beg and Khan2007), however, focuses onsimplicityand computational
A fast inverse consistent deformable image registration method 6149
Figure 2. Demonstration of the proposed inversely consistent registration method. Matching
points A and B are in image I and image J, respectively. After n passes, A is moved to point A?
and B is moved to point B?. A?and B?are in close proximity, but are not perfectly registered. Vn
and Unare the overall motion fields. The delta motion field ?Vnand ?Unare computed for each
efficiency. As illustrated in figure 2, I and J are symmetrically deformed pass-by-pass toward
each other. Inand Jndenote I and J deformed after pass n. Registration is achieved on Inand
Jnwhen Inand Jnmatches.
At pass n, a delta motion field ?Vn, is computed by minimizing a symmetric optical flow
system cost equation (to be discussed in the following section) using modified optical flow
algorithms. The two overall motion fields, Vnfor image I and Unfor image J, are updated by
accumulating ?Vnand −?Vnas
Vn= ?Vn◦ Vn−1
Un= (−?Vn) ◦ Un−1
and Inand Jnare then updated as
In= I ◦ Vn
Jn= J ◦ Un.
The two new deformed images Inand Jnwill be used for the next pass.
Initially, V0= U0= 0. Because ?Vnis a ‘pull-back’ motion field (defined on the voxel
grid of Inand Jn, instead of the voxel grid of I and J), the Vn?= −Unfor pass numbers n > 1,
therefore Vnand Unare updated individually. The magnitude of ?Vnis forced to be less than
0.4 voxel size in order to ensure the smoothness and invertibility of Vnand Unas discussed
below. If the registration direction is reversed, it can be shown that Inand Jnwill be swapped,
and consequently Vnand Unwill be swapped.
The final motion fields, VIJwhich registers I to J, and UJIwhich registers J to I, are
from the last Vnand Un. It can be shown that VIJand UJIare inversely consistent to each other.
If the registration direction is reversed, Vnand Unsimply swap. VIJand UJIwill also be simply
swapped. Because VIJand UJIare inversely consistent, the final motion fields computed in the
forward and the backward registration directions are inversely consistent. Both VIJand UJI
can be computed in one step regardless of the registration direction.
2.2. Symmetric optical flow system cost equation
At pass n, we compute the delta motion fields ?Vnand ?Unto achieve further image
registration between the current deformed In−1and Jn−1. In−1is deformed using ?Vnand
6150 D Yang et al
generates Inaccording to
In= In−1◦ ?Vn= In−1− ∇In−1· ?Vn
Jn−1is deformed using ?Unand generates Jnaccording to
Jn= Jn−1◦ ?Un= Jn−1− ∇Jn−1· ?Un
?Vnand ?Unare solved by minimizing the following new system cost equation:
To simplify the new equation, we add another hard constraint on ?Vnand ?Un
((Jn− In)2+ β2R(?Vn) + β2R(?Un))d?.
?Vn+ ?Un= 0(24)
and we select the smoothness regularity function R( ) so that
R(?Vn) = R(−?Vn)
and let α2= 2β2, then the system cost equation could be rewritten in the following differential
form using Taylor expansion:
which is simplified into
where IS= In−1+ Jn−1and Id= Jn−1− In−1.
One can see that equation (27) has exactly the same form as equation (3). This means
that the intermediate deformation fields ?Vncould be solved with the same algorithms that
solve equation (3) while having ?Un= −?Vn. Most regularization functions, including the
ordinary optical flow global smoothness function R(V) = tr((∇V)T(∇V)), are good choices
for equation (25).
((Jn−1− In−1+ (∇Jn−1+ ∇In−1) · ?Vn)2+ α2R(Vn))d?
((Id+ ∇Is· ?Vn)2+ α2R(?Vn))d?,
2.3. Solving the system cost equation
2.3.1. Case 1: Horn–Schunck (HS) optical flow algorithm.
(HS) (Horn and Schunck 1981) algorithm solves equation (3) using the following iterative
where Vkis the motion field at iteration k,¯Vkis Vkaveraged for each pixel in the neighborhood
of that pixel.
To solve the new system cost equation (27), the iterative equation is modified slightly to
The original Horn–Schunck
where IS= In−1+ Jn−1and Id= Jn−1− In−1.
After all iterations are finished, the last Vk+1is ?Vn, the desired solution for equation (27).
A fast inverse consistent deformable image registration method 6151
2.3.2. Case II: using demons algorithm.
solves equation (3) using this iterative solution
where Gσis Gaussian lowpass filter with a window width σ, k is the iteration number.
To solve equation (27), we replace the gradient terms with ∇In−1+ ∇Jn−1. We also do
not have to use multiple iterations because we are already applying multiple passes at the In
and Jnlevels. In this way, the equation can be reduced to
The original demons algorithm (Thirion 1998)
(I ◦ Vk− J)∇J
|∇J|2+ (I ◦ Vk− J)2
2.4. Inversion of Vnand Un
2.4.1. Guarantee of invertibility.
fields can be computed. To be invertible, Vnand Unshould be smooth, without folding and
the determinant of the Jacobian matrix should be strictly positive defined (Leow et al 2005).
However, neither the HS algorithm nor the demons algorithm guarantees the invertibility for
Vnand Un. Therefore, the following small-step multiple pass approach is used to ensure it.
then Vnand Un, which are accumulating ?Vnand −?Vn, will also be diffeomorphic. Such
an approach has been reported previously (Cootes et al 2004, Rueckert et al 2006). Rueckert
et al reported that the maximal displacement of the control points for their cubic B-Spline
algorithm needs to be less than 0.40 of the spacing of the control points for the motion field to
be diffeomorphic. Such a conclusion could be indirectly applied to the optical flow algorithms
by treating voxels in each image as B-Spline control points. If ?Vnis less than 0.4 voxel size,
?Vnwill be diffeomorphic and Vnand Unwill also be diffeomorphic.
We used the following ad hoc step after every pass to explicitly reduce the magnitude of
?Vnto 0.4 voxel if it is greater than 0.4 voxel
0.4 × ?Vn/|?Vn|
There are alternative approaches to guarantee diffeomorphism for ?Vn. Vercauteren
et al (2007) reported using exp(?Vn) to replace ?Vn. The term exp(?Vn) is approximated
by composing ?Vn/n for n times. For example, exp(?Vn) = (?Vn/16) ◦ (?Vn/16) ◦ ··· ◦
(?Vn/16). Because ?Vn/n is diffeomorphic if n is large, exp(?Vn) will be diffeomorphic.
One problem of this approach is that exp(?Vn) ?= ?Vn, neither in direction nor in magnitude,
therefore exp(?Vn) is a rough but diffeomorphic approximation of ?Vn.
Neither our magnitude limiting procedure nor the method of using exp(?Vn) is perfect.
If ?Vnis accurate, then errors will be introduced by either method. Such errors have to be
recovered in the next pass, and could slow down the overall convergence. Our method is,
however, simpler to implement and more computationally efficient.
Smoothness of ?Vnafter the magnitude limiting procedure should not be a concern
because ?Vnis discrete and the largest possible magnitude difference of ?Vnbetween two
adjacent voxels is 0.4 × 2 = 0.8. However, if more smoothness is desired, ?Vncan be
smoothed by a Gaussian lowpass filter as Gσ(?Vn) → ?Vn, where σ is the window size. The
maximal magnitude of ?Vnwill still be less than 0.4 after such a lowpass filtering step and
the smoothed ?Vnwill still be diffeomorphic.
Vnand Unmust be invertible so that the final motion
|?Vn| ? 0.4
|?Vn| > 0.4.
6152 D Yang et al
2.4.2. Motion field inversion.
way, as used in many diffeomorphism algorithms, is to integrate (accumulate) the inverse of
the delta motion fields during the passes. This means to integrate the inverse of ?Vnand
?Un(?Un= −?Vn). Because magnitude of ?Vnis small, (?Vn)−1can be approximated as
−?Vn. Integration of (?Vn)−1and (?Un)−1is slightly different from computation of Vnand
Unby accumulating ?Vnand ?Unbecause (?Vn)−1and (?Un)−1are the push-forward motion
fields while ?Vnand ?Unare the pull-back motion fields. However, the way to approximate
(?Vn)−1by −?Vndoes not work very well with multigrid approaches because a small ?Vnin
the coarse image resolution stage equals to larger motion in the finer resolution stage and will
have larger approximation errors. To reduce such error in the finer resolution stage, the spatial
step (maximal |?Vn|) in the coarse stage must be very small, much smaller than 0.4 voxel.
Using very small spatial steps contradicts the idea of using the multigrid approach since the
multigrid approach is applied to improve computation speed.
Methods todirectlycompute theinversemotionfieldhave been reported(Christensenand
Johnson 2001, Cachier and Rey 2000). Ashburner reported a fast method based on the idea of
tetrahedral and affine transformation inversion (Ashburner et al 2000). We used this method
in this work because it is computationally efficient and accurate. The method is already
implemented in the statistical parametric mapping (SPM) (Friston 2006) version 5 package.
We evaluated the code from SPM and demonstrated that it worked well for all our tested cases
with averaged error <0.05 pixel and maximal error <0.1 pixel.
Vnand Uncan be inverted by a few different ways. The easiest
2.5. The entire procedure
The entire inverse consistency method can be described in following pseudo code:
(1) Let the pass number n = 0 and V0= U0= 0.
(2) Compute the deformed image Inand Jnaccording to equations (17) and (18).
(3) Use one of the modified optical flow algorithms to perform registration between Inand Jn
and compute ?Vn+1.
(4) Limit the magnitude of ?Vn+1according to equation (32), and optionally smooth ?Vn+1
with a Gaussian lowpass filter.
(5) Let n = n + 1, update the overall motion fields Vnand Unaccording to equation (15) and
(16) and optionally smooth Vnand Unwith another Gaussian lowpass filter.
(6) If the results have not converged and n is less than the maximal step number allowed, and
then go back to step 2 for the next pass. Convergence is determined by checking whether
the maximal magnitude of ?Vnis less than a user set value, for example, 0.01 voxel.
(7) Otherwise, compute the final deformation fields according to equations (19) and (20).
The entire procedure is similar to a regular asymmetric registration procedure, with
additional steps 4 and 7. An important difference is that the computation needs to be carried
out for both images to update Inand Jn, Vnand Un, while a regular asymmetric procedure often
only needs to compute similar variables for one image. Optional smoothing in step 4 helps
to smooth ?Vnafter the magnitude of ?Vnis limited. Optional smoothing in step 5 helps to
diffuse the motion from high contrast regions into neighborhoods with low contrast regions
The proposed method is implemented primarily in MATLAB with image processing toolbox.
The motion field inversion procedure is implemented in C/C++. Besides the multiple passes
A fast inverse consistent deformable image registration method 6153
Figure 3. Yosemite images: (a) the Yosemite 0 image, used as moving image, (b) the Yosemite 1
image, used as fixed image, overlaid with ground truth motion field, (c) the difference image.
approach, we also used the multigrid approach to sequentially carry out the registration in
multiple down-sampled image resolution stages. We used five stages for the pig lung image
set, and four stages for Yosemite, liver and kidney and patient lung image sets. The number
of stages is selected to ensure capturing of the largest possible image deformations. We used
eight passes for each stage. For the HS algorithm, we used five iterations for each pass with
α = 0.2. For the demons algorithm, we used σ = 2 pixels and did not use multiple iterations.
Before the two images were registered, their intensities were always normalized to [0, 1]
by dividing by the common maximal intensity value. The Laplacian pyramid down-sampling
filter (Burt and Adelson 1983) was used to half-sample the images in the multigrid approach.
Bilinear (for 2D images) or trilinear (for 3D images) interpolation was used for situations
where interpolation is needed. The differential mask [−1 8 0 −8 1]/12 was used for all
3.1. Image data sets
We used three 2D image data sets and one 3D image data set to test the new method. All three
2D image datasets are accompanied by ground truth motion fields. The 3D image data did
not have ground truth deformation fields, therefore manually selected landmarks were used
for accuracy validation purposes.
3.1.1. Yosemite sequence—2D.
sequence, which was originally generated at SRI (Barron et al 1994). The Yosemite image
truth motion field is known. Maximal magnitude of motion is 5.19 pixels. Figure 3 shows
both images and their difference. Evaluation using this image dataset would make it possible
to directly compare our method to other reported deformable registration algorithms.
We used the first two images from the Yosemite 2D image
3.1.2. CT images of pig lung phantom—2D.
phantom (Yang et al 2007b), with pixel size of 0.2441 × 0.2441 mm. The CT slice was
deformed according to a synthesized motion field. The original CT slice is used as the moving
image. The generated one is used as the fixed image. The synthesized motion field is the
ground truth. The maximal motion was 25 pixels. Figure 4 shows the original and the
deformed CT slice, the difference image and the optical flow vector field. We also deformed
We used a cropped CT slice of a pig lung
6154D Yang et al
Figure 4. Pig lung CT images: (a) the original slice, (b) the generated image, (c) the difference
image, (d) the generated image with ground truth motion field vectors. The difference image is
limited between [−0.5, 0.5] in order to show the difference better.
the CT slice with a similar synthesized motion field with maximal motion magnitude of
5 pixels. The second generated image is used for convergence analysis.
3.1.3. Patient upper abdominal CT scan—2D.
abdominal CT scan, which contains liver and kidney, with pixel size 0.9766 × 0.9766 mm.
This CT scan has much lower image contrast than the pig lung CT scan. It is good to test
for low image contrast situations. We deformed this CT scan with a synthesized motion field.
The maximal motion was 10 pixels. Figure 5 shows the original and the deformed CT slice,
the difference and the synthesized motion field. For convergence analysis, we used another
similar motion field of maximal magnitude of 3 pixels to deform the CT slice.
We used a transverse slice of patient upper
3.1.4. Patient 4D-CT images.
2007). The first volume represents an exhalation phase and the second one represents an
inhalation phase. Dimensions are 512 × 512 × 152 for both phases, with voxel size of
0.9766 × 0.9766 × 2.5 mm. For this study, we first cropped the 3D-CT volumes to delete
almost everything but lung and then half-sampled every 2D transverse slice. The volume
dimensions became 122 × 150 × 110, and voxel size became 1.9532 × 1.9532 × 2.5 mm.
Figure 6 shows the coronal slices of both volumes, the difference image and the checkerboard
image. In total, 38 corresponding landmarks have been manually selected by physicians on
the original 512 × 512 × 152 volumes. Among the 38 landmarks, 17 were in the right lung,
17 were in the left lung, 2 for heart and 2 for aorta. The landmark selection procedure was
reported in Brock (2007).
We used two 3D-CT volumes from a patient 4D-CT dataset.
3.2. Evaluation procedures
We performed several quantitative comparisons to evaluate the proposed inverse consistency
method against the corresponding asymmetric optical flow algorithms. We always used the
same parameters, multigrid and multiple pass settings in order to make the comparison as fair
A fast inverse consistent deformable image registration method6155
Figure 5. The liver and kidney CT images: (a) the original slice, (b) the generated image, (c) the
difference, (d) the generated image overlaid with motion field vectors. The difference image is
scaled and limited between [−0.2, 0.2] in order to visualize the differences better.
Figure 6. Patient 4D-CT images. (a) The exhaled phase—the moving image, (b) the inhaled
phase—the fixed image, (c) the difference.
3.2.1. Accuracy validation.
error vector field Verris computed as
After the resulting motion field is computed, the displacement
Verr= V − Vgt,
6156 D Yang et al
where V is the computed field and Vgtis the ground truth field. Mean, standard deviation and
maximal values are computed for |Verr|. For the 2D images, the mean and standard deviation
for absolute angular error Aerrare also computed. Aerris defined as
Aerr= |tan−1(Vy/Vx) − tan−1(Vy,gt/Vx,gt)|,
where tan−1is the inverse tangent function, Vyand Vxare the x and y component of V, and
Vy,gtand Vx,gtare the x and y component of Vgt. The ground truth motion fields are available
for the 2D images; Verrand Aerrare computed for the entire image.
point is not located in the center of a voxel, liner interpolation was used to compute V for
the landmark point. We did not compute angular errors for this image dataset. Instead, we
computed the mean, max and standard deviation for Verr,x, Verr,yand Verr,zwhere Verr,x, Verr,y
volumes are also called as LR (left–right), AP (anterior–posterior) and SI (superior–inferior)
registration accuracy. We avoided to use them in this work because (1) they are not accurate
measurements of registration accuracy since better MSE or MI does not directly translates into
higher accuracy, (2) MI is not useful for single modality image registration analysis.
in both forward and backward directions. For the proposed inverse consistency method, we
only performed registration once in the forward direction and the motion fields for both
registration directions were computed from the results of a single computation step. Inverse
consistency error ICE is computed according to equation (9) using the motion fields of both
3.2.3. Convergence analysis.
converging to the ground truth and converging to a stable solution. Note that converging
to a stable solution, does not necessarily mean that the stable solution is the known ground
truth as these problems are non-convex. We only used 2D image sets for such convergence
study, because computation cost will be too expensive for the 4D-CT data. Registrations for
convergence analysis were performed without using the multigrid approach so that the native
convergence ability of the methods can be evaluated.
We evaluated the algorithms for two types of convergence,
3.2.4. Analysis of accuracy improvement.
consistency optical flow algorithm is more accurate than the inverse-inconsistency version
of the HS and the demons algorithms (see section 4).
could contribute for this accuracy improvement: (1) using spatial gradients of both images;
(2) the symmetric formation that deforms both images to the middle. To test our conjecture,
we performed experimental simulations as follows. We used the sum of the gradients of
both images to replace the single image gradient in the original optical flow algorithms, and
performed registration in asymmetric way as the original asymmetric optical flow algorithms.
This means using equation (27) to solve V as defined in equation (1). Results of such modified
asymmetric optical flow algorithms were compared to the results by the original algorithms
and the proposed inverse consistency algorithms. The same multigrid, multiple pass and
smoothing parameters were used for all three algorithms to ensure that the comparisons were
We have observed that the proposed inverse
We conjecture that two things