Incorporating Prior Knowledge on Class Probabilities into Local Similarity Measures for Intermodality Image Registration

Abstract

We present a methodology for incorporating prior knowledge on class probabilities into the registration process. By using knowledge from the imaging modality, pre-segmentations, and/or probabilistic atlases, we construct vectors of class probabilities for each image voxel. By defining new image similarity measures for distribution-valued images, we show how the class probability images can be nonrigidly registered in a variational framework. An experiment on nonrigid registration of MR and CT full-body scans illustrates that the proposed technique outperforms standard mutual information (MI) and normalized mutual information (NMI) based registration techniques when measured in terms of target registration error (TRE) of manually labeled fiducials.

Full text

Page 1

Incorporating Prior Knowledge on Class
Probabilities into Local Similarity Measures for
Intermodality Image Registration
Matthias Hofmann1,2,3, Bernhard Sch¨ olkopf2,
Ilja Bezrukov2,3, and Nathan D. Cahill1,4
1Department of Engineering Science, University of Oxford, Oxford, UK
2Max Planck Institute for Biological Cybernetics, T¨ ubingen, Germany
3Department of Radiology, University Hospital T¨ ubingen, Germany
4School of Mathematical Sciences, Rochester Institute of Technology, NY, USA
Abstract. We present a methodology for incorporating prior knowledge
on class probabilities into the registration process. By using knowledge
from the imaging modality, pre-segmentations, and/or probabilistic at-
lases, we construct vectors of class probabilities for each image voxel. By
defining new image similarity measures for distribution-valued images,
we show how the class probability images can be nonrigidly registered in
a variational framework. An experiment on nonrigid registration of MR
and CT full-body scans illustrates that the proposed technique outper-
forms standard mutual information (MI) and normalized mutual infor-
mation (NMI) based registration techniques when measured in terms of
target registration error (TRE) of manually labeled fiducials.
1 Introduction
Intermodality image registration describes the task of aligning two images that
have been acquired using different modalities and thus have values with different
meanings. Typical registration methods are designed in terms of three compo-
nents: the allowable space of geometric transformations, the similarity measure
relating two images, and the optimization technique for finding the optimal value
of the similarity measure over the space of allowable transformations.
Mutual information (MI) [1,2] is one of the most commonly used similar-
ity measures in intermodality image registration. The underlying assumption is
that if images values are treated as realizations of some underlying probabil-
ity distribution function, then the mutual information between both images is
maximized when they are aligned. The reason for the success of MI and related
measures such as normalized mutual information (NMI) [3] lies in the fact that
these measures do not make any underlying assumptions about the modalities.
This allows them to be used for a whole range of inter-modality registration
problems. In practice, however, most volume registration problems in medical
imaging applications use images acquired from a limited number of modalities,
including MR, CT, PET, and Ultrasound. When prior knowledge about the im-
ages, modalities, or optimal transformations is available, this knowledge can be

Page 2

exploited to enable better registration results than would be possible with MI
or NMI.
Prior knowledge has been previously exploited in a number of ways for im-
age registration [4–9]. Ashburner et al. [4] have shown how incorporating prior
knowledge about an object’s shape and scale can significantly improve accuracy
and reliability for affine registration. Leventon and Grimson [5], Guetter et al. [6],
Chung et al. [7], and Sabuncu and Ramadge [8] have all proposed variations on
the idea of learning the optimal joint distribution from exemplar aligned images
and then designing a similarity measure that reflects the difference between the
learned joint distribution and an observed joint distribution. Lee et al. [9] have
focused directly on learning an optimal similarity measure for a specific pair of
image modalities, assuming exemplar aligned images from those modalities are
provided.
While these techniques are useful for certain intermodal registration prob-
lems, they seem to rely directly on the image intensities; hence, they may not be
directly generalizable to all types of intermodal registration. Whereas CT scan-
ners are calibrated to yield images with an accuracy of within a few Hounsfield
units, this is not possible with MR scanners and MR typical field inhomogeneities
lead to severe intensity bias. Furthermore, it is not necessarily true that exem-
plar aligned images will always be available; this is especially true in situations
when nonrigid registration is required.
Other previous work has attempted to use prior information in the form of
tissue class probabilities to aid in registration. D’Agostino et al. [10] describe
how to align images by minimizing the Kullback-Leibler divergence between the
actual joint class distribution and an ideal joint class distribution, although their
formulation assumes that individual class probabilities in pairs of correspond-
ing voxel positions in two images are independent. Lorenzen et al. [11] avoid
making this assumption and propose an approach based on local KL divergence.
Although Lorenzen’s approach is applicable for the more general problem of
multiple image registration, it requires the additional step of atlas construction
even when only two images must be registered.
In this paper, we show how to design a nonrigid registration algorithm based
on vector valued images. Such a nonrigid registration algorithm exploits prior
knowledge about the imaging modalities, but unlike other techniques, it does
not require exemplar aligned images, independence assumptions, or atlas con-
struction. We compare the performance of the proposed nonrigid registration
technique versus standard MI-based registration on full-body MR/CT images,
measuring the results in terms of target registration error (TRE) on manually
labeled fiducial points.
2 Prior Knowledge on Class Probabilities
In medical images, the intensity at a given voxel depends on the underlying tissue
class at the voxel’s position. If it is possible to infer the underlying tissue class
at each position in an image, this knowledge could potentially be exploited by

Page 3

Fig.1. Estimating class probabilities using Bayes’ rule and pre-segmented images.
Red/blue colors indicate high/low probabilities, respectfully.
a registration algorithm. In many situations, however, it is difficult to uniquely
identify the underlying tissue class at a voxel solely from the intensity value at
that voxel. This can be remedied by using additional information that may be
available directly from the imaging device, from a pre-segmentation, or from a
probabilistic atlas.
One example of additional information that may be provided directly from an
imaging device is the phase information in MR imaging that is available after MR
reconstruction [12]. This information is not usually used for diagnosis, but may
be useful in inferring the underlying tissue class. If no additional direct source
of information is available, class probabilities can be estimated based on the
intensity values by using pre-segmented images. Assuming that pre-segmented
images and the image to be registered have similar intensity characteristics, the
probability that a particular voxel has tissue class C, given the voxel intensity
I, is given by p(C|I). According to Bayes’ rule, p(C|I) ∝ p(I|C) · p(C), where
p(C) is a prior probability on how often class C is expected to occur. Figure 1
provides an illustration of the process of estimating class probabilities at each
voxel in a full-body MR scan.
Alternatively, if there is no direct source of additional information available,
and pre-segmentation is not desirable, probabilistic atlases can be used to provide
prior information about which tissue class is expected at different positions in
the patient [13,14]. Since the anatomy of different subjects shows structural sim-
ilarities, the images of different subjects can be brought into alignment through
intersubject registration. For example, a template MR image for which tissue
labels are available can be registered to a patient MR image. Applying the same
transformation on the label image yields a tissue label prediction for the patient
MR image. Performing this procedure for several template MR image and label
pairs can be used to derive a position-depedendent prior probability p(C|X).

Page 4

This prior probability can be combined with the intensity-derived class estimate
using Bayes’ rule:
p(C|I,X) ∝ p(C|X) · p(I|C)
Whether estimated directly from the imaging modality, or indirectly from
pre-segmented images or probabilistic atlases, once class probabilities have been
derived for each voxel, they can be used to form distribution-valued images.
These distribution-valued images can be subsequently used for registration, hope-
fully yielding better results than registration based solely on the original intensity-
valued images.
(1)
3 Nonrigid Registration Preliminaries
In this section, we describe a variational framework for registering two images,
using MI or NMI as a similarity measure. In the next section, we will extend
this framework and introduce new similarity measures to allow for vector-valued
and distribution-valued images.
Consider two images, a reference image R and a floating image F, both
as functions mapping Ω ⊆ Rd?→ RN. Define a deformation Φ : Rd?→ Rdby
Φ(x) = x−u(x), and call u the displacement. The general form of the variational
registration problem is given by:
min
u
E(R,F,u) := S(u) + αJ(R,Fu) ,
(2)
where J is a similarity measure that quantifies the similarity between the refer-
ence image R and the deformed floating image Fu:= F(Φ), S is a regularizer
that ensures that the minimization problem is well-posed and that the solution
is smooth in some sense, and α is a weighting parameter.
In this paper, we will assume the use of the curvature regularizer [15]:
S(u) :=
?
Ω
d
?
j=1
(∆uj(x))2dx .
(3)
Normally, variational registration proceeds by identifying the Euler-Lagrange
equations associated with (2) and embedding them in an artificial time. When
the curvature regularizer is used, this process yields the evolution equation:
∂tu(x,t) + ∆2u(x,t) = −αf(x,R,Fu),
u(x,0) = u(0)(x) ,
x ∈ Ω,t > 0,
(4)
(5)
where u(0)(x) is typically chosen to be 0, and where Dirichlet or Neumann bound-
ary conditions are typically imposed on the boundary of Ω. The equilibrium
solution of (4)–(5) yields a stationary point of (2).
To handle large deformations, we follow a strategy similar to fluid registration
[16], and solve the evolution equation in terms of the velocity field v instead of

Page 5

the displacement field u; i.e.,
∂tv(x,t) + ∆2v(x,t) = −αf(x,R,Fu),
v(x,0) = 0 ,
x ∈ Ω, t > 0,
(6)
(7)
where the velocity and displacement are related by the material derivative:
v(x,t) = ∂tu(x,t) + (∇u(x,t))Tv(x,t).
The vector field f(x,R,Fu) that forms the nonhomogeneous term in (4) is
called the force vector, and it arises from the Gˆ ateaux derivative of J. Force
vectors can typically be determined analytically from the similarity measure at
hand. For the MI and NMI, we first present the similarity measures and then
their corresponding force vectors. The MI and NMI are defined by:
(8)
MI(R,Fu) := H(R) + H(Fu) − H(R,Fu) ,
NMI(R,Fu) :=H(R) + H(Fu)
H(R,Fu)
where H(A) and H(A,B) are the marginal and joint entropies, defined by:
?∞
H(A,B) = −
−∞
and where pA(a) and pA,B(a,b) are the values of the probability density func-
tion of A at a, and the joint probability density function of (A,B) at (a,b),
respectively.
The corresponding force vectors for MI and NMI are given by:
(9)
,
(10)
H(A) = −
−∞
?∞
pA(a)logpA(a)da ,
?∞
(11)
−∞
pA,B(a,b)logpA,B(a,b)dadb ,
(12)
fMI(x,R,Fu) = PH(x;Fu) − PH(x;R,Fu) ,
fNMI(x,R,Fu) =PH(x;Fu) − NMI(R,Fu)PH(x;R,Fu)
(13)
H(R,Fu)
,
(14)
where
PH(x;Fu) =
1
|Ω|
1
|Ω|
?
?
p
pFu(Fu(x))
?
Fu(Fu(x))
?
∇Fu(x) ,
(15)
PH(x;R,Fu) =
∂
∂f[pR,Fu(R(x),f)]f=Fu(x)
pR,Fu(R(x),Fu(x))
?
∇Fu(x) .
(16)
The force vector for MI is derived in [17], and the force vector for NMI follows
a similar derivation.
Once the force vector has been chosen, the evolution equations (4) or (6)
can be approximately solved by introducing a discretization that is backward in
time and centered in space, and then by using Fourier methods [18] to resolve the
linear system that arises at each time step. If (6) is being solved, the displacement
field u can be computed from the velocity field v at each time step by Euler
integration of (8).