Page 1

Combined Volumetric and Surface Registration

Gheorghe Postelnicu+,

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA 02176

USA (postelni@nmr.mgh.harvard.edu).

Lilla Zöllei+, and

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA 02176

USA lzollei@nmr.mgh.harvard.edu.

Bruce Fischl

Martinos Center for Biomedical Imaging, Massachusetts General Hospital, Boston, MA 02176

USA fischl@nmr.mgh.harvard.edu.

Abstract

In this paper, we propose a novel method for the registration of volumetric images of the brain that

optimizes the alignment of both cortical and subcortical structures. In order to achieve this,

relevant geometrical information is extracted from a surface-based morph and diffused into the

volume using the Navier operator of elasticity, resulting in a volumetric warp that aligns cortical

folding patterns. This warp field is then refined with an intensity driven optical flow procedure

that registers noncortical regions, while preserving the cortical alignment. The result is a combined

surface and volume morph (CVS) that accurately registers both cortical and subcortical regions,

establishing a single coordinate system suitable for the entire brain.

Keywords

Cortical and subcortical alignment; elastic warp; magnetic resonance imaging (MRI); volumetric

registration

I. Introdution

PAIR-WISE brain image registration is an active area of research in the medical imaging

community. Various algorithms have addressed the generic problem of registering

information from two brain scans using an array of different techniques that can be broadly

classified as either volume based or surface based. Volumetric registration (see [1] for a

thorough survey) seeks a 3-D deformation field that is driven by either raw intensity

information or features derived from image intensities. A different approach is to extract

geometric features from surface models of anatomical structures such as the neocortex, and

to reformulate the complex correspondence problem in a surface matching framework.

Both of these approaches have advantages and weaknesses. Surface-based methods [2]–[5]

have been shown to accurately align the highly complex folding pattern of the human

cerebral cortex, and to result in increased statistical power for averaging of functional data

across subjects, presumably due to their alignment of functionally homologous regions

© 2009 IEEE

Correspondence to: Gheorghe Postelnicu.

+Contributed equally.

NIH Public Access

Author Manuscript

IEEE Trans Med Imaging. Author manuscript; available in PMC 2009 October 14.

Published in final edited form as:

IEEE Trans Med Imaging. 2009 April ; 28(4): 508–522. doi:10.1109/TMI.2008.2004426.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 2

across subjects [3]. This accuracy stems from the direct use of geometric information that is

generally unavailable to volumetric methods, and the relatively close relationship between

folding patterns and functional properties of the neocortex. Conversely, volumetric methods

[6]–[10] provide a correspondence field across the entire brain, including subcortical and

ventricular regions that are not in the domain of the surface-based alignment procedures, but

typically fail to align corresponding cortical folds across subjects.

The failure of nonlinear volumetric procedures to align cortical folding patterns stems from

the highly nonconvex nature of the driving energy functionals when initialized with affine

transforms. Fundamentally, a 12 parameter affine transform, which is typically used as an

initial estimate of the displacement fields, does not bring many homologous folds into close

enough correspondence to allow nonlinear techniques to align them. In essence, if the

initialization of a volumetric nonlinear warp does not bring a fold within one half the

interfold spacing of the homologous fold then intensity-based algorithms will converge to

the incorrect solution.

In this paper, we propose a method that combines surface-based and volumetric approaches,

preserving the strengths of each. This is accomplished by integrating surface-based

information into a volumetric registration procedure through the use of a biomechanical

deformation model. The result is a 3-D displacement field that aligns both cortical folding

patterns and subcortical structures across subjects. While the idea of using surface-based

registration to drive volumetric registration is not new [11]–[13], here we focus on the

automated and accurate registration of both cortical folding patterns and noncortical regions

in 3-D space.

Regularizing registration procedures by employing biophysical models of the underlying

deformable objects is an attractive approach in the field of medical image registration as it

brings to bear branches of mathematics that allow the generation of correspondence fields

with specified properties (e.g., smoothness and invertibility). One of the first such

presentations of this type of registration was that of Bajcsy et al. [6], who built on the work

of Broit [14], where images were considered to be solid linear elastic bodies constrained at

certain position by applied external forces. Extensions of this early work have been since

proposed by Davatzikos [7], Ferrant [12], and Toga [13], who use sparse displacement fields

extracted from curves or surfaces to produce a sparse displacement field on the boundary of

the cortex, which is then diffused using a volumetric elastic model. A drawback of these

methods is that they are accurate close to the boundary driving the deformation, but they fail

to properly align structures distant from it.

Collins [15] was the first to propose a method that addresses this problem. He used surface

information through manually extracted sulcal traces together with intensity information to

obtain a deformation field that was accurate both in the cortical areas as well as in regions

distant from the cortex.

Other hybrid methods, such as HAMMER [16] implicitly incorporate surface as well as

volume information in the alignment through the use of feature vectors that represent both

intensity as well as geometric information in the registration process. More recent work

extending HAMMER [17] uses the result of HAMMER as an initialization to further

improve cortical surface alignment between the two subjects being registered.

Our current approach [18] is similar to that of Joshi et al. [19], [20]. These two methods

have the same basic ingredients. First, a surface-based registration algorithm is applied and

is used to derive a volumetric displacement field. Subsequently, an intensity-based

registration is computed to match regions distant from the surfaces. It is worth pointing out

that the surface registration used in the two approaches differ significantly in that the

Postelnicu et al.

Page 2

IEEE Trans Med Imaging. Author manuscript; available in PMC 2009 October 14.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 3

surface-based registration method employed by Joshi et al. is not automated, requiring a

trained neuroanatomist to specify a set of corresponding landmarks across subjects. The

current algorithm extends our preliminary work [18], and uses an automated algorithm for

surface-based registration [21] to obtain an initial estimate of a displacement field that aligns

two individual brain images. The resulting alignment is based on a biophysical model of the

brain, thus yielding a smooth, invertible deformation field. Using this deformation as an

initialization, we then apply an intensity-based registration [9] and [22].

Initializing the deformation field by aligning cortical folds provides an extremely well-

constrained alignment of noncortical regions, which can then be brought into register using a

nonlinear intensity-based volumetric deformation.

The remainder of this paper is organized as follows. In Section II we give an overview of the

preprocessing steps that lead to accurate and topologically correct cortical surface models

given an MRI image, as well as of the surface-based registration that brings the geometric

features of these surfaces into register across subjects. The extraction and registration of the

cortical surfaces are performed automatically using previously described techniques and

freely available software (http://surfer.nmr.mgh.harvard.edu/fswiki), hence, we only

describe them briefly. In Section III we provide details about the diffusion of the surface-

based registration results into the volume, using the Navier operator derived from the theory

of linear elasticity. We detail the discretization of the Navier operator, the numerical scheme

used in the implementation, and the technical aspects of working with tetrahedral meshes. In

Section IV, we briefly explain the intensity-based registration algorithm that was previously

presented in [9]. Finally in Section V we present our results on a varied set of real medical

data sets, showing the accuracy of the resulting registration procedure both cortically and

throughout the brain.

II. Surface-Based Registration

We present an algorithm for registering two structural brain scans, arbitrarily denoted target

and moving. This registration involves three main steps. First, cortical folding patterns are

registered using information derived from cortical surface models. Then this warp is

projected back to the volumetric domain and diffused using linear elasticity constraints.

Finally an optical flow intensity-based registration algorithm is applied, using the output of

the second step as an initialization. This framework results in a nonlinear displacement field

which simultaneously registers cortical and subcortical structures.

In the preprocessing step, the input brain scans are independently processed to obtain

accurate, topologically correct reconstructions of the cortical surfaces (see [21], [23]–[26]

for details). As a result we obtain four detailed cortical surfaces for each brain scan. For

each hemisphere, one of the surfaces is extracted from the boundary between the gray and

white matter (denoted the white surface), while the other surface is automatically positioned

at the pia between cortical gray matter and cerebrospinal fluid (CSF) (denoted the pial

surface).

The reconstruction of the cortical surfaces is a complex procedure [24] that is broken into a

number of subtasks. First, intensity variations due to magnetic field inhomogeneities are

corrected [27] and a normalized intensity image is created from a T1-weighted anatomical 3-

D MRI data set [9]. Next, extracerebral voxels are removed, using a “skull-stripping”

procedure [23]. The intensity normalized, skull-stripped image is segmented based on the

geometric structure of the gray-white interface. Cutting planes are then computed that

separate the cerebral hemispheres and disconnect subcortical structures from the cortical

component [24]. This generates a preliminary segmentation that is partitioned using a

connected components algorithm. Any interior holes in the white matter components are

Postelnicu et al.

Page 3

IEEE Trans Med Imaging. Author manuscript; available in PMC 2009 October 14.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 4

filled, resulting in a single filled volume for each cortical hemisphere. Finally, the resulting

volume is covered with a triangular tessellation and deformed to produce an accurate and

smooth representation of the gray-white interface as well as the pial surface. This surface

departs from a simple spherical topology due to the presence of subcortical gray matter as

well as various midbrain structures. These topological “defects” are automatically removed,

resulting in a surface that is both geometrically accurate and topologically correct [23], [25],

[28].

Each of the corresponding surfaces of the target and moving scans are then registered in the

spherical space [3], [29]. The algorithm minimizes the following energy functional:

(1)

where Jp measures the alignment, based on cortical depth and curvature information, while

the other two terms act as regularizers. JA is a topology-preservation term, while Jd controls

the amount of metric distortion allowed

(2)

(3)

In (2) and (3), superscripts denote integration time with 0 being the starting point, T and V

are the number of triangles and vertices in the tessellation, is the position of vertex i at

iteration n, N(i) is the set of neighbors of the ith vertex and Ai denotes the oriented area of

triangle i. The result of the surface registration is a 1-to-1 mapping that transports all

surfaces of the moving scan to the counterpart surfaces in the target image

, where S can be any of the left/right pial/white surfaces of the brain.

An example of the representation of the sulcal patterns being registered in spherical space is

displayed in Fig. 1. On the left-hand side it is the subject to be registered, in the middle the

already registered subject and on the right-hand side the target that is displayed. The

colormap we use represents average convexity information, or the signed distance a point

has to move to reach the inflated surface [21].

III. Volumetric Registration of the Cortical Structures

When the surface registration is completed, we obtain a displacement vector field that

provides a 1-to-1 mapping between the hemisphere surfaces of the target and moving brain

scans in Euclidean space. In the following we show how we diffuse this vector field from

the cortical surfaces to the rest of the volume.

Let

the target image as: ϕ(x) = x + u(x) where

goal is to find a function ϕ such that ϕ(x) = Freg(x), for any

white surfaces of brain ), where Freg is the specified displacement through the surface

registration procedure. Since the surfaces represent a space of co-dimension 1, in order for

this problem to be well-posed, we impose an additional regularity constraint. We require the

displacement field to be an elastic deformation, i.e., a smooth, orientation-preserving

be the target image domain. We define an arbitrary transformation of

denotes the displacement field. The

(one of the left/right pial/

Postelnicu et al.

Page 4

IEEE Trans Med Imaging. Author manuscript; available in PMC 2009 October 14.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript

Page 5

deformation which satisfies the equations of static equilibrium in elastic materials. This

means that we (additionally) require u to satisfy

(4)

where is an operator we define below.

The choice of the operator

displacement field has numerous solutions that have been proposed in the registration

literature. Some of the widely used ones are the thin plate splines, proposed by Bookstein

[30], and the B-splines-based free-form deformations proposed by Rueckert et al. [31]. We

have chosen to use the Navier operator from the linear elasticity theory together with the

finite element method. This choice was motivated by the high level of flexibility needed in

order to satisfy the constraints imposed by the displacement fields on all four surface models

of the brain (left and right hemisphere white matter and pial surfaces).

and the discretization method to numerically extrapolate the

We consider this component of the registration framework to be the most technically

significant. The remainder of this section is divided as follows. We start with a short

presentation of the partial differential equation of static elasticity. Next, we derive the

external forces directly from the output of the surface registration algorithm, as a set of

prescribed displacements. Finally, technical aspects regarding our discretization method are

presented.

A. Elastic Operator

In order to solve the problem stated above, we use the equilibrium equation for elastic

materials. This states that at equilibrium, the elastic energy equals the external forces applied

to the body [32]

(5)

Here, ∇u = ((∂ u)/(∂ x), (∂ u)/(∂ y), (∂ u)/(∂ z)) denotes the gradient and the second Piola-

Kirchoff stress tensor is defined as

and

is the Green-St. Venant strain tensor [32]. Here λ and μ are the Lamé elastic constants that

characterize the elastic properties of an isotropic material. The linear approximation to the

above operator uses the Fréchet derivative of

(6)

since no deformation occurs in the absence of external forces. Finally,

by dropping the nonlinear terms in

is computed

, which results in

Postelnicu et al.

Page 5

IEEE Trans Med Imaging. Author manuscript; available in PMC 2009 October 14.

NIH-PA Author Manuscript

NIH-PA Author Manuscript

NIH-PA Author Manuscript