ITK-based Registration of Large Images from
Light Microscopy: A Biomedical Application
K. Mosaliganti1, T. Pan2, R. Machiraju1, K. Huang2, and J. Saltz2
Department of 1Computer Science and Engineering, 2Biomedical Informatics
The Ohio State University, Columbus, Ohio, USA
Abstract. Inactivation of the retinoblastoma gene in mouse embryos
results in morphological changes in the placenta, which has been shown
to aﬀect fetal survivability. The construction of a 3D virtual placenta
aids in accurately quantifying structural changes using image analysis.
The placenta dataset consisted of 786 images totaling 550 GB in size,
which were registered into a volumetric dataset using ITK’s registra-
tion framework. The registration process faces many challenges arising
from the large image sizes, damages during sectioning, staining gradients
both within and across sections, and background noise leading to local
solutions. In this work, we implement a rigorous ITK-based preprocess-
ing pipeline for removing noise and employ a novel 2-level optimization
strategy for enhanced registration in ITK. We provide 3D visualizations
and numerical results to demonstrate our improvements.
Many human diseases have genetic bases that manifest phenotypically as func-
tional and/or structural deﬁciencies. The retinoblastoma (Rb) gene is one of the
ﬁrst to be associated with a speciﬁc cancer, and has been studied extensively
in both human cells and mouse models . Recently, it has been shown that
inactivation of Rb (Rb−) in mouse embryo results in morphological changes in
the placenta, including reduction in vascularity of the labyrinth layer. Figure
1 shows a sample slide with the approximate labyrinth,spongiotrophoblast and
glycogen layers. The labyrinth layer serves the critical function of gas, nutrient,
and waste exchange between the mother and the fetus and hence understanding
its interaction with neighboring tissue layers is important. A decrease in vas-
cularity is thought to contributes to fetal death at 14.5 days of gestation .
Image analysis of the structural changes in mouse placenta provides an oppor-
tunity for quantitative correlation of genetic changes with phenotype expressions.
Speciﬁcally, the goals of image analysis are segmentation of the tissue layers, vi-
sualization of the ﬁnger-like inﬁltrations into the labyrinth, and quantitative
measurement of volume and surface area of the layers. Registration of the tissue
sections is therefore, an important pre-processing step for various algorithms as
shown in Fig. 1 (Right), and its accuracy directly aﬀects the quantiﬁcation and
Fig. 1. Left: Placenta slice showing diﬀerent tissue types. A small region magniﬁed to
show labyrinth layer (within red boundaries), spongiotrophoblast layer (between the
red and green boundaries) and spongiotrophoblast and glycogen inﬁltrations (yellow
circles). Right: Overall image analysis framework
interpretation of the results. Mutant (Rb−) placenta was harvested at 13.5 days
of gestation and prepared using a standard histological protocol. It was ﬁxed in
paraﬃn and sectioned at 3µm thickness using a microtome. Serial sections were
mounted on glass slides and stained with hematoxylin and eosin. The slides are
scanned in an Aperio ScanScope slide digitizer at 200x magniﬁcation. The image
dimensions on average were (15K x 15K x 3). The entire placenta produced 786
color images, ranging in size from 500 MB to 1 GB each. The manual nature
of the tissue preparation process required extensive preprocessing steps as well
as modiﬁcations to the registration process. Our pipeline was implemented in in
the National Library of Medicine’s (NIH/NLM) Insight Segmentation and Reg-
istration Toolkit (ITK) . The following artifacts were observed in the images:
Fig. 2. A: background noise from acquisition. B: tissue tear and shear. C,D: consecutive
sections have non-linear intensity variations.
1. Orientation diﬀerences: Sections are mounted in diﬀerent orientations on glass
slides due to the manual nature of the process.
2. Luminance gradient: Sections mounted close to the edge of the glass slide
produce images with signiﬁcant luminance gradients (Fig. 2A).
3. Non-tissue noise: Dust and air bubbles in the slide may cause artifacts.
4. Damaged and missing sections: During sectioning and mounting, sections are
occasionally torn, folded, or discarded entirely. (Fig. 2B).
5. Staining variations: Diﬀerences in section thickness, staining duration, and
Fig. 3. Left: Pre-processing pipeline. Right: ITK-based registration framework. Trans-
forms are scaled when passed from a lower to higher resolution.
stain concentration result in color variations (Fig. 2C,D).
In what follows, Section 2 explains the ITK framework for noise removal
with the details of our registration framework components in Section 3. Section
4 provides details on our new optimization strategy with experimental and vi-
sualization results. Finally, we report our conclusions and outline our plans for
the future in Section 5.
2 Preprocessing Framework
Defective Section Exclusion: Defective sections are identiﬁed using manual in-
spection and excluded from the pipeline. They are later accounted for in the 3D
reconstruction and visualization process.
Tissue Detection: The ﬁrst step separates the tissue (foreground) from the back-
ground (noise). We use the k-BCC clustering approach  with multi-resolution
kd-tree optimizations for tissue detection, as it performs well in the presence of
luminance gradient, background noise, and staining variations. It is a variation
of the popular k-means algorithm adapted to the light microscopy images. It is
validated for the placenta slices and compared with the Expectation Maximiza-
tion (EM) and k-means clustering in . Tissue pixels are identiﬁed and stored
as binary masks which are used in the level-set based edge detection and the
PCA alignment of sections.
Level-Sets for edge detection The binary mask obtained from the clustering al-
gorithm has a poor description of the placenta boundary as shown in Figure 4.
Moreover, parts of the placenta interior that were initially occupied by tissues of
low luminance (such as vasculature) are clustered as background. These hollow
regions also contribute to noise when evaluating the registration match across
images since they are usually not well-correlated. Therefore, by using level-sets
, an accurate boundary description is obtained by growing a contour inwards
from the edges of the image boundary.
PCA Alignment: The possible translations and rotations in a plane deﬁne the
transform space for the placenta images. Using a priori knowledge that mouse
placenta sections are typically elongated in shape, we use principle component
analysis (PCA) to estimate tissue orientation. The orientations are used to ini-
tialize registration, thus restricting the transform search space. PCA is performed
Fig. 4. Left: An example binary mask obtained from the KBCC algorithm. Center:An
example tissue mask image from level-sets. Right: Top sequence shows the initial ran-
dom orientation of the placenta images. The bottom sequence shows PCA-aligned
for each tissue mask image. The gleaned orientation angle is used to rotate and
align the original histology images. Figure 4 shows the same two sequences of
images before and after PCA alignment.
3 Mutual Information Based Registration
While the placenta images are acquired using the same staining protocol, they
have multimodal characteristics due to non-linear variations in pixel intensities.
This may be attributed to diﬀerent stain absorptions, tissue thickness and lu-
minance gradients. Under these conditions, it is diﬃcult to detect appropriate
landmarks for point or surface-based registration . Intramodal registration
methods that rely on linear correlation of pixel values are also inadequate .
Mutual information (MI)  based methods are eﬀective in registering multi-
modal images with non-linear pixel intensity correlations.
We perform registration as an optimization process that searches for an im-
age transform that allows the closest similarity between consecutive images. The
input is a pair of stationary (S) and moving (M) images. The framework itera-
tively passes Mthrough 4 stages: transform,metric,optimizer, and interpolator
as shown in Figure 3 (Right). At a given iteration, the transform stage applies
transform Tnon Mwhich is then resampled onto a grid in the interpolator
stage to yield Tn(M). The metric stage computes the goodness of ﬁt between S
and Tn(M). Based on the similarity metric values from current and prior itera-
tions, the optimizer produces a reﬁnement in transform Tn+1. At convergence,
Sand Tn(M) have the optimal metric value thereby aligning the images. For
the placenta dataset, we register each image to its predecessor, and the pairwise
transforms were merged sequentially to obtain each image’s global transform.
We adopt regular step gradient descent optimizer  in the interpolator and
optimizer stages. We model the transform using rigid 2D transform, which allows
2D rotation and translation. This choice is based on the physical conditions that
the tissue does not elastically deform during sectioning or mounting, that the
microtome produces parallel sections, and that all images are acquired at the
same magniﬁcation. The only variable components of the transform are in-plane
rotation and translation.
Fig. 5. Application of the regular step gradient descent (A) and the two-level opti-
mizer (B) on PCA-aligned slides. Note that the regular step gradient descent settles
in local solutions. Right: Plot showing the metric values after convergence (blue). The
improvement in metric values using the two-level versus the standard optimizer is also
shown (red). Note that the negative of mutual information is plotted.
We apply the multiresolution approach, using 2-level image pyramids as
shown in Fig. 3 (Right). Optimal transforms obtained from a lower magniﬁca-
tion (Xn) are scaled and used as initialization for registration of the next higher
magniﬁcation (Xn+1). The process is repeated for each magniﬁcation level to
obtain the ﬁnal transforms. We note that at magniﬁcations higher than 20x, the
computational cost for registration outweighs the improvements in accuracy.
4 Two-level Optimization
Tissue detection and PCA-based image alignment signiﬁcantly improved reg-
istration performance by reducing the search space for the optimal transform.
However, 3D visualization of the registered placenta revealed that the transforms
remained suboptimal. Experiments with diﬀerent optimizers such as conjugate-
gradient and gradient descent did not aﬀect performance signiﬁcantly due to
overtly noisy metrics. We propose the use of a novel two-level optimization
scheme by searching the transform space neighborhood with a random walk
Optimizers such as regular-step gradient descent and conjugate gradient ()
increase the rate of convergence by reﬁning the transform based on their current
learning rates from observed MI gradients. However, their performance degrades
rapidly since the learning rates are sensitive to data noise. In the placenta data,
we observe that ITK optimizers converge to local solutions as shown in Fig. 5(A)
for most image pairs (better MI values were later observed with the two-level
We use a two-level optimization strategy, which perturbs converged solutions
via random walks and then restarts the normal registration process. In essence,
the perturbation introduces a step in the transform space that is larger than
the speciﬁed learning rate of the regular step gradient descent optimizer. The
step is a restricted translation and rotation around the PCA initialization. If the
previous result was a local minimum, then the perturbation may result in a better
solution with higher MI (lower -MI) as shown in Fig. 5 (B) and a new solution
is realized. The process is repeated until the MI does not improve even after an
user-deﬁned Nperturbations. Similar to standard optimizers, convergence to a
global solution is not guaranteed. However, the registration is likely to perform
better since this approach allows hill climb against the gradient direction.
Fig. 6. Left: 3D visualization of the labyrinth layer within the placenta slices, shown as
white boundaries. Right: A single ﬁnger like inﬁltration from the spongiotrophoblast
into the labyrinth is zoomed and shown.
In the placenta images, we determined that maximum transform deviation
of PCA aligned images is ±80 pixels in translation and ±10 degrees in rotation
around the tissue centroid. The hybrid optimization strategy lead to global solu-
tions in 99% of the images within N=20 iterations. The remaining 1% converges
within N=50 iterations. Figure 5 (Right) shows the plots of the ﬁnal MI values
of all the images (blue) using the two-level optimizer. The improvement of the
two-level optimizer compared to standard optimizer is shown as a diﬀerence in
the MI values at convergence (red). The average negative MI of all images using
the standard and two-level optimizers are -0.122 and -0.142 respectively. Fig-
ure 6 (Left) shows the 3D visualization of the labyrinth layer and the placenta
contours. Figure 6 (Right) show the zoomed cross-section of a single ﬁnger like
inﬁltration from the spongiotrophoblast into the labyrinth layer. Figure 7 is yet
another example of ﬁnger visualization.
5 Conclusion and Future Work
In this paper, we described our experience in registering large serial sections
of a histology sample. Our contribution involves methods to adapt the ITK
registration process for real-world data such as the mouse placenta. We utilized
pre-processing techniques to account for acquisition artifacts, defective histology
sections, and large parameter search space. We also employed a multi-resolution
implementation of MI registration algorithm. We propose a new optimization
Fig. 7. Both images are cropped volumes from the original dataset focusing on inﬁltra-
tions into the labyrinth layer. In both images the labyrinth is shown in blue. In the top
image glycogen is shown in yellow and placenta boundary in white, while the bottom
image highlights glycogen in red. The cross-hairs indicate the center of the inﬁltration
into the labyrinth layer.
strategy that has a higher probability of convergence to global solutions albeit
with longer processing times. Our results show an improvement in the ﬁnal 3D
reconstructions and overall lower MI metric values.
1. Mouse Genome Sequencing Consortium: Initial sequencing and comparative analysis
of the mouse genome. Nature 420 (2002) 520–562
2. Wu, L., de Bruin, A., Saavedra, H.I., Starovic, M., Trimboli, A., Yang, Y., Opavska,
J., Wilson, P., Thompson, J.C., Ostrowski, M.C., Rosol, T.J., Woollett, L.A., Wein-
stein, M., Cross, J.C., Robinson, M.L., Leone, G.: Extra-embryonic function of Rb
is essential for embryonic development and viability. Nature 421 (2003) 942–947
3. Ib´a˜nez, L., Schroeder, W.: The ITK Software Guide. The Insight and Registration
Toolkit [www.itk.org], Kitware, Inc. (2003)
4. Rao, K., Machiraju, R., Heverhagen, J., Saltz, J., Knopp, M.: Exploratory segmen-
tation using geometric tessellations (in presss). In: Proceedings of 10th International
Fall Workshop on Vision, Modeling and Visualization. (2005)
5. Malladi, R., Sethian, J.A., Vermuri, B.C.: Shape modeling with front propagation:
A level set approach. IEEE Trans. on Pattern Analysis and Machine Intelligence
17(2) (1995) 158–174
6. Maintz, J.A.B.: Retrospective registration of tomographic brain images. Ph.D.
dissertation, Univ. Utrecht, Utrecht, The Netherlands (1996)
7. Maes, F., Vandermeulen, D., Suetens, P.: Medical image registration using mutual
information. Proceedings of the IEEE 91(10) (1996) 1699–1722
8. Maes, F., Collignon, A., Vandermeulen, D., Marchal, G., Suetens, P.: Multimodality
image registration by maximization of mutual information. IEEE Trans. Medical
Imaging 16(2) (1997) 187–198
9. Maes, F., Vandermeulen, D., Suetens, P.: Comparative evaluation of multiresolution
optimization strategies for multimodality image registration by maximization of
mutual information. Medical Image Analysis 3(4) (1999) 373–386