Fast interactive integration of cross-sectional image datasets and surface data for morphometric analysis.
ABSTRACT To investigate external facial morphology and cell proliferation patterns and their relationship with cleft lip malformation in mice, we need to compare samples of mice tissue photographs and surface reconstructions from micro-CT scans obtained from mouse embryos. Tissue samples obtained through digital photography are typically misaligned with respect to each other, which prevents further analysis. We have developed a system for fast interactive alignment of these image stacks for volume reconstruction and data visualization and analysis in 3D. The system is designed to work in multiprocessor environments and can utilize an arbitrary number of processors, cutting down significantly the turnaround time and allowing users to quickly process sets of hundreds of high resolution images using a combination of automated and interactive tools. Additional modules are used to reconstruct the shape of the original subject. Our system is interactive, fully scalable and can be applied to any photographic sliced dataset, regardless of subject and reduces significantly the processing time for stack alignment.
- SourceAvailable from: fitrieamelia.comACM SIGGRAPH Computer Graphics. 01/1987; 21:163--169.
- [show abstract] [hide abstract]
ABSTRACT: A broad understanding of the relationship between gene activation, pattern formation and morphogenesis will require adequate tools for three-dimensional and, perhaps four-dimensional, representation and analysis of molecular developmental processes. We present a novel, computer-based method for the 3D visualization of embryonic gene expression and morphological structures from serial sections. The information from these automatically aligned 3D reconstructions exceeds that from single-section and whole-mount visualizations of in situ hybridizations. In addition, these 3D models of gene-expression patterns can become a central component of a future developmental database designed for the collection and presentation of digitized, morphological and gene-expression data. This work is accompanied by a web site (http://www.univie.ac.at/GeneEMAC).Nature Genetics 07/2000; 25(2):147-52. · 35.21 Impact Factor
Fast Interactive Integration of Cross-
Sectional Image Datasets and Surface Data
for Morphometric Analysis
Oscar MERUVIA-PASTOR1a, Jung SOHa, Mei XIAOa, Eric SCHMIDTb,
Cairine LOGANb, Julia C. BOUGHNERb, Nicholas JONESb, David OSBORNa,
Johanna SANTIAGO a, Julian GITTLEMANb,
Benedikt HALLGRÍMSSONb, Christoph W. SENSENa
aSun Center of Excellence for Visual Genomics, Department of Biochemistry and
Molecular Biology, Faculty of Medicine, University of Calgary
bMorphometrics Laboratory, Department of Cell Biology & Anatomy
Faculty of Medicine, University of Calgary
Abstract. To investigate external facial morphology and cell proliferation patterns
and their relationship with cleft lip malformation in mice, we need to compare
samples of mice tissue photographs and surface reconstructions from micro-CT
scans obtained from mouse embryos. Tissue samples obtained through digital
photography are typically misaligned with respect to each other, which prevents
further analysis. We have developed a system for fast interactive alignment of
these image stacks for volume reconstruction and data visualization and analysis in
3D. The system is designed to work in multiprocessor environments and can
utilize an arbitrary number of processors, cutting down significantly the
turnaround time and allowing users to quickly process sets of hundreds of high
resolution images using a combination of automated and interactive tools.
Additional modules are used to reconstruct the shape of the original subject. Our
system is interactive, fully scalable and can be applied to any photographic sliced
dataset, regardless of subject and reduces significantly the processing time for
Keywords. Image Registration, Histology, Parallel Processing, 3D Surface
Reconstruction, Mouse Development
Identifying cell proliferation changes within and across subjects at different stages of
growth is important in morphometrics research. In particular, we want to model the
relationship between cell proliferation patterns, external facial morphology and cleft lip
malformation in mice. To investigate this relationship, various types of data sets are
generated for each mouse embryo specimen. A 3D representation of the surface of a
subject is obtained by processing MRI, CT or micro-CT scans (see Figure 1).
1 Corresponding Author: Dr. Oscar Meruvia-Pastor (email@example.com)
Figure 1: Surface models obtained from micro-CT scans of several mouse embryos before they were
sectioned in thin slices. The surface models were cropped to show only the upper body, since we are
interested exclusively in studying the region that contains the facial features.
Micro-CT scan data cannot be used to identify cell proliferation patterns, because
the images resulting from CT scans do not reveal any differences between particular
cell types. In order to identify cell proliferation patterns, particular cells are localized
using antibody binding, and antibody-antigen interactions. These cells become visible
only after slicing and digitally photographing samples from the subjects. In this process,
technicians tend to spend several hours taking photographs using different illumination
setups, mainly white- versus blue- or UV light, which highlights the different cell types.
Allowing technicians to photograph each sample disregarding the alignment of the
samples and to take batches of samples using different lighting conditions in separate
batches would save several hours of work per specimen. Because each picture is taken
individually, the resulting images are misaligned and can not be directly compared
with each other before proper alignment. The disarray also prevents direct analysis of
this data with respect to the surface model previously obtained through micro-CT scans.
For this reason, a fast alignment technique (registration) is needed to convert data
taken from photographs of tissue slices to an aligned volumetric dataset that can be
then mapped to a 3D surface model.
2. Tools and Methods
We have created a parallelized, multiprocessor-oriented alignment system that uses the
Insight segmentation and registration toolkit (ITK) . Since registration algorithms do
not guarantee successful alignment in all cases (e.g. [2,6,7]), we also provide users with
an interactive alignment tool (AlignAStack & AlignAPair) that complements the
automatic registration tool and is used to correct exceptional cases of misalignment.
Finally, a 3D model alignment tool (AlignAModel) is used to find a mapping from the
surface model generated with micro-CT scan data to the model originated through
surface reconstruction from the stack of photographs.
2.1. Automatic Registration
Since each image in the stack is a digital photograph of a slice of tissue that has been
manually placed in the microscope and photographed disregarding the orientation of
previously photographed slices, the user ends up with a sequence of images which
correspond to the original specimen, but which are all misaligned with respect to each
other and cannot be directly mapped to the original 3D surface model obtained from the
Following a sequence of steps using our system, users can automatically
manipulate high-resolution image sets, which in the case of a mouse embryos’ upper
body typically contain 70-150 images each. The first step is to do binary segmentation
of the individual images. The second step is the automatic alignment of image pairs.
The last step (consolidation) produces the aligned stack of images. Each step of the
registration (segmentation, pair alignment and/or stack alignment) can de done in
parallel. A process scheduler can spawn an arbitrary number of independent processes
that act concurrently on the stack. Our solution therefore allows a multiprocessing
machine or cluster of processors to act on the individual images of the stack
concurrently, thus significantly reducing the time spent to align the whole stack.
2.1.1. Parallel Registration of a Stack of Images
When aligning a stack of N misaligned sequence of images (indexed I1 to IN), one safe
approach would be to take the first two images in the stack, align them by having the
first image fixed and the second image moved to coincide with the first, and then move
on to take the second and third images, have the second image (already aligned with
image I1) be the fixed image, move the third image to be aligned with the second and
move on to the next pair, and so on until the whole stack of images has been processed.
This solution would have the stack aligned in linear time, depending on N. From a
computational point of view, the process of aligning a pair of images Ii-1, Ii using the
sequential algorithm described before, is the equivalent of generating a 2D
transformation Ti (represented as a 3×3 matrix in homogeneous coordinates), that could
be applied on image Ii to bring it into alignment with image Ii-1.
126.96.36.199. Generating the Alignment Transformations
To align the stack of images in parallel, a scheduler provides each processor with a
single pair of sequential images (e.g.: Ii-1 and Ii), that the processor takes as input to
generate the transformation Ti (i∈[2..N]). This transformation can clearly be computed
by a processor with disregard of other processors working on other pairs of images that
are also part of the same stack. Once all the transformations T2 to TN have been
computed we can move to the consolidation stage. Note that at this stage, no single
image has been actually aligned, i.e. we defer the application of the transformations on
the images to the next stage.
188.8.131.52. Consolidating the Registration Process
In the consolidation stage, which is also executed in parallel, we apply a transformation
Tfinal to each image Ii such that it is aligned with respect to the first image in the stack
(which is the reference image):
Ii_aligned = Tfinal·Ii
For an arbitrary image in the stack Ii, the transformation that will bring it into
alignment with the first image in the stack is the composition of the transformation that
would have been necessary to bring image Ii in alignment with its predecessor Ii-1 (Ti),
with the transformations that would have aligned the predecessor with its own
predecessor and so on, up to the first image I1:
Tfinal = T2·T3·T4· … ·Ti
In the previous stage we computed each transformation from T2 to TN. In the next
stage, each processor is provided with an image Ii and the set of transformations T2..i;
then, the processor computes the transformation Tfinal for each image and applies this
transformation to the image.
A job scheduler provides the images and the transformations to a set of processors
until all images in the stack have been processed. The key to this method is to defer the
actual alignment of the images (application of the transformations) until all
transformations T2 to TN are computed and Tfinal can be found for each image.
2.2. AlignAPair and AlignAStack
We have developed an interactive tool, AlignAPair/AlignAStack (see Figure 2), to
assist the user in fixing the particular cases where the automatic alignment failed,
which typically would happen when the automatic alignment’s optimization algorithm
gets stuck in a local minimum while searching for solutions. After a user performs the
manual alignment, an additional pass of automatic registration can be applied to
improve the alignment results, since the automatic registration tools are particularly
effective for fine tuning.
Figure 2: Screenshots of AlignAStack. Mouse embryos are sliced and photographed to identify cell
proliferation regions close to the embryo’s face. During this process, the images produced are misaligned
with respect to each other. Parallel registration aligns most of the images in the stack, but for exceptional
cases where parallel registration fails, we use AlignAStack.
AlignAStack handles images in the stack as if they were tightly coupled to each
other. When a user performs a single alignment operation using AlignAStack, all
images that are subsequent to the reoriented or moved image are affected by the user’s
operation as if they were stuck to the image originally displaced. This prevents
individual user manipulations from creating misalignments in the images that are
adjacent to the images being fixed. This approach is useful in our system because
automatic alignment precedes manual alignment and several sections of the stack of
images have already been aligned, so the user can focus on solving exclusively those
cases where automatic alignment was not successful. AlignAPair is a simplified
version of AlignAStack, where the user can align two images, for instance, one of
which is the image of a tissue sample illuminated with white light while the other is the
image of the same sample illuminated with UV light.
2.3. Surface Reconstruction
Figure 3: Once the stack of images has been aligned, we use a surface reconstruction algorithm (marching
cubes) to recreate the surface of the model generated from the tissue slices taken from the subject.
To find the mapping between the information contained in the slice data and the
morphometric data from the micro-CT scans, a 3D surface model is built from the stack
of aligned images (See Figure 3). For this purpose, we have implemented a version of
marching cubes  by Bourke2, but this can also be done using other algorithms or
libraries such as VTK ; an extensive survey of 3D surface reconstruction algorithms
is available from Newman and Yia . Figure 3 shows samples of the surfaces
reconstructed from 3D models.
Figure 4: To integrate the histological data with the morphometric data in the same 3D space, the surface
model from the slice data (Figure 3) is aligned with the model obtained from the micro-CT scan (Fig. 1)
Since the micro-CT model and the slice-based model were generated with different
imaging procedures, we need to find a 3D affine transformation that binds both 3D
models. To do this, we have created AlignAModel (Figure 4), an interactive program
that receives the 3D models and lets the user change the translation, orientation and
scale of one of the models to have it match with the other. AlignAModel provides
automatic centering and automated rescaling and supports different rendering styles,
using blended transparency, stencil buffers to highlight differences, and checkerboard
overlays. The output of AlignAModel is a 3D transformation (represented as a 4×4
matrix in homogeneous coordinates) used to map data from the image space of the slice
data to the 3D space of the micro-CT surface model.
2 Available at: http://local.wasp.uwa.edu.au/~pbourke/geometry/polygonise/
We have used the tools and methods presented to map the position of cells in the
histological samples to the surface model obtained from the micro-CT scan, in an
attempt to conduct morphometric analysis of the cleft lip malformation during mouse
development. We have evaluated the efficiency of the parallel registration procedure
with several datasets. With approximately 15 processes working concurrently in a
multiuser environment, a stack of 73 high-resolution (2 Megapixels) images is
automatically aligned in about 20 minutes, whereas the sequential process for this takes
about 5 hours in the same setting. The fundamentals of our multiprocessor supported
registration method are solid, generic, and can be used to speed-up existing registration
systems, such as  and . In addition, the system is fully scalable to an arbitrary
number of processors: in a multiprocessor or cluster environment with a large number
of processors, each processor could tackle one image pair from the stack and the whole
set could be automatically aligned in about the same time taken to align a single pair.
The methods described here are generic and can be applied to a variety of experimental
setups where sectioning and 3D reconstruction are needed, (e.g. [2,6,7]). Additional
modules and tools are being designed within the Cell Proliferation Visualization project
to produce analysis tools for the integration of data from several specimens. In the
future, we plan to enhance our system by implementing an algorithm that distributes
the alignment process for a single pair of images over several processors. This work
has been supported by Genome Canada through Genome Alberta; the NRC of
Canada’s IRA Program; Alberta Science and Research Authority; Western Economic
Diversification; the Governments of Canada and of Alberta through the WE
Partnership Agreement; the iCORE/Sun Microsystems IR Chair program; the Alberta
Network for Proteomics Innovation; and the Canada Foundation for Innovation.
 Yoo, T (ed.) Insight into Images “Principles and Practice for Segmentation, Registration, and Image
Analysis” ISBN:1568812175 A.K.Peters, 2004.
 Karen P.; Jirkovská M.; Tomori Z.; Demjénová E.; Janácek J.; Kubínová L. Three-dimensional
computer reconstruction of large tissue volumes based on composing series of high-resolution confocal
images by GlueMRC and LinkMRC software. Microscopy Research and Technique Volume 62 Issue 5,
pp 415 – 422, Wiley, 2003 http://dx.doi.org/10.1002/jemt.10405
 Lorensen, W. E.; Cline, H. E. Marching Cubes: A High Resolution 3D Surface Construction Algorithm.
SIGGRAPH Computer. Graphics. v.21 n.4, p.163-169, July 1987.
 Newman, T. S. and Yia, H. “A Survey of the Marching Cubes Algorithm.” Computers & Graphics
Volume 30, Issue 5, pp 854-879, October 2006. http://dx.doi.org/10.1016/j.cag.2006.07.021
 Schroeder, W.; Martin, K.; Lorensen, B. The Visualization Toolkit, An Object-Oriented Approach To
3D Graphics, 4th edition, ISBN-10: 193093419X, Kitware, Inc. 2006.
 Streicher, J.; Weninger, W. J.; Müller, G. B. External marker-based automatic congruencing: A new
method of 3D reconstruction from serial sections. The Anatomical Record Volume 248, No 4, pp 583-
602, 1997. Wiley-Liss, Inc. http://dx.doi.org/10.1002/(SICI)1097-0185(199708)248:4<583::AID-AR10>3.0.CO;2-L
 Streicher, J.; Donat, M. A.; Strauss, B.; Sporle, R.; Schughart, K.; Muller, G. B. Computer-based Three-
Dimensional Visualization of Developmental Gene Expression. Nature Genetics Volume 25, Issue 2,
pp.147 – 152, June 2000. http://dx.doi.org/10.1038/75989