Image segmentation based on Bayesian network-Markov random field model and its application to in vivo plaque composition.

Conference Paper (PDF Available) · January 2006with36 Reads
DOI: 10.1109/ISBI.2006.1624872 · Source: DBLP
Conference: Proceedings of the 2006 IEEE International Symposium on Biomedical Imaging: From Nano to Macro, Arlington, VA, USA, 6-9 April 2006
Abstract
Combining Bayesian network (BN) and Markov random field (MRF) models, this paper presents an effective supervised image segmentation algorithm. Representing information from different features, a Bayesian network generates the probability map for each pixel via the conditional PDF (probability density function) learned from a limited training data set. Considering the spatial relation and a priori knowledge of the image, MRF theory is used to generate a reasonable segmentation by minimizing the proposed energy functional. Applying this algorithm to multi-contrast MR image in vivo plaque composition measurement shows comparable results with expert manual segmentation
IMAGE SEGMENTATION BASED ON BAYESIAN NETWORK-MARKOV RANDOM FIELD
MODEL AND ITS APPLICATION TO IN VIVO PLAQUE COMPOSITION
Fei Liu
*
, PhD; Dongxiang Xu, PhD; Chun Yuan, PhD; William Kerwin, PhD
*
Email: feil@u.washington.edu
Department of Radiology, University of Washington, Seattle, WA
ABSTRACT
Combining Bayesian network (BN) and Markov Random
Field (MRF) models, this paper presents an effective
supervised image segmentation algorithm. Representing
information from different features, a Bayesian network
generates the probability map for each pixel via the
conditional PDF (probability density function) learned from
a limited training data set. Considering the spatial relation
and a priori knowledge of the image, MRF theory is used to
generate a reasonable segmentation by minimizing the
proposed energy functional. Applying this algorithm to
multi-contrast MR image based in vivo plaque composition
measurement shows comparable results with expert manual
segmentation.
1. INTRODUCTION
Classification of a medical image is to segment component
tissues based upon unique attributes of those tissues. In
order to get accurate classification, it is beneficial to include
information from different aspects together. Thus, to
analyze the relationships among all the valuable features,
from a statistical point of view, by Bayesian theory, we need
to know the joint distribution of these features. But due to
the limited training dataset, memory requirement and
calculation complexity, it is unrealistic to estimate this joint
distribution directly. Furthermore, other than the relation
among features associated to each pixel, in order to get a
reasonable segmentation result, the spatial relation between
pixels also need to be considered. This paper will provide
an effective framework to handle these issues from both
theory and application aspects.
2. RELATED WORKS
2.1. Bayesian Network
A Bayesian network for a set of variables
^
`
Vv
consists
of a network structure which is a directed acyclic graph
(DAG) encoding a set of conditional independence
assertions about variables in and a set of local probability
distributions associated with each variable [1]. It is a
powerful tool for knowledge representation and inference
under conditions of uncertainty. From the existing training
data, it is capable to qualitatively learn the reasoning
structure which shows the logical relation among the
features. It can also quantitatively learn the conditional
probability between the related variables. It was not
considered as a classifier until the discovery that naïve-
Bayesian networks, a very simple kind of BN that assumes
the attributes are independent given the class node, is
surprisingly effective [2]. Based on conditional
independence information, the joint probability in a BN
could be factorized as the product of conditional
probabilities. This will reduce the requirement on the size of
the training dataset. From an observation of
V
which may
include the features such as intensity and texture, the
probability to be classified as specified class
C
,
Pr |CV
could be inferred from the learned BN.
2.2. Markov Random Field
A natural way to incorporate spatial correlations into a
segmentation process is to use MRF as a prior model [3][4].
An MRF assumes that the information contained in the local
structure of images is sufficient to describe the global
image. This hypothesis provides a convenient and consistent
way for modeling observed images with a priori knowledge
about the restrictions imposed on the simultaneous labeling
of connected neighboring units. It takes into account the
spatial dependencies in the image through the conditional
probability that a pixel belongs to a certain class given the
classes of its neighbors. Through this model the
segmentation problem is converted into a MAP (Maximum
A-Posteriori) problem.
3. PROPOSED SUPERVISED SEGMENTATION
METHOD
As a supervised image segmentation algorithm, the
proposed algorithm includes two parts – probability map
generation based on a BN and MAP estimation based on a
MRF. In the first part, combining all valuable information,
each pixel will be individually assigned a probability value
to be each given class. By generating such a probability
1410-7803-9577-8/06/$20.00 ©2006 IEEE ISBI 2006
map, BN provides a mechanism to convert the problem
from feature space to image domain.
Figure 1. Proposed image segmentation model
Although assigning the pixel in the probability map to the
highest probability class presents a reasonable classification
of pixels, due to noise and artifact in the image, the
classification result may not be a reasonable segmentation.
Thus, in the second part, considering the a priori knowledge
on the image model and the spatial relation between
individual pixels, the MRF based model will be taken into
account to generate a reasonable segmentation.
Specifically, given a decomposition of the image domain
1
N
i
i
R
R
*
, and region
i
R
is labeled as class
i
L
R , the total
probability by the segmentation
^
`
i
R
is



1
,
i
i
N
LR
i
R
ER P xydxdy
¦
³³
(1)
where
,
j
Pxyrepresents the probability at pixel
,
y to be
class
1, ,
j
M "
. Obviously, without any constraints, for
the given estimation of
P
,
E
R
could get the maximum
value as
arg max ,
i
i
E P x y dxdy
:
³³
(2)
But because of noise and some other artifacts, in order to
get a reasonable segmentation, some regulators from a priori
knowledge are necessary. One commonly used regulator is
the constraint on total boundary length[5][6]. Considering
this, in this paper we will design the energy model as





11
,
i
i
NN
i
LR
ij
R
E R g P x y dxdy R
O
*
¦¦
³³
(3)
where
g
is a monotonically decreasing function which
converts the maximizing functional problem into searching
for a minimum. Based on Gibbs-Markov random field
theory[3][5], we will use
g
in the form of
log
g
xx
.
i
R
*
indicates the boundary length of region
i
R
,
O
is a
positive value to control the trade off between boundary
smoothness and total probability.
To minimize the energy in (3), in this paper, we will choose
a level set based active contour model. The active contour
model realizes a closed binary boundary and the level set
method easily handles topological changes.
M
level set
functions will be used to represent
M
classes of regions.
The energy functional is designed as
 


1
1
1
2
2
1
1
2
M
ii
i
M
i
i
M
i
i
E g P H dxdy
Hdxdy
H
dxdy
O
O
:
:
:
) )
)
§·
)
¨¸
©¹
¦
³³
¦
³³
¦
³³
(4)
where
i
)
representing one region as
,0
i
xy)
iff
,
x
y is enclosed by the contour of zero level set
,0
i
xy)
, and
H )
is the Heaviside function. The
second term represents the total boundary length and the
third one guarantees each pixel belongs to one and only one
class.
1
O
and
2
O
are positive weighting constants.
At the beginning, pixels are labeled to the highest
probability class. And
i
) is initialized by the
corresponding labeled regions. Then,
i
E ) is minimized
in a deepest decreasing way until convergence by



1
2
1
1
i
i
i
i
i
M
i
i
gP div
t
H
O
G
O
§·
§·
)
¨¸
¨¸
¨¸
)
¨¸
w)
©¹
)
¨¸
w
§·
¨¸
)
¨¸
¨¸
¨¸
¨¸
©¹
©¹
¦
(5)
where

i
i
i
H
G
w)
)
w)
.
4. APPLICATION TO IN VIVO PLAQUE
COMPOSITION
MRI is a promising noninvasive technique for
characterizing atherosclerotic plaque composition in vivo,
with an end-goal of assessing plaque vulnerability. Because
of limitations arising from acquisition time, achievable
resolution, contrast-to-noise ratio, patient motion, and the
effects of blood flow, automatically identifying plaque
composition remains a challenging task in vivo. Numerous
studies have shown that MRI exhibits high contrast for
internal plaque features, but that combined information from
multiple MRI contrast weightings is critical for
distinguishing all plaque components [7]. Replacing
subjective, manual segmentation with an automated
segmentation alternative would have several benefits. Aside
Feature
Space
Image
Domain
Segme-
ntation
Bayesian
Network
MRF
Model
Feature
Relation
Pixel
Relation
142
from saving time in image review, automated segmentation
would reduce the considerable amount of training required
to read these images and the corresponding inter-rater
variability.
4.1. Materials
MR images from 26 patients scheduled for carotid
endarterectomy (CEA) were obtained on a GE Signa 1.5T
MR scanner. Following CEA surgery, the excised carotid
plaque was histologically analyzed to identify plaque
components including necrotic core, calcification, loose
matrix, and fibrous tissue. In total, 142 image locations
were identified with corresponding histology. At each
location, five images were obtained with T1, T2, PD (proton
density), TOF (time-of-flight) and CET1 (contrast-
enhanced) weightings. In-plane resolution was 0.63 mm,
resulting in a pixel size of 0.31 mm after zero-filled
interpolation, and the slice thickness was 2.0mm.
To generate ground truth for training and evaluation of the
segmentation algorithm, images from all 26 subjects were
manually segmented based on established MRI criteria [8]
and knowledge of the histological results. The review was
conducted by an expert radiologist (BC) in conjunction with
the histologist (MSF). As in histology, three types of tissue
– calcification, necrotic core and loose matrix – were
identified and the remainder was assigned to fibrous tissue.
Images from the remaining contrast weightings were
aligned with the drawn contours with the aid of an
automated registration algorithm [9].
From this data set, 14 subjects (84 locations) were assigned
to a training set and 12 subjects (58 locations) were
assigned to a testing data set. The average size of each
tissue type was confirmed to be similar between training
and testing groups.
4.2. Methods
In an initial preprocessing step, coil correction removes the
inhomogeneity caused by the coil and MR images are
normalized by dividing the median intensity of the 4cm x
4cm region of interest aligned with the center of the wall
contour.
In previous work [10][11] intensity from multi-contrast MR
was the only feature used in plaque segmentation. Here, we
assume for the given tissue that the intensity distribution is
conditionally independent of the position in the plaque,
yielding the naïve-Bayesian structure in Fig 2. For each
pixel the observed morphology measurement
,dt m
(distance to lumen and thickness of the plaque at the pixel),
and intensity vector
x
from multi-contrast MR, yields the
probability for this pixel to be tissue
i
T . This leads to the
deduced BN as




4
1
||Pr
Pr | ,
||Pr
iii
i
jjj
j
pTp T T
T
pTp T T
¦
xm
mx
xm
(6)
where the conditional PDF (probability density function) is
estimated by the Parzen window method from the training
set. The proposed segmentation method in plaque
composition is named MEPPS (Morphology Enhanced
Probabilistic Plaque Segmentation).
Figure 2. Naïve-Bayesian used in the application
4.3. Results
Fig. 3 shows a segmentation example based on the proposed
segmentation framework. Results with and without the
additional morphological information are displayed, as are
the histology-guided manual drawings, and the
corresponding histology section. To visualize the pixel-wise
probabilities in Fig. 3 (a) and (b), each pixel has been color
coded to indicate the tissue with the highest probability. The
intensity represents the difference between the highest and
second highest probabilities, essentially providing a
confidence metric in the classification. Also shown are the
final contours delineating the tissue regions.
From this
example, two aspects are apparent. First, the benefits of
morphological information are apparent by noting that
without morphological information, the necrotic core is
incorrectly divided into two disjoint regions and a region of
calcification is incorrectly identified in a thin-walled region
near the left of the image. Second, the use of active contour
methods to delineate the final regions has successfully
overcome the presence of “holes” in the probability map
that might otherwise have been misclassified.
Table 1. Correlations (R) of histology-guided measurements
(total area per location) with manual segmentation, intensity-
based automatic segmentation, and MEPPS.
Tissue Manual Intensity-based MEPPS
Necrotic Core 0.84 0.78 0.88
Calcification 0.87 0.88 0.91
Loose Matrix 0.57 0.57 0.64
Fibrous Tissue 0.88 0.83 0.91
T
i
x m
143
(a) (b)
(c) (d)
Figure 3. Segmentation results showing: (a) intensity based
probability map and segmentation (b) MEPPS segmentation
(c) histology confirmed manual segmentation on T1 image
(d) segmentation on histological image. Comparing with
intensity based method, MEPPS accurately detects more
calcification regions and shows more reasonable shaped
necrotic core.
For overall validation, we examined the correlation between
automatic and histology-guided manual segmentation. The
areas of each tissue type in each of the 58 test locations
were used for comparison and the results are compiled in
Table 1. The benefits of using morphology in addition to
intensity are apparent, given the higher correlations for
MEPPS compared to the results based on intensity alone.
5. CONCLUSION
Based on BN and MRF theory, this paper presented a
supervised image segmentation framework. BN is capable
of combining available features for each individual pixel
and mapping from feature space into a probability map.
Then, considering prior knowledge of the image model and
the spatial relation between individual pixels, an MRF based
model is used to generate reasonable segmentation.
Applying this framework to the in vivo plaque segmentation
problem by combining multi-contrast intensity and
morphology information demonstrates that reliable,
automated, in vivo segmentation of carotid plaque
components is possible and quantitatively comparable to
manual results. Combining morphological information with
intensity to make the final decision proves to increase the
accuracy of the segmentation result.
6. REFERENCES
[1] Hecherman D. A tutorial on learning with Bayesian
networks. technical report, MSR-TR-95-06.
[2] Cheng J, Greiner R. Learning Bayesian belief network
classifiers: algorithms and system. proceedings of the
fourteenth canadian conference on artificial
intelligence. 2001;141–151.
[3]
Geman S, Geman D. Stochastic Relaxation, Gibbs
Distributions, and the Bayesian Restoration of Images.
IEEE Trans. Pattern Anal. Machine Intell.1984; 6: 721-
741.
[4]
Derin H, Elliot H. Modeling and segmentation of noisy
and textured images using Gibbs random field. IEEE
Tran. Pattern Anal. Mach. Intel. 1987;9: 39-55.
[5]
Paragios N, Deriche R. Coupled Geodesic Active
Regions for Image Segmentation: A Level Set
Approach. ECCV 2000; 224-240.
[6] Chan TF, Vese LA. Active contours without edges.
IEEE T Image Process 2001;10:266-277.
[7] Yuan C, Kerwin WS. MRI of atherosclerosis. Journal
of Magnetic Resonance Imaging 2004; 19:710-719.
[8] Saam T, Ferguson MS, Yarnykh VL, Takaya N, Xu D,
Polissar NL, Hatsukami TS, Yuan C. Quantitative
evaluation of carotid plaque composition by in vivo
MRI. Arterioscler Throm Vas 2005;25:234-239.
[9]
Kerwin WS, Yuan C. Active edge maps for medical
image registration. Proceedings of SPIE
2001;4322:516-526.
[10] Clarke SE, Hammond RR, Mitchell JR, Rutt BK.
Quantitative assessment of carotid plaque composition
using multi-contrast MRI and registered histology.
Magnet Reson Med 2003;50:1199-1208.
[11]
Adame IM, Geest RJ, Wasserman BA, Mohamed MA,
Reiber JH, Lelieveldt BP. Automatic segmentation and
plaque characterization in atherosclerotic carotid artery
MR images. MAGMA. 2004;16:227-34.
144
    • "Their experimental results demonstrate the promising capability of the proposed BN model for both automatic image segmentation and effective interactive image segmentation. In [12] They presents a novel probabilistic unsupervised image segmentation framework called Irregular Tree- Structured Bayesian Networks (ITSBN).By integrating non-parametric density estimation technique with the traditional precision-recall framework, the method is more robust to boundary inconsistency due to human subjects. They experimentally show the improvement of ITSBN over the baseline method which motivates us to further investigate models of similar type. "
    Full-text · Article · May 2012 · IEEE Transactions on Image Processing
    • "These predications are then fused with the BN prior model for scene segmentation. Liu et al. [24] combine BN and MRF to form an image segmentation approach. The BN generates a probability map for all pixels. "
    [Show abstract] [Hide abstract] ABSTRACT: We propose a new Bayesian network (BN) model for both automatic and interactive image segmentation. A multilayer BN is constructed from an oversegmentation to model the statistical dependencies among superpixel regions, edge segments, vertices, and their measurements. The BN also incorporates various local constraints to further restrain the relationships among these image entities. Given the BN model and various image measurements, belief propagation is performed to update the probability of each node. Image segmentation is generated by the most probable explanation inference of the true states of both region and edge nodes from the updated BN. Besides the automatic image segmentation, the proposed model can also be used for interactive image segmentation. While existing interactive segmentation (IS) approaches often passively depend on the user to provide exact intervention, we propose a new active input selection approach to provide suggestions for the user's intervention. Such intervention can be conveniently incorporated into the BN model to perform actively IS. We evaluate the proposed model on both the Weizmann dataset and VOC2006 cow images. The results demonstrate that the BN model can be used for automatic segmentation, and more importantly, for actively IS. The experiments also show that the IS with active input selection can improve both the overall segmentation accuracy and efficiency over the IS with passive intervention.
    Full-text · Article · Oct 2011
    • "1) Learning Conditional Probability Distributions: Parameter learning in the proposed CG model consists of two parts. First, the log-likelihood can be maximized with respect to the CPT parameter , i.e., (9) where the constraint comes from the definition of a conditional probability. Maximizing (9) leads to the following analytical solution of the optimal parameters: "
    [Show abstract] [Hide abstract] ABSTRACT: Chain graph (CG) is a hybrid probabilistic graphical model (PGM) capable of modeling heterogeneous relationships among random variables. So far, however, its application in image and video analysis is very limited due to lack of principled learning and inference methods for a CG of general topology. To overcome this limitation, we introduce methods to extend the conventional chain-like CG model to CG model with more general topology and the associated methods for learning and inference in such a general CG model. Specifically, we propose techniques to systematically construct a generally structured CG, to parameterize this model, to derive its joint probability distribution, to perform joint parameter learning, and to perform probabilistic inference in this model. To demonstrate the utility of such an extended CG, we apply it to two challenging image and video analysis problems: human activity recognition and image segmentation. The experimental results show improved performance of the extended CG model over the conventional directed or undirected PGMs. This study demonstrates the promise of the extended CG for effective modeling and inference of complex real-world problems.
    Full-text · Article · Oct 2011
Show more