# Image segmentation based on Bayesian network-Markov random field model and its application to in vivo plaque composition.

**Abstract**

Combining Bayesian network (BN) and Markov random field (MRF) models, this paper presents an effective supervised image segmentation algorithm. Representing information from different features, a Bayesian network generates the probability map for each pixel via the conditional PDF (probability density function) learned from a limited training data set. Considering the spatial relation and a priori knowledge of the image, MRF theory is used to generate a reasonable segmentation by minimizing the proposed energy functional. Applying this algorithm to multi-contrast MR image in vivo plaque composition measurement shows comparable results with expert manual segmentation

IMAGE SEGMENTATION BASED ON BAYESIAN NETWORK-MARKOV RANDOM FIELD

MODEL AND ITS APPLICATION TO IN VIVO PLAQUE COMPOSITION

Fei Liu

*

, PhD; Dongxiang Xu, PhD; Chun Yuan, PhD; William Kerwin, PhD

*

Email: feil@u.washington.edu

Department of Radiology, University of Washington, Seattle, WA

ABSTRACT

Combining Bayesian network (BN) and Markov Random

Field (MRF) models, this paper presents an effective

supervised image segmentation algorithm. Representing

information from different features, a Bayesian network

generates the probability map for each pixel via the

conditional PDF (probability density function) learned from

a limited training data set. Considering the spatial relation

and a priori knowledge of the image, MRF theory is used to

generate a reasonable segmentation by minimizing the

proposed energy functional. Applying this algorithm to

multi-contrast MR image based in vivo plaque composition

measurement shows comparable results with expert manual

segmentation.

1. INTRODUCTION

Classification of a medical image is to segment component

tissues based upon unique attributes of those tissues. In

order to get accurate classification, it is beneficial to include

information from different aspects together. Thus, to

analyze the relationships among all the valuable features,

from a statistical point of view, by Bayesian theory, we need

to know the joint distribution of these features. But due to

the limited training dataset, memory requirement and

calculation complexity, it is unrealistic to estimate this joint

distribution directly. Furthermore, other than the relation

among features associated to each pixel, in order to get a

reasonable segmentation result, the spatial relation between

pixels also need to be considered. This paper will provide

an effective framework to handle these issues from both

theory and application aspects.

2. RELATED WORKS

2.1. Bayesian Network

A Bayesian network for a set of variables

^

`

Vv

consists

of a network structure which is a directed acyclic graph

(DAG) encoding a set of conditional independence

assertions about variables in and a set of local probability

distributions associated with each variable [1]. It is a

powerful tool for knowledge representation and inference

under conditions of uncertainty. From the existing training

data, it is capable to qualitatively learn the reasoning

structure which shows the logical relation among the

features. It can also quantitatively learn the conditional

probability between the related variables. It was not

considered as a classifier until the discovery that naïve-

Bayesian networks, a very simple kind of BN that assumes

the attributes are independent given the class node, is

surprisingly effective [2]. Based on conditional

independence information, the joint probability in a BN

could be factorized as the product of conditional

probabilities. This will reduce the requirement on the size of

the training dataset. From an observation of

V

which may

include the features such as intensity and texture, the

probability to be classified as specified class

C

,

Pr |CV

could be inferred from the learned BN.

2.2. Markov Random Field

A natural way to incorporate spatial correlations into a

segmentation process is to use MRF as a prior model [3][4].

An MRF assumes that the information contained in the local

structure of images is sufficient to describe the global

image. This hypothesis provides a convenient and consistent

way for modeling observed images with a priori knowledge

about the restrictions imposed on the simultaneous labeling

of connected neighboring units. It takes into account the

spatial dependencies in the image through the conditional

probability that a pixel belongs to a certain class given the

classes of its neighbors. Through this model the

segmentation problem is converted into a MAP (Maximum

A-Posteriori) problem.

3. PROPOSED SUPERVISED SEGMENTATION

METHOD

As a supervised image segmentation algorithm, the

proposed algorithm includes two parts – probability map

generation based on a BN and MAP estimation based on a

MRF. In the first part, combining all valuable information,

each pixel will be individually assigned a probability value

to be each given class. By generating such a probability

1410-7803-9577-8/06/$20.00 ©2006 IEEE ISBI 2006

map, BN provides a mechanism to convert the problem

from feature space to image domain.

Figure 1. Proposed image segmentation model

Although assigning the pixel in the probability map to the

highest probability class presents a reasonable classification

of pixels, due to noise and artifact in the image, the

classification result may not be a reasonable segmentation.

Thus, in the second part, considering the a priori knowledge

on the image model and the spatial relation between

individual pixels, the MRF based model will be taken into

account to generate a reasonable segmentation.

Specifically, given a decomposition of the image domain

1

N

i

i

R

R

*

, and region

i

R

is labeled as class

i

L

R , the total

probability by the segmentation

^

`

i

R

is

1

,

i

i

N

LR

i

R

ER P xydxdy

¦

³³

(1)

where

,

j

Pxyrepresents the probability at pixel

,

x

y to be

class

1, ,

j

M "

. Obviously, without any constraints, for

the given estimation of

P

,

E

R

could get the maximum

value as

arg max ,

i

i

E P x y dxdy

:

³³

(2)

But because of noise and some other artifacts, in order to

get a reasonable segmentation, some regulators from a priori

knowledge are necessary. One commonly used regulator is

the constraint on total boundary length[5][6]. Considering

this, in this paper we will design the energy model as

11

,

i

i

NN

i

LR

ij

R

E R g P x y dxdy R

O

*

¦¦

³³

(3)

where

g

is a monotonically decreasing function which

converts the maximizing functional problem into searching

for a minimum. Based on Gibbs-Markov random field

theory[3][5], we will use

g

in the form of

log

g

xx

.

i

R

*

indicates the boundary length of region

i

R

,

O

is a

positive value to control the trade off between boundary

smoothness and total probability.

To minimize the energy in (3), in this paper, we will choose

a level set based active contour model. The active contour

model realizes a closed binary boundary and the level set

method easily handles topological changes.

M

level set

functions will be used to represent

M

classes of regions.

The energy functional is designed as

1

1

1

2

2

1

1

2

M

ii

i

M

i

i

M

i

i

E g P H dxdy

Hdxdy

H

dxdy

O

O

:

:

:

) )

)

§·

)

¨¸

©¹

¦

³³

¦

³³

¦

³³

(4)

where

i

)

representing one region as

,0

i

xy)

iff

,

x

y is enclosed by the contour of zero level set

,0

i

xy)

, and

H )

is the Heaviside function. The

second term represents the total boundary length and the

third one guarantees each pixel belongs to one and only one

class.

1

O

and

2

O

are positive weighting constants.

At the beginning, pixels are labeled to the highest

probability class. And

i

) is initialized by the

corresponding labeled regions. Then,

i

E ) is minimized

in a deepest decreasing way until convergence by

1

2

1

1

i

i

i

i

i

M

i

i

gP div

t

H

O

G

O

§·

§·

)

¨¸

¨¸

¨¸

)

¨¸

w)

©¹

)

¨¸

w

§·

¨¸

)

¨¸

¨¸

¨¸

¨¸

©¹

©¹

¦

(5)

where

i

i

i

H

G

w)

)

w)

.

4. APPLICATION TO IN VIVO PLAQUE

COMPOSITION

MRI is a promising noninvasive technique for

characterizing atherosclerotic plaque composition in vivo,

with an end-goal of assessing plaque vulnerability. Because

of limitations arising from acquisition time, achievable

resolution, contrast-to-noise ratio, patient motion, and the

effects of blood flow, automatically identifying plaque

composition remains a challenging task in vivo. Numerous

studies have shown that MRI exhibits high contrast for

internal plaque features, but that combined information from

multiple MRI contrast weightings is critical for

distinguishing all plaque components [7]. Replacing

subjective, manual segmentation with an automated

segmentation alternative would have several benefits. Aside

Feature

Space

Image

Domain

Segme-

ntation

Bayesian

Network

MRF

Model

Feature

Relation

Pixel

Relation

142

from saving time in image review, automated segmentation

would reduce the considerable amount of training required

to read these images and the corresponding inter-rater

variability.

4.1. Materials

MR images from 26 patients scheduled for carotid

endarterectomy (CEA) were obtained on a GE Signa 1.5T

MR scanner. Following CEA surgery, the excised carotid

plaque was histologically analyzed to identify plaque

components including necrotic core, calcification, loose

matrix, and fibrous tissue. In total, 142 image locations

were identified with corresponding histology. At each

location, five images were obtained with T1, T2, PD (proton

density), TOF (time-of-flight) and CET1 (contrast-

enhanced) weightings. In-plane resolution was 0.63 mm,

resulting in a pixel size of 0.31 mm after zero-filled

interpolation, and the slice thickness was 2.0mm.

To generate ground truth for training and evaluation of the

segmentation algorithm, images from all 26 subjects were

manually segmented based on established MRI criteria [8]

and knowledge of the histological results. The review was

conducted by an expert radiologist (BC) in conjunction with

the histologist (MSF). As in histology, three types of tissue

– calcification, necrotic core and loose matrix – were

identified and the remainder was assigned to fibrous tissue.

Images from the remaining contrast weightings were

aligned with the drawn contours with the aid of an

automated registration algorithm [9].

From this data set, 14 subjects (84 locations) were assigned

to a training set and 12 subjects (58 locations) were

assigned to a testing data set. The average size of each

tissue type was confirmed to be similar between training

and testing groups.

4.2. Methods

In an initial preprocessing step, coil correction removes the

inhomogeneity caused by the coil and MR images are

normalized by dividing the median intensity of the 4cm x

4cm region of interest aligned with the center of the wall

contour.

In previous work [10][11] intensity from multi-contrast MR

was the only feature used in plaque segmentation. Here, we

assume for the given tissue that the intensity distribution is

conditionally independent of the position in the plaque,

yielding the naïve-Bayesian structure in Fig 2. For each

pixel the observed morphology measurement

,dt m

(distance to lumen and thickness of the plaque at the pixel),

and intensity vector

x

from multi-contrast MR, yields the

probability for this pixel to be tissue

i

T . This leads to the

deduced BN as

4

1

||Pr

Pr | ,

||Pr

iii

i

jjj

j

pTp T T

T

pTp T T

¦

xm

mx

xm

(6)

where the conditional PDF (probability density function) is

estimated by the Parzen window method from the training

set. The proposed segmentation method in plaque

composition is named MEPPS (Morphology Enhanced

Probabilistic Plaque Segmentation).

Figure 2. Naïve-Bayesian used in the application

4.3. Results

Fig. 3 shows a segmentation example based on the proposed

segmentation framework. Results with and without the

additional morphological information are displayed, as are

the histology-guided manual drawings, and the

corresponding histology section. To visualize the pixel-wise

probabilities in Fig. 3 (a) and (b), each pixel has been color

coded to indicate the tissue with the highest probability. The

intensity represents the difference between the highest and

second highest probabilities, essentially providing a

confidence metric in the classification. Also shown are the

final contours delineating the tissue regions.

From this

example, two aspects are apparent. First, the benefits of

morphological information are apparent by noting that

without morphological information, the necrotic core is

incorrectly divided into two disjoint regions and a region of

calcification is incorrectly identified in a thin-walled region

near the left of the image. Second, the use of active contour

methods to delineate the final regions has successfully

overcome the presence of “holes” in the probability map

that might otherwise have been misclassified.

Table 1. Correlations (R) of histology-guided measurements

(total area per location) with manual segmentation, intensity-

based automatic segmentation, and MEPPS.

Tissue Manual Intensity-based MEPPS

Necrotic Core 0.84 0.78 0.88

Calcification 0.87 0.88 0.91

Loose Matrix 0.57 0.57 0.64

Fibrous Tissue 0.88 0.83 0.91

T

i

x m

143

(a) (b)

(c) (d)

Figure 3. Segmentation results showing: (a) intensity based

probability map and segmentation (b) MEPPS segmentation

(c) histology confirmed manual segmentation on T1 image

(d) segmentation on histological image. Comparing with

intensity based method, MEPPS accurately detects more

calcification regions and shows more reasonable shaped

necrotic core.

For overall validation, we examined the correlation between

automatic and histology-guided manual segmentation. The

areas of each tissue type in each of the 58 test locations

were used for comparison and the results are compiled in

Table 1. The benefits of using morphology in addition to

intensity are apparent, given the higher correlations for

MEPPS compared to the results based on intensity alone.

5. CONCLUSION

Based on BN and MRF theory, this paper presented a

supervised image segmentation framework. BN is capable

of combining available features for each individual pixel

and mapping from feature space into a probability map.

Then, considering prior knowledge of the image model and

the spatial relation between individual pixels, an MRF based

model is used to generate reasonable segmentation.

Applying this framework to the in vivo plaque segmentation

problem by combining multi-contrast intensity and

morphology information demonstrates that reliable,

automated, in vivo segmentation of carotid plaque

components is possible and quantitatively comparable to

manual results. Combining morphological information with

intensity to make the final decision proves to increase the

accuracy of the segmentation result.

6. REFERENCES

[1] Hecherman D. A tutorial on learning with Bayesian

networks. technical report, MSR-TR-95-06.

[2] Cheng J, Greiner R. Learning Bayesian belief network

classifiers: algorithms and system. proceedings of the

fourteenth canadian conference on artificial

intelligence. 2001;141–151.

[3]

Geman S, Geman D. Stochastic Relaxation, Gibbs

Distributions, and the Bayesian Restoration of Images.

IEEE Trans. Pattern Anal. Machine Intell.1984; 6: 721-

741.

[4]

Derin H, Elliot H. Modeling and segmentation of noisy

and textured images using Gibbs random field. IEEE

Tran. Pattern Anal. Mach. Intel. 1987;9: 39-55.

[5]

Paragios N, Deriche R. Coupled Geodesic Active

Regions for Image Segmentation: A Level Set

Approach. ECCV 2000; 224-240.

[6] Chan TF, Vese LA. Active contours without edges.

IEEE T Image Process 2001;10:266-277.

[7] Yuan C, Kerwin WS. MRI of atherosclerosis. Journal

of Magnetic Resonance Imaging 2004; 19:710-719.

[8] Saam T, Ferguson MS, Yarnykh VL, Takaya N, Xu D,

Polissar NL, Hatsukami TS, Yuan C. Quantitative

evaluation of carotid plaque composition by in vivo

MRI. Arterioscler Throm Vas 2005;25:234-239.

[9]

Kerwin WS, Yuan C. Active edge maps for medical

image registration. Proceedings of SPIE

2001;4322:516-526.

[10] Clarke SE, Hammond RR, Mitchell JR, Rutt BK.

Quantitative assessment of carotid plaque composition

using multi-contrast MRI and registered histology.

Magnet Reson Med 2003;50:1199-1208.

[11]

Adame IM, Geest RJ, Wasserman BA, Mohamed MA,

Reiber JH, Lelieveldt BP. Automatic segmentation and

plaque characterization in atherosclerotic carotid artery

MR images. MAGMA. 2004;16:227-34.

144

- CitationsCitations10
- ReferencesReferences16

- "Their experimental results demonstrate the promising capability of the proposed BN model for both automatic image segmentation and effective interactive image segmentation. In [12] They presents a novel probabilistic unsupervised image segmentation framework called Irregular Tree- Structured Bayesian Networks (ITSBN).By integrating non-parametric density estimation technique with the traditional precision-recall framework, the method is more robust to boundary inconsistency due to human subjects. They experimentally show the improvement of ITSBN over the baseline method which motivates us to further investigate models of similar type. "

- "These predications are then fused with the BN prior model for scene segmentation. Liu et al. [24] combine BN and MRF to form an image segmentation approach. The BN generates a probability map for all pixels. "

[Show abstract] [Hide abstract]**ABSTRACT:**We propose a new Bayesian network (BN) model for both automatic and interactive image segmentation. A multilayer BN is constructed from an oversegmentation to model the statistical dependencies among superpixel regions, edge segments, vertices, and their measurements. The BN also incorporates various local constraints to further restrain the relationships among these image entities. Given the BN model and various image measurements, belief propagation is performed to update the probability of each node. Image segmentation is generated by the most probable explanation inference of the true states of both region and edge nodes from the updated BN. Besides the automatic image segmentation, the proposed model can also be used for interactive image segmentation. While existing interactive segmentation (IS) approaches often passively depend on the user to provide exact intervention, we propose a new active input selection approach to provide suggestions for the user's intervention. Such intervention can be conveniently incorporated into the BN model to perform actively IS. We evaluate the proposed model on both the Weizmann dataset and VOC2006 cow images. The results demonstrate that the BN model can be used for automatic segmentation, and more importantly, for actively IS. The experiments also show that the IS with active input selection can improve both the overall segmentation accuracy and efficiency over the IS with passive intervention.- "1) Learning Conditional Probability Distributions: Parameter learning in the proposed CG model consists of two parts. First, the log-likelihood can be maximized with respect to the CPT parameter , i.e., (9) where the constraint comes from the definition of a conditional probability. Maximizing (9) leads to the following analytical solution of the optimal parameters: "

[Show abstract] [Hide abstract]**ABSTRACT:**Chain graph (CG) is a hybrid probabilistic graphical model (PGM) capable of modeling heterogeneous relationships among random variables. So far, however, its application in image and video analysis is very limited due to lack of principled learning and inference methods for a CG of general topology. To overcome this limitation, we introduce methods to extend the conventional chain-like CG model to CG model with more general topology and the associated methods for learning and inference in such a general CG model. Specifically, we propose techniques to systematically construct a generally structured CG, to parameterize this model, to derive its joint probability distribution, to perform joint parameter learning, and to perform probabilistic inference in this model. To demonstrate the utility of such an extended CG, we apply it to two challenging image and video analysis problems: human activity recognition and image segmentation. The experimental results show improved performance of the extended CG model over the conventional directed or undirected PGMs. This study demonstrates the promise of the extended CG for effective modeling and inference of complex real-world problems.

Data provided are for informational purposes only. Although carefully collected, accuracy cannot be guaranteed. The impact factor represents a rough estimation of the journal's impact factor and does not reflect the actual current impact factor. Publisher conditions are provided by RoMEO. Differing provisions from the publisher's actual policy or licence agreement may be applicable.

This publication is from a journal that may support self archiving.

Learn more