Human Visual System Based Multi-Histogram Equalization for Non-Uniform Illumination and Shoadow Correction

Abstract

Images that do not have uniform brightness pose a challenging problem for image enhancement systems. As histogram equalization has been successfully used to correct for uniform brightness problems, we propose a new histogram equalization method that utilizes human visual system based thresholding as well as logarithmic processing techniques. Whereas previous histogram equalization methods have been limited in their ability to enhance these images, we demonstrate the effectiveness of this new method by enhancing a range of images with shadowing effects and inconsistent illumination. The images shown include images captured professionally and with cell phone cameras. Comparison with other methods are presented

Full text

Page 1

HUMAN VISUAL SYSTEM BASED MULTI-HISTOGRAM EQUALIZATION FOR NON-
UNIFORM ILLUMINATION AND SHADOW CORRECTION

Eric Whartona, Dr. Karen Panettaa, Dr. Sos Agaianb

aTufts University, Department of Electrical and Computer Engineering, Medford, MA 02155
bThe University of Texas at San Antonio, 6900 North Loop 1604 West, San Antonio, TX 78249

ABSTRACT

Images that do not have uniform brightness pose a
challenging problem for image enhancement systems. As
histogram equalization has been successfully used to correct
for uniform brightness problems, we propose a new
histogram equalization method that utilizes human visual
system based thresholding as well as logarithmic processing
techniques. Whereas previous histogram equalization
methods have been limited in their ability to enhance these
images, we will demonstrate the effectiveness of this new
method by enhancing a range of images with shadowing
effects and inconsistent illumination. The images shown
will include images captured professionally and with cell
phone cameras. Comparison with other methods will be
presented

Index Terms— Image Enhancement, Human Visual
System, Logarithmic Arithmetic, Histogram Equalization

1. INTRODUCTION

In this paper, we introduce the human visual system based
multi-histogram equalization method. This algorithm
capitalizes on the advantages
equalization with the benefit of an effective quantitative
measure to ensure optimal results as well as local histogram
equalization while removing useless information to avoid
the production of artifacts. We compare the results of the
proposed algorithm against the results of the leading
adaptive and local algorithms on a large number of images,
presenting a representative collection. This demonstrates
the effectiveness of the human vision based multi-histogram
equalization algorithm.
Histogram equalization and its variations are outlined in
[1][2][3][4][5]. These papers also identify deficiencies with
the algorithms, such as the introduction of artifacts or over-
enhancement of regions. Further, experimental results show
that these methods are unable to correct for non-uniform
illumination and shadows. Local, or adaptive, histogram
equalization divides the image into semi-overlapping blocks
and equalizes these blocks separately.
of multi-histogram
Multi-histogram equalization has generally been
limited to bi-histogram equalization, and previous results
have shown tri-histogram equalization to have no consistent
advantage. In this paper, we demonstrate a method that
overcomes this limitation by separating the image into
different regions of illumination instead of thresholding by
simple pixel intensity. In this manner, histogram
equalization can be used on each region to correct for non-
uniform illumination. In order to perform this segmentation,
we utilize the model of the human visual system.
This paper is organized as follows: Section 2 presents
background information. Section 3 will present the
methods. Section 4 will present the results of computer
simulations. Section 5 will be a discussion of results and
some concluding comments are made.

2. BACKGROUND

In this section, we provide a brief description of the
standard histogram equalization algorithm, the Logarithmic
Image Processing (LIP) model, and the Logarithmic AME
performance measure.

2.1. Histogram Equalization

Standard histogram equalization uses a cumulative density
function to attempt to force a uniform probability density
function for an image, remapping the pixel intensities
according to the following formula:
()(
maxmin
YYxf
−+=
Where f(x) is the pixel intensity of the output image, x is the
pixel intensity of the input image, Ymax and Ymin are the
desired maximum and minimum for the output range,
respectively, and P(x) is the cumulative density, where
P(Xmax) = 1.
This can then be modified into a local method.
Adaptive, or local, histogram equalization divides the image
into partially overlapping blocks which are then equalized
separately and averaged. Generally, using more windows
gives a better quality enhancement at the cost of
computational and time complexity.
This can also be modified into bi-histogram
equalization. This is done by splitting the image into two
)()
min
xPY
⋅

(1)

Page 2

(a)

(b)

(c)

(g)
Figure 1. Bed image enhanced using the variants of histogram equalization; (a)Original bed image, bed image enhanced using
(b)dualistic sub-image histogram equalization, (c)minimum mean brightness error bi-histogram equalization, (d)bi-histogram
equalization with parameters chosen using the Logarithmic AME
sub-images based on a threshold, with all values above the
threshold in one image and all those below in the other. The
two images are then equalized separately and recombined.
Many attempts have been made to define the optimal value
of this threshold. However, as no enhancement measure has
been used, these methods can be improved upon. Figure 1
compares the results of several of these methods to the
results using the Logarithmic AME measure of image
enhancement.

2.2. Logarithmic Image Processing (LIP) model

The LIP model was introduced by Jourlin and Pinoli to give
a non-linear framework for image processing [6]. This is
designed to both maintain the pixel values inside the range
[0,M) as well as to more accurately process images from a
human visual system point of view. The LIP model can be
summarized as follows:
M
b
ab
baba
−+=⊕

(2)
gM
a
Mba
−
−
=Θ

(3)
c
M
(
f
MMac
⎟
⎠
⎞
⎜
⎝
a
⎛−
1
−=⊗


(4)
(
ϕ
) )b)(
1
ba
ϕϕ⋅=∗
−
(5)
Where we use ⊕ as LIP addition, Θ as LIP subtraction, ⊗
as LIP scalar multiplication, and * as LIP grayscale
multiplication. Also, a and b are any grey tone pixel values,
is the maximum value of the range, and c is a constant. In
general, a and b correspond to the same pixel in two
different images that are being added, subtracted, or
multiplied. For the grayscale multiplication, the functions
ϕ and
ϕ
are defined as:
⎛−
⋅−=
Ma1ln)(
ϕ
1
−
⎟
⎠
f
⎞
⎜
⎝
M
f

(6)
⎥⎦
⎤
⎢⎣
⎡
⎟
⎠
⎞
⎜
⎝
⎛−
−⋅=
−
M
Maexp1)(
1
ϕ

(7)
Where α and β are user-defined operating parameters which
can be fine-tuned for the specific images being processed
[8].

2.3. Measure of Image Enhancement

A problem of image enhancement has always been to
develop a quantitative measure
enhancement. The Logarithmic AME and Logarithmic
AMEE measures utilize the Michelson Contrast as well as
Fisher’s Law or Entropy [7]. The measures function by
segmenting an image into k1 x k2 sized blocks, assessing
each block separately according to the following formulae,
and averaging the results.
∑∑
==
11
21
20
ij
kk
to assess image
⎟⎟
⎠
⎞
⎞
⎜⎜
⎝
⎛
⊕
Θ
⊗⊗=Φ
12
21
,:min,:max
⎛
ln
,:min
w
I
,: max
w
ln
11
)(log
kk
lklk
w
lk
w
lk
kk
I
II
AME

(8)
∑∑
=
1
i
I
=
⎟⎟
⎠
⎜⎜
⎝
⊕
Θ
∗
⊕
Θ
⊗=Φ
12
21
1
,:min,: max
,:min
w
I
,:max
w
,:min,:max
,: min
w
I
,: max
w
21
1
k
)(log
kk
j
lklk
w
lk
w
lk
lklk
w
lk
w
lk
kk
I
II
I
II
k
AMEE

(9)
Where
w
max;
lk
I
,
and
w
min;
lk
,
are the local maximum and
minimum, respectively. The summations use LIP
arithmetic.
It is important to note that this method is relative. This
means that it is not an arbitrary standard of “goodness,” but
rather a means of comparing similarly processed images.
Further, larger or smaller numbers are not necessarily better,
instead this depends on the image.

3. METHODS

In this section, we present the human vision thresholding
(HVT) algorithm and the multi-histogram equalization using
HVT method.

3.1. Human Visual System Based Image Enhancement

Human Visual System (HVS) based Image Enhancement
aims to emulate the way in which the human visual system
discriminates between useful and useless data [8] and is
based on the background illumination and the gradient. The
former is arrived at using the following formula:

Page 3

2),(),(
24
1
) , (i
4
1
2
1
),(
'
÷
⎥
⎦
⎥
⎤
⎢
⎣
⎢
⎡
+
⎟⎟
⎠
⎞
⎜⎜
⎝
⎛
+=
∑
Q
∑
Q
yxXlkXjXyxB

(10)
Where B(x,y) is the background intensity at each pixel,
X(x,y) is the input image, Q is all of the pixels which are
directly up, down, left, and right from the pixel, and Q’ is
all of the pixels diagonally one pixel away. We must also
define a parameter BT, which is the maximum difference in
the image, arrived at using:
()
min),( maxyxXBT
−=
Further, the gradient information is needed, which is
arrived at in the following formula:
) 1,(),(
1
+−=
yxXyxXG
where X’(x,y) is the gradient information and G1, G2 are the
directional gradients. Finally, we must also know some
parameters concerning the human eye itself, which we will
call Bxi, I = 1,2,3 and Ki, I = 1,2,3. These are arrived at
using the following formulas:
BB
11
α
=
⎛
⋅=
β
() )y,(xX

(11)
() 2
÷
),( '
), 1(),(
21
2
+=
+−=
GGyxX
yxXyxXG

(12)
T
X
x
Tx
Tx
BB
BB
33
22
α
α
=
=

(13)
313
212
1
/
),(
),( '
max
100
K
1
x
x
BKK
BK
yxB
yx
K
=
=
⎟⎟
⎠
⎞
⎜⎜
⎝

(14)
Where α1, α2, α3 are parameters based upon the three
different regions of response characteristics displayed by the
human eye. As α1 is the lower saturation level, it is
effective to set this to 0. For α2, α3, it is necessary to
determine these experimentally, or using the measure.
Using this information, the image is first broken up into
the different regions of human visual response. These
different regions are characterized by the minimum
difference between two pixel intensities for the human
visual system to register a difference. Next, these three
regions are thresholded, removing the pixels which do not
constitute a noticeable change for a human observer and
placing these in a fourth image. These four images are
arrived at using the following formula:
Pixels Remaining All
),(
)
2
,
y
( '
x
&),(
),(
),( '
&),(
),(
),( '
B
&),(
such that
),(4 Im
),(3Im
),(2Im
),(1 Im
33
123
212
K
B
yxX
ByxB
K
yxB
yxX
ByxBB
K
yx
yxX
ByxBB
yxX
yxX
yxX
yxX
x
xx
xx
≥≥
≥≥≥
≥≥≥
=
=
=
=

(15)
These four images are then enhanced separately and
recombined to form the enhanced image.

3.1. HVS Based Multi-Histogram Equalization
Experimental results have
improvement between tri-histogram equalization using the
standard thresholding method
shown no significant
and bi-histogram
equalization. Another method is necessary to achieve better
results using multi-histogram equalization.
By separating the image into regions by the quality of
illumination, such as over-illuminated, well illuminated, and
under-illuminated, traditional histogram equalization can be
used on each region to correct for non-uniform illumination.
For this, we utilize the human visual system to segment the
image, using the measure of image enhancement to select
α2 and α3. The first three images are then equalized
separately and unionized, with the remaining pixels filled in.
In summary, the algorithm is executed as follows:
Input Image
Step 1: Segment image
Thresholding algorithm
Step 2: Equalize images 1, 2, and 3 separately
Step 3: Recombine the pixels in the three equalized
images
Step 4: Fill in the missing pixels
Output Image

4. COMPUTER SIMULATIONS

In this section, we present the results of computer
simulations. The proposed method has been tested using a
variety of images, including uniformly illuminated, well and
poorly illuminated, and non-uniformly illuminated, and
images with shadows. The test images include images that
were produced by high-quality professional digitization
methods as well as lower quality cell phone cameras.
We compare against the results of other leading
histogram equalization methods. The comparison is made
both by visual inspection and the Logarithmic AME
performance measure. The results show the proposed
method to produce more visually pleasing enhanced images
and more consistently produce optimal enhanced images.
Figure 2 shows the resulting images for the bed image.
As can be seen from visual inspection, the proposed
algorithm creates the most visually pleasing enhanced
image. It corrects for the largely under-illuminated scene,
allowing details such as the bed and pictures to be seen
clearly. Further, properly illuminated details such as the
light and the image in the window can still be seen clearly.
Figure 3 shows the resulting images for a camera phone
image. The original image, in figure 3.a, has regions of
under-illumination, over-illumination,
illumination. The enhanced images using bi-histogram
equalization only enhance one section, making either the
under-illuminated background visible or correcting the
over-illuminated foreground. The image in figure 3.f, using
the proposed method, corrects both regions such that all
areas of the image can be clearly seen.
Figure 4 summarizes the results for a representative
range of images, showing the results of the quantitative
measure for the enhanced images. As can be seen from the
using Human Vision
and proper

Page 4

(a)

(b)

(c)

(d)
Figure 2: Bed image enhanced using the variants of histogram equalization; (a)Original bed image, bed image enhanced using
(b)recursive mean-separate histogram equalization, (c)brightness preserving histogram equalization with maximum entropy
(d)human vision based multi-histogram equalization

(a)

(b)

(c)

(d)

(e)
(f)
Figure 3: Cave image enhanced using the variants of histogram equalization; (a)Original cave image, cave image enhanced using
(b)dualistic sub-image histogram equalization, (c)minimum mean brightness error bi-histogram equalization, (d)recursive mean-
separate histogram equalization, (e)brightness preserving histogram equalization with maximum entropy, (f) human vision based
multi-histogram equalization
numbers, the proposed algorithm consistently outperforms
the other histogram equalization algorithms.
Image HVSMHE DSHE MMBEBHE
bed 973.8 1006
cave 277.1 222.8
faces 641.2 530.4
street 237.2 159.2
school 635.6 442.7
Figure 4: Logarithmic AME results for the proposed algorithm
and other equalization methods

5. CONCLUSION

In this paper we introduced a new method for performing
histogram equalization, resulting in a more robust histogram
equalization that is able to correct images with regions of
over-illumination, under-illumination,
illumination. We compared the results of the proposed
algorithm to the results of the leading histogram
equalization methods. The effectiveness of the proposed
method was shown both visually and quantitatively. This
demonstrates that the Multi-Histogram Equalization method
using HVT is effective for the enhancement of images with
regions of improper illumination.

6. REFERENCES

[1] Y. Wang, Q. Chen, B. Zhang, “Image Enhancement Based on
Equal Area Dualistic Sub-Image Histogram Equalization Method,”
IEEE Transactions on Consumer Electronics, vol. 45, no. 1, pp.
68-75, Nov. 1998.

RMSHE
1006
178.8
475.9
139.1
353.6
BPHEME
991.8
189.3
454.7
135.9
417.9
1007
178.2
449.0
128.6
372.7
and proper
[2] S.D. Chen, A.R. Ramli, “Contrast Enhancement using
Recursive Mean-Separate Histogram Equalization for Scalable
Brightness Preservation”, IEEE Transactions on Consumer
Electronics, vol. 49, no. 4, pp. 1301-1309, Nov. 2003.

[3] S.D. Chen, A.R. Ramli, “Minimum Mean Brightness Error
Bi-Histogram Equalization in Contrast Enhancement,” IEEE
Transactions on Consumer Electronics, vol. 49, no. 4, pp. 1310-
1319, Nov. 2003.

[4] C. Wang, Z. Ye, “Brightness Preserving Histogram
Equalization with Maximum Entropy: A Variational Perspective,”
IEEE Transactions on Consumer Electronics, vol. 51, no. 4, pp.
1326-1334, Nov. 2005.

[5] A.M. Vossepoel, B.C. Stoel, A.P. Meershoek, “Adaptive
Histogram Equalization Using Variable Regions,” in Proc. IEEE
Conference on Pattern Recognition, Rome, pp. 351-353, Nov.
1988.

[6] M. Jourlin, J.C. Pinoli, “Logarithmic Image Processing; The
Mathematical and Physical Framework for the Representation and
Processing of Transmitted Images,” Advances in Imaging and
Electron Physics, vol. 115, pp. 129-196, 2001.

[7] E. Wharton, S. Agaian, K. Panetta, “Comparative Study of
Logarithmic Enhancement Algorithms
Measure,” in Proc. SPIE Electronic Imaging, San Jose, CA, #6064
12, Jan. 2006.

[8] E. Wharton, S. Agaian, K. Panetta, “A Logarithmic Measure
of Image Enhancement,” in Proc. SPIE Defense and Security
Symposium, Orlando, FL, #62500P, Apr. 2006.
with Performance