Information Preserving Color Transformation for Protanopia and Deuteranopia

Abstract

In this letter, we proposed a new recoloring method for people with protanopic and deuteranopic color deficiencies. We present a color transformation that aims to preserve the color information in the original images while maintaining the recolored images as natural as possible. Two error functions are introduced and combined together to form an objective function using the Lagrange multiplier with a user-specified parameter lambda. This objective function is then minimized to obtain the optimal settings. Experimental results show that the proposed method can yield more comprehensible images for color-deficient viewers while maintaining the naturalness of the recolored images for standard viewers.

Full text

IEEE SIGNAL PROCESSING LETTERS, VOL. 14, NO. 10, OCTOBER 2007 711
Information Preserving Color Transformation
for Protanopia and Deuteranopia
Jia-Bin Huang, Yu-Cheng Tseng, Se-In Wu, and Sheng-Jyh Wang, Member, IEEE
Abstract—In this letter, we proposed a new recoloring method
for people with protanopic and deuteranopic color deficiencies. We
present a color transformation that aims to preserve the color in-
formation in the original images while maintaining the recolored
images as natural as possible. Two error functions are introduced
and combined together to form an objective function using the La-
grange multiplier with a user-specified parameter
. This objective
function is then minimized to obtain the optimal settings. Experi-
mental results show that the proposed method can yield more com-
prehensible images for color-deficient viewers while maintaining
the naturalness of the recolored images for standard viewers.
Index Terms—Color deficiency, image processing, Lagrange
multiplier, recoloring.
I. INTRODUCTION
D
UE to the increasing use of colors in multimedia con-
tents to convey visual information, it becomes more impor-
tant to perceive colors for information interpretation. However,
roughly around 5%–8% of men and 0.8% of women have cer-
tain kinds of color deficiency. Unlike people with normal color
vision, people with color deficiency have difficulties discrimi-
nating certain color combinations and color differences. Hence,
multimedia contents with rich colors, which can be well dis-
criminated by people with normal color vision, may sometimes
cause misunderstanding to people with anomalous color vision.
Humans’ color vision is based on the responses to photons
in three different types of photoreceptors, which are named
“cones” and are contained in the retina of human eyes [1].
The peak sensitivities of these three distinct cones lie in the
long-Wavelength (L), middle-wavelength (M), and short-wave-
length (S) regions of the spectrum. Anomalous trichromacy is
frequently characterized by a shift of one or more cone types
so that the pigments in one type of cone are not sufficiently
distinct from the pigments in others. For example, L-Cones are
more like M-Cones in protanomaly and M-Ccones are more
like L-Cones in deuteranomaly. On the other hand, dichromats
have only two distinct pigments in the cones and entirely lack
one of the three cone types. Lack of L-cones is referred to
as protanopia, lack of M-cones is referred to as deuteranopia,
Manuscript received October 15, 2006; revised February 11, 2007. This work
was supported by the National Science Council of the Republic of China under
Grant NSC-94-2219-E-009-008. The associate editor coordinating the review
of this manuscript and approving it for publication was Dr. Konstantinos N.
Plataniotis.
The authors are with the Department of Electronics Engineering, National
Chiao Tung University, Hsin-Chu 30050, Taiwan, R.O.C. (e-mail: mysoul-
foryou.ee91@nctu.edu.tw).
Color versions of one or more of the figures in this paper are available online
at http://ieeexplore.ieee.org.
Digital Object Identifier 10.1109/LSP.2007.898333
and lack of S-cones is referred to as tritanopia. Among these
three types of dichromats, protanopia and deuteranopia have
difficulty in distinguishing red from green, while tritanopia
has difficulty in discriminating blue from yellow. So far, many
research works have been conducted on simulating color-defi-
cient vision [2]–[5]. These approaches represent color stimuli
as vectors in the three-dimensional LMS space, where three
orthogonal axes L, M, and S represent the quantum catch for
each of the three distinct cone types. Since the dichromatic
vision is the reduced form of trichromatic vision, the lack of
one cone type can be simulated by collapsing one of the three
dimensions into a constant value.
To enhance the comprehensibility of images for color-defi-
cient viewers, daltonization is proposed in [6] to recolor images
for dichromats. In [6], the authors first increase the red/green
contrast in the image and then use the red/green contrast in-
formation to adjust brightness and blue/yellow contrast. In [7],
Ishikawa
et al. described the manipulation of webpage colors for
color-deficient viewers. They first decompose a webpage into a
hierarchy of colored regions and determine “important” pairs of
colors that are to be modified. An objective function is then de-
fined to maintain the distances of these color pairs, as well as to
minimize the extent of color remapping. This approach is fur-
ther extended to deal with full-color images in [8]. On the other
hand, Seuttgi Ymg et al. [9] proposed a method to modify colors
for dichromats and anomalous trichromats. For dichromats, a
monochromatic hue is changed into another hue with less sat-
uration, while for anomalous trichromats, the proposed method
tends to keep the original colors. In [10], Rasche et al. use a
linear transform to convert colors in the CIELAB color space
and enforce proportional color differences during the remap-
ping. Based on the same constraint for color deficiency, the au-
thors further improve the optimization process by using the ma-
jorization method [11].
Basically, all the aforementioned works may generate im-
ages that are more comprehensible to color-deficient viewers.
However, recolored images may look very unnatural to viewers
with normal vision. From an application viewpoint, images in a
public place may be simultaneously observed by normal people
and color-deficient people. For example, in a public transporta-
tion system, many advertisements and traffic maps are delivered
in colors. Without concerning the needs of deficient observers,
color-deficient people may have difficulty in understanding the
image contents. On the contrary, if only concerning the needs
of color-deficient people, then these recolored images may look
annoying to normal observers. Hence, in this letter, we aim to
develop a recoloring algorithm that can automatically construct
a transformation to maintain details for color-deficient viewers
while preserving naturalness for standard viewers.
1070-9908/$25.00 © 2007 IEEE

712 IEEE SIGNAL PROCESSING LETTERS, VOL. 14, NO. 10, OCTOBER 2007
Fig. 1. Rotation operation in the – plane.
II. COLOR REPRODUCTION FOR PROTANOPIA
AND
DEUTERANOPIA
A. Color Reproduction Method
In this letter, we focus on protanopia and deuteranopia,
which are the major types of color deficiency. In order to
mimic the color perception of protanopia and deuteranopia, we
adopt Brettel’s algorithm [2] to simulate the perceived images.
Here, we adopt CIELAB color space as the working domain.
In both protanopia and deuteranopia, there is strong correlation
between the original colors and the simulated colors in the
values of
and , while there is a weak correlation between
the original
and the perceived . That is, the original color
information in
gets lost significantly. To retain the infor-
mation in
, a reasonable way is to do some kind of image
warping so that the information of
is mapped onto the
axis in the CIELAB color space.
In our approach, we aim to maintain the color differences of
color pairs in the CIELAB color space while keeping the recol-
ored images as natural as possible. To keep the recolored image
natural, three premises are adopted. First, the recolored image
has the same luminance as the original image. Second, colors
with the same hue in the original image still have the same hue
after recoloring. Third, the saturation of the original colors is
not altered after recoloring. In our approach, a rotation oper-
ation is adopted in the
– plane to transform the informa-
tion of
onto the axis, as illustrated in Fig. 1. Here, we
assume some color stimuli
have the same in-
cluded angle
with respect to the axis. The rotation opera-
tion maps these colors to new colors
, which lay
on another line with the included angle
. If ignoring
the nonlinear property of the iso-hue curves in the CIELAB
color space [13], this rotation process simultaneously changes
the hue of
with the same amount of hue. Hence,
the transformed colors
still share the same hue
after color transformation. Moreover, the saturation of the orig-
inal color
is also preserved.
In mathematics, this rotation operation can be formulated as
a matrix multiplication. That is, we have
(1)
where
and are the CIELAB values of the
recolored color and the original color, respectively.
is a
monotonically decreasing function of
. Since the color differ-
ence along the
axis can be well discriminated by protanopic
Fig. 2. (a) Function
with three parameters: , and . (b) Func-
tion
with parameters
and for a half plane.
and deuteranopic viewers, ) decreases to zero when ap-
proaches
. In this letter, we define to be
(2)
for the right half-plane of the
– plane, where ranges from
to . Here, represents the maximal change of
the included angle and
represents the degree of the decreasing
rate. These two parameters will be specified by optimizing an
objective function based on the contents of the original color
image. For the left half-plane
,wedefine the
function in a similar manner but with different and .
This is because in practice, we may want the right half and the
left half of the
– plane to have different transformations, as
shown in Fig. 2(a). Moreover, since
approaches zero when
colors are close to the
axis, crossover of colors can be avoided
when crossing the b
axis.
In Fig. 2(b), we show the plot of the transformed hue
versus the original hue for the right-half – plane.
If the
is positive, then the quadrant with positive
will be compressed while the quadrant with negative
will be expanded and vice versa. To avoid
colors crossover in the compressed quadrant, we require
(3)
By combining (2) and (3), we have
(4)
Since
ranges from to , the LHS of (4) has the
lower bound
. Thus, we can obtain the constraint
. On the other hand, the constraint in (4) is not necessary
in the expanded quadrant. Hence, we introduce two parameters
and , one for each quadrant. For the compressed quadrant,
the constraint in (4) is required, while for the expanded region,
no constraint is needed for
and . In the proposed algo-
rithm, there would be six parameters in total. Their notations
and meanings are listed in Table I.
B. Optimization Using Detail and Naturalness Criteria
In this section, we introduce two criteria, one for detail pre-
serving and the other for naturalness preserving. For each color

HUANG et al.: INFORMATION PRESERVING COLOR TRANSFORMATION FOR PROTANOPIA AND DEUTERANOPIA 713
TABLE I
P
ARAMETERS FOR
RECOLORING
pair in the original color domain, we first calculate the perceived
color difference with respect to a person with normal vision.
Then, for the corresponding color pair in the transformed color
domain, we calculate the perceived color difference with re-
spect to a person with protanopic or deuteranopic deficiencies.
As mentioned above, we follow Brettel’s algorithm [2] to simu-
late the color perception for protanopia and deuteranopia. In our
criterion, we wish these two perceived color differences to be as
similar as possible. Hence, we define an error function to be
(5)
where
and range over the colors contained in the images,
is a perceptual color difference metric, is our recoloring
function, and
denotes the simulated color perception
using Brettle’s algorithm. By minimizing this error function, we
can preserve color details of the original image.
On the other hand, we attempt not to dramatically modify the
color perception of the color images since a severe modification
may make the recolored image extremely unnatural for normal
viewers. Hence, we define another error function to be
(6)
where
ranges over all the colors in the original color image.
Minimizing this error function shortens the color distance
between the original colors and the corresponding remapped
colors. To preserve both details and naturalness, we combine
these two error functions using the Lagrange multiplier with a
user-specified parameter
. Here, we further normalize these
two error functions by their arithmetic means to achieve similar
order of magnitude. That is, the total error is written as
(7)
To minimize the objective function in (7), we roughly
estimate
and in the initialization stage
with
, and fixed to 1. Then we use the
Fletcher–Reeves conjugate-gradient method with the constraint
in (4) to obtain the optimal solution. By choosing different
values of
, users may adjust the tradeoff between details and
naturalness. A larger
makes the recolored image more natural
Fig. 3. (a) Original image. (b) Recolored by the Daltonization method with a
middle-level correction [6]. (c) Recolored by Rasche’s method [10]. (d) Recol-
ored by our proposed method with
. (e) Recolored by our proposed
method with
. (f)–(j): Corresponding color images perceived by people
with deuteranopic color deficiency.
for normal viewers, while a smaller
makes the recolored
image more comprehensible for color-deficient viewers.
One more thing to mention is about the nonlinear property
of the iso-hue curves in the CIELAB color space [13]. That is,
two colors with the same included angle
in the – plane
may not have the same value of hue. Due to this nonlinear prop-
erty, colors with the same hue in the original image may gen-
erate colors with different hues in the recolored image. To solve
this problem, we may simply apply the hue-linearization process
mentioned in [14] as a preprocessing and then apply the delin-
earization process after the recoloring algorithm.
III. E
XPERIMENTAL
RESULTS
In Fig. 3, we demonstrate some experimental results for the
“flower” image. Fig. 3(a)–(e) shows the images perceived by
normal viewers, while Fig. 3(f)–(j) presents the images per-
ceived by viewers with deuteranopic deficiency. We can ob-
serve that the color contrast between the red flower and the
green leaves is lost for people with deuteranopic deficiency.
We compare our method with the Daltonization method [6] and
Rasche’s method [10], as shown in Fig. 3(b)–(e) and (g)–(j). We
may observe that even though the Daltonization method with
a middle-level correction may also preserve the naturalness of
the recolored image for normal people, the contrast between the
flower and leaves looks very poor for deuteranopic people. On
the other hand, even though Rasche’s method may create great
contrast for deuteranopic people, the naturalness of the recol-
ored image is extremely poor for people with normal vision. In
comparison, our method may well preserve both details and nat-
uralness at the same time.
To verify the effect of
, we also demonstrate in Fig. 3(e) that
our proposed method will produce an extremely unnatural re-
colored image if
. Furthermore, in Table II, we compare
the naturalness error and detail error among different methods,
based on (5) and (6). In our approach, the naturalness error de-
creases while detail error increases when
rises. For the Dal-
tonization method, even though its naturalness error is less than
ours, its detail error becomes extremely high. On the other hand,
even though Rasche’s method has a smaller detail error, its natu-
ralness error is larger. These experimental results show that both
naturalness and detail can be properly preserved by our method.

714 IEEE SIGNAL PROCESSING LETTERS, VOL. 14, NO. 10, OCTOBER 2007
TABLE II
C
OMPARISON OF
NATURALNESS
ERROR AND
DETAIL ERROR
Fig. 4. (a) Original image. (b) Perceived image by protanopic viewer. (c) Per-
ceived image by deuteranopic viewer. (d) Recolored image for protanopia. (e)
Perceived image of (d) by protanopic viewers. (f) Recolored image for deuteran-
gopia. (g) Perceived image of (f) by deuteranopic viewers.
In Fig. 4, we show more examples to verify the effectiveness of
the proposed method.
We also used Thurstone’s Law of Comparative Judgment [12]
for subjective evaluation. In our subjective experiments, ten par-
ticipants with normal vision were involved and six represen-
tative color images were chosen, as shown in Fig. 5. All ten
participants were graduate students with some background in
video coding and image processing. Since we have difficulty in
finding color-deficient viewers, we adopted Brettel’s algorithm
[2] to mimic the perception of protanopia and deuteranopia. In
the first experiment, each of the six images was, respectively, re-
colored by the Daltonization method, Rasche’s method, and our
method with
. For each image, the original image was
first shown to the participants. Then, exhaustive paired compar-
isons were performed over the recolored images, and the partic-
ipants were asked to choose the more natural image from each
pair. This experiment is to evaluate the naturalness of the re-
colored images from the viewpoint of normal viewers. In the
second experiment, Brettel’s algorithm was applied over the
original images and recolored images to simulate the perceived
images for deuteranopia. Exhaustive paired comparisons were
performed again over the simulated images, and the participants
were asked to choose the more comprehensible image from each
pair. This experiment is to evaluate the comprehensibility of the
recolored images from the viewpoint of deuteranopic viewers.
The results of these two subjective experiments were analyzed
based on Thurstone’s Law of Comparative Judgment [12]. The
scaling of data is shown in Fig. 6. Fig. 7(a) indicates that both
our method and the Daltonization method produce more natural
images, while Fig. 7(b) indicates that our method may preserve
more details than the other two methods.
Fig. 5. Six images for the subjective evaluation.
Fig. 6. Experimental results. (a) Scales from the “naturalness” experiment. (b)
Scales from the “comprehensibility” experiment.
IV. C
ONCLUSION
We have presented in this letter a new recoloring method for
people with protanopic or deuteranopic deficiency. We propose
a color transformation that can yield more comprehensible im-
ages for protanopic or deuteranopic viewers while maintaining
the naturalness of the recolored images for standard viewers.
The same procedure can be extended to the case of tritanopia,
in which blue and yellow tones cannot be well distinguished.
The experimental results show that our proposed method per-
forms subjectively better than others, in terms of comprehensi-
bility and naturalness.
R
EFERENCES
[1] Wandell, Foundations of Vision. Sunderland, MA: Sinauer, 1995.
[2] H. Brettel, F. Vi’enot, and J. Mollon, “Computerized simulation of
color appearance for dichromats,” J. Optic. Soc. Amer. A, vol. 14, no.
10, pp. 2647–2655, Oct. 1997.
[3] G. Meyer and D. Greenberg, “Color-defective vision and computer
graphics displays,” IEEE Comput. Graph. Appl., vol. 8, no. 5, pp.
28–40, Sep. 1988.
[4] S. Kondo, “A computer simulation of anomalous color vision,” in Color
Vision Deficiencies. Amsterdam, The Netherlands: Kugler & Ghe-
dini, 1990, pp. 145–159.
[5] J. Walraven and J. W. Alferdinck, “Color displays for the color blind,”
in Proc. IS&T/SID 5th Color Imaging Conf., 1997, pp. 17–22.
[6] R. Dougherty and A. Wade, Daltonize. [Online]. Available:
http://www.vischeck.com/daltonize/.
[7] M. Ichikawa, K. Tanaka, S. Kondo, K. Hiroshima, K. Ichikawa, S.
Tanabe, and K. Fukami, “Web-page color modification for barrier-free
color vision with genetic algorithm,” Lecture Notes Comput. Sci., vol.
2724, pp. 2134–2146, 2003.
[8] M. Ichikawa, K. Tanaka, S. Kondo, K. Hiroshima, K. Ichikawa, S.
Tanabe, and K. Fukami, “Preliminary study on color modification for
still images to realize barrier-free color vision,” in Proc. IEEE Int. Conf.
Systems, Man, Cybernetics, 2004, pp. 36–41.
[9] S. Yang and Y. M. Ro, “Visual contents adaptation for color vision
deficiency,” in Proc. IEEE Int. Conf. Image Process., Sep. 2003, vol. 1,
pp. 453–456.
[10] K. Rasche, R. Geist, and J. Westall, “Detail preserving reproduction
of color images for monochromats and dichromats,” IEEE Comput.
Graph. Appl., vol. 25, no. 3, pp. 22–30, May–Jun. 2005.
[11] K. Rasche, R. Geist, and J. Westall, “Re-coloring images for gamuts of
lower dimension,” EuroGraphics, vol. 24, no. 3, pp. 423–432, 2005.
[12] W. S. Torgerson, Theory and Method of Scaling. New York: Wiley,
1967.
[13] G. Hoffmann, CIELab Color Space. [Online]. Available: http://www.
fho-emden.de/~hoffmann/cielab03022003.pdf.
[14] G. J. Braun, F. Ebner, and M. D. Fairchild, “Color Gamut mapping
in a hue-linearized CIELab color space,” in Proc. IS&T/SID6th Color
Imaging Conf., 1998, pp. 163–168.