ArticlePDF Available

High Dynamic Range Display Systems

Authors:

Abstract and Figures

The dynamic range of many real-world environments exceeds the capabilities of current display technology by several orders of magnitude. In this paper we discuss the design of two different display systems that are capable of displaying images with a dynamic range much more similar to that encountered in the real world. The first display system is based on a combination of an LCD panel and a DLP projector, and can be built from off-the-shelf components. While this design is feasible in a lab setting, the second display system, which relies on a custom-built LED panel instead of the projector, is more suitable for usual office workspaces and commercial applications. We describe the design of both systems as well as the software issues that arise. We also discuss the advantages and disadvantages of the two designs and potential applications for both systems.
Content may be subject to copyright.
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed for
profit or direct commercial advantage and that copies show this notice on the first page or
initial screen of a display along with the full citation. Copyrights for components of this
work owned by others than ACM must be honored. Abstracting with credit is permitted. To
copy otherwise, to republish, to post on servers, to redistribute to lists, or to use any
component of this work in other works requires prior specific permission and/or a fee.
Permissions may be requested from Publications Dept., ACM, Inc., 1515 Broadway, New
York, NY 10036 USA, fax +1 (212) 869-0481, or permissions@acm.org.
High Dynamic Range Display Systems
Helge Seetzen
1,2
, Wolfgang Heidrich
2
, Wolfgang Stuerzlinger
3
, Greg Ward
1
, Lorne Whitehead
2
,
Matthew Trentacoste
2
, Abhijeet Ghosh
2
, Andrejs Vorozcovs
3
1)
Sunnybrook Technologies,
2)
The University of British Columbia,
3)
York University
Figure 1: From left to right: our projector-based display showing an HDR image; our LED-based HDR display showing a discrete and a
smooth intensity ramp (the top half of the discrete ramp and the bottom half of the smooth ramp have each been covered by a 1% transparent
filter to illustrate high luminance content on the left side of the image, which cannot be captured by the camera); a color-coded original HDR
image; HDR photograph taken off the screen of our projector-based system; HDR photograph taken off a conventional monitor displaying
the tone-mapped image.
Abstract
The dynamic range of many real-world environments exceeds the
capabilities of current display technology by several orders of mag-
nitude. In this paper we discuss the design of two different display
systems that are capable of displaying images with a dynamic range
much more similar to that encountered in the real world. The first
display system is based on a combination of an LCD panel and
a DLP projector, and can be built from off-the-shelf components.
While this design is feasible in a lab setting, the second display
system, which relies on a custom-built LED panel instead of the
projector, is more suitable for usual office workspaces and commer-
cial applications. We describe the design of both systems as well as
the software issues that arise. We also discuss the advantages and
disadvantages of the two designs and potential applications for both
systems.
Keywords: Hardware Novel Display Technologies; Rendering
Perceptually Based Rendering; Image and Video Processing
Image Processing; Methods and Application Signal Processing;
Hardware – Framebuffer Algorithms.
CR Categories: B.4.2 [INPUT/OUTPUT AND DATA COMMU-
NICATIONS]: Input/Output Devices—Image display; I.3.3 [COM-
PUTER GRAPHICS]: Picture/Image Generation—Display algo-
rithms; I.3.4 [COMPUTER GRAPHICS]: Graphics Utilities—
Device drivers; I.4.0 [IMAGE PROCESSING AND COMPUTER
VISION]: General—Image displays.
1 Introduction
In the past few years, the limited dynamic range of both imaging
devices and displays has received a lot of attention in the computer
graphics community. Algorithms have been developed for captur-
ing both photographs [Mann and Picard 1994; Debevec and Ma-
lik 1997; Robertson et al. 1999; Mitsunaga and Nayar 1999] and
videos [Kang et al. 2003] with extended dynamic range.
Simultaneously, tone mapping operators have also been devel-
oped for compressing the dynamic range so that the images can be
displayed on the familiar 8 bit/channel displays with typical con-
trast ratios of about 300 : 1, including any conventional Cathode
Ray Tube (CRT), Liquid Crystal (LCD), and projector-based dis-
play. While these tone mapping operators (e.g. [Schlick 1994; Lar-
son et al. 1997; Tumblin and Turk 1999; Durand and Dorsey 2002]
among others) allow for displaying high-dynamic-range (HDR) im-
ages in a recognizable and even aesthetically pleasing way, nobody
would confuse a photograph rendered in this fashion with, say,
watching the same scene through a window. The dynamic range
of conventional displays is simply insufficient to create the optical
sensation of watching a real sunset or driving a car into oncoming
traffic at night. Note that this is not just an issue of top intensity:
simply increasing the brightness of a conventional display would
wash out the dark tones and turn them into a medium gray. What is
needed is a significant expansion of the contrast or dynamic range
of the display.
In this paper we describe two alternative designs for HDR dis-
play systems. We have built prototypes of both, and discuss both
the optical design, and related software issues such as display cali-
bration and the rendering of HDR images on both displays.
Both display systems are based on the fundamental idea of using
an LCD panel as an optical filter of programmable transparency to
modulate a high intensity but low resolution image from a second
display. For example, assume we have any display with a contrast
range of c
1
: 1 between the darkest and the brightest intensity pro-
760
© 2004 ACM 0730-0301/04/0800-0760 $5.00
ducible by that display. If we now put an LCD panel with a contrast
ratio of c
2
: 1 in front of the first one, then the (theoretical) contrast
of the combined system is (c
1
· c
2
) : 1. In practice, the first display
needs to be able to produce a very high intensity image, because
color LCD panels only have a transparency of about 3-8%, even
when switched to ‘white’, so that most energy is actually absorbed.
Another reason for using a display with a very high base intensity
is that a lot of the HDR images we would like to show have, in fact,
very bright regions in them.
Based on this principle, we have derived two alternative designs
for HDR displays. In the first design (Section 4), a video projector
based on Digital Light Projector (DLP) technology serves as the
base display. In this version, we directly focus the projector onto
the back of the LCD panel. Since the illuminated area is much
smaller than during regular use of a projector, the light density is
dramatically improved, yielding the high top intensities that we are
aiming for. While this design works well in a laboratory setting,
it has several drawbacks that restrict its use for a wider class of
applications. In particular, these are a large form factor, significant
power consumption and heat development, as well as calibration
issues.
To overcome these issues, we have devised a second design (Sec-
tion 5), in which the projector is replaced with a low-resolution
array of ultra-bright LEDs. The intensity of every LED can be pro-
grammed individually, yielding a low resolution version of the de-
sired image. High frequency features are introduced by attaching
a high-resolution LCD panel to the front of this LED array, and
adjusting its transparency accordingly. This design makes use of
results from psychophysics, which show that very high contrast, al-
though important on a global scale, cannot be perceived by humans
at high spatial frequencies (see Section 3).
The two displays we developed have dynamic ranges well be-
yond 50, 000 : 1, and a maximum intensity of 2700cd/m
2
and
8500cd/m
2
, respectively. This compares to a typical dynamic
range of about 300 : 1, and a maximum intensity of about
300cd/m
2
for a typical desktop display.
The design of both systems and their appropriate rendering algo-
rithms, as well as the advantages and disadvantages are detailed in
Sections 4 and 5. In Section 6 we discuss possible applications for
our display technology before we conclude with some remarks and
future research directions in Section 7.
2 Related Work
The class of image processing techniques for coping with the dis-
crepancy between real world luminances and those that fit within
the limited gamut of a conventional output device is collectively
called tone-mapping. Tumblin and Rushmeier [1993] introduced
this concept to computer graphics, though their early work did not
address dynamic range limitations per se. The first tone-mapping
operator to tackle dynamic range reduction was Chiu et al. [1993],
who used a spatially varying exposure ramp over the image. How-
ever, this approach led to disturbing “reverse gradients” typically
seen as halos around light sources. Later work by Larson et
al. [1997] returned to a global operator for dynamic range reduction
based on histogram adjustment to avoid these artifacts, with local
variations to simulate disability glare due to high contrast bound-
aries in a scene. Pattanaik et al. [1998] developed what some re-
searchers consider the ultimate still image operator based on the
human visual system, incorporating color adaptation, local con-
trast, and dynamic range. However, even this operator exhibited
some reverse-gradient effects near high contrast boundaries due to
its local spatial filters, leading other researchers to take a different
approach.
The basic challenge for a spatially varying tone-mapping opera-
tor is that it needs to reduce the global contrast of an image without
affecting the local contrast to which the human visual system (HVS)
is sensitive. To accomplish this, an operator must segment the HDR
image, either explicitly or implicitly, into regions the HVS does not
correlate during dynamic range reduction. The first researchers to
successfully accomplish this in an automatic tone-mapping were
Tumblin and Turk [1999] with their LCIS operator. However, LCIS
sometimes produces odd-looking images, which bear little rela-
tion or resemblance to the original scene brightnesses. More re-
cent operators by Ashikhmin [2002], Fattal et al. [2002], Reinhard
et al. [2002], Durand & Dorsey [2002], and Choudhury & Tum-
blin [2003], are much more successful in separating contrast differ-
ences that matter to vision from those that do not.
Regrettably, none of these techniques has been validated or sup-
ported by psychophysical research, so the resulting images remain
as the only evidence that these methods have any real validity for re-
producing our experience of an HDR scene on a low dynamic range
device. Most of the methods contain free parameters that must be
set by the user based on preference. Usually, little can be said about
the visual impact that these parameters have on the image in terms
of visibility, contrast, brightness, or human visual response in gen-
eral.
HDR displays offer two benefits in this area. The first, immedi-
ate benefit is to researchers, who until now have had no means for
the controlled display of dynamic HDR imagery in their studies.
Our HDR displays are already helping researchers to test out their
hypotheses regarding the effects of tone-mapping on HDR scenes.
The longer term benefit will be felt when HDR displays penetrate
the professional and eventually the consumer markets, reducing or
eliminating the need for tone-mapping. Although we expect tone-
mapping to continue to be a requirement for many types of out-
put, such as hard copy, the availability of HDR displays will likely
reduce the need for dynamic range compression for many critical
applications in the years to come.
3 Remarks on Human Perception
The human visual system has tremendous capabilities but also some
limitations. Some of these limitations are an integral part of the
theory underlying the HDR devices presented in this paper and the
following sections will describe these in detail. In general, our eye
has evolved to deal with the vast dynamic range available to us in
our daily environment, ranging from starlight to sunlight over at
least an eight order of magnitude luminance range. To cope with
this range, the eye uses a complex adaptation system. For the pur-
pose of this paper, we use a simple model of adaptation with two
time scales: those mechanisms working at time scale of the order of
minutes and those with a shorter time scale. The former are of little
interest from the point of view of HDR display development. The
latter are very interesting as they are the primary reason why current
displays cannot provide realistic representations of real world HDR
scenes. The eye can capture approximately 5 orders of magnitude
of dynamic range effectively simultaneously. No conventional dis-
play technology comes close to this. Yet, there are limitations to
this capability as described below.
3.1 Local Contrast Perception
While we can see a vast dynamic range across a scene, we are un-
able to see more than a small portion of it in small regions (cor-
responding to small angles). Different researchers report differ-
ent values for the threshold past which we cannot make out high
contrast boundaries, but most agree that the maximum perceivable
contrast is somewhere around 150 : 1 [Vos 1984]. Scene contrast
boundaries above this threshold appear blurry and indistinct, and
the eye is unable to judge the relative magnitudes of the adjacent
regions.
761
1e-06
1e-05
0.0001
0.001
0.01
0.1
1
-2
-1.5
-1
-0.5
0
0.5
1
1.5
2
Relative Intensity
Angle (in degrees)
Point Spread Function of the Human Eye
Figure 2: The point spread function of the human eye according to
Moon&Spencer [1945].
This inherent limitation can be explained locally by the scatter-
ing properties of the eye. From Moon & Spencer’s original work
on glare [1945], we know that any high contrast boundary will scat-
ter at least 4% of its energy on the retina to the darker side of the
boundary, obscuring the visibility of the edge and details within a
few degrees of it (Figure 2). If the contrast of an edge is 25 : 1, then
details on the darker side will be competing with an equal amount
of light scattered from the brighter side, reducing visible contrast by
a factor of 2 in the darker region. When the edge contrast reaches a
value of 150 : 1, the visible contrast on the dark side is reduced by
a factor of 12, rendering details indistinct or invisible.
However, we cannot claim high contrast content has no effect
clearly it does. An observer will notice when one region is much
brighter than another, both by the challenge it creates in viewing
the boundary, and by the accommodation that goes on when shift-
ing from side to side. When the threshold is very large, observers
may even experience discomfort as they attempt to see detail near
a bright source, as any driver knows from their nighttime travels.
A photographic print of oncoming headlights is merely an allusion
to the real experience – it cannot duplicate the visceral experience
of glare, or reproduce the effect it has on a human observer. It is
exactly this kind of experience that an HDR display can uniquely
reproduce.
The HDR display technology described in this paper only ex-
ploits the inability of humans to see detail in the immediate vicinity
of a high-contrast boundary; it makes no assumptions about our
overall response to varying brightnesses. Relative (and even ab-
solute) brightnesses are maintained, and edges will be reproduced
exactly when they are below the maximum contrast of the front dis-
play – about 400 : 1 in our current prototype. Only when this range
is exceeded is some fidelity lost near high contrast boundaries, but
this effect is well below the detectable threshold, and has not been
visible in any of our experiments [Seetzen et al. 2003].
3.2 Just Noticeable Difference Steps vs. Contrast
and Bit Depth
A key question to ask in designing any display system is: how many
distinct input/output levels are necessary to cover the desired range
without banding or similar quantization artifacts? For conventional
displays, this question is often answered by considering a single
viewer adaptation level and the number of bits required to represent
suitable steps on a particular gamma curve. This may be adequate
if the dynamic range being considered is small, but fails when a
display is capable of levels much brighter and much darker than
ambient.
To answer this question in the context of a HDR displays, we
turn to psychophysical research in the area of Just Noticeable Dif-
ferences. A JND is the smallest detectable luminance difference at
a given luminance level. Visual psychologists have established a
complete theory for calculating JNDs over the luminance range rel-
evant for our HDR displays [Barten 1992; Barten 1993], which al-
lows us to establish the useful luminance level of the HDR display.
Adding a JND to a particular luminance level effectively defines
the next useful step on the luminance scale of the display since it is
clearly redundant to provide addressable luminance levels between
those two levels if the eye cannot perceive any difference.
0.01
0.1
1
10
100
1000
10000
100000
0
200
400
600
800
1000
1200
Luminance in cd/m
2
JND index
# Just Noticeable Difference Levels for Different Maximum Luminance Levels
Figure 3: The number of just noticeable difference (JND) steps
for different maximum intensities according to the model by
Barten [1992,1993].
Based on Barten’s original work, an analytical formula for JNDs
was derived for a DICOM standard [2001] (see Figure 3). This
model predicts 962 JND steps in the luminance range of the pro-
jection based HDR display described in Section 4 and 1139 JND
in the luminance range of the LED based HDR display (Section 5).
Our goal therefore is to reproduce at least this many steps on each
display from the darkest to brightest output level, and in both cases
our step sizes are maintained well below a JND throughout the lu-
minance range. In the following sections we will use the notion of
JNDs instead of contrast and bit depth to characterize the perfor-
mance of the HDR systems, but continue to use these more familiar
terms for the individual components (LCD and projector) that make
up the HDR display.
Recently, Muka and Reiker [2002] have argued that, for conven-
tional displays with a typical dynamic range of 300 : 1 or so, an
8-bit representation of images is sufficient for medical diagnosis.
They argue that the difference between an 8-bit digital display and
a 10-bit or higher bit depth is minimal, and perhaps not noticeable
at all. However, as the range of displayable luminances increases,
so does the number of JND steps required to cover that range, which
is reflected in the numbers presented above.
4 System 1: Projector-based Display
As outlined in the introduction, the first HDR display system we
built modulates the image from a projector with a transmissive
LCD. This system is detailed in the following.
762
4.1 Hardware Setup
In a conventional LCD, two polarizers and a liquid crystal are used
to modulate the light coming from a uniform backlight, typically a
fluorescent tube assembly. The light is polarized by the first polar-
izer and transmitted through the liquid crystal where the polariza-
tion of the light is rotated in accordance with the control voltages
applied to each pixel of liquid crystal. Finally, the light exits the
LCD by transmission through the second polarizer. The luminance
level of the light emitted at each pixel is controlled by the polariza-
tion state of the liquid crystal. It is important to point out that LCDs
cannot completely prevent light transmission - even at the darkest
state of a pixel, light is emitted and as such the dynamic range of an
LCD is defined by the ratio between the light emitted at the bright-
est state and the light emitted in the darkest state. For a high end
LCD, this ratio is usually around 300 : 1, with monochromatic spe-
cialty LCDs (e.g. those for medical imaging) going up to 700 : 1.
The luminance range of the display can easily be adjusted by con-
trolling the brightness of the backlight, but the dynamic range ratio
will remain the limiting factor. In order to maintain a reasonable
‘black’ level of about 1cd/m
2
, the LCD is thus limited to a maxi-
mum brightness of about 300cd/m
2
.
The basic modification introduced by the HDR technology in-
volves inserting a second light modulator and increasing the bright-
ness of the backlight. These two modulators in series provide an ex-
tremely dark state with a very low light emission, which then makes
it possible to increase the brightness of the backlight dramatically
without losing the ‘black’ state. Optically, this series of modulators
results in multiplication of the individual dynamic ranges.
For the projector-based HDR display presented in this paper, the
backlight and the first modulator are combined into a single DLP
using a Digital Mirror Device with a dynamic range of 800 : 1.
The three central components of the HDR display are then the pro-
jector, the LCD and the optics that couple the two. Using these
components, each image on the HDR display is the result of modu-
lated light coming from the projector which is directed onto the rear
of the transmissive LCD by the optics system, modulated a second
time by the LCD, and properly diffused for viewing.
The projector used in the HDR display is an Optoma DLP
EzPro737 digital mirror projector. To reduce unnecessary light loss,
we have removed the color wheel from the projector, resulting in
a monochrome display system with a threefold increase in bright-
ness due to the absence of the color filters. New control electronics
have been integrated into the commercially available projector to
re-synchronize it in absence of this color wheel.
The LCD panel is a 15” XGA color LCD made by Sharp (Sharp
LQ150X1DG0). It is driven by an EarthVision AD2 LCD con-
troller, which allows a direct VGA connection. The LCD panel has
been separated from the conventional backlight and all of the opti-
cal layers behind the display have been removed to create a trans-
missive image modulator.
The optics used in the HDR display include the conventional
projection lens of the EzPro projector, and a Fresnel lens directly
behind the LCD display to collimate the projected light into a nar-
row viewing angle for maximum brightness of the HDR display and
to avoid color distortion due to diverging light passing through the
color filters of the LCD. Finally, a standard LCD diffuser has been
used to redistribute the collimated light into a reasonable viewing
angle.
All three components have been installed in a single housing
with appropriate alignment mechanisms to create a close match-
ing of the DLP and LCD pixels. The alignment can be fine-tuned
through the controls of the DLP projector. However, a perfect
match is impractical as alignment at the sub-pixel level is hard to
achieve and almost impossible to maintain. To avoid moir
´
e patterns
and alignment artifacts associated with even a minor misalignment,
we have deliberately blurred the projector image and compensate
for that blur in the LCD image as described in the following sec-
tion.
LCD
Fresnel Lens and Diffuser
Projector
LCD
Controller
Dual-VGA Graphics
Card in PC
Figure 4: Top: schematic of the HDR display including the pro-
jector, LCD and optics. Bottom: actual photograph of the display.
Both projector and LCD are driven by a single dual-VGA graphics
card.
Using this configuration, the light output of each pixel of the
HDR display is effectively the result of two modulations, first by
the DLP and then by the LCD pixel, along the same optical path.
The upper boundary of the dynamic range results from full trans-
mission of both pixels (i.e. the 255
th
level on both modulators),
and the lowest boundary from the lowest possible transmission of
both modulators (i.e. the 0
th
level on both modulators). Since the
DLP has a dynamic range of 800 : 1 and the LCD a dynamic range
of 300 : 1, the theoretical dynamic range of the HDR display is
240, 000 : 1. Imperfections in the optical path introduce noise that
reduces the dynamic range to a measured 54, 000 : 1. The lumi-
nance values matching these boundaries are a result of the bright-
ness of the projector and the transmission of the LCD. In this case,
the Optoma EzPro737 is rated at 1200 Lumens, or approximately
2400 Lumens once the RGB color filters are removed (since each
filter for red, green and blue eliminates approximately 2/3 of the
incoming light). The Sharp LCD panel has a measured transmis-
sion of approximately 7.6% in the white state (this is quite high for
an LCD since even the theoretical maximum for a color LCD with-
out any losses is only 16% due to the light reduction of 50% at the
polarizer and another 66% due to the RGB color filter). Assuming
that the light emitted by the HDR display is diffused across a solid
angle ω, the maximum luminance is then given by:
L
max
=
Φ
max
where A is the area of the LCD and Φ
max
is the maximum outgo-
ing flux. In the HDR display prototype, the flux is approximately
182 Lumens (2400 Lumens ×7.6%). The area A is the area of the
763
15” LCD (697cm
2
) and the solid angle of diffusion ω is approx-
imately 0.66sr (40
diffusion horizontally, 15
vertically). The
maximum luminance for this particular configuration is then ap-
proximately 3, 956cd/m
2
. We actually measured a luminance of
2, 700cd/m
2
Lumens. The theoretical minimum luminance is less
than 0.01cd/m
2
, while our measurements resulted in a value of
0.05cd/m
2
. Clearly, a shift of this range toward even higher lumi-
nance values would be possible with a brighter projector or a more
transmissive LCD. Unlike a standard low dynamic range display,
even an order of magnitude increase of the maximum luminance
would not significantly reduce the quality of the ‘black’ state since
1cd/m
2
is still a very satisfying ‘black’, especially if other parts of
the image contain very high luminance values.
Within that luminance range, a very large number of differ-
ent combinations of output settings for the DLP and LCD can be
achieved. If both systems were linear 8-bit devices then the total
number of combinations would be 256
2
, over 17,000 of which are
distinct. Due to the non-linear gamma of each system, the actual
range of distinct addressable steps is different, but still significantly
larger than what is needed to display the 962 JND steps necessary
to provide all visible and distinguishable luminance steps in the
measured luminance range of the system (including all losses) of
0.05cd/m
2
to 2, 700cd/m
2
.
4.2 Driving the Projector Display
To correctly render HDR images on this display, we need to analyze
the image formation process of the system. Let us assume for the
moment that both the projector and the LCD panel are perfectly
linear, and that both have the same dynamic range. For now, let
us also neglect the blurring of the projector image. Under these
assumptions, we can achieve the target intensity by normalizing the
intensity range of the HDR image to 0 . . . 1, and using the square
root of this normalized intensity to drive both the projector and the
LCD panel. This even split between pixel values on the projector
and the LCD panel is preferable to a scenario where one value is
very large and the other is very small, since quantization artifacts
are relatively larger for small values.
In reality, neither the projector nor the LCD have a linear re-
sponse, and we also need to compensate for the blurring of the pro-
jector image. We do this in the following way: we choose a simple
estimate of what the projector intensity should be, and then simu-
late the effect of response function and blurring. Finally, the pixel
values of the LCD panel are chosen such that they compensate for
these effects.
LCD
projector
1
2 3
4
5 6
I
I
p
1
I
r
1
1
(
I)
p
1
I
r
1
2
(
I
p
1
I
)
I
Figure 5: Rendering algorithm for the projector-based display.
The complete rendering algorithm then works as follows (also
see Figure 5): we take the square root of the original HDR image
with intensity I (1). The resulting image (2) represents the target
intensity
I for the projector. We map these intensities into projec-
tor pixel values by applying the inverse of the projector’s response
function r
1
(3). The projector now produces an image of intensity
r
1
(r
1
1
(
I)) =
I, except that the image is actually blurred ac-
cording to p
1
, the projector’s point spread function (PSF). To simu-
late this blurring, we convolve the projector intensities with the PSF
(4) and divide the result out from the original HDR image to get the
target LCD transparency (5). For the final pixel values of the LCD,
we apply the inverse of the panel’s response function r
2
(6).
An exposure sequence of the PSF p
1
is depicted in Figure 6.
Note the vertical lines visible in images with larger exposure times.
These are the RGB subpixels of the LCD panel. In order to speed
up the computation, we do not use the measured PSF directly, but
fit a tensor-product Gaussian to p
1
. We usually set the focus of the
projector such that the fitted Gaussian has a standard deviation of
about 2-3.5 pixels, so that we can use a 2D separable filter of width
13 for the convolution.
Figure 6: Point spread function of the projector.
These image processing steps are possible both completely in
software and using recent programmable graphics hardware (graph-
ics processing unit, GPU). The convolution is the most difficult part
of the GPU implementation, but with the separable approximation
it can be readily implemented as a pixel shader on both the latest
ATI and NVIDIA chips. With this approach we achieve frame rates
of 30 frames per second or more on current hardware.
0.0001
0.001
0.01
0.1
1
0
50
100
150
200
250
Response (% of maximum)
Pixel value
Response function - Projector display
LCD panel transparency
Projector intensity
Figure 7: Response function of both the LCD panel and the DLP
projector in the projector-based display.
Figure 8 shows the results of this image factorization on a por-
tion of Debevec’s Stanford Memorial Church HDR photograph. On
the left side, you can see a grayscale image that corresponds to the
square root of the original intensity values. Convolving that im-
age with the PSF of the projector yields the center image. This
is the predicted image that will be produced by the projector. Fi-
nally, the right image is the color LCD panel image that corrects
for the blurriness of the projector. It is interesting to note that the
LCD panel image is essentially an edge-enhanced image with low
frequency components attenuated or removed. This is particularly
noticeable for the widths of the window frames. Interestingly, the
algorithm that we apply here is very similar in principle to a local
tone mapping operator. This means that our LCD panel image is
almost a tone mapped version of the original HDR image, although
our method is clearly not designed for that purpose.
4.3 Discussion of the Projector Display
The projector based HDR display hardware provides a tool to
present high quality HDR images but has several drawbacks. In ad-
764
Figure 8: Factoring an HDR photograph for our projector-based
display. Left: square root of the intensity. Center: blurred image
which is predicted to be the image generated by the de-focused pro-
jector. Right: edge enhanced LCD panel image that corrects for the
blurriness of the projector image.
dition to the obvious form factor problem due to the optical length
required by the projector, the power consumption, cost, thermal
management and video bandwidth requirements are high compared
to a conventional display.
High power consumption and the resulting thermal management
requirements are a consequence of the image creation mechanism
inside the projector. Unlike a cathode ray tube (CRT) display, where
light is created only in the regions of the image that are supposed to
be bright, an LCD or DLP projector creates a uniform light distri-
bution that is then modulated by the LCD or DLP mirror chip. The
power consumption of an LCD or DLP projector is thus indepen-
dent of the image and always very high as there has to be enough
light produced by the lamp such that a full screen ‘white’ can be
shown. Combined with the low modulation efficiency of the LCD
or DLP this causes the high power consumption. In the HDR dis-
play the situation is worse than in a conventional, single-modulator
display. The lamp of the projector has to emit enough light to allow
a full screen image at the highest possible brightness of the HDR
display. To achieve 10, 000cd/m
2
on a 15” screen we would need
an outgoing flux of approximately 500 Lumens (see Section 4.1).
Even with a very high transmission LCD this requires at least 5000
Lumens to be emitted from the projector. In the prototype pre-
sented in Section 4.1 the color wheel/filter of the projector has al-
ready been removed to reduce the losses in the projector but even so
the modulation efficiency of the projector is slightly less than 50%.
The lamp thus has to produce in the order of 10, 000 Lumens. Yet,
in almost all HDR images the area that is actually at such a high
brightness of 10, 000cd/m
2
is very small. In fact, a random selec-
tion of 100 HDR images indicated that average HDR images have
less than 10% of the image content in the high luminance range
(above 3000cd/m
2
) and that the average luminance over all im-
ages was less than 800cd/m
2
for indoor scenes and 2, 100cd/m
2
for outdoor scenes. The projector HDR display consequently cre-
ates a factor of between 12.5 and 4.75 too much light at any given
time.
The projector is also the cause of the high bandwidth require-
ments. Even though the image projected by the projector onto the
back of the LCD is blurred, the projector itself is still a high res-
olution display which requires high resolution input data. As a re-
sult, the projector-based HDR display needs a high resolution video
stream going to the LCD and a similar size video stream to the pro-
jector. This creates a requirement for a dual output graphic card and
imposes limits to the frame rate of the display due to the computa-
tional requirements.
Finally, the cost of a high brightness projector is very high which
makes this version of the HDR display unsatisfactory for commer-
cial purposes. Cost also presents a barrier to larger screen sizes
as the brightness requirements increase linearly with area, and the
cost curve for projectors is very steep with brightness (while the
step from 2, 000 Lumens needed for a 15” display to a 4, 000 Lu-
mens projector for a 20” display is only twice as high, the next step
from 20” to 40” TV size would require a 15, 000 Lumens projector
at a price that is over 20 times higher than that of a 4, 000 Lumens
projector).
Yet, for research applications, the display is a valuable tool. The
high cost is in part due to the use of fully finished consumer prod-
ucts instead of individual components, but this also makes it possi-
ble to assemble the system without significant development of cus-
tom electronics. Since the drawbacks mentioned above in no way
diminish the actual image quality, the projector-based HDR display
provides researchers with a relatively simple to build solution with
very high image quality.
5 System 2: LED-based Display
As seen in the discussion of the projector-based HDR display in
Section 4.3 there are significant obstacles to overcome.
To realize the dream of television or computer displays present-
ing images that look indistinguishable from the real world, it is not
sufficient to merely show images with the appropriate luminance
range and resolution; it is also necessary to make a commercially
viable system that achieves these higher quality images within to-
day’s hardware and software infrastructure and market price points.
The version of the HDR display described in this section retains
the high image quality of the projector display and overcomes the
commercialization barriers: power, thermal, cost, form factor and
infrastructure demands on the graphics card.
5.1 Hardware Setup
We have already seen in Section 4.1 that software correction can
compensate for a low resolution of the rear image of the HDR dis-
play. It is important to realize that this correction works perfectly, as
long as the local image contrast does not exceed the dynamic range
of the front modulator. From the psychology theory presented in
Section 3 we can establish the largest size of a rear image pixel. For
this version of the HDR display we have used light emitting diodes
(LED) at the largest possible size allowed by the veiling luminance
effect which has been validated previously through experimental
tests [Seetzen et al. 2003].
A prototype has been build using 12mm Lumiled Luxeon 1 Watt
white LEDs (LXHL- PWO1) on a hexagonal close-packing matrix
where each LED is individually controlled over its entire dynamic
range with 1024 addressable steps. 760 LEDs have been mounted
behind an 18.1” L.G. Philips LCD with a 500 : 1 dynamic range
and 1280 × 1024 resolution. On full screen white/black the max-
imum luminance is measured as 8, 500cd/m
2
and the minimum
luminance is zero, since all LEDs are off. The minimum lumi-
nance is less than 0.03cd/m
2
on a checkerboard pattern larger than
20mm. As mentioned in Section 3.2, the Barten model [Barten
1992; Barten 1993] predicts 1139 JNDs for this luminance range.
The system is capable of displaying images at video rates.
5.2 Driving the LED Display
The principal rendering algorithm for the LED-based system is
quite similar to that for the projector-based display, as shown in
Figure 9. The primary difference between the two display systems
from a rendering perspective is that the PSF of an LED has a much
wider support than the one for a pixel of the projector. Also of
importance is the fact that the LEDs are arranged on a hexagonal
grid rather than a rectangular grid. These differences have two con-
sequences. Firstly, because of the wider support of the PSF, it is
advisable to come up with a better way to choose the LED val-
ues. Since the supports of the PSFs for neighboring LEDs overlap,
determining the optimal LED value is essentially a de-convolution
765
problem, as explained below. Secondly, because of both the hexag-
onal geometry and wider support of the PSF, the convolution (4)
has to be implemented differently.
LED
LCD
1
2 2a
4
5 6
3
r
1
2
(
I
p
1
I
L
)
I
I
I
L
p
1
I
L
I
p
1
I
L
r
1
1
(I
L
)
Figure 9: Rendering algorithm for the LED-based display.
We address the first issue by adding an additional stage (2a) to
derive the target intensities I
L
for every individual LED. To this end
we first down-sample the image to the resolution of the LED array,
and then solve for the values, taking the overlap of the PSFs into
account. This is essentially a de-convolution problem, the full solu-
tion of which would require solving a sparse linear equation system
with as many unknowns as there are LEDs. This is not an option for
interactive applications, and furthermore de-convolution algorithms
are known to be numerically unstable. We can approximate a so-
lution with a single Gauss-Seidel iteration over neighboring LED
pixels. This amounts to a local weighted average of neighboring
LED target values, where some of the weights are negative. In our
design, these LED values (2a) can be forwarded directly to the con-
troller electronics, which correct for nonlinearities in the LED re-
sponse in hardware.
As before, we rely on the LCD panel to compensate for any dif-
ferences between the LED values and the target image. To this end,
we need to forward-simulate the low-frequency image (4) gener-
ated by the LED panel in order to derive the LCD pixel values.
We use two different approaches for software and hardware imple-
mentations. With GPUs, we use a splatting approach, and simply
draw screen aligned quadrilaterals with textures of the PSF into the
framebuffer. Alpha blending is used to accumulate the results. In
software, we approximate the measured PSF with a Gaussian, and
implement the reconstruction by convolving a low resolution image
with the approximate PSF, and then up-sampling to the full resolu-
tion.
0.001
0.01
0.1
1
0
50
100
150
200
250
Response (% of maximum)
Pixel value
Response function - LED display
LCD panel transparency
Figure 10: Response function of the LCD panel in the LED-based
display. The response of the individual LEDs (not shown here) is
designed to be linear.
Figure 11 shows the factorization of an HDR image into LED
component and LCD panel component. Due to the wider support
of the PSF of the LEDs compared to the PSF of a projector pixel in
the first setup, the LED image is even more low-pass filtered than
before. As a result, the compensation performed in the LCD panel
is more significant, and is visible in the right image.
Figure 11: Factorization of an HDR photograph for the LED-based
display. Left: LED contribution as a result of convolving LED val-
ues with the LED point spread function. Right: edge enhanced
LCD panel image that corrects for the blurriness of the LED image.
5.3 Discussion of the LED Display
The use of LEDs overcomes the power consumption and thermal
problem as a result of intelligent light production, and it eliminates
other commercialization barriers outlined in Section 4.3. Form fac-
tor is no longer an issue as the LED matrix can be the same thick-
ness as a conventional LCD backlight. The video bandwidth re-
quirements are dramatically reduced due to the lower resolution of
the LED matrix. As a result, only a few hundred 8-bit values are
needed over a standard image, and these are added to the front of
the DVI video stream going to the LCD and stripped off by the
controller hardware inside the display. Cost remains an issue but
significantly less so than with the projector system.
One slight disadvantage is that the rendering algorithm for the
LED-based display is computationally more demanding than the al-
gorithm for the projector-based display due to the larger support of
the PSF. On a GeForce FX we currently achieve about 10 fps. for a
fullscreen image factorization into LCD and LED components. The
rendering time is mostly limited by having to work around limita-
tions in the support of floating point framebuffers and textures on
current GPUs. The next generation of GPUs will support a more
complete set of operations, including blending and bi-linear inter-
polation, which should improve the performance of our algorithm
by a factor of 3-4.
6 Applications
We have developed four simple applications to test our display tech-
nology, and to demonstrate its potential in a number of application
domains. The first one is a simple HDR image viewer that works
with both displays (Figure 12, top left). It allows the user to load an
HDR image and show it while interactively adjusting the exposure
settings (i.e. the absolute scale of intensity). This application was
also used to generate the images in Figure 1. The three images to
the right of that figure show a color-coded comparison of an origi-
nal radiance map, an HDR photograph taken off our projector-based
display, and finally a photograph taken off a standard LCD screen.
The intensities of the photograph from the HDR display are similar
but not identical to the values in the original radiance map. The
differences are mostly due to imperfections in both the calibration
of the display to absolute intensities, and in the image acquisition
process. Clearly both intensity and dynamic range of our display
are vastly superior to the standard monitor.
766
Figure 12: Screen photographs of the different applications we implemented. The exposure times of the two images in each pair differ by
4 stops. Top left: HDR image viewer. Top right: interactive rendering of measured BRDFs. Bottom left: volume rendering. Bottom right:
medical image viewer.
The second application we developed is related to interactive
photorealistic rendering. We modified a DirectX application for
displaying BRDFs measured with linear light source reflectome-
try [Gardner et al. 2003], and replaced its tone mapping step with
the rendering algorithm for our display (Figure 12, top right). Other
interactive applications that can render into floating point buffers
can easily be modified in a similar fashion. As pointed out in
Section 5.2, the rendering performance on current GPUs is limited
due to missing features such as alpha blending with floating point
framebuffers and lack of bi-linear interpolation for floating point
textures. These issues should be resolved in upcoming generations
of GPUs.
Based on a similar principle, we have built a simple volume ray-
caster that runs on a GPU (Figure 12, bottom left). It allows for
operations such as rotation, slicing, and adjustments to the transfer
function. Both the actual volume rendering algorithm and the pro-
cessing for our displays is implemented in a single Cg shader [Mark
et al. 2003].
Finally, a simple medical imaging viewer and browser has been
implemented to allow radiologist to view medical images in high
dynamic range (Figure 12, bottom right).
7 Conclusions and Future Work
In this paper we have presented two designs for HDR displays,
one based on a projector setup and one based on an LED array.
The two displays we developed have dynamic ranges of well be-
yond 50, 000 : 1, and a maximum intensity of 2700cd/m
2
and
8500cd/m
2
, respectively. We have described both hardware and
software aspects of these display systems, and have described a
number of applications for this kind of technology.
There are plenty of opportunities for future work. On the hard-
ware side, the image quality of the LED-based HDR display can
be further enhanced by replacing each white LED with a triplet
of red, green and blue LEDs or a larger number of colored LEDs
if more primaries are desired (e.g. red, blue, blue-green, yellow-
green). Color LEDs have very sharply defined spectral emission
pattern as opposed to the broad band emission of fluorescent tubes
used in conventional LCD backlights. This creates pure primaries
and allows for a significant increase in the display color gamut. A
conventional LCD achieves approximately 66% of the NTSC color
gamut while a tri-color LED based system can achieve 98% with
additional red and blue outside the NTSC gamut [Ohtsuki et al.
2002].
The actively controlled matrix of LEDs can also be used to very
easily implement backlighting schemes proposed by the industry
such as flashing the backlight in segment in sync with the LCD re-
fresh to reduce motion blur artifacts [Fisekovic et al. 2001]. Like-
wise, the LED array simplifies problems of global luminance non-
uniformity found in conventional displays resulting from a non-
uniform light output of the fluorescent tube backlight, lifetime prob-
lems due to backlight failure, and many other limitations of a con-
ventional LCD backlight.
At this point the calculation of the rear and front image layer are
executed by the graphics card but standardized calculations of this
kind can easily be done by dedicated hardware inside the display
as soon as standards for floating point video signals (such as exten-
sions to the DVI standard) become available. This will reduce the
load on the graphic card, allow software to directly send floating
point data to the display, and have the display handle the necessary
computations for blur correction.
In addition to these directions for further developing the hard-
ware, there are a number of potential applications of this display
technology in computer graphics, visualization, human-computer
interaction, and perception research. Commercial applications in-
clude proofing for the film and special effects industries, since the
dynamic range of film significantly exceeds what can currently be
shown on standard displays, and eventually even home theater. For
volume rendering in particular, the higher dynamic range might
open up possibilities for completely new rendering algorithms. We
also plan to explore new interaction metaphors, such as use of HDR
imaging in user interfaces. The extra brightness should be useful
for directing the user’s attention to important events. Finally, an
interesting research direction is to expand the work on human per-
ception research in order to challenge and expand on assumptions
that have been made in the context of traditional display technology.
Other researchers have already used our display to run perceptual
767
comparisons of tone mapping operators.
8 Acknowledgments
We would like to thank Chris Tchou for his work on the initial in-
teractive rendering demo for the linear light source reflectometry
data [Gardner et al. 2003]. Paul Debevec made the Daguerreotype
reflectance data from the same project available to us (see Figure 12
and the video). The HDR photographs used throughout this paper
were also provided by Paul Debevec, while the CT data set was
provided by Klaus Engel. Thanks to the anonymous reviewers for
their valuable comments.
This work was in part supported by NSERC, IRIS/Precarn, and
the BC Advanced Systems Institute.
References
ASH IKH MIN , M. 2002. A tone mapping algorithm for high contrast
images. In Proc. of Eurographics Workshop on Rendering 2002,
145–156.
BA RTEN , P. 1992. Physical model for the contrast sensitivity of
the human eye. In Proc. SPIE, vol. 1666, 57–72.
BA RTEN , P. 1993. Spatio-temporal model for the contrast sensi-
tivity of the human eye and its temporal aspects. In Proc. SPIE,
vol. 1913-01.
CHI U, K., HER F, M., SHIRLEY, P., SWAMY, S., WAN G, C., AND
ZIM MER MAN , K. 1993. Spatially nonuniform scaling functions
for high contrast images. In Proc. Graphics Interface ’93, 245–
253.
CHO UDH URY, P., A ND TUMBLIN, J. 2003. The trilateral filter for
high contrast images and meshes. In Proc. of the Eurographics
Symposium on Rendering, 186–196.
DEB EVE C, P., AN D MA LIK , J. 1997. Recovering high dynamic
range radiance maps from photographs. In Proc. of ACM SIG-
GRAPH ’97, 369–378.
DICOM. 2001. Digital Imaging and Communications in Medicine
(DICOM). http://www.medical.nema.org/dicom/2001.html,
ch. Part 14: Grayscale Standard Display Function.
DUR AND , F., AND DORSEY, J. 2002. Fast bilateral filtering for
the display of high-dynamic-range images. ACM Trans. Graph.
(special issue SIGGRAPH 2002) 21, 3, 257–266.
FATTAL, R., LISC HIN SKI , D., AND M.WERMAN. 2002. Gradient
domain high dynamic range compression. ACM Trans. Graph.
(special issue SIGGRAPH 2002) 21, 3, 249–256.
FIS EKOVI C, N., NAU TA, T., CORN ELISSEN, H., A ND BRUI NIN K,
J. 2001. Improved motion-picture quality of AM-LCDs using
scanning backlight. In IDW 2001 Proceedings, 1637–1640.
GAR DNE R, A., TCHOU, C., HAWKI NS, T., A ND DEB EVE C, P.
2003. Linear light source reflectometry. ACM Trans. Graph.
(special issue SIGGRAPH 2003) 22, 3, 749–758.
KAN G, S. B., UYT TEN DAE LE, M., WINDER, S., A ND SZELI SKI ,
R. 2003. High dynamic range video. ACM Trans. Graph. (spe-
cial issue SIGGRAPH 2003) 22, 3, 319–325.
LAR SON , G. W., RUSH MEI ER, H., AN D PIATKO , C. 1997.
A visibility matching tone reproduction operator for high dy-
namic range scenes. IEEE Trans. on Visualization and Computer
Graphics 3, 4, 291–306.
MAN N, S., AN D PICARD , R. 1994. Being ’undigital’ with digi-
tal cameras: Extending dynamic range by combining differently
exposed pictures. Tech. Rep. 323, M.I.T. Media Lab Perceptual
Computing Section. Also appears, IS&T’s 48th annual confer-
ence, Cambridge, MA, May 1995.
MAR K, W. R., GL ANVILLE, R. S., AK ELE Y, K., AND KILGARD,
M. J. 2003. Cg: A system for programming graphics hard-
ware in a C-like language. ACM Trans. Graph. (special issue
SIGGRAPH 2003) 22, 3, 896–907.
MIT SUNAGA, T., AN D NAYAR , S. K. 1999. Radiometric self
calibration. In Proc. of IEEE CVPR, 472–479.
MOO N, P., AND SPENCER, D. 1945. The visual effect of non-
uniform surrounds. Journal of the Optical Society of America
35, 3, 233–248.
MUK A, E., AND REIK ER, G. 2002. Reconsidering bit depth for
radiological images – is eight enough? In Proc. SPIE, vol. 4686,
177–188.
OHT SUK I, H., NAK ANI SHI , K., MO RI, A., SA KAI , S., YACHI ,
S., AND TIMMERS, W. 2002. 18.1-inch XGA TFT-LCD with
wide color reproduction using high power led-backlighting. In
Proc. Society for Information Display International Symposium,
1154–1157.
PATTA NA IK, S. N., FERWER DA, J. A., FAIR CHILD, M. D., AND
GRE ENB ERG , D. 1998. A multiscale model of adaptation and
spatial vision for realistic image display. In Proc. of ACM SIG-
GRAPH ’98, 287–298.
REI NHA RD, E., STARK , M., SHI RLE Y, P., AN D FERWERDA, J.
2002. Photographic tone reproduction for digital images. ACM
Trans. Graph. (special issue SIGGRAPH 2002) 21, 3, 267–276.
ROB ERTS ON, M., BOR MAN, S., AND STE VEN SON, R. 1999.
Dynamic range improvements through multiple exposures. In
Proc. of International Conference on Image Processing (ICIP)
’99, 159–163.
SCH LIC K, C. 1994. Quantization techniques for visualization of
high dynamic range pictures. In Proc. of Eurographics Workshop
on Rendering ’94, 7–20.
SEE TZE N, H., WHITEHEAD , L., AND WA RD, G. 2003. A high dy-
namic range display using low and high resolution modulators.
In Society for Information Display Internatiational Symposium
Digest of Technical Papers, 1450–1453.
TUM BLI N, J., AN D RUSH MEI ER, H. 1993. Tone reproduction for
realistic images. IEEE Computer Graphics and Applications 13,
6, 42–48.
TUM BLI N, J., A ND TURK, G. 1999. LCIS: A boundary hierar-
chy for detail-preserving contrast reduction. In Proc. of ACM
SIGGRAPH ’99, 83–90.
VOS , J. 1984. Disability glare - a state of the art report. CIE
Journal 3, 2, 39–53.
768
... However, these methods mostly target LDR images rather than HDR images. To render HDR images on dual-panel LCD displays, Seetzen et al. [27] proposed a HDR image rendering method. This method decomposed HDR images into two luminance layers using the square root of the image luminance channel. ...
Article
Full-text available
High dynamic range (HDR) displays based on liquid crystal panels require local dimming algorithms to reproduce content with high fidelity and HDR. However, most local dimming algorithms are developed by using hand-crafted features and most of them focus on low dynamic range images rather than HDR images. In addition, few local dimming algorithms can manage well the conflicting requirements of maintaining image quality while minimising power consumption. In this paper, we propose a genetic algorithm for backlight local dimming (GABLD) that generates images for the HDR display based on the given fitness function and the HDR content about to be displayed while considering the impact of power consumption and image quality. The effectiveness of the GABLD method is demonstrated in three aspects: the fidelity of reconstruction of HDR images, the power consumption, and the visualisation of HDR content on HDR displays. The results show that the proposed method has good performance in terms of both the image quality and power consumption, and it retains more details especially for areas with highlights when compared to traditional methods.
... The notion of Perceptual Displays is not necessarily a new concept. 3D displays & High Dynamic Range (HDR) displays introduced back in the 2000s are examples of the impactful technical hardware advancements towards building displays that are closer to what we see in the real world [4,5]. Although, trying to define & optimize "display user experience" leveraging physiological research has become more broadly used in the last decade [6,7]. ...
... Based on the size of the LED chip, the technology is divided into two parts: mini-LEDs and micro-LEDs. High dynamic range is one of the important features for next-generation displays which require high peak brightness and excellent dark state of the display system simultaneously [13,14]. This can be achieved in LCD by local dimming. ...
Chapter
Full-text available
Electronic displays have always amused people since their beginning from cathode-ray tube screens to modern thin panel displays. Quantum dot (QD) assisted displays are a new addition to the display family; promising more vivid colors and flexibility to modern displays. This chapter offers a comprehensive overview of QDs as a next-generation technology for displays. The unique size-dependent chemical, optical, electronic, and optoelectronic features of QDs make them the most suitable candidate to be used in modern displays owing to wide color gamut and high-power efficiency with lower barrier protection requirements. Herein, we extensively present the display applications of different types of QDs in the existing display technologies and their comparative analysis. Future scope along with the recycling aspects of display devices has also been covered within the sphere in this chapter.
... Based on these two methods, novel mini-LED backlight systems supporting spatial light intensity modulation are proposed to prevent the light crosstalk and to maintain the luminance and color uniformity of the zones 34,75-79 . On the software side, Seetzen et al. 80 proposed a local dimming algorithm (called maximum method algorithm 80 ) to systematically optimize the LCD transmittance and spatial profile of mini-LED backlight. The brightness of the mini-LEDs in each zone is determined by the brightest pixel in the zone. ...
Article
Full-text available
Liquid crystal displays (LCDs) and photonic devices play a pivotal role to augmented reality (AR) and virtual reality (VR). The recently emerging high-dynamic-range (HDR) mini-LED backlit LCDs significantly boost the image quality and brightness and reduce the power consumption for VR displays. Such a light engine is particularly attractive for compensating the optical loss of pancake structure to achieve compact and lightweight VR headsets. On the other hand, high-resolution-density, and high-brightness liquid-crystal-on-silicon (LCoS) is a promising image source for the see-through AR displays, especially under high ambient lighting conditions. Meanwhile, the high-speed LCoS spatial light modulators open a new door for holographic displays and focal surface displays. Finally, the ultrathin planar diffractive LC optical elements, such as geometric phase LC grating and lens, have found useful applications in AR and VR for enhancing resolution, widening field-of-view, suppressing chromatic aberrations, creating multiplanes to overcome the vergence-accommodation conflict, and dynamic pupil steering to achieve gaze-matched Maxwellian displays, just to name a few. The operation principles, potential applications, and future challenges of these advanced LC devices will be discussed.
Article
High dynamic range (HDR) displays with dual-panels are one type of displays that can provide HDR content. These are composed of a white backlight panel and a colour LCD panel. Local dimming algorithms are used to control the backlight panel in order to reproduce content with high dynamic range and contrast at a high fidelity. However, existing local dimming algorithms usually process low dynamic range (LDR) images, which are not suitable for processing HDR images. In addition, these methods use hand-crafted features to estimate the backlight values, which may not be suitable for many kind of images. In this work, a novel deep learning based local dimming method is proposed for rendering HDR images on dual-panel HDR displays. The method uses a Convolutional Neural Network (CNN) to directly predict backlight values, using as input the HDR image that is to be displayed. The model is designed and trained via a controllable power parameter that allows a user to trade off between power and quality. The proposed method is evaluated against seven other methods on a test set of 105 HDR images, using a variety of quantitative quality metrics. Results demonstrate improved display quality and better power consumption when using the proposed method compared to the best alternatives.
Article
Full-text available
Machine vision systems that capture images for visual inspection and identification tasks have to be able to perceive a scene under a range of illumination conditions. To achieve this, current systems use circuitry and algorithms that compromise efficiency and increase complexity. Here we report bioinspired vision sensors that are based on molybdenum disulfide phototransistors and exhibit time-varying activation and inhibition characteristics. Charge trap states are intentionally introduced into the surface of molybdenum disulfide, enabling the dynamic modulation of the photosensitivity of the devices under different lighting conditions. The light-intensity-dependent characteristics of the sensors match Weber’s law in which the perceived change in stimuli is proportional to the light stimuli. The approach offers visual adaptation with highly localized and dynamic modulation of photosensitivity under different lighting conditions at the pixel level, creating an effective perception range of up to 199 dB. The phototransistor arrays exhibit image contrast enhancement for both scotopic and photopic adaptation.
Article
Full-text available
With well-established methods for producing photo-realistic results, the next big challenge of graphics and display technologies is to achieve perceptual realism --- producing imagery indistinguishable from real-world 3D scenes. To deliver all necessary visual cues for perceptual realism, we built a High-Dynamic-Range Multi-Focal Stereo Display that achieves high resolution, accurate color, a wide dynamic range, and most depth cues, including binocular presentation and a range of focal depth. The display and associated imaging system have been designed to capture and reproduce a small near-eye three-dimensional object and to allow for a direct comparison between virtual and real scenes. To assess our reproduction of realism and demonstrate the capability of the display and imaging system, we conducted an experiment in which the participants were asked to discriminate between a virtual object and its physical counterpart. Our results indicate that the participants can only detect the discrepancy with a probability of 0.44. With such a level of perceptual realism, our display apparatus can facilitate a range of visual experiments that require the highest fidelity of reproduction while allowing for the full control of the displayed stimuli. (More information on project page: https://www.cl.cam.ac.uk/research/rainbow/projects/hdrmfs/)
Article
Full-text available
In this paper, we investigate visual similarity for high dynamic range (HDR) images. We collect crowdsourcing data through a web-based experimental interface, in which the participants are asked to choose one of the two candidate images as being more similar to the query image. Triplets forming the query-and-candidates sets are obtained by random sampling from existing HDR data sets. Experimental control factors include choice of tone mapping operator (TMO), choice of distance metric, and choice of image feature. The image features that we experiment with are chosen from the features that are commonly used in the usual low dynamic range setting including features learned via Convolutional Neural Networks. The set of image features also includes combined features where the combination coefficients are estimated using logistic regression. We compute correlations between human judgments and quantitative features to understand how much each feature contributes to visual similarity. Combined features yield nearly 84% agreement with human judgments when applied on tone mapped images. Though we observed that using common features directly on raw or linearly scaled HDR images yield subpar correlation estimates compared to using them on tone mapped HDR images, we did not observe significant effect due to the choice of TMO on the estimates. As an application, we propose an improvement to style-based tone mapping for more correctly imparting desired styles to HDR images with different characteristics.
Article
Full-text available
A "sample and hold" artifact is present in all AMLCDs causing motion blur even when very fast LC effects are used. A new backlighting principle for LCD-TV applications is proposed. It improves the motion picture quality tremendously by eliminating this artifact without need for an especially fast LC effect.
Article
The latest real-time graphics architectures include programmable floating-point vertex and fragment processors, with support for data-dependent control flow in the vertex processor. We present a programming language and a supporting system that are designed for programming these stream processors. The language follows the philosophy of C, in that it is a hardware-oriented, general-purpose language, rather than an application-specific shading language. The language includes a variety of facilities designed to support the key architectural features of programmable graphics processors, and is designed to support multiple generations of graphics architectures with different levels of functionality. The system supports both of the major 3D graphics APIs: OpenGL and Direct3D. This paper identifies many of the choices that we faced as we designed the system, and explains why we made the decisions that we did.
Article
The physical model for the spatial contrast sensitivity of the eye that we presented in a previous paper is extended to the temporal domain by making some assumptions about the temporal behavior of the photoreceptors and the lateral inhibition in the ganglion cells. In this way a complete spatio-temporal model is obtained that gives a simple explanation of the complicated spatio-temporal behavior of the eye. Spatio-temporal and temporal contrast sensitivity curves calculated with the model are compared with measurements published in literature.
Article
Based on observations made at our institution that diagnostic images could be read on cathode ray tube (CRT) displays controlled with 8-bit hardware, a reconsideration of the bit depth for primary interpretation of radiological images seemed in order. Using actual CRT performance and human visual system (HVS) models with target size, surround luminance and external noise parameters (detector, display and image), CRT luminance modulation as a function of bit depth is compared with the HVS detection threshold modulation. While best case HVS performance requires, at least, 10-bit control to avoid creating luminance artifacts, probable HVS performance is estimated when targets are small, surround luminance is not equal to target luminance and external noise is included as a mask. In this light, the HVS threshold modulation is elevated to such an extent that 8-bit hardware is sufficient. It is shown that when implemented in 8-bit space at the display, the DICOM display function standard creates additional noise and potentially, artifacts. Acceptable image display in an 8-bit space will be discussed with respect to display data representation alternatives such gamma space, which is based on the intrinsic (uncorrected) CRT display function.
Article
The contrast sensitivity of the human eye and its dependence on luminance and display size is described on the basis of internal noise in the visual system. With the addition of a global description of the optical MTF of the eye, a complete physical model is obtained for the spatial contrast sensitivity function. The results of this model are compared with measurements published by various authors.
Article
The long standing controversy about whether disability glare is essentially a matter of intra-ocular light scattering or also has a neural inhibitory component is discussed. The controversy now seems to be decided in favour of the stray-light-only explanation. Next the angular dependency of the stray-light veil is discussed and a complete light profile from O degree to 100 degree angular distance to the point glare source is given. This stray-light could be attributed mainly to approximately equal scatter contributions from cornea, crystalline lens and retina. The main uncertainty exists in the age effect: experimental data not only show considerable spread, but they never cover the whole angular domain. The tentative introduction of an age factor in the stray-light formula must therefore be considered with due reserve.
Article
We develop 18.1-inch XGA TFT-LCD using high power Light Emitting Diodes (LEDs) as light sources for backlighting. By matching the transmissive spectra of color-filter to LEDs radiation, the higher brightness and the wider color reproduction of 95%-NTSC are achieved on the front-of-screen.
Article
Citation PARRY MOON and DOMINA EBERLE SPENCER, "The Visual Effect of Non-Uniform Surrounds," J. Opt. Soc. Am. 35, 233-247 (1945) http://www.opticsinfobase.org/josa/abstract.cfm?URI=josa-35-3-233
Article
Based on observations made at our institution that diagnostic images could be read on cathode ray tube (CRT) displays controlled with 8-bit hardware, a reconsideration of the bit depth for primary interpretation of radiological images seemed in order. Using actual CRT performance and human visual system (HVS) models with target size, surround luminance and external noise parameters (detector, display and image), CRT luminance modulation as a function of bit depth is compared with the HVS detection threshold modulation. While best case HVS performance requires, at least, 10-bit control to avoid creating luminance artifacts, probable HVS performance is estimated when targets are small, surround luminance is not equal to target luminance and external noise is included as a mask. In this light, the HVS threshold modulation is elevated to such an extent that 8-bit hardware is sufficient. It is shown that when implemented in 8-bit space at the display, the DICOM display function standard creates additional noise and potentially, artifacts. Acceptable image display in an 8-bit space will be discussed with respect to display data representation alternatives such gamma space, which is based on the intrinsic (uncorrected) CRT display function.© (2002) COPYRIGHT SPIE--The International Society for Optical Engineering. Downloading of the abstract is permitted for personal use only.
Conference Paper
A new method is presented that takes as an input a high dynamic range image and maps it into a limited range of luminance values reproducible by a display device. There is significant evidence that a similar operation is performed by early stages of human visual system (HVS). Our approach follows functionality of HVS without attempting to construct its sophisticated model. The operation is performed in three steps. First, we estimate local adaptation luminance at each point in the image. Then, a simple function is applied to these values to compress them into the required display range. Since important image details can be lost during this process, we then re-introduce details in the final pass over the image.