Perceptual limit to display resolution of images as per visual acuity

Abstract

Achieving ultimate visual realness of natural images on a display requires high resolution, so that artifacts due to finite image resolution are undetectable. An image resolution of 30 cycles/degree (cpd) or one pixel/arc-minute is often used as the criterion for viewing conditions when assessing displayed image quality. It is reasoned that if the pixel size is smaller than the separable angle of normal vision (20/20), the pixel structure is invisible and doesn't negatively affect image quality. However, it is not clear whether 30 cpd resolution is adequate to prevent seeing artifacts, especially for observers with better than 20/20 vision. We conducted experiments to find the threshold resolution of natural images and its dependence on visual acuity. Three objects were used; each object was presented 60 times at 5 resolutions (19.5, 26, 39, 52, or 78 cpd) next to the same image at a resolution of 156 cpd. Forty-five observers with visual acuity of 20/20 or better were asked to make a forced-choice distinction between the image pair in regard to resolution. Each observer indicated which image of the pair appeared at a higher resolution. The results show that the mean resolution for 75% correct responses for each of the visual acuity groups increased from more than 30 cpd as visual acuity increased and reached a plateau at 40-50 cpd at -0.3 logMAR.

Full text

Perceptual limit to display resolution of images as per visual acuity
Kenichiro Masaoka*
a
, Takahiro Niida
b
, Miya Murakami
b
, Kenji Suzuki
b
,
Masayuki Sugawara
a
, Yuji Nojiri
a
a
NHK Science & Technical Research Laboratories, 1-10-11 Kinuta, Setagaya-ku, Tokyo 157-8510, Japan;
b
Department of Orthoptics and Visual Science, School of Health Sciences,
International University of Health and Welfare, 2600-1 Kitakanemaru, Otawara, Tochigi 324-8501, Japan
Keywords: image resolution, visual acuity, MTF, visual realness
ABSTRACT
Achieving ultimate visual realness of natural images on a display requires high resolution, so that artifacts due to finite
image resolution are undetectable. An image resolution of 30 cycles/degree (cpd) or one pixel/arc-minute is often used as
the criterion for viewing conditions when assessing displayed image quality. It is reasoned that if the pixel size is smaller
than the separable angle of normal vision (20/20), the pixel structure is invisible and doesn’t negatively affect image
quality. However, it is not clear whether 30 cpd resolution is adequate to prevent seeing artifacts, especially for observers
with better than 20/20 vision. We conducted experiments to find the threshold resolution of natural images and its
dependence on visual acuity. Three objects were used; each object was presented 60 times at 5 resolutions (19.5, 26, 39,
52, or 78 cpd) next to the same image at a resolution of 156 cpd. Forty-five observers with visual acuity of 20/20 or
better were asked to make a forced-choice distinction between the image pair in regard to resolution. Each observer
indicated which image of the pair appeared at a higher resolution. The results show that the mean resolution for 75%
correct responses for each of the visual acuity groups increased from more than 30 cpd as visual acuity increased and
reached a plateau at 40-50 cpd at -0.3 logMAR.
1. INTRODUCTION
The minimum required image resolution at which an increment in resolution is undetectable to the human eye is one of
the criteria for ultimate visual realness. Regarding the requirements of image quality assessment, the ITU-R recommends
the ‘design viewing distance’ [1] as 3H for HDTV [2] and 6H for SDTV [3] (H: screen height). The image resolution at
those viewing distances corresponds to about 30 cycles per degree (cpd) or one pixel per arc minute (the minimum
separable angle for 20/20 vision). It is often reasoned that if the pixel size is smaller than the separable angle, the
scanning line or pixel structure is invisible and doesn’t negatively affect image quality. However, it is not clear whether
the image resolution is high enough for visual realness of natural images. First, the visual acuity of emmetropic eyes is
often better than 20/20 and may even reach 20/10. However, there is no standard required image resolution for specific
viewers’ visual acuity. For viewers with better than 20/20 vision, it is not clear whether the image resolution of 30 cpd is
still high enough or should be increased, say, proportionally to visual acuity. Furthermore, individuals with visual
hyperacuity can detect a vernier offset of a few seconds of arc [4]. Taking this into account, an image resolution of 30
cpd would not be enough to ensure the visual realness of images. In fact, Masaoka et al. [5] used a paired-comparison
procedure to quantify the realness of the images versus each other or versus real objects. Both real objects and images
were viewed through a synopter. The results indicated that realness of an image increased as the image resolution
increased to about 40-50 cpd and reached a plateau. Secondly, it would be an oversimplification to suppose that image
resolution based on the viewers’ minimum separable angle suffices visual realness. In a strict sense, the contrast
sensitivity of the viewers’ eyes, the modulation transfer function (MTF) of the shooting system and the image content
should be taken into account. The MTF of the shooting system gradually decreases as the spatial frequency increases,
and drops down at the Nyquist frequency to prevent aliasing. Therefore natural images are generally different from
visual acuity test charts, such as the Landolt C, in that test charts have a greater number of high spatial frequency
components. We conducted experiments to examine the discrimination threshold of image resolution and the relationship
between image resolution and visual acuity.
Human Vision and Electronic Imaging XIII, edited by Bernice E. Rogowitz, Thrasyvoulos N. Pappas,
Proc. of SPIE-IS&T Electronic Imaging, SPIE Vol. 6806, 68061G, © 2008 SPIE-IS&T · 0277-786X/08/$18
SPIE-IS&T/ Vol. 6806 68061G-1
2008 SPIE Digital Library -- Subscriber Archive Copy

II I I
20H
INormal
I
I
ILowCS
I!
HL
0.1 0.0 -0.1 -0.2 -0.3 -0.4 -0.5 -0.6
Visual acuity [1ogMAR]
2. METHODS
2.1 Observers
Forty-five students from the Department of Orthoptics and Visual Science at the School of Health Sciences in the
International University of Health and Welfare participated in our experiments as observers. The observers were age 19
to 24 (mean: 21) with normal vision. They were not experts in electronic imaging, but were familiar with visual blur and
knew what “image resolution” meant. Their vision was optimally corrected on a daily basis; their vision was not further
corrected in the experiments. Each observer’s visual acuity, contrast sensitivity, subjective refraction, and objective
refraction were measured both before and after the experiment. Visual acuity was measured using single Landolt Cs
ranging up to -0.6 logMAR in 0.1 logMAR steps. Table 1 shows the visual acuity scale. The C charts were designed to
be viewed from a distance of 4.45 m, which equaled the viewing distance in the experiments. Figure 1 shows a histogram
of binocular visual acuity of the observers measured before experiments. Each observer’s binocular visual acuity was
equal to or better than his or her monocular visual acuity as explained by binocular summation. Each observer’s contrast
sensitivity was measured with the CAT2000 Contrast Sensitivity Accurate Tester (Neitz Instruments Co., Ltd., Tokyo,
Japan). Five observers had contrast sensitivities lower than the normative limits. Objective refraction was measured
using the ARK-700A auto ref/keratometer (Nidek Co., Ltd., Tokyo, Japan). Decreased visual acuity, decreased contrast
sensitivity, or aggravation of refraction was not found for any observer after the experiments.
Table 1. Visual acuity scale.
logMAR Foot Meter Decimal
0.1 20/20 6/6 0.79
0.0 20/20 6/6 1.00
-0.1 20/16 6/4.8 1.26
-0.2 20/13 6/3.8 1.58
-0.3 20/10 6/3 2.00
-0.4 20/8.0 6/2.4 2.51
-0.5 20/6.3 6/1.9 3.16
-0.6 20/5.0 6/1.5 3.98
Figure 1. Histogram of the observers’ binocular visual acuity. Low
CS means observers who had contrast sensitivities lower
than the normative limits.
2.2 Apparatus
Objects were shot with a CCD-based imaging colorimeter (PM-1400; Radiant Imaging, Inc., Duvall, Washington, USA)
and a telephoto lens (Nikon Ai AF-S, Nikkor ED 300mm F4D; Nikon Corporation, Tokyo, Japan). The colorimeter had a
native resolution of 3072 × 2048 pixel (pixel size: 9 µm × 9 µm), 2-stage Peltier-cooled 14-bit CCD with full-frame
technology, measuring the luminance and chromaticity accurately. The colorimeter measured CIE 1931 XYZ values
using a set of three filters based on the 1931 CIE 2-degree color matching function. The XYZ values of the central CCD
area of 1700 × 1700 pixels were used as the original image to be processed into lower resolutions. The shooting distance
was 4.55 m, focused at f/16.
For displaying images, a color LCD monitor (ColorEdge CG221; EIZO Nanao Corporation, Hakusan, Japan) was used;
this monitor featured a wide color range approximately equal to the Adobe RGB color space [6]. The 22.2-inch screen
had a display area of 1920 × 1200 pixels (pixel size: 0.249 mm × 0.249 mm). The screen was calibrated to have uniform
color and brightness with a gamma of 2.2 across the entire screen, using the monitor’s 12-bit look-up table that has a
palette of 4,096 gray scales and outputs the most appropriate 256 grayscale tones (8 bits) for red, green, and blue. The
SPIE-IS&T/ Vol. 6806 68061G-2

white level was set to 183 cd/m
2
and the black level to 0.61 cd/m
2
. The white point was set to D65. Images were
displayed from a PC connected to the display via a DVI cable. The display was set in a room surrounded with achromatic
walls and the viewing distance from the observer’s eyes was 4.45 meters. The walls at the back of the display were
illuminated at a luminance of about 15% of the display white level, according to the recommendation of ITU-R BT 500
[7]. Under these dim surrounding conditions, the white level was 184 cd/m
2
and the black level was 1.06 cd/m
2
.
2.3 Materials
Figure 2 shows the three objects used in our experiments. These objects, which were within the depth of field (DOF) of
the colorimeter, have many details in their shapes. As stated before, the shooting distance was equal to the viewing
distance. This allowed the perspective as well as the perceived size of the image to appear identical to the real objects.
Food Model ship Butterflies
Figure 2. Evaluation objects.
2.4 Image processing
The original images of 1700 × 1700 pixels shot with the colorimeter were downscaled to 850 × 850 pixel images with a
resolution ranging from 19.5 cpd to 156 cpd, as shown in Table 2. Figure 3 shows the image processing flow chart for
lowering image resolution. First, the original images of 14-bit XYZ values were filtered by anti-aliasing lowpass filters.
The filters were two-dimensional FIR filters designed by the window method using a Hamming window. The Matlab
fwind1 function was used to design the n × n filters, having a size nearly inversely proportional to the downscaling ratio,
where n = 8k + 1 (k is shown in Table 2). The lowpass filtered images were then downscaled into six different image
sizes (1/k) by decimation. Each downscaled image was filtered by an unsharp contrast enhancement filter to compensate
for the drop of the MTF of the shooting system at high frequencies. A 3 × 3 two-dimensional negative Laplacian filter
designed by the Matlab fspecial function with an alpha parameter of 0.2 was used for unsharp contrast enhancement.
After conversion from XYZ tristimulus values to linear RGB (red-green-blue) values based on the primary colors of the
display, the images were encoded with the inverse of the display’s gamma of 2.2, forming 8-bit non-linear R’G’B’
values. Finally, the R’G’B’ images were upscaled into 850 × 850 pixels by using the nearest-neighbor method to
simulate square-pixel structure.
Figure 4 shows the MTFs of six image resolutions and the spatial frequency response (SFR) of the imaging colorimeter,
measured based on slanted-edge analysis using the SFR algorithm, as defined in ISO 12233 [8]. The horizontal axis in
the left side is in cycles per degree, while that in the right side is normalized by the Nyquist frequency. The MTFs at the
Nyquist frequency were about 10%, and the spatial frequencies that correspond to 50% MTF were about 0.74 times of
the Nyquist frequency for each image resolution. Figure 5 shows the slanted edge image and edge profile for each image
resolution. The over-shooting and under-shooting of the edges were about 5% of the luminance range for all image
resolutions. Figure 6 shows the center portions of 375 × 375-pixel ‘model ship’ images. Note that lower resolution
images have a square pixel structure.
SPIE-IS&T/ Vol. 6806 68061G-3

1.11
.
0.9
0.8
0.7
06L
H
I
______
LI I"
SFK
156 cpd
03H
U
iocpct
I
__52cpd
I
0
0 1 2
10
Frequency [cpd]
10
11 i
I
1*
N
U.FI ______
SFR
O.4H
0
l56cpd
O.3H
0
78cpd
52cpd
0.2
I
39cpd
0
26cpd
.' H
19.5cpd
o o.i 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9
Frequency [Nyquist freq.]
200 400
Pixel
400
r
350
3OU
1:1/f 1
I
/l II
,FK
1
l56cpd
U
/öCU
5O A
52cpd
i9 cpu
26cpd
I
19.Dcpcl I
-50
•- -4 -3 -2 -1 U 1 2 3 4
Pixel
Table 2. Parameters for image processing.
Resolution
[cpd]
Downscaling
Ratio
(1/k)
Upscaling
Ratio
(k/2)
156 1/2 1
78 1/4 2
52 1/6 3
39 1/8 4
26 1/12 6
19.5 1/16 8
Figure 3. Image processing flow chart.
Figure 4. Spatial frequency characteristics. The left figure shows the MTFs of each image resolution at frequencies in cpd,
while the right figure shows those at frequencies normalized by Nyquist frequency.
Figure 5. Slanted edge image (left) and the edge profile (right).
Original image (1700 × 1700 pixels, 14-bit XYZ)
Anti-aliasing lowpass filtering
Downscaling (850 × 850 pixels or less)
Enhancement filtering
Conversion from XYZ to RGB
Encoding with a gamma correction of 2.2 (R’G’B’)
Upscaling by the nearest-neighbor method
(850 × 850 pixels, 8-bit R’G’B’)
SPIE-IS&T/ Vol. 6806 68061G-4

156 cpd 78cpd 52 cpd
39 cpd 26 cpd 19.5 cpd
Figure 6. ‘Model ship’ images at six resolutions. These images were obtained from the center portions of 375×375-pixel
‘model ship’ images simulating a square pixel structure.
2.5 Experiment
Each object was presented 60 times at 5 resolutions (19.5, 26, 39, 52, or 78 cpd) next to the same image at a resolution of
156 cpd. Each observer was asked to make a two-alternative forced-choice (2AFC) distinction between the image pair in
regard to resolution. Each observer indicated which image of the pair appeared to be at a higher resolution (i.e. the image
at 156 cpd). The order of the 60 image pairs was set in ascending order by resolution. Observers were notified of the
correctness for each choice by a sound for correct and incorrect answers, respectively. The ascending order of image
resolution and sound feedback of correctness were set to make the observers’ criteria as high as possible. Each image
pair was shown for 10 seconds, or until the observer made a choice (using a simple joypad to make the selection),
followed by a gray image. If the observer made a choice within 10 seconds, the image pair disappeared and a gray image
was shown for 1 second. Then, the next image pair was shown. If the observer did not make a choice within 10 seconds,
a gray image was shown until the observer made a choice. Then, the gray image lasted for another second followed by
the next image pair. There were three sessions for each observer, one session for one object. The order of the three
sessions was randomized for each observer. Each observer stayed in the experiment room to adapt to the room luminance
for 10 minutes before each session and took a 20-minute break after each session. Each observer was allowed to watch
the high resolution image before each session to find portions which appeared easy to detect.
SPIE-IS&T/ Vol. 6806 68061G-5

20 40 60 80 20 40 60 80 20 40 60 80 20 40 60 80 20 40 60 80
Resolution [cpd] Resolution [cod]
3. RESULTS
Figure 7 shows the correct probabilities for each resolution of each object for each observer. To predict the
discrimination thresholds of resolution at the 75% correct probability, the logistic regression was used to fit the correct
probabilities of each object for each observer, except for some observers with low contrast sensitivity.
Figure 7. Correct probabilities of each observer with logistic curves. Each graph has a legend including the observer’s
number, binocular visual acuity, and vision correction (Glasses, Contact lens, or no correction). The legend is
colored yellow if the observer had low contrast sensitivity.
SPIE-IS&T/ Vol. 6806 68061G-6

100
90
I
'U
— 60 I
a 50
T 11 1
I
C
I
0
V
30
1
0 Fnod
h
I
I
V
DUt1I111CS
20F
U.U -U.1 -U.2
-Ui
-U.4 -U. -U.b
Visual
acuity [1ogMARI
Figure 8 shows the discrimination thresholds of resolution at 75% correct responses for each visual acuity group. The
markers of visual acuity groups from -0.2 logMAR to -0.4 logMAR, which consisted of more than three observers, show
the mean resolution with error bars of ±1 S.D., while the other markers show each individual observer’s discrimination
thresholds of resolution. The results of five observers with low contrast sensitivity were excluded. The dotted line in
Figure 8 shows the resolution of 30/MAR in cpd, which is the resolution based on the visual acuity of the minimum
separable angle. The results show that the upper discrimination threshold of resolution increased from more than 30 cpd
and reached a plateau at 40-50 cpd at -0.3 logMAR.
Figure 8. Discrimination thresholds of resolution for each visual acuity group. The markers of visual acuity groups from -0.2
logMAR to -0.4 logMAR, which consisted of more than three observers, show the mean resolution with error bars
of ±1 standard deviation, while the other markers show the observers’ discrimination thresholds. The dotted line
shows the resolution of 30/MAR in cpd.
4. DISCUSSION
Each observer showed a constant correct probability for each of the all 60 trials. According to observer subjective reports,
they tried to evaluate more than one portion of each image to prevent blur caused by concentrating on the same image
section, and to confirm their choices. Figure 8 shows that the upper discrimination threshold of resolution increased from
more than 30 cpd and reached a plateau at 40-50 cpd at -0.3 logMAR. The gap between the discrimination threshold and
30/MAR at a visual acuity of -0.3 logMAR or more might be due to the time limit of the image presentation, because
there were fewer cues in the high frequency range of natural images and it took time to detect a difference between
resolution image pairs. To confirm the cause of the plateau, we conducted further experiments. Fifteen observers with
-0.2 to -0.5 logMAR participated in the subsequent experiment. Each observer compared each image pair without a time
limit. Before each 60 trials, the observers were trained to find image sections where clarity was easy to detect by
changing the viewing distance.
Figure 9 shows the correct probabilities for resolution of each object for each observer. To predict the discrimination
thresholds of resolution at the 75% correct probability, logistic regression was used to fit the correct probabilities of each
object for each observer, although some correct probabilities reached 100% at frequencies up to 52 cpd, where the
logistic fitting curves are not reliable. Figure 10 shows the discrimination thresholds of resolution for each visual acuity
group. The discrimination threshold for ‘model ship’ image for the -0.3 logMAR group (20/10 vision) reached 60 cpd.
When observers were allowed to compare image pairs without a time limit and through adequate pre-training, the upper
discrimination threshold of resolution of natural images increased as the visual acuity increased, and was close to the
SPIE-IS&T/ Vol. 6806 68061G-7

Resolution Lcpd] Resolution [cpdj
100
90
V
'U
_60[
a 50
I
I
30 _____________________
I • Food I
I
V niteiiiies
20F . .
U.U -U.1 -U.2
-Ui
-U.4 -U. -U.b
Visual
acuity [1ogMARI
resolution of 30/MAR in cpd. In typical TV viewing, however, the amount of pre-training needed to achieve this result
was considered excessive. Furthermore, Observer 43 and Observer 45, who showed high discrimination thresholds, took
longer times to evaluate critical image pairs. Figure 11 shows their evaluation time for each object. Other observers took
approximately 15 seconds to evaluate critical image pairs.
Figure 9. Correct probabilities for each observer. Each graph has a legend including the observer’s number, binocular visual
acuity, and vision correction (Glasses, Contact lens, or no correction).
Figure 10. Discrimination thresholds of resolution for each visual acuity group. The markers of the visual acuity groups of -0.3
logMAR (10 observers), show the mean resolution with error bars of ±1 standard deviation, while the other markers
show each individual observer’s discrimination thresholds. The dotted line shows the resolution of 30/MAR in cpd.
SPIE-IS&T/ Vol. 6806 68061G-8

60
.
I . 1nnd
Model ship
,-
tiuuerriies
T Ii
ii
I]
No43 /1
10 -0.4logMAR
Naked vision
20 4U bU 3U
Resolution [cpd]
60
.
I . 1nnd I TH
50
—
Model
ship
I
,-
rsuuerriies
/ I
/1]
No.45
/,
10 -0.5 logMAR
Naked vision
i/il
Figure 11. Evaluation time with error bars of ±1 standard deviation for Observer 43 (left) and Observer 45 (right) in the
experiments without a time limit.
5. CONCLUSION
We conducted experiments to find the threshold resolution of natural images and its dependence on visual acuity. When
observers were allowed to compare image pairs over 10 seconds, the mean resolution for 75% correct responses for each
of the visual acuity groups increased from more than 30 cpd as visual acuity increased and reached a plateau at 40-50
cpd at -0.3 logMAR. With no time limit and adequate training, the discrimination threshold was close to 30/MAR cpd,
which is an image resolution based on a visual acuity of the minimum separable angle. However, the amount of
pre-training and evaluation time needed to achieve this result was considered excessive. Our results suggest that, in
TV-viewing situations, display resolution of 40-50 cpd is adequate to address viewers with 20/20 vision or better to
prevent the presence of artifacts interfering with finite image resolution.
ACKNOWLEDGMENTS
We would like to thank the students of the Department of Orthoptics and Visual Science at the School of Health Sciences
in the International University of Health and Welfare for setting up and calibrating the equipment for the study,
participating in our experiments as observers, and helping with the visual function tests of the observers.
REFERENCES
1. ITU-R Recommendation BT. 1127, “Relative quality requirements of television broadcast systems,” 1994.
2. ITU-R Recommendation BT. 710-4, “Subjective assessment methods for image quality in high-definition
television,” 1998.
3. ITU-R Recommendation BT. 1128-2, “Subjective assessment of conventional television systems,” 1997.
4. Westheimer, G. & McKee, S. P., “Spatial configurations for visual hyperacuity,” Vision Research, 17, pp. 941-948,
1977.
5. Masaoka, K., Emoto, M., Sugawara, M., Nojiri, Y., “Comparing realness between real objects and images at various
resolutions,” Proc. of the SPIE, 6492, pp. 64921F, 2007.
6. Adobe RGB (1998) Color Image Encoding, May 2005.
7. ITU-R Recommendation BT. 500-11, “Methodology for the subjective assessment of the quality of television
pictures,” 2006.
8. ISO 12233:2000, Photography - Electronic still picture cameras - Resolution measurements.
SPIE-IS&T/ Vol. 6806 68061G-9