Available via license: CC BY-NC-ND 4.0
Content may be subject to copyright.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, TVCG-2021-08-0343 1
Predicting Subjective Discomfort Associated
with Lens Distortion in VR Headsets During
Vestibulo-Ocular Response to VR Scenes
Tsz Tai Chan, Yixuan Wang, Richard Hau Yue So and Jerry Jia
Abstract— With advances in Virtual Reality (VR) technology, user expectation for a near-perfect experience is also increasing.
The push for a wider field-of-view can increase the challenges of correcting lens distortion. Past studies on imperfect VR
experiences have focused on motion sickness provoked by vection-inducing VR stimuli and discomfort due to mismatches in
accommodation and binocular convergence. Disorientation and discomfort due to unintended optical flow induced by lens
distortion, referred to as dynamic distortion (DD), has, to date, received little attention. This study examines and models the effects
of DD during head rotations with various fixed gazes stabilized by vestibulo-ocular reflex (VOR). Increases in DD levels
comparable to lens parameters from poorly designed commercial VR lenses significantly increase discomfort scores of viewers
in relation to disorientation, dizziness, and eye strain. Cross-validated results indicate that the model is able to predict significant
differences in subjective scores resulting from different commercial VR lenses and these predictions correlated with empirical
data. The present work provides new insights to understand symptoms of discomfort in VR during user interactions with static
world-locked / space-stabilized scenes and contributes to the design of discomfort-free VR headset lenses.
Index Terms— Virtual reality, lens distortion, visual discomfort, motion sickness, disorientation, vestibulo-ocular reflex
—————————— ◆ ——————————
1 INTRODUCTION
OMMERCIAL VR headsets currently rely on optics to
project pixels on a head-steered near-eye display in
order to form images of space-stabilized virtual worlds [1].
To create a realistic virtual world, the geometry and
texture of a 3D object needs to be accurately reproduced
spatially. The spatial mapping between the display and
virtual world is critically important to geometric
reproduction. With advances in VR technology, users
expect a perfect mapping experience when involved in
world-locked VR scenes in which all virtual objects remain
spatially stable relative to the physical world (i.e., world-
stable). As a result, images of a VR world-locked object will
be visible, and appear spatially stable, when their angular
positions fall within the field-of-view of a head-steered VR
display. In practice, the mapping is not perfect because the
transmitted light field from a VR display is different from
the naturally occurring light field from a real environment
that the virtual world-locked VR scenes are simulating [1]
(Fig. 1a). The issue is attributed to the optical layout of VR
headsets. The optical layout of a VR headset can be
considered as an ideal lens with extra lens distortions.
Theoretically, a distortion correction process to pre-
transform a displayed image should be able to cancel out
any distortion by the lens. This can be achieved with ray
tracing and calibration through a set of mapping files
generated specifically to all locations on the display. Fig.
1a illustrates how a distorted display can be corrected; Fig.
1b presents a 2D spatial transformation in the form of a 2D
map.
————————————————
• T.T. Chan is with the Department of Industrial Engineering and Decision
Analytics, The Hong Kong University of Science and Technology, Hong
Kong, China. E-mail: ttchanac@connect.ust.hk.
• Y. Wang is with the Department of Chemical and Biological Engineering,
The Hong Kong University of Science and Technology, Hong Kong, China.
E-mail: ywanggx@connect.ust.hk.
• R. H. Y. So is with the Department of Industrial Engineering and Decision
Analytics and Department of Chemical and Biological Engineering, The
Hong Kong University of Science and Technology, Hong Kong, China. E-
mail: rhyso@ust.hk.
• J. Jia is with the Facebook Reality Laboratory, CA, USA. E-mail:
Jerry.Jia@fb.com.
(T.T. Chan and Y. Wang are co-first authors.)
xxxx-xxxx/0x/$xx.00 © 200x IEEE Published by the IEEE Computer Society
Fig. 1. (a) Distortion correction in a VR headset allows virtual content
to appear regular and normal. (b) Example of a distortion correction
map that is independent of content and saved on the VR headset.
C
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
2 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, TVCG-2021-08-0343
During the calculation of the correction mapping files, a
fixed pupil location relative to the lens is assumed;
however, this assumption does not hold in many situations.
One typical example where this assumption fails is during
reading with the help of vestibule-ocular reflex (VOR), a
response to stabilize the eye gaze during head movements.
When a viewer turns his / her head while keeping the eye
gaze stable (VOR-type movement), the eye pupil will move
relative to the lens in the opposite direction of the head
movement to continue to fixate on the object of attention
(see Fig. 2). Thus, the assumption of a fixed pupil location
relative to the lens is violated, and a dynamically changing
distortion (referred to as “dynamic distortion (DD)”)
results from the incomplete correction.
The DD can cause observations of “shifting floors” and
“curved walls”. As floors and walls are fixtures that are
normally not moving, their movements may affect the
reference rest frame’s judgment and even lead to motion
sickness symptoms, such as disorientation and dizziness
according to the rest frame hypothesis [2, 3, 4]. A review of
relevant literature failed to find any study of DD among
VR users. In this study, we refer to symptoms of motion
sickness induced by the DD during VOR-type head
motions as “DD-VOR discomfort”. The occurrence of DD-
VOR discomfort, if proven, can be an example to support
the rest frame hypothesis (RFH) of motion sickness [2, 3, 4].
From the perspective of sensory conflict theory [5, 6], the
mismatch between visual and vestibular inputs could also
be an explanation for DD-VOR discomfort.
The occurrence of motion sickness symptoms among
VR users has stymied the growth of the industry. Most
previous studies on VR discomfort focused on hardware
or vection-provoking visual content. Regarding hardware,
the effect of display latency has been the subject of many
studies [7, 8, 9, 10, 11]. As to the influence of visual content
on motion sickness, the effects of vection-provoking visual
stimuli has also been the subject of many studies [ 12, 13,
14, 15, 16]. Solutions to reduce discomfort have been
proposed [12, 13, 17, 18]. Examples include reduction of
field-of-view [17, 18, 19] and controlling the speed of
navigation [12, 13]. Models have also been proposed to
predict vection induced motion sickness [20, 21].
In summary, despite advances in VR technology,
discomfort and motion sickness symptoms are still
prevalent among users. While effects of hardware (e.g.,
display latency) and vection induced VR content have been
the focus of research, motion sickness due to imperfect lens
correction (DD-VOR discomfort) has received little
attention. One possible reason for the lack of a study on
DD-VOR discomfort among VR users is that it takes a
collaborative effort between the VR headset manufacturer
and the researchers to manipulate and control the relevant
parameters. This paper reports the first study to examine
the DD-VOR discomfort and proposes a predictive model.
In this study, we focus on the effects of DD-VOR
discomfort caused by lens distortion. Both the influence of
the vection-provoking scenes and latency are controlled.
All VR scenes remain world-locked static without passive
navigation. Users can still choose what they see through
head steering. The latency is around 10ms or less with
displays updated at 90Hz.
2 CAUSES OF DD-VOR DISCOMFORT
The historical development of the theoretical
understanding of DD-VOR discomfort can be summarized
as three progressive steps: (i) sensory conflict theory, i.e.,
the theory that humans suffer motion sickness due to
conflict in received sensory cues; (ii) physical causes; and
(iii) sensory conflict theory specifically applied to DD-VOR.
These three steps are broadly similar to the three levels of
DD-VOR discomfort users experience.
2.1 Sensory conflict theory
Motion sickness is a general syndrome characterized by
symptoms such as nausea, stomach discomfort, cold
sweats and disorientation [2, 6]. About one-third of the
population is currently susceptible to this condition with a
range from moderate to extreme nausea [22]. A widely
accepted hypothesis for motion sickness is sensory conflict
theory [6, 23], which is also referred to as sensory
rearrangement theory [5]. It proposes that the conflict
among motion information perceived from different
sensory modalities (e.g., vestibular, visual, and
proprioceptive systems) and the expected sensory inputs
based on previous experience can provoke discomfort [5,
6]. The sensory conflict theory has received support from
reported correlations among perception of sensory conflict,
neural activation at the reticular formation in the
brainstem, neural activation of the autonomic nervous
system and reported symptoms of motion sickness [24].
Studies of motion sickness among users of virtual reality
systems have led to the development of the rest-frame
hypothesis (RFH) [25]. This hypothesis predicts that the
brain adopts the intrinsic Euclidean frame of reference to
process spatial information. The rest-frame refers to some
specific spatial features which an observer assumes to be
space stationary (i.e., world-locked) [2, 4]. If DDs cause an
assumed space stationary feature in a virtual environment
to move, it would induce sensory mismatch and, hence,
motion sickness [25].
2.2 Physical cause of DD
The pupil location is a vital factor in causing DD. Pupil
locations relative to a VR lens will shift if the eyes rotate
relative to the head. If, initially, the eye’s gaze is aligned
Fig. 2. Vestibulo-ocular reflex in VR headset leads to angular
displacement of pupils related to the near-eye display. Before VOR
movement, the pupil is looking through the center of the lens; and after
the VOR movement, it is looking at the same displayed object but
through the side of the lens.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
CHAN ET AL.: PREDICTING SUBJECTIVE DISCOMFORT ASSOCIATED WITH LENS DISTORTION IN VR HEADSETS DURING VESTIBULO-OCULAR 3
RESPONSE TO VR SCENES
with the optical axis of the VR lens, during a VOR-type
head movement when the gaze is fixated ahead, the lens
rotates away with the head forcing the pupil location to
move off the optical axis of the lens as illustrated in Fig. 3.
With the dynamic change of pupil location, light from
the same pixel on the display is received at slightly
different angular directions into the eye. More importantly,
it is passing through a different part of the lens and
subjected to different lens distortion. If pupil locations are
known, current technology can predict and correct the
lens distortion through a ray-tracing program [26]. Fig. 4
illustrates an example of such a correction procedure. The
lens distortion is illustrated by arrays of red dots on a black
angular grid. At 0 degrees (before VOR), there is no lens
distortion. When the head has turned 20 degrees (after the
VOR), red dots shift from the intersection points indicating
the corresponding optical distortion which is angular
position dependent and lens dependent. If we connect each
pair of corresponding red dots after the VOR with its black
interaction point, we can create a 2D vector field across the
field-of-view of the lens (Fig. 5). This 2D vector field is also
called optical flow. Fig. 5 illustrates the optical flow from
three different existing commercial VR lens designs
simulated with VOR head movements from 0 to 20° to the
right direction of the user.
2.3 DD-VOR Discomfort explained by sensory
conflict theory
DD occurs when a VR user is performing a VOR-type head
movement (Fig. 5). In this scenario, as the head motion is
initiated and driven by the user, the inertial sense of
motion (i.e., the vestibular cue) is perfectly perceived. Due
to DD, the visual input is distorted, causing a sensory
conflict and its subsequent discomfort and motion sickness.
One example is research conducted by Stratton (1897) who
examined the effects of wearing inverted and reversed lens.
Even though the vestibular input was perfectly perceived,
the locomotion and general psychomotor performance
were disturbed due to distorted visual inputs. The
Fig. 4. Illustrations of lens distortion by angular grids (grid size is approximately 3.3 by 3.3 degrees). (a) The grid is not distorted when the head
is pointing straight ahead. (b, c) After a 20° rightward head rotation, the grid is distorted as illustrated by the red dots representing the interaction
points after distortion.
Fig. 5. 2D vector field (i.e., optical flow) representations of the DD induced by a rightward 20° head rotation with three different lenses. Each
vector is a line connecting the black intersections and red dot in Fig. 4. Both axes are in degrees and vectors have been magnified 10 times for
better illustrations.
Fig. 3. (View from top of the head) Optical root cause for DD: during VOR, eye pupil location changes relative to the lens. This changes the
perceived distortion pattern of the lens accordingly.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
4 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, TVCG-2021-08-0343
participant reported discomfort and nausea on the first
and second day. On removing the lens after an adaptation
period, the participant also experienced symptoms of
nausea [27]. Likewise, participants experiencing DD-VOR
are hypothesized to report discomfort. DD-VOR
discomfort can be accounted for by the sensory conflict
theory, and specifically, it could be a supportive example
for the RFH. In the real world, the information about self-
motion, object motion, and rest-frame can be inferred from
visual cues without disturbance, and that information will
be consistent with those inferred by inertial cues from the
vestibular systems. However, in VR and augmented reality
(AR), when an observer makes VOR-type head movements,
due to the afore-mentioned DD, the visual cues presented
to the observer will be incorrect. The observable effect is
“visual distortion” where visual cues are unexpected and
occasionally annoying. Examples of reported observations
in these cases include: “the edge of building is distorted”,
“straight frames look bent”, etc. In addition, observers may
infer self-motion and / or orientation that are inconsistent
with those inferred from the inertial cues. Prolonged
exposure to such conflict can lead to symptoms of motion
sickness in susceptible populations. Fig. 6 illustrates the
logic of how sensory conflicts can be induced by visual
distortion through a block diagram with a comparison
between the visual-vestibular interactions in the real world
and the VR world.
3 MODEL DEVELOPMENT
3.1 Verification of motivation of model
development: significant DD-VOR discomfort
effects
Before developing the model, a user study was conducted
to verify the strength of the effects. Two dynamic
distortion conditions were derived from a lens (lens A):
one with a scaling factor of 0.5 (A-1), the other with a
scaling factor of 2 (A-2). The scaling factor was defined to
magnify (scaling factor > 1) or reduce (scaling factor < 1)
the dynamic distortion. It was hypothesized that condition
A-2 would be associated with significantly higher
discomfort ratings. Participants (n=8, see Section 4.3) were
exposed to each of the two selected conditions for a total of
20 minutes. During this time, they were asked to focus on
a central eye-fixation point and turn their heads 16° to 20°
to the right and back and repeated according to a 100-bpm
metronome (see more details in Section 4). At the end of
the test, they rated the DD-VOR discomfort they
experienced, with a number from 0 to 5. Before and after
the 20-minute test, they were also asked to fill in Simulator
Sickness Questionnaires (pre-SSQ and post-SSQ) [28].
Results indicated that the SSQ scores were significantly
increased after the 20-minute exposure for both DD
conditions (paired t-tests, mean pre-SSQ and post-SSQ
scores of A-1 condition: 0.935 vs. 20.10, p = 0.0008; mean
pre-SSQ and post-SSQ scores of A-2 condition: 1.40 vs.
23.38, p=0.0019). Also, the A-2 condition resulted in
significantly higher DD-VOR discomfort scores than A-1
condition at the end of the 20-minute test (paired t-test, p =
0.0098, Fig. 7). Examinations of the symptoms reported in
the SSQ questionnaires indicate that “general discomfort,”
“vertigo,” and “fullness of head” were more frequently
reported in A-2 than in A-1 condition (increases from 37.5%
to 62.5% to 75%). Results of the verification study confirms
the presence of a genuine and significant DD-VOR
discomfort effect.
3.2 Overview of the model development process
Based on the above verified significant effects of visual
distortion and symptoms of motion sickness, an analytical
model was developed and trained with data collected in
further studies (see Section 4). The model starts with the
selection of lens design and the pupil location relative to
the center of the lens. As illustrated in Figs. 3, 4 and 5,
optical flow in terms of 2D vector field representing the DD
was generated (Section 3.3). The optic flow was then
Fig. 6. Framework for understanding discomfort during VOR-type head motion. In the real world, consistent sense of motion can be inferred
from visual cues and inertial cues (top) not in VR or AR (bottom). Defective visual cues produce conflicts
Fig. 7. Significant higher DD-VOR discomfort scores were reported for
the dynamic distortion conditions with aggravated distortion.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
CHAN ET AL.: PREDICTING SUBJECTIVE DISCOMFORT ASSOCIATED WITH LENS DISTORTION IN VR HEADSETS DURING VESTIBULO-OCULAR 5
RESPONSE TO VR SCENES
passed through a series of mathematical operations to
account for (i) eccentricity effects (Section 3.4); (ii) spatial-
temporal interactions (Section 3.5); and (iii) influence in
terms of 16 optic flow pattern features (Section 3.5). The
processed DD features were mapped to predicted DD-
VOR discomfort scores and distortion scores through a
regression model (Section 3.6). The regression model was
pre-trained using data from the psychophysical
experiment (see Section 4). Fig. 8 illustrates the procedure
of how the model predicts DD-VOR discomfort and
distortion scores associated with the use of a particular lens
in VR applications.
3.3 Quantification of dynamic lens distortions
using optical flow
The DD can be represented by an optical flow map (a 2D
vector field) with a fixed radius of 40 degrees from lens
center and step size of 3.3 degrees (Fig. 9). As illustrated in
Figs. 3, 4 and 5, an optical flow map is generated between
the pupil location when the VOR starts, and the pupil
location when the VOR ends using optical ray tracing. This
is referred to as the uncompensated optical flow (Fig. 9a).
The displayed content at the fixation point will shift
according to the optical flow (i.e., distortion). To maintain
the fixation, the eyes will move to follow the displayed
content through smooth pursuit, and the perceived optical
flow would be different from the uncompensated optical
flow map. To account for the effects of corrective eye
movements, the optical flow map is recalibrated by
subtracting the vector at the new fixation point from the
original, resulting in compensated optical flow (Fig. 9b). In
this study, the compensated optical flow was used. Fig. 10
illustrates the optical flow representing the DD, after
corrective eye movements, of the four lenses studied in the
experiment. The DDs were associated with a rightward
head rotation of 20 degrees. Kindly note that Fig. 10 is just
a snapshot; in the study, DD was dynamically updated
according to the measured head rotations.
3.4 Sensitivity weighting of distortions according
to eccentricity
Past studies reported that visual sensitivity decreases with
eccentricity, and that the rate of changes of sensitivity
along the horizontal axis and the vertical axis are different
[29, 30, 31]. Literature on the motion sensitivity during
VOR-type head motion is absent. As a first attempt, we
followed the past literature and applied the reported visual
sensitivity weighting (see Fig. 11). In summary, visual
distortion closer to the eye fixation point (i.e., foveal) has a
higher influence on users. We acknowledge that this
weighting is only preliminary and can be changed in the
future.
3.5 Quantifying spatial-temporal interactions
between VOR head movements and distortions
Two elements were used to quantify the optical flow
representations of DDs for scoring its influence on
subjective responses; one relates to distortion intensity, the
other relates to distortion pattern. For distortion intensity,
the magnitudes of all vectors () in an optical flow such as
one of those illustrated in Fig. 10 were averaged as a scalar
Fig. 10. Optic flows representing the DD after corrective eye movements for the four lenses studied in the experiment. The DD was specific to a
rightward head rotation of 20 degrees. Both axes are in degrees and the vector magnitude have been scaled up 10 times for ease pf of
visualization.
Fig. 8. Block diagram illustrating how the model predicts DD-VOR
discomfort and distortion scores to be expected from users of VR
headsets fitted with a particular lens.
Fig. 9. (a) Example of uncompensated optical flow without considering
the corrective eye movement. X, y axes are virtual angle space with
reference to fixation as a blue cross. (b) Compensated optical flow
based on center plot where the whole map is subtracted by the vector
at the fixation point (blue cross).
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
6 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, TVCG-2021-08-0343
(S) (in degrees). A natural log function was then applied to
estimate the perceived strength of stimuli (M) with
reference to the Weber-Fechner’s law. As the threshold
stimulus is unknown, a manually tuned value of 0.016
degrees was used for modeling purposes based upon the
minimum dynamic distortion threshold collected in pilot
runs. A value of “1” was added before the logarithmic
operation to enable continuing prediction of M for S more,
equal, or less than . The relevant equations are as follows:
In Eqt. (1): S, So, vi are same as explained in the text,
and n is the total number of vectors in each optical flow.
As explained in Section 2.3, DD-VOR discomfort can be
caused by conflicting perceived motion (Fig. 6). We
propose to decompose distortion patterns into
components along translational and rotational axes from
an egocentric viewpoint. As a first attempt, the total
distortion represented by optical flow is decomposed into
16 components according to 16 patterns: upward,
downward, rightward, leftward, expansion, contraction,
clockwise (CW), anti-clockwise (anti-CW), expansion-X,
contraction-X, shear-X-CW, shear-X-anti-CW, expansion-Y,
contraction-Y, shear-Y-CW, and shear-Y-anti-CW (Fig. 12).
The selection of component directions is mainly based on
optical flow associated with natural motions such as going
forward (expansion), plus a few patterns such as shear, to
relate to the distortion score. Each of the 16 components is
determined by applying dot product between the vectors
in the original optic flow and the corresponding
component pattern. Then, the component distortion
intensity Si is calculated as the average vector’s magnitude
from the resulting optic flow after the dot product
operation. In summary, the original lens distortion as
represented by the optic flow is decomposed into 16
feature components. The decomposition of a lens
distortion into 16 features will enable finer mapping
between distortions and subjecting discomfort (Section 3.6).
A weighting parameter is then defined for each
component as a measure of the alignment of vectors of a
particular pattern with the vectors of the original optic
flow representing the lens distortion:
where is the index for the distortion pattern component
and has a value from 1 to 16 for indexing the 16 patterns.
The process of calculate is illustrated in Fig. 13. The
higher the alignment, the larger the value of . This has
been achieved by calculating the angle difference between
the corresponding vectors in the original optic flow and a
particular pattern flow. This generates a distribution of
angles, and, by fitting a von Mises distribution, the
probability of angles within -15 degrees and 15 degrees is
extracted to be the parameter which has a value from 0
to 1. The values of were determined by the DD pattern.
Different lens designs and pupil deviations from the center
of the lens would lead to different values. In other
words, for the same head rotations (hence pupil
deviations), different lenses will have different weighting
distributions of values. Likewise, for the same lens,
different head rotations will result in different weighting
distributions of values.
3.6 Modeling subjective ratings as functions of the
featured distortion
Following the above steps, the optical flow representing
the lens distortion can be transformed to a 16-dimensional
feature vector array (). This “M” quantifies the perceived
dynamic distortion during a VOR (DD-VOR). Values of M
are completely objective and can be calculated for any lens
and any VOR movement with known starting and end
locations. To establish the correlational relationships
between this objective arrays M and the subjective visual
discomfort scores, a regression model is used to estimate
the DD-VOR discomfort and distortion scores as follows:
Fig. 12. 16 different patterns with reference to fixation at the blue cross.
The red arrows represent the expected distortion direction of the
pattern. These patterns will be used as masking filter to extract and
decompose a lens distortion into 16 components.
Fig. 11. Visual sensitivity weightings along the horizontal and vertical
axes along the eccentricity of the location of in the FOV based on [29,
30 and 31].
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
CHAN ET AL.: PREDICTING SUBJECTIVE DISCOMFORT ASSOCIATED WITH LENS DISTORTION IN VR HEADSETS DURING VESTIBULO-OCULAR 7
RESPONSE TO VR SCENES
where “Score” is either the measured DD-VOR discomfort
score or the distortion score (see Section 4 on
psychophysical experiments); “” represents a constant
accounting for inter-subject variability in motion sickness
and distortion susceptibility; “ is the parameter array
relating the subjective severity scores to M (the 16 vector
maps) featuring the qualified DD-VOR; and “ represents
other factors contributing to the score that was not
controlled or not in scope of study. The values of
were estimated through regression models to fit the
reported data in the valuation experiments to be described
in the following sections with minimizing mean-squared
error. In particular, “α” was estimated from the data to
represent the data variances due to individual variations,
hence, individual susceptibility while “β” was estimated
from the data to represent mean (of subjects) data
variances due to M (the 16 vector maps), hence, average
influence of M.
4 PSYCHOPHYSICAL EXPERIMENTS
4.1 Objectives and hypothesis
To explore the perceptual effects of the dynamic distortion,
we conducted two experiments. We intended to verify, by
manipulating the visual distortion perceived by
participants during the experiment, that the dynamic
distortion would induce subjective discomfort and other
motion sickness symptoms like disorientation and
dizziness (DD-VOR discomfort). It was hypothesized that
the visual distortion would be correlated with the DD-
VOR discomfort. We hypothesized that conditions with
larger dynamic distortion magnitudes would lead to more
severe reported DD-VOR discomfort (H1), and lenses with
different distortion would cause noticeable differences in
subjective scores (H2).
4.2 Variables and designs of experiment
4.2.1 Independent variables: DD conditions
Four lenses (A, B, C and D) were used in the experiment.
They were provided by the partnering company and the
model names have been hidden. To expand the diversity
of dynamic distortion conditions to be investigated, we
simulated five modes (mode #1, #2, #3, #4 and #5) of
distortion patterns based on the existing lens designs and
four levels of absolute average intensity (average vector
magnitudes: 0.05, 0.10, 0.15, 0.20 degrees) (see Table 1).
These provided 80 different conditions (4 lenses x 5 modes
of patterns x 4 levels of intensity). In addition, the original
lens distortions were scaled to 0.5, 1 and 2 times of their
intensity. These are the 12 conditions (4 lenses x mode 1 –
the intrinsic pattern x 3 scaling factors). The original
intrinsic distortion patterns were represented by the
condition with mode 1 and unity scaling factor. The
definitions of modes are documented in Table 1. Modes 1
Fig. 13. Illustration of procedure, using the rightward component as example, to calculate each component’s probability values of : how likely
the viewer will observe this corresponding component.
TABLE 1
DYNAMIC DISTORTION CONDITIONS TESTED IN THE FIRST EXPERIMENT
Lens
Mode (Variants of lens)
Intensity of Distortion
A
B
C
D
#1
Original lens distortion
For all 5 modes, four levels of
average intensity (S = 0.05, 0.10,
0.15, 0.20 degrees) were adopted
(80 conditions).
For 4 lenses in mode #1, 3 scaling
factors (0.5x, 1x, 2x) were adopted
(extra 12 conditions).
#2
Lens distortion from 0 to 10° was applied during head rotation from 0 to 10°.
#3
Lens distortion from 10 to 20° was applied during head rotation from 0 to 10°.
#4
Lens distortion from 20 to 30° was applied during head rotation from 0 to 10°.
#5
Lens distortion from 0 to 32° was compressed and applied during head
rotation from 0 to 16 degrees.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
8 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, TVCG-2021-08-0343
to 4 used regional distortion patterns extracted from the 4
lenses and mode 5 used spatially compressed distortion
patterns extracted from the 4 lenses. These modes were
designed to expand the range of lens distortion to be
studied. In this paper, data associated with the original
intrinsic lens distortions are reported.
4.2.2 Dependent variables: Subjective distortion
and discomfort
To obtain a quantitative perceptual evaluation of
conditions of dynamic distortion, we used two predictive
subjective scales (Figs. 14 and 15). If we allow participants
to stay in each of the 92 VR conditions for 20 minutes to
incubate measurable symptoms, the experiment will take
at least 2 years to complete as participants will need more
than 7 days to recover between conditions to reduce the
effects of learning [32]. To study this phenomenon in an
efficient way, we first examined all the 92 conditions in a
single study with short exposures (1-2 minutes). To verify
the accuracy of the predictive scores, a second validation
experiment with two representative conditions selected
from the 92 conditions was conducted. During this
validation experiment, 20-minute exposures were used for
each condition (Table 1, Section 4.6). The two conditions
were generated from the original distortion of lens A with
scaling factors of 0.5x and 2x.
In the first experiment, all participants were exposed to
each of the 92 DD conditions for a short duration (about 1-
2 minutes) in random order. After each exposure, they
were asked to predict, with a number from 0 to 5, the
possible severity of dizziness, disorientation, and
discomfort if they were in that condition for 20 minutes
(Fig. 14). The number is referred to as “DD-VOR
discomfort score” in the following text. Similarly, their
perceived difference in deformation/ distortion were rated
on a separate scale, the Distortion Scale (Fig. 15). For this
question, participants were asked to give a number from 0
to 3, which is referred to as a “distortion score”.
4.3 Participants
A pilot test with three participants was conducted to
estimate the optimal sample size to distinguish two
dynamic distortion conditions. They were asked to report
the DD-VOR discomfort scores for a series of conditions
including two target conditions: lens C-mode #1 and lens
D -mode #1. Assuming that the two averaged DD-VOR
scores (2.633 for lens C-mode #1 and 1.367 for lens D-mode
#1) and the standard deviation of the difference was 0.833,
a power analyses for the paired sample t test was
conducted. Results indicated that at least six participants
were needed to get a desired power of 0.80 at the
significance level of 0.05 for the main effects that we were
seeking [33].
Nine participants (four females) from the Hong Kong
University of Science and Technology with normal or
corrected-to-normal vision participated in the experiment.
One male participant discontinued his participation due to
discomfort, and his data was therefore not included in the
subsequent analyses. Informed consent was obtained from
all participants, and the experiment was approved by the
Human Research Ethics Committee of the Hong Kong
University of Science and Technology. None of the eight
participants took part in the pilot test.
4.4 Method and procedures
4.4.1 Method
A modified commercial headset based on Oculus Rift CV1
with a customized lens module and software provided by
Facebook was used. Its intrinsic DD is close to the median
of the four lenses. The display resolution was 1440 × 1600
per eye (roughly 21 pixels per degree), the refresh rate was
90 Hz and the FOV was about 100 degrees. A visual scene
full of buildings was created based on Unity (Unity
2018.3.0 release) with a virtual asset called “Windridge City”
accessible through the Unity asset store (Fig. 16). A white
sphere in the middle of FOV with a diameter of 0.025°, was
the fixation point. The distance from the observer and the
fixation point was eight meters.
Fig. 14. Dizziness, disorientation, and discomfort scales used in the experiment. Both the English and Chinese translations were shown to the
participants.
Fig. 15. Distortion scale used in the experiment. Both the English and Chinese translations were shown to the participants.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
CHAN ET AL.: PREDICTING SUBJECTIVE DISCOMFORT ASSOCIATED WITH LENS DISTORTION IN VR HEADSETS DURING VESTIBULO-OCULAR 9
RESPONSE TO VR SCENES
The head motion of each participant was trained to be
similar in the range from 16° to 20° to the right with a 100-
bpm metronome. Head motion was controlled with a
visual feedback training session; a metronome was used as
the audio cue to regulate the head motion frequency. When
the rotation angle reached 16°, a short visual feedback bar
on the fixation point switched from horizontal to vertical.
When the rotation angle went over 20°, the vertical bar
would turn back and remain horizontal. Hearing the
metronome beats, participants would finish a round-trip
head rotation between 0 and the range of 16° to 20° in two
beats, and then rest for another beat. Fig. 17b illustrates the
visual feedback bar, and Fig. 17b shows examples of
typical head motion time histories of the participants. This
visual feedback bar was used throughout the training but
was removed during the actual experiment to avoid
distraction to the participants. Measured head motions
indicated that participants were able to maintain their
head movements within the 16° to 20° range (Fig. 17b).
4.4.2 Procedure
Participants were required to keep focusing on the fixation
point and rotate their heads back and forth from center to
the right (in a range from 16 to 20 degrees). The
background visual stimulation was space stationary (i.e.,
world-locked), and the only visual motion was from the
dynamic distortion when participants were carrying out a
VOR-type head motion. Participants were thoroughly
briefed on the procedure and the questions that they were
going to be asked. In the training sessions, ten to fifteen
conditions were randomly selected to enable participants
to become familiar with the procedure with the use of the
visual feedback bar as a control of head motion magnitude.
With several head rotations, they were able to report scores
for the presented condition. In the main experiment, the
visual feedback bar was removed as the participants were
well trained in controlling their head motions already.
Measured head motions indicated that the participants
were able to maintain their head motions within the range
of 16 to 20 degrees (Fig. 17B). Both the DD-VOR Discomfort
scores and distortion scores were collected for each
condition following the same procedure. The presentation
order of conditions was randomized, and there was a
break for five minutes after twelve conditions to avoid
fatigue.
4.5 Results and analyses
When we compared the original distortion conditions
(mode #1, scaling factor 1x) of the four lenses, different
perception scores were reported (Fig. 18). Data have been
tested to be normally distributed (Shapiro-Wilk test, DD-
VOR discomfort score: p = 0.49; distortion score: p = 0.22).
Results of paired t-tests showed significant differences in
reported distortion scores and predicted DD-VOR
discomfort scores between lenses A and C (DD-VOR
discomfort score: p = 0.0157; distortion score: p = 0.0057),
between lenses A and D (DD-VOR discomfort score: p =
0.0038; distortion score: p = 0.0056), between B and C (DD-
VOR discomfort score: p = 0.0067; distortion score: p =
0.0015), between lenses B and D (DD-VOR discomfort
score: p = 0.031), and between lenses C and D (DD-VOR
discomfort score: p = 0.0041; distortion score: p = 0.0014).
The distortion score and DD-VOR discomfort score were
strongly correlated (Pearson’s ρ = 0.746, p<0.001).
The average DD-VOR discomfort scores by the lens,
mode and intensity are illustrated in Fig. 19. The potential
of the lenses to provoke DD-VOR discomfort is in the order
of C>B≈A>D. DD-VOR discomfort scores for different
modes were found to be roughly the same. The means of
DD-VOR discomfort scores increased with the intensity.
This supports hypotheses H1 and H2 and demonstrates
the importance of correcting lens distortion in future VR
headsets.
Results of the analyses of variance (ANOVAs) on DD-
VOR discomfort scores of 80 conditions (excluding the 12
Fig. 16. Simplified illustration of a subject’s view in the experiment.
The virtual environment used was more sophisticated, but the license
agreement does not allow public display or reproduction of it.
Fig. 17. Illustrations of how head motion was controlled in the experiments. Participants were trained to rotate head horizontally to the right to a
range of 16 to 20° and back to 0°. (a) During the training session, participants could see a black bar through the fixation point and hear a 100-
bpm metronome. The bar turned vertical when the head rotated in the range of 16 to 20° and horizontal when the head rotation was out of that
range. (b) Recorded head movement time histories indicate that during the experiment, the head motion roughly ranged from 16 to 20° oscillating
in a uniform frequency.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
10 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, TVCG-2021-08-0343
intrinsic lens distortion conditions) rated by all
participants showed that the main effects of type of lens,
mode, average intensity of distortion and the interaction
between lens and average intensity were found to be
significant [Lens: F(3, 21)=12.440, p<0.001; Mode: F(4,
28)=3.114, p=0.031, Average intensity: F(3, 21)=32.969,
p<0.001; Lens*Average Intensity: F(9, 63)=6.130, p<0.001].
Post hoc analysis indicated that, except between lens A and
B, the differences between reported discomfort scores
associated with each pair of lenses were significant. The
differences between different intensity levels were also
statistically significant.
4.6 Second experiment: to validate the accuracy of
participants’ prediction of discomfort
The validity of the predicted score was verified using
scores collected from the same participants after 20-minute
exposures to selected conditions. Two conditions of lens A
with original distortion (mode # 1) were selected: one with
a scaling factor of 0.5 (A-1) and the other with a scaling
factor of 2 (A-2). The same group of participants was
exposed to each of two selected conditions for 20 minutes
with gazes focused on a fixation point and the same
horizontal head motion. At the end of each exposure, the
same group participants were asked to rate the DD-VOR
discomfort they experienced with a number from 0 to 5.
Before and after each 20-minute exposure, they were asked
to fill in the Simulator Sickness Questionnaire (SSQ)
adopted from [28] to record their symptoms and
corresponding severity. Each exposure was separated by
at least 7 days. To verify if the predicted DD-VOR
discomfort score used in the experiment was indicative of
the actual discomfort reported after a 20-minute exposure,
we compared two sets of scores. The scores predicted
within a short period were significantly correlated with the
scores rated after the 20-minute test (Pearson’s ρ = 0.521, p
= 0.038). Results of test-and-retest comparisons indicated
that the predicted and actual DD-VOR discomfort scores
are statistically consistent (Cronbach’s = 0.773)[34]. The
result verified that the quick predictive rating questions as
shown in Fig. 14 were able to measure the predictive
discomfort, dizziness, and disorientation scores as if after
the 20-minute exposure.
5 MODEL RESULTS AND APPLICATIONS
Subjectively reported data from the experiment were used
to estimate parameters in the model through an optimizer
by minimizing mean-squared error between the ratings
and predicted score. To evaluate the model results, the
data from the experiment was separated into a training set
and a testing set. One of the goals of this study was to
estimate the performance of the design of a new lens,
which is the lens B in the experiment. The testing set was
therefore made up of subjective reported data of lens B
conditions. The rest of the data from lenses A, C and D
formed the training set.
5.1 Training and Results
Fig. 20 illustrates the results of a comparison between
reported scores and predicted scores in the experiment
categorized into a training set and a testing set. The
Fig. 18. Comparison of DD-VOR discomfort scores (Left) and distortion scores (Right) for the four original lens conditions (lens A, B C and D,
mode #1). The significant differences in corresponding perception scores between lenses are labelled with asterisk(s) in each figure (Paired t-
test, * for p < 0.05 and ** for p < 0.01).
Fig. 19. Mean DD-VOR discomfort scores of different lenses, modes, and average intensity: data from 80 conditions (4 lenses * 5 modes * 4
Average intensities). Results of ANOVA can be found in the text. Main effects of lens and average intensity are significant (p<0.001).
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
CHAN ET AL.: PREDICTING SUBJECTIVE DISCOMFORT ASSOCIATED WITH LENS DISTORTION IN VR HEADSETS DURING VESTIBULO-OCULAR 11
RESPONSE TO VR SCENES
prediction generally aligned with the data from the
experiment. To benchmark the model, a naïve baseline
model using mean value of ratings as the prediction was
compared. Five-fold cross-validation was conducted by
randomly splitting the training dataset into 5 equal sized
groups, where each validation using 4 groups’ data as
training data and 1 group’s data as the validation test data.
Results are summarized in Table 2, where mean absolute
error (MAE), root-mean-square error (RMSE), normalized
RMSE (NRMSE) by dividing range of measured data, and
R-squared values are shown. Comparing the NRMSE, the
model performed slightly better in predicting DD-VOR
discomfort than Distortion scores, as shown in Table 2,
which may be because the design of the 16 features in the
model leaned towards global optic flow of self-motion. A
skewed outcome in Distortion score prediction is also
observed in that the model tends to over-estimate the low-
severity conditions and under-estimate the high-severity
conditions.
5.2 Predicting new lens performance
To evaluate the model’s ability to predict the lens’s
overall performance, data are grouped by different lens
designs, and the comparison between mean predicted
scores and reported scores are plotted as bar charts in Fig.
21. The prediction outcomes of DD-VOR discomfort by
lens were comparable to the mean reported scores,
including the test lens (i.e., lens B used in the experiment),
while the Distortion score prediction was under-estimated
for the test lens (Fig. 21). This suggests there may be some
optical features that were not captured by the current
version of the model. Further investigations of the DD
patterns of the four lenses indicated that DD of lens B has
more neighboring optic flow vectors that are abruptly
different while the corresponding changes are smoother
with the other three lenses (Fig. 10). Future work is needed.
We also acknowledge that the current 16 patterns as shown
in Fig. 12 covers more than just the horizontal axis which
was the dominating axis of head rotations investigated in
the experiments. Indeed, internal weightings (pi) inside the
model also confirm this observation. Future research will
involve head rotations in all directions.
TABLE 2
MODEL PERFORMANCE METRICS
DD-VOR discomfort scores
Distortion scores
MAE
RMSE
NRMSE
R-squared
MAE
RMSE
NRMSE
R-squared
Baseline model
0.764
0.942
0.200
0
0.502
0.622
0.222
0
Our model:
Training set
0.501
0.642
0.137
0.536
0.371
0.457
0.163
0.461
5-fold cross-
validation
0.513
0.653
0.140
0.503
0.382
0.470
0.168
0.427
Testing set
0.547
0.683
0.145
0.450
0.415
0.519
0.185
0.274
Fig. 20. Comparison of reported scores and predicted scores. Each dot represents a tested condition from a participant in the experiment.
Fig. 21. Comparison of mean reported scores and predicted scores grouped by lens, including the test lens (lens B).
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
12 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, TVCG-2021-08-0343
5.3 Estimating ratings at different percentiles of
population
Subject’s constants were used in the model development
(3). To account for inter-subject variability, a normal
distribution was fitted to the subjective predictive rating
data of all subjects in the first experiment. A scale at
different percentile of population can be estimated
through the fitted curve as shown in Fig. 22 to determine
an estimated specification for larger population in
evaluating optical flow. When evaluating lens designs or
optical flow’s impact on human perception, the scale at the
90th percentile can be chosen to cover a larger population
in scoring. In other words, the model can be used to predict
the scores for a chosen percentile of population and can be
used for the industry.
6 DISCUSSION
With the public’s increasing expectation of perfection in
VR experiences, the quest for providing a discomfort-free
VR experience is intensifying. Following the sensory
conflict theory and the rest-frame hypothesis, motion
sickness provoked with dynamic moving VR scene has
been the subject of many studies [2 to 24]. The general
conclusion is that a vection-inducing VR moving scene is a
major cause of motion sickness. On the other hand, there
are situations in which viewers experience discomfort
during normal viewing of VR world-locked scenes without
vection- inducing VR moving scenes. This latter situation
is important, but research is incomplete. This study fills
this gap by presenting empirical data as well as a validated
prediction model. Results from the user experiments
indicated that subjective discomfort (dizziness,
disorientation) is significantly affected by the type of lens
and intensity of visual distortion, with their two-way
interactions being significant as well [Lens: F(3, 21)=12.440,
p<0.001; Intensity: F(3, 21)=32.969, p<0.001; Lens*Intensity:
F(9, 63)=6.130, p<0.001]. This is a new and important
finding as it suggests that a VR user inspecting a world-
locked VR scene could experience discomfort even in the
absence of vection-inducing motion. For the first time, the
dynamic distortion (DD) caused by lens during VOR type
head motion has been shown to cause significant increases
in discomfort, disorientation, and dizziness. Furthermore,
a validated model to quantify and predict the subjective
discomfort as functions of lens design has been reported.
6.1 Applications
The most direct application of the work is twofold. First, it
provides concrete evidence that lens distortion remaining
after correction by image pre-transformation, can cause
visual discomfort, disorientation, and dizziness. Second,
the model provides a tool to predict subjective discomfort
associated with different lens designs. The scope of
application of the current predictive models goes beyond
just lens distortion during VOR. For example, camera
distortion to image pixels in mixed reality VR design can
also be represented in the form of optical flow and hence,
the similar methodology reported in this study can be
extended to predict the associated visual discomfort scores
as well.
With the quest for VR headsets with a wider field-of-
view, lens design and associated distortion correction
methods become ever more challenging. The model
presented in this study represents a way to digitally
prototype and predict user feedback before manufacturing
a lens. This will not only reduce costs but also enable
greater experimentation on lens design.
The reported work represents the first study that has
directly examined and modelled the influence of the
remaining lens distortion in VR systems after their
(imperfect) compensation that has erratically assumed the
pupil locations always remain at the center of the VR
displays. This remaining lens distortion is called dynamic
distortion (DD). This work is new and is linked to the
literature on visually induced motion sickness. As this DD
occurs in nearly all VR systems, the authors hope that this
first attempt will be followed by more research so that
future VR users can enjoy discomfort-free experience.
6.2 Limitations and future work
Being the first attempt on this topic, we acknowledge that
the present study has limitations. All DDs examined were
associated with four commercial VR lens and rightward
head rotations of 16 to 20 degrees. Also, only one virtual
environment was tested with eight participants. Future
work to examine more distortion conditions, different
types of head motions and involve more participants are
desirable. Due to non-disclosure agreement, we are unable
to disclose the specific model numbers of the commercial
lenses. Notwithstanding that, the representative distortion
patterns of the 4 lenses used have been presented in Figs. 5
and 10.
Fig. 22. Subject’s constant (a) resulted from the model and corresponding percentiles estimated by normal distribution.
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
CHAN ET AL.: PREDICTING SUBJECTIVE DISCOMFORT ASSOCIATED WITH LENS DISTORTION IN VR HEADSETS DURING VESTIBULO-OCULAR 13
RESPONSE TO VR SCENES
The current prediction model underestimated the
distortion scores for the test lens. One possible reason is the
abruptly changing DD pattern leading to a jerky rather
than smooth distortion. We acknowledge that none of the
current 16 pattern features were designed to capture the
abruptness of DD. More studies are needed to improve the
prediction models.
We acknowledge that a perfect eye tracker can
theoretically eliminate DD-VOR Discomfort by monitoring
the pupil location and perform adaptive DD compensation.
However, the shortest response delay of current state-of-
the-art eye-trackers in VR is around 15 ms [35]. A saccade
can have a speed up to 500 degrees per second and a delay
of 15 ms will introduce non-trivial tracking error that can
be counterproductive. Notwithstanding that, how the
response delays of eye tracking system will affect DD-VOR
discomfort and is there a minimal delay for eye tracking
above which the use of eye tracking will be counter-
productive will be desirable future work.
7 CONCLUSION
The current study investigated and modeled the
perceptual effects associated with remaining
uncompensated lens distortion. Such distortions can cause
unintended optical flow, called dynamic distortion (DD),
during head rotations. Experimental data indicated that
increases in DD can significantly increase discomfort
scores (called DD-VOR discomfort) and perceived image
distortion (p<0.001, ANOVAs). A better design of lenses
has been shown to significantly reduce DD-VOR
discomfort (p<0.001, paired t-test). A model has been
developed to predict DD-VOR discomfort and distortion
scores for new lens designs. The predictions from the
model are consistent with the results from the user
experiments. The predicted ranking of lenses in terms of
comfort is also found to be consistent with the expected
quality of the lens design.
In summary, this study evaluated and predicted the
user experience during a VR experience with headsets
featuring different lens designs for the first time. The
model developed in this study can guide the development
of new designs of optical layout as well as to evaluate
performances of existing lenses. Since DD occurs in all VR
lens, future work to improve and expand the scope of
applications for the model is desirable.
ACKNOWLEDGEMENTS
This study is partially funded by Facebook Reality
Laboratory and Hong Kong University Grants Council.
The authors would like to thank Phil Guan and Brant
Lewis from Facebook Reality Laboratory for their
constructive comments, and Yudong He and Tingyi Wang
for their help on conducting experiment.
Corresponding author: T.T. Chan
REFERENCES
[1] G. A. Koulieris, K. Akşit, M. Stengel, R. K. Mantiuk, K. Mania,
and C. Richardt, “Near‐Eye Display and Tracking Technologies
for Virtual and Augmented Reality,” Comput. Graph. Forum,
vol. 38, no. 2, pp. 493–519, May 2019.
[2] J. D. Prothero, “The Role of Rest Frames in Vection, Presence and
Motion,” 1998.
[3] D. Parker and J. Prothero, “A Unified Approach to Presence and
Motionsickness,” in Virtual and Adaptive Environments, CRC
Press, 2003, pp. 47–66.
[4] Z. Cao, “The effect of rest frames on simulator Discomfort
reduction,” no. July 2017, 2017.
[5] J. T. Reason, “Motionsickness: a special case of sensory
rearrangement.,” Adv. Sci., vol. 26, no. 130, pp. 386–93, Jun. 1970.
[6] C. M. Oman, “Sensory conflict in motionsickness: an Observer
Theory approach,” in Pictorial Communication In Real And
Virtual Environments, 1991.
[7] A. Kinsella, "The effect of 0.2 Hz and 1.0 Hz frequency and 100
ms and 20-100 ms amplitude of latency on simulatory sickness
in a head mounted display," M.S. thesis, Department of Applied
Psychology, Clemson University, 2014. [Online]. Available:
https://tigerprints.clemson.edu/all_theses/1848/
[8] M. E. St. Pierre, S. Banerjee, A. W. Hoover, and E. R. Muth, “The
effects of 0.2 Hz varying latency with 20-100 ms varying
amplitude on simulator Discomfort in a helmet mounted
display,” Displays, vol. 36, pp. 1–8, Jan. 2015.
[9] M. Ramshaw and S. Cutchin, “Reducing Motionsickness
Resulting From Movement inside Virtual Reality Environments,”
in International Conference on Artificial Reality and Telexistence
Eurographics Symposium on Virtual Environments, 2020.
[10] S. Palmisano, R. S. Allison, and J. Kim, “Cybersickness in Head-
Mounted Displays Is Caused by Differences in the User’s Virtual
and Physical Head Pose,” Front. Virtual Real., vol. 1, p. 587698,
Nov. 2020.
[11] R. H. Y. So, “An investigation of the effects of lags on motion
sickness with a head-coupled visual display,” in United
Kingdom Informal Group Meeting on Human Response to
Vibration, 1994.
[12] D. J. Chen, B. Bao, Y. Zhao, and R. H. Y. So, “Visually induced
motionsickness when viewing visual oscillations of different
frequencies along the fore-and-aft axis: keeping velocity versus
amplitude constant,” Ergonomics, vol. 59, no. 4, pp. 582–590, Apr.
2016.
[13] R. H. Y. So, W. T. Lo, and A. T. K. Ho, “Effects of Navigation
Speed on Motionsickness Caused by an Immersive Virtual
Environment,” Hum. Factors J. Hum. Factors Ergon. Soc., vol. 43,
no. 3, pp. 452–461, Sep. 2001.
[14] C. C. T. Guo, D. J. Z. Chen, I. Y. Wei, R. H. Y. So, and R. T. F.
Cheung, “Correlations between individual susceptibility to
visually induced motion sickness and decaying time constant of
after-nystagmus,” Appl. Ergon., vol. 63, pp. 1–8, Sep. 2017.
[15] B. Keshavarz, A. E. Philipp-Muller, W. Hemmerich, B. E. Riecke,
and J. L. Campos, “The effect of visual motion stimulus
characteristics on vection and visually induced motion sickness,”
Displays, vol. 58, pp. 71–81, Jul. 2019.
[16] F. Bonato, A. Bubka, and S. Palmisano, “Combined pitch and roll
and cybersickness in a virtual environment.,” Aviat. Space.
Environ. Med., vol. 80, no. 11, pp. 941–5, Nov. 2009.
[17] A. Kemeny, P. George, F. Merienne, F. Colombet, and F.
Mérienne, “New VR Navigation Techniques to Reduce
Cybersickness,” 2017.
[18] A. S. Fernandes and S. K. Feiner, “Combating VR Discomfort
through subtle dynamic field-of-view modification,” 2016 IEEE
This work is licensed under a Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 License. For more information, see https://creativecommons.org/licenses/by-nc-nd/4.0/
This article has been accepted for publication in a future issue of this journal, but has not been fully edited. Content may change prior to final publication. Citation information: DOI
10.1109/TVCG.2022.3168190, IEEE Transactions on Visualization and Computer Graphics
14 IEEE TRANSACTIONS ON VISUALIZATION AND COMPUTER GRAPHICS, TVCG-2021-08-0343
Symp. 3D User Interfaces, 3DUI 2016 - Proc., pp. 201–210, 2016.
[19] S. Zhang, A. Kurogi, and Y. Ono, “VR Discomfort in continuous
exposure to live-action 180°video,” in 26th IEEE Conference on
Virtual Reality and 3D User Interfaces, VR 2019 - Proceedings,
2019, pp. 1269–1270.
[20] N. Padmanaban, T. Ruban, V. Sitzmann, A. M. Norcia, and G.
Wetzstein, “Towards a Machine-Learning Approach for
Discomfort Prediction in 360° Stereoscopic Videos,” IEEE Trans.
Vis. Comput. Graph., vol. 24, no. 4, pp. 1594–1603, Apr. 2018.
[21] W. Kim, S. Lee, and A. C. Bovik, “VR Discomfort Versus VR
Presence: A Statistical Prediction Model,” IEEE Trans. Image
Process., vol. 30, pp. 559–571, 2021.
[22] R. H. Y. So, C. M. Finney, and R. S. Goonetilleke,
“Motionsickness susceptibility and occurrence in Hong Kong
Chinese,” in Contemporary Ergonomics, Taylor & Francis, 1999.
[23] B. Keshavarz, B. E. Riecke, L. J. Hettinger, and J. L. Campos,
“Vection and visually induced motionsickness: how are they
related?,” Front. Psychol., vol. 6, p. 472, 2015.
[24] C. M. Oman, “Are evolutionary hypotheses for motionsickness
‘just-so’ stories?,” in Journal of Vestibular Research: Equilibrium
and Orientation, 2012, vol. 22, no. 2–3, pp. 117–127.
[25] G. H. Crampton, Motion and Space Discomfort. 1990.
[26] A. de Castro, S. Barbero, S. Ortiz, and S. Marcos, “Accuracy of
the reconstruction of the crystalline lens gradient index with
optimization methods from Ray Tracing and Optical Coherence
Tomography data,” Opt. Express, vol. 19, no. 20, p. 19265, Sep.
2011.
[27] G. M. Stratton, “Vision without inversion of the retinal image,”
Psychol. Rev., vol. 4, no. 4, pp. 341–360, Jul. 1897.
[28] R. S. Kennedy, N. E. Lane, K. S. Berbaum, and M. G. Lilienthal,
“Simulator Discomfort Questionnaire: An Enhanced Method for
Quantifying Simulator Discomfort,” Int. J. Aviat. Psychol., vol. 3,
no. 3, pp. 203–220, Jul. 1993.
[29] F. H. McColgin, “Movement Thresholds in Peripheral Vision,” J.
Opt. Soc. Am., vol. 50, no. 8, p. 774, 1960.
[30] S. P. McKee and K. Nakayama, “The Detection of Motion in the
Peripherl Visual Field,” Vision Res., vol. 24, no. l, pp. 25–32, 1984.
[31] H. Strasburger, I. Rentschler, and M. Jüttner, “Peripheral vision
and pattern recognition: A review,” J. Vis., vol. 11, no. 5, pp. 1–
130, 2011.
[32] J. D. Moss, J. Austin, J. Salley, J. Coats, K. Williams, and E. R.
Muth, “The effects of display delay on simulator Discomfort,”
Displays, vol. 32, no. 4, pp. 159–168, Oct. 2011.
[33] Power Analysis for Two-group Independent sample t-test | R
DATA ANALYSIS EXAMPLES,” 2008. [Online]. Available:
https://stats.idre.ucla.edu/r/dae/power-analysis-for-paired-
sample-t-test/. [Accessed: 09-Nov-2021].
[34] R. Eisinga, M. Te Grotenhuis, and B. Pelzer, “The reliability of a
two-item scale: Pearson, Cronbach, or Spearman-Brown?,” Int. J.
Public Health, vol. 58, no. 4, pp. 637–642, 2013.
[35] N. Stein, D. C. Niehorster, T. Watson, F. Steinicke, K. Rifai, S.
Wahl, M. Lappe, “A Comparison of Eye Tracking Latencies
Among Several Commercial Head-Mounted Displays,” I-
Perception., vol. 12, no. 1, pp. 1-16, 2021.
Chan, Tsz Tai
Tsz Tai Chan received a B.Eng. in Mechanical
Engineering (2017) and M.Phil. in Industrial
Engineering and Decision Analytics (2021)
from the Hong Kong University of Science and
Technology. He has experience on various
data analytics projects in manufacturing and
healthcare industry. He is now working as an
optical engineer at the Facebook Reality Lab.
His main interests are computational models
for analyzing visual data, especially 3D computer vision methods and
their applications.
Wang, Yixuan
Yixuan Wang received a BSc in Science
(2017), majoring in Physics, from Liyun
College, Beijing Normal University, Beijing,
China. She is currently a PhD student at the
Hong Kong University of Science and
Technology, under the supervision of Prof.
Richard So. Her research interests include
investigating behavioral and physiological
indicators, modeling individual differences in
visually induced motion sickness and exploring the mechanism of
visually induce motion sensations.
So, Richard Hau Yue
Prof. Richard SO received a B.S. in Electronics
Engineering (1987) and Ph.D. in Applied
Sciences (1995) from University of
Southampton. He joined the Hong Kong
University of Science and Technology in 1995
and is a Professor in Industrial Engineering and
Decision Analytics. He studies human
binocular vision and develops computational
models of human vision. He is an elected
Fellow of International Ergonomics Association, Chartered Institute of
Ergonomics and Human Factors, and the Hong Kong Institute of
Engineers. He has authored and co-authored more than 50 referred
journal manuscripts and more than 100 conference papers.
Jia, Jerry
Jerry Jia is a system engineer and human
perception specialist at Facebook Reality Lab.
He advocates for user experience-centric
product design and an integration of hardware
system, algorithms and human vision system in
product development for virtual reality and
augmented reality applications. He received his
Ph.D. in Material Science and Engineering
from Columbia University in the City of New
York (2011), B.S. in Physics (2005) and B.A. in Philosophy (2005)
from Peking University in Beijing.
