ArticlePDF Available

An fMRI Study to Analyze Neural Correlates of Presence during Virtual Reality Experiences

Authors:

Abstract and Figures

In the field of virtual reality (VR), many efforts have been made to analyze presence, the sense of being in the virtual world. However, it is only recently that functional magnetic resonance imaging (fMRI) has been used to study presence during an automatic navigation through a virtual environment. In the present work, our aim was to use fMRI to study the sense of presence during a VR-free navigation task, in comparison with visualization of photographs and videos (automatic navigations through the same environment). The main goal was to analyze the usefulness of fMRI for this purpose, evaluating whether, in this context, the interaction between the subject and the environment is performed naturally, hiding the role of technology in the experience. We monitored 14 right-handed healthy females aged between 19 and 25 years. Frontal, parietal and occipital regions showed their involvement during free virtual navigation. Moreover, activation in the dorsolateral prefrontal cortex was also shown to be negatively correlated to sense of presence and the postcentral parietal cortex and insula showed a parametric increased activation according to the condition-related sense of presence, which suggests that stimulus attention and self-awareness processes related to the insula may be linked to the sense of presence.
Content may be subject to copyright.
© The Author 2013. Published by Oxford University Press on behalf of The British Computer Society. All rights reserved.
For Permissions, please email: journals.permissions@oup.com
doi:10.1093/iwc/iwt037
An fMRI Study to Analyze Neural
Correlates of Presence during Virtual
Reality Experiences
Miriam Clemente
1
, Beatriz Rey
1,2
, Aina Rodríguez-Pujadas
3
, Alfonso
Barros-Loscertales
3
, Rosa M. Baños
2,4
, Cristina Botella
2,5
, Mariano
Alcañiz
1,2
, and César Ávila
3
1
I3BH/LabHuman, Instituto Interuniversitario de Investigación en Bioingeniería y Tecnología orientada al
Ser Humano, Universitat Politècnica de València, Valencia, Spain
2
Ciber, Fisiopatología Obesidad y Nutrición, CB06/03 Instituto de Salud Carlos III, Spain
3
Departamento de Psicología Básica, Clínica y Psicobiología, Universitat Jaume I, Castellón de la
Plana, Spain
4
Labpsitec, Universitat de Valencia, Valencia, Spain
5
Labpsitec, Universitat Jaume I, Castellón, Spain
Corresponding author: mclemente@labhuman.i3bh.es
Inthefieldofvirtualreality(VR),manyeffortshavebeenmadetoanalyzepresence,thesenseofbeing
in the virtual world. However, it is only recently that functional magnetic resonance imaging (fMRI)
hasbeenusedtostudypresenceduringanautomaticnavigationthroughavirtual environment.In the
present work, our aim was to use fMRI to study the sense of presence during a VR-free navigation
task, in comparison with visualization of photographs and videos (automatic navigations through
the same environment). The main goal was to analyze the usefulness of fMRI for this purpose,
evaluating whether, in this context, the interaction between the subject and the environment is
performed naturally, hiding the role of technology in the experience. We monitored 14 right-handed
healthy females aged between 19 and 25 years. Frontal, parietal and occipital regions showed their
involvementduring free virtual navigation.Moreover,activationinthe dorsolateral prefrontalcortex
was also shown to be negatively correlated to sense of presence and the postcentral parietal cortex
and insula showed a parametric increased activation according to the condition-related sense of
presence, which suggests that stimulus attention and self-awareness processes related to the insula
may be linked to the sense of presence.
RESEARCH HIGHLIGHTS
We analyzed the sense of presence experienced in a virtual environment using fMRI.
Free navigation in a virtual environment was compared with video and photographs.
The main presence results obtained for the comparison between navigation and video.
Cuneus, post-central parietal lobe and insula were the principal activated areas.
Negative correlation with questionnaires found in the dorsolateral prefrontal cortex.
Keywords: presence; virtual reality; human computer interaction (HCI)
Editorial Board Member: Prof. Timothy Bickmore
Received 31 October 2012; Revised 4 April 2013; Accepted 29 June 2013
1. INTRODUCTION
Virtual reality (VR) is a technology that has been widely
used to simulate reality. One of the most important concepts
of measuring in VR applications is presence. Presence can
be defined as the sense of being there, inside the virtual
environment (VE), although your body is physically located
Interacting with Computers, 2013
Interacting with Computers Advance Access published July 31, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
2 Miriam Clemente et al.
elsewhere (Baños et al., 2000; Sadowski and Stanney, 2002;
Sheridan, 1992; Slater and Wilbur, 1997). In our research,
we used functional magnetic resonance imaging (fMRI) to
evaluate brain activation associated with sense of presence
duringnavigationin aVE, which, as faraswe know, is amethod
that has not yet been applied for this purpose.
The measure of presence has a major role in the study of
the human–computer interaction (HCI), in order to improve the
designofVEsandtomeasuretheeffectthatthoseimprovements
have over the subject. If the subject’s sense of presence
increases inside the VR, it means that the interaction between
the computer-generated world and the subject has improved.
Accordingto Riva et al.(2003), ‘asmediabecomes increasingly
interactive,perceptuallyrealistic andimmersive,the experience
of presence becomes more realistic’. Moreover, as Sjölie et al.
(2010) remarked, measuring the brain activity while interacting
naturally with the system allows for correlating the activity in
specific brain areas with hidden cognitive states, which can be
related to aspects of the interface and the interaction. The final
aim in HCI is to develop systems ‘that minimize the barrier
between the human’s cognitive model of what users want to
accomplishand the computer’sunderstanding of theusers’task’
(Sharp et al., 2007). If we can disclose the neural correlates
that hide behind the sense of presence, maybe we will be able
to develop adaptive brain–computer interfaces (BCIs) which
allowthe environmentto change depending on the user’s needs.
This will help one to accomplish a satisfying experience for the
user, making the VEs disappear from the subject’s awareness
(disappearance of mediation; Riva et al., 2003).
Although presence is inherent to exposure to VEs, most
previous studies that combined VR with fMRI did not evaluate
this concept or its influence on the virtual experience. The
first studies in this area aimed to demonstrate the value of
combining both techniques and the different possibilities that
this association could offer (e.g., Baumann et al., 2003; Mraz
et al., 2003). Other studies have analyzed brain activation
associated with the performance of specific navigation tasks
in controlled VEs. For example, Astur et al. (2005) analyzed
spatial memory during navigation in a virtual radial arm maze
and reported changes in bilateral hippocampus activation. Pine
et al. (2002) conducted a study of the neuronal correlates of
different spatial navigation conditions using VEs as stimuli.
They found that navigation ability correlated with activation
in different regions, including the right-frontal and right-
anterior medial temporal lobe. Hartley et al. (2003) also used
fMRI to compare brain activations between two experimental
conditions: wayfinding and route following. They obtained
between-subjects correlation of activation with performance
in the anterior hippocampus for the former condition, and
in the head of caudate for the latter. In another study,
Mellet et al. (2010) compared brain activations while mentally
estimating the distances between landmarks placed in a
learned environment. One group of subjects performed the
learning during a real walking task and the other in an
equivalent virtual walking task of the same environment.
They found a left-lateralization of the brain activations
in the virtual learners, compared with the real learners.
In another study, Sjölie et al. (2010) analyzed the brain
activations due to a mental rotation task. Although they did not
measure presence, two other important VR parameters were
analyzed: 3D-motion and interactivity. For this purpose, they
compared between three experimental conditions: one without
motion (still), one with automatic non-interactive motion
(auto) and the last one with interactively controlled motion
(interactive). They found that the addition of interactivity
increased the activation of the main areas of the mental
rotation network, including frontal and preparatory motor
areas.
Turning now to presence, several authors have theorized
about the concept and its implications (e.g., Kim and Biocca,
2006;Lombard andDitton,2006; Schuemieet al.,2001). Inside
a VE, presence refers to the sense of being in it instead of
being in the real room where the experience is taking place.
According to Heeter (1992), in the natural world, this process is
engaged since birth; while in theVE this derivesfrom feeling as
if you existed as a separate form within a virtual world that also
existed. Sanchez-Vives and Slater (2005) also pointed out that
insidethe virtualexperience, youare atthe sametime conscious
of the ‘place’ and the ‘events’ and simultaneously conscious
of that there are no such place of events; however, you still
behave and think as if the place were real and the events were
happening. As your consciousness of the differences between
the real and virtual place and events blurs, the barrier between
your mind and the VE diminishes, improving your interaction
with the computer-generated world. And that is because, as
Loomis (1992) remarked, ‘presence is a fundamental property
ofconsciousness’.Therefore,it isunlikelyto beunidimensional
(Kim andBiocca, 2006).The International Society for Presence
Research (2000)
proposed that presence could be considered
from
several major dimensions, based on the findings of
different studies in the matter. The first dimension is spatial
presence, the subjects’ belief that they are really inside the VE.
The second issensory presence, which is related to the subjects’
perception of the VE as they would perceive the real world,
divided into visual, auditory and tactile perception. Social
realism refers to the subjects’ perception that objects, events
and people that appear in the VE could exist in the real
world. Engagement occurs when the subject feels the VE to
be involving. Finally, social presence refers to communication
with other people or entities inside the VE.
However, many theories about what is presence have been
formulated. For example, presence has been defined as the
facultyofbeing-in-the-world (Flachand Holden,1998;Zahorik
and Jenison, 1998). In this assumption, presence is described
as the ‘successfully supported action in the environment’.
As Sanchez-Vives and Slater (2005) explained, following this
theory, presence would not only be the sense of ‘being there’,
but also the ability to ‘do’ there. In this line of argument, the
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
Correlates of Presence during Virtual Reality 3
possibilityof movementthrough theenvironmentby thesubject
would increase his sense of presence.
Regarding measurement techniques for presence, traditional
methods are questionnaire-based (e.g., Baños et al., 2000;
Lessiter et al., 2001; Usoh et al., 2000; Witmer and Singer,
1998). To avoid the problems inherent to subjective measures,
objective techniques have also been proposed, mainly based on
psychophysiologicalmeasures. Forexample,as skinconductiv-
ity and heart rate can be related to the anxiety level of the users,
theycould constitute an indicator ofthe presence that the user is
feeling in environments that generate these kinds of responses
(Dillon et al., 2000; Meehan et al., 2001). Furthermore, recent
studies have analyzed presence from a neuroscience point of
view, and it has been stated that VR is not only a tool for neu-
roscience, but that presence in VEs is also an object of study
for neuroscientists (Sanchez-Vives and Slater, 2005).
We can also remark some studies which focused on the con-
cept of presence using other brain activation related techniques,
such as transcranial Doppler (TCD) or electroencephalography
(EEG). Two recent studies (Alcañiz et al., 2009; Rey et al.,
2010) proposed that TCD could be used as a brain activitymea-
surement technique to study presence in VEs. Results of these
two studies showed changes in blood flow velocity in the major
cerebral arteries of the participants during moments associated
withdifferentlevelsof presence indifferentimmersiveand nav-
igational conditions.With regardto EEG,we can emphasizethe
work developed by Baumgartner et al. (2006), which compared
activation in children and adolescents while watching a video
of a virtual roller coaster and concluded that it evoked activa-
tionsinparietalbrainareas.Furthermore,thechildrenreporteda
higherspatialpresenceexperiencethantheadolescents(showed
by a lower activation in the frontal brain).
However, as already indicated, there are no fMRI studies
that have analyzed brain activation associated with presence
during a free navigation in a VE. The advantages that fMRI
provides compared to other image techniques are based on its
betterspatial resolutiontogetherwith adecenttemporal one,the
factthatit isnotinvasiveand thatdoesnot useionizing radiation
(unlike other image techniques such as computed yomography
or X-rays), not provoking secondary effects on the subject.
The spatial resolution of the technique is suitable to observe
specific brain areas and neural networks that are activated
during the presence experience, which may not be possible
with other techniques. As far as we know, the only previous
related study was conducted by Baumgartner et al. (2008),
which applied fMRI to analyze brain activation associated with
presence in VR. However, they did not use VEs in which
the participants could navigate freely. They compared brain
activation in children and adults while the subjects watched
a VR video of an automatic navigation, with the same roller
coaster scenario used in the previous EEG study (Baumgartner
et al., 2006), to identify which areas of the brain were activated
with the experience of presence and to identify differences
between the age groups due to brain maturity. They compared
the differences between a high presence and a low presence
environment. Results from the fMRI analysis showed that the
presenceexperienceevokedbythevirtualrollercoasterscenario
was associated with an increase in activation in a distributed
network. A preliminary neuroanatomical model of presence
can be extracted from these results. As discussed later by
Jäncke et al. (2009), the distributed network that is activated
during the presence experience includes the dorsal and ventral
visual stream, the parietal cortex, the premotor cortex, the
mesial temporal areas (including the hippocampus, amygdala
and insula), the brainstem and the thalamus. Baumgartner et al.
(2008) explained the activation of the parietal lobe based on the
definition of presence as ‘an egocentric spatial experience of
VEs’. The parietal lobe is believed to be activated as a result
of its role in ‘generating an egocentric view by translation
of the retinal coordinates to head-centered coordinates’. They
also concluded that there are some brain areas, such as the
prefrontal cortex, that continue maturing throughout life and
may be associated with ‘the modulation of the inter-individual
differences in the experience of presence’. The more intense
the sense of presence, the smaller was the activation in the
dorsolateral prefrontal cortex (DLPFC) (Jäncke et al., 2009).
They also remarked the activation of the insula as part of
the distributed network that generates presence. The insula
is considered to participate in the sense of self-awareness
and body-ownership, which leads to the formation of your
‘body schema’. This means that if this area is activated in
the participants of a presence-inducing task, their feeling of
embodiment inside the VE will have increased during the
experience, making the barrier between the virtual and the real
body disappears. This will contribute to an improvement in the
participants’interaction with theVE. This concept is important
when considering HCI applications of presence.
In the present study, the main goal was to analyze if brain
imaging techniques can be used to evaluate the sense of
presencestimulatedwhile thesubjectinteracts withacomputer-
generatedenvironment.Forthispurpose,welookedforthebrain
areas that were activated in relation to the sense of presence
during a VR paradigm. However, the most relevant objective of
the present work is not only to map the brain areas related to
presence, but also to study if the sense of presence itself can
be stimulated and measured using fMRI. If this can be proved,
it will mean that the interaction between the environment and
the subject is performed naturally, and that the technology has
become ‘invisible’to the subject. In a previous publication, the
questionnaire results of thiswork were presentedin comparison
with other brain imaging techniques (Clemente et al., 2011),
now we want to complete those questionnaire results with
those obtained from the fMRI data. Once the importance of the
concept of presence inside the HCI theory is understood, fMRI
can be seen as the tool to analyze the existence of the sense of
presencein a moreobjectivewaythan theuse of questionnaires.
What is more, fMRI allows the continuous measure of slow
variations in the sense of presence in time, so it will allow us to
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
4 Miriam Clemente et al.
have a complete report of the HCI along time; thing that cannot
be accomplished with the traditional measurement tools. This
can workas asteady feedback ofthe efficiencyof theVR world.
We know from previous studies that the sense of presence is
influenced by the possibility of self-controlling the navigation
(Alcañiz et al., 2009; Welch et al., 1996). In order to analyze
the brain activation associated to changes in the level of
presence, different navigation paradigms will be compared
in our experimental design. Specifically, we will compare
brain activation during an experimental condition where
the participants could navigate freely, with less immersive
configurations (visualization of still imagesof the environment,
and visualization of an automatic navigation—video—through
the same environment).The selection of the three experimental
conditions has been made based on the definition of
Sanchez-Vives and Slater (2005) of the concept of presence as
the ability to ‘do’ inside the VR, so the more you do inside the
VE, the more presence you will feel. Comparing the conditions
of free and guided navigation (video), we expect to measure
the differences in the level of presence due to the self-control
of the movement; while the still-photographs condition will act
as baseline condition. From this point of view, the increase of
activity between the three experimental conditions would be
translated to an increase in the sense of presence, losing the
consciousness of the existing barrier between the real and
the virtual world. In order to ensure that there were differences
in the level of presence between the different experimental
conditions, the sense of ‘being there’ was evaluated by means
of a validated questionnaire (Usoh et al., 2000), which has
been applied to obtain a subjective measure of the spatial
dimension of presence in the different conditions and subjects.
The main hypothesis of our research was that brain activation
would be higher during a navigation task than during a video
or photograph task in areas such as the cuneus and the parietal
lobe, which we know are related to presence from previous
studies. Taking into account the results of Baumgartner et al.
(2008), we also expected to find negative correlations between
the activation in the DLPFC and self-reported presence scores.
2. MATERIALS AND METHODS
2.1. Subjects
For this study, we recruited 14 right-handed women, none
of them with any medical or psychological disorders, aged
between 19 and 25 (mean age 21.64). The participants’
hand dominance was tested using the Edinburgh Handedness
Inventory (Oldfield, 1971). These women were students, were
paid for their participation in the study and were recruited
from the Universitat Jaume I in Castellón and the Universitat
Politècnica de València.
Ethical approval was obtained from the authors’ institution,
and each subject signed a written informed consent prior to
participation.
2.2. Environments
The VEs were programmed using GameStudio software
(Conitec Datensysteme GmbH, Germany), which allowed us
to develop 3D objects and virtual worlds with which we could
interact and navigate. Our VE consisted of an everyday, clean
bedroom (with a bed, a closet, and a desk with some books on
it) where participants could navigate freely.
To allow us to identify the specific areas of the brain that
were activatedfor each task, we dividedthe paradigm into three
conditions developed with the same VE: in the first, only pho-
tographs of the roomcould be visualized (four photographs dis-
playedfor4.5 seachwith0.5sofblackscreenbetweenthem);in
thesecond, avideo of anautomatic navigationthrough the same
room could be observed (with a duration of 20s); and in the last
one, the participant could navigate freely for 20s in the VE.
In order to prevent the subjects from staying still during the
navigationperiod, they were instructed to perform a search task
which forced them to move through the environment and kept
them engaged with the stimuli. This task consisted in searching
for some red keys that randomly appeared and disappeared in
the environment, and counting the number of them that they
had seen (maximum of 4, remaining in the VE for 5s). They
were not encouraged to find them all, or to find them as quickly
as possible, they were only told to continue searching for them
duringeach period.To preventdifferencesbetween thedifferent
phases ofthe experiment,this counting task wasalso performed
during the other two conditions. During the photograph period,
some of the images showed featured keys and some did not,
and the subjects had to count the number of keys they saw.
During the video task, the keys appeared randomly in the
environment as the camera moved through it. After each task,
subjects were questioned about the number of keys they had
found (they had to answer in a short period of 4s). While they
were conducting the tasks, the researcher checked that they
had answered properly. The number of keys counted is not
relevant, it was just included to avoidthe subjects to remain still
during the experimental conditions. Between phases, a black
screen appeared to give subjects a rest period during which
brain activation could decay to its baseline values (6s) before
the label indicating the next task appeared (2s). The total time
between tasks was 12s. At the beginning of the experiment
there were 14 s of black screen to compensate for T1 saturation
effects. Each of the three experimental conditions was repeated
six timesin a counterbalanced order topreventeffectsproduced
by the order in which they were presented. The total time of the
complete experimentwas 12 min52 s.A scheme of the protocol
can be seen in Figure 1.
To learn about the tasks that had to be performed inside the
scannerroom,subjectsunderwent aprior trainingsession where
they were introduced to the VR navigation and to the tasks.
Theywere alsoshownthe differencesbetween thephotographs,
videos and navigation, and practiced the hand movement using
thejoystickasitwasgoingtobedoneduringthescannersession.
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
Correlates of Presence during Virtual Reality 5
Figure 1. Diagram of the experimental design.
This training session was conducted in a supplementary VE to
preventhabituation. In order to preventdifferences inactivation
caused by the motor task, subjects were instructed to move the
joystick continuously during the video and photograph tasks
in the same way as they did during the navigation period.
The joystick movement was made just to compensate the
brain activations caused by motor tasks between the different
conditions. During the fMRI scan, the VR application checked
the total time that they spent moving the joystick in each
condition to guarantee that the motor movements had been
continuous in all the cases.
2.3. Post-fMRI questionnaires
After the scanner session, subjects had to answer the questions
in the Slater, Usoh and Steed (SUS) questionnaire (Usoh et al.,
2000) to evaluate the levelof presence that they felt during each
task. The questionnaire consisted in six, 7-point Likert-type,
questions that had to be answered depending on the strength of
the ‘being there’sensation experienced, where 1 corresponded
tonot feelingthereat alland7 tothehighest senseof beingthere
(as experienced in the real world).A midscale value of 4 would
correspond to an intermediate level of being there, experienced
by the subject as the midpoint between the feeling in the real
world and not feeling there at all. Subjects had to complete
three questionnaires, one for each task, all containing the same
questions.
2.4. fMRI procedures
All subjects were scanned in a 1.5 T Siemens Avanto Magnetic
Resonance scanning device (General Hospital, Castellón,
Spain).We used an adaptedmagnetic resonance(MR) helmetto
prevent head movement. To display the environments, we used
MRI-compatible videogoggles,VisualStimDigital (Resonance
Technology Inc., Los Angeles, USA); and, for the navigation,
we used an adapted joystick (Resonance Technology Inc.,
Los Angeles, USA). First, as is indicated for fMRI studies
(AmaroandBarker,2006),weacquiredthesagittalT1-weighted
structural images of the brain (224 × 256 matrix covering the
brain with 176 contiguous 1 mm slices, repetition time (TR) =
11ms, echo time (TE) = 4.94ms, flip angle (FA) = 15
, voxel
size = 1.04 × 1.04mm). Then the functional scanning was
launched, synchronized with the VEs. Functional images were
obtainedusingaT2
single-shotecho-planarimagingsequence.
We used 30 contiguous 4.2 mm interleaved axial slices (parallel
to the line between the anterior commissure (AC) and the
posterior commissure (PC) or the AC–PC line) covering the
entirevolumeofthebrainwitha64×64matrix(TR = 2000ms,
TE = 30 ms, FA = 90
, voxel size = 3.5 × 3.5mm).
2.5. Data analysis
2.5.1. Questionnaire analysis
We analyzed the results of the SUS questionnaires using the
program SPSS 17.0 (IBM Corporation, Somers, NY, USA).
Apart from the individual responses to the six questions
associated with each of the periods (photographs, video and
navigation), we calculated an additional measurement: SUS
mean. This is the mean score across the six questions that have
already been described in previous studies (Usoh et al., 2000).
We carried outa non-parametricFriedmanTest tocompare SUS
responses (dependent variables: questions 1–6 and the SUS
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
6 Miriam Clemente et al.
mean) for the different experimental conditions: photographs,
videoand navigation.Post hoctests weremade withaWilcoxon
Signed-Rank test with Bonferroni correction.
2.5.2. fMRI analysis
To analyze the fMRI data we used Statistical Parametric
Mapping software (SPM8, Wellcome Department of Imaging
Neuroscience, London, UK), launched with MatlabVersion 7.1
(MathWorks, Natick, MA, USA). The first seven scans were
excluded from the analysis to eliminate the decay of the fMRI
signal associated with the moment when magnetization reaches
equilibrium. The first step was to align the images to the AC–
PC line. Then we began the preprocess (Friston et al., 1995)
realigning the functional images (estimate and reslice option),
coregistering them to the structural images and segmenting this
latter anatomical scan to obtain the gray matter, white matter
andcerebrospinal fluidimages.Wethen normalizedthe resliced
functionalvolumeswith thenormalization parametersextracted
aftersegmentationandnormalizationoftheanatomicalvolumes
for each subject separately (template provided by the Montreal
Neurological Institute [MNI]). We did not have to exclude any
volunteer due to movements or distortions during the fMRI.
Finally, we smoothed the images using a Gaussian kernel (full-
width at half-maximum of 8 × 8 × 8mm).
In a first fixed-effect level analysis (with the aim of detecting
changes in the blood-oxygen-level-dependent (BOLD) signal
between conditions in a single subject), the individual contrasts
comparing between the different experimental conditions are
obtained. In this analysis, the functional time series for
each subject and for each condition were modeled with a
box-car function convolved with the hemodynamic response
function. As a result, the ‘navigation > video’, ‘navigation >
photographs’ and ‘video > photographs’ contrasts for
each subject were obtained. The parameters for the motion
correction were employed as regressors of non-interest. We
also applied a 128s high-pass filter to eliminate the low-
frequency components in the signal caused by scanner motion
and warming.
Group tests were performed at second level random effect
analysis, where the group of subjects is taken into account.
Taking into account the results obtained in previous similar
studies (Baumgartner et al., 2008; Pine et al., 2002), we tested
for task-related activation by performing a one-sample t -test
including contrast images of estimated parameters from all the
subjects for the differences of interest between conditions. In
total, we performed three one-sample t-test, for the contrasts
‘navigation > video’, ‘navigation > photographs’ and
‘video > photographs’.
Once we obtained the brain activation maps for each group-
level contrast, we conducted a second-level multiple regression
analysis to evaluate the existent relationship between brain
activation in the aforementioned contrasts (‘navigation >
video’, ‘navigation > photographs’and ‘video > photographs’)
andthesubjectivescoresfromthequestionnaires.Weperformed
three new group-level analysis (for the three contrasts of
interest),whereweusedascovariatethedifferencesbetweenthe
SUS mean results for the experimental conditions compared in
the contrast (see Baumgartner et al., 2008). The covariate then
is a vector of 14 components, one for each subject. The value
of the component of each subject is obtained by subtracting
the value of the SUS mean for the second condition of the
contrast compared from the value of the SUS mean of the
first condition of the contrast. For example, our main interest
lay in the ‘navigation > video’ contrast, where we obtained
the correlation analysis between the ‘navigation > video’
contrast and the responses from the SUS questionnaires for the
‘navigation SUS mean score—video SUS mean score’.
Finally, we studied the brain areas that showed a linear
parametric modulation of the activation levels and their
associated subjective level of presence (SUS mean of
each condition minus the global SUS mean) for the three
experimental conditions (navigation, videos and photographs),
accordingto increasedsenseof presence(followinga procedure
described in previousstudies, such as Geake and Hansen, 2005;
Scheibe et al., 2006; Smith, 2004).
Results from statistical tests at group level were considered
significant if 10 or more adjacent voxels passed the statistical
threshold of p<0.001 (uncorrected). To obtain the specific
brain areas that are activated in each contrast, we used the
xjView(http://www.alivelearn.net/xjview8/)softwareutilityfor
SPM that uses the MNI coordinates system.
3. RESULTS
3.1. Questionnaire results
The answers to the SUS questionnaire, given after fMRI
monitoring during exposure to the different experimental
conditions, showed the between-subject variations. As sense
of presence is subjective, each person can experience the
conditions with a different grade of affectation. Mean values
in each condition are shown in Table 1.
Results from applying the non-parametric Friedman Test
showedthat therewere significantdifferencesbetweenthe three
experimental conditions for all the questions except question 5
(results can be observed in Table 1, columns 5 and 6), including
theSUS mean value.If we observethe results for eachquestion,
we
can see that the greatest Chi-square value (χ
2
= 16,
p<0.001) is observed for question 1.
Posthocanalyses basedonWilcoxonSigned-Rank testswere
conducted on the SUS mean results with Bonferroni correction,
resulting in a significance level set at p<0.0167. There were
nosignificant differencesbetween the photographand thevideo
tasks (Z = 1.174, p = 0.241 > 0.0167). However, there
was a statistically significant increment in the SUS mean in
the navigation versus photographs (Z = 2.805, p = 0.005 <
0.0167) and the navigation versus video comparisons (Z =
2.550, p = 0.011 < 0.0167).
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
Correlates of Presence during Virtual Reality 7
Table 1. SUS responses to the questionnaires for each task and results of the Friedman test for each question (7-point Likert scale) and the
mean score
Photographs Video Navigation χ
2
P
SUS question 1: feeling of ‘being there’ 3.14 ± 0.39 3.79 ± 0.45 4.43 ± 0.43 16.00 <0.001
SUS question 2: feeling that the room is real 2.79 ± 0.48 3.14 ± 0.48 3.50 ± 0.50 6.75 0.034
SUS question 3: how real do you remember the room? 2.00 ± 0 .31 2.50 ± 0.42 3.14 ± 0.49 10.90 0.004
SUS question 4: feeling of being inside the room or observing it 3.14 ± 0.38 3.14 ± 0.49 4.07 ± 0.45 6.45 0.004
SUS question 5: memory of the room as similar to being in other places 3.43 ± 0.44 3.50 ± 0.42 4.00 ± 0.46 5.25 0.072
SUS question 6: did you think you were really in the room? 2.71 ± 0.37 3.00 ± 0.50 3.50 ± 0.53 6.07 0.048
SUS mean 2.87 ± 0.33 3.18 ± 0.40 3.77 ± 0.43 12.29 0.002
Note. The mean score and standard error of the mean for the 14 subjects are presented in each cell.
Table 2. Brain area activation results for the ‘navigation > video’, ‘navigation > photographs’ and ‘video > photographs’contrasts
Anatomical region H x (mm) y (mm) z (mm) t score Cluster size
Postcentral/Parietal Lobe L 47 18 59 5.7771 19
Insula R 34 0 17 5.4986 10
Navigation > Video Cuneus/Occipital Lobe R 10 91 26 5.3162 10
Extra-nuclear/Sub-lobar R 27 42 13 5.2809 16
Calcarine/Middle Occipital gyrus/Occipital Lobe R 24 98 0 5.0042 22
Cerebellum Anterior Lobe L 43 49 37 5.0356 56
Cerebellum Posterior Lobe L 8 74 25 5.8061 17
Navigation > Photographs Superior and Middle Occipital Lobe/Cuneus L 22 84 21 10.919 1085
Lingual/Occipital Lobe R 3 70 5 6.7976 83
Superior Frontal Lobe R 24 0 55 9.8225 143
Inferior Temporal Lobe/BA37 R 48 70 4 9.57 1147
Lingual Gyrus/Inter-Hemispheric R 3 74 0 5.28 38
Video > Photographs Inferior Frontal Operculum/Sub-Gyral/Frontal Lobe R 41 11 21 5.99 44
Supramarginal Gyrus/Inferior Parietal Lobe R 55 39 30 4.83 13
Middle Frontal Gyrus/Frontal Lobe R 27 0 51 5.91 73
Middle Frontal Gyrus/Frontal Lobe L 29 4 51 4.22 10
Note: H column refers to the hemisphere of each activation.
3.2. Imaging results
3.2.1. Contrasts results
Our fMRI paradigm was divided into three different tasks
(photographs,videosandnavigation)thatwewantedtocompare
to obtain the contrasting brain activations. We obtained results
for the three contrasts between tasks, the most relevant for the
purposes of our study being those concerning the differences in
activationbetweenthefreenavigationandtheguidednavigation
(video). Therefore, we selected the contrast ‘navigation >
video’ and looked for the main activated brain regions. We
foundactivationsintherightcuneusandleftparietallobeamong
others (see the upper part of Table 2 and Figure 2). Other brain
regions activated in the ‘navigation > video’ contrast were the
right calcarine, right sub-lobar and right insula.
Regarding the ‘navigation > photographs’ contrast, new
activations were seen in the left cerebellum, both in the anterior
and posterior lobes, and in the superior frontal lobe. There
were activations in some areas of the occipital lobe, such as
the cuneus, the left and right middle occipital lobe, and the
right lingual gyrus. Finally, we also found activations in areas
of the parietal lobe, such as the precuneus (see the middle part
of Table 2 and Figure 3).
Forthe ‘video > photographs’contrast, wefound activations
in the right inferior temporal lobe, the right lingual gyrus, the
right inferior frontal lobe, the right supramarginal gyrus and
the right and left middle frontal lobe (see the inferior part of
Table 2 and Figure 4). It is important to mention here that with
the inverse contrasts (‘video > navigation’, ‘photographs >
navigation’and ‘photographs > video’), we did not obtain any
significant activation results.
3.2.2. Positive and negative correlation results
We conducted a second-level multiple regression analysis for
our contrast of interest ‘navigation > video’, from which we
obtained correlations between the fMRI results and the SUS
mean values obtained from the questionnaires. On the one
hand, for the ‘navigation > video’ contrast, results showed a
negative correlation (Table 3) between the questionnaire results
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
8 Miriam Clemente et al.
Figure 2. Main brain activation results for the ‘navigation > video’ contrast, centered in the right cuneus. Results are shown in neurological
coordinates. The color bar represents statistical t -values.
and the activation observed in the DLPFC of the right frontal
lobe, which can be seen in Figure 5. On the other hand, a
positive correlation was found between the responses to the
questionnaires and activation in the left lingual gyrus, left
cerebellum anterior lobe, left middle inferior temporal lobe,
left sub-gyral, left calcarine, left superior temporal lobe, left
middle temporal gyrus and left cuneus. All these results can
be seen in Table 3. Moreover, the graph showing the positive
correlation for the lingual gyrus (the most remarkable of the
positive correlation results) is shown in Figure 6.
3.2.3. Parametric analysis results
Finally, we tested for the possible existence of an increasing
linear trend in theactivation corresponding to thethree reported
levelsofpresence(SUSmeanofeachconditionminustheglobal
SUS mean values) according to the experimental conditions.
The results showed that an increasing linear trend for the
different presence-related conditions (photographs, videos and
navigation) was observed in the activations in the right insula
(x = 41, y =−14, z = 13; t = 4.22, p<0.001, 10 cluster
size)andtheleft postcentralparietalgyrus(x =−47,y =−18,
z = 59, t = 6.67, p<0.001, 10 cluster size) for the three
experimental conditions (see Figure 7).
4. DISCUSSION
The principal aim of our study was to analyze whether subjects
could feel presence while navigating in a VE, analyzing the
results using fMRI. As mentioned in Section 1, if this is so,
this would mean that the interaction between the computer-
generated world and the subject is naturally performed and
the barrier between technology and reality has been reduced;
which is a major interest in HCI. As mentioned previously, we
tried to generate an increase in the sense of presence between
the different experimental conditions by means of the increase
in the actions the user has to perform in the VE. Presence
was then especially motivated by the free navigation condition,
where it is the user who controls the movement along the
environment. This free navigation in aVE was shown to induce
a higher feeling of presence in the participants than a guided
navigation condition (that in a comparison would act as the
low presence condition). Contrasting the functional activation
seen during these two conditions (‘navigation > video’),
results showed a higher activation of the parietal and occipital
brain regions, including the cuneus, during the navigation
condition,ashypothesized,butalsoactivationoftherightinsula.
These areas are included in the distributed network activated
by presence that was described in Section 1 of this paper.
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
Correlates of Presence during Virtual Reality 9
Figure 3. Main brain activation results for the ‘navigation > photographs’ contrast, centered in the left superior occipital lobe. Results are shown
in neurological coordinates. The color bar represents statistical t-values.
Moreover, the differential subjective sense of presence reported
by participants in the questionnaires between the navigation
and video conditions was shown to be inversely correlated to
the activation of the DLPFC, and directly correlated to the
activationofthelingualgyrusandcuneusandotheroccipitaland
temporal regions. Finally, we observed a linear increase in the
activation of the rightinsula and left postcentral parietalregions
according to the subjective sense of presence reported for each
condition (SUS mean of each condition minus the global SUS
mean values). Among the multiple brain areas activated by the
‘navigation > video’ contrast, we can highlight the cuneus
and the post-central parietal lobe, which have been related to
working memory and navigation tasks (Haldane et al., 2008;
Mishkin and Ungerleider, 1982). These results are comparable
to those obtained in other presence studies that have been
conducted using fMRI (Baumgartner et al., 2008) or can be
extrapolated to the results obtained with other techniques such
as TCD (Alcañiz et al., 2009) or EEG (Baumgartner et al.,
2006),always considering the limitedspatial resolutionof these
techniques. Regarding the ‘navigation > photographs’ and
‘video > photographs’ contrasts, they showed some similar
activations, such as the lingual gyrus, the cuneus, the frontal
lobe or the occipital lobe. In this section, we will discuss all
these items in more detail.
The subjects answered three SUS questionnaires (one for
each experimental condition) where they evaluated the level of
presence they felt. In each question, they value between 1 (not
feeling there at all) and 7 (highest sense of ‘being there’) the
presence experience. The results confirmed that a higher level
of presence was induced during the free navigation than during
the photograph and guided navigation conditions. Furthermore,
we can observe how the mean value for subjective sense of
presence increased for each condition, observing the lowest
score for the photographs and the highest score for navigation.
Specifically, the Friedman Test showed significant differences
between the experimental conditions for all the questions and
the SUS mean except for question 5, which evaluated how
the user remembered the experience in comparison to a real
one. The largest difference between experimental conditions
in response to the questionnaire was found in Question 1,
which asked directly about the sense of being in the virtual
world. Finally, post hoc analysis based on the Wilcoxon
Signed-Rank tests showed no significant differences in the
comparison of the photograph and the video tasks, but that
thereweresignificantdifferencesfortheothertwocomparisons:
photographs versus navigation and video versus navigation.
Therefore, as hypothesized, there were significant differences
betweenthe levelof presenceexperiencedduringthe navigation
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
10 Miriam Clemente et al.
Figure 4. Main brain activation results for the ‘video > photographs’ contrast, centered in the right inferior temporal lobe. Results are shown in
neurological coordinates. The color bar represents statistical t-values.
Table 3. Brain areas showing negative and positive correlations between activation in the ‘navigation > video’ contrast and averaged
questionnaire results.
Anatomical region Hemisphere x(mm) y(mm) z(mm) t score Cluster size Correlation
Frontal Lobe R 45 21 17 4.8747 21 Negative
Lingual/Parahippocampal Gyrus/Limbic Lobe L 19 53 0 6.1862 114 Positive
Temporal_Mid/Inferior Temporal Gyrus/Temporal Lobe L 61 14 21 5.9193 25 Positive
Cerebellum_4_5/Culmen/Cerebellum Anterior Lobe L 19 35 21 5.763 18 Positive
Cuneus/Precuneus/Parietal Lobe L 12 74 38 5.7275 23 Positive
Temporal_Sup/Superior Temporal Gyrus/Temporal Lobe L 68 28 9 5.3975 21 Positive
Sub-Gyral/Temporal Lobe L 36 11 12 5.1038 14 Positive
Calcarine/Cuneus/Occipital Lobe L 1 91 9 5.044 13 Positive
Middle Temporal Gyrus/Temporal Lobe L 29 74 17 4.6807 10 Positive
condition and that experienced during the other two conditions.
As indicated, a previous study by Welch et al. (1996) analyzed
this connection between presence and navigation and their
results are in accordance with the present study. They used
two levels of interaction, the subject as an active or a passive
driver, and observed that the interactivity increased the sense of
presence the subject experienced.
One purpose of our research was to test the hypothesis
that fMRI is an appropriate way to explore brain activation
related to presence in a VE when comparing between different
experimental conditions, allowing us to obtain objective
differences in brain activation associated with the different
levels of presence that the subjects have experienced. The main
contrastthatweanalyzedwasthe‘navigation > video’contrast,
to evaluate the differences in brain activation between two
conditionsthatinduceddifferentlevelsofpresence,asmeasured
with the SUS questionnaire. In the following paragraphs, the
results from this principal contrast will be analyzed in detail.
As explained in Section 3, one of the most significant activated
areas is the cuneus, part of the occipital lobe. This area has been
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
Correlates of Presence during Virtual Reality 11
Figure 5. Graph of results for the ‘navigation > video’ contrast, showing the negative correlation between the activation in the DLPFC (contrast
estimates difference) and the questionnaire results (navigation SUS mean - video SUS mean). The color bar represents statistical t-values.
Figure 6. Graph of results for the ‘navigation > video’ contrast, showing the positive correlation between the activation in the lingual gyrus
(contrast estimates difference) and the questionnaire results (navigation SUS mean - video SUS mean). The color bar represents statistical t -values.
relatedin previousstudies tothe visualprocessing (Peraniet al.,
2001). The cuneus is known to receive visual information from
the contralateral superior retina, and the processing that occurs
in the area is modulated by other effects, such as attention,
working memory or reward expectation (Haldane et al., 2008;
Vanni et al., 2001). Our study relates cuneus activation to the
subjectivesense of presence experiencedduring free navigation
in a VE. Another region included in the results is the calcarine
sulcus, also part of the occipital lobe where the primary visual
cortex is concentrated (Belliveau et al., 1991; Le Bihan et al.,
1993).
Another brain region that showed significant activation
during the task was the post-central parietal lobe. Between
the usual areas considered to be part of the presence network,
the parietal lobe is involved in determining spatial sense
and navigation, directly associated with the navigation in the
VE (Johnson et al., 1996; Mishkin and Ungerleider, 1982).
Moreover, Mellet et al. (2010) found that left activation of the
parietal lobe was higher while navigating through a VE than
while navigating through a real one. We also found activation
in the insula, usually related to emotion and the regulation of
the body’s homeostasis, including perception, motor control of
handandeyemovements,self-awareness,cognitivefunctioning
andinterpersonalexperience(Craig,2009;Karnathetal.,2005).
As pointed out in Section 1, the most important of these items
for our study are self-awareness, sense of agency and sense
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
12 Miriam Clemente et al.
Figure 7. BOLD signal change in response to the different
experimental conditions around MNI coordinates (46, 18, 59),
corresponding to the left parietal lobe. Observe the increase in signal
between experimental conditions.
of body ownership, because they are closely related to the
sense of presence experienced inside the VE. The sense of
body ownership allows you to discriminate your individual’s
own body and perceptions; forming the ‘body schema’ that
covers the dynamic distributed network of procedures aimed
at guiding your behavior (Haans and Ijsselsteijn, 2012). Our
results showed a parametric increase in the activation of the
right insula according to the sense of presence experience in
the conditions. Recent studies (Dodds et al., 2011) have found
evidencethatthe right insulamay beactivatedby a combination
of attentional and response control demands, playing a role in
the processing of sensory stimuli that are relevant to current
goals. During navigation in a virtual world, decisions are
constantly made based on evaluation of the sensory stimuli
guiding our behavior in the VE. Our results suggest that the
insula may play a key role in guiding behavior in the VE based
on the presented stimuli and the sense of presence. Moreover,
according to Sjölie (2012), attention and behavior are essential
to develop the sense of presence, increasing the precision in the
predictionsabout the environment and thesynchronization with
it, and avoiding prediction errors from sources outside the VE.
All these results are consistent with those obtained in
the Baumgartner et al. (2008) research. They generated
different levels of presence by means of two different types
of environment, one that induced a high arousal experience
and another that induced a low arousal experience. They
placed particular emphasis on the parietal lobe as one of
the most important areas related to presence and egocentric
spatial processing, something which was also observed in
our results. They also mentioned significant activations in the
cuneus, middle occipital gyrus and areas involved in emotional
processing, such as the insula; activationsin these brain regions
were also observed in our study and associated with the
condition which induced the higher level of presence.
Although our results can be compared with those obtained in
previous presence studies, those comparisons should be done
carefully. Each brain area is involved in several other functions
not related with presence, and the network described before is
not a closed one to the study of presence. The activation of
those areas does not necessarily imply stimulation of the sense
of presence. As Jäncke et al. (2009) explained, it is a network
involved in the control of many other psychological functions,
and ‘the psychological specificity cannot be inferred simply
by identifying the activated brain structures’. Moreover, our
primary aim was to demonstrate the validity of fMRI as a tool
toevaluatepresence;nottomapthebrainnetworkinvolvedinits
stimulation. The fMRI is a great tool to measure brain activity,
butthesize andcharacteristics ofthemachinemakesimpossible
to use it in real situations. If it can be proved that inside the
fMRI, the subject can feel presence inside a VE; this could lead
to the use of VR to approach the subject to the equivalent real
situation while being scanned. Moreover, as aforementioned,
demonstrating that the sense of presence can be stimulated
proves that the interaction between the computer-generated
environment and the subject is performed naturally, making the
technology ‘invisible’to the user. Obtaining activation in brain
areasthathavebeenpreviouslyrelatedtopresenceisremarkable
in the sense of showing that our results are not random, and
that our initial hypothesis has been accomplished. The main
objective of our fMRI research is then to bring into agreement
with previous presence theories, not to show new results on the
matter.
In a more theoretical perspective, the degree of presence in a
VEmaybeconsideredasthe degreeofsynchronizationbetween
the environment and the subject’s mental reality. In our case,
the subjects view the VE for the first time in their lives during
the scan, but due to the increasing familiarity of humans with
virtual phenomena, this should lead to the internalization of
mental simulations of theVEs, which matches with the activity
theory so popular in the HCI circles (Sjölie, 2012). So, the
fact that the subjects are not familiarized with the environments
should not prevent the sense of presence. Moreover, the central
nervous system is capable of incorporating the new tools and
technological artifacts that we use in the virtual experiences
to its representation of the body schema, integrating them
in a functional unity with our biological limbs and sensory
receptors (Haans and Ijsselsteijn, 2012), helping the interface
transparencyor‘disappearanceofmediation’(Rivaetal.,2003).
Referringto thesearchtask thesubjects hadtoperform inside
the environment, it was designed to avoid them staying still
during the experiences, but the fact of identifying an objective
to perform inside the virtual world enhances a major sense
of presence in the subjects (Riva et al., 2011). In fact, if the
performer becomes ‘emotionallyand intellectually engaged’by
the task developed, higher levels of presence can be achieved
(Waterworth et al., 2002); which leads to a state of loss
of self-consciousness (Riva et al., 2011), as we previously
discussed.
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
Correlates of Presence during Virtual Reality 13
Wewillnowdiscusstheothercontrastsevaluatedinourstudy.
As we have previously said, the results for the ‘navigation >
photographs’ contrast showed the activation of the cuneus and
the parietal lobe, two of the most important results obtained in
the‘navigation > video’contrast,whichwehavehighlighted as
havingbeenpreviouslyrelatedtopresenceduringthenavigation
task. Moreover, the activation of the cuneus may reflect an
increase inthe visual processing due tothe change in the optical
flow between both conditions. We have also found activations
in the cerebellum and the frontal lobe, results that coincide
with those obtained by Pine et al. (2002), who also evaluated
differencesbetweenfreeandguidednavigation.Thecerebellum
may have been activated because of its role in the control of
movement (Willshaw, 1999; Wolf et al., 2009). The frontal lobe
is related to the planning of the navigation task (Baumgartner
et al., 2006; Owen et al., 1990). It is also important to remark
that the cuneus, precuneus, middle occipital lobe and frontal
lobes are areas which were also activated in the research by
Baumgartner et al. (2008).
Regarding the ‘video > photographs’ results, there are
coincidenceswith thestudy byPine etal.(2002) inthe temporal
and frontal lobes. Our results agree only with those from
Baumgartner et al. (2008) in the middle frontal lobe, which is
oftenreferredtoasbeinginvolvedinvariousexecutivefunctions
as, for example, the planning of movement (Baumgartner
et al., 2006; Owen et al., 1990). The fact that there are
no other coincidences between the results may be explained
by the lower sense of presence stimulated during the two
conditions (video and photographs) compared here. Moreover,
inside the temporo-occipital cluster, it is worthy to remark the
bilateral activation of the V5/MT area, part of the extrastriate
visual cortex, which plays a main role in the perception of
movement (Born and Bradley, 2005), due to the addition of
visual movement in the video condition.
With regard to the correlation analysis comparing brain
activation and responses to questionnaires, we found a negative
correlation in the prefrontal cortex, more specifically in the
dorsolateral area, which agrees with the result obtained by
Baumgartner et al. (2008) for the measurement of presence in
video tasks, although at an inferior location within the DLPFC.
This area is related to executive processing within working
memory (Petrides, 2000) and controls the visual information
that comes from the visualization of the VE, being involved in
the decrease of the sense of presence (Koechlin et al., 2003).
Moreover, Jäncke et al. (2009) also remarked its importance in
modulatingandgeneratingtheactivityofthenetworkassociated
with the experience of presence. Regarding the positive
correlations, we obtained significant activations in the lingual
gyrus, cerebellum; middle, sub-gyral and superior temporal
lobe;calcarine and cuneus.Allof theseareasare relatedtosense
of presence, which explains why their activation gets higher
alongwiththeincreaseofthequestionnairesscores.Particularly
remarkable is the result for the lingual parahippocampal gyrus,
more specifically the activation of the parahippocampal area,
a sub-region of the parahippocampal cortex related to spatial
orientation and encoding and recognition of scenes (Aguirre
et al., 1996; Epstein and Kanwisher, 1998).
Referring to the parametric analysis, it showed a lineal trend
between the three tasks associated with an increased feeling
of presence in the insula and parietal lobe, two of the most
significant areas we emphasized for the ‘navigation > video’
contrast, and which are related to self-awareness (Craig, 2009;
Karnath et al., 2005) and navigation sense in a VE (Johnson
et al., 1996; Mishkin and Ungerleider, 1982), respectively.
The
fact that these two areas showed a positive correlation
with questionnaire scores, and in the parametric analysis, is
an indicator of their relation to sense of presence.
To finish this discussion, we will address some of the
limitations of our study. The study was conducted using a
specific group of participants, namely 14 right-handed women.
They were all right-handed to prevent noise effects of manual
lateralization on brain activation in virtual/spatial processing.
The subjects were all women to reduce variability generated
by gender differences. There are some previous studies which
show that women present a higher activation in the presence
of emotional stimuli than men. In fact, Canli et al. (2001)
indicated that they chose women because they respond more
intensely to sensitive stimuli. Theyalso maintained that women
show a greater psychological reaction according to their value
judgment than men. Some other studies concerning emotional
arousal have also concluded that women demonstrate higher
activation when shown disgusting images than when shown
pleasant ones, while men do not demonstrate any difference
(Lang et al., 1998).A great deal of previous studies concerning
visual stimuli has been conducted with women (e.g., Dilger
et al., 2003; Ochsner et al., 2002). Another limitation of our
study was the small sample size, which restricts the statistical
power to detect changes in the BOLD signal.
In our study, we added the continuous movement of the
joystick to compensate the differences between experimental
conditions in the activations caused by the motor tasks.
However, there were differences in the active planning between
the free navigation condition and the other two tasks, and these
differences could not be prevented because they are one of the
causes of the differences in the feeling of presence between
experimental conditions. We should also remark as a limitation
the low significance level we used for the statistical analysis
of the fMRI data (p<0.001 (uncorrected) may be a liberal
threshold). Maybe the use of a 3 T scanner could improve the
results obtained here.
5. CONCLUSIONS
In conclusion, the activation of the cuneus, the insula and the
parietal areas should be noted, especially the latter, due to its
relationship with thenavigational aspectsof theVR experience.
Asshownin Section 4,our final resultsare consistent withthose
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
14 Miriam Clemente et al.
from other studies concerning navigation in VR, presence in
VR studied with other brain imaging techniques and presence
during an automatic navigation in a VE studied with fMRI.
Moreover,insulaactivationinVR anditsparametric association
with thesense of presence experienced ineach of the conditions
raises questions regarding its role in the virtual experience.
However, the brain activation results may be seen just as a
proof of the utility of fMRI as a tool to evaluate presence, and
the important consequences that this could have in the field of
the HCI. Although in this study we have generated differences
in presence with changing navigation conditions, possible
future research could involve more arousing environments,
with different content, to analyze other factors that can induce
presence. Moreover, the demonstration that presence is related
to measurable differences in brain activity, even inside an
unfriendly environment as it is an MR machine, opens the door
to future studies combining VR with fMRI for psychological
treatments and psychopathological applications.
FUNDING
Thisstudywasfundedbythe MinisteriodeEducaciónyCiencia
Spain, Project Game Teen (TIN2010-20187) and partially
by projects Consolider-C (SEJ2006-14301/PSIC), ‘CIBER
of Physiopathology of Obesity and Nutrition, an initiative
of ISCIII’, the Excellence Research Program PROMETEO
(Generalitat Valenciana. Conselleria de Educación, 2008-157)
and the Consolider INGENIO program (CSD2007-00012).The
work of Miriam Clemente was supported by the Generalitat
Valenciana under a VALi+d Grant.
REFERENCES
Aguirre, G.K., Detre, J.A.,Alsop, D.C.and D’Esposito, M. (1996)The
parahippocampus subserves topographical learning in man. Cereb.
Cortex, 6, 823–829. doi:10.1093/cercor/6.6.823.
Alcañiz,M.,Rey,B.,Tembl,J.andParkhutik,V.(2009)Aneuroscience
approach to virtual reality experience using transcranial Doppler
monitoring. Presence, 18, 97–111. doi:10.1162/pres.18.2.97.
Amaro Jr, E. and Barker, G.J. (2006) Study design in fMRI:
basic principles. Brain Cogn., 60, 220–232. doi:10.1016/j.bandc.
2005.11.009.
Astur, R.S., Germain, S.A., Baker, E.K., Calhoun, V., Pearlson, G.D.
and Constable, R.T. (2005) fMRI hippocampal activity during a
virtual radial arm maze. Appl. Psychophysiol. Biofeedback, 30,
307–317. doi: 10.1007/s10484-005-6385-z.
Baños, R.M., Botella, C., García-Palacios, A., Villa, H., Perpiñá, C.
and Alcañiz, M. (2000) Presence and reality judgment in virtual
environments: a unitary construct? Cyberpsychol. Behav., 3, 327–
335. doi:10.1089/10949310050078760.
Baumann, S., Neff, C., Fetzick, S., Stangl, G., Basler, L., Vereneck,
R. and Schneider, W. (2003) A virtual reality system for
neurobehavioral and functional MRI studies. Cyberpsychol.
Behav., 6, 259–266. doi:10.1089/109493103322011542.
Baumgartner, T., Speck, D., Wettstein, D., Masnari, O., Beeli, G.
and Jäncke, L. (2008) Feeling present in arousing virtual reality
worlds: prefrontal brain regions differentially orchestrate presence
experience in adults and children. Front. Hum. Neurosci., 2, 1–12.
doi: 10.3389/neuro.09.008.2008.
Baumgartner, T., Valko, L., Esslen, M. and Jäncke, L. (2006). Neural
correlate of spatial presence in an arousing and noninteractive
virtualreality:anEEG and psychophysiology study. Cyberpsychol.
Behav., 9, 30–45. doi:10.1089/cpb.2006.9.30.
Belliveau, J.W., Kennedy, D.N., McKinstry, R.C., Buchbinder, B.R.,
Weisskoff, R.M., Cohen, M.S., Vevea, J.M., Brady, T.J. and Rosen,
B.R. (1991) Functional mapping of the human visual cortex
by magnetic resonance imaging. Science, 254, 716–719. doi:
10.1126/science.1948051.
Born, R. and Bradley, D. (2005) Structure and function of visual
area MT. Annu. Rev. Neurosci., 28, 157–89. doi:10.1146/
annurev.neuro.26.041002.131052.
Canli, T., Zhao, Z., Desmond, J.E., Kang, E., Gross, J. and Gabrieli,
J.D.E. (2001) An fMRI study of personality influences on brain
reactivity to emotional stimuli. Behav. Neurosci., 115, 33–42. doi:
10.1037/0735-7044.115.1.33.
Clemente, M., Rodríguez, A., Rey, B., Rodríguez, A., Baños, R.M.,
Botella, C., Alcañiz, M. and Ávila, C. (2011) Analyzing the level
of presence while navigating in a virtual environment during an
fMRIscan. P. Camposetal. (Eds.):INTERACT2011, PartIV,Lect.
NotesComput. Sci. 6949, 475–478. doi: 10.1007/978-3-642-23768
-3_61.
Craig, A.D. (2009) How do you feel now? The anterior
insula and human awareness. Nat. Rev. Neurosci., 10, 59–70.
doi:10.1038/nrn2555.
Dilger, S., Straube, T., Mentzel, H.J., Fitzek, C., Reichenbach,
J., Hecht, H., Krieschel, S., Gutberlet, I. and Miltner, W.H.R.
(2003) Brain activation to phobia-related pictures in spider phobic
humans: an event-related functional magnetic resonance imaging
study. Neurosci. Lett., 348, 29–32. doi: 10.1016/S0304-3940(03)
00647-5.
Dillon, C., Keogh, E., Freeman, J., and Davidoff, J. (2000) Aroused
and Immersed: The Psychophysiology of Presence. In Proceedings
of 3rd International Workshop on Presence, Delft University of
Technology, Delft, The Netherlands, 27–28 March 2000.
Dodds,C.M.,Morein-Zamir,S.andRobbind,T.W.(2011)Dissociating
inhibition, attention, and response control in the frontoparietal
network using functional magnetic resonance imaging. Cereb.
Cortex, 21, 1155–1165. doi: 10.1093/cercor/bhq187.
Epstein, R. and Kanwisher, N. (1998) A cortical representa-
tion of the local visual environment. Nature, 392, 598–601.
doi:10.1038/33402.
Flach, J.M. and Holden, J.G. (1998) The reality of experi-
ence: Gibson’s way. Presence-Teleop. Virt. Environ. 7, 90–95.
doi:10.1162/105474698565550.
Friston, K.J., Holmes, A.P., Poline, J-B., Grasby, P.J., Williams,
S.C.R., Frackowiak, R.S.J. and Turner, R. (1995) Analysis of
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
Correlates of Presence during Virtual Reality 15
fMRI time-series revisited. Neuroimage, 2, 45–53. doi:10.1006/
nimg.1995.1007.
Geake, J.G. and Hansen, P.C. (2005) Neural correlates of intelligence
as revealed by fMRI of fluid analogies. NeuroImage, 26, 555–564.
doi: 10.1016/j.neuroimage.2005.01.035.
Haans,A. and Ijsselsteijn,W.A. (2012) Embodiment and telepresence:
towardsan comprehensive theory of presence. Interact.Computers,
24, 211–218. doi:10.1016/j.intcom.2012.04.010.
Haldane, M., Cunningham, G., Androutsos, C. and Frangou, S.
(2008) Structural brain correlates of response inhibition in
bipolar disorder I. J. Psychopharmacol., 22, 138–143. doi:
10.1177/0269881107082955.
Hartley, T., Maguire, E.A., Spiers, H.J. and Burgess, N. (2003) The
well-worn route and the path less traveled: distinct neural bases of
route following and wayfinding in humans. Neuron, 37, 877–888.
doi: 10.1016/S0896-6273(03)00095-3.
Heeter, C. (1992) Being there: the subjective experience of presence.
Presence: Teleop. Virt. Environ., 1, 262–271.
International Society for Presence Research (2000) The concept of
presence: explication statement. http://www.ispr.info/ (accessed
October 28, 2011).
Jäncke, L., Cheetham, M. and Baumgartner, T. (2009) Virtual reality
and the role of the prefrontal cortex in adults and children. Front.
Neurosci., 3, 52–59. doi: 10.3389/neuro.01.006.20009.
Johnson, P.B., Ferraina, S., Bianchi, L. and Caminiti, R. (1996)
Corticalnetworksforvisualreaching:physiologicalandanatomical
organizationoffrontaland parietal lobearmregions.Cereb.Cortex,
6, 102–119. doi: 10.1093/cercor/6.2.102.
Karnath, H.O., Baier, B. and Nägele, T. (2005) Awareness of the
functioning of one’s own limbs mediated by the insular cortex?
J. Neurosci., 25, 7134–7138. doi: 10.1523/?JNEUROSCI.1590-
05.2005.
Kim, T. and Biocca, F. (2006) Telepresence via television: two
dimensions of telepresence may have different connections to
memory and persuasion. J. Comput. Mediated Commun., 3. doi:
10.1111/j.1083-6101.1997.tb00073.x.
Koechlin, E., Ody, C. and Kouneiher, F. (2003) The architecture of
cognitive control in the human prefrontal cortex. Science, 302,
1181–1185. doi: 10.1126/science.1088545.
Lang, P.J., Bradley, M.M., Fitzsimmons, J., Cuthbert, B.N., Scott,
J.D., Moulder, B. and Nangia, V. (1998) Emotional arousal and
activationofthevisualcortex:anfMRIanalysis.Psychophysiology,
35, 199–210. doi: 10.1111/1469-8986.3520199.
Le Bihan, D., Turner, R., Zeffiro, T.A., Cuénod, C.A., Jezzard, P. and
Bonnerot, V. (1993) Activation of human primary visual cortex
during visual recall: a magnetic resonance imaging study. Proc.
Natl. Sci. USA, 90, 11802–11805.
Lessiter, J., Freeman, J., Keogh, E. and Davidoff, J. (2001) A
cross-media presence questionnaire: the ITC-sense of pres-
ence inventory. Presence-Teleop. Virt. Environ., 10, 282–298.
doi:10.1162/105474601300343612.
Lombard, M. and Ditton, T. (2006)At the heart of it all: the concept of
presence. J. Comput. Mediated Commun. 3. doi: 10.1111/j.1083-
6101.1997.tb00072.x.
Loomis, J.M. (1992) Distal attribution and presence. Presence, 1,
113–118.
Meehan, M., Insko, B., Whitton, M. and Brooks, F.P. Jr. (2001)
Physiological measures of presence in virtual environments.
In Proceedings of 4th International Workshop on Presence,
Philadelphia, USA, 21–23 May 2001.
Mellet, E., Laou, L., Petit, L., Zago, L., Mazoyer, B. and Tzourio-
Mazoyer, N. (2010) Impact of the virtual reality on the neural
representations of an environment. Hum. Brain Mapp., 31,
1065–1075. doi: 10.1002/hbm.20917.
Mishkin, M. and Ungerleider, L.G. (1982) Contribution of striate
inputs to the visuospatial functions of parieto-preoccipital cortex
in monkeys. Behav. Brain Res., 6, 57–77. doi: 10.1016/0166-
4328(82)90081-X.
Mraz, R., Hong, J., Quintin, G., Staines, W.R., McIlroy, W.E.,
Zakzanis, K.K. and Graham, S.J. (2003) A platform for
combining virtual reality experiments with functional mag-
netic resonance imaging. Cyberpsychol. Behav., 6, 359–368.
doi:10.1089/109493103322278736.
Ochsner, K.N., Bunge, S.A., Gross, J.J. and Gabrieli, J.D.E.
(2002) Rethinking feelings: an fMRI study of the cognitive
regulation of emotion. J. Cogn. Neurosci., 14, 1215–1229.
doi:10.1162/089892902760807212.
Oldfield, R.C. (1971) The assessment and analysis of handed-
ness: the Edinburgh inventory. Neuropsychologia, 9, 97–113.
doi:10.1016/0028-3932(71)90067-4.
Owen, A.M., Downes, J.J., Sahakian, B.J., Polkey, C.E. and Robbins,
T.W. (1990) Planning and spatial working memory following
frontallobelesionsin man. Neuropsychologia, 28, 1021–1034. doi:
10.1016/0028-3932(90)90137-D.
Perani, D., Fazio, F., Borghese, N.A., Tettamanti, M., Ferrari, S.,
Decety, J. and Gilardi, M.C. (2001) Different brain correlates for
watching real and virtual hand actions. NeuroImage, 14, 749–758.
doi: 10.1006/nimg.2001.0872.
Petrides, M. (2000) The role of the mid-dorsolateral prefrontal
cortex in working memory. Exp. Brain Res., 133, 44–54. doi:
10.1007/s002210000399.
Pine, D.S., Grun, J., Maguire, E.A., Burgess, N., Zarahn,
E., Koda, V., Fyer, A., Szeszko, P.R. and Bilder, R.M.
(2002) Neurodevelopmental aspects of spatial navigation: a
virtual reality fMRI study. NeuroImage, 15, 396–406. doi:
10.1006/nimg.2001.0988.
Riva, G., Davide, F. and Ijsselsteijn, W.A. (2003) Being There:
Concepts, Effects and Measurement of User Presence in Synthetic
Environments. IOS Press, Amsterdam, The Netherlands.
Riva, G.,Waterworth,J.A.,Waterworth,E.L. andMantovani,F. (2011)
From intention to action: the role of presence. New Ideas Psychol.,
29, 24–37. doi:10.1016/j.newideapsych.2009.11.002.
Rey, B.,Alcañiz, M., Tembl, J. and Parkhutik,V. (2010) Brain activity
andpresence:apreliminarystudyindifferentimmersiveconditions
using transcranial Doppler monitoring. Virt. Real., 14, 55–65. doi:
10.1007/s10055–009-0141-2.
Sanchez-Vives, M.V. and Slater, M. (2005) From presence to
consciousness through virtual reality. Nat. Rev. Neurosci., 6, 332–
339. doi:10.1038/nrn1651.
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
16 Miriam Clemente et al.
Sadowski,W.andStanney, K.(2002) Presenceinvirtual environments.
In K. Stanney (ed.), Handbook of Virtual Environments:
Design, Implementation, andApplications, pp. 791–806. Lawrence
Erlbaum Associates, New Jersey.
Scheibe, C., Wartenburger, I., Wüstenberg, T., Kathmann, N.,
Villringer, A. and Heekeren, H.R. (2006) Neural correlates of
the interaction between transient and sustained processes: a
mixed blocked/event-related fMRI study. Hum. Brain Mapp., 27,
545–551. doi: 10.1002/hbm.20199.
Schuemie, M.J., Van Der Straaten, P., Krijn, M. and Van Der
Mast, C.A.P.G. (2001) Research on presence in virtual reality:
a survey. Cyberpsychol. Behav., 4, 183–201. doi:10.1089/
109493101300117884.
Sharp, H., Rogers,Y. and Preece, J. (2007)Interaction Design: Beyond
Human–Computer Interaction, 2nd ed. Wiley, NewYork.
Sheridan, T.B. (1992) Musings on telepresence and virtual presence.
Presence-Teleop. Virt. Environ., 1, 120–126.
Sjölie, D. (2012) Presence and general principles of brain function.
Interact. Comput. doi:10.1016/j.intcom.2012.04.004
Sjölie, D., Bodin, K., Elgh, E., Eriksson, J., Janlert, L.-E. and
Nyberg,L. (2010)Effectsof interactivity and 3D-motion on mental
rotation brain activity in an immersive virtual environment. In
Proceedingsofthe28thInternationalConferenceonHumanFactors
in Computing Systems, pp. 869–878. ACM, Atlanta, GA.
Slater, M. and Wilbur, M. (1997) A framework for immersive
virtual environments (FIVE)—speculations on the role of presence
in virtual environments. Presence-Teleop. Virt. Environ., 6,
603–616.
Smith, S.M. (2004) Overview of fMRI analysis. Br. J. Radiol., 77,
167–175. doi: 10.1259/bjr/33553595.
Usoh, M., Catena, E.,Arman, S. and Slater, M. (2000) Using presence
questionnaires in reality. Presence-Teleop. Virt. Environ., 9, 497–
503. doi:10.1162/105474600566989.
Vanni, S., Tanskanen, T., Seppä, M., Uutela, K. and Hari, R. (2001)
Coinciding early activation of the human primary visual cortex and
anteromedial cuneus. Proc. Natl Acad. Sci. USA, 98, 2776–2780.
doi: 10.1073/pnas.041600898.
Waterworth, J. A., Waterworth, E. L. and Westling, J. (2002)
Presence as performance: the mystique of digital participation. In
Proceedings of 5th Annual International Workshop on Presence,
Porto, Portugal, October 2002.
Welch, R., Blackmon, T., Liu,A., Mellers, B. and Stark, L. (1996) The
effects of pictorial realism, delay of visual feedback, and observer
interactivity on the subjective sense of presence. Presence-Teleop.
Virt. Environ., 5, 263–273.
Willshaw, D. (1999) The cerebellum as a neuronal machine. In
Proceedings of Self-Learning Robots III Brainstyle Robotics:
The Cerebellum Beyond Function Approximation (Ref. No.
1999/049), IEE Workshop, London, UK, 3-1 January 1999. doi:
10.1049/ic:19990258.
Witmer, B.G. and Singer, M.J. (1998) Measuring presence in virtual
environments: a presence questionnaire. Presence-Teleop. Virt.
Environ., 7, 225–240. doi:10.1162/105474698565686.
Wolf, U., Rapoport, M.J. and Schweizer, T.A. (2009) Evaluating
the affective component of the cerebellar cognitive affective
syndrome. J. Neuropsychiatry Clin. Neurosci., 21, 245–253. doi:
10.1176/appi.neuropsych.21.3.245.
Zahorik, P. and Jenison, R.L. (1998) Presence as being-in-the-
world. Presence-Teleop. Virt. Environ., 7, 78–89. doi:10.1162/
105474698565541.
Interacting with Computers, 2013
by guest on August 26, 2013http://iwc.oxfordjournals.org/Downloaded from
... Previous research has shown that the insula and insula-related self-awareness and sensory attention process regions have been linked to game immersion (Ju and Wallraven, 2019). Additionally, immersion related experiences of a sense of presence (Nacke and Lindley, 2008) in VR studies found that prefrontal regions (Baumgartner et al., 2006), and activity of left and right dorsolateral prefrontal cortex (DLPFC) were associated with upregulating and downregulating of sense of presence (Baumgartner et al., 2008;Clemente et al., 2013). ...
... Already several studies have tried to find brain regions associated with subjective gaming experiences, however, investigations of subjective gaming experiences were usually limited to one or two dimensions and resulting brain regions for same gaming experience were inconsistent across studies. For instance, previous studies have investigated brain regions associated with a single gaming experience such as flow (Baumgartner et al., 2006(Baumgartner et al., , 2008Clemente et al., 2013), and found that main brain regions for decoding flow experiences were varied across studies, including the somatosensory Ju 10.3389/fnhum.2022.1013991 and motor regions (Klasen et al., 2011), prefrontal regions (Yoshida et al., 2014;de Sampaio Barros et al., 2018) and visual processing related regions (Ju and Wallraven, 2019) being the main regions associated with flow. ...
Article
Full-text available
Studying how gaming experiences are encoded is important to understand the effects of gaming on the brain. Although studies have investigated neural correlates of gaming experiences, the brain patterns related to the full range of subjective experiences across different types of games are yet to be identified. The present study used three custom-made, immersive driving games with different input dynamics (controlling a car, a boat, or a spaceship) and different mechanics to assess subjective gaming experiences in a magnetic resonance imaging scanner. A correlational analysis identified several brain networks associated with different subjective gaming experiences, including visual and attentional processing networks. The contributions of these networks were further validated using meta-analysis-based functional term decoding. The results of the present study point to a range of perceptual, motivational, and control networks that are engaged during active gameplay.
... Articles HCI evaluation [56], [55], [93], [66], [102], [79], [3], [5], [63], [91], [19], [118], [83], [26], [44], [31], [92], [132], [49], [14], [71], [12], [25], [78], [47], [142], [133], [134], [28], explicit control [35], [72], [119], [145], [151], [101], [99], [114], [51], [52], [33], [150], [98], [106], [76], [90], [70], [69], [103], [39], [84], [65], [95], [36], implicit open loop [121], [131], [108], [58], [140], [122], [2], [107], [7], [109], [27], [73], [112], implicit closed loop [128], [126], [130], [1], [111], [152], [147], [115], [15], neurofeedback [45], [43], [54], [81], [6], mental state assessment [138], [77], [50], [23], [154], [61], [157], [139], [110], [125], [11], [88], [34], [148], [48], [158], [96], other [127], [123], [104], [85], [21], [136], [87], [41], [38], [10], [4], [24], [97], category). Systems which are able to bring all components together show that using brain signals in runtime can yield substantial usability improvements [1,152] or unlock completely novel kinds of applications [103]. ...
... For EEG processing, we use the MNE toolbox [17]. For machine learning and evaluation algorithms, we use scikit [28] and custom routines build on numpy and scipy. " (Putze et al. [112]) Brain Signal Integration Brain Input effect: This attribute describes how the output of the brain input processing influences the design, the behavior, or the content of the application or experimental paradigm. ...
Article
In human-computer interaction (HCI), there has been a push towards open science, but to date, this has not happened consistently for HCI research utilizing brain signals due to unclear guidelines to support reuse and reproduction. To understand existing practices in the field, this paper examines 110 publications, exploring domains, applications, modalities, mental states and processes, and more. This analysis reveals variance in how authors report experiments, which creates challenges to understand, reproduce, and build on that research. It then describes an overarching experiment model that provides a formal structure for reporting HCI research with brain signals, including definitions, terminology, categories, and examples for each aspect. Multiple distinct reporting styles were identified through factor analysis and tied to different types of research. The paper concludes with recommendations and discusses future challenges. This creates actionable items from the abstract model and empirical observations to make HCI research with brain signals more reproducible and reusable.
... Changes in physiological signals such as heart rate or galvanic skin response might be indicators of presence experience indices (Grassini and Laumann 2020;Meehan et al. 2001) (see also Supplementary Material A). Different neuroscientific studies using the electroencephalogram (EEG) or functional magnet resonance tomography (fMRT) found an increased parietal activation and a decreased activation of the dorsolateral prefrontal cortex (DLPFC) during higher levels of presence (Baumgartner et al. 2006Clemente et al. 2014;Jäncke et al. 2009;. It is assumed that an egocentric or body-centered representation of space provided by the parietal lobe might be essential for the feeling of being in VR (Baumgartner et al. 2006. ...
... Differences in presence experience might also affect brain activity measures. For instance, higher levels of presence are associated with an increased parietal brain activation (Baumgartner et al. 2006Clemente et al. 2014;Jäncke et al. 2009;). Hence, a general increased presence experience in the present study might have led to an overall increased parietal brain activation. ...
Article
Full-text available
The subjective presence experience in virtual reality (VR) is associated with distinct brain activation patterns. Particularly, the dorsolateral prefrontal cortex (DLPFC) seems to play a central role. We investigated the effects of electric brain stimulation (transcranial direct current, tDCS) on the presence experience as well as on brain activity and connectivity. Thirty-eight participants received either anodal ( N = 18) or cathodal ( N = 20) stimulation of the DLPFC before interacting in an immersive VR as well as sham stimulation. During VR interaction, EEG and heart rate were recorded. After VR interaction, participants rated their subjective presence experience using standardized questionnaires. Cathodal stimulation led to stronger brain connectivity than sham stimulation. Increased brain connectivity was associated with numerically lower levels of subjective presence. Anodal stimulation did not lead to changes in brain connectivity, and no differences in subjective presence ratings were found between the anodal and sham stimulation. These results indicate that cathodal tDCS over the DLPFC leads to a more synchronized brain state, which might hamper the activity in networks, which are generally associated with the evolvement of the subjective presence experience. Our results underline the importance of the DLPFC for the presence experience in VR.
... In 2012, MR was tested in several spatial navigation studies in which all subjects described immersive experiences, which were confirmed through fMRI monitoring the neuronal activation of the parahippocampal areas, cuneus, and occipital regions [30]. In 2014, Clemente Bellido et al. [31] found proof that users can feel presence similar to physical space inside a MR environment using fMRI, speculating that this could lead to interactions between computer-generated environments and humans in a natural manner as the technology becomes more 'invisible' to the user. While providing reason to assume some experiential transmissibility between a digital MR model and a constructed project, the data from such studies also seemed to conflict with the predominant architectural theory regarding the embedded nature of architectural experience within the specific materiality and location of objects and places such as that articulated by Norberg-Schultz [9]. ...
Article
Full-text available
The case study documents the design process of the physical and digital versions of the heritage-valued Nordic pavilion at the Venice Biennale. The case facilitated a multiuser collaboration in mixed reality (MR), studying the technologies’ influence upon user interactions and design decision making. Retitled as the ‘Sami Pavilion’ in tribute to the Sami artists from Norway, Sweden, and Finland, which the exhibition featured, the case study took place between 2019 and 2022, primarily during the COVID-19 pandemic. The context of the case study prompted a need to explore MR methods to overcome travel restrictions. While MR has shown some interesting utility in design research, the literature indicates the need for more concrete case work. It also was necessary to design a custom solution for multiuser collaboration. As the transferability of predictive design decisions in MR to the physical building relied upon replication between user experiences, the case embodied interesting challenges to prevalent Nordic architectural theory, particularly that of ‘genius loci’ or the ‘spirit of place’, which was a strong component of the heritage value of the building and, thus, an important design narrative. The case study documents how artworks and positions of artworks were tested in various configurations within the MR model by curators and designers to simulate the spatial experience of the design options. Several key design decisions were made based on the unique vantage points offered in MR. The MR model was then used to generate 2D technical documentation and installation instructions, which were installed on site. Studies to check the relationship between the MR model and the finished, physical result were conducted. Findings depicted a high degree of transferability between the MR model and the physical exhibition while noting discrepancies between the field of view (FOV) in the MR and physical spaces in which objects felt smaller in the real-life, built result. Possibilities and limitations for future MR implementation in the design and construction of complex projects in heritage situations are discussed in addition to implications for current architectural theory regarding place and experience from a non-dualistic perspective.
... Our findings showed a decrease in the theta band power in the VRCT in the parietal brain area. This area of the brain is said to be actively involved in creating the feeling of presence in VR applications [46,47]. The superior parietal lobe participates in visual imagery [48], and mental transformations of the body-in-space [49]. ...
Article
Full-text available
We investigated the effectiveness of virtual-reality-based cognitive training (VRCT) and exercise on the brain, cognitive, physical and activity of older adults with mild cognitive impairment (MCI). Methods: This study included 99 participants (70.8 ± 5.4) with MCI in the VRCT, exercise, and control groups. The VRCT consisted of a series of games targeting different brain functions such as executive function, memory, and attention. Twenty-four sessions of VRCT (three days/week) were performed, and each session was 100 min long. Exercise intervention consisted of aerobic and resistance trainings performed in 24 sessions for 60 min (2 times/week for 12 weeks). Global cognitive function was measured using the Mini-Mental State Examination (MMSE) test. Resting-state electroencephalography (EEG) of the neural oscillatory activity in different frequency bands was performed. Physical function was measured using handgrip strength (HGS) and gait speed. Results: After the intervention period, VRCT significantly improved the MMSE scores (p < 0.05), and the exercise group had significantly improved HGS and MMSE scores (p < 0.05) compared to baseline. One-way analysis of variance (ANOVA) of resting-state EEG showed a decreased theta/beta power ratio (TBR) (p < 0.05) in the central region of the brain in the exercise group compared to the control group. Although not statistically significant, the VRCT group also showed a decreased TBR compared to the control group. The analysis of covariance (ANCOVA) test showed a significant decrease in theta band power in the VRCT group compared to the exercise group and a decrease in delta/alpha ratio in the exercise group compared to the VRCT group. Conclusion: Our findings suggest that VRCT and exercise training enhances brain, cognitive, and physical health in older adults with MCI. Further studies with a larger population sample to identify the effect of VRCT in combination with exercise training are required to yield peak benefits for patients with MCI.
... Sin embargo, las características del EV pueden afectar la manera en que nuestro cerebro reacciona ante estos estímulos lo que puede traducirse como una correlación entre la sensación de inmersión y la actividad cerebral. Por ejemplo, la activación en la corteza prefrontal dorsolateral (memoria de trabajo) también muestra una correlación negativa con el sentido de presencia y la corteza parietal post central y la ínsula (vinculadas con la atención y la felicidad) mostraron un aumento de activación de acuerdo con la sensación de presencia relacionada con la condición, lo que sugiere que la atención del estímulo y la autoconciencia que son los procesos relacionados con la ínsula, pueden estar relacionados con la sensación de presencia (Clemente et al. 2014). En otro estudio, Mellet y colaboradores (2010) compararon las activaciones cerebrales mientras estimaban mentalmente las distancias entre puntos de referencia colocados en un entorno aprendido. ...
Article
Full-text available
En Odontopediatría, aproximadamente uno de siete individuos de la población experimenta elevados niveles de miedo y ansiedad dental, el cual influye en la salud oral y el tratamiento farmacológico, conductual o cognitivo han sido empleados para su intervención con resultados variantes (Aminabadi, et al. 2012). La literatura para el manejo del dolor en esta población reconoce que la distracción, por medio de las nuevas tecnologías, es efectiva para reducir el dolor y mejorar la experiencia en los tratamientos dentales. Con base en esto, se describe la experiencia de llevar a cabo un proyecto interdisciplinario entre la psicología y la odontopediatría en la que se aplican como técnica de distracción para el manejo del dolor el uso de la Realidad virtual (RV), Se describen los resultados en un paciente y se discute su relevancia para el trabajo entre ambas disciplinas facilitado por las nuevas tecnologías de la información.
... In order to better understand the role of each brain region shown to correlate with sense of presence, some researchers have relied on functional magnetic resonance imaging or functional near-infrared spectroscopy to get a snapshot of which brain regions become active while in VR. In [165], for example, frontal, parietal and occipital regions showed involvement during free virtual navigation and activation in the dorsolateral prefrontal cortex was shown to be negatively correlated to sense of presence, hence corroborating some of the EEG findings. In turn, brain regions responsible for balance and vestibular (located in the cerebellum) inputs were shown to be active during cybersickness events [166]. ...
Article
Full-text available
Virtual reality (VR) applications, especially those where the user is untethered to a computer, are becoming more prevalent as new hardware is developed, computational power and artificial intelligence algorithms are available, and wireless communication networks are becoming more reliable, fast, and providing higher reliability. In fact, recent projections show that by 2022 the number of VR users will double, suggesting the sector was not negatively affected by the worldwide COVID-19 pandemic. The success of any immersive communication system is heavily dependent on the user experience it delivers, thus now more than ever has it become crucial to develop reliable models of immersive media experience (IMEx). In this paper, we survey the literature for existing methods and tools to assess human influential factors (HIFs) related to IMEx. In particular, subjective, behavioural, and psycho-physiological methods are covered. We describe tools available to monitor these HIFs, including the user’s sense of presence and immersion, cybersickness, and mental/affective states, as well as their role in overall experience. Special focus is placed on psycho-physiological methods, as it was found that such in-depth evaluation was lacking from the existing literature. We conclude by touching on emerging applications involving multiple-sensorial immersive media and provide suggestions for future research directions to fill existing gaps. It is hoped that this survey will be useful for researchers interested in building new immersive (adaptive) applications that maximize user experience.
... Studies attempting to identify the neural correlates of presence remain rare, in part due to the difficulties of bringing a virtual reality environment into an fMRI scanner (Clemente et al., 2014;Jäncke et al., 2009;Sjölie et al., 2014). Among the few studies that have been conducted, the dorsolateral prefrontal cortex (DLPFC) has emerged as one of the relevant brain areas in shaping the reported degree of presence and immersiveness (Baumgartner et al., 2008;Jäncke et al., 2009). ...
Preprint
Full-text available
Extended Reality (XR), encompassing various forms of virtual reality (VR) and augmented reality (AR), has become a powerful experimental tool in consciousness research due to its capability to create holistic and immersive experiences of oneself and surrounding environments through simulation. One hallmark of a successful XR experience is when it elicits a strong sense of presence, which can be thought of as a subjective sense of reality of the self and the world. Although XR research has shed light on many factors that may influence presence (or its absence) in XR environments, there remains much to be discovered about the detailed and diverse phenomenology of presence, and the neurocognitive mechanisms that underlie it. In this chapter, we analyse the concept of presence and relate it to the way in which humans may generate and maintain a stable sense of reality during both natural perception and virtual experiences. We start by reviewing the concept of presence as developed in XR research, covering both factors that may influence presence and potential ways of measuring presence. We then discuss the phenomenological characteristics of presence in human consciousness, drawing on clinical examples where presence is disturbed. Next, we describe two experiments using XR that investigated the effects of sensorimotor contingency and affordances on a specific form of presence related to the sense of objects as really existing in the world, referred to as ‘objecthood’. We then go beyond perceptual presence to discuss the concept of 'conviction about reality', which corresponds to people's beliefs about the reality status of their perceptual experiences. We finish by exploring how the novel XR method of ‘Substitutional Reality’ can allow experimental investigation of these topics, opening new experimental directions for studying presence beyond the ‘as-if’ experience of fully simulated environments.
... Studies investigating the subjective response to cybersickness have consistently reported an increase in motion sickness scores after VR experiences compared to before VR experiences (Kennedy et al., 2000;Lim et al., 2021;Min et al., 2004;Moss et al., 2008). Studies have also examined objective responses to cybersickness by observing physiological responses using functional magnetic resonance imaging (fMRI), electroencephalography (EEG), and electrocardiography (ECG) (Clemente et al., 2013;Garcia-Agundez et al., 2019;Heo and Yoon, 2020;Naqvi et al., 2015). EEG has an excellent temporal resolution that enables brain activity changes to be observed in real time during VR experiences; thus, it has been widely used to investigate cybersickness before and after VR experiences (Chen et al., 2010(Chen et al., , 2012Heo and Yoon, 2020). ...
Article
Full-text available
Cybersickness refers to the uncomfortable side effects, such as headaches, dizziness, and nausea, felt while experiencing virtual reality (VR). This study investigated cybersickness in people with sensitivity to motion sickness using electroencephalography (EEG), the Simulator Sickness Questionnaire (SSQ), and simple VR content. Based on the scores from the Motion Sickness Susceptibility Questionnaire (MSSQ), 40 males in their twenties were selected as the sensitive group (n = 20) and non-sensitive group (n = 20). The experiment contained two conditions: a baseline condition representing a resting state and a cybersickness condition in which watching VR content induced cybersickness. The SSQ score increased significantly after watching the VR content in both groups. The sensitive group showed significantly lower absolute power in the beta and gamma bands than the non-sensitive group. The cybersickness condition showed significantly increased delta and decreased alpha compared to the baseline condition. We evaluated EEG and SSQ to identify subjective symptoms and objective physiological changes associated with cybersickness.
Chapter
Remote simultaneous interpreting (RSI) draws on Information and Communication Technologies to facilitate multilingual communication by connecting conference interpreters to in-presence, virtual or hybrid events. Early solutions for RSI involved interpreters working in interpreting booths with ISO-standardised equipment. However, in recent years, cloud-based solutions for RSI have emerged, with innovative Simultaneous Interpreting Delivery Platforms (SIDPs) at their core, enabling RSI delivery from anywhere. SIDPs recreate the interpreter’s console and work environment (Braun 2019) as a bespoke software/videoconferencing platform with interpretation-focused features. Although initial evaluations of SIDPs were conducted before the Covid-19 pandemic (e.g., DG SCIC 2019), research on RSI (booth-based and software-based) remains limited. Pre-pandemic research shows that RSI is demanding in terms of information processing and mental modelling (Braun 2007; Moser-Mercer 2005), and suggests that the limited visual input available in RSI constitutes a particular problem (Mouzourakis 2006; Seeber et al. 2019). Besides, initial explorations of the cloud-based solutions suggest that there is room for improving the interfaces of widely used SIDPs (Buján and Collard 2021; DG SCIC 2019). The experimental project presented in this paper investigates two aspects of SIDPs: the design of the interpreter interface and the integration of supporting technologies. Drawing on concepts and methods from user experience research and human-computer interaction, we explore what visual information is best suited to support the interpreting process and the interpreter-machine interaction, how this information is best presented in the interface, and how automatic speech recognition can be integrated into an RSI platform to aid/augment the interpreter’s source-text comprehension.
Article
Full-text available
Virtual environments (VEs) are the most advanced human-computer interfaces yet developed. Researchers, by the development of new methods, theories, and technologies, have endeavored to make effective VEs. The definition of effectiveness changes based on the application of the VE. For flight simulators, training transfer is important. For architectural walkthroughs, accurate perception of space is important. For treatment of phobias and post-traumatic stress disorders, presence-evoking in patients the feeling that they are near the source of their phobia or stress-is important [Hodges, 1994]. It is on this last concept, presence, that this paper focuses.
Article
Full-text available
Presence and reality judgment are two important variables to take into account in the virtual reality field. So far, scientific literature has paid (and pays) attention to the construct of presence, trying to offer assessment measures that could seize such an elusive concept. However, the concept of reality judgment has received less attention, and, frequently, it has been subsumed into the concept of presence. Not much effort has been dedicated to test whether or not both constructs refer to the same domain. Most likely there are relationships between both variables, but it is also possible that they have differentiated domains. The aim of the present work is to design a self-report measure that assesses both constructs, and to carry out the validation process with Spanish and North American samples.
Article
Full-text available
Recent research in neuroscience has tried to understand human action from two different but converging perspectives: the cognitive and the volitional. On one side, cognitive studies analyze how action is planned and controlled in response to environmental conditions. On the other side, volitional studies analyze how action is planned and controlled by a subject's needs, motives and goals. In this paper we suggest that the notion of presence may be the missing link between these two approaches, explaining how can we differentiate between perception, action and concepts.In particular, a consideration of presence can explain how can we distinguish between a perceived action, a planned or an executed one. We argue that the evolutionary role of presence is the control of agency through the unconscious separation of “internal” and “external” and the enaction/reenaction of intentions.The model makes sense in terms of evolutionary psychology and is beginning to be supported by evidence of the neural and other physical correlates of action, imitation and self-monitoring. Another strength of this model is that it provides testable predictions about how to improve the experience of presence in media: maximal presence in a mediated experience arises from an optimal combination of form and content, able to support the intentions of the user.
Article
There is a wide range of functional magnetic resonance imaging (fMRI) study designs available for the neuroscientist who wants to investigate cognition. In this manuscript we review some aspects of fMRI study design, including cognitive comparison strategies (factorial, parametric designs), and stimulus presentation possibilities (block, event-related, rapid event-related, mixed, and self-driven experiment designs) along with technical aspects, such as limitations of signal to noise ratio, spatial, and temporal resolution. We also discuss methods to deal with cases where scanning parameters become the limiting factor (parallel acquisitions, variable jittered designs, scanner acoustic noise strategies).
Article
In this work ,we focus ,on demonstrating, a real ,time communication interface which enhances text communication by detecting from real time typed text, the extracted emotions, and displaying on the screen appropriate facial expression images ,in real time. The displayed expressions are represented in terms of expressive ,images ,or sketches of ,the communicating ,persons. This interface makes ,use of a ,developed ,real time ,emotion ,extraction engine from text. The emotion extraction engine and extraction rules are discussed together with a description of the interface, its limits and future direction of such interface. The extracted emotions are mapped into displayed facial expressions. Such interface can be used ,as a ,platform ,for a number ,of future ,CMC experiments. The developed ,online communication ,interface brings together remotely located collaborating parties in a ,shared electronic space for their communication. In its current state the interface allows the participant to see at a glance,all other online participants and ,all those who ,are engaged ,in communications.,An important ,aspect of the ,interface is that for two users engaged in communication, the interface locally extracts emotional states from the content of typed ,textual sentences automatically. Subsequently it displays discrete expressions mapped,from extracted emotions to the remote screen of the other person. It also analyses/extracts the ,intensity/duration of ,the emotional
Article
What explains the experience of ''being there'' in a simulated or mediated environment? In recent years, research has pointed to various technological and psychological factors deemed important in eliciting this so-called experience of telepresence, including interactivity, sensory-motor integration, media trans-parency, and distal attribution. However, few theories exist that can combine these findings in a coherent framework. In the present paper, we formulate such a theoretical framework. We will argue that the experience of telepresence is a consequence of the way in which we are embodied, and that it extends naturally from the same ability that allows us to adjust to a slippery surface, or to the weight of a hammer. The importance of embodiment in the understanding of telepresence has been stated before, but these works have not yet fully addressed what it means to be embodied. We argue that ''having a human body'' means having a specific morphology, a body schema, and a body image. Subsequently we describe how tools and technological artifacts may be incorporated at each of these levels of embodi-ment, and the implications thereof for the experience of telepresence.
Article
Recent developments in general theories of cognition and brain function make it possible to consider the concept of presence from a new perspective, based in general principles of brain function. The importance of interaction with reality for the development and function of the brain and human cognition is increasingly emphasized. The brain is explained as implementing a generative model of the current environment. Whether this environment is real or virtual does not matter. Mental simulations are created for whatever one interacts with, when possible. This view provides a basis for relating human experiences in virtual environments to several theories that explain cognition and brain function on many levels, from ultimate evolutionary motivations to plausible neural implementations. The purpose of this paper is not to provide yet another definition of presence but to suggest explanations of phenomena commonly related to presence, with a basis in general principles of brain function. Such principles are employed to explain how, and why, interaction with our environment, and internalization of objects and tools therein, play an essential role in human cognition. This provides a rich basis for further analysis of how central aspects of presence, such as breaks in presence or the perceptual illusion of non-mediation, may work on a fundamental level. More general descriptions of such phenomena have advantages such as being easier to relate to new contexts and technologies, and opening up for additional inspiration and confirmation from other disciplines such as cognitive neuroscience. In addition to an account of general principles for brain function and a discussion about the concept of presence in light of these, this paper also relates this discussion to a number of previous accounts of presence, and to practical implications and applications for interaction design.