Between Real and Unreal: Investigating Presence and Task Performance The Design of a Pilot Study (Short paper accepted for presentation: 2nd international workshop on Presence)

Abstract

Summary This document describes on-going work investigating the relation of the notion of presence in virtual environments and task performance. Three initial experiments are being designed to identify significant factors which influence humans' memory related to a learning experience run under various conditions, as well as their perception of the environment. Although results from this study are not complete yet, the initial hypothesis relates to a conclusion made by Slater et al(8), which stated that there is no particular reason to expect presence to improve performance. The motivation of this study is therefore, to conduct experiments in an effort to find a possible link between these two concepts; this would be an outcome which will help define a set of design guidelines to specific applications. The task required subjects to attend a seminar, which was video recorded. The audio data from this seminar is going to be used again by two different sets of subjects, one of them attending a 3D seminar with audio and the other listening to the audio without any visual information involved. The goal of these initial experiments is to analyze data taken from the two mediated experiments associated with data gathered during the real-life seminar. The results could form a base for the design of a follow-up experiment involving a Shared Virtual Environment setting.

Full text

Between Real and Unreal: Investigating Presence and Task Performance
The Design of a Pilot Study
(Short paper accepted for presentation: 2nd international workshop on Presence)
Katerina Mania
Hewlett Packard Labs External Research
University of Bristol
E-mail: mania@cs.bris.ac.uk
Tel: +44-117-9545249
Fax: +44-117-9545208
Alan Chalmers
University of Bristol
E-mail: alan@cs.bris.ac.uk
Tel: +44-117-9545150
Summary
This document describes on-going work investigating the relation of the notion of presence in virtual
environments and task performance. Three initial experiments are being designed to identify significant
factors which influence humans’ memory related to a learning experience run under various conditions, as
well as their perception of the environment. Although results from this study are not complete yet, the initial
hypothesis relates to a conclusion made by Slater et al[8], which stated that there is no particular reason to
expect presence to improve performance. The motivation of this study is therefore, to conduct experiments
in an effort to find a possible link between these two concepts; this would be an outcome which will help
define a set of design guidelines to specific applications. The task required subjects to attend a seminar,
which was video recorded. The audio data from this seminar is going to be used again by two different sets
of subjects, one of them attending a 3D seminar with audio and the other listening to the audio without any
visual information involved. The goal of these initial experiments is to analyze data taken from the two
mediated experiments associated with data gathered during the real-life seminar. The results could form a
base for the design of a follow-up experiment involving a Shared Virtual Environment setting.
Introduction
With the advent of new technologies, especially networked 3D graphics and virtual reality, the concept of
presence has become an active area of research between different disciplines, from computer science and
engineering to psychology and philosophy. According to Lombard and Ditton[5], presence is defined as the
perceptual illusion of non-mediation that occurs when a user fails to perceive or acknowledge the existence
of a medium and responds as if the medium was not there. Heeter[2] distinguished three types of the
subjective experience of presence in a virtual world: personal presence, social presence and environmental
presence. In relation to task performance, Sheridan[7] asked if the sense of 'presence' is a concomitant
benign phenomenon, or even a distraction, stating that what is important in performing a task is having
enough information in the proper form.

In this preliminary pilot study we are initially trying to answer the following questions:
- Does presence, perceived as the richness of sensory information, correlate in any way with task
performance in a mediated learning environment?
- Will the subjects acquire the same amount of information from the real and virtual seminar?
- Will the subjects perceive the same level of details of the environment in the real and virtual world?
Design of the experiments
The purpose of this limited pilot study is to associate data taken from subjects attending a real seminar with
relevant data taken from different sets of subjects attending two mediated seminars. In addition, the
authors' goal is to explore this specific experimental methodology and acquire a clearer idea about its
limitations and restrictions.
The real seminar
The real seminar took place in the Department of Computer Science of the University of Bristol. There were
20 subjects attending the seminar, 8 female and 12 male, all computer science students who agreed to
participate on a voluntary basis. The duration of the seminar was 15 minutes. The topic was the Boer War
in South Africa which was chosen as it was unlikely that any of the audience would have prior knowledge
on the matter. Just before the seminar started, the participants were asked how they would rate their
knowledge of the subject on a scale from 0 to 10. Only one of the subjects reported a ‘1’. The seminar was
video recorded using a high resolution digital camera which had a steady viewpoint. The lecturer used a
standard overhead projector and 12 slides during the talk. The audience was not informed previously about
the topic of the seminar nor the contents of the questionnaire that followed.
The task questionnaire
The subjects were asked to leave the seminar room after the seminar was completed to an adjacent room,
where they were asked to complete a questionnaire. There was not any restriction of time for this task. The
questionnaire consisted of 12 questions, each of which had four possible answers. Eight of them were
asking for data recollection relevant to the theme of the seminar (the Boer War). Half of these had answers
which were included on the slides projected on the wall and half were mentioned by the lecturer but not
included on the slides. This means that half of the questions had correct answers mentioned through audio
and visual stimuli and the other half through only audio stimuli. The remaining four were designed to test
the subjects’ perception of the environment. In general, every question had one possible correct answer,
one similar to the latter and two totally irrelevant ones.

Figure 1: DrAlan Chalmers during the real seminar
VR seminar
The sound from the video of the seminar room was extracted and stored in a sound file. A model of the
seminar room was constructed in 3D Studio MAX and converted to VRML. A slide show was implemented,
incorporating the actual slides of the real talk already taken place, into the virtual room. A different set of
subjects are going to be asked to listen to the sound file while at the same time they will be viewing the
slide show projected on the wall of the 3D reconstruction of the seminar room, on a 21 inch monitor. They
will be able to navigate (walk) in the virtual room and zoom in and out from the slides. At the end, they will
fill in the same questionnaire given after the real-life seminar.
Audio seminar
Yet another set of people are going to listen to the real seminar’s sound file, without any visual information
involved. The subjects are going to be asked to fill in the part of the original questionnaire relevant to the
content of the seminar.

Figure 2: The 3D model of the seminar room
Discussion
While designing the pilot-study, no interactivity was allowed at this point, in an attempt to isolate the
different elements of a mediated experience (sound, visual stimuli, interactivity) and get comparable results
between the real and the virtual seminars.
In general, different ways of perceiving or measuring presence, relating this subjective notion with task
performance, are complicated issues to address. This relation (if it is concluded that it generally exists),
could be strongly task and technology-involved dependent. The experiments described here should be
regarded as an attempt to draw some initial conclusions towards the design of follow-up experiments; the
authors' main goal is the exploration of the basic concepts and principles towards an extended and more
complete experimental design which will associate real-life situations to relevant virtual environment
simulations.
References
[1] Barfield, W., Sheridan, T., Zeltzer, D., Slater, M. (1995). Presence and Performance within Virtual
Environments. In Barfield, W. and Furness, T. (eds), Virtual Environments and Advanced Interface Design,
Oxford University Press.
[2] Heeter, C., (1992). Being there: The Subjective Experience of Presence, Presence 1(2).
[3] Held, R., Durlach, N. (1992). Telepresence. Presence: Teleoperators and Virtual Environments, 1(1).
[4] Kim, T. and Biocca, F. (1997). Telepresence via Television: Two dimensions of telepresence may have
different connections to memory and persuasion, http://www.ascusc.org/jcmc/vol3/issue2/kim.html
[5] Lombard, M., Ditton, T. At the Heart of It All: The Concept of Presence,
http://www.ascusc.org/jcmc/vol3/issue2/lombard.html
[6] Shapiro, M., McDonald G. (1992). I'm Not a Real Doctor, but I Play One in Virtual Reality: Implications of
Virtual Reality for Judgments about Reality, Journal of Communication 42(4).
[7] Sheridan, T. (1992). Musings on Telepresence and Virtual Presence. Presence: Teleoperators and
Virtual Environments, 1(1).
[8] Slater, M., Linakis, V., Usoh M., Kooper, R., (1996). Immersion, Presence and Performance in Virtual
Environments: An Experiment with Tri-Dimensional Chess, ACM Virtual Reality Software and Technology
(VRST), Mark Green (ed.), ISBN: 0-89791-825-8, p 163-172.
[9] Steuer, J. (1995). Defining Virtual Reality: Dimensions determining Telepresence. In: Biocca, F. and
Levy, M.R.(eds), Communication in the age of Virtual Reality, Hilldale, Nj: Lawrence Erlbaum Associates.