ArticlePDF Available

Virtual memory palaces: immersion aids recall

Springer Nature
Virtual Reality
Authors:

Abstract and Figures

Virtual reality displays, such as head-mounted displays (HMD), afford us a superior spatial awareness by leveraging our vestibular and proprioceptive senses, as compared to traditional desktop displays. Since classical times, people have used memory palaces as a spatial mnemonic to help remember information by organizing it spatially and associating it with salient features in that environment. In this paper, we explore whether using virtual memory palaces in a head-mounted display with head-tracking (HMD condition) would allow a user to better recall information than when using a traditional desktop display with a mouse-based interaction (desktop condition). We found that virtual memory palaces in HMD condition provide a superior memory recall ability compared to the desktop condition. We believe this is a first step in using virtual environments for creating more memorable experiences that enhance productivity through better recall of large amounts of information organized using the idea of virtual memory palaces.
This content is subject to copyright. Terms and conditions apply.
Vol.:(0123456789)
1 3
Virtual Reality (2019) 23:1–15
https://doi.org/10.1007/s10055-018-0346-3
ORIGINAL ARTICLE
Virtual memory palaces: immersion aids recall
EricKrokos1 · CatherinePlaisant2· AmitabhVarshney3
Received: 21 October 2016 / Accepted: 3 May 2018 / Published online: 16 May 2018
© The Author(s) 2018
Abstract
Virtual reality displays, such as head-mounted displays (HMD), afford us a superior spatial awareness by leveraging our
vestibular and proprioceptive senses, as compared to traditional desktop displays. Since classical times, people have used
memory palaces as a spatial mnemonic to help remember information by organizing it spatially and associating it with sali-
ent features in that environment. In this paper, we explore whether using virtual memory palaces in a head-mounted display
with head-tracking (HMD condition) would allow a user to better recall information than when using a traditional desktop
display with a mouse-based interaction (desktop condition). We found that virtual memory palaces in HMD condition
provide a superior memory recall ability compared to the desktop condition. We believe this is a first step in using virtual
environments for creating more memorable experiences that enhance productivity through better recall of large amounts of
information organized using the idea of virtual memory palaces.
Keywords Immersion· Experimental methods· HMD· 3D navigation· Visualization· Psychology· Training· Education·
User study· Perception· Presence
1 Introduction
Throughout history, humans have relied on technology to
help us remember information. From cave paintings, clay tab-
lets, and papyrus to modern paper, audio, and video, we have
used technology to encode and recall information. This paper
addresses the question of whether virtual environments could
be the next step in our quest for better tools to help us memo-
rize and recall information. Virtual reality displays, in contrast
to traditional displays, can combine visually immersive spatial
representations of data with our vestibular and proprioceptive
senses. The technique of memory palaces provides a natural
spatial mnemonic to assist in recall. Since classical times,
people have used memory palaces (method of loci), by taking
advantage of the brain’s ability to spatially organize thoughts
and concepts (Julian 1976; Roediger 1979; Knauff 2013). In a
memory palace, one mentally navigates an imagined structure
to recall information (Yates 1992; Harman 2001). Even the
Roman orator Cicero is believed to have used the memory pal-
ace technique by visualizing his speeches and poems as spatial
locations within the auditorium he was in (Yates 1992; Godwin-
Jones 2010). Spatial intelligence has been associated with a
heightened sense of situational awareness and of relationships
in one’s own surroundings (Mayer etal. 2001; Gardner 2006).
Research in cognitive psychology has shown that recall is
superior in the same environment in which the learning took
place (Godden and Baddeley 1975). Such findings of context-
dependent memory have interesting implications for virtual
environments that have not yet been fully explored. Imagine, for
instance, a victim of a street aggression being asked to recall the
appearance details of their assailant. Virtual environments that
mirror the scene of the crime could provide superior assistance
in recall by placing the victim back into such an environment.
In this paper, we present the results of a user study that
examined if virtual memory palaces could assist in superior
The original version of this article was revised due to a
retrospective Open Access order.
* Eric Krokos
ekrokos@umiacs.umd.edu
Catherine Plaisant
plaisant@cs.umd.edu
Amitabh Varshney
varshney@umiacs.umd.edu
1 University ofMaryland, College Park, AV. Williams 4406,
CollegePark, USA
2 University ofMaryland, College Park, Hornbake
Bldg. 2117C, CollegePark, USA
3 University ofMaryland, College Park, AV. Williams 2119,
CollegePark, USA
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
2 Virtual Reality (2019) 23:1–15
1 3
recall of faces and their spatial locations aided by the con-
text-dependent immersion afforded by a head-tracked head-
mounted display (HMD condition) as compared to using a
traditional desktop display with a mouse-based interaction
(desktop condition). To explore this question, we designed
an experiment where participants were asked to recall spe-
cific information in the two environments: the HMD condi-
tion and the desktop condition. We created the virtual mem-
ory palaces prior to the start of the study. Our hypotheses
are as follows:
Hypothesis 1: The participant memory recall accuracy
will be higher in the HMD condition as compared to the
desktop condition due to the increased immersion.
Hypothesis 2: Participants will have higher confidence in
their answers in the HMD condition as compared to the
desktop condition.
The experiment was a within-subject,
2×2×2
Latin-square
design, ensuring all the different combinations of variables
and factors were accounted for. The experimental results of
our study support both hypotheses.
2 Related work
Memory palaces have been used since the classical times
to aid recall by using spatial mappings and environmental
attributes. Figure1 shows a depiction of a memory palace
attributed to Giulio Camillo in 1511. The idea was to map
words or phrases onto a mental model of an environment (in
this case an amphitheater) and then recall those phrases by
mentally visualizing that part of the environment.
An important component of the memory palace technique
is the subjective experience of being virtually present in the
palace, even when one is physically elsewhere. This notion
of presence has long been considered central to virtual
environments, for evaluation of their effectiveness as well
as their qualitySkarbez etal. (2017). More precisely, Slater
(2009) developed the idea of place illusion (PI), referring
to the aspects of presence “constrained by the sensorimotor
contingencies afforded by the virtual reality system.” Sen-
sorimotor contingencies are those actions which are used in
the process of perceiving the virtual world, such as moving
the head and eyes to change gaze direction or seeing around
occluding objects to gain an understanding of the space
(O’Regan and Noë 2001). Slater (2009) therefore concluded
that establishing presence or “being there” for lower-order
immersive systems such as desktops is not feasible. In con-
trast, the sensorimotor contingencies of walking and look-
ing around facilitated by head-mounted displays contribute
to their higher-order immersion and establishing presence.
Recent research in cognitive psychology(Repetto etal.
2016) suggests that the mind is inherently embodied. The
way we create and recall mental constructs is influenced
by the way we perceive and move(Barsalou 2008; Shapiro
2010). The memory system that encodes, stores, recognizes,
embodies, and recalls spatial information about the environ-
ment is called spatial memory (Madl etal. 2015). Several
studies have found that embodied navigation and memory
are closely connected (Leutgeb etal. 2005; Buzsáki and
Moser 2013). Madl etal. (2015) state that there are several
different types of brain mechanisms involved in process-
ing spatial representations in the brain. Grid cells in the
entorhinal cortex, used for path integration, are activated
by changes in movement direction and speed (Moser etal.
2008; Burgess 2008). Head-direction cells activate in the
medial parietal cortex when the head points in a given direc-
tion, providing information on viewing direction (Baumann
and Mattingley 2010). Border cells and boundary vector
cells in the subiculum and entorhinal cortex activate in close
proximity to environment boundaries, depending on head
direction (Burgess 2008; Lever etal. 2009). Lastly, place
cells in the hippocampus activate in specific spatial loca-
tions, independent of orientation, providing an internal rep-
resentation of the environment (Ekstrom etal. 2003; Hartley
etal. 2014). It is believed that place cell fields arise from
groups of grid and boundary cells which activate for differ-
ent spatial scales and environmental geometry to provide
a sense of location (Barry etal. 2006; Kim etal. 2011). In
addition, these hippocampal cells also provide information
about place–object associations, associating place cell rep-
resentations of specific locations with the representations of
specific objects in recognition memory (Brown and Aggle-
ton 2001; Hok etal. 2005). This leads us to the possibil-
ity that a spatial virtual memory palace, experienced in an
Fig. 1 Giulio Camillo’s depiction of a memory palace (1511 AD).
Memory palaces like this have been used since the classical times as
a spatial mnemonic
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
3Virtual Reality (2019) 23:1–15
1 3
immersive virtual environment, could enhance learning and
recall by leveraging the integration of vestibular and proprio-
ceptive inputs (overall sense of body position, movement,
and acceleration) (Hartley etal. 2014).
2.1 Memory palaces onadesktop monitor
Legge etal. (2012) compared the use of the traditional
method of loci using a mental environment against a 3D
graphics desktop environment. In this study, the subjects
were divided into three groups. The first group was instructed
to use a mental location or scene, the second group was a
3D graphics scene, and the third (control) group was not
informed on the use of any mnemonic device. The subjects
in the three groups were given 10–11 uncorrelated words
and asked to memorize the words with their mnemonic
device, if any. The users then recalled the words serially.
This study found that the users who used a graphics desktop
environment as the basis for their method of loci performed
better than those using a mental scene of their choice, and
those who were not instructed on a memory strategy did
not perform as well as those who were instructed to use the
memory strategy. Fassbender and Heiden (2006) compared
the ability of users to recall a list of 10 words when using a
desktop compared to memorizing the word list. The authors
created a navigable 3D castle with 4 sections and 10 objects,
where each object has a visual and audible component, with
the idea that a user will associate a word with that object.
First, each user was given 10 words to memorize and then
were asked to recall as many as they could after a 2-min
distraction task. Next, each user was explained and shown
the 3D castle on a desktop. After being given time to learn
the associations between the words, images, and audio, the
users were evaluated on their ability to recall the words in
the 3D castle on the desktop. The study found that there
was no significant difference between the users’ ability to
immediately recall the words after a 2-min break, but after
one week there was a 25% difference in recall in favor of the
3D graphics desktop memory palace environment condition.
The above studies show that compared to a purely mental
mnemonic, a graphics desktop setup is better in assisting
retention and recall.
Both of these studies have been carried out on desktops
and not in immersive HMDs. In our study, we compare the
performance of users on a desktop compared with an immer-
sive HMD.
2.2 Memory palaces onmultiple displays
The efficacy of varying immersion levels by changing the
field of view has also been studied in the context of pro-
cedural training(Bowman and McMahan 2007). Sownda-
rarajan etal. (2008) compared subject performance for a
simple and complex procedural task (involving a different
number of steps and interactions), but with two different
fields of view—one with a laptop and the other with a large
rear-projected L-shaped display. The study had participants
trained on two procedures, and the performance with the two
levels of immersion was compared. The study found that
higher levels of immersion (in this case, field of view) were
more effective in learning complex procedures that reference
spatial locations. In addition, there was no statistical dif-
ference in performance for the simple task for the different
levels of immersion. Ragan etal. (2010) carried out a user
study in which participants were asked to memorize and
recall the sequence of placement of virtual objects on a grid
shown on three rear-projected screens (one front and two
side screens). The participants were divided into multiple
groups that performed the task with different fields of view
and fields of regard. The field of view is the size of the visual
field seen in one instant, while the field of regard is the total
size of the visual field that can be seen by a user(Bowman
and McMahan 2007). Both are measured in degrees of visual
angle. Ragan etal. found that higher field of view and field
of regard produced a statistically significant performance
improvement.
The above studies examined the effectiveness of memory
recall of objects, their locations, and the sequence of place-
ment actions, in a limited field of view and field of regard
in monoscopic display environments with multiple moni-
tors. The field of regard in these studies did not surround
the viewer completely. In our study, we wanted to examine
the effectiveness of stereoscopic, spherical field of regard
afforded by modern HMDs compared to a desktop for mem-
ory recall of objects and their spatial locations.
2.3 Search andrecall inhead‑mounted displays
Pausch etal. (1997) studied whether immersion in a virtual
environment using a HMD aids in searching and detection of
information. For their study, they created a virtual room with
letters distributed on walls, ceiling, and floor. A user was
placed in the center of this room and was asked whether a set
of letters was present or not. The test was conducted using a
HMD and a traditional display with a mouse and keyboard.
They found that when the search target was present, the
HMD and the traditional display had no statistically signifi-
cant difference in performance. However, when the target
was absent, the users were able to confirm its absence faster
in the HMD than on the traditional display. In addition, the
users that used the HMD first and then moved to a traditional
desktop had better performance than those who used the
desktop first and then the HMD. This suggests a positive
transfer effect from the HMD to a desktop. Our user study is
highly influenced by the study of Pausch etal. (1997), but in
our study, users perform recall rather than search.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
4 Virtual Reality (2019) 23:1–15
1 3
Ruddle etal. (1999) compared user navigation time and
relative straight-line distance accuracy (amount of wasteful
navigational movement) between a HMD and a traditional
desktop. Users were then asked to learn the layout of two
virtual buildings: one using a HMD, and the other using a
desktop. After familiarizing themselves with the buildings,
each user was placed in the lobby of that building and were
told to go to each of five named rooms and then return to
the lobby. They found that the users wearing the HMD had
faster navigation times and less waste-full movement and
were more accurately able to estimate distances, compared
to those using a desktop.
Mania etal. (2003) examined accuracy and confidence
levels associated with recall and cognitive awareness in
a room filled with objects such as pyramids, spheres, and
cubes. Participants were exposed to one of the following
scenarios: (a) a virtual room using a HMD, (b) a rendered
room on a desktop, or (c) a real room experienced through
glasses designed to restrict the field of view to
30
to match
that of the HMD and desktop. All the four walls of the room
were distinct. After 3min of exposure, the participants were
given a paper containing a representation of the room which
included numbered positions of objects in the various loca-
tions. The participants were asked to recall which objects
were present and where they were located in the room, and
to give a confidence and awareness state with each answer.
The study evaluated the participants immediately after the
exposure and then again after one week. The study found
that immediately after the exposure the participants had the
most accurate recall in the real-world scene and were slightly
less accurate and confident in the HMD and least accurate
and confident on the desktop. After one week, the overall
scores and confidence levels dropped consistently across
the board, with the viewing condition having no effect on
the relative reduction in performance. In this inspirational
study, the participants only experienced one display. In our
study, the participants were exposed to both the desktop and
the HMD. This makes it possible to compare recall for the
same user across the two display modalities. Further, to use
the context provided by immersion, the participants in our
study were asked to recall the information while viewing the
same virtual scenes on the same display, rather than record-
ing their answers on a representation of the scene on paper.
Harman etal. (2017) explored immersive virtual environ-
ments for memory recall by having participants take on the
role of a boarding an airplane in a virtual airport. After the
experience, the participants were asked about the tasks they
performed. The participants who experienced the virtual
airport in a HMD had more accurate recall than those who
used the desktop. In this study, each participant used either
a HMD or a desktop. Also, the evaluation of the memory
recall was done outside of the visual experience, through
a questionnaire. In our study, not only do participants
experience the virtual environment in both, HMD and desk-
top, but are also asked to recall in the same environment in
which they experienced the information.
2.4 Embodied interaction andrecall
Virtual walk-throughs have been one of the earliest appli-
cations of virtual worlds(Brooks Jr etal. 1992). Brooks
(1999) studied whether active participants had superior
recall of the layout of a 3D virtual house on a desktop com-
pared to passive participants. Active participants controlled
camera navigation via a joystick, while passive participants
observed the navigation. They found that active participants
had a superior environment layout recall compared to those
who were passive. However, they also found that there was
no statistically significant difference between the recall or
recognition of objects (such as furniture or entrances and
exits of a room) or their positions within the environment
between the active and passive participants. This suggests
that memory was only enhanced for those aspects of the
environment that were interacted with directly—particularly
the environment which was navigated.
Richardson etal. (1999) had users learn the layout of a
complex building through either 2D maps, physically walk-
ing through the real building, or through a 3D virtual repre-
sentation of that building built using the Doom II engine and
shown on a desktop. The study found that when the building
was a single floor, the real-world and virtual-environment-
trained users had comparable results. However, when the
building had two floors, relative view orientation during
learning and testing mattered. If the participants were in the
same orientation that they had used during learning, they
were able to navigate the environment just as well as those
who were physically in the environment. However, partici-
pants were susceptible to disorientation if their starting-out
views were different between their training and testing. The
authors concluded that training in the virtual and real-world
environments likely used similar cognitive mechanisms.
Wraga etal. (2004) compared the effectiveness of ves-
tibular and proprioceptive rotations in assisting recall by
having participants recall on which of the four walls was a
object located relative to their orientation before and after
rotation. Participants were placed in a virtual room with
four distinctly colored alcoves on four walls and given time
to learn and recognize the alcoves. Participants would then
rotate, either using the HMD accelerometer or a joystick, to
find a certain object on one of the alcoves as described by
the tester. Once the user was looking at that object on one
of the alcoves, their view would be frozen and the tester
would ask the participant to state where a particular (differ-
ent) alcove was relative to their orientation. They found that
users in a HMD were better able to keep track of the objects
by rotating their heads as compared to using a joystick. In
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
5Virtual Reality (2019) 23:1–15
1 3
another experiment, the authors also found that users in a
HMD who controlled their bearing in a virtual world by
actively rotating in a swivel chair were better able to keep
track of an object than those that were being rotated by a
tester. In our study, we expect vestibular and propriocep-
tive inputs to improve performance in the HMD. We study
how well people can recall information regardless of their
orientation. In addition, our objects are distributed in more
than four unique locations.
Perrault etal. (2015) leveraged the method of loci tech-
nique by allowing participants to link gestural commands,
which would control some system, to physical objects within
a real room. They compared their interaction technique to a
mid-air swipe menu which relies on directional swiping ges-
tures. Their idea was to leverage spatial, object, and semantic
memory to help users learn and recall a large number of ges-
tures and commands. In a home environment, participants
were shown a command (or stimulus) on a television and
then performed a motion that a Microsoft Kinect would track
and record as representing that command. For the mid-air
swipe, the participant would perform a 2-segment marking
menu gesture. For the physical loci, the participant would
simply point at an object in the environment that they wanted
associated with the command, such as a chair or poster. Once
the gestures and physical loci were trained, the participants
went into the recall phase. In this phase, a command would
be presented on the screen and the user had to quickly and
accurately perform the corresponding gesture. The system
would then show whether the participant performed the cor-
rect gesture or pointed at the correct loci object that they
originally assigned for that command. The authors found
that users, when using their physical loci technique, had
superior command recall and were more robust compared
to the more traditional mid-air swipe menu.
3 Method
A memory palace is a spatial mnemonic technique where
information is associated with different aspects of the imag-
ined environment, such as people, objects, or rooms, to assist
in their recall(Yates 1992; Harman 2001). The goal of our
user study was to examine whether a virtual memory pal-
ace, experienced immersively in a head-tracked stereoscopic
HMD, can assist in recall better than a mouse-based inter-
action on a traditional, non-immersive, monoscopic desk-
top display. Previous work has examined the role of spatial
organization, immersion, and interaction in assisting recall.
This study is different from the previous work in several
ways. First, we are focusing on spatial memory using a 3D
model of a virtual memory palace, rather than relying on
other forms of memory (such as temporal/episodic). Sec-
ond, both the training and testing (recall) phases take place
within the same virtual memory palace. Third, participants
used both the desktop and HMD displays, which allows us
to compare each participant’s recall across displays. Lastly,
the content used in previous studies was either abstract,
verbal, textual, visually simplistic, low in diversity, or time
based, whereas our study uses faces, with unique and diverse
characteristics.
3.1 Participants
Our user study for this research was carried out under IRB
ID 751321-1 approved on August 7, 2015, by the Univer-
sity of Maryland College Park IRB board. In this study, we
recruited 40 participants, 30 male and 10 female, from our
campus and surrounding community. Each participant had
normal or corrected-to-normal vision (self reported). The
study session for each participant lasted around 45min.
3.2 Materials
For this study, we used a traditional desktop with a 30 inch
(76.2)cm—diagonal monitor and an Oculus DK2 HMD.
The rendering for the desktop was configured to match that
of the Oculus with a resolution of
1920 ×1080
pixels (across
the two eyes) with a rendering field of view (FOV) of
100
.
In order to give the desktop display the same field of view as
the HMD, the participants were positioned with their heads
10 inches (25.4cm) away from the monitor. The software
used to render the 3D environments on both the desktop
and HMD was identical and was designed in-house using
C++ and OpenGL-accelerated rendering. The rendering
was designed to replicate a realistic looking environment as
closely as possible, incorporating realistic lighting, shadows,
and textures. The models (the medieval town and palace)
were purchased through the 3D modeling distribution Web
site TurboSquid (3DMarko 2011, 2014).
3.3 Design
The participants were shown two scenes, on two display con-
ditions (head-tracked HMD and a mouse-based interaction
desktop), and two sets of faces (within-subject design), all
treated as independent variables, with the measured accu-
racy of recall as the dependent variable. The two scenes
(virtual memory palaces) consist of pre-constructed pal-
ace and medieval town environments filled with faces. We
decided to use faces given the previous work (Harris 1980;
McCabe 2015) showing the effectiveness of memory palaces
aiding users in recalling face-name pairs. We used faces as
the objects to be memorized and carefully partitioned them
into two sets of roughly equal familiarity. We quantified the
familiarity of the faces using Google trends data over the
four months preceding the study. The faces are shown in
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
6 Virtual Reality (2019) 23:1–15
1 3
Appendix” (at the very end of the paper) in Figs.11 and12,
and the Google trends statistics are presented in Tables1
and2. There was no statistically significant difference
between the two sets of Google trends data:
p=0.45
>
0.05
.
The faces in the palace and medieval town were hand
positioned for each environment, before the start of the
study, and remained consistent throughout the study. We
distributed the faces at varying distances from the users’
location (see Fig.2) so that they surrounded and faced the
user. Since we used perspective projection, the sizes of the
faces varied. However, the distribution of the angular resolu-
tion of the faces across the two sets/environments was not
statistically different, with
p=0.44
>
0.05
(see Table3 in
Appendix”).
Users were allowed to freely rotate their view but not
translate. This effectively simulated a stereoscopic spherical
panoramic image with the participant at its center. Our moti-
vation behind this study design decision was that if even this
limited level of immersion could show an improvement in
recall, it could lead to a better-informed exploration of how
greater levels of immersion relate to varying levels of recall.
3.4 Procedure
First, each participant familiarized themselves with all the
42 faces and their names used in the study. The participants
received a randomly permuted collection of printouts, each
containing a face-name pair used in the study. Participants
were given as much time as needed until they stated when
they were comfortable with the faces. In general, partici-
pants did not spend more than 5min on this familiarization.
Next, each participant was told about the training and
testing procedure, including how many faces were going to
be in each scene (21), how much time they had to view the
faces (5min), how the breaks would work, that the faces
would be replaced with numbers in the recall phase, and that
they were to give a name and confidence for their recalled
faces for each numbered position. In almost every case, we
recorded the answer as the name explicitly recalled by the
participant. However, in rare, exceptional circumstances,
when the participants gave an extremely detailed and unam-
biguous description of the face (“fat, wore a wig, was King
of France, and is not Napoleon” for King Louis), we marked
it correct. Next, each participant was placed either in front
of a desktop monitor with a mouse or inside a head-tracked
stereoscopic HMD. They were given as much time as they
desired to get comfortable, looking around the scene without
numbers or faces. The users rotated the scene on a desktop
monitor with a mouse, and in the HMD setup they rotated
their head and body, but no further navigation was possible.
Once each participant was comfortable with the setup and
the controls, a set of 21 faces were added to the 3D scene
and distributed around the entire space as shown in Fig.2.
We used two such scenes—a palace and a medieval town,
shown in Fig.3. The faces were divided into two consistent
sets used for the whole study; if a face appeared in one set
(or scene) for a given participant, it would not be shown
again in the second set or scene.
To cover all possible treatments of the
2×2×2
Latin-
square design, each participant was tested in both scenes,
both display conditions (HMD and desktop), and both sets
of faces, with their relative ordering counterbalanced across
participants. The 21 faces within the scene were presented to
the participants all at once, and the participants were able to
Fig. 2 Locations of faces and numbers in the virtual memory palaces
used in our user study a an ornate palace and b a medieval town. Note
that this is not the view the participants had during the experiment,
and these pictures are used to convey the distribution of the face loca-
tions. The participants would have been placed in the middle of these
scenes surrounded by the faces as shown in Fig.3
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
7Virtual Reality (2019) 23:1–15
1 3
view and memorize the faces in any order of their choosing.
The faces were deterministically placed in the same order
for all participants. However, since the participants were free
to look in any direction, the order of presentation of faces
was self-determined. Each participant was given 5min to
memorize the faces and their locations within the scene.
After the 5-min period, the display went blank and each par-
ticipant was given a 2-min break in which they were asked a
series of questions. Questions we asked included how each
participant learned about the study, what their profession/
major was, and what were their general hobbies or interests.
In the second half of the study, during the break for the
alternative display, we asked how often a participant used
a computer, what their previous experience was with VR,
and their general impressions of VR. We consistently asked
these questions of each participant, but did not record the
responses.
The reasons for these study design decisions are rooted in
foundational research in psychology on memory. From the
seminal work byMiller (1956), we learn that the working
memory(Baddeley and Hitch 1974) can only retain
items. According to Atkinson and Shiffrin (1968), the infor-
mation in the short-term memory decays and is lost within a
period of 15–30s. We feel confident that having participants
recall 21 faces after a 2-min break will engage their long-
term memory.
After the 2-min break, the scene would reappear on the
display with numbers having replaced the faces, as shown
in Fig.4. Each participant was then asked to recall, in any
order, which face had been at each numbered location. Dur-
ing this recall phase, each participant could look around and
explore the scene just as they did in the training phase, using
the mouse on the desktop or rotating their head-tracked
HMD. Each participant had up to 5min to recall the names
of all the faces in the scene. Once the participant was con-
fident in all their answers, or the 5-min period had passed,
the testing phase ended. After a break, each participant was
placed in the other display that they had not previously tested
with. The process was then repeated with a different scene
and a different set of 21 faces to avoid information overlap
from the previous test.
For each numbered location in the scene, the participants
verbally recalled the name of the face at that location, as
well as a confidence rating for their answer, ranging from 1
to 10, with 10 being certain. If a participant had no answer
for a location, it was given a score of 0. The results were
Fig. 3 The two virtual memory palace scenes used in our user study a
an ornate palace and b a medieval town, as seen from the view of the
participants
Fig. 4 Virtual memory palace: recall phase
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
8 Virtual Reality (2019) 23:1–15
1 3
hand-recorded by the study administrator, keeping track of
the number, name, user confidence, and any changes in a
previously given answer.
To mitigate any learning behavior from the first trial to
the second, we employed a within-subject trial structure,
using a 2 (HMD-condition to desktop-condition vs desktop-
condition to HMD-condition) × 2 (Scene 1 vs Scene 2 ) × 2
(Face Set 1 vs Face Set 2) Latin-square design. By alternat-
ing between the displays shown first (2), the scenes (2), and
the faces (2), we expect to mitigate any confounding effects.
At the end, each participant was tested on the two display
conditions, desktop and HMD, on two different scenes, and
with two different sets of 21 faces. We note that participants
could have used personal mnemonics to help remember the
locations and ordering of faces. However, since we evaluated
recall for each participant over a desktop and a HMD, their
performance should be counterbalanced between the two
display conditions.
4 Results
Our hypothesis is that a virtual memory palace experienced
in an immersive head-tracked HMD (the HMD condition)
will lead to a more accurate recall than on a mouse-con-
trolled desktop display (the desktop condition). In addition,
we hypothesized that participants should be more confident
in their answers in the headset and make fewer mistakes
or errors in recall. Our null hypothesis is that there is no
statistical difference between the accuracy and confidence
of results between the HMD and desktop conditions and
that there is no statistical difference in the ordering of the
display conditions.
We confirmed using a four-way mixed ANOVA that there
were no statistically significant effects on recall due to the
scenes (palace and town)
F(1, 79)=0.27, p
>
0.05
, the two
sets of 21 faces
F(1, 79)=0.27, p
>
0.05
, or the ordering
of display conditions (HMD followed by desktop vs desk-
top followed by HMD)
F(1, 79)=1.93, p
>
0.05
. We found
that there was a statistically significant effect for the dis-
play condition (HMD vs desktop) with
F(1, 79)=4.6
and
p
<
0.05
. This means participants were able to recall better
in the HMD condition as compared to the desktop condition,
permitting us to reject the null hypothesis.
4.1 Task performance
The overall average recall performance of participants in
the HMD condition was 8.8% higher compared to the desk-
top condition with the mean recall accuracy percentage for
HMD condition at 84.05% and the desktop condition at
75.24%. Using a paired t test with Bonferroni–Holm cor-
rection, we calculated
p=0.0017
<
0.05
which shows that
our result was statistically significant. In Fig.5, we present
the overall performance of the users in the HMD condition
as compared to the desktop condition.
4.2 Errors andskips
The recall accuracy measures the number of correct
answers. In addition, we kept track of when participants in
our user studies made an error in recall (i.e., gave an incor-
rect answer) or skipped answering (i.e., did not provide an
answer). We show the percentile distribution of the aver-
age number of erroneous answers per participant for each
display modality in Fig.6. Participants in the HMD condi-
tion made on average fewer errors than those in the desktop
condition. The total number of errors in the HMD condition
for 40 people was 33 out of 840, and in the desktop condi-
tion it was 56 out of 840. In addition, the difference in the
incorrect answers was statistically significant, shown using
a paired t test with Bonferroni–Holm correction resulting in
p=0.0195
<
0.05
.
Fig. 5 The overall average recall performance of participants in the
HMD condition was 8.8% higher compared to the desktop condition.
The median recall accuracy percentage for HMD was 90.48% and
for desktop display was 78.57%. The figure shows the first and third
quartiles for each display modality
Fig. 6 The distribution of incorrect answers for each display modality
showing the median, first, and third quartiles
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
9Virtual Reality (2019) 23:1–15
1 3
In Fig.7, we showed that the number of faces for which
participants skipped an answer in the desktop condition was
significantly higher than in the HMD condition. This was
shown to be statistically significant using a paired t test with
Bonferroni–Holm correction with
p=0.0062
<
0.05
, which
reinforces that participants in the HMD had better recall than
those on the desktop.
4.3 Condence
Previous work by (Mania and Chalmers 2001; Mania etal.
2003) examined user confidence with recall accuracy. This
allows us to study not only the objective recall accuracy
but also the subjective certainty of the user answers. We
asked each participant to indicate their confidence on a scale
of 1–10, with 10 being certain, as a measure of how cer-
tain they were in the correctness of their response, for each
answer. The confidence scores aggregated across all the 40
participants and all the 42 faces that each studied are shown
in Fig.8.
From Fig.8, we can see that users were slightly more
confident in the HMD condition than on the desktop con-
dition. The average confidence values for the HMD and
desktop conditions were 9.4 and 9.1 respectively, ignoring
skips. For the highest confidence, a confidence score equal
to 10, there was a statistical difference between the num-
ber of correct answers given in the HMD and the desktop
conditions, with
p=0.009
<
0.05
using a Chi-square test,
and with
p=0.022
<
0.05
including Yates community cor-
rection. However, confidence is not always an indication of
correctness. We wanted to see whether the HMD condition
was giving a false sense of confidence. Figure9 shows the
number of errors given in each display based on the confi-
dence of participant answers.
The results in Fig.9 show that when the users were less
error-prone in the HMD condition, their confidence was bet-
ter-grounded in the recall accuracy than when in the desktop
condition. In general, participants were more often correct
in the HMD condition than for the desktop condition for a
given confidence level.
4.4 Ordering eect
In our study, we alternated the order in which participants
were exposed to the displays. Figure10 shows the accuracy
when using the desktop first followed by the HMD versus
using the HMD first and then the desktop.
For both the desktop and HMD conditions, users started
with roughly the same performance (accuracy) on both the
desktop and HMD (desktop-1 and HMD-1 in Fig.10), but
when going to the other display, the performance changed.
When users went from a desktop to a HMD, their perfor-
mance generally improved. However, when the users went
from a HMD to a desktop, their performance surprisingly
decreased. When comparing each participant’s first trials,
the desktop-1 and HMD-1, their distribution of recall scores
was not significantly different with
p=0.62
>
0.05
, but they
Fig. 7 The distribution of faces skipped during recall for each display
modality showing the median, first, and third quartiles
Fig. 8 The overall confidence scores of participants in the HMD con-
dition and the desktop condition. Each participant gave a confidence
score between 1 and 10 for each face they recalled. Those in the
HMD condition are slightly more confident about their answers than
those in the desktop condition
Fig. 9 The number of errors made for each display condition for vari-
ous confidence levels
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
10 Virtual Reality (2019) 23:1–15
1 3
were for the second trials, the HMD-2 and desktop-2, with
p=0.025
<
0.05
.
5 Discussion
We next report some interesting observations based on a
questionnaire the participants filled out after the study. All
our participants were expert desktop users, but almost none
had experienced a HMD before. We believe that if there
were to be any implicit advantage, it would lie with the desk-
top, given the overall familiarity with it. Although we gave
the participants enough time to get comfortable in the HMD
before we began the study, we observed that many were not
fully accustomed to the HMD, even though they performed
better in it. We asked each participant which display they
preferred for the given task of recall. We explicitly stated
that their decision should not be based on the novelty or
“coolness” of the display or the experience. All but two of
the 40 participants stated they preferred the HMD for this
task. They further stated that they felt more immersed in the
scene and so were more focused on the task. In addition, a
majority of the users (70%) reported that HMD afforded
them a superior sense of the spatial awareness which they
claimed was important to their success. Approximately a
third mentioned that they actively used the virtual memory
palace setup by associating the information relative to their
own body. This ability to associate information with the
spatial context around the body only adds to the benefit of
increased immersion afforded by the HMD.
We note the interesting results we obtained with the
display ordering. When starting with the desktop and then
using the HMD, we observed a significant improvement
as compared to starting with the HMD and then using the
desktop. A possible explanation for this could be that those
who used the HMD first are able to benefit from the HMD’s
superior immersion, which they lose when they transfer to
the desktop. However, when the users start on the desktop
they invest a greater effort to memorize the information and
therefore when they transfer to the HMD, they not only keep
their dedication but also gain from the improved immersion.
5.1 Study limitations
In general, it is a difficult design decision to balance the
goals of experimental control and ecological validity. In
our study, we placed the faces for a particular face set in
the same locations for all participants. However, since the
participants were free to look in any direction, the order of
presentation of faces was self-determined. We could have
restricted the participants to look at the faces in a prede-
termined order. However, we allowed the participants to
look around freely, so that the results would achieve greater
ecological validity. Randomization of faces could have led
to unintended consequences; having the Dalai Lama’s face
next to Abraham Lincoln’s in one instantiation could alter
its memorability, as could the opportune positioning of the
Dalai Lama on a roof-top background. To avoid such inter-
object semantic saliency confounds, we decided to preserve
the same ordering of faces for all participants that viewed
the scene with a given set of faces. We recognize that not
randomizing the stimuli in a within-subject design could
introduce a bias. To make sure that this did not result in any
significant effects, we carried out a four-way mixed ANOVA
(reported at the beginning of Sect.4) and we did not find any
statistically significant effects on recall due to the scenes,
face sets, or the ordering of the display conditions. Previous
research, such as Loomis etal. (1999), points out the trade-
offs between experimental control and ecological validity for
virtual environments. Parsons (2015) persuasively argues for
designing virtual environment studies that strike a balance
between naturalistic observation and the need for exacting
control over variables.
The modality of interactive exploration of the virtual
environment in the two conditions was different (head track-
ing versus mouse tracking). Thus, differences in the recall
performance may be explained by this diverse interaction
modality. Our study did not attempt to distinguish the role
Fig. 10 The performance of participants going from a desktop to a
HMD and from a HMD to a desktop, showing the median, first and
third quartiles
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
11Virtual Reality (2019) 23:1–15
1 3
of proprioceptive and vestibular information from visual
stimuli, but examined them in the respective contexts of
immersive HMD and desktop display conditions. It will be
interesting to examine the relative advantage of the diverse
interaction modalities with the same display modality, in
future user studies.
5.2 Conclusions
We found that the use of virtual memory palaces in HMD
condition improves recall accuracy when compared to
using a traditional desktop condition. We had 40 partici-
pants memorize and recall faces on two display–interaction
Fig. 11 Face Set 1, containing
21 faces
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
12 Virtual Reality (2019) 23:1–15
1 3
modalities for two virtual memory palaces, with two dif-
ferent sets of faces. The HMD condition was found to have
8.8% improvement in recall accuracy compared to the desk-
top condition, and this was found to be statistically signifi-
cant. This suggests an exciting opportunity for the role of
immersive virtual environments in assisting in recall. Given
the results of our user study, we believe that virtual memory
palaces offer us a fascinating insight into how we may be
able to organize and structure large information spaces and
navigate them in ways that assist in superior recall.
One of the strengths of virtual reality is the experience
of presence through immersion that it provides (Sanchez-
Vives and Slater 2005; Skarbez etal. 2017). If memory
recall could be enhanced through immersively experienc-
ing the environment in which the information was learned,
it would suggest that virtual environments could serve as a
valuable tool for various facets of retrospective cognizance,
including retention and recall.
Fig. 12 Face Set 2, containing
21 faces
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
13Virtual Reality (2019) 23:1–15
1 3
5.3 Future work
Our study provides a tantalizing glimpse into what may lie
ahead in virtual-environment-based tools to enhance human
memory. The next steps will be to identify and characterize
what elements of virtual memory palaces are most effective
in eliciting a superior information recall. At present, we have
only studied the effect of in-place stereoscopic immersion,
in which the participants were allowed to freely rotate their
viewpoint but not translate. It will be valuable to study how
the addition of translation impacts information recall in a
virtual memory palace.
Other directions of future studies could include ele-
ments in the architecture of the virtual memory palaces
such as their design, the visual saliency of the structure of
model(Kim etal. 2010), their type, and various kinds of lay-
outs and distribution of content that could help with recall.
Another interesting future work would be to allow people
to build their own virtual memory palaces, manipulate and
organize the content on their own, and then ask them to
recall that information. If their active participation in the
organization of the data in virtual memory palaces makes a
meaningful difference, then that could be further useful in
designing interaction-based virtual environments that could
one day assist in far superior information management and
recall tools than those currently available to us. Yet another
interesting future direction of research could be to compare
elements of virtual memory palaces that are highly personal
versus those that could be used by larger groups. Much as
textbooks and videos are used today for knowledge dissemi-
nation, it could be possible for virtual memory palaces to
be used one day for effective transfer of mnemonic devices
among humans in virtual environments.
Acknowledgements We would like to extend our sincere appreciation
to the anonymous reviewers who helped us refine this paper that signifi-
cantly improved its presentation. We appreciate the support of the NSF
Grants 14-29404, 15-64212, the State of Maryland’s MPower initiative,
and the NVIDIA CUDA Center of Excellence. Any opinions, findings,
conclusions, or recommendations expressed in this article are those
of the authors and do not necessarily reflect the views of the research
sponsors. Lastly, we would like to thank the 40 study participants.
Open Access This article is distributed under the terms of the Crea-
tive Commons Attribution 4.0 International License (http://creat iveco
mmons .org/licen ses/by/4.0/), which permits use, duplication, adapta-
tion, distribution and reproduction in any medium or format, as long
as you give appropriate credit to the original author(s) and the source,
provide a link to the Creative Commons license and indicate if changes
were made.
Appendix
See Tables1, 2 and 3.
Table 1 Trend scores for each face for Face Set 1 from Google Trends,
with the average trend score of 30.5 and standard deviation of 21.86.
The data were collected from April, May, June, and July of 2015
FaceSet1 APR MAY JUN JUL AVG
Martin Luther King 14 14 10 8 11.5
Bill Gates 48 50 48 47 48.25
Mahatma Gandhi 57 55 57 54 55.75
Donald Duck 66 71 71 65 68.25
Buzz Lightyear 37 38 36 41 38
George Washington 21 21 15 15 18
George Bush 2 2 2 2 2
Oprah Winfrey 13 12 10 12 11.75
Taylor Swift 59 79 69 68 68.75
Steve Jobs 2 3 2 3 2.5
Michael Jackson 3 3 4 3 3.25
Harry Potter 6 7 8 11 8
Stephen Hawking 43 36 31 30 35
Mona Lisa 38 38 31 29 34
Shrek 9 10 9 9 9.25
Frodo Baggins 19 18 19 17 18.25
Albert Einstein 44 43 39 34 40
Vladimir Putin 36 31 27 22 29
Galileo Galilei 34 35 32 35 34
King Louis XVI 65 73 60 56 63.5
Napoleon Bonaparte 42 44 46 34 41.5
Table 2 Trend scores for each face for Face Set 2 from Google Trends,
with the average trend score of 29.83 and standard deviation of 18.32.
The data were collected from April, May, June, and July of 2015
FaceSet2 APR MAY JUN JUL AVG
Abraham Lincoln 39 35 26 25 31.25
Katy Perry 37 37 35 34 35.75
Hillary Clinton 32 11 12 13 17
Arnold Schwarzenegger 25 25 34 39 30.75
Tom Cruise 17 16 15 28 19
Batman 27 27 29 37 30
Mickey Mouse 76 75 73 78 75.5
Marilyn Monroe 49 56 64 45 53.5
Testudo 2 2 2 3 2.25
Winston Churchill 48 50 38 36 43
Barbie 42 42 44 45 43.25
Mark Zuckerberg 21 20 18 19 19.5
Robin Williams 2 1 1 2 1.5
Dalai Lama 26 26 32 36 30
Kim Jong-un 20 30 17 16 20.75
Harrison Ford 21 15 12 15 15.75
Bill Clinton 22 18 14 14 17
Michelle Obama 8 6 8 9 7.75
Queen Victoria 48 55 42 40 46.25
Cleopatra 56 52 50 51 52.25
Nikola Tesla 33 36 32 37 34.5
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
14 Virtual Reality (2019) 23:1–15
1 3
References
3DMarko (2011) Medieval town 01 Dubrovnik. https ://www.turbo squid
.com/FullP revie w/Index .cfm/ID/63241 5. Retrieved 10 May 2018
3DMarko (2014) Palace interior 02. https ://www.turbo squid .com/FullP
revie w/Index .cfm/ID/80957 7. Retrieved 10 May 2018
Atkinson RC, Shiffrin RM (1968) Human memory: a proposed system
and its control processes. Psychol Learn Motiv 2:89–195
Baddeley AD, Hitch G (1974) Working memory. Psychol Learn Motiv
8:47–89
Barry C, Lever C, Hayman R, Hartley T, Burton S, O’Keefe J, Jeffery
K, Burgess N (2006) The boundary vector cell model of place cell
firing and spatial memory. Rev Neurosci 17(1–2):71–98
Barsalou LW (2008) Grounded cognition. Annu Rev Psychol
59:617–645
Baumann O, Mattingley JB (2010) Medial parietal cortex
encodes perceived heading direction in humans. J Neurosci
30(39):12,897–12,901
Bowman DA, McMahan RP (2007) Virtual reality: how much immer-
sion is enough? Computer 40(7):36–43
Brooks BM (1999) The specificity of memory enhancement during
interaction with a virtual environment. Memory 7(1):65–78
Brooks FP Jr, Airey J, Alspaugh J, Bell A, Brown R, Hill C, Nimscheck
U, Rheingans P, Rohlf J, Smith D, Turner D, Varshney A, Wang
Y, Weber H, Yuan X (1992) Six generations of building walk-
through: final technical report to the National Science Foundation
TR92-026, Department of Computer Science. University of North
Carolina at Chapel Hill
Brown MW, Aggleton JP (2001) Recognition memory: what are the
roles of the perirhinal cortex and hippocampus? Nat Rev Neurosci
2(1):51–61
Burgess N (2008) Spatial cognition and the brain. Ann N Y Acad Sci
1124(1):77–97. https ://doi.org/10.1196/annal s.1440.002
Buzsáki G, Moser EI (2013) Memory, navigation and theta rhythm in
the hippocampal-entorhinal system. Nat Neurosci 16(2):130–138
Ekstrom AD, Kahana MJ, Caplan JB, Fields TA, Isham EA, Newman
EL, Fried I (2003) Cellular networks underlying human spatial
navigation. Nature 425(6954):184–188
Fassbender E, Heiden W (2006) The virtual memory palace. J Comput
Inf Syst 2(1):457–464
Gardner H (2006) Multiple intelligences: new horizons. Basic books,
New York. ISBN 978-0465047680
Godden DR, Baddeley AD (1975) Context-dependent memory in
two natural environments: on land and underwater. Br J Psychol
66(3):325–331
Godwin-Jones R (2010) Emerging technologies from memory palaces
to spacing algorithms: approaches to second-language vocabulary
learning. Lang Learn Technol 14(2):4
Harman J (2001) Creating a memory palace using a computer. In: CHI
’01 extended abstracts on human factors in computing systems,
pp 407–408
Harman J, Brown R, Johnson D (2017) Improved memory elicitation
in virtual reality: new experimental results and insights. In: IFIP
conference on human–computer interaction, Springer, pp 128–146
Harris JE (1980) Memory aids people use: two interview studies. Mem
Cogn 8(1):31–38
Hartley T, Lever C, Burgess N, O’Keefe J (2014) Space in the brain:
how the hippocampal formation supports spatial cognition. Philos
Trans R Soc B 369(1635):20120,510
Hok V, Save E, Lenck-Santini P, Poucet B (2005) Coding for spatial
goals in the prelimbic/infralimbic area of the rat frontal cortex.
Proc Natl Acad Sci USA 102(12):4602–4607
Julian J (1976) The origin of consciousness in the breakdown of the
bicameral mind. The Adelphi, Milan
Kim Y, Varshney A, Jacobs DW, Guimbretière F (2010) Mesh saliency
and human eye fixations. ACM Trans Appl Percept (TAP) 7(2):12
Kim J, Delcasso S, Lee I (2011) Neural correlates of object-in-place
learning in hippocampus and prefrontal cortex. J Neurosci
31(47):16,991–17,006
Knauff M (2013) Space to reason: a spatial theory of human thought.
MIT Press, Cambridge
Legge EL, Madan CR, Ng ET, Caplan JB (2012) Building a memory
palace in minutes: equivalent memory performance using virtual
versus conventional environments with the method of loci. Acta
Psychol 141(3):380–390
Leutgeb S, Leutgeb JK, Moser MB, Moser EI (2005) Place cells, spa-
tial maps and the population code for memory. Curr Opin Neu-
robiol 15(6):738–746
Lever C, Burton S, Jeewajee A, O’Keefe J, Burgess N (2009) Bound-
ary vector cells in the subiculum of the hippocampal formation. J
Neurosci 29(31):9771–9777
Loomis JM, Blascovich JJ, Beall AC (1999) Immersive virtual environ-
ment technology as a basic research tool in psychology. Behav Res
Methods Instrum Comput 31(4):557–564
Madl T, Chen K, Montaldi D, Trappl R (2015) Computational cog-
nitive models of spatial memory in navigation space: a review.
Neural Netw 65:18–43
Mania K, Chalmers A (2001) The effects of levels of immersion on
memory and presence in virtual environments: a reality centered
approach. CyberPsychol Behav 4(2):247–264
Mania K, Troscianko T, Hawkes R, Chalmers A (2003) Fidelity
metrics for virtual environment simulations based on spatial
memory awareness states. Presence Teleoper Virtual Environ
12(3):296–310
Table 3 Angular resolution of
faces in the town and palace
scenes, with the average and
standard deviation of the
angular resolutions of the set of
faces for each scene
The difference in angular reso-
lutions between the two scenes
was not statistically significant
with
(p=0.44
>
0.05)
Face number Town Palace
1 6.61 7.86
2 7.29 7.86
3 5.72 7.81
4 6.92 7.81
5 7.29 8.02
6 5.83 8.02
7 7.81 4.73
8 7.81 5.46
9 7.60 5.46
10 5.98 4.73
11 6.04 6.51
12 7.44 6.40
13 4.94 4.94
14 4.68 4.94
15 7.44 6.09
16 4.37 6.09
17 5.26 5.46
18 4.53 5.36
19 7.81 7.55
20 5.72 6.35
21 7.81 6.35
Average 6.42 6.37
Standard Dev 1.17 1.16
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
15Virtual Reality (2019) 23:1–15
1 3
Mayer JD, Salovey P, Caruso DR, Sitarenios G (2001) Emotional intel-
ligence as a standard intelligence. Emotion 1(3):232–242
McCabe JA (2015) Location, location, location! Demonstrating
the mnemonic benefit of the method of loci. Teach Psychol
42(2):169–173
Miller GA (1956) The magical number seven, plus or minus two: some
limits on our capacity for processing information. Psychol Rev
63(2):81
Moser EI, Kropff E, Moser MB (2008) Place cells, grid cells, and
the brain’s spatial representation system. Annu Rev Neurosci
31:69–89
O’Regan JK, Noë A (2001) A sensorimotor account of vision and
visual consciousness. Behav Brain Sci 24(5):939–973
Parsons TD (2015) Virtual reality for enhanced ecological validity and
experimental control in the clinical, affective and social neuro-
sciences. Front Hum Neurosci 9:660
Pausch R, Proffitt D, Williams G (1997) Quantifying immersion in
virtual reality. In: Proceedings of the 24th annual conference on
computer graphics and interactive techniques, SIGGRAPH ’97,
pp 13–18
Perrault ST, Lecolinet E, Bourse YP, Zhao S, Guiard Y (2015) Physical
loci: leveraging spatial, object and semantic memory for command
selection. In: Proceedings of the 33rd annual ACM conference on
human factors in computing systems, CHI ’15, pp 299–308
Ragan ED, Sowndararajan A, Kopper R, Bowman DA (2010) The
effects of higher levels of immersion on procedure memorization
performance and implications for educational virtual environ-
ments. Presence Teleoper Virtual Environ 19(6):527–543
Repetto C, Serino S, Macedonia M, Riva G (2016) Virtual reality as
an embodied tool to enhance episodic memory in elderly. Front
Psychol 7:1839:1–1839:4
Richardson AE, Montello DR, Hegarty M (1999) Spatial knowledge
acquisition from maps and from navigation in real and virtual
environments. Mem Cognit 27(4):741–750
Roediger HL (1979) Implicit and explicit memory models. Bull Psy-
chon Soc 13(6):339–342
Ruddle RA, Payne SJ, Jones DM (1999) Navigating large-scale virtual
environments: what differences occur between helmet-mounted
and desktop displays? Presence Teleoper Virtual Environ
8(2):157–168
Sanchez-Vives MV, Slater M (2005) From presence to consciousness
through virtual reality. Nat Rev Neurosci 6(4):332–339
Shapiro L (2010) Embodied cognition. Routledge, New York. ISBN
978-0415773423
Skarbez R, Brooks FP Jr, Whitton MC (2017) A survey of presence
and related concepts. ACM Comput Surv 50(6):96:1–96:39. https
://doi.org/10.1145/31343 01
Slater M (2009) Place illusion and plausibility can lead to realistic
behaviour in immersive virtual environments. Philos Trans R Soc
Lond B Biol Sci 364(1535):3549–3557
Sowndararajan A, Wang R, Bowman DA (2008) Quantifying the ben-
efits of immersion for procedural training. In: Proceedings of the
2008 workshop on immersive projection technologies/emerging
display technologies, IPT/EDT ’08, pp 2:1–2:4
Wraga M, Creem-Regehr SH, Proffitt DR (2004) Spatial updating of
virtual displays. Mem Cognit 32(3):399–415
Yates FA (1992) The art of memory, vol 64. Random House, New York
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com
... Using virtual memory palaces [19] in HMD conditions improves memorization accuracy compared to using traditional desktop conditions. The results of the study [19] suggest an interesting opportunity for the role of immersive virtual environments in aiding in equipping. ...
... Using virtual memory palaces [19] in HMD conditions improves memorization accuracy compared to using traditional desktop conditions. The results of the study [19] suggest an interesting opportunity for the role of immersive virtual environments in aiding in equipping. The experience of presence through immersion is undoubtedly one of the strengths of VR [20]. ...
... The experience of presence through immersion is undoubtedly one of the strengths of VR [20]. Assuming that it would be possible to enhance the memorability effect through immersive experiencing of the environment in which education and training would take place, virtual environments could effectively serve as a valuable tool for various aspects of retrospective cognition, including memorization [19]. ...
Article
Full-text available
This study examines the effects of different levels of immersion in Virtual Reality (VR) on memory retention and learning outcomes within industrial safety training. A controlled experiment was conducted using a VR simulation of firefighting in an industrial environment, with varying combinations of immersive elements such as audio guidance and visual management cues. Memory retention was assessed immediately after the simulation and re-evaluated three months later. The results indicate that while overall immersion levels did not produce statistically significant differences in memory retention, specific elements, particularly the use of audio guidance and visual management, showed a significant positive impact on short-term memory retention. However, the simultaneous use of these elements occasionally led to cognitive overload, reducing the overall effectiveness. These findings suggest that the design of VR training programs should prioritize a balanced incorporation of immersive elements to optimize learning outcomes without overwhelming participants. The study contributes to the ongoing discourse on the application of VR in industrial training and highlights the nuanced role of individual immersive features in enhancing cognitive processes.
... In a lesson that introduced 25 new words, the first group (VRH) achieved markedly better results. It was further found that retention of information increased when a VRH environment was utilized [56]. In other words, using VRH enables a more effective learning experience and plays a significant role in retaining the learned information. ...
... In contrast, when words were processed deeply, recall was enhanced, and long-term retention improved without the need for repetition or memorization [65]. For example, Krokos et al. [56] found that information displayed in space and associatively linked to elements presented in the learning environment increased the depth of processing and memory retention. ...
Article
Full-text available
There is a need to find innovative learning methods that enable accelerated learning of a foreign language. This study examined the effect of computer-assisted language learning (CALL) in acquiring a foreign language, which combines cognitive and emotional stimuli in the background. The study explored two factors related to the acquisition of a foreign language: the duration and scope of the learning process and the depth of internalization of the newly acquired language. Another objective was to assess the learning method in two learning environments, 2D and VR, to determine if the learning environment affects the learning results and leads to better vocabulary retention. One hundred native French speakers, with an average age of 47.5, participated in the study and had no prior knowledge of the newly acquired language. We randomly divided the participants into two groups (2D and VR). They studied 550 words in a new language for five days: 30 min each evening and 15 min in the morning. The post-learning test pointed out that both groups improved their vocabulary scores significantly. Approximately one month after the learning experience, we administered a knowledge retention test to 32 participants and found that the level of knowledge had been retained. Finally, background variables (e.g., gender, age, and previous knowledge of the newly acquired language) did not affect the learning results. The findings indicate that CALL, which integrates background cognitive and emotional stimuli in both learning environments, significantly accelerates learning pace, broadens the scope of newly acquired words, and ensures retention. The level of improvement observed in our study is notably higher than that reported in the literature for studies that had previously evaluated CALL and in-class language acquisition.
... Research has shown that there are a number of advantages to utilising virtual reality in educational settings (Cheng and Tsai, 2013;Schott and Marshall, 2018). Its uses have been demonstrated to improve students' attitudes towards learning (Goldin and Katz, 2007;Hõrak, 2019;Lazar and Panisoara, 2018), improve their comprehension of the subject matter, boost their engagement in learning, encourages student-focused pedagogy and active learning, enhance memorisation (Krokos et al., 2019), creates pleasant classroom environments (Kaplan-Rakowski and Wojdynski, 2018;Chen et al., 2022), and lowers anxiety (Gruber and Kaplan-Rakowski, 2020;Kaplan-Rakowski and Gruber, 2022). On the whole, it results in better educational outcomes. ...
Article
Full-text available
This research endeavor aimed to develop and validate a comprehensive scale to assess educators’ knowledge, perceptions, and readiness (KPR) towards the adoption of Extended Reality (XR) technologies in classroom settings. XR (Virtual Reality -VR, Augmented Reality-AR, and Mixed Reality-MR) has transpired as a ground-breaking tool within the educational landscape, but its effective integration completely relies on educators. Grounded in three theoretical frameworks- Technological Pedagogical Content Knowledge (TPACK) for Knowledge, Technology Acceptance Model (TAM) for Perception, and Unified Theory of Acceptance and Use of Technology (UTAUT) for Readiness- the initial scale included 41 items that were drawn/inspired from different papers published in international journals. After this, expert reviews and Content Validity Index (CVI) calculation was undertaken, which resulted in three items being removed, resulting in a 38-item scale. The scale was then administered to 700 University educators across India in the mid of 2024. Exploratory Factor Analysis (EFA) and Confirmatory Factor Analysis (CFA) validated the scale structure, confirming three distinct dimensions (KPR) with strong internal consistency (> 0.90). The KPR-XR scale offers a reliable means to examine the critical factors (KPR) that influence educators' adoption of XR technologies, providing important implications for educational practice and policy.
... Over the past 20 years, several studies have shown that VR training tools effectively acquire the required skills, which are then transferred to the real world [17][18][19]. Research by Krokos et al. [20] even suggests that participants remember more information and can better apply the knowledge they gained after participating in VR-based training than traditional skill development forms. However, research by Cooper et al. [21] has shown that VR-based training improves the effectiveness of skill acquisition/improvement, but on the condition that the simulated training environment is of high quality or includes additional task-relevant sensory cues. ...
Article
Full-text available
(1) Background: This article presents research results from a project to develop a simulator and training program for tram drivers based on virtual reality tools. This article aims to present the research results on the risk assessment of adverse events to develop training scenarios for tram drivers. (2) Methods: The research methodology involved four steps, including the identification of adverse events based on interviews with experts, estimation of risk parameters, risk assessment using fuzzy logic, and risk evaluation to identify events recommended for the training program. (3) Results: Thirteen adverse events related to tram driving were identified and ranked according to the proposed classification. A risk assessment was then performed for the selected group of events using fuzzy logic models. The results made it possible to recommend situations that should be mapped in the virtual world as part of a tram driver training program. (4) Conclusions: The results made it possible to distinguish a group of events that should constitute the training area dedicated to tram drivers. Including the developed emergency scenarios in the training program allows employees to examine their reactions to stressful or dangerous situations and better prepare themselves for future duties.
... While the literature presents a number of cognitive advantages of immersive visualization, studies assessing its effect on memorability yielded mixed results, indicating that its potential retention improvements may be context-dependent and limited to certain workload conditions [11,38,55,116]. Under the premise that both encoding and testing occur under the same immersive conditions, Krokos et al. suggested that VR's embodied interaction with virtual objects could yield superior information recall in comparison with desktop displays [59]. Buttussi and Chittaro observed improved retention of spatial knowledge regarding distance estimations and spatial relations with VR compared to a touchscreen device [14]. ...
Conference Paper
Full-text available
Although Virtual Reality (VR) has undoubtedly improved human interaction with 3D data, users still face difficulties retaining important details of complex digital objects in preparation for physical tasks. To address this issue, we evaluated the potential of visuohaptic integration to improve the memorability of virtual objects in immersive visualizations. In a user study (N=20), participants performed a delayed match-to-sample task where they memorized stimuli of visual, haptic, or visuohaptic encoding conditions. We assessed performance differences between these encoding modalities through error rates and response times. We found that visuohaptic encoding significantly improved memorization accuracy compared to unimodal visual and haptic conditions. Our analysis indicates that integrating haptics into immersive visualizations enhances the memorability of digital objects. We discuss its implications for the optimal encoding design in VR applications that assist professionals who need to memorize and recall virtual objects in their daily work.
Article
Purpose Responding to calls for accountants to engage with modern technologies and explore data visualisation within a three-dimensional (3D) environment, this study aims to explore whether social and environmental accounting (SEA) data visualisation is a promising use case for 3D game engine technology. Design/methodology/approach Drawing on visual perception and embodiment theories, this study uses photo-elicitation, a qualitative research method, to explore the usefulness of two-dimensional (2D) and 3D visualisations of sustainability information in a 3D virtual environment. This study provides three stimuli: numerical data, 2D visualisations and 3D visualisations, and asks open-ended questions regarding future applications. Twelve semi-structured interviews were conducted with academics, preparers and users of sustainability reports to obtain responses to these stimuli. Findings The key finding is that visualisation of SEA information may indeed be a strong use case for 3D game technology, but only for certain data and for certain audiences. Presenting information within a 3D virtual environment offered enhanced engagement and contextual understanding but reduced navigation speed and data clarity. Participants were enthusiastic about the potential of a museum-like experience, incorporating interactivity and community, but felt that the appropriate audience was more likely to be novices than experts. Practical implications This study suggests that deploying 3D game engine technology can be a powerful tool for presenting sustainability information but requires significant resources. The optimum audience is likely to be novices, and a key design principle is to ensure the virtual environment supports, rather than overwhelms, the information presented within that environment. Originality/value This study introduces a novel application of 3D visualisation technology within the SEA context, offering original insights into its potential to enhance user understanding and decision-making capabilities. This study highlights the technology’s value not as a replacement for traditional reporting but as a supplementary educational tool. The study also provides a novel setting for the photo-elicitation method, demonstrating this approach’s utility in a 3D environment.
Chapter
Three-dimensional virtual reality (VR) environments, whether utilized on desktop platforms or immersive screens, have been acknowledged for facilitating innovative and highly captivating methods of engaging with digital content across various domains of application. Extensive research conducted in recent years has consistently indicated that the utilization of 3D interfaces, as opposed to their 2D counterparts, can result in improvements across multiple dimensions of performance. These improvements encompass a more comprehensive comprehension and retention of information, an increased capacity for creative and efficient collaboration, and the ability to seamlessly integrate multiple sources of information into workflows in a more expedient manner. Concurrently, the integration of digital content, such as documents, audio-visual content, and web browsers, into 3D spaces is often determined by the creators of said spaces, based on aesthetic considerations or on a case-by-case basis influenced by the specific workflow. This chapter aims to present the findings of an experiment that was conducted to gain a deeper understanding of how users prefer to arrange digital content within their 3D environments, taking into account factors such as the subject matter, the format of the content (e.g., text-based, image, or audio-visual), and the presence of 3D objects within the space. The outcomes of this experiment, as outlined in this chapter, can offer valuable insights for the development of future design methodologies for 3D VR and may also offer support for the implementation of automated solutions for content arrangement.
Chapter
This chapter explores the potential of desktop virtual reality in enhancing memory and learning, along with an examination of advertising evolution and online formats. Two studies, with a total of fifty participants, aimed to compare memory retention of advertisements in 3D highlighted and embedded formats, in both 2D browser and desktop VR environments. The findings suggest a distinct impact of 3D highlighting in VR contexts on memory retention compared to traditional 2D settings. This aspect holds significant importance in educational contexts, particularly as traditional e-learning is gradually transitioning towards VR-based education. In virtual educational settings, instructors can continuously present essential diagrams, making necessary information more accessible and promoting improved comprehension among learners.
Article
Full-text available
The authors have claimed that emotional intelligence (EI) meets traditional standards for an intelligence (J. D. Mayer, D. R. Caruso, & P. Salovey, 1999). R. D. Roberts, M. Zeidner, and G. Matthews (2001) questioned whether that claim was warranted. The central issue raised by Roberts et al. concerning Mayer et al. (1999) is whether there are correct answers to questions on tests purporting to measure EI as a set of abilities. To address this issue (and others), the present authors briefly restate their view of intelligence, emotion, and EI. They then present arguments for the reasonableness of measuring EI as an ability, indicate that correct answers exist, and summarize recent data suggesting that such measures are, indeed, reliable.
Article
Full-text available
In the last decade, embodiment has dramatically influenced our conception of cognition. In this new frame, episodic memory, and particularly memory decline have been reinterpreted. Interventions supporting memory in the aging population address the connection between mind and body. Here, we discuss the use of Virtual Reality (VR) as an innovative tool to support episodic memory in older adults.
Book
An argument against the role of visual imagination in reasoning that proposes a spatial theory of human thought, supported by empirical and computational evidence. Many scholars believe that visual mental imagery plays a key role in reasoning. In Space to Reason, Markus Knauff argues against this view, proposing that visual images are not relevant for reasoning and can even impede the process. He also argues against the claim that human thinking is solely based on abstract symbols and is completely embedded in language. Knauff proposes a third way to think about human reasoning that relies on supramodal spatial layout models, which are more abstract than pictorial images and more concrete than linguistic representations. He argues that these spatial layout models are at the heart of human thought, even thought about nonspatial relations in the world. For Knauff the visual images that we so often associate with reasoning are only in the foreground of conscious experience. Behind the images, the actual logical work is carried out by reasoning-specific operations on these spatial layout models. Knauff also offers a solution to the problem of indeterminacy in human reasoning, introducing the notion of a preferred layout model, which is one layout model among others that has the best chance of being mentally constructed and thus guides the further process of thought. Knauff's "space to reason" theory covers the functional, the algorithmic, and the implementational level of analysis and is corroborated by psychological experiments, functional brain imaging, and computational modeling.
Article
The presence construct, most commonly defined as the sense of “being there,” has driven research and development of virtual environments (VEs) for decades. Despite that, there is not widespread agreement on how to define or operationalize this construct. The literature contains many different definitions of presence and many proposed measures for it. This article reviews many of the definitions, measures, and models of presence from the literature. We also review several related constructs, including social presence, copresence, immersion, agency, transportation, reality judgment, and embodiment. In addition, we present a meta-analysis of existing presence models and propose a model of presence informed by Slater’s Place Illusion and Plausibility Illusion constructs.
Conference Paper
Eliciting accurate and complete knowledge from individuals is a non-trivial challenge. In this paper, we present the evaluation of a virtual-world based approach, informed by situated cognition theory, which aims to assist with knowledge elicitation. In this approach, we place users into 3D virtual worlds which represent real-world locations and ask users to describe information related to tasks completed in those locations. Through an empirical A/B evaluation of 62 users, we explore the differences in recall ability and behaviour of those viewing the virtual world via a virtual reality headset and those viewing the virtual world on a monitor. Previous results suggest that the use of a virtual reality headset was able to meaningfully improve memory recall ability within the given scenario. In this study, we adjust experiment protocol to explore the potential confounds of time taken and tool usability. After controlling for these possible confounds, we once again found that those given a virtual reality headset were able to recall more information about the given task than those viewing the virtual world on a monitor.