Conference PaperPDF Available

Pedagogical Agents in Educational VR: An in the Wild Study


Abstract and Figures

Pedagogical agents are theorized to increase humans’ effort to understand computerized instructions. Despite the pedagogical promises of VR, the usefulness of pedagogical agents in VR remains uncertain. Based on this gap, and inspired by global efforts to advance remote learning during the COVID-19 pandemic, we conducted an educational VR study in-the-wild (𝑁 = 161). With a 2 × 2 + 1 between subjects design, we manipulated the appearance and behavior of a virtual museum guide in an exhibition about viruses. Factual and conceptual learning outcomes as well as subjective learning experience measures were collected. In general, participants reported high enjoyment and had significant knowledge acquisition. We found that the agent’s appearance and behavior impacted factual knowledge gain. We also report an interaction effect between behavioral and visual realism for conceptual knowledge gain. Our findings nuance classical multimedia learning theories and provide directions for employing agents in immersive learning environments.
Content may be subject to copyright.
Pedagogical Agents in Educational VR: An in the Wild Study
Gustav Bøg Petersen
Department of Psychology
University of Copenhagen
Copenhagen, Denmark
Aske Mottelson
Department of Psychology
University of Copenhagen
Copenhagen, Denmark
Guido Makransky
Department of Psychology
University of Copenhagen
Copenhagen, Denmark
Pedagogical agents are theorized to increase humans’ eort to
understand computerized instructions. Despite the pedagogical
promises of VR, the usefulness of pedagogical agents in VR re-
mains uncertain. Based on this gap, and inspired by global eorts
to advance remote learning during the COVID-19 pandemic, we
conducted an educational VR study in-the-wild (
161). With a
1between subjects design, we manipulated the appearance
and behavior of a virtual museum guide in an exhibition about
viruses. Factual and conceptual learning outcomes as well as sub-
jective learning experience measures were collected. In general,
participants reported high enjoyment and had signicant knowl-
edge acquisition. We found that the agent’s appearance and behav-
ior impacted factual knowledge gain. We also report an interac-
tion eect between behavioral and visual realism for conceptual
knowledge gain. Our ndings nuance classical multimedia learning
theories and provide directions for employing agents in immersive
learning environments.
Human-centered computing Virtual reality
; Empirical
studies in HCI;
Applied computing
Interactive learning envi-
Immersive Virtual Reality; Educational Technology; Learning; Cog-
nitive Load, Pedagogical Agents
ACM Reference Format:
Gustav Bøg Petersen, Aske Mottelson, and Guido Makransky. 2021. Peda-
gogical Agents in Educational VR: An in the Wild Study. In CHI Conference
on Human Factors in Computing Systems (CHI ’21), May 8–13, 2021, Yoko-
hama, Japan. ACM, New York, NY, USA, 12 pages.
Imagine visiting your favorite museum in the connes of your own
home through the technology of virtual reality (VR); not having to
stand in line to explore exhilarating exhibitions, and at the same
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than ACM
must be honored. Abstracting with credit is permitted. To copy otherwise, or republish,
to post on servers or to redistribute to lists, requires prior specic permission and/or a
fee. Request permissions from
CHI ’21, May 8–13, 2021, Yokohama, Japan
©2021 Association for Computing Machinery.
ACM ISBN 978-1-4503-8096-6/21/05. . . $15.00
time having your own private museum guide at hand. The goal of
this study is to examine the role of a virtual museum guide (hence-
forth, a pedagogical agent) and its design in producing learning
during a tour in an educational VR museum.
VR has recently received increased attention as an educational
tool [
]. Theoretically, the contents of VR experiences are only
bounded by the limits of our imagination [
]. In practice, however,
educational VR often takes the shape of simulations or depictions of
reality, and VR is typically combined with face-to-face instruction
when used in the classroom (e.g., [
]). Such use of VR is in line
with the blended learning systems approach [11].
Blended learning systems are in certain situations not ideal, be-
cause of cost or practicalities of mixing digital and classical learning.
There are situations where real-life circumstances force us to rely
solely on remote technology for instruction. This, for instance, is
evident during the coronavirus disease 2019 (COVID-19) pandemic,
where lock down has forced many teachers and students to tran-
sition into a remote learning style [
]. New approaches to remote
teaching, including use of pedagogical agents and VR for remote
learning, are in such situations desired.
Unlike avatars, that are controlled by humans [
], pedagogical
agents are anthropomorphous computer-controlled virtual charac-
ters that are used in online learning environments to serve various
instructional goals [
]. The purpose of using pedagogical agents
is to mimic the social processes that usually take place in real-life
teaching. According to multimedia learning theories, people are
inclined to treat computerized agents as social partners if they ex-
hibit social cues [
]. This, in turn, should motivate the learner
to make sense of the presented material [
]. However, there is
disagreement with regards to the usefulness of pedagogical agents.
Opponents claim that the visual appearance of a pedagogical agent
adds unnecessary distraction to the learning experience [
]. This
view is consistent with the notion that the human working memory
is limited and that embellished materials therefore impede learn-
ing [
]. For VR, where it is possible to render pedagogical agents in
realistic 3D with higher behavioral realism compared to traditional
media, their eect on learning is even less supported.
Inspired by the eorts to reorganize teaching into online formats
during the COVID-19 pandemic, we created a virtual museum on
the topic of viral diseases. The museum was accessed by participants
‘in the wild’ using their own Oculus Quest head-mounted display
(HMD), an approach which has been proven feasible in previous VR
research [
]. We were interested in how the addition of a
social entity to the museum impacted participants’ learning outputs
and subjective ratings of the learning experience. Specically, we
investigated the role of the pedagogical agent’s appearance and
behavior. The hypothesis behind this was that a more realistic
CHI ’21, May 8–13, 2021, Yokohama, Japan Petersen et al.
pedagogical agent would induce a stronger sense of interacting
with a social partner, and in turn enhance learning.
Our results show a large eect on learning (Cohen’s
They also show that participants enjoyed the learning (
We report that the presence of a pedagogical agent diminished fac-
tual knowledge acquisition; yet, for learning conceptual knowledge,
an interaction eect between visual and behavioral realism shows
that the presence of pedagogical agents in educational VR might
be worthwhile.
2.1 VR in Education
Recent work in educational psychology and HCI emphasize the
potentials of learning in immersive environments (e.g., [
]); these works together show how instructional and visual design
of virtual environments are imperative for learning outcomes.
Jensen and Konradsen [
] recently conducted a review of re-
search on education and training with the current generation of
HMDs. They argue that VR does not automatically cause learning
but rather provides a possibility for accessing simulations that might
induce learning. Furthermore, with regard to cognitive skills acqui-
sition, immersive experiences risk overwhelming learners unless
they are designed specically for the purpose and not overly inter-
active [
]. According to Jensen and Konradsen [
], the biggest
barriers to adopting VR in education and training are a lack of con-
tent as well as the technical skills necessary, which challenge many
instructors. Radianti et al. [
] similarly reviewed research on VR in
higher education, showing how VR is used in many dierent elds,
but for teaching engineering and computer science in particular.
Furthermore, VR is used for teaching various types of knowledge,
including procedural, practical, and declarative knowledge [43].
Studies that compare the relative eectiveness of media in pro-
moting educational outcomes can be used to illuminate the advan-
tages of VR in education compared to traditional media. Findings
show that VR is an eective medium for promoting quality expe-
riences, yet with mixed results for objective learning outcomes
compared to traditional media [25, 29, 35].
Comparisons of the learning eects of immersive and non-immersive
learning environments, respectively, show that immersion is asso-
ciated with higher self-ecacy, enjoyment, and interest [
Parong and Mayer [
] compared the eects of administering a
biology lesson in VR or as a slideshow on a PC. They found that
the VR group reported signicantly higher ratings of motivation,
interest, engagement, and aect than the group who used a PC.
The VR group, however, scored worse on a post-test of factual
knowledge [
]. Similarly, Makransky et al. [
] found that learn-
ers gained more knowledge from a lesson when the material was
presented via a PC than via VR. Parong and Mayer [
] showed that
lower retention scores when learning via VR, as opposed to desktop,
is related to extraneous cognitive load. These ndings indicate that
VR may tax the cognitive resources of learners heavily. Careful
attention to the instructional design of educational VR lessons is
therefore necessary.
2.2 Social agency theory
Social agency theory is a frequently used theoretical framework
from multimedia learning that explains the use of pedagogical
agents [
]. According to social agency theory, social cues in mul-
timedia lessons can prime a feeling of social presence in learners,
which leads to deeper cognitive processing and more learning [
Social presence refers to a psychological state in which virtual so-
cial actors are experienced as actual social actors in either sensory
or non-sensory ways [
]. Social agency theory states that people
are attentive to social cues when interacting with computerized
agents, and that these may induce a feeling of interacting with
another social being. This will activate social rules such as the co-
operation principle, meaning that the learner will try to make sense
of the instructional message; consequently, with a social agent, the
learner will make a deeper eort to understand and process the
computerized instructional message [33].
According to Mayer [
], several kinds of social cues can in-
duce a feeling of social presence in the learner. Two such cues are
specically relevant to educational VR: image cues (i.e., displaying a
pedagogical agent who narrates the material) and embodiment cues
(i.e., making the pedagogical agent display human-like behavior
such as gesturing, movement, eye contact, etc.). Taken together, em-
bodied pedagogical agents should, theoretically, give rise to social
presence and, therefore, deeper processing and better learning [
Pedagogical agents have also been criticized for adding unnecessary
complexity to learning environments. According to this view, the
visual presence of a pedagogical agent is merely a seductive detail,
and what really matters is the narration it provides [45].
In reviewing the eect of adding image and embodiment cues to
multimedia lessons, Mayer [
] concluded that there is moderate
evidence that embodiment cues improve learning (
36), but
less support for the eect of adding a speaker’s image to the screen
20). This can be formulated as the image principle: people
do not necessarily learn more deeply from a multimedia lesson
when the speaker’s image is on the screen compared to not on
the screen [
]; and the embodiment principle: people learn more
deeply when pedagogical agents display human-like gesturing,
movement, eye contact, and facial expressions [
]. Importantly,
however, the principles are based on ndings in classic multimedia
learning environments (such as desktop computers), often with non-
humanoid pedagogical agents or humanoid pedagogical agents of
low realism. Consequently, there is a need to revisit the the image
and embodiment principles with 3D and immersive media such
as VR [
]. This is relevant since VR is known to induce feelings
of social presence in learners [
] and thereby potentially deep
cognitive processing. Here follows a description of the current state
of research regarding agents in VR and other media, as well as
related work on social presence in virtual environments.
2.3 Agents in less immersive media
The literature contains a number of relevant HCI studies using
non HMD-based technology (e.g., projection systems or PC) that
can inform the design of pedagogical agents in VR; this section
provides a short description of a select few. Kartiko et al. [
] used
a projection system to test the impact of virtual actors’ visual com-
plexity (i.e., amount of visual information) on science learning, and
Pedagogical Agents in Educational VR CHI ’21, May 8–13, 2021, Yokohama, Japan
Figure 1: Participants in this study were immersed in a virtual museum with an exhibition on the topic of viruses (left). Partici-
pants could freely walk or teleport around the museum that would feature 3D objects relevant to the topic of the exhibition. A
museum tour guide would lead the participant and use white boards with animated presentations to explain the topic. Before
and after experiencing the museum, participants responded to a knowledge test on the subject of the exhibition (right).
found no eect of manipulation with regard to learning outcomes.
Wang et al. [
] explored the impact of dierent virtual agents
presented via augmented reality (AR) on a simple object nding
task. Although there were no eect of manipulation on completion
time, participants gazed more often at human-like agents compared
to non-human. Kim et al. [
] investigated users’ perceptions of
AR agents acting as lab assistants, and reported that participants
displayed most trust and social presence when an agent had a hu-
man body and was capable of speech, gestures, and locomotion.
During a problem-solving task on PC, Groom et al. [
] found that
participants liked a human virtual agent the most when it displayed
inconsistent behavioral realism compared to agents either consis-
tently low or high in behavioral realism (however, scores were
generally low). The same pattern was reported with regard to par-
ticipants’ levels of comfort. Kizilcec et al. [
] compared the impact
of presenting the instructor’s face strategically (i.e., when learners
should focus on spoken text independent of lecture slides) vs. con-
stantly during video instruction. Results indicated that strategic
presentation induced higher social presence relative to constant
presentation but no dierence in achieved course grade. Taking
a more practice-oriented approach, Veletsianos et al. [
] provide
a framework, ‘EnALI’, to enhance agent-learner interactions en-
compassing 15 guidelines, including suggestions regarding agent
characteristics such as designing them to communicate in a polite
and positive manner.
2.4 Pedagogical agents in VR
Only a few studies have been conducted concerning pedagogical
agents in VR and their eect on learning outcomes and experiences
(e.g., [
]). Typically, these do not include a no-agent condition,
making it dicult to be conclusive about the image and embod-
iment principles in VR; specically if the presence of the agent
has an eect on the learned subject. One study investigated the
eect of realism of agents on learning in VR exhibitions [
]. In-
terestingly, participants rated the absence of an agent as higher in
‘humanness’ compared to a realistic agent; the study, however, did
not nd a signicant eect on learning, possibly because of low
power. Makransky et al. [30] designed two pedagogical agents for
VR and used them to teach middle school students about laboratory
safety. A robot-like drone was intended to be more appealing to
boys; a young female scientist was posited to be more inviting to
girls. They demonstrated that boys learned better with the drone,
and that girls learned better with the female scientist. This suggests
that gender-specic design of pedagogical agents could be impor-
tant in educational VR. The general lack of empirical studies on
pedagogical agents in immersive learning environments makes it
dicult to reason about the impact of virtual agents’ appearance
and behavior on learning outcomes.
2.5 Social presence in virtual environments
Social agency theory proposes that social presence during multi-
media learning is a central mechanism that leads to deeper cogni-
tive processing and consequently better learning outcomes. Oh et
al. [
] recently conducted a systematic review of the predictors of
social presence in virtual environments. Their ndings emphasize
the importance of visual representation of virtual communication
partners. Specically, they found that (i) people feel higher levels
of social presence when a visual representation is available rather
than not; (ii) behavioral realism is a powerful predictor of social
presence; (iii) there are mixed results with regard to the eect of
delity and human-likeness on social presence (some studies show
an eect, others none); and (iv) that a ‘consistency eect’ possibly
exists – level of behavioral realism should be consistent with level
of visual delity to maintain high levels of social presence [
When applying these ndings to pedagogical agents and social
CHI ’21, May 8–13, 2021, Yokohama, Japan Petersen et al.
agency theory, there are some similarities. First, there is empiri-
cal evidence for the eect of image cues as people generally feel
higher social presence when the speaker is visible. Second, there is
empirical evidence for the inuence of embodiment cues, as people
generally feel higher social presence when the speaker displays
realistic behavior. Furthermore, Oh et al. [
] show the potential
for delity (i.e., visual realism) and consistency between behavioral
and visual realism in pedagogical agents to be important sources
of social presence and thereby possibly learning.
Based on social agency theory and research on social presence
in virtual environments, four dierent pedagogical agents for the
VR Museum were constructed. These agents varied by a combi-
nation of two levels of visual and behavioral realism, for a total
of four agent conditions. Additionally, a control condition with-
out a pedagogical agent (hence, only narration) was included. The
participants signed up for the experiment online, and installed the
experimental application on their Oculus Quest device. Self-report
inside the VR application collected variables related to learning (i.e.,
knowledge gain and enjoyment), in addition to variables related to
the pedagogical agent (i.e., humanness, attractiveness, and social
presence). Lastly, free text feedback was collected after study com-
pletion in participants’ web browser. See Table 1 for a full list of
the included variables. All procedures performed during the study
were approved by the institutional ethical committee.
3.1 Preregistration: Hypotheses and analyses
We preregistered the experimental study alongside hypotheses,
study plan, and a statistical analyses plan (see
The data collection, study design, and statistical analyses followed
the preregistration, only with a minor deviation, as we, due to an
unexpected high participation interest collected slightly more data
than intended (162 participants instead of 150).
We preregistered the below ve hypotheses. In summary, these
speculate that pedagogical agents, and their realism, cause higher
social presence which will lead to more learning.
Participants in conditions with pedagogical agents of high
visual realism, compared to low visual realism, will report
higher social presence.
Participants in conditions with pedagogical agents of high
behavioral realism, compared to low behavioral realism, will
report higher social presence.
Participants who report higher social presence will have
higher knowledge acquisition.
Learning with a virtual pedagogical agent leads to higher
knowledge acquisition compared to only learning with a
An interaction eect between visual and behavioral real-
ism of the pedagogical agent exists, such that consistency
(high/high or low/low) leads to more learning than inconsis-
tency (high/low or low/high).
3.2 Participants
Following recommendations on conducting unsupervised VR stud-
ies [
], participants were recruited to install our experimental
application onto their own devices, and conduct the study at their
discretion. A total of 162 participants, recruited on social media,
participated in the experiment using their own VR headset over the
course of 11 days. Most of the participants found our advertisement
on Reddit (132), but some found it on Facebook (11) or Twitter
(6). Participants were reimbursed with a gift certicate worth $15
USD (or the equivalent in their preferred currency). All of the
demographics answers are nominal as participants answered the
questions within VR by pointing (see Figure 1, right). Participants
were mostly male (134 male, 24 female, 4 non-binary). Roughly half
of the participants were between 18-29 (88), the rest were: 30-39
(41), 40-49 (22), 50-59 (10), and 60+ (3). Based on IP, participants
were identied to be located in 23 dierent countries, among the
most common: United States (78), United Kingdom (22), Canada
(11), Dominican Republic (11), and Mexico (7). The participants’
educational level ranged from ‘High school or less’ (73), ‘Bachelor’
(57), ‘Master’ (28), and ‘PhD’ (6). The resulting sample is, as the
above shows, rather diverse. However, the majority were from a
cohort with expert VR familiarity. This follows from limiting partic-
ipation to only participants who own an Oculus Quest themselves,
and who have the ability to install custom applications onto their
device. This is also evident from self reports of VR experience, as
the majority of participants had extensive VR experience, having
been immersed more than 50 times (94); the remaining had mostly
some experience (10-50 times,
35), or little experience (1-10
times, 𝑁=29).
One participant was excluded for taking too long, as dened in
the preregistration (
). Eight participants were recorded
with a negative knowledge acquisition; these were kept in the
sample as an exclusion criteria based on learning outcomes was not
established prior to data collection, and because of the relatively
limited amount of participants who did not learn. The analyses
presented are therefore conducted on 161 participants.
tests were conducted to assess the equivalence of conditions
on demographic variables. These were all non-signicant: age (
93), gender (
60), education (
27), and English prociency
92). Hence, the assigned groups did not signicantly dier
based on demography.
3.3 Apparatus
The virtual environment was developed using Unity 2020. The
application was targeted Oculus Quest only. The environment was
an exhibition hall equipped with animated 3D models related to
the topic of viruses (see Figure 1, left). Most of the 3D models were
found on the Unity Asset store. The pedagogical agent was taken
from the Microsoft Rocketbox repository [
]. An American female
voice actor was employed for recording the manuscript. Ambient
museum background sounds were present during the simulation.
3.4 Design
2between subjects design was employed, with an additional
control group for a total of ve conditions. Condition was assigned
randomly at run time on the device. The independent variables
manipulated were (i) behavioral realism, with the levels high and
low and (ii) visual realism, also with the levels high and low. For
high behavioral realism, the pedagogical agent featured gesturing,
Pedagogical Agents in Educational VR CHI ’21, May 8–13, 2021, Yokohama, Japan
Figure 2: Experimental design of the study. High behavioral realism entailed eye contact, gesturing, lip sync, and natural
movements (left). High visual realism entailed rendering the agent as a human (top) rather than in monochrome (bottom).
The control group experienced the simulation without a pedagogical agent (far right).
eye contact, idle animations, speech and lip synchronization, and
movement by walking (see Figure 2 left). Conversely, low behavioral
realism entailed neither of these, and instead featured a static agent
that would move by gliding over the oor (see Figure 2, right).
For high visual realism the pedagogical agent would look like a
human female museum tour guide (see Figure 2, top); for low visual
realism the same humanoid agent employed a black monochrome
mesh (see Figure 2, bottom). The control condition did not feature a
pedagogical agent, yet with the narration intact. This experimental
design allowed an investigation of the importance of both behavior
and appearance of pedagogical agents in virtual learning environ-
ments, also, it made comparisons between having an agent and
no agent possible. This way, the design enabled verication of the
image principle (concerning the appearance vs. absence of pedagog-
ical agents), the embodiment principle (concerning the presence
vs. absence of human-like behavior in pedagogical agents), as well
as potential new principles derived from Oh et al. [
] concern-
ing the eect of high vs. low visual realism in pedagogical agents
and, nally, the eect of consistency vs. inconsistency between
behavioral and visual realism in pedagogical agents. On a broader
level, the experimental design enabled an assessment of the two
opposing views in the eld: that pedagogical agents facilitate vs.
impede learning.
3.5 Dependent measures
Eight variables were measured from a total of 40 questions; two of
these (factual and conceptual knowledge [2]) were objective ques-
tions about the learning topic. Three variables relating to the inter-
action with the pedagogical agent were measured. Two measures,
humanness and attractiveness, were from Ho and MacDorman’s
measures of the Uncanny Valley Eect [
] (‘eeriness’ was omitted
to reduce the length of the within-VR questionnaire). Social pres-
ence [
] was included, which measures the subjective experience
of being present with a ‘real’ person. Additionally, enjoyment and
cognitive load were measured. Validated subjective scales were
employed for all psychological variables. The knowledge questions
were administered, in the same order, both before and after the
study to study pre-to-post changes on learning. The subjective mea-
sures were only administered after the study. All questions were
answered on a virtual screen within the VR application by pointing
CHI ’21, May 8–13, 2021, Yokohama, Japan Petersen et al.
Variable Category Questions Type Min/max Reference
Factual knowledge Learning outcome 10 Multiple choice 0-10 [2]
Conceptual knowledge Learning outcome 10 Multiple choice 0-10 [2]
Perceived enjoyment Experience 3 5-point Likert 1-5 [26]
Perceived humanness Uncanny valley 5 Semantic dierential 1-5 [14]
Attractiveness Uncanny valley 5 Semantic dierential 1-5 [14]
Social presence Pedagogical agent 5 5-point Likert 1-5 [27]
Intrinsic cognitive load Cognitive load 1 5-point Likert 1-5 [5]
Extraneous cognitive load Cognitive load 1 5-point Likert 1-5 [5]
Table 1: Dependent measures used in the study.
and pulling the trigger on the controller (see Figure 1, right). For
each knowledge question four possible answers were provided.
The collected variables were analyzed using analysis of variance
(ANOVA), specically the results presented are computed using the
Rfunction car::Anova.
3.6 Developing the learning material and
outcome test
To underline the potentials of using home VR as a commodity
educational tool, especially during the global health crisis, a virtual
museum exhibition about viruses was chosen. In addition to a
brief introduction to general virology, the exhibition progressed
as a learning tour through three viral diseases: measles, Zika virus
disease, and COVID-19.
The narration that accompanied the exhibition was developed
with inspiration from a national biology teaching repository about
epidemics and pandemics targeted 13-15 year-olds, as well as other
relevant information sources such as the World Health Organiza-
tion. The target group for such simulations is therefore potentially
large. The environment features slides on virtual screens to supple-
ment the narrations with relevant visuals such as a depiction of a
baby suering from microcephaly when learning about Zika virus
To have a direct measure of participants’ knowledge acquisition
as a result of experiencing the simulation, a multiple choice test
was developed, that contained questions about the information pre-
sented during the simulation. The test was developed with experts
in educational psychology and psychometrics, and measured both
factual and conceptual knowledge; that is, cognitive objectives of
recalling and understanding, respectively [
]. The initial version of
the test had a total of 10 questions. To estimate the diculty of the
test, 112 participants were recruited on Amazon Mechanical Turk
(AMT) to conduct the test without any preparation (10 minutes; 1$
pay). The resulting median score was 7 out of 10, with roughly 10%
of the participants answering all questions correctly. As a result of
this test, the number and the diculty of questions were increased.
A next iteration of the test had 20 questions, and was also tested
on AMT. This time, with 75 participants, the median score was
11/20. In this iteration, participants scored between 6 and 15 points,
which attested that it was neither too easy or too dicult. The
nal learning outcome test therefore held 20 questions. It included
ten questions on factual knowledge, such as How many deaths has
measles vaccination prevented?, and ten questions on conceptual
knowledge, such as How do vaccines help the body develop immunity
to diseases?.
3.7 Procedure
For an overview of the study procedure, please consult Figure 3.
Participants signed up to our study using an online survey. Upon
giving informed consent to data collection and study participation,
a brief guide to installing our experimental application followed.
The open app store SideQuest
was utilized for the purpose of
easing the installation burden. After completing the installation
Install VR
Sign up
Confirm ID
Pre test Intro-
COVID-19 Post test
ID code
Figure 3: Visual overview of the study procedure. Yellow boxes denote PC-based user activity, gray are within-VR question-
naires, blue are core learning material, and the red shows meta activity. The entire procedure took about one hour to complete.
Pedagogical Agents in Educational VR CHI ’21, May 8–13, 2021, Yokohama, Japan
instructions, participants were instructed to launch the application,
take the headset on, and follow within-VR guidelines. As such,
guidelines were not provided before immersion. It was not possible
to skip parts of the experience.
The virtual environment began with a brief introduction of the
controls, the purpose, and the content. This was provided both
in audio and on information displays blended into the museum
environment. Participants could freely walk around or teleport
themselves by pointing and clicking ‘A’ on the controller. The exhi-
bition progressed as the participant followed the tour guide around
to the dierent displays constituting the core learning material. A
knowledge test was conducted, before beginning the guided tour.
A general introduction to the topic of virology was followed by
presentations of three viruses: Measles, Zika, and COVID-19. After
completing the museum tour, participants conducted an identical
knowledge test, in addition to a survey about subjective measures
and demographics. Upon completion, a unique code emerged on
a screen; this code had to be entered into the online form where
participants signed up to ensure valid participation (and to qualify
for reimbursement).
The mean duration of the immersion was 20.0 minutes (
𝑆𝐷 =
Here, quantitative ndings are reported. They relate to the knowl-
edge acquisition and the eect of manipulations on subjective and
objective measures. A visual inspection of the collected variables
showed that data followed normal distributions (e.g., see Figure 4).
The analyses are therefore based on parametric tests. Furthermore,
the study collected subjective measures from scales that have pre-
viously been validated for parametric testing.
4.1 How much did they learn?
See Table 2 for an overview of dierences between pre- and post
scores. Out of a combined maximum of 20 points, the mean pre-
score was 11.0 (
𝑆𝐷 =
8). The mean pre-score for factual knowl-
edge was 3.5 (
𝑆𝐷 =
5); it was 7.6 (
𝑆𝐷 =
for conceptual
knowledge. For the post test, the combined mean score was 15.1
𝑆𝐷 =
8). The mean post score for factual knowledge was 6.2
𝑆𝐷 =
8); for conceptual knowledge the mean was 8.9 (
𝑆𝐷 =
That shows that participants, on average, increased their tests scores
with 4.0 points (
𝑆𝐷 =
9) after experiencing the virtual exhibition
(see Figure 4); this dierence was also signicant, shown with a
repeated measures ANOVA: 𝐹(1,160)=305.9, 𝑝 <.0001, 𝑑 =1.4.
Factual Conceptual
Pre test, M (SD) 3.5 (1.5) 7.6 (2.1)
Post test, M (SD) 6.2 (1.8) 8.9 (1.5)
Dierence +2.7 +1.3
Welch’s t-test, p <.0001 <.0001
Cohen’s d1.6 0.7
Table 2: Scores obtained in the pre- and post tests, respec-
0 5 10 15 20
Test score
Post score
Pre score
Figure 4: Pre- to post test scores: participants performed bet-
ter on the knowledge tests after experiencing the virtual ex-
4.2 The eect of agent on subjective measures
Figure 5 shows the mean reported scores on the subjective measures
relating to interaction with the agent, divided by experimental
For humanness (Figure 5, A), the absence of a pedagogical agent
yields comparable humanness to an agent with high behavioral
realism. Also, high behavioral realism resulted in higher humanness
compared to low behavioral realism;
, 𝑝 =.
005. In
other words, behavior, but not appearance, of the virtual agent
aected whether participants experienced the agent as human.
For attractiveness (Figure 5, B), the experimental manipulations
of agent had less of an impact, and the eect of manipulation was
not signicant.
The high behavioral realism conditions showed signicantly
higher social presence (Figure 5, C);
, 𝑝 =.
02. A
comparable dierence for appearance was not found.
In summary, our ndings suggest that behavior of pedagogical
agents (gesturing, eye contact, natural movements) impact sub-
jective social accounts of the agent, while appearance to a lesser
degree does. It should be noted that the absence of an agent results
in comparable, or even higher, reports of humanness and attractive-
ness (but not social presence). This counter intuitive nding, that
no agent leads to high reports of attractiveness and humanness,
was also reported by Rzayev et al. [44].
4.3 The eect of agent on learning
Figure 6 shows the mean dierence between pre- and post-test
knowledge scores for each condition. Two learning outcomes were
measured; factual and conceptual knowledge gain [
]. Factual knowl-
edge relates to recalling (e.g., numbers, places, years) while concep-
tual knowledge relates to understanding (e.g., explaining, connect-
ing, transferring).
Both the appearance and behavior of a pedagogical agent had an ef-
fect on factual knowledge gain (see Figure 6, A). One-way ANOVAs
showed signicant eects:
5; 3
], 𝑝 =[.
Post hoc Tukey’s HSDs showed that high visual realism as well
as high behavioral realism signicantly diered with the control
condition. This shows that, for learning facts, the presence of a
pedagogical agent is not ideal; rather, the agent, and its visual and
behavioral delity impede factual retention.
CHI ’21, May 8–13, 2021, Yokohama, Japan Petersen et al.
Control Low appearance High appearance
Mean humanness
Control Low appearance High appearance
Mean attractiveness
Control Low appearance High appearance
Mean social presence
Control High behavior Low behavior
Figure 5: Means of subjective measures by experimental
manipulation: humanness (A), attractiveness (B), and social
presence (C). Error bars show 95% condence intervals.
For conceptual knowledge, a two-way ANOVA showed a sig-
nicant interaction eect between appearance and behavior (see
Figure 6, B);
, 𝑝 =.
003. Oh et al. [
] reported a
‘consistency eect’; they state that, for learning, consistency of an
agent is preferred (i.e., that delity of behavioral and visual realism
ideally match). Our ndings contradict this nding, as we observe a
higher conceptual knowledge gain when behavior and appearance
are incongruent.
4.4 Enjoyment
Participants generally reported high enjoyment rates for the learn-
ing experience, with a mean score of
(𝑆𝐷 =
of a
maximum 5. These enjoyment rates were consistent across exper-
imental manipulations, see Figure 7. Together with the general
positive knowledge acquisition, this tells us, that the virtual mu-
seum was received positively as a new form of remote learning
during the global health crisis. This nding is consistent with previ-
ous research that associated VR with higher enjoyment compared
to less immersive media [24].
Control High behavior Low behavior
Figure 6: Mean dierences between pre- and post test
scores. For factual knowledge (A), absence of a pedagogi-
cal agent leads to higher pre- to post scores. For concep-
tual learning (B), we nd a signicant interaction eect be-
tween behavior and appearance, namely that incongruence
(low/high or low/high) leads to better learning than congru-
ence (high/high or low/low). Error bars show 95% CI.
4.5 Cognitive load
As previous ndings in educational VR suggest that specic in-
structional designs in VR learning environments lead to increased
cognitive load [
], a subjective measure for both intrinsic (di-
culty of subject) and extraneous (diculty of instruction) cognitive
load [5] were collected.
Participants reported comparable intrinsic and extraneous cogni-
tive load for all conditions. As such, medians for all ve conditions
for both cognitive load measures were 2 out of 5. There were no
signicant dierences between conditions. It should be noted that
cognitive load was assessed via single items. Use of full scales could
have provided further insights (e.g. [1]).
Pedagogical Agents in Educational VR CHI ’21, May 8–13, 2021, Yokohama, Japan
Control Low appearance High appearance
Mean enjoyment
Control High behavior Low behavior
Figure 7: Means of enjoyment ratings: participants consis-
tently experienced the virtual exhibition as enjoyable. Error
bars show 95% condence intervals.
4.6 Brief Summary of hypotheses and ndings
We preregistered ve hypotheses related to the realism of peda-
gogical agents, their eect on social presence, and in turn learning.
We hypothesized that high visual realism would lead to high social
presence (H1); that high behavioral realism would lead to high
social presence (H2); that high social presence would lead to better
learning (H3); that agents would be better than no agents for learn-
ing (H4); and that there would be an interaction eect between
visual and behavioral realism, where consistency would be better
than inconsistency for learning (H5). In relation to our ndings,
specically, only H2 was conrmed, namely that behavior of an
avatar signicantly impacts social presence. On the contrary, we did
not nd support for H1; that is, that appearance of an agent impacts
social presence. We did not nd a signicant correlation between
social presence and factual learning (rather slightly negative, Pear-
, 𝑝 =.
15). Yet, for conceptual learning we do nd
it signicantly correlated to social presence (
, 𝑝 =.
Consequently, H3 has a more nuanced answer. Importantly, for H4,
we nd an opposite eect for factual learning. For conceptual learn-
ing we did nd an interaction eect, but rather the reverse than
hypothesized in H5. Consequently we nd partial support for the
opposite eect for H4 (for factual learning) and H5 (for conceptual
We conducted a VR experiment ‘in the wild’ during the COVID-19
pandemic. In general, our ndings show that unsupervised, remote
learning in VR is feasible as all participants enjoyed the experience
and improved on a knowledge test. Furthermore, we show that the
design of pedagogical agents in educational VR impacts learning
depending on the type of learning considered. Including a pedagog-
ical agent leads to lower factual knowledge acquisition compared
to only including a narration. Looking at conceptual information
acquisition, a pedagogical agent may aid learning. These ndings
expand classical multimedia learning theory in the context of VR.
Our nding that pedagogical agents are useful for learning about
concepts but not facts could be explained by the diering nature
of the two types of information in combination with the human
capacity for selectively attending to certain stimuli [
]. During the
lesson, factual information, such as specic dates and numbers,
was presented very quickly and therefore imposed large demands
on attention at specic moments in time. In contrast, much of the
conceptual information had broader explanations and therefore al-
lowed short diversions in attention. Thus, the addition of a detailed
agent would specically have a negative eect on factual learning
by stealing attention at critical moments. Although we did not use
eye-tracking, other research corroborates people’s tendency to gaze
at human agents [
]. This echoes the issue raised by Veletsianos et
al. [
] that pedagogical agents may be mesmerizing and misdirect
attention from the task.
Our participants were expert VR users, hence a novelty eect
most likely did not interfere with the results. The novelty eect
refers to a heightened motivation to use something simply on ac-
count of its newness [
]. Novelty may confound the results of
media studies as the increased eort and attention could result in
achievement gains that would not occur if the learner was familiar
with the medium [
]. In that light, and in combination with the
relatively high participation count, our results are a reliable source
of evidence on the ecacy of pedagogical agents in immersive
learning environments.
5.1 Theoretical implications
The ndings of this study has a number of implications for social
agency theory and its derived learning principles in the context of
educational VR.
5.1.1 The image principle. The image principle suggests that using
visible pedagogical agents has a small eect on learning compared
to only using narration. As some previous studies found negligi-
ble or even negative eects of presenting an image of the speaker,
however, it led Mayer to conclude that people do not necessar-
ily learn more from lessons when the speaker’s image is on the
screen compared to when it is not [
]. Importantly, the theory is
based on relatively dated empirical evidence, some of which date
back 20 years ago, where learning environments and pedagogical
agents were less sophisticated. We examined the image principle
with highly sophisticated pedagogical agents rendered using state
of the art consumer VR technology. Our ndings show that for
factual learning, including a pedagogical agent of high visual or
behavioral realism leads to less learning compared to using an ‘in-
visible’ speaker. Realistically looking agents presumably distract
the learner, yet, we did not record a change in subjectively mea-
sured cognitive load. In contrast, learning about concepts was not
hampered by the inclusion of pedagogical agents.
To sum up, our ndings suggest a renement of the image princi-
ple when applied to educational VR. When compared to only a nar-
ration, pedagogical agents do not lead to higher factual knowledge
gain. On the contrary, realistic pedagogical agents may actually
hamper learning of factual information. This was not the case when
learning about conceptual information.
5.1.2 The embodiment principle. The embodiment principle fo-
cuses on the absence vs. presence of behavioral cues in pedagogical
agents. Previous research points to the benets of embodiment,
which led Mayer to conclude that when pedagogical agents display
CHI ’21, May 8–13, 2021, Yokohama, Japan Petersen et al.
human-like gesturing, movement, facial expressions, etc. as op-
posed to appearing static, it leads to better learning. In the present
study we did not nd a positive eect of behavioral realism on
factual knowledge gain. In fact, participants who learned from
behaviorally realistic pedagogical agents (i.e., agents exhibiting
gesturing, eye contact, speech and lip synchronization) increased
their scores slightly less on the factual knowledge test than partici-
pants who learned from static pedagogical agents (although this
warrants further investigation). Consequently, our ndings do not
corroborate the embodiment principle when learning about facts.
When learning about concepts, however, our ndings indicate that
the embodiment principle exists if the agents are of low but not
high visual realism. In the latter scenario, a reversed embodiment
principle is actually found. This is an important addition to the
embodiment principle in educational VR.
5.1.3 Visual realism and consistency. Oh et al. [
] reviewed deter-
minants of social presence in virtual environments and found that
agents’ visual realism and consistency in appearance and behavior
could be important sources of social presence. We built on these
ndings and tested if visual realism and consistency inuenced
learning of factual and conceptual information. Our results indicate
that visual realism impacts factual learning negatively. One possible
explanation for this could be the uncanny valley eect, which refers
to the relation between the human-likeness of an entity and the per-
ceiver’s anity for it [
]. The theory posits that anity increases
as a function of the human-likeness of articial humans until it
reaches a valley where anity suddenly drops. This corresponds to
a point where there is a relatively high degree of human-likeness in
an entity combined with evidence that it is articial, and this is ac-
companied by a creepy sensation [
]. The visually realistic agents
might have caused a creepy sensation in the participants, lowering
their motivation to understand the learning material. This would be
consistent with the feedback reported by some of the participants
learning with visually realistic agents: “You could replace the guide
with a robot. A clearly non-human robot guide wouldn’t be so o
putting [sic]” or “the guide looks creepy”. In terms of consistency, our
results, again, indicate an eect in the opposite direction of what
we had hypothesized. There was a signicant interaction eect
between visual and behavioral realism for conceptual knowledge
gain; knowledge gain was higher when these were incongruent.
5.1.4 Social presence as a learning mechanism. Social presence
during multimedia learning is posited to be an important construct
that leads to better learning outcomes [
]. Consistent with social
agency theory, we found that behavioral realism had a signicant
eect on social presence. However, an increase in subjective social
presence was not unequivocally associated with more learning.
This shows that social presence does not necessarily lead to more
learning, and that other constructs than those provided by social
agency theory are important for learning with virtual agents.
5.2 Limitations and future research directions
Pedagogical agents can take on a variety of instructor roles dur-
ing learning [
]. Similar to Baylor and Kim [
], the agents in the
present study performed the role of an expert as their primary
function was to provide accurate and concise information. How-
ever, agents can also inhabit other roles, such as motivators whose
primary function is to provide encouragement [
]. Future research
should investigate learning with pedagogical agents inhabiting
dierent roles.
The participants in this study primarily consisted of expert VR
users. This was an advantage in terms of limiting novelty eects, but
future research should investigate the ecacy of remote learning
with pedagogical agents in a non-expert sample.
We investigated factual and conceptual learning as pre- to post-
test changes on a multiple choice question test administered in
VR before and after the lesson. However, we did not investigate
transfer of learning (i.e., when learning in one context improves
performance in another context [
]). Transfer is a key goal in edu-
cation, and future research should therefore consider investigating
transfer eects of learning with agents in VR.
Another limitation concerns the relevancy of the pedagogical
agents and their behavior in terms of teaching the content of the
VR simulation. For instance, it could be argued that an agent would
be more relevant for teaching specic procedures compared to
teaching facts, as its behavior could then be tied more easily to the
learning material (e.g., if demonstrating how to perform a certain
procedure). Theoretically, this would map onto classical psycho-
logical theories of model learning as put forward by scholars such
as Albert Bandura. Future research could therefore examine pro-
cedural learning from pedagogical agents in VR. One promising
direction could be to examine whether watching an agent perform
a procedure produces stronger procedural learning compared to
hearing about it.
While out-of-lab VR experimentation allows for a larger and
more heterogeneous sample which gives higher statistical power
and ecological ndings at a lower cost [
], it comes with the cost of
reduced internal validity. Specically, we cannot control the size and
constraints of the participants’ surroundings or a strict adherence
to the study protocol. Qualitative ndings are also hard to collect
using this paradigm. Some of the aspects discussed in this paper,
concerning for instance creepiness of agents, might be benecial
to investigate further using traditional laboratory protocols.
Based on inquiry by some of the participants in regards to invit-
ing lock downed family members and friends to participate in the
study, we allowed multiple participation from the same IP (a total of
17 recurrent IP addresses were recorded). This resulted in a slightly
more diverse participant pool than otherwise expected. We note
that this makes it technically possible for a single participant to
complete the study multiple times, we however, do not suspect this
have inuenced the reported data.
Furthermore we collected free form text answers as part of the de-
brieng by asking for any feedback. Although many wrote lengthy
comments of mostly feature requests, the focus of the study was
quantitative, and an extensive analysis of qualitative data was there-
fore neither signicant nor the scope. If the aim is qualitative, a
recommendation for future online user studies is therefore to be
specic about any desired type of qualitative data and to formulate
open-ended questions accordingly.
The general low number of non-males in the study was a lim-
itation along with the fact that we did not manipulate the sex of
Pedagogical Agents in Educational VR CHI ’21, May 8–13, 2021, Yokohama, Japan
the agent. Participants’ gender did not reveal considerable learning
dierences at post-test: M 15.4, 15.0, and 16.0 (F/M/X).
Remote learning in the connes of your own home through VR
is an enjoyable and eective alternative to the real-life classroom,
especially during global health crises. Caution should be taken,
however, when it comes to incorporating a pedagogical agent into
the virtual learning experience. Our ndings expand upon classical
multimedia learning theory in the context of VR, and provide new
insights into the scholarly discussion about whether pedagogical
agents are facilitators of learning or merely unnecessary distrac-
tions; our ndings suggest that the answer depends on the type of
knowledge in question. Behavior of a pedagogical agent did in fact
increase social presence and humanness, yet the eects on learning
were mixed. If the material to be learned is factual, a pedagogical
agent may impede learning. Contrary, if the material to be learned
is conceptual, a pedagogical agent may be worthwhile.
We would like to thank Martin Kampmann for designing and de-
veloping the virtual environment used in the study. We would also
like to thank all the people who participated in the experiment.
Martin S. Andersen and Guido Makransky. 2021. The validation and further
development of a multidimensional cognitive load scale for virtual environments.
Journal of Computer Assisted Learning 37, 1 (2021), 183–196.
Lorin W. Anderson, David R. Krathwohl, Peter W. Airasian, Kathleen A. Cruik-
shank, Richard E. Mayer, Paul R. Pintrich, James Raths, and Merlin C. Wittrock.
2001. A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s
Taxonomy of Educational Objectives. Longman.
Sarune Baceviciute, Aske Mottelson, Thomas Terkildsen, and Guido Makransky.
2020. Investigating Representation of Text and Audio in Educational VR Using
Learning Outcomes and EEG. In Proceedings of the 2020 CHI Conference on Human
Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for
Computing Machinery, New York, NY, USA, 1–13.
Amy L. Baylor and Yanghee Kim. 2004. Pedagogical agent design: The impact of
agent realism, gender, ethnicity, and instructional role. In International conference
on intelligent tutoring systems. Springer, 592–603.
Gabriele Cierniak, Katharina Scheiter, and Peter Gerjets. 2009. Explaining the
split-attention eect: Is the reduction of extraneous cognitive load accompanied
by an increase in germane cognitive load? Computers in Human Behavior 25, 2
(2009), 315–324.
Richard E. Clark and Terrance G. Craig. 1992. Research and theory on multi-
media learning eects. In Interactive multimedia learning environments. Springer,
Joseph Crawford, Kerryn Butler-Henderson, Jürgen Rudolph, Matthias Glowatz,
Rob Burton, Paola A. Magni, and Sophia Lam. 2020. COVID-19: 20 Countries’
Higher Education Intra-Period Digital Pedagogy Responses. Journal of Applied
Teaching and Learning (JALT) 3, 1 (2020).
Peter Dayan, Sham Kakade, and P. Read Montague. 2000. Learning and selective
attention. Nature neuroscience 3, 11 (2000), 1218–1223.
Jesse Fox, Sun Joo (Grace) Ahn, Joris H. Janssen, Leo Yeykelis, Kathryn Y. Segovia,
and Jeremy N. Bailenson. 2015. Avatars Versus Agents: A Meta-Analysis Quanti-
fying the Eect of Agency on Social Inuence. Human–Computer Interaction 30,
5 (2015), 401–432.
Mar Gonzalez-Franco, Eyal Ofek, Ye Pan, Angus Antley, Anthony Steed, Bernhard
Spanlang, Antonella Maselli, Domna Banakou, Nuria Pelechano, Sergio Orts
Escolano, Veronica Orvahlo, Laura Trutoiu, Markus Wojcik, Maria V. Sanchez-
Vives, Jeremy Bailenson, Mel Slater, and Jaron Lanier. 2020. Importance of
rigging for procedural avatars. Microsoft Rocketbox a public library. https:
// [Online; accessed April 1, 2020].
Charles R. Graham. 2006. Blended learning systems: Denition, current trends, and
future directions. John Wiley & Sons.
Victoria Groom, Cliord Nass, Tina Chen, Alexia Nielsen, James K. Scarborough,
and Erica Robles. 2009. Evaluating the eects of behavioral realism in embodied
agents. International Journal of Human-Computer Studies 67, 10 (2009), 842–849.
Ste Heidig and Geraldine Clarebout. 2011. Do pedagogical agents make a
dierence to student motivation and learning? Educational Research Review 6, 1
(2011), 27–54.
Chin-Chang Ho and Karl MacDorman. 2017. Measuring the Uncanny Valley
Eect. International Journal of Social Robotics 9, 1 (2017), 129–139. https://doi.
org/10.1007/s12369-016- 0380-9
Lasse Jensen and Flemming Konradsen. 2018. A review of the use of virtual reality
head-mounted displays in education and training. Education and Information
Technologies 23, 4 (2018), 1515–1529. 9676-0
Iwan Kartiko, Manolya Kavakli, and Ken Cheng. 2010. Learning science in a vir-
tual reality application: The impacts of animated-virtual actors visual complexity.
Computers & Education 55, 2 (2010), 881–891.
Kangsoo Kim, Luke Boelling, Steen Haesler, Jeremy N. Bailenson, Gerd Bruder,
and Gregory F. Welch. 2018. Does a Digital Assistant Need a Body? The Inuence
of Visual Embodiment and Social Behavior on the Perception of Intelligent Virtual
Agents in AR. In 2018 IEEE International Symposium on Mixed and Augmented
Reality (ISMAR). 105–114.
Paul A. Kirschner. 2002. Cognitive load theory: Implications of cognitive load
theory on the design of learning. Learning and Instruction 12, 1 (2002), 1–10. 7
René F. Kizilcec, Jeremy N. Bailenson, and Charles J. Gomez. 2015. The in-
structor’s face in video instruction: Evidence from two large-scale eld studies.
Journal of Educational Psychology 107, 3 (2015), 724–739.
Sara Klingenberg, Maria L. M. Jørgensen, Gert Dandanell, Karen Skriver,
Aske Mottelson , and Guido Makransky. 2020. Investigating the eect of teaching
as a generative learning strategy when learning through desktop and immersive
VR: A media and methods experiment. British Journal of Educational Technology
51, 6 (2020), 2115–2138.
Michael Koch, Kai von Luck, Jan Schwarzer, and Susanne Draheim. 2018. The
Novelty Eect in Large Display Deployments - Experiences and Lessons-Learned
for Evaluating Prototypes. In ECSCW 2018 Exploratory Papers. European Society
for Socially Embedded Technologies (EUSSET).
Kwan Min Lee. 2006. Presence, Explicated. Communication Theory 14, 1 (2006),
Xiao Ma, Megan Cackett, Leslie Park, Eric Chien, and Mor Naaman. 2018. Web-
Based VR Experiments Powered by the Crowd. In Proceedings of the 2018 World
Wide Web Conference (Lyon, France) (WWW ’18). International World Wide Web
Conferences Steering Committee, Republic and Canton of Geneva, CHE, 33–43.
Guido Makransky, Niels Koch Andreasen, Sarune Baceviciute, and Richard E.
Mayer. 2020. Immersive Virtual Reality Increases Liking but Not Learning with
a Science Simulation and Generative Learning Strategies Promote Learning in
Immersive Virtual Reality. Journal of Educational Psychology (2020). https:
Guido Makransky, Stefan Borre-Gude, and Richard E. Mayer. 2019. Motivational
and cognitive benets of training in immersive virtual reality based on multiple
assessments. Journal of Computer Assisted Learning 35, 6 (2019), 691–707. https:
Guido Makransky and Lau Lilleholt. 2018. A Structural Equation Modeling
Investigation of the Emotional Value of Immersive Virtual Reality in Education.
Educational Technology Research and Development (2018), 1141–1164. https:
// 9581-2
Guido Makransky, Lau Lilleholt, and Anders Aaby. 2017. Development and
validation of the Multimodal Presence Scale for virtual reality environments: A
conrmatory factor analysis and item response theory approach. Computers in
Human Behavior 72 (2017), 276–285.
Guido Makransky, Gustav B. Petersen, and Sara Klingenberg. 2020. Can an
immersive virtual reality simulation increase students’ interest and career aspira-
tions in science? British Journal of Educational Technology 51, 6 (2020), 2079–2097.
Guido Makransky, Thomas S. Terkildsen, and Richard E. Mayer. 2019. Adding
immersive virtual reality to a science lab simulation causes more presence but
less learning. Learning and Instruction 60 (2019), 225 – 236.
Guido Makransky, Philip Wismer,and Richard E. Mayer. 2019. A gender matching
eect in learning with pedagogical agents in an immersive virtual reality science
simulation. Journal of Computer Assisted Learning 35, 3 (2019), 349–358. https:
David M. Markowitz, Rob Laha, Brian P. Perone, Roy D. Pea, and Jeremy N.
Bailenson. 2018. Immersive Virtual Reality Field Trips Facilitate Learning About
Climate Change. Frontiers in Psychology 9, 2364 (2018), 1–20.
CHI ’21, May 8–13, 2021, Yokohama, Japan Petersen et al.
Ati Suci Dian Martha and Harry B. Santoso. 2019. The Design and Impact of the
Pedagogical Agent: A Systematic Literature Review. Journal of Educators Online
16, 1 (2019).
Richard E. Mayer. 2014. Principles based on social cues in multimedia learning:
Personalization, voice, image, and embodiment principles. In The Cambridge
handbook of multimedia learning. Cambridge University Press, 345–370.
Richard E. Mayer. 2019. Thirty years of research on online learning. Applied
Cognitive Psychology 33, 2 (2019), 152–159.
Oliver A. Meyer, Magnus K. Omdahl, and Guido Makransky. 2019. Investigating
the eect of pre-training when learning through immersive virtual reality and
video: A media and methods experiment. Computers & Education 140 (2019).
Masahiro Mori, Karl F. MacDorman, and Norri Kageki. 2012. The Uncanny
Valley [From the Field]. IEEE Robotics Automation Magazine 19, 2 (2012), 98–100.
Aske Mottelson and Kasper Hornbæk. 2017. Virtual Reality Studies Outside the
Laboratory (VRST ’17). Association for Computing Machinery, New York, NY,
USA, 1–10.
Catherine S. Oh, Jeremy N. Bailenson, and Gregory F. Welch. 2018. A Systematic
Review of Social Presence: Denition, Antecedents, and Implications. Frontiers
in Robotics and AI 5, 114 (2018), 1–35.
Jocelyn Parong and Richard E. Mayer. 2018. Learning science in immersive
virtual reality. Journal of Educational Psychology 110, 6 (2018), 785–797. https:
Jocelyn Parong and Richard E. Mayer. 2021. Cognitive and aective processes
for learning science in immersive virtual reality. Journal of Computer Assisted
Learning 37, 1 (2021), 226–241.
David N. Perkins and Gavriel Salomon. 1992. Transfer of learning. In International
encyclopedia of education. Pergamon Press.
[42] Gustav B. Petersen, Sara Klingenberg, Richard E. Mayer, and Guido Makransky.
2020. The virtual eld trip: Investigating how to optimize immersive virtual
learning in climate change education. British Journal of Educational Technology
51, 6 (2020), 2098–2114.
Jaziar Radianti, Tim A. Majchrzak, Jennifer Fromm, and Isabell Wohlgenannt.
2020. A systematic review of immersive virtual reality applications for higher
education: Design elements, lessons learned, and research agenda. Computers &
Education 147 (2020).
Rufat Rzayev, Gürkan Karaman, Katrin Wolf, Niels Henze, and Valentin Schwind.
2019. The Eect of Presence and Appearance of Guides in Virtual Reality Ex-
hibitions. In Proceedings of Mensch Und Computer 2019 (Hamburg, Germany)
(MuC’19). Association for Computing Machinery, New York, NY, USA, 11–20.
Noah L. Schroeder, Olusola O. Adesope, and Rachel Barouch Gilbert. 2013. How
Eective are Pedagogical Agents for Learning? A Meta-Analytic Review. Journal
of Educational Computing Research 49, 1 (2013), 1–39.
Mel Slater. 2009. Place Illusion and Plausibility Can Lead to Realistic Behaviour in
Immersive Virtual Environments. Philosophical transactions of the Royal Society
B364 (2009), 3549–3557.
Anthony Steed, Sebastian Frlston, Maria Murcia Lopez, Jason Drummond, Ye Pan,
and David Swapp. 2016. An In the Wild Experiment on Presence and Embodiment
using Consumer Virtual Reality Equipment. IEEE Transactions on Visualization &
Computer Graphics 22, 04 (2016), 1406–1414.
George Veletsianos, Charles Miller, and Aaron Doering. 2009. Enali: A Research
and Design Framework for Virtual Characters and Pedagogical Agents. Journal
of Educational Computing Research 41, 2 (2009), 171–194.
George Veletsianosand Gregory S. Russell. 2014. Pedagogical agents. In Handbook
of research on educational communications and technology. Springer, 759–769.
Isaac Wang, Jesse Smith, and Jaime Ruiz. 2019. Exploring Virtual Agents for
Augmented Reality. In Proceedings of the 2019 CHI Conference on Human Factors in
Computing Systems (Glasgow, Scotland UK) (CHI ’19). Association for Computing
Machinery, New York, NY, USA, 1–12.
... Although there are theoretical grounds [58] showing that collaborative learning can reduce cognitive load and yield better learning outcomes, previous reviews [92,111] have also argued that classic VR video viewing systems struggle to meet students' needs for social interaction and collaboration. To build on these prior studies and work towards addressing these challenges, we chose to investigate the role of collaborative tools and shared control systems for VR collaborative video viewing , and selected to measure considering both the knowledge acquisition and factors highly relative to learning and collaboration: collaboration (e.g., [21,112]), social presence (e.g., [17,48,88,91]), cognitive load (e.g., [23,88]), and satisfaction (e.g., [14,112]). Section 4.4.2 ...
... Although there are theoretical grounds [58] showing that collaborative learning can reduce cognitive load and yield better learning outcomes, previous reviews [92,111] have also argued that classic VR video viewing systems struggle to meet students' needs for social interaction and collaboration. To build on these prior studies and work towards addressing these challenges, we chose to investigate the role of collaborative tools and shared control systems for VR collaborative video viewing , and selected to measure considering both the knowledge acquisition and factors highly relative to learning and collaboration: collaboration (e.g., [21,112]), social presence (e.g., [17,48,88,91]), cognitive load (e.g., [23,88]), and satisfaction (e.g., [14,112]). Section 4.4.2 ...
... In other words, these avatars might not have been useful enough for people to attend to. Prior studies noted the realism of an avatar could influence social interaction quality [88,101]. Thus, future researchers could explore participants' visual preferences and priority on collaborative VR video viewing systems (e.g., using eye-tracking technique [16,98]) to better guide this area. ...
Full-text available
Virtual Reality (VR) has a noteworthy educational potential by providing immersive and collaborative environments. As an alternative but cost-effective way of delivering realistic environments in VR, using 360-degree videos in immersive VR (VR videos) received more attention. Although many studies reported positive learning experiences with VR videos, little is known about how collaborative learning performs on VR video viewing systems. In this study, we implemented two collaborative VR video viewing modes based on the way of group video control, synchronized or shared (Sync mode) and non-synchronized or individual (Non-sync mode) video control, against a conventional VR video viewing setting (Basic mode). We conducted a within-subject study (N = 54) in a lab-simulated remote learning environment. Our results show that collaborative VR video modes (Sync and Non-sync mode) improve users' learning experiences and collaboration quality, especially with shared video control. Our findings provide directions for designing and employing collaborative VR video tools in online learning environments.
... We found five papers that involved tasks related to learning activities. Three evaluated agents [46,49,64], one evaluated the avatar of another person [15], and one evaluated both a self-avatar and an agent. [39] These papers include activities in VR such as a guided tour through a museum exhibition [46,49], a virtual field trip [64], a recall and object finding task (spatial learning) [15], and giving lab instructions and explaining theoretical concepts to students [39]. ...
... Three evaluated agents [46,49,64], one evaluated the avatar of another person [15], and one evaluated both a self-avatar and an agent. [39] These papers include activities in VR such as a guided tour through a museum exhibition [46,49], a virtual field trip [64], a recall and object finding task (spatial learning) [15], and giving lab instructions and explaining theoretical concepts to students [39]. The main goal of these papers was to investigate the instructor's representation [15,39,46,49,64] in learning activities in VR. ...
... [39] These papers include activities in VR such as a guided tour through a museum exhibition [46,49], a virtual field trip [64], a recall and object finding task (spatial learning) [15], and giving lab instructions and explaining theoretical concepts to students [39]. The main goal of these papers was to investigate the instructor's representation [15,39,46,49,64] in learning activities in VR. Regarding appearance, upperbody [15] or full-body [39,46,49,64] were used. ...
Full-text available
Augmented Reality (AR) and Virtual Reality (VR) are pushing from the labs towards consumers, especially with social applications. These applications require visual representations of humans and intelligent entities. However, displaying and animating photo-realistic models comes with a high technical cost while low-fidelity representations may evoke eeriness and overall could degrade an experience. Thus, it is important to carefully select what kind of avatar to display. This article investigates the effects of rendering style and visible body parts in AR and VR by adopting a systematic literature review. We analyzed 72 papers that compare various avatar representations. Our analysis includes an outline of the research published between 2015 and 2022 on the topic of avatars and agents in AR and VR displayed using head-mounted displays, covering aspects like visible body parts (e.g., hands only, hands and head, full-body) and rendering style (e.g., abstract, cartoon, realistic); an overview of collected objective and subjective measures (e.g., task performance, presence, user experience, body ownership); and a classification of tasks where avatars and agents were used into task domains (physical activity, hand interaction, communication, game-like scenarios, and education/training). We discuss and synthesize our results within the context of today's AR and VR ecosystem, provide guidelines for practitioners, and finally identify and present promising research opportunities to encourage future research of avatars and agents in AR/VR environments.
... Therefore, in the following section, we summarize research that focuses on the effects of clothing of both humans (e.g. Johnson et al., 2014;Beege et al., 2019) and pedagogical agents (Schmidt et al., 2019;Petersen et al., 2021). ...
... In contrast, in a museum context, Schmidt et al. (2019) found positive effects on learning and presence, if an agent was represented as a museum guide rather than an astronaut. However, Petersen et al. (2021) observed heterogeneous results regarding the effects of the thematically appropriate agent design on learning in a virtual museum. Comparing an agent dressed as museum guide with the same agent displaying only black mesh and a control group without an agent, they found that all the investigated pedagogical agents seemed to hinder the acquisition of factual knowledge. ...
Full-text available
Pedagogical agents are often used to enhance social cues in learning materials. The inclusion of pedagogical agents raises several design questions, for example on what kind of clothing the agent should wear. Further, it is not yet clear how the setting of an animated learning video (i.e., the digitally created background) affects learning. In an online experiment (N = 200), we investigated whether creating thematically appropriate clothing and setting has some added value in that it improves learning outcomes in comparison to more neutral assets. Whereas all participants acquired knowledge from the animated video, there were no main effects of clothing and setting for any of the dependent variables, but an interaction for learning outcomes, indicating that the appropriately dressed agent worked better combined with the inappropriate setting than with the appropriate setting. Overall, given those non-significant main effects and the small effect size of the interaction, there seem to be some degrees of freedom for designers of pedagogical agents in animated learning videos. However, these degrees of freedom may be limited to at least moderate (i.e., neutral) levels of appropriateness.
... For instance, it has enhanced and extended classroom education [11], assisted in science education [12]- [14], raised social issues awareness in an engaging way [15], and simulated natural disaster protocols for public safety [16]. There is also an investigation on how to improve the effectiveness of IL via pedagogical agents [17]. However, the literature seems to indicate that research on learning outcomes, intervention characteristics, and assessment measures associated with XR in education has been sparse [18]- [20]. ...
Full-text available
Metaverse, an alternative universe for play, work and interaction, has become a captivating topic for academia and industry in recent times. This opens the question on what a metaverse for education, or edu-metaverse, should look like. It is believed that this metaverse for learning should be grounded by a pedagogical theory. Particularly, we propose a constructivist metaverse learning theory with eight actionable principles to guide the edu-metaverse and its applications. With this metaverse learning theory, we further propose the framework for an edu-metaverse; it is essentially walkable yellow pages that connect knowledge. The core idea is to combine the structure of knowledge graphs and the immersion of virtual reality in order to facilitate association, exploration and engagement in learning. Our current prototype for this edu-metaverse vision, K-Cube VR, is also presented. We have tested K-Cube VR for the introduction of course topics to our students and the results indicate that our edu-metaverse framework benefits students by providing a focused environment and structured learning on the topics of a course, akin to a mind map. Overall, in this paper, we present an edu-metaverse design that is rooted in a constructivist pedagogy that already shows promising results from a pilot user study via our metaverse prototype.
... For pedagogical agent representation, there have been works discussing implications for the design of pedagogical agents in VR based on comparisons between the presence of a real instructor immersed with students in the environment vs. training with recordings of instructors [12,51]. Petersen et al. [42] discuss the effect of the appearance and behavior of pedagogical agents on knowledge gain. ...
Conference Paper
Full-text available
Figure 1: Four Spatial Representations for placement of learning content in VRLEs. (a) World-anchored: on a TV screen fixed in the environment. (b) User-anchored (Controllers): on a panel anchored to the VR controllers. (c) User-anchored (HMD): on a panel anchored to the head-mounted display. (d) Object-anchored: on a panel anchored to the object associated with current instruction. ABSTRACT A recent surge in the application of Virtual Reality in education has made VR Learning Environments (VRLEs) prevalent in fields ranging from aviation, medicine, and skill training to teaching factual and conceptual content. In spite of multiple 3D affordances provided by VR, learning content placement in VRLEs has been mostly limited to a static placement in the environment. We conduct two studies to investigate the effect of different spatial representations of learning content in virtual environments on learning outcomes and user experience. In the first study, we studied the effects of placing content at four different places-world-anchored (TV screen placed in the environment), user-anchored (panel anchored to the wrist or head-mounted display of the user) and object-anchored (panel anchored to the object associated with current content)-in the VR environment with forty-two participants in the context of learning how to operate a laser cutting machine through an immersive tu-torial. In the follow-up study, twenty-two participants from this study were given the option to choose from these four placements to understand their preferences. The effects of placements were examined on learning outcome measures-knowledge gain, knowledge * transfer, cognitive load, user experience, and user preferences. We found that participants preferred user-anchored (controller condition) and object-anchored placement. While knowledge gain, knowledge transfer, and cognitive load were not found to be significantly different between the four conditions, the object-anchored placement scored significantly better than the TV screen and head-mounted display conditions on the user experience scales of attractiveness, stimulation, and novelty.
... Laboratory settings have the advantages of control and internal validity, but their ecological validity is highly limited [4]. Our out-of-the-lab data collection did not control where, when, how long and via which laptop or desktop participants could join, allowing more natural behaviour [40,59,46]. In addition, most datasets only include either mouse or keyboard data, while we opted for evaluations on both modalities. ...
Analysing and modelling interactive behaviour is an important topic in human-computer interaction (HCI) and a key requirement for the development of intelligent interactive systems. Interactive behaviour has a sequential (actions happen one after another) and hierarchical (a sequence of actions forms an activity driven by interaction goals) structure, which may be similar to the structure of natural language. Designed based on such a structure, natural language processing (NLP) methods have achieved groundbreaking success in various downstream tasks. However, few works linked interactive behaviour with natural language. In this paper, we explore the similarity between interactive behaviour and natural language by applying an NLP method, byte pair encoding (BPE), to encode mouse and keyboard behaviour. We then analyse the vocabulary, i.e., the set of action sequences, learnt by BPE, as well as use the vocabulary to encode the input behaviour for interactive task recognition. An existing dataset collected in constrained lab settings and our novel out-of-the-lab dataset were used for evaluation. Results show that this natural language-inspired approach not only learns action sequences that reflect specific interaction goals, but also achieves higher F1 scores on task recognition than other methods. Our work reveals the similarity between interactive behaviour and natural language, and presents the potential of applying the new pack of methods that leverage insights from NLP to model interactive behaviour in HCI.
... These contributions are relevant to the areas of immersive learning technologies [40,41] that are discussed in the AR/MR/VR [42][43][44][45] and human-computer interaction [46][47][48] communities, as it involves the introduction of a virtual reality approach for logogram learning. Precisely, the research is aligned with the field of embodied learning [49,50] as it involves bodily gestures using arms and legs and speeches to facilitate lexical item retrieval and procedural memories as a cross-modal prime. ...
Full-text available
A logogram is a type of writing system in which each character represents a word. Compared to segmental scripts where the alphabets reflect sounds, learning logograms are disengaging, since each character is not linked to its pronunciation. This paper presents Logogram VR, a virtual reality edutainment game that uses a treadmill and controllers to teach Hanja, which uses logograms. Hanja is a traditional Korean language writing system comprising over 8000 Chinese characters. The system leverages the logogram’s feature that each letter stands for each vocabulary item, as an embodied learning strategy. Specifically, it incorporates each character’s meaning into the VR learning environment, accompanied by gamified actions using a treadmill and VR controllers. We evaluated the system with 33 participants to test its overall usability, while determining the desirable playtime and number of characters for the further enhancement of it. We demonstrated and assessed the system with 125 visitors at an exhibition to disseminate it and verify the results with a wider population sample. The user studies revealed that the system provides a playful experience for learning Hanja without severe motion sickness. The differences in age groups showed that the embodiment approach utilizing meanings and actions in VR may be an effective logogram edutainment strategy, particularly among adolescents.
Analysing and modelling interactive behaviour is an important topic in human-computer interaction (HCI) and a key requirement for the development of intelligent interactive systems. Interactive behaviour has a sequential (actions happen one after another) and hierarchical (a sequence of actions forms an activity driven by interaction goals) structure, which may be similar to the structure of natural language. Designed based on such a structure, natural language processing (NLP) methods have achieved groundbreaking success in various downstream tasks. However, few works linked interactive behaviour with natural language. In this paper, we explore the similarity between interactive behaviour and natural language by applying an NLP method, byte pair encoding (BPE), to encode mouse and keyboard behaviour. We then analyse the vocabulary, i.e., the set of action sequences, learnt by BPE, as well as use the vocabulary to encode the input behaviour for interactive task recognition. An existing dataset collected in constrained lab settings and our novel out-of-the-lab dataset were used for evaluation. Results show that this natural language-inspired approach not only learns action sequences that reflect specific interaction goals, but also achieves higher F1 scores on task recognition than other methods. Our work reveals the similarity between interactive behaviour and natural language, and presents the potential of applying the new pack of methods that leverage insights from NLP to model interactive behaviour in HCI.KeywordsInteractive Behaviour ModellingNatural Language ProcessingMouse and Keyboard InputOut-of-the-lab Dataset
Full-text available
Immersive virtual reality (IVR) simulations for education have been found to increase affective outcomes compared to traditional media, but the effects on learning are mixed. As reflection has previously shown to enhance learning in traditional media, we investigated the efficacy of appropriate reflection exercises for IVR. In a 2 × 2 mixed‐methods experiment, 89 (61 female) undergraduate biochemistry students learned about the electron transport chain through desktop virtual reality (DVR) and IVR (media conditions). Approximately, half of each group engaged in a subsequent generative learning strategy (GLS) of teaching in pairs (method conditions). A significant interaction between media and methods illustrated that the GLS of teaching significantly improved transfer ( d = 1.26), retention ( d = 0.60) and self‐efficacy ( d = 0.82) when learning through IVR, but not DVR. In the second part of the study, students switched media conditions and the experiment was repeated. This time, significant main effects favoring the IVR group on the outcomes of intrinsic motivation ( d = 0.16), perceived enjoyment ( d = 0.94) and presence ( d = 1.29) were observed, indicating that students preferred IVR after having experienced both media conditions. The results support the view that methods enable media that affect learning and that the GLS of teaching is specifically relevant for IVR. Practitioner Notes What is already known about this topic Previous research has found a media effect with Immersive Virtual Reality (IVR) in education leading to better motivational outcomes compared to less immersive media, but effects on learning outcomes are mixed. There is evidence that Generative Learning Strategies (GLSs) such as summarizing and enacting can increase learning in IVR. There is also evidence that some instructional methods, such as pretraining, may be beneficial for learning in IVR. What this paper adds Evidence that the GLS of teaching improves self‐efficacy, retention and transfer in educational IVR. An interaction effect between media (DVR/IVR) and method (GLS/no‐GLS) on self‐efficacy, retention and transfer supporting the theoretical view that method enables media. No difference in perceived enjoyment, motivation and presence for students who were new to learning through these media (DVR/IVR), but differences became significant when students learned through the other media first with students preferring IVR. Implications for practice and/or policy Since IVR learning experiences can be highly engaging and also cognitively demanding, it is beneficial to introduce reflection exercises after an IVR learning experience to ensure that students reflect over the material and integrate it with their long‐term memory. One effective solution is to engage students in the GLS of teaching after an IVR simulation, thereby prompting them to select relevant information, organize it into a coherent structure and elaborate on it by incorporating it with their existing knowledge.
Full-text available
As immersive virtual reality (IVR) systems proliferate in classrooms, it is important to understand how they affect learning outcomes and the underlying affective and cognitive processes that may cause these outcomes. Proponents argue that IVR could improve learning by increasing positive affective and cognitive processing, thereby supporting improved performance on tests of learning outcome, whereas opponents of IVR contend that it could hurt learning by increasing distraction, thereby disrupting cognitive learning processes and leading to poorer learning outcomes. In a media comparison study, students viewed a biology lesson either as an interactive animated journey in IVR or as a slideshow on a desktop monitor. Those who viewed the IVR lesson performed significantly worse on transfer tests, reported higher emotional arousal, reported more extraneous cognitive load and showed less engagement based on EEG measures than those who viewed the slideshow lesson, with or without practice questions added to the lessons. Mediational analyses showed that the lower retention scores for the IVR lesson were related to an increase in self-reported extraneous cognitive load and emotional arousal. These results support the notion that immersive environments create high affective and cognitive distraction, which leads to poorer learning outcomes than desktop environments.
Full-text available
Lay Description What is already known about this topic Cognitive load theory is broadly applied in different fields of education. Leppink's Cognitive Load Scale (CLS) is widely used in traditional learning situation. Measuring cognitive load is important in virtual learning environments (VLE). Valid and reliable measures of cognitive load are important to support instructional design in VLE. What this paper adds Through three studies, we investigated the validity and reliability of the CLS and developed the extraneous cognitive load (EL) dimension into three sub‐scales relevant for VLE: EL instructions, EL interaction and EL environment. We named the he new measure the Multidimensional Cognitive Load Scale for Virtual Environments (MCLSVE). We investigated the validity of the CLS (Study 1) and the MCLSVE (Study 2 and 3) using the Partial Credit Model (PCM), Confirmatory Factor Analysis (CFA) and correlations with retention tests. The studies provide initial evidence for the validity and reliability of the MCLSVE. Implications for practice and/or policy Researchers using virtual reality can apply the MCLSVE to obtain a multidimensional measure of extraneous cognitive load. The adapted version of the Cognitive Load Scale can differentiate meaningfully between easy and difficulty learning situations and can thus be used to examine the difficulty in learning situation using virtual reality. The study provides a framework for developing multidimensional conceptualizations of EL for other settings where instructions and explanations are not the only source of EL.
Full-text available
Science‐related competencies are demanded in many fields, but attracting more students to scientific educations remains a challenge. This paper uses two studies to investigate the value of using Immersive Virtual Reality (IVR) laboratory simulations in science education. In Study 1, 99 (52 male, 47 female) seventh (49) and eighth (50) grade students between 13 and 16 years of age used an IVR laboratory safety simulation with a pre‐ to posttest design. Results indicated an overall increase in interest in science and self‐efficacy, but only females reported an increase in science career aspirations. Study 2 was conducted with 131 (47 male, 84 female) second (77) and third (54) year high school students aged 17 to 20 and used an experimental design to compare the value of using an IVR simulation or a video of the simulation on the topic of DNA‐analysis. The IVR group reported significantly higher gains from pre‐ to posttest on interest, and social‐outcome expectations than the video group. Furthermore, both groups had significant gains in self‐efficacy and physical outcome expectations, but the increase in career aspirations and self‐outcome expectations did not reach statistical significance. Thus, results from the two studies suggest that appropriately developed and implemented IVR simulations can address some of the challenges currently facing science education. Practitioner Notes What is already known about this topic Science‐related skills are becoming increasingly important as these are in high demand, not only in traditional science occupations, but also in other fields of work and in our daily lives. Thus, it is desirable to inspire students to pursue careers within science. According to the social cognitive career theory (SCCT), students’ educational choice goals (ie, career aspirations) are shaped by their interests, self‐efficacy and outcome expectations. Students report low levels of interest in science and several studies find that positive attitudes toward science decline with age, from primary through the secondary school years. Unfavorable attitudes toward science could be attributed to science education failing to engage students at a satisfactory level. Immersive Virtual reality (IVR) is touted for its potential to offer inspiring learning experiences that increase interest and self‐efficacy. What this paper adds A systematic investigation of how IVR laboratory simulations can increase science interest and career aspirations in middle school (aged 13 to 16) and high school (aged 17 to 20) students. Evidence that IVR‐based learning experiences can significantly increase students’ interest in science topics. An indication that an IVR‐based simulation led to a significant pre‐ to posttest increase in science aspirations among 13‐ to 16‐year‐old female students. Implications for practice and/or policy IVR‐based simulations are specifically relevant when the goal of an educational intervention is to increase students’ situational interest and social‐outcome expectations in a science topic. Provided the right instructional design, IVR might help bridge the gender difference within science education in middle school (ie, students between ages of 13 and 16). Although IVR‐based simulations can increase situational interest, longitudinal interventions are needed to create lasting effects on career aspirations in science.
Full-text available
The Coronavirus 2019 (COVID-19) pandemic has created significant challenges for the global higher education community. Through a desktop analysis leveraging university and government sources where possible, we provide a timely map of the intra-period higher education responses to COVID-19 across 20 countries. We found that the responses by higher education providers have been diverse from having no response through to social isolation strategies on campus and rapid curriculum redevelopment for fully online offerings. We provide in our discussion a typology of the types of responses currently undertaken and assess the agility of higher education in preparing for the pandemic. We believe there are significant opportunities to learn from the pedagogical developments of other universities, in order to strengthen our collective response to COVID-19 now and into the future.
Conference Paper
Full-text available
This paper reports findings from a between-subjects experiment that investigates how different learning content representations in virtual environments (VE) affect the process and outcomes of learning. Seventy-eight participants were subjected to an immersive virtual reality (VR) application, where they received identical instructional information, rendered in three different formats: as text in an overlay interface, as text embedded semantically in a virtual book, or as audio. Learning outcome measures, self-reports, and an electroencephalogram (EEG) were used to compare conditions. Results show that reading was superior to listening for the learning outcomes of retention, self-efficacy, and extraneous attention. Reading text from a virtual book was reported to be less cognitively demanding, compared to reading from an overlay interface. EEG analyses show significantly lower theta and higher alpha activation in the audio condition. The findings provide important considerations for the design of educational VR environments.
Full-text available
We investigated the instructional effectiveness of using an interactive and immersive virtual reality (IVR) simulation versus a video for teaching scientific knowledge in two between-subject experiments. In Experiment 1, 131 high school students (84 females) used a science simulation that involved forensic analysis of a collected DNA sample in a virtual laboratory environment rendered in IVR or as a video covering the same material. In Experiment 2, 165 high school students (111 females) replicated the experiment with approximately half of each group being asked to engage in the generative learning strategy of enactment after the lesson--i.e., carrying out the learned procedures with concrete manipulatives. Across both experiments, the IVR groups reported significantly higher perceived enjoyment and presence than the video group. However, no significant differences were found between media for procedural knowledge in Experiment 1 and 2, or transfer in Experiment 2. Also, there was no difference in declarative knowledge across media in Experiment 1, and there was a media effect favoring video in Experiment 2 (ηp2 = 0.028). Enactment lead to significantly better procedural knowledge (ηp2 = 0.144) and transfer (ηp2 = 0.088) in the IVR group but not in the video group. In conclusion, learning in IVR is not more effective than learning with video but incorporating generative learning strategies is specifically effective when learning through IVR. The results suggest that the value of IVR for learning science depends on how it is integrated into a classroom lesson.
Full-text available
Researchers have explored the benefits and applications of virtual reality (VR) in different scenarios. VR possesses much potential and its application in education has seen much research interest lately. However, little systematic work currently exists on how researchers have applied immersive VR for higher education purposes that considers the usage of both high-end and budget head-mounted displays (HMDs). Hence, we propose using systematic mapping to identify design elements of existing research dedicated to the application of VR in higher education. The reviewed articles were acquired by extracting key information from documents indexed in four scientific digital libraries, which were filtered systematically using exclusion, inclusion, semi-automatic, and manual methods. Our review emphasizes three key points: the current domain structure in terms of the learning contents, the VR design elements, and the learning theories, as a foundation for successful VR-based learning. The mapping was conducted between application domains and learning contents and between design elements and learning contents. Our analysis has uncovered several gaps in the application of VR in the higher education sphere—for instance, learning theories were not often considered in VR application development to assist and guide toward learning outcomes. Furthermore, the evaluation of educational VR applications has primarily focused on usability of the VR apps instead of learning outcomes and immersive VR has mostly been a part of experimental and development work rather than being applied regularly in actual teaching. Nevertheless, VR seems to be a promising sphere as this study identifies 18 application domains, indicating a better reception of this technology in many disciplines. The identified gaps point toward unexplored regions of VR design for education, which could motivate future work in the field.
Conference Paper
Full-text available
Virtual reality (VR) enables users to experience informal learning activities, such as visiting museum exhibitions or attending tours independent of their physical locations. Consequently, VR offers compelling use cases by making informal learning and education accessible to a broader audience and simultaneously reducing the carbon footprint. For many learning activities, the presence of a human guide is essential for participants' experience. The effect of the presence of a guide and its appearance in VR is, however, unclear. In this paper, we compare a real-world guide with a realistic, an abstract, and an audio-only representation of a virtual guide. Participants followed four multimodal presentations while we investigated the effect on comprehension, presence, co-presence and the perception of the guide. Our results show that even a realistic presentation of a guide results in significantly lower co-presence, humanness, and attractiveness compared to a human guide. Qualitative results and participants' feedback indicate that having no visual representation of the guide helps to focus on the content but can reduce the connection with the guide.
Immersive Virtual Reality (IVR) is being used for educational virtual field trips (VFTs) involving scenarios that may be too difficult, dangerous or expensive to experience in real life. We implemented an immersive VFT within the investigation phase of an inquiry‐based learning (IBL) climate change intervention. Students investigated the consequences of climate change by virtually traveling to Greenland and exploring albedo and greenhouse effects first hand. A total of 102 seventh and eighth grade students were randomly assigned to one of two instructional conditions: (1) narrated pretraining followed by IVR exploration or (2) the same narrated training material integrated within the IVR exploration. Students in both conditions showed significant increases in declarative knowledge, self‐efficacy, interest, STEM intentions, outcome expectations and intentions to change behavior from the pre‐ to post‐assessment. However, there was a significant difference between conditions favoring the pretraining group on a transfer test consisting of an oral presentation to a fictitious UN panel. The findings suggest that educators can choose to present important prerequisite learning content before or during a VFT. However, adding pretraining may lead to better transfer test performance, presumably because it helps reduce cognitive load while learning in IVR. Practitioner Notes What is already known about this topic? Immersive virtual reality (IVR) simulations lead to higher presence but may lead to less learning when the content is not designed based on the affordances of the technology. One explanation for this finding is that cognitive load may be higher in IVR. The pretraining principle (ie, individuals learn more deeply from interactive multimodal learning environments when they receive pretraining on relevant prior knowledge) can be particularly effective in IVR‐based learning compared to learning through a video. Evidence shows that instructional design principles such as segmentation and generative learning strategies such as summarization can improve learning in IVR simulations. What this paper adds An investigation of the value of two different approaches to designing immersive virtual field trips (VFTs) within a real educational middle school context. Evidence that VFTs featuring IVR climate change simulations, in the context of inquiry‐based learning (IBL), can increase important variables such as declarative knowledge, interest in science and intentions to take climate action in seventh and eighth grade students. Evidence that presenting important learning content before a VFT leads to higher transfer scores. Implications for practice and/or policy Implementing an immersive VFT within the context of an IBL intervention provides students with relevant and engaging learning experiences and results in increased knowledge and interest in science. In the design of instruction using VFTs, educators can choose to either present prerequisite learning content prior to a VFT or during a VFT. However, adding pretraining has an advantage in terms of higher transfer scores.