ArticlePDF Available

Abstract and Figures

Recent advances in virtual reality (VR) technology allow for potential learning and education applications. For this study, 99 participants were assigned to one of three learning conditions: traditional (textbook style), VR and video (a passive control). The learning materials used the same text and 3D model for all conditions. Each participant was given a knowledge test before and after learning. Participants in the traditional and VR conditions had improved overall performance (i.e. learning, including knowledge acquisition and understanding) compared to those in the video condition. Participants in the VR condition also showed better performance for ‘remembering’ than those in the traditional and the video conditions. Emotion self-ratings before and after the learning phase showed an increase in positive emotions and a decrease in negative emotions for the VR condition. Conversely there was a decrease in positive emotions in both the traditional and video conditions. The Web-based learning tools evaluation scale also found that participants in the VR condition reported higher engagement than those in the other conditions. Overall, VR displayed an improved learning experience when compared to traditional and video learning methods.
(page number not for citation purpose)
*Corresponding author. Email:
Research in Learning Technology 2018. © 2018 D. Allcoat and A. von Mühlenen. Research in Learning Technology is the journal of the Association for
Learning Technology (ALT), a UK-based professional and scholarly society and membership organisation. ALT is registered charity number 1063519. This is an Open Access article distributed under the terms of the Creative Commons Attribution 4.0 International License
(, allowing third parties to copy and redistribute the material in any medium or format and to remix,
transform, and build upon the material for any purpose, even commercially, provided the original work is properly cited and states its license.
Citation: Research in Learning Technology 2018, 26: 2140 -
Research in Learning Technology
Vol. 26, 2018
Learning in virtual reality: Effects on performance,
emotion and engagement
Devon Allcoat* and Adrian von Mühlenen
Department of Psychology, University of Warwick, Coventry, UK
(Received 12 June 2018; nal version received 23 October 2018)
Recent advances in virtual reality (VR) technology allow for potential learning and
education applications. For this study, 99 participants were assigned to one of three
learning conditions: traditional (textbook style), VR and video (a passive control).
The learning materials used the same text and 3D model for all conditions. Each
participant was given a knowledge test before and after learning. Participants in
the traditional and VR conditions had improved overall performance (i.e. learn-
ing, including knowledge acquisition and understanding) compared to those in the
video condition. Participants in the VR condition also showed better performance
for ‘remembering’ than those in the traditional and the video conditions. Emotion
self-ratings before and after the learning phase showed an increase in positive emo-
tions and a decrease in negative emotions for the VR condition. Conversely there
was a decrease in positive emotions in both the traditional and video conditions.
The Web-based learning tools evaluation scale also found that participants in the
VR condition reported higher engagement than those in the other conditions.
Overall, VR displayed an improved learning experience when compared to tradi-
tional and video learning methods.
Keywords: VR; education; experience; mood
This paper is part of the special collection Mobile Mixed Reality Enhanced Learning, edited by Thom
Cochrane, Fiona Smart, Helen Farley and Vickel Narayan. More papers from this collection can be
found here
Interactive technology is progressing at an incredibly fast rate, and advances in virtual
reality (VR) technology have led to many potential new applications. Commercial VR
headsets are widely used for entertainment purposes, with many individuals’ expe-
riences of VR being from video games and other widely distributed media, as these
media are widely advertised and well known, leading to higher popularity. However,
VR has broader application possibilities, thanks to signicant advances in the tech-
nology, including the technology now available in a mobile format.
VR technologies allow the user to see and interact with virtual environments and
objects. Modern VR is delivered through a headset, which allows the user to see – and
in some cases, hear – the 3D environment. In this way the user is totally immersed in
the virtual environment, as it replaces the physical environment around them. Immer-
sion and engagement can be considered intrinsically linked in virtual environments
D. Allcoat and A. von Mühlenen
2 Citation: Research in Learning Technology 2018, 26: 2140 -
(page number not for citation purpose)
(McMahan 2003). Mount et al. (2009) discussed the relationship between immersion,
presence and engagement. They explored what it means for a learner to be immersed
and considered immersion and engagement in 3D virtual environments, to outline
how 3D virtual environments can be used to enhance learner engagement.
VR boasts a number of features that could be useful for education: it presents
environments in 3D, it is interactive and it is able to give audio, visual and even haptic
feedback. Presenting learning materials in 3D can be especially benecial for teaching
subjects where it is important to visualise the learning materials (e.g. in chemistry or
in engineering). Though visualising is one of the most obvious benets of VR, this
could also be accomplished with simple video. However, videos are passive learning
objects, whereas VR allows for a direct interaction with the environment. Interactivity
and feedback can be valuable for all subjects, as there are specic benets of interac-
tive learning because it promotes active learning instead of passive learning.
The usefulness of VR in education might also depend on the type of learning.
Learning styles theories suggest that there are various ways to learn, and some individ-
uals learn better with some methods than others, as they have different approaches to
information processing. The well-known visual–auditory–kinaesthetic learning styles
model (Barbe, Swassing, and Milone 1979) suggests there are three types of learning
styles: visual, auditory and kinaesthetic. VR allows all three of these learning styles
to be targeted in one application, as VR headsets allow for complex visual renderings,
audio and movement tracking. Though there has been much contention over learning
styles theories, as discussed in the following, having one learning environment that
can encompass multiple learning styles could be very benecial as it would be suitable
for a much wider range of individuals.
Other learning styles models include the importance of learning through different
perceptual modalities, many of which are able to be targeted in VR (for an overview of
various learning styles theories, see Cassidy 2004). It has been suggested that having a
variety of learning methods is valuable; Gaytan and McEwen (2007) concluded that
it is benecial to use a variety of instructional methods to appeal to students’ learning
preferences. VR activities could be designed to include multiple learning methods, so
learners can choose to engage with the learning materials in the manner that interests
them the most, as students have a preference for multiple modes of information pre-
sentation (Lujan and DiCarlo 2006).
Scholars now are more critical of learning styles theories (e.g. Pashler et al. 2008;
Riener and Willingham 2010), stating that, though there are many theories, there is
little empirical evidence for learning styles. However, others still consider it impor-
tant to be aware of varying sensory modalities and learning approaches because of
students’ differing learning habits and preferences (e.g. Hawk and Shah 2007; Kharb
etal. 2013). The impact of learning styles on e-learning is also debated (Truong 2016),
including how best to design adaptive virtual learning environments whilst consider-
ing learning styles (Kanninen 2008). There are potential benets of targeting multiple
methods of learning within VR, to allow for different information processing. This
could be not only a result of learning methods and individuals’ preferences but also
an illustration of how different types of information may be better presented in some
formats than others (e.g. language may be best learnt with audio, whereas engineering
may be better suited to visualisation).
VR is not necessarily equally suitable for all subject areas; benets of visualis-
ing are more signicant in some subjects than others. As such, VR applications
may be more suited to some areas of education than others. The revised Bloom’s
Research in Learning Technology
Citation: Research in Learning Technology 2018, 26: 2140 - 3
(page number not for citation purpose)
taxonomy (Bloom et al. 1956; see also Anderson et al. 2001) suggests that there is
not simply one way in which information is processed and learnt; instead it presents
learning as a hierarchy of learning, consisting of six stages that involve cognitive
processes from simplest to most complex (from remember, understand, apply, anal-
yse and evaluate to create). It is suggested that these different types of learning can
be processed differently; some methods of study that are used in education are only
applicable to some subjects. Debates, for example, are often good at engaging stu-
dents with material that requires critical thinking (Camp and Schnader 2010; Scott
2008) but are less suited to learning more concrete information, such as for sciences
like physics or chemistry. VR, for example, may be less benecial for learning to
play a musical instrument that requires tactile feedback, such as a guitar, but may
be particularly useful for topics where spatial arrangement is important or there are
dynamic changes.
Though not many empirical studies have yet been conducted, VR has been com-
pared to traditional learning in some areas. In one study a group of military students
were taught with either the lecture-based teaching methods that are traditionally
used for the subject material (corrosion prevention and control) or with an immersive
VR-based teaching method (Webster 2015). They found that whereas the traditional
learning group had an improvement of 11%, the VR group had a higher improvement
of 26%.
Bellamy and Warren (2011) conducted a case study using simple online interactive
simulations which mimicked real experiments. Eighty-three per cent of their students
reported that they found these online simulations helpful or very helpful, and their
demonstrators stated that the students seemed much better prepared and more will-
ing to answer questions when they had done the online simulations. These and other
examples promote for learning the usefulness of simulated environments as alterna-
tives to real-life scenarios.
Creating educational applications for VR could be a laborious and costly
endeavour, so it is important to investigate whether these applications are useful
for learning or not. Therefore, explorative research can help answer whether the
development of educational applications for this type of hardware is worth pursu-
ing. As VR technology has only recently become more accessible and affordable,
research in the past using VR in educational and pedagogic settings has typically
used smaller sample sizes with less rigorous methodologies. This study looks to
address that, considering not only test performance (used as a measure of learning)
but also other outcomes of using VR for learning, such as effects on emotion and
All participants were rst-year Psychology students at the University of Warwick
(UK), who completed the study for course credit. A total of 99 participants
(84females, 15 males) who were 19 years of age on average were assigned randomly
to one of three learning conditions: traditional (textbook style), VR and video. All
participants reported normal or corrected-to-normal vision. The study was approved
by the university’s Humanities and Social Sciences Research Ethics Committee, and
all participants gave informed written consent and were aware of their right to with-
draw at any time.
D. Allcoat and A. von Mühlenen
4 Citation: Research in Learning Technology 2018, 26: 2140 -
(page number not for citation purpose)
The questionnaires and learning materials were presented on a 19" LCD computer
screen (1920 × 1080 pixels, 60 Hz) using Microsoft Word and Qualtrics. Responses
were collected through mouse and keyboard. A HTC Vive (Xindian, New Taipei,
Taiwan) (Figure 1) was used for the VR condition. The headset weighs 550 g and dis-
plays a 3D environment via two OLED displays (1080 × 1200 pixels per eye, 90 Hz)
with a eld of view of 100 × 110 degrees. Participants controlled the VR environment
with the standard handheld HTC Vive controller.
Learning materials
The learning materials used the same text and 3D model of a plant cell for all three
conditions. The VR condition presented the model from the application ‘Lifeliqe
Museum’ on the HTC Vive headset, allowing the participants to see and interact with
a 3D model, with accompanying descriptive text (Figure 1). The 3D plant cell model
was fully interactive, allowing participants to highlight individual cell parts, change
the size of the cell and rotate it. They could also teleport around the virtual room,
with the plant cell appearing as a oating object in the room with them, which they
could navigate around. A menu was available, virtually attached to one of the con-
trollers, showing names of each part of the plant cell. Participants could select one of
these parts from the menu (e.g. the Golgi apparatus) and it would highlight the part
on the 3D model. This could also be done the opposite way, by selecting the part on
the model, which would highlight the name on the menu. A written explanation of
the purpose of each part of the plant cell was also available on this menu. The option
of a narrator was disabled for this study, in order to remove audio learning as a con-
founding variable.
Figure 1. e HTC Vive headset and examples of the 3D model used as learning material
for all conditions from the Lifeliqe Museum virtual reality environment.
Research in Learning Technology
Citation: Research in Learning Technology 2018, 26: 2140 - 5
(page number not for citation purpose)
The video condition used a 2D recording from the HTC Vive, matched from
participants in the VR condition and presented on a computer screen. Participants
were informed that they could navigate this video at will (play or pause, fast forward,
rewind), as they would in a distance learning scenario. This was a control to the VR
condition as it presented the same visual information, with the same graphics, but did
not have other VR features, such as interactivity and immersive 3D display. As such,
this condition acted as a carefully matched control condition.
The textbook condition used screenshots of the 3D model with the same accom-
panying text and presented them on a computer screen as a PDF le (Figure 2). This
ensured that all three groups had the same information and visuals to learn with, with
the only difference being the format in which these materials were presented.
Rating scales
An adapted version of the Differential Emotions Scale (DES, Izard et al. 1974), with
nine emotion categories (interest, amusement, sadness, anger, fear, anxiety, contempt,
surprise and elatedness), was used to measure participants’ mood before and after the
learning phase. Participants were asked to rate to which extent the emotional adjec-
tives, each represented with three words (e.g. surprised, amazed, astonished), applied
to them on a scale from 1 (not at all) to 5 (very strongly). Five of the categories related
to negative emotions, and four related to positive emotions.
The Web-based Learning Tools (WBLT) Evaluation Scale questionnaire (Kay
2011) was used to measure engagement. The WBLT Evaluation Scale asks partici-
pants to rate what they thought about the learning tools across 13 questions on a scale
of 1 (strongly disagree) to 5 (strongly agree). The questions included items such as
‘the learning object helped teach me a new concept’ and ‘I would like to use the learn-
ing object again’. The questions can be grouped into the three categories ‘learning’,
‘design’ and ‘engagement’.
The procedure was the same for each participant, starting with a pretest and the DES,
followed by the learning phase. For the learning phase participants were instructed to
learn as much as they could from the learning materials, and all conditions were given
the same amount of time (7 min). After the learning phase, participants completed a
post-test consisting of the same questions as the pretest, the DES, the WBLT and one
Figure 2. Example of the textbook conditions, using the same text and screenshots from
the Lifeliqe Museum virtual reality environment.
D. Allcoat and A. von Mühlenen
6 Citation: Research in Learning Technology 2018, 26: 2140 -
(page number not for citation purpose)
question that allowed for qualitative feedback. The improvement from pretest to post-
test was used as the main measure of learning performance. This method was used in
order to account for any participants with prior knowledge of the subject (plant cells).
Questions used for the test were either sourced directly from a British AQA Biology
A-Level exam or were in the same style as these questions.
The 17 biology knowledge questions were marked as correct or incorrect and used in
the calculation of an overall percentage correct, separately for each participant. The
top half of Table 1 shows the average knowledge scores in the pretest and in the post-
test, together with the difference scores, as an indicator for learning. Here the overall
difference between pretest and post-test is referred to as ‘performance’ to differentiate
it from the ‘learning’ scores of the WBLT Evaluation Scale. The corresponding aver-
age condence ratings are given in the bottom half of the table.
The knowledge scores were analysed with a mixed-design ANOVA with the between-
subject factor condition (textbook, video, virtual) and the within-subject factor test
(pre-, post). The ANOVA revealed a signicant main effect for test, F(1,96)=273.25,
p< 0.001, ηp
2 = 0.740, indicating that knowledge improved overall by 23.2% from pre-
test to post-test, and a signicant test × condition interaction, F(2,96) = 6.80, p=0.002,
2 = 0.124. The signicant interaction was further analysed with two split-up ANOVAs,
separately for pretest and for post-test. The ANOVA on the post-test data revealed a
signicant condition effect, F(2,96) = 3.51, p = 0.034, ηp
2 = 0.068. Post-hoc least sig-
nicant difference (LSD) showed that participants in the VR condition scored signi-
cantly higher than participants in the video condition (56.5% vs. 43.9%, respectively;
p= 0.009). The pretest ANOVA showed no signicant effect (p = 0.793).
The condence ratings showed a similar pattern of results as the knowledge data
(see bottom half of Table 1). The equivalent mixed-design ANOVA revealed a signi-
cant effect for test, F(1,96) = 266.96, p < 0.001, ηp
2 = 0.736, as a result of participants
being more condent in the post-test than in the pretest (3.24 vs. 2.24, respectively), as
well as a signicant test × condition interaction, F(2,96) = 5.80, p = 0.004, ηp
2 = 0.108,
because of less condence gain in the video than in the VR or textbook condition
(0.71 vs. 1.12 and 1.18, respectively).
Table 1. Number of participants (N), knowledge scores (percentage correct) and condence
ratings (1–5) in the pretest and post-test separately for the three conditions.
Condition NPretest Post-test Difference
Knowledge scores
Virtual 34 28.1% 56.5% 28.5%
Video 34 27.9% 43.9% 16.1%
Textbook 31 25.3% 50.2% 24.9%
Condence ratings
Virtual 34 2.24 3.35 1.12
Video 34 2.33 3.04 0.71
Textbook 31 2.14 3.32 1.18
Research in Learning Technology
Citation: Research in Learning Technology 2018, 26: 2140 - 7
(page number not for citation purpose)
The knowledge questionnaire data was further analysed by splitting the questions
into two categories on the basis of Bloom’s taxonomy (Bloom et al. 1956). The rst
group (12 questions) related to the remembering of information, whereas the sec-
ond group (5 questions) was more concerned with the understanding of information.
The overall percentage correct in each category is shown in Figure 3. A 3 × 2-way
ANOVA on the remembering scores showed a signicant test × condition interac-
tion, F(2,96)=6.28, p = 0.003, ηp
2 = 0.116. Further split-up ANOVAs and LSD tests
revealed that in the post-test participants scored signicantly higher in the VR than
in the video and the textbook condition (53.1% vs. 40.6% and 43.6; p = 0.008 and
p= 0.041, respectively). The corresponding analysis of the understanding scores also
revealed a signicant interaction, F(2,96) = 3.15, p = 0.047, ηp
2 = 0.062; however,
further tests showed no difference between VR and textbook, but scores in the video
condition were lower than scores in the VR and textbook conditions (50.2% vs. 60.2%
and 62.3%; p = 0.071 and p = 0.79, respectively). In summary, participants in the VR
group showed better remembering than participants in the textbook group, but there
was no difference between the two groups in terms of understanding.
Emotional response
DES ratings were split into the two categories: positive emotions (interest, amuse-
ment, surprise and elatedness) and negative emotions (sadness, anger, fear, anxiety
and disgust), and average ratings are shown in Figure 4. A 3 × 2-way ANOVA with the
factors condition and test on the positive emotions revealed a signicant main effect of
condition, F(2,96) = 13.24, p < 0.001, ηp
2 = 0.216, and a signicant interaction effect,
F(2,96) = 31.40, p < 0.001, ηp
2 = 0.395. The signicant interaction was further analysed
with three split-up t-tests, to see whether ratings changed from pre- to post-test. Posi-
tive emotion signicantly increased from 3.2 to 3.8 in the VR condition, t(30) = 4.73,
p < 0.001, and signicantly decreased in the video condition, t(33) = 4.92, p < 0.001,
and in the textbook condition, t(30) = 4.37, p < 0.001. The corresponding ANOVA
on the negative emotions also revealed a signicant interaction effect, F(2,96) = 4.37,
p = 0.015, ηp
2 = 0.084, which was a result of a signicant decrease in negative emotion
VR Video Textbook
Test score (%)
Learning condion
Pre-test Post-test Pre-test Post-test
VR VideoTextbook
Test score (%)
Learning condion
Figure 3. Percentage test scores and standard error mean (SEM) (error bars) for the
remembering questions (le) and for the understanding questions (right). VR, virtual reality.
D. Allcoat and A. von Mühlenen
8 Citation: Research in Learning Technology 2018, 26: 2140 -
(page number not for citation purpose)
(from 1.7 to 1.3) in the VR condition, t(30) = 4.20, p < 0.001, and no change in the
video or textbook condition (both p’s > 0.50).
Learning experience
Average WBLT ratings were grouped into the three categories ‘learning’, ‘design’
and ‘engagement’ and calculated separately for each category (see Figure 5). Three
separate one-way ANOVAs revealed a signicant effect of condition for each of the
three subscales (all p < 0.001). Post-hoc LSD tests showed that both learning and
engagement ratings were signicantly higher in the VR than in the textbook con-
dition (p = 0.005 and p < 0.001, respectively), and they were signicantly higher in
VR Video Textbook
Rang scale
Learning condion
Posive emoons
VR VideoTextbook
Rang scale
Learning condion
Negave emoons
Pre-test Post-test Pre-test Post-test
Figure 4. Mean rating and SEM (error bars) for positive emotions (le) and for negative
emotions (right).
VR Vide
Learning condion
WBLT evaluaon scale
Learning Design Engagement
Rang scale
Figure 5. Mean WBLT ratings and SEM (error bars) for learning, design and engage-
ment. WBLT, Web-based Learning Tools.
Research in Learning Technology
Citation: Research in Learning Technology 2018, 26: 2140 - 9
(page number not for citation purpose)
the textbook than in the video condition (p < 0.001 and p = 0.016, respectively). For
design, ratings were signicantly higher in the VR and textbook conditions than in
the video condition (both p < 0.001), but there was no difference between the VR and
the textbook condition.
Qualitative feedback
Qualitative data was also gathered; participants were asked as part of their online
questionnaire: ‘What did you think of the format of the learning materials/the equip-
ment used?’ The question was optional, and about half of the participants (n = 52)
gave some written feedback. Each participant who responded with qualitative feed-
back was grouped into positive, negative and mixed feedback, and the overall counts
for each category and condition are given in Table 2.
Multiple participants reported that the video learning material was ‘confusing’,
with one participant stating that it was ‘engaging but confusing’ and another say-
ing it was ‘difcult to navigate’. Participants described the textbook-style learning
materials as ‘basic’, ‘boring’ and ‘bland’. There were discrepancies in reports, with
some participants stating the materials were ‘clear’ and ‘easy to learn from’ but
others expressing that the materials were ‘unclear’ and the diagrams ‘weren’t very
helpful’. On the other hand, participants found that the VR was ‘difcult’ to use,
often clarifying ‘at rst’, but found it more ‘engaging’, with one participant stating
that it ‘made learning more exciting’ and another stating that it was ‘very useful and
The qualitative feedback suggests that because of the difculty using the equip-
ment ‘at rst’, future studies may benet from giving VR participants a trial period
with the equipment rst, to familiarise themselves with controls. Similarly, video
recording of VR would not be suitable as a primary learning condition (as opposed
to a control condition as in this study) because of the jarring and confusing nature of
the movement and interaction when they are not controlling it, with one participant
stating that the ‘video felt all over the place’.
The aim of this study was to consider the effects of using VR headsets for learn-
ing. Overall, participants in both the VR and the textbook-style conditions showed
better learning than participants in the video condition. Further breakdown of the
learning data showed that participants in the VR condition were better at ‘remem-
bering’ than those in the video and traditional conditions, and participants in both
VR and traditional conditions were better at ‘understanding’ than those in the video
Table 2. Number of participants who responded with qualitative feedback in grouped types:
positive, negative and mixed feedback.
Condition Positive Negative Mixed
Virtual 5 3 5
Video 2 13 2
Textbook 1 15 6
D. Allcoat and A. von Mühlenen
10 Citation: Research in Learning Technology 2018, 26: 2140 -
(page number not for citation purpose)
That the VR condition showed better test results compared to the video condi-
tion suggests that the learning in the VR condition is not a result of the graphics
or visuals of the equipment, as these were the same in both conditions. Instead, the
learning appears to be attributable to either the 3D immersion or the interactivity
of the VR environment. A further study may benet from comparing VR to other
active learning methods. This study compares interactive VR, an active learning
method, to passive video watching and traditional textbook-based methods. The
distinction between active learning and passive learning plays an important role in
many existing educational theories. There is evidence that active learning is bene-
cial to students (e.g. Pereira-Santos, Prudêncio and Carvalho 2017), which could
suggest that the benets found for VR are simply the benets of active learning.
However, active learning is not always found to be better than passive learning
(e.g.Haidet et al. 2004); therefore, the benets shown in VR may also be a result of
other factors.
The current results show a difference in learning stages as dened by Bloom’s tax-
onomy; further research into the other stages would be of interest. This study looked
at the lower ends of the learning hierarchy, remembering and understanding. VR may
compare differently to traditional methods for applying, analysing, evaluating and
creating. In particular, the 3D aspects of VR, along with the interactivity it affords,
may be benecial for ‘creating’ in many subjects. Alternatively, participants’ unfamil-
iarity with the equipment, which they hadn’t used before, may mean that improve-
ments of the VR condition were diminished, as individuals need time to adapt to new
technology systems (e.g. Cook and Woods 1996). This could explain why participants
in the VR condition were not signicantly better at ‘understanding’ compared to par-
ticipants in the traditional condition.
VR was also found to have a very positive impact on mood, with participants
having an overall increase in positive emotions and an overall decrease in negative
emotions. Conversely the other conditions showed a decrease in positive emotions.
Enjoyment has been previously linked as an important part of student performance
(e.g. Goetz et al. 2006; Valiente, Swanson, and Eisenberg 2012). This suggests that
using VR headsets can have a positive impact on the learning experience.
The WBLT Evaluation Scale also shows that engagement can be increased through
the use of VR. The importance of student engagement has been recognised previ-
ously (e.g. Kuh 2009; Strydom, Mentz and Kuh 2010; Wolf-Wendel, Ward and Kinzie
2009). Participants also rated the VR environment higher for learning, demonstrating
that they felt that they had learnt better from the VR. Student self-rating of learning
has been shown to be a valid measure of student performance (Benton, Duchon, and
Pallett 2013), with participants here reporting higher learning in the VR condition,
which was found for ‘remembering’.
The positive effects on emotion and engagement in VR are important benets
for both within and outside classroom learning (e.g. distance learning, self-teaching).
These aspects of learning are sometimes overlooked, with the focus being on other
outcomes, such as test scores. However, it has been demonstrated that individuals’
emotions, engagement and motivation are highly linked with each other and they are
all important aspects of learning (Pintrich 2003).
This research has demonstrated how VR can replicate or complement traditional
learning methods. It is important to consider how VR technology allows for learning
beyond the classroom. The technology, though suitable for classroom use, is also par-
ticularly suitable for distance learning, self-teaching and other learning environments.
Research in Learning Technology
Citation: Research in Learning Technology 2018, 26: 2140 - 11
(page number not for citation purpose)
This can be achieved, as the equipment can allow for rich, detailed learning environ-
ments that can be programmed to any scenario. Such VR environments can allow for
learning that could not be replicated in reality (e.g. dangerous environments or experi-
ments) or would be too costly to be accessible (e.g. expensive equipment or materials).
Future studies, for example, may want to consider the possible advantages of
the auditory options available with equipment such as VR, which were not utilised
for this project as they may be a confounding variable. As discussed, this could be
of interest in relation to learning styles, which are prevalent in a number of learn-
ing theories, though the concept of learning styles has received some criticism (e.g.
Pashler et al. 2008). Regardless of learning styles, there may be some benet to
including audio to increase immersion and engagement (e.g. Paterson and Conway
2014; Wharton and Collins 2011).
Many VR headsets also share the benets of mobile learning, most obviously the
VR headsets that run through mobile phones. Though not as powerful and capable
of detailed environments as PC-based VR headsets like the HTC Vive and Oculus
Rift, these mobile headsets share many of the same benets. Though the headset used
in the study, the HTC Vive, is currently only mobile with the use of a portable back-
pack PC, there is a new mobile, portable version of the HTC Vive headset called the
‘Vive Focus’. The Vive Focus is currently available for developers and is expected to
be released later this year, which means that applications such as the one used in this
study will be fully mobile, allowing for more exible learning.
Overall, VR does seem to be a potential alternative to traditional textbook-style
learning, with similar performance levels and improved mood and engagement.
These benets may have a longer-term impact on learning, such as improvements
resulting from the learning experience. However, the results may be partially because
of the novelty of the VR equipment, so the improvements may not be sustained over
longitudinal studies. Conversely, these improvements could increase over time, as
individuals become more familiar with the equipment and more able to navigate it
easily. Therefore, further longitudinal studies are needed to address these questions.
VR does show great potential, not only as an option to supplement or replace tra-
ditional learning methods, but to develop novel learning experiences that have not
been used before.
Anderson, L.W., et al., (2002) A Taxonomy for Learning, Teaching, and Assessing: A Revision of
Bloom’s Taxonomy of Educational Objectives, Longman, New York, vol. 41, p. 302.
Barbe, W., Milone, M. & Swassing, R. (1988) Teaching through Modality Strengths, Zaner-
Bloser, Columbus, OH.
Bellamy, M., & Warren, A. (2011) Using Online Practicals to Support Lab Sessions, Unpublished, [Online] Available at:
Benton, S., Duchon, D. & Pallett, W. (2013) ‘Validity of student self-reported ratings of learn-
ing’, Assessment & Evaluation in Higher Education, vol. 38, no. 4, pp. 377–388. doi:10.1080/
Bloom, B. S., et al., (1956) Taxonomy of Educational Objectives: The Classication of Edu-
cational Goals: Handbook I Cognitive Domain, Longmans, Green and Co LTD, London.
Camp, J. & Schnader, A. (2010) ‘Using debate to enhance critical thinking in the accounting
classroom: the Sarbanes-Oxley act and U.S. Tax Policy’, Issues in Accounting Education,
vol. 25, no. 4, pp. 655–675. doi:10.2308/iace.2010.25.4.655.
D. Allcoat and A. von Mühlenen
12 Citation: Research in Learning Technology 2018, 26: 2140 -
(page number not for citation purpose)
Cassidy, S. (2004) ‘Learning styles: an overview of theories, models, and measures’, Educational
Psychology, vol. 24, no. 4, pp. 419–444. doi:10.1080/0144341042000228834.
Cook, R. & Woods, D. (1996) ‘Special section: adapting to new technology in the operating
room’, Human Factors: The Journal of the Human Factors and Ergonomics Society, vol. 38,
no. 4, pp. 593–613. doi:10.1518/001872096778827224.
Gaytan, J. & McEwen, B.C. (2007) ‘Effective online instructional and assessment strategies’,
American Journal of Distance Education, vol. 21, pp. 117–132.
Goetz, T., et al., (2006) ‘A hierarchical conceptualization of enjoyment in students’, Learning
and Instruction, vol. 16, no. 4, pp. 323–338. doi:10.1016/j.learninstruc. 2006.07.004.
Haidet, P., et al., (2004) ‘A controlled trial of active versus passive learning strategies in a
large group setting’, Advances in Health Sciences Education, vol. 9, no. 1, pp. 15–27.
Hawk, T. & Shah, A. (2007) ‘Using learning style instruments to enhance student learning’,
Decision Sciences Journal of Innovative Education, vol. 5, no. 1, pp. 1–19. doi:10.1111/
Izard, C. E., et al., (1974) The Differential Emotions Scale: A Method of Measuring the Mean-
ing of Subjective Experience of Discrete Emotions, Vanderbilt University, Department of
Psychology, Nashville, TN.
Kanninen, E. (2008) Learning Styles and e-Learning, Master of Science Thesis, Tampere
University of Technology, vol. 12, pp. 1–76. doi:10.4018/978-1-4666-4313-0.ch020.
Kay, R. (2011) ‘Evaluating learning, design, and engagement in web-based learning tools
(WBLTs): the WBLT evaluation scale’, Computers in Human Behavior, vol. 27, no. 5,
pp.1849–1856. doi:10.1016/j.chb.2011.04.007.
Kharb, P. (2013) ‘The learning styles and the preferred teaching–learning strategies of rst year
medical students’, Journal of Clinical and Diagnostic Research, vol. 7, no. 6, pp. 1089–1092.
Kuh, G. (2009) ‘What student affairs professionals need to know about student engagement’,
Journal of College Student Development, vol. 50, no. 6, pp. 683–706. doi:10.1353/csd.0.0099.
Lujan, H.L. & DiCarlo, S.E. (2006) ‘First-year medical students prefer multiple learning styles’,
Advances in Physiology Education, vol. 30, pp. 13–16.
McMahan, A. (2003) ‘Immersion, engagement, and presence: a method for analyzing 3-D
video games’, in The Video Game Theory Reader, Routledge, London, pp. 67–86.
Mount, N., et al., (2009) ‘Learner immersion engagement in the 3D virtual world: principles
emerging from the DELVE project’, Innovation in Teaching and Learning in Information and
Computer Sciences, vol. 8, no. 3, pp. 40–55. doi:10.11120/ital.2009.08030040.
Pashler, H., et al., (2008) ‘Learning styles’, Psychological Science in the Public Interest, vol. 9,
no. 3, pp. 105–119. doi:10.1111/j.1539-6053.2009.01038.x.
Paterson, N., & Conway, F. (2014) ‘Engagement, immersion and presence: the role of audio
interactivity in location-aware sound design’, in Oxford Handbook of Interactive Audio,
eds K. Collins, B. Kapralos & H. Tessler, 1st edn., Oxford University Press, Oxford,
pp.263–280. doi:10.1093/oxfordhb/9780199797226.013.016.
Pereira-Santos, D., Prudêncio, R. & de Carvalho, A. (2017) ‘Empirical investigation of active
learning strategies’, Neurocomputing, [Online] Available at:
Pintrich, P. (2003) ‘A motivational science perspective on the role of student motivation
in learning and teaching contexts’, Journal of Educational Psychology, vol. 95, no. 4,
pp.667–686. doi:10.1037/0022-0663.95.4.667.
Riener, C. & Willingham, D. (2010) ‘The myth of learning styles’, Change: The Magazine of
Higher Learning, vol. 42, no. 5, pp. 32–35. doi:10.1080/00091383.2010.503139.
Scott, S., (2009) ‘Perceptions of students’ learning critical thinking through debate in a
technology classroom: a case study’, The Journal of Technology Studies, vol. 34, pp. 39–44.
Research in Learning Technology
Citation: Research in Learning Technology 2018, 26: 2140 - 13
(page number not for citation purpose)
Strydom, J. F., Mentz, M. & Kuh, G. D. (2010) ‘Enhancing success in higher education by
measuring student engagement in South Africa’, Acta Academica, vol. 42, pp. 1–13.
Truong, H. (2016) ‘Integrating learning styles and adaptive e-learning system: current develop-
ments, problems and opportunities’, Computers in Human Behavior, vol. 55, pp. 1185–1193.
Valiente, C., Swanson, J. and Eisenberg, N. (2011) ‘Linking students’ emotions and academic
achievement: when and why emotions matter’, Child Development Perspectives, vol. 6, no.2,
pp. 129–135. doi:10.1111/j.1750-8606.2011.00192.x.
Webster, R. (2015) ‘Declarative knowledge acquisition in immersive virtual learning environ-
ments’, Interactive Learning Environments, vol. 24, no. 6, pp. 1319–1333. doi: 10.1080/
Wharton, A., and Collins, K. (2011) ‘Subjective measures of the inuence of music customization
on the video game play experience: a pilot study’, Game Studies: the International Journal of
Computer Game Research, vol. 11, no. 2. [Online] Available from:
articles/wharton_Collins [Accessed: 16th November 2018]. doi: 10.4172/2161-0487.1000191.
Wolf-Wendel, L., Ward, K. and Kinzie, J. (2009) ‘A tangled web of terms: the overlap and
unique contribution of involvement, engagement, and integration to understanding col-
lege student success’, Journal of College Student Development, vol. 50, no. 4, pp. 407–428.
... VR learning games can engage learners in scientific practices, real-life problem solving, and reflection on their actions [31]. Among the motivations for VR use in education is allowing individuals facing cost prohibition [32], time constraints [33], inaccessible locations [34], risky activities, such as exploring cliffs and canyons [32], or hazardous training [35] to experience situations that would be otherwise impossible [36]. Desktop VR is an advantageous entry option for immersive learning since its virtual experiences can be delivered by computers, gaming devices, or any device with a Web browser and internet connection (i.e., WebGL interfaces). ...
... VR learning games can engage learners in scientific practices, real-life problem solving, and reflection on their actions [31]. Among the motivations for VR use in education is allowing individuals facing cost prohibition [32], time constraints [33], inaccessible locations [34], risky activities, such as exploring cliffs and canyons [32], or hazardous training [35] to experience situations that would be otherwise impossible [36]. Desktop VR is an advantageous entry option for immersive learning since its virtual experiences can be delivered by computers, gaming devices, or any device with a Web browser and internet connection (i.e., WebGL interfaces). ...
Conference Paper
A prototype desktop Virtual Reality (dVR) exploration game was developed as a curriculum enhancement activity for promoting watershed literacy with middle school students. It focuses on the spatial components, geography, and history of their local watershed. The dVR exploration game was implemented in the summer of 2021 in eastern USA during the COVID-19 pandemic with 35 learners aged 10–14 during summer school. Immediately before and after gameplay completion, the participants answered a 9-item watershed literacy measure assessing essential elements of watershed understandings. The preliminary findings revealed players’ improved ability to identify their own local watershed and how it connected to the ocean by rivers, creeks, and human-made structures. The watershed literacy measure was found to be a valid and reliable instrument.
... En contraste con los estudios anteriores, numerosos trabajos señalan avances exitosos en la integración de RVI en las aulas (Allcoat y von Mühlenen, 2018;Radianti et al., 2020;). A este respecto, Tang et al. (2020) afirman que la RVI puede mejorar las habilidades en análisis geométrico y la creatividad de los estudiantes, en comparación con los enfoques tradicionales. ...
Full-text available
El uso de la Realidad Virtual Inmersiva para la enseñanza y el aprendizaje de las matemáticas es una línea de creciente interés, en la que hacen falta investigaciones empíricas sobre el impacto de su inclusión en las aulas. Presentamos un estudio llevado a cabo con maestros en formación, en el que el software NeoTrie VR es utilizado en un grupo experimental para trabajar la noción de poliedro, siguiendo el modelo de Vinner. El análisis de datos cuantitativos muestra los errores más comunes en los que incurren los participantes a la hora de identificar las clases a las que pertenecen un conjunto de figuras tridimensionales. El contraste de resultados del grupo experimental con un grupo control arroja diferencias significativas, que implican que el trabajo con el software contribuye a la adquisición de una imagen mental más rica y conectada de los conceptos estudiados. Los maestros en formación del grupo experimental han podido identificar en mayor grado las características relevantes de los poliedros, sus tipos y conexiones entre ellos, así como reducir los errores detectados en la prueba inicial.
... Moreover, immersive VR can engage students and promote cognitive and affective learning outcomes (Di Natale et al., 2020) in formal education contexts (Krassmann et al., 2022). Among the motivations for VR use in education is allowing individuals to experience situations that would be otherwise impossible (Freina & Ott, 2015) due to cost prohibition (Allcoat & von Mühlenen, 2018), spatial inaccessibility (Zhao et al., 2020), and risky or hazardous limitations (Engelbrecht et al., 2019). However, using VR of any type in schools still faces barriers to adoption, including its state-of-the-art nature, teachers' preparation, as well as the availability of appropriate learning materials aligned to school curriculum standards (Jensen & Konradsen, 2018). ...
Full-text available
Digital game-based and gamified technologies hold much promise to promote the development of learners' conceptual knowledge. This efficacy study used desktop virtual reality (dVR) to investigate how the place-based digital gameful learning experience (DGLE) "Watershed Explorers", implemented as a curriculum enhancement activity, impacted students' watershed literacy as well as their attitudes towards playing and learning during dVR gameplay. Thirty-five middle school students completed a pretest-posttest content assessment measuring watershed literacy, a motivation to play with a dVR game survey, and a perception of learning with dVR games survey. Results showed that using "Watershed Explorers" as a curriculum enhancement activity can positively impact learning, especially middle school students' ability to identify their own local watershed and its connections to the ocean. Most participants perceived this DGLE as a helpful and engaging type of learning technology that can promote stimulating ways of presenting content materials and make school learning more interesting. Implications for the design of future iterations of “Watershed Explorers” to enhance learning are also discussed.
... Virtual reality is considered an effective tool to study perception [71] and performance [2]. However, VR studies either examine perceptual processes (e.g., [6,32,78]) or performance-related aspects (e.g., [35,58,65]). ...
While user's perception and performance are predominantly examined independently in virtual reality, the Action-Specific Perception (ASP) theory postulates that the performance of an individual on a task modulates this individual's spatial and time perception pertinent to the task's components and procedures. This paper examines the association between performance and perception and the potential effects that tactile feedback modalities could generate. This paper reports a user study (N=24), in which participants performed a standardized Fitts's law target acquisition task by using three feedback modalities: visual, visuo-electrotactile, and visuo-vibrotactile. The users completed 3 Target Sizes × 2 Distances × 3 feedback modalities = 18 trials. The size perception, distance perception, and (movement) time perception were assessed at the end of each trial. Performance-wise, the results showed that electrotactile feedback facilitates a significantly better accuracy compared to vibrotactile and visual feedback, while vibrotactile provided the worst accuracy. Electrotactile and visual feedback enabled a comparable reaction time, while the vibrotactile offered a substantially slower reaction time than visual feedback. Although amongst feedback types the pattern of differences in perceptual aspects were comparable to performance differences, none of them was statistically significant. However, performance indeed modulated perception. Significant action-specific effects on spatial and time perception were detected. Changes in accuracy modulate both size perception and time perception, while changes in movement speed modulate distance perception. Also, the index of difficulty was found to modulate all three perceptual aspects. However, individual differences appear to affect the magnitude of action-specific effects. These outcomes highlighted the importance of haptic feedback on performance, and importantly the significance of action-specific effects on spatial and time perception in VR, which should be considered in future VR studies.
In our previous publications, we introduced the concepts of Course Authoring Tools (CAT), and Didactical Structural Templates (DST) which are a further development of the CAT. DSTs are defined as a possibility to describe the didactical structure of a course, a study program, or an applied game in an abstract way. The idea of DSTs is based on the structure of IMS Learning Design (IMS-LD), which is a quasi-standard for modelling learning structures. We have shown what DSTs are useful for and that there is a need for an editor for DSTs which we already presented in combination with the Didactical Structural Template Manager (DSTM). In this paper, we will focus on how DSTs can be used as a structure for Applied Games (AG) as applied gaming content for an LMS course and as a stand-alone AG. We will demonstrate the possibility, of how to provide two different kinds of AGs: one that is based on a complete DST, and one that is based only on a part of a DST. Finally, this paper presents the relevant state of the art, the conceptual modeling, and the relevant implementations. The paper comes to a close with a summary and a list of the remaining challenges.
Full-text available
The Caribbean’s metaverse evolution accelerated due to the Covid-19 pandemic. This paper focuses on the metaverse, XR, and NFT and emphasises the Caribbean’s contribution to the virtual environment. A bibliometric analysis of metaverse-themed research identified the rapid increase in publications in 2021 and 2022 and that titles with XR (AR, VR or MR) occurred three times more than blockchain (including NFT). An evolving dataset was created based on a continuous scoping literature review of Industry 4.0 and its enabling technologies. This enables the creation of a new definition of the metaverse, understanding the UX benefits of XR and its applications' areas of foci, highlighting investment in XR-based projects, and illustrating the Caribbean-themed NFT and XR projects. This dataset revealed that UX benefits are linked to XR element features that are relevant, contextual, customised, hands-free and intuitive. It also revealed that XR applications have areas of foci that can enable machine control or data interface, designing and testing, remote support, education, customer engagement, remote collaboration or entertainment and escapism. Analysis of 54 XR papers revealed that the most popular area of focus was education (including training, learning and understanding). An evaluation of global investments in XR development showed funding ranged from USD 70K to USD 100M, and there needs to be focused financial support for Caribbean projects. This justifies continued research into factors influencing funding and encouraging Caribbean XR development. In addition, this research promotes regionally developed XR projects and NFTs. The paper's originality is the reductionist definition of the metaverse: a space designed for users by users, which can satisfy whomever, whatever, however, wherever and whenever. It manifests the user's extended reality, facilitated through XR technologies that enable Industry 4.0 (I4.0). As such, the metaverse can be considered the practical implementation of I4.0.
Full-text available
This year we are pleased to present our inaugural Practitioner Proceedings volume, which includes twenty-nine submissions from the Practitioner's track, at the 8th International Conference of the Immersive Learning Research Network (iLRN 2022). This Practitioner Proceedings volume is published by iLRN and separately from the IEEE-technically sponsored Academic Proceedings.
Children with reading and writing difficulties, such as dyslexia, have been directly affected by the Covid-19 situation because they could not have the teacher’s face-to-face support. Consequently, new devices and technological applications are being used in educational contexts to improve the interest of learning. This paper presents the design of a Virtual Reality Serious Game called DixGame. This game is a pedagogical tool specifically oriented to children between 8 and 12 years old with dyslexia. Two immersive mini-games are included in this game: a Whack-a-mole and a Memory, which try to improve different skills keeping the children focused on tasks. Whack-a-mole aims to work on the attention and visual and reading agility by recognizing correct letters and words. Memory aims to improve memory and attention ability by pairing letter-cards. The mini-game structure permits to incorporate new levels or games and the progressive increment of difficulty allows the autonomous treatment.KeywordsVirtual realitySerious gameDyslexiaEducationChildren
Conference Paper
Area measurement has a high priority in mathematics school education. Nevertheless, many students have problems understanding the concept of area measurement. An AR tool for visualizing square units on objects in the real world is developed to enable teachers to support understanding already in primary school. This work-in-progress paper presents the initial test version and discusses the first teaching experiment results. The students’ feedback and use of the app showed possible adaptations of the AR tool, e.g., that the idea of dynamic geometry could be incorporated in the future.
Full-text available
Many games and consoles today allow for a player to substitute a personal music playlist into the video game. We examined the influence that a player's choice of music has on the player's experience in one particular game, Fallout 3: Operation Anchorage. Players specifically chose music for the purpose of relieving anxiety, improving tactics and to experience immersion. Results showed that players were unable to predict what music would improve their immersion, but were able to choose appropriate music to influence game playing tactics and anxiety levels.
Full-text available
The author investigated the interaction effect of immersive virtual reality (VR) in the classroom. The objective of the project was to develop and provide a low-cost, scalable, and portable VR system containing purposely designed and developed immersive virtual learning environments for the US Army. The purpose of the mixed design experiment was to compare lecture-based and immersive VR-based multimedia instruction, in terms of declarative knowledge acquisition (i.e. learning) of basic corrosion prevention and control with military personnel. Participants were randomly assigned to the control group (N = 115) or investigational group (N = 25) and tested immediately before and after training. The author accessed learning outcomes from the pre-exam and post-exam scores and VR system usability from exit questionnaires. Results indicate that both forms of instruction will increase learning. VR-based did produce higher gain scores and there was a statistically significant interaction between instruction type and time.
Full-text available
While critical thinking is necessary for accountants in today's business world, cultivating students' critical thinking skills in an accounting classroom can be a challenge. The extant literature suggests that debate is a well-established pedagogical tool for enhancing student critical thinking skills, yet debate is not often used effectively in accounting classrooms. We provide suggestions for developing debates for use in the accounting classroom and two examples of debates used by the authors. The first requires students to argue for or against the extension of tax provisions currently being deliberated by Congress. The second requires students to examine the provisions of Sarbanes-Oxley and propose amendments to the bill.
Full-text available
This paper investigates the issues surrounding the use of 3D virtual worlds to enhance learner immersion through improved learner engagement. It is based on findings from the JISC-funded DEsign of Learning spaces in 3D Virtual Environments (DELVE) project at the University of Nottingham and the Open University. Given continued confusion about the term immersion, what it means for a learner to be immersed, and the relationship between immersion, presence and engagement, notions of immersion and engagement in 3D virtual environments are explored in the context of previous published studies ranging from virtual reality to psychology. The resultant improved understanding of the terminology is then used as the basis for coding results from a qualitative, inductive analysis of 20 students that undertook a substantive learning task in the virtual environment Second Life. Emergent themes from the analysis identify key factors that act to both enhance and restrict learner engagement in 3D virtual worlds and a set of principles for practitioners who wish to use 3D virtual environments to enhance learner engagement is presented.
Many predictive tasks require labeled data to induce classification models. The data labeling process may have a high cost. Several strategies have been proposed to optimize the selection of the most relevant examples, a process referred to as active learning. However, a lack of empirical studies comparing different active learning approaches across multiple datasets makes it difficult identifying the most promising strategies, or even assessing the relative gain of active learning over the trivial random selection of instances. In this study, a comprehensive comparison of active learning strategies is presented, with various instance selection criteria, different classification algorithms and a large number of datasets. The experimental results confirm the effectiveness of active learning and provide insights about the relationship between classification algorithms and active learning strategies. Additionally, ranking curves with bands are introduced as a means to summarize in a single chart the performance of each active learning strategy for different classification algorithms and datasets.
Improving student success and throughput rates are key challenges facing South African higher education. International research shows that a focus on student engagement can help to enhance student learning and other desired outcomes as well as the efficiency and effectiveness of higher education systems. This article documents the psychometric properties of the South African Survey of Student Engagement (SASSE), providing a sound basis on which to promote large-scale studies of student engagement-related interventions. Using this contextualized measure will allow South African institutions to engage in national and international benchmarking with countries such as the USA, Canada and Australia. c UV/UFS.
Learning styles which refer to students’ preferred ways to learn can play an important role in adaptive e-learning systems. With the knowledge of different styles, the system can offer valuable advice and instructions to students and teachers to optimise students’ learning process. Moreover, e-leaning system which allows computerised and statistical algorithms opens the opportunity to overcome drawbacks of the traditional detection method that uses mainly questionnaire. These appealing reasons have led to a growing number of researches looking into the integration of learning styles and adaptive learning system. This paper, by reviewing 51 studies, delves deeply into different parts of the integration process. It captures a variety of aspects from learning styles theories selection in e-learning environment, online learning styles predictors, automatic learning styles classification to numerous learning styles applications. The results offer insights into different developments, achievements and open problems in the field. Based on these findings, the paper also provides discussion, recommendations and guidelines for future researches.
Critical thinking is often a desired compe- tency for graduates of a technology program. Organizational members have uttered concern about students' inability to think critically. Although traditional pedagogical techniques, such as lectures and examinations, center on knowledge acquisition, debates in the technolo- gy classroom can effectively facilitate critical thinking. The purpose of this study was to gath- er via questionnaires the perceptions of technol- ogy students on the debate process used in the classroom to increase critical thinking. Ov erall, the students believed that the debate process was a useful learning activity. The results of the questionnaire revealed that students believed that the debates helped them understand the topic better, learn new knowledge, and gain an understanding of the debate process. In addi- tion, students thought that the debates increased their critical thinking skills.