ArticlePDF Available

Abstract and Figures

This study examined whether an interviewer’s nonverbal behavior influences observers’ competence ratings in a recruitment interview using 360-degree videos experienced with immersive virtual reality (VR-cardboard) and 2D screen displays. Participants ( n = 110) observed a recruitment interview and assessed three competences of the applicant (behavior in a team, customer care, and sales skill). We used a 2 × 2 design with the nonverbal behavior (positive vs. negative) of the interviewer and display type (VR-cardboard vs. 2D screen display) as between-subjects factors. After observing interview sequences and providing competence ratings, participants also rated different aspects of immersion using the augmented reality immersion questionnaire (ARI; Georgiou and Kyza in Int J Hum Comput Stud 98: 24–37, 2017) and their overall satisfaction with the experience. For two of the three competences (customer care and behavior in a team), we found that observers gave higher competence ratings when the interviewer’s nonverbal behavior was positive compared to when it was negative. This social influence effect was similar for 360-degree videos experienced with immersive VR and 2D screen displays. VR resulted in higher immersion than 2D screen displays regarding the dimensions of flow and presence . Our results suggest that the ARI questionnaire can be used to reliably assess 360-degree videos experienced with immersive VR and 2D screen displays.
Content may be subject to copyright.
Vol.:(0123456789)
1 3
Virtual Reality (2022) 26:669–686
https://doi.org/10.1007/s10055-021-00540-3
S.I. : VR ANDCOGNITIVE SCIENCE
Nonverbal behavior ofinterviewers influences thecompetence ratings
ofobservers inrecruitment interviews: astudy investigating social
influence using 360‑degree videos withvirtual reality and2D screen
displays
ThomasWyssenbach1 · MelinaZeballos1 · StefanLoosli1 · AdrianSchwaninger1
Received: 28 October 2020 / Accepted: 11 May 2021 / Published online: 19 June 2021
© The Author(s) 2021
Abstract
This study examined whether an interviewer’s nonverbal behavior influences observers’ competence ratings in a recruitment
interview using 360-degree videos experienced with immersive virtual reality (VR-cardboard) and 2D screen displays. Par-
ticipants (n = 110) observed a recruitment interview and assessed three competences of the applicant (behavior in a team,
customer care, and sales skill). We used a 2 × 2 design with the nonverbal behavior (positive vs. negative) of the interviewer
and display type (VR-cardboard vs. 2D screen display) as between-subjects factors. After observing interview sequences and
providing competence ratings, participants also rated different aspects of immersion using the augmented reality immersion
questionnaire (ARI; Georgiou and Kyza in Int J Hum Comput Stud 98: 24–37, 2017) and their overall satisfaction with the
experience. For two of the three competences (customer care and behavior in a team), we found that observers gave higher
competence ratings when the interviewer’s nonverbal behavior was positive compared to when it was negative. This social
influence effect was similar for 360-degree videos experienced with immersive VR and 2D screen displays. VR resulted in
higher immersion than 2D screen displays regarding the dimensions of flow and presence. Our results suggest that the ARI
questionnaire can be used to reliably assess 360-degree videos experienced with immersive VR and 2D screen displays.
Keywords Virtual reality· 360-degree video· Social influence· Nonverbal behavior· Immersion· Job recruitment
interview
1 Introduction
Virtual reality (VR) bears the potential to disrupt today’s
education (Allcoat and Von Mühlenen 2018; Checa and
Bustillo 2020; Radianti etal. 2020). It allows students to
experience and engage in virtual scenarios as if they were
there (Jerald 2016; Rupp etal. 2019). VR allows educa-
tors to deliver learning content in a more immersive way
than slides, texts, or 2D videos. A promising VR applica-
tion for students in higher education is to increase aware-
ness of possible biases in potential future workplace situ-
ations (Innocenti etal. 2012; Salmanowitz 2018), such as
judging applicants’ competences in a recruitment interview.
The main task of the rater in a recruitment interview is to
judge the competences of an applicant objectively (Dipboye
2017). Research on situations other than job interviews has
shown that people are influenced by their environment
(Cialdini 2001). One important aspect is social influence
(e.g., Cialdini and Goldstein 2004, for an overview of social
influence), which often occurs through nonverbal behavior
(Guyer etal. 2019). To our knowledge, no previous study has
investigated whether the nonverbal behavior of interviewers
influences observers who must rate the competences of a job
applicant in a recruitment interview. To close this gap, we
conducted a study in which participants observed a recruit-
ment interview and rated three competences of the inter-
viewed person (customer care, behavior in a team, and sales
skill). We used a 2 × 2 design with the nonverbal behavior
(positive vs. negative) of the interviewer and display type
(VR-cardboard vs. 2D screen display) as between-subjects
* Thomas Wyssenbach
thomas.wyssenbach@fhnw.ch
1 School ofApplied Psychology, University ofApplied
Sciences andArts Northwestern Switzerland, Olten,
Switzerland
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
670 Virtual Reality (2022) 26:669–686
1 3
factors. In the remaining introduction, we present the edu-
cational background of this study and summarize previous
research on job interviews, social influence through nonver-
bal behavior, and immersive VR versus 2D displays before
explaining the current study and the expected results.
1.1 Educational background ofthis study
Traditional classroom-based lecturing uses slides, texts, and
2D videos. The use of VR can increase engagement and
enjoyment, promote self-learning, and enable multisensory
learning (Papanastasiou etal. 2019). This is particularly use-
ful for theories and concepts that are related to perception,
action, and decision making in real-life situations (for further
potentials of VR-use in education see, e.g., Freina and Ott
2015; Martín-Gutiérrez etal. 2017; Jensen and Konradsen
2018; Markowitz etal. 2018; Radianti etal. 2020; Wu etal.
2020). We applied a total of four interactive VR modules
in a lecture on general psychology for first-semester under-
graduate psychology students. With the first VR module, the
students were introduced to VR technology by interactively
mixing colored lights in a dark room and viewing different
colored objects in a virtual environment. In a session two
weeks later, we introduced two subsequent VR modules to
demonstrate biases of distance perceptions under various
environmental conditions (e.g., daylight vs. night; detailed
vs. limited visual context). The current study evaluates the
fourth VR module, which was deployed again two weeks
later. Its goal was to raise awareness of social influence bias
by nonverbal behavior in a job recruitment interview. This
module combines the topics of cognitive interpretations of
perceived visual and auditive stimuli with a topic relevant
to the students’ future careers (occupational, organizational,
and business psychology students). Furthermore, the VR
module set up for the study served multiple purposes in the
class-based lecture. The students’ ratings of the evaluated
competences were summarized as graphs in real time and
used to discuss rating biases and their implications. Addi-
tionally, the disclosure and explanation of the study design
introduced the students to key elements of experimental
designs. Finally, students received first-hand experience
using VR in a non-game application.
1.2 Job recruitment interviews andnonverbal
behavior
Job recruitment interviews continue to be one of the most
widely used methods for evaluating the fit between potential
applicants’ competences and job requirements (Arvey and
Campion 1982; Schmidt and Hunter 1998; Ryan etal. 1999;
Salgado etal. 2004; Macan 2009). In fact, “the interview is
the most-used procedure in personnel selection across all
countries. Job applicants expect to be interviewed; managers
expect to interview some or all candidates” (Viswesvaran
and Ones 2018, p. 460). For example, a survey conducted
among human resources (HR) personnel in the German-
speaking part of Switzerland revealed that 99% of the
respondents included a form of interview in their selection
processes, while 71% of these were at least semistructured
(König etal. 2010, p. 23). A meta-analysis by Hausknecht
etal. (2004) revealed that candidates also perceive inter-
views favorably compared to alternative selection methods
(see also Dipboye etal. 2012). However, job recruitment
interviews differ in several aspects. One important aspect
is standardization across candidates regarding the questions
asked and the evaluation (Campion etal. 1997). A structured
interview enhances reliability and predictive validity (e.g.,
Campion etal. 1988; McDaniel etal. 1994; Schmidt and
Hunter 1998). Therefore, structured interviews are often the
preferred choice in practice. Interview formats with two or
more interviewers/observers (also called panel interviews)
enhance the accuracy of the assessment (Huffcutt etal.
2013). Moreover, applicants perceive that interviews with
more than one interviewer are more fair (Blackman 2017,
p. 187).
An essential part of communication and human interac-
tion is nonverbal (see Bonaccio etal. 2016, for an over-
view in the workplace context). Nonverbal behavior is an
effective means of social influence: it affects recipients’
attitudes, behaviors, and beliefs (e.g., Guyer etal. 2019).
Nonverbal cues are mostly emitted unintendedly and uncon-
sciously. Visually observable positive nonverbal behaviors
are related, for example, to eye gaze, nodding of the head in
agreement, and smiling, while negative nonverbal cues can
be expressed with a furrowed brow, stern look, and tighten-
ing of the mouth (Butler and Geis 1990, p. 49; Owen etal.
2012, p. 366). Whereas many studies have analyzed the
effects of applicants’ nonverbal behavior on interviewers
ratings (e.g., Parsons and Liden 1984; Gifford etal. 1985;
DeGroot and Gooty 2009; Huffcutt etal. 2011, for a more
advanced model), only a few studies have investigated the
effects of interviewers’ behavior on the applicant (e.g., Liden
etal. 1993). To our knowledge, no study has investigated
whether social influence through the nonverbal behavior of
interviewers influences fellow interviewers or raters.
1.3 Display types andperceived immersion
VR enables users to immerse themselves in and experience
virtual content as if they were there (Jerald 2016; Lanier
2017; Slater 2018). VR offers new ways of studying human
perception and behavior (e.g., Blascovich etal. 2002; Pan
and Hamilton 2018). Highly realistic VR content includes
360-degree videos, also called spherical or immersive vid-
eos (e.g., Violante etal. 2019). This type of content has
limited natural interaction capabilities typically associated
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
671Virtual Reality (2022) 26:669–686
1 3
with immersive VR. However, 360-degree videos allow
users to look around at all angles from the location where
the 360-degree camera captured the footage. In addition to
experiencing 360-degree videos on an immersive VR head-
set (e.g., VR-cardboard), users can view the content on a 2D
screen display (e.g., notebook or tablet display). Instead of
simply turning the head naturally to change perspective in
VR, the viewing angle is controlled on a 2D screen display
using the computer mouse (notebook computer) or a touch
screen (tablet computer). Immersive VR has the advantage
that visual distractions from the user’s real environment are
not perceived. The user’s visual focus remains on the virtual
world.
Only a few studies have compared immersive VR and
traditional 2D screen displays. For example, Schutte and
Stilinović (2017) found that VR evoked significantly more
empathy than 2D screen displays when participants watched
a documentary about a girl in a refugee camp. Niu etal.
(2019) compared subjective emotions evoked by short
film sequences in immersive VR and 2D screen displays
and found that immersive VR caused significantly higher
arousal (intensity of the emotion) than the same sequences
viewed on a 2D screen display. Research comparing VR to
real-world experiences showed high ecological validity of
VR. For example, watching a 360-degree video of a scenic
landscape in immersive VR evoked similar emotions as see-
ing it in reality (Chirico and Gaggioli 2019). Kinateder and
Warren (2016) studied socially influenced human behavior
in evacuation scenarios. Their findings also yielded similar
behaviors in immersive VR as in real life, supporting the
ecological validity of immersive VR compared to real-life
behavior. Other studies compared job interviews conducted
as videoconferences on 2D screen displays and conducted
face-to-face. For example, Sears etal. (2013) and Black-
smith etal. (2016) found that applicants received better
interview ratings in face-to-face interviews. These authors
argue that technology use limits the perception of inter-
personal skills, such as effectively reading nonverbal cues.
Overall, studies suggest that nonverbal cues are less effec-
tive in 2D screen video-mediated communications than the
face-to-face equivalent (e.g., Doherty-Sneddon etal. 1997;
Bos etal. 2002; Proost etal. 2020).
One reason for the better performance of VR compared to
2D screen displays could be the degree of immersion (e.g.,
Kumar etal. 2017). From a technical perspective, immersion
in VR can be described as “the extent to which the computer
displays are capable of delivering an inclusive, extensive,
surrounding and vivid illusion of reality to the senses of
a human participant” (Slater and Wilbur 1997, p. 604ff).
However, more recent research characterizes immersion
as a psychological state that “people attain when they are
intensely involved in an activity and are experiencing a high
level of enjoyment and fulfillment” (Cheng etal. 2015, p.
234ff). Based on game immersion theory, Brown and Cairns
(2004) operationalized immersion on three levels (engage-
ment, engrossment, and total immersion). Each level has
specific barriers that players need to overcome before reach-
ing the next stage. Cheng etal. (2015) refined this concept
for video games and learning applications and shaped sub-
factors for each of the three levels. They described engage-
ment as the first stage of immersion: users are interested in
the topic and style of the application. The second stage is
engrossment, in which users focus on the application’s con-
tent and gradually lose awareness of their environment and
self-perception. Total immersion represents the third and
final stage: users experience feelings of presence (a sense
of actually being there) and detach entirely from reality
(Cheng etal. 2015, p. 235). Existing questionnaires meas-
uring aspects of immersion for VR were typically designed
to evaluate fully interactive VR applications (e.g., Sutcliffe
etal. 2005) in immersive VR only (see also Suh and Prophet
2018; Schwind etal. 2019, for overviews of different scales).
Only a few studies have evaluated aspects of immersion
based on game immersion theory using VR or comparing
VR to other media. However, a few recent publications have
successfully applied Georgiou and Kyza’s (2017) augmented
reality immersion (ARI) questionnaire (Voit etal. 2019) or
parts of it (Gallup etal. 2019; Tennant etal. 2020). Thus, in
the current study, we slightly adapted and applied the ARI
questionnaire (see Method for details).
1.4 Current study
We investigated whether the nonverbal behavior of an inter-
viewer influences the competence ratings of observers in a
recruitment interview using 360-degree videos. We used a
2 × 2 design with the nonverbal behavior (positive vs. nega-
tive) of the interviewer and display type (VR-cardboard vs.
2D screen display) as between-subjects factors. Participants
acted as interview observers who had to rate the applicant’s
competences (behavior in a team, customer care, sales skill)
in the recruitment interview. To assess perceived immersion
depending on the display type (VR-cardboard vs. 2D screen),
participants completed the ARI questionnaire (Georgiou and
Kyza 2017). Additionally, they rated their overall satisfac-
tion with the experience.
1.5 Research questions andexpected results
Based on the reviewed literature mentioned above, we
defined our research questions and hypotheses: First, does
the nonverbal behavior of interviewers influence the com-
petence ratings of observers in recruitment interviews? We
expected that positive nonverbal behavior of the interviewer
would lead to higher competence ratings than negative non-
verbal behavior of the interviewer (and vice versa; e.g.,
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
672 Virtual Reality (2022) 26:669–686
1 3
Cialdini and Goldstein 2004; Cialdini 2007). Second, does
display type (VR-cardboard vs. 2D screen display) influence
the competence ratings of observers in recruitment inter-
views? Previous studies reported higher performance ratings
for face-to-face interviews compared to technology mediated
interviews (Sears etal. 2013; Blacksmith etal. 2016). How-
ever, to our knowledge, no previous study examined whether
display type (VR-Cardboard vs. 2D screen display) influ-
ences the competence ratings of observers in recruitment
interviews. Therefore, we did not formulate a hypothesis for
the second research question. Third, is there an interaction
between nonverbal behavior and display type on competence
ratings of observers in recruitment interviews? Because
research on this question is also scarce, we did not formulate
a hypothesis on this third research question. Fourth, does VR
result in higher immersion than 2D screen displays? The first
two stages of immersion (engagement, engrossment) focus
mainly on content- and application-related topics (Brown
and Cairns 2004; Cheng etal. 2015), which were identical
for the immersive VR and 2D screen display conditions.
Thus, we expected to find higher perceived immersion on
the subscales of flow and presence (both related to total
immersion) for VR because the technology is capable of
making users experience a feeling of being there and detach-
ing from reality (Slater and Wilbur 1997). Fifth, can the
ARI questionnaire (Georgiou and Kyza 2017) be applied
to compare 360-degree videos displayed with VR versus
displaying them on 2D screens? We expected that applying
the ARI scales would differentiate between immersive VR
and 2D screen display (construct validity) on the relevant
subscales of total immersion and that the ARI scales would
show good reliability.
2 Method
2.1 Participants
One hundred and seventeen (117) first-semester psychology
students simultaneously participated in the experiment in the
General Psychology I lecture. Participation was voluntary
and took place during the scheduled lecture time. All par-
ticipants gave informed consent before starting the experi-
ment. Seven students did not complete the questionnaire due
to technical issues during the experiment (e.g., smartphone
incompatibility with VR-cardboard, low battery); thus, we
excluded their data. The final sample consisted of 110 partic-
ipants (n = 110; 74% female; age: M = 24.1years, SD = 4.6).
All had previously experienced VR-cardboard by conduct-
ing a cardboard VR module on color and two modules on
distance perception in the same lecture two weeks before
the experiment. The study complied with the American Psy-
chological Association Code of Ethics and was conducted
in accordance with the European General Data Protection
Regulation.
2.2 Experimental design
We used a 2 × 2 design, with the interviewer’s nonverbal
behavior (positive vs. negative) and display type (VR-card-
board vs. 2D screen display) as between-subject factors.
The participants rated three competences of the applicant
(behavior in a team, customer care, and sales skill) based on
360-degree videos of a job recruitment interview. Finally,
the participants completed the ARI questionnaire (Georgiou
and Kyza 2017) on perceived immersion and answered one
item on overall satisfaction with the experience.
2.3 Procedure
The experiment was conducted in a lecture. We welcomed
the students and explained that they would be observers of a
job interview to provide competence ratings of a job appli-
cant. We told the participants that they would be randomly
distributed to the VR-cardboard or the 2D screen condi-
tion and that they could experience the experiment later at
home using both display types if they wished to do so. All
participants were asked to remain seated at their table in
the classroom during the experiment. We explained to the
participants that they could stop the experiment anytime
and should stop the experiment immediately if they felt any
discomfort (this was important for the immersive VR con-
ditions because, in rare cases, participants can feel adverse
effects such as sensation disorientation, oculomotor issues,
or visually induced motion sickness; e.g., Lubeck etal. 2015;
Kim etal. 2018). To our knowledge, this was not the case;
all participants without technical difficulties completed the
experiment. Six persons were available on-site to help in
case of technical issues. After the instructions, the partici-
pants logged into a web-based survey and education plat-
form (developed by Waimanoo 2020) using their notebook
or tablet computer and an anonymous ID. The platform ran-
domly distributed the participants to four groups: 2D screen
display with positive nonverbal behavior, 2D screen display
with negative nonverbal behavior, VR-cardboard with posi-
tive nonverbal behavior, and VR-cardboard with negative
nonverbal behavior. All participants were then asked to con-
nect their headphones to the device (notebook, tablet, or
smartphone) and adjust the audio volume to a comfortable
level based on an audio sample. Participants in the immer-
sive VR condition then mounted their smartphone on their
VR-cardboard (all students had received a Google cardboard
V2, Google 2020a, in the previous lecture two weeks ear-
lier). The experiment started for all students simultaneously
(Fig.1).
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
673Virtual Reality (2022) 26:669–686
1 3
In the VR-cardboard conditions, participants could
change the viewpoint by moving their head. Interaction ele-
ments, such as buttons, were activated by either maintaining
the look-at position for three seconds (also called “fusing”;
with a cursor animation; Google 2020b) or clicking on a
physical button positioned at the top right-hand side of the
VR-cardboard. To ensure easy usability for both display
types and keep the interaction modalities comparable, in the
2D screen display condition, participants could move their
viewpoint with a mouse or touchscreen (click-and-drag).
They could select buttons with a screen-centered cursor. A
click on the left mouse button, a tab on the screen (tablets),
or the keyboard’s enter key was used to activate a button.
Once ready, a click on an “OK” button started the experi-
ment with the first 360-degree video (introduction video;
Fig.2a), welcoming the applicant into the room and intro-
ducing the interview setting and the persons involved. Then,
three video sequences were displayed (one for each compe-
tence; Fig.2b, for an example); the order of these sequences
was counterbalanced across participants using Latin square
(Jones and Kenward 2015, p. 140). After each interview
sequence, the video faded out again, and instructions
appeared to rate the corresponding competence (Fig.2c, for
an example). Once answered, a click on an “OK” button
started the next video sequence until all three competences
were rated. A brief closing video represented the end of the
interview setting. Participants in the VR-cardboard condition
were asked to remove the VR-cardboard and continue on
their notebook or tablet computer. Subsequently, all partici-
pants completed the ARI questionnaire (Georgiou and Kyza
2017) and rated an extra item on the overall satisfaction with
the experience on their notebook or tablet computer.
2.4 Materials
As visual and auditory stimuli, multiple 360-degree videos
were created in cooperation with Waimanoo (2020) to rep-
resent realistic video sequences of a recruiting interview.
The 360-degree camera captured the videos from a natu-
ral viewing position of the rater or the actual participant
(Fig.2). We did not manipulate the camera position for all
clips to reduce the risk of simulator sickness (e.g., Lubeck
etal. 2015). To retain the participant’s attention on the rel-
evant actions, we kept the job interview setting’s surround-
ings simple (Fig.3) with a minimum of visual distractions
(Pirker etal. 2020). The interview setting is a generic office
environment consisting of a table, a fictional company logo,
a plant, and a round table with the three protagonists: the
interviewer, the applicant, and the rater. From the rater’s
perspective, this setup allowed the participants to observe
the interviewer and applicant simultaneously, even with the
limited horizontal field of view of a VR-cardboard (between
60° and 100°, depending on the specific smartphone used;
Powell etal. 2016).
We captured the video footage with a dual-lens GoPro
Fusion camera and then stitched the content to monoscopic,
spherical 360-degree videos (for a general overview of typi-
cal capturing workflows, see Feurstein 2019). An Olympus
LS-P4 recorded the audio. The separate video and audio
materials were merged and edited (for illustration, see Fig.4)
using Adobe Premiere Pro version 13.1.3. Furthermore, we
aligned the center of the “primary action” (Pirker etal. 2020,
p. 323) to the viewer’s forward view. To minimize potential
technical issues with the Wi-Fi bandwidth in the classroom
and limited cache on the students’ devices, we lowered the
resolution to 1920 × 960pixels and set the target bitrate to
Fig. 1 Photograph of the experiment in the classroom lecture
Fig. 2 Screenshots of the 360-degree videos used in the study. a Picture of the introduction video showing the job applicant entering the room.
b Example picture of the interview. c Question and ten-point Likert scale for rating the competence behavior in a team
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
674 Virtual Reality (2022) 26:669–686
1 3
5Mbps, which was a compromise for good image quality
combined with high functionality on the variety of student
devices used during the experiment.
We implemented the scenario of a job applicant applying
for a salesperson position in an electronic retail store. We
chose this scenario because first-semester students in higher
education have had experiences with salespersons in such a
store. Thus, it could be assumed that they could imagine an
appropriate job requirement profile. In addition to technical
knowledge, salespersons should also exhibit competences in
other categories, such as working well in a team, interacting
with customers, and sales skills (for a meta-analysis of the
determinants of salespersons’ performance, Churchill etal.
1985; for a study analyzing situational interviews for a sales-
related job, Weekley and Gier 1987; for detailed job analysis
for sales-related jobs, Phillips 1992; for a study evaluating
a skill-set of sales-related jobs, Rentz etal. 2002). Accord-
ingly, we chose three specific competences for each of these
three categories: behavior in a team, customer care, and sales
skill.
The first 360-degree video served as an introduction (the
same video for all test conditions) and aimed to familiarize
Fig. 3 Layout of the interview
setting. The interviewer, the job
applicant, and the rater were
located around a table. The
360-degree camera was placed
at the position of the rater’s
chair. The room’s setting was
kept simple, with a minimum of
visual distractors
Rater
(360-degree camera)
Fictional
company
logo
Interviewer
Job applicant
Entrance
Plant
Table
Fig. 4 Editing process of the 360-degree video footage. All scenes
were recorded multiple times. The content was then assembled and
edited using Adobe Premiere Pro’s masking functions (red frame;
depending on the nonverbal behavior). In the final videos, only the
visual and auditive footage of the interviewer differed between the
positive and negative nonverbal behavior conditions
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
675Virtual Reality (2022) 26:669–686
1 3
the rater with the virtual scene and build rapport in the inter-
view setting (e.g., Barrick etal. 2010; Swider etal. 2016).
The interviewer (female; professional theater actress) wel-
comed the participant and explained the interview proce-
dure. Then, a door buzzer sounded, the door opened, and
the interviewer welcomed the applicant (male; professional
theater actor). After they both sat down at a table, the inter-
viewer briefed the applicant and the rater on the proce-
dure. This introduction video ended after 1min 32s. The
360-degree content blurred out, and written text appeared
with instructions for the participant to focus on the upcom-
ing competence rating. A click on the “OK” button started
the video sequence for the first competence rating. For each
of the three competences (each video sequence was 42s), the
interviewer asked one of the following questions.
Behavior in a team: For this competence, the interviewer
asked a situational question derived from Campion etal.
(1994, p. 999): “Suppose you had an idea for a change in
work procedure to enhance quality, but there was a problem
in that some members of your work team were against any
type of change. What would you do in this situation?”.
Customer care: This question was derived from an exam-
ple noted in Weekley and Gier (1987, p. 485): “A customer
comes into the store to pick up a watch he had left for repair.
The repair was supposed to have been completed a week
ago, but the watch is not yet back from the repair shop.
The customer becomes very angry. How would you handle
this situation?” For a better fit in the chosen scenario, we
replaced the term “watch” with “smartphone.”
Sales skill: The interviewer’s question posed for this com-
petence was derived from Campion etal. (1997, p. 668):
“Suppose you were giving a sales presentation and a difficult
technical question arose that you could not answer. What
would you do?”.
The applicant’s reply to the question was the same in all
four conditions within every competence. We defined and
evaluated the replies to each competence question with HR
experts. The applicant’s replies corresponded to answers
rated as average and contained no salient cues related to
high or low performers. For each competence, we edited
two versions (positive nonverbal behavior of the interviewer,
negative nonverbal behavior of the interviewer) using Adobe
Premiere Pro’s masking functions. Only the interviewer’s
visual and audio differed between the two versions, while all
other content remained the same. The interviewer responded
to the applicant’s reply by giving nonverbal cues only. For
the positive nonverbal behavior condition, the interviewer
showed facial expressions (for an overview on facial expres-
sions, see Cohn etal. 2007), such as cheek raising, head
up, and head down, along with other nonverbal visual cues
of pleasure (for studies applying these cues, see Butler and
Geis 1990; Owen etal. 2012), such as friendly eye gaze
to the rater, nodding, and authentic smiles (see Fig.4, top
right). For the negative condition, the interviewer showed
facial expressions such as brow lowering, chin raising, and
lip tightening, along with other nonverbal cues of displeas-
ure, such as giving stern looks to the rater, frowning, or
humming quietly (negative nonverbal behavior; see Fig.4,
bottom right). To keep the credibility of the interviewer
as perceived by the observer consistent (Williamson etal.
2013), we maintained the intensity and number of nonverbal
indications for all conditions (Borkenau and Liebler 1995;
Larsen and Shackelford 1996; for examples, see Fig.5 in
Appendix).
In the closing video (length: 30s), the interviewer said
goodbye to the applicant, who left the room. The interviewer
told the participant (rater) to continue on his/her notebook or
tablet computer for the final questionnaire. The total length
of the 360-degree videos per participant was 4min 18s.
For the immersive VR conditions, the total time wearing a
VR-cardboard was under 8min (avoiding potential adverse
effects on the participants’ ocular system; see, e.g., Turn-
bull and Phillips 2017; Elias etal. 2019), including reading
instructions and rating the competences.
All video sequences were evaluated by three experts (one
professor and two research scientists) working in the field
of personnel assessment. They unanimously confirmed that
the videos corresponded to a realistic interview setting, the
interviewer asked realistic questions, the applicant’s replies
were short but realistic, and that the nonverbal behavior of
the interviewer was clearly identifiable as either positive or
negative. The experts also mentioned that professional inter-
viewers would often pose follow-up questions and that their
facial expressions would be less pronounced.
2.5 Measures
Competence ratings: We asked the participants to rate the
applicant’s competencies (behavior in a team, customer
care, sales skill) directly after the presentation of the spe-
cific video sequence on a ten-point Likert scale ranging from
“low” (1) to “high” (10), see Fig.2, right for an example.
Immersion questionnaire: We applied the ARI ques-
tionnaire (Georgiou and Kyza 2017) to measure perceived
immersion aspects based on game immersion theory (Brown
and Cairns 2004; Cheng etal. 2015). Cronbach’s
𝛼
-values
reported by Georgiou and Kyza (2017) are shown in Table1.
The model consists of three main factors, each with two
subscales: engagement is composed of usability and interest,
engrossment consists of emotional attachment and focus of
attention, and total immersion includes flow and presence.
We used the items of the ARI questionnaire (Georgiou
and Kyza 2017) and replaced the term “AR” with “VR”
in the four items mentioning AR. A native English and
German speaker translated the items into German (initial
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
676 Virtual Reality (2022) 26:669–686
1 3
translation; see, e.g., Beaton etal. 2000). The items were
then translated back to English by another native speaker
of both languages (back-translation), followed by a review
by the authors. Table6 in Appendix contains the Eng-
lish and German items (the latter were used in the current
study). We used a seven-point Likert scale ranging from
“totally disagree” (1) to “totally agree” (7), identical to
Georgiou and Kyza (2017).
Overall satisfaction: To gather an overall impression of
whether the participants liked the experience, we added
one extra item at the end of the survey: “How did you
like the VR module [with VR-cardboard/on the 2D screen
display]?” The text indicated in square brackets changed
depending on the display type condition. A seven-point
Likert scale ranging from “very bad” (1) to “very good”
(7) was applied.
2.6 Analysis
For psychometric and descriptive statistics, analyses of vari-
ance (ANOVAs), and post hoc comparisons, we used the
free and open statistical software JAMOVI version 1.2.27.0
(The Jamovi Project 2020) with alpha set to 0.05. We report
the effect sizes of ANOVAs using η2
p (partial eta-squared)
and interpret η2
p-effect sizes of .01, .06, and .14 as small,
medium, and large effects, respectively (Cohen 1988, p.
368). Significant interaction effects of the ANOVAs were
further investigated with post hoc tests using Holm–Bonfer-
roni corrections (Holm 1979).
3 Results
All analyses refer to the same data set; thus, the participants
per condition remained the same for all analyses. No outliers
were detected or removed.
3.1 Effects ofnonverbal behavior anddisplay type
oncompetence ratings
Table2 shows the number of participants per condition,
means, standard deviations, and results of two-way ANOVA
statistics (according to the American Psychological Asso-
ciation 2020) for the competence ratings as dependent
variables.
We conducted two-way between-subject ANOVAs to
explore the impact of nonverbal behavior and display type
on the three competence ratings using two-way independ-
ent ANOVAs (Table2). We found significant main effects
of the interviewer’s nonverbal behavior on the competence
ratings of the job applicant with regard to behavior in a team
Table 1 Cronbach’s
𝛼
Values of the ARI Scales and Subscales
Reported by Georgiou and Kyza (2017, p. 33)
n =162 (Georgiou and Kyza 2017, p. 30)
Scales Cronbach’s
𝛼
Engagement .77
Interest (4 items) .80
Usability (4 items) .82
Engrossment .88
Emotional attachment (3 items) .76
Focus of attention (3 items) .87
Total immersion .82
Presence (4 items) .80
Flow (3 items) .87
Table 2 Number of participants,
means, standard deviations, and
two-way ANOVA statistics for
evaluation of competences
n = 110. ANOVA analysis of variance, DT display type, NVB nonverbal behavior
*p < .05
Variable DT: VR-cardboard DT: 2D screen AN O VA
n M SD n M SD Effect F(1, 106) η2
p
Behavior in a team
NVB: negative 27 4.85 1.70 25 5.16 2.44 DT 0.36 .00
NVB: positive 29 5.72 1.44 29 5.83 1.56 NVB 4.99* .05
DT x NVB 0.09 .00
Customer care
NVB: negative 27 4.44 1.69 25 5.64 1.66 DT 2.39 .02
NVB: positive 29 5.79 2.29 29 5.66 1.37 NVB 3.97* .04
DT x NVB 3.80 .04
Sales skill
NVB: negative 27 5.11 2.04 25 4.96 2.03 DT 0.07 .00
NVB: positive 29 5.45 2.43 29 5.79 1.24 NVB 2.39 .02
DT x NVB 0.51 .00
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
677Virtual Reality (2022) 26:669–686
1 3
and customer care. For competence behavior in a team, par-
ticipants who experienced positive nonverbal behavior gave
higher ratings (M = 5.78, SD = 1.49) than those who expe-
rienced negative nonverbal behavior (M = 5.00, SD = 2.08).
Competence customer care showed similar effects: the par-
ticipants in conditions with positive nonverbal behavior
(M = 5.72, SD = 1.87) rated the applicants higher than those
in conditions with negative nonverbal behavior (M = 5.02,
SD = 1.77). However, these ratings did not depend on the
display type; we found no significant effects of display type
and no significant interaction of nonverbal behavior with the
display type. No main effects or interaction of the display
type with nonverbal behavior reached statistical significance
for competence sales skill. Considering the ten-point Likert
scale used for the three competency ratings (behavior in a
team, customer care, sales skill), the means of each group
(ranging from M = 4.44 to M = 5.83) shown in Table2 indi-
cate average ratings with no signs of ceiling or floor effects.
3.2 Effects ofnonverbal behavior anddisplay type
onperceived immersion
Table3 shows the psychometric properties of the immersion
scales we used based on the ARI questionnaire of Georgiou
and Kyza (2017). We found very good reliability (DeVellis
2016, p. 136) for all scales, with Cronbach’s
𝛼
ranging from
.79 to .91.
We conducted two-way between-subject ANOVAs to
explore the impact of nonverbal behavior and display type
Table 3 Psychometric properties for ARI scales and subscales
n = 110
Scales M SD Cronbach’s
𝛼
Engagement 5.33 1.02 .83
Interest (4 items) 4.98 1.27 .89
Usability (4 items) 5.67 1.22 .79
Engrossment 4.61 1.29 .90
Emotional attachment (3 items) 4.62 1.34 .87
Focus of attention (3 items) 4.60 1.41 .81
Total immersion 3.40 1.38 .91
Presence (4 items) 3.09 1.48 .90
Flow (3 items) 3.82 1.58 .87
Table 4 Means, standard
deviations, and two-way
ANOVA statistics for the ARI
subscales
n = 110. ANOVA analysis of variance, DT display type, NVB nonverbal behavior
**p < .01
Variable DT: 2D screen DT: VR-cardboard ANO VA
M SD M SD Effect F(1, 106) η2
p
Engagement: usability
NVB: negative 5.65 1.15 6.00 1.05 DT 0.02 .00
NVB: positive 5.76 1.07 5.34 1.49 NVB 1.43 .01
DT × NVB 2.80 .03
Engagement: interest
NVB: negative 4.52 1.18 5.49 1.16 DT 2.00 .02
NVB: positive 5.13 1.19 4.83 1.40 NVB 0.01 .00
DT × NVB 7.23** .06
Engrossment: emotional attachment
NVB: negative 4.58 1.21 5.08 1.02 DT 0.44 .00
NVB: positive 4.52 1.31 4.36 1.67 NVB 2.37 .02
DT × NVB 1.67 .02
Engrossment: focus of attention
NVB: negative 4.46 1.25 5.09 1.17 DT 2.77 .03
NVB: positive 4.33 1.46 4.59 1.62 NVB 1.39 .01
DT × NVB 0.52 .01
Total immersion: flow
NVB: negative 3.62 1.48 4.44 1.68 DT 7.14** .06
NVB: positive 3.26 1.44 4.01 1.55 NVB 1.77 .02
DT x NVB 0.02 .00
Total immersion: presence
NVB: negative 2.81 1.44 3.39 1.61 DT 9.56** .08
NVB: positive 2.53 1.19 3.65 1.47 NVB 0.00 .00
DT × NVB 0.97 .01
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
678 Virtual Reality (2022) 26:669–686
1 3
on the immersion scales (see Table4). The analysis of
the immersion variable engagement showed a significant
interaction in the subscale of interest between nonverbal
behavior and display type. Post hoc analysis revealed sig-
nificantly higher interest ratings in the negative nonverbal
behavior with VR-cardboard condition compared to the
negative nonverbal behavior with 2D screen, t(106) = 2.83,
p < 0.05.
The ANOVAs also yielded a significant main effect
of the display type on both subscales of total immersion:
flow (VR-cardboard: M = 4.21, SD = 1.61; 2D screen
display: M = 3.43, SD = 1.46) and presence (VR-card-
board: M = 3.53, SD = 1.53; 2D screen display: M = 2.67,
SD = 1.31). The VR-cardboard condition led to higher rat-
ings of both subscales compared to the 2D screen display
condition.
3.3 Effects ofnonverbal behavior anddisplay type
onoverall satisfaction withtheexperience
We further analyzed overall satisfaction with a two-way
between-subject ANOVA (see Table 5). The ANOVA
suggested a significant main effect of the display type
(VR-cardboard: M = 5.37, SD = 1.46; 2D screen display:
M = 4.68, SD = 1.43). Thus, experiencing 360-degree vid-
eos with a VR-cardboard led to higher overall satisfaction
than viewing the videos on a 2D screen display.
4 Discussion
We studied whether an interviewer’s nonverbal behavior
(positive vs. negative) impacts the competence ratings of
observers in a job recruitment interview using 360-degree
videos experienced with two different display types (VR-
cardboard vs. 2D screen display). First-semester psy-
chology students (n = 110) in the role of raters observed
a recruitment interview and assessed three competences
(behavior in a team, customer care, and sales skill) of an
applicant. We found that positive nonverbal behavior of
the interviewer resulted in higher ratings for two of three
competences (behavior in a team and customer care) com-
pared to negative nonverbal behavior by the interviewer.
This effect was similar for 360-degree videos viewed with
immersive VR and on 2D screen displays; we found no
effect on display type for the three competence ratings.
Our data did not show an interaction of nonverbal behav-
ior and display type on the competence ratings. However,
360-degree videos resulted in higher immersion when
experienced with VR than when viewed on 2D screen
displays with regard to flow and presence. Thus, the two
relevant ARI subscales of total immersion (flow and pres-
ence) differentiated between VR and 2D screen display
(construct validity). All applied immersion scales showed
good reliability. In the following sections, we discuss each
key result in the context of the research questions, hypoth-
eses, and relevant literature.
4.1 Effects ofnonverbal behavior oncompetence
ratings
The interviewer’s nonverbal behavior influenced observers’
competence ratings in the job recruitment interview. Overall,
the results support our first hypothesis on social influence
trough nonverbal behavior. The participants, acting as inter-
view observers, rated the job applicant’s competence behav-
ior in a team and customer care significantly higher when the
interviewer reacted with positive nonverbal behavior to the
job applicant’s replies than when the interviewer’s nonverbal
behavior was negative. This effect of social influence can be
explained by social proof (also called informational influ-
ence, Cialdini and Goldstein 2004; Cialdini 2007), a form
of seeking accuracy for judgments in uncertain situations
(Wood 2000). These results have highly relevant practical
implications. To increase the validity of interview ratings
(e.g., Huffcutt etal. 2013; Blackman 2017; and further prac-
tical reasons), recruitment interviews often include multiple
persons from the organization’s side. However, these persons
may unintentionally influence others and can unknowingly
be socially influenced, which is by no means intended in job
interviews with multiple interviewers or raters. However, our
results show no significant difference between positive and
Table 5 Means, standard
deviations, and two-way
ANOVA statistics for overall
satisfaction
n = 110. ANOVA analysis of variance, DT display type, NVB nonverbal behavior
**p < .01
Variable DT: VR-cardboard DT: 2D screen A NOVA
M SD M SD Effect F ratio df η2
p
Overall satisfaction
NVB: negative 4.59 1.47 5.80 1.15 DT 7.06** 1, 106 .06
NVB: positive 4.76 1.41 5.00 1.60 NVB 1.35 1, 106 .01
DT x NVB 3.14 1, 106 .03
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
679Virtual Reality (2022) 26:669–686
1 3
negative nonverbal behavior with competence sales skills. A
possible explanation may be that the participants felt more
competent in judging this specific competence because it can
be considered part of the everyday experience if a salesper-
son’s reaction leads to real sales success. We will return to
this point in the section on limitations and further research.
From an educational perspective, our study has shown
that 360-degree videos are useful to increase students’
awareness regarding the effects of social influence through
nonverbal behavior in job application interviews. With the
integration of didactical aspects (e.g., Fowler 2015; with
focus on active learning: e.g., Theobald etal. 2020), such as
the subsequent real-time visualization of the experiment’s
data and class discussion, this procedure can lead to the
desired reflections by students. On the other hand, these
achievements can only be seen as an initial step to reduce
the risk of biases such as these occurring in job interviews.
Before individuals act as interviewers or observers in real
recruitment interviews, thorough training on interviewing
and rating applicants’ competences is highly recommended
(Alcañiz etal. 2018, for leadership training in VR; Doll
2018, for training on conducting interviews). Our study
suggests that such training would benefit from increasing
awareness of possible social influence effects through non-
verbal behavior that could result in biases when judging job
applicants’ competence.
4.2 Effects ofdisplay type (VR‑cardboard vs. 2D
screen display) oncompetence ratings
The effect of social influence through nonverbal behavior
was similar for 360-degree videos viewed in VR-cardboard
and 2D screen displays. To our knowledge, no previous
research has investigated the effects of nonverbal behavior
on observers of job recruitment interviews. Therefore, we
did not formulate a hypothesis on whether the effect of social
influence depends on the display type used to experience
360-degree videos. Our results suggest that the display type
(VR-cardboard vs. 2D screen display) does not significantly
impact competence ratings. Further, we did not formulate a
hypothesis for interaction effects of nonverbal behavior and
display type on competence ratings at job interviews. On
all three competence ratings, the ANOVAs did not yield an
interaction effect. These findings may imply that using 2D
screen displays for awareness-raising experiences (or train-
ing) on social influence biases could be sufficient in practice.
Currently, the use of immersive VR in large higher educa-
tion courses still demands extra effort. Although the use of
low-cost VR-cardboards, combined with the use of students’
smartphones, does not necessarily result in high hardware
acquisition costs, the complete implementation requires
more effort. Securing full technical functionality and com-
patibility on all device types (e.g., different screen sizes,
operating systems, browser types, and versions) demands
additional testing and, thus, potentially more development
resources. The alternative, purchasing an adequate set of
homogenous stand-alone, untethered VR devices, is costly
and involves additional necessary cleaning measures to
meet hygiene standards. In contrast, viewing (360-degree)
videos on notebook/tablet computers can be achieved rela-
tively quickly. However, immersive VR in a class-based sce-
nario has several benefits compared to 2D screen displays.
First, immersive VR automatically absorbs potential visual
distractors (e.g., neighboring students). Second, as higher
education prepares students for future needs at work (e.g.,
twenty-first-century skills), such non-game applications are
feasible ways to familiarize students with VR’s emerging
trends and technology. Third, the overall satisfaction ratings
showed that the students liked using immersive VR (with
VR-cardboard) significantly better than 2D screen displays
for viewing 360-degree video content. Finally, technology
improves rapidly, and VR solutions become more affordable
every year.
4.3 Effects ofnonverbal behavior anddisplay type
onperceived immersion
360-degree videos have higher immersion when viewed in
VR than when viewed on 2D screen displays with regard
to flow and presence. Game immersion theory (Brown and
Cairns 2004) suggests that players need to overcome specific
barriers on an immersion level before reaching the next stage
(for learning context, see also Cheng etal. 2015). The first
stage (engagement) and the second stage (engrossment) of
perceived immersion focus more on the content (e.g., the
story) and the application’s use (e.g., interaction modes).
These were kept nearly identical for the VR-cardboard and
2D screen display conditions. As expected, the subscale
ratings on the two first stages did not differ with regard to
the display type. However, the subscale means ranged from
M = 4.46 to M = 6.00 (see Table4). Taking into account the
seven-point Likert scale, these values were high, indicat-
ing that the two stages were achieved. The analysis of the
subscales of total immersion (presence and flow) showed
that they were both significantly higher for the VR-card-
board condition than for the 2D screen display condition.
These findings align with our expectations as immersive VR
has the advantage of providing a higher perception of total
immersion (subscales of presence and flow) than 2D screen
displays.
An interesting point was discovered in the interaction
between nonverbal behavior and display type on the subscale
of interest. The VR-cardboard group with negative nonverbal
behavior perceived the experience with significantly higher
interest than the group with positive nonverbal behavior. One
possible explanation is the evocation of emotions through
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
680 Virtual Reality (2022) 26:669–686
1 3
nonverbal behavior or social influence. To our knowledge,
however, no previous research examines social influence that
triggers emotions (i.e., their nature, valence, and intensity),
which then influences aspects of perceived immersion. We
will return to this point in the section on limitations and
further research.
4.4 Adapted ARI questionnaire
As mentioned in Introduction and Method, we decided to
slightly adapt the ARI questionnaire (Georgiou and Kyza
2017) to be used with 360-degrees videos presented using
a VR-cardboard and a 2D screen display. Our results sug-
gest that the adapted ARI questionnaire can be used to reli-
ably measure different aspects of immersion when viewing
360-degree videos in VR-cardboard and on a 2D screen
display. The psychometric properties of the adapted ARI
questionnaire showed very good reliability with our data
(Cronbach’s
𝛼
-values ranging from .79 to .91). Remarkably,
with the exception of the subscales usability and focus of
attention, the reliabilities with our sample were even higher
than the reported values of the original ARI questionnaire
that was used in another context (Georgiou and Kyza 2017).
4.5 Limitations andfurther research
As mentioned in Method section, all video sequences were
evaluated by three experts working in the field of personnel
assessment. They unanimously confirmed that the interview
setting, interviewer questions and answers of the applicant
were realistic, but professional interviewers would show less
pronounced emotional expressions. Our study highlights the
importance of conducting job assessment interviews in a
benevolent manner without showing pronounced facial
expressions. We tested lay raters (first-semester psychology
students) in our study. It is unclear whether our findings
of social influence bias through nonverbal behavior would
also affect professional raters in job recruitment interviews.
It would be interesting to investigate this aspect in a future
study and compare students’ ratings to those of experienced
job interviewers’ ratings. Furthermore, our data relied on the
judgment of three competences based on situational ques-
tions only. A more comprehensive range of job-relevant
aspects (e.g., Lambert etal. 2014, for sales-related jobs) and
the inclusion of behavior description questions (e.g., Taylor
and Small 2002; Huffcutt etal. 2004; Klehe and Latham
2006; Culbertson etal. 2017; Hartwell etal. 2019) would
allow further conclusions about which type of competencies
or questions are especially likely to socially influence raters.
Our results are very promising regarding the application of a
slightly adapted ARI questionnaire (Table6 in Appendix) to
measure the three levels of perceived immersion (as defined
in Brown and Cairns 2004). However, further research with
more participants would be beneficial to validate the model
structure of game immersion theory (Cheng etal. 2015) in
more detail. Finally, the observed interaction of nonverbal
behavior and the display type on the immersion scale of
interest potentially opens further research questions. Why
and to what extent do immersive VR experiences with neg-
ative nonverbal behavior compared to positive nonverbal
behavior arouse interest, but not when displayed on a 2D
screen? A future study analyzing viewers’ emotions (e.g.,
Diemer etal. 2015; Chirico and Gaggioli 2019; Niu etal.
2019) induced by nonverbal behavior (positive vs. negative)
could more precisely reveal the reasons for the observed
interaction effect on interest.
5 Conclusion
Recruitment interviews are a widely used method for recruit-
ing new personnel and often include multiple persons from
the hiring organization. This study evaluated whether an
interviewer’s nonverbal behavior (positive vs. negative)
impacts observers’ competence ratings of an applicant
using 360-degree videos on two different display types (VR-
cardboard vs. 2D screen display). The participants observed
interview sequences in the role of a rater and assessed three
competences (behavior in a team, customer care, and sales
skill) of the applicant. The interviewer’s nonverbal behav-
ior influenced these ratings on two out of three evaluated
competences (behavior in a team and customer care). Over-
all, these findings confirm a social influence bias and are
therefore highly relevant for practice, particularly because
interviews with multiple interviewers/raters are gener-
ally considered valid. Additionally, our study showed that
360-degree videos of recruitment interviews can be used for
awareness-raising experiences on social influence through
nonverbal behavior. Interestingly, the effect of social influ-
ence was independent of the display type: watching the
360-degree videos with immersive VR and on 2D screen dis-
plays resulted in similar ratings of competence. This raises
the question of whether it is worth the extra effort of using
immersive VR in a classroom to demonstrate the effect of
social influence through nonverbal behavior. However, the
participants in the immersive VR condition rated the overall
satisfaction significantly higher than the participants in the
2D screen display condition. Furthermore, the participants
perceived a higher level of flow and presence (defined as the
third level of immersion, total immersion) when experienc-
ing the 360-degree videos in immersive VR compared to
2D screen displays. This finding is in line with the under-
lying game immersion theory. The first and second levels
of immersion (engagement with the subscales of usability
and interest; engrossment with the subscales of emotional
attachment and focus of attention) did not differ between
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
681Virtual Reality (2022) 26:669–686
1 3
the display types because the constructs relate mainly to
the content and the implementation (which were similar for
both display conditions). Thus, the evaluated scales of the
ARI questionnaire (Georgiou and Kyza 2017, with the term
AR” modified to “VR”; see the Table6 in Appendix) dem-
onstrated construct validity and high reliability. Accordingly,
our results suggest that the ARI questionnaire is an appropri-
ate instrument to rate different aspects of immersion when
assessing 360-degree videos using immersive VR and 2D
screen displays.
Appendix
See Fig.5 and Table 6.
Fig. 5 Examples of positive and negative nonverbal behaviors shown in the 360-degree videos for each competence (behavior in a team, cus-
tomer care, and sales skill)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
682 Virtual Reality (2022) 26:669–686
1 3
Acknowledgements We thank Nina Imboden for her support with
the literature research; Christin Meier (interviewer) and Lukas Gasser
(applicant) for their professional acting performance; Nina Wyssenbach
for the 360-degree video capturing and editing; Amanda Folie for the
support in technical implementation (video streaming and webVR);
Dorothee Tautz and Nina Wyssenbach for the technical support during
data collection.
Authors’ contributions All authors contributed to the study conception,
design, material preparation, and data collection. Thomas Wyssenbach
and Adrian Schwaninger performed the data analysis. Thomas Wys-
senbach wrote the first draft of the manuscript with feedback from the
co-authors. All authors read and approved the final manuscript.
Funding Open Access funding provided by Fachhochschule Nordwest-
schweiz FHNW. This research was funded by the University of Applied
Sciences and Arts Northwestern Switzerland (FHNW).
Availability of data and material The analyzed data of the current study
and the 360-degree video material used in the current study are avail-
able from the corresponding author upon reasonable request.
Code availability Not applicable.
Table 6 Immersion questionnaire used in the current study (Items and Structure Based on Georgiou and Kyza 2017, p. 34ff)
Reverse-coded items are marked with an asterisk; English formulations deviating from the original ARI questionnaire (Georgiou and Kyza 2017)
are marked with square brackets
Factor English German translation (used in the current study)
Engagement: Interest I liked the activity because it was novel Ich mochte die Aktivität, weil sie neuartig war
Engagement: Interest I liked the type of the activity Ich mochte die Art der Aktivität
Engagement: Interest I wanted to spend the time to complete the activity
successfully
Ich wollte Zeit aufbringen, um die Aktivität erfol-
greich abzuschliessen
Engagement: Interest I wanted to spend time to participate in the activity Ich wollte Zeit investieren, um an der Aktivität
teilzunehmen
Engagement: Usability It was easy for me to use the [VR] application Es war einfach für mich, das VR-Modul zu nutzen
Engagement: Usability* I found the [VR] application confusing Ich fand das VR-Modul verwirrend
Engagement: Usability* The [VR] application was unnecessarily complex Das VR-Modul war unnötig komplex
Engagement: Usability I did not have difficulties in controlling the [VR]
application
Ich hatte bei der Bedienung des VR-Moduls keine
Schwierigkeiten
Engrossment: Emotional attachment I was curious about how the activity would progress Ich war neugierig, wie die Aktivität weitergeht
Engrossment: Emotional attachment I was often excited since I felt as being part of the
activity
Ich war oftmals begeistert, da ich mich als Teil der
Aktivität fühlte
Engrossment: Emotional attachment I often felt suspense by the activity Ich empfand die Aktivität oft als spannend
Engrossment: Focus of attention If interrupted, I looked forward to returning to the
activity
Wenn ich unterbrochen wurde, freute ich mich
darauf, zur Aktivität zurückzukehren
Engrossment: Focus of attention Everyday thoughts and concerns faded out during
the activity
Alltägliche Gedanken und Sorgen verblassten
während der Aktivität
Engrossment: Focus of attention I was more focused on the activity rather on any
external distraction
Ich war mehr auf die Aktivität fokussiert, als auf
irgendwelche äussere Ablenkungen
Total immersion: presence I felt that I was in a highly realistic activity, in
which I could hardly separate what was virtual
or real
Die Aktivität fühlte sich so authentisch an, dass ich
dachte die virtuellen Figuren/Objekte wären real
Total immersion: presence I felt that what I was experiencing was something
real, instead of a fictional activity
Ich hatte das Gefühl, dass das was ich erlebt habe,
etwas Reales und nicht eine fiktive Aktivität war
Total immersion: presence I was so involved in the activity, that in some cases
I wanted to interact with the virtual characters/
objects directly
Ich war so sehr in die Aktivität vertieft, dass ich in
einigen Fällen direkt mit den virtuellen Figuren/
Objekten interagieren wollte
Total immersion: presence I so was involved, that I felt that my actions could
affect the activity
Ich war so vertieft, dass ich das Gefühl hatte meine
Handlungen können die Aktivität beeinflussen
Total immersion: flow I didn’t have any irrelevant thoughts or external
distractions during the activity
Ich hatte keine irrelevanten Gedanken oder äussere
Ablenkungen während der Aktivität
Total immersion: flow The activity became the unique and only thought
occupying my mind
Die Aktivität wurde zum einzigen und alleinigen
Gedanken, der mich beschäftigte
Total immersion: flow I lost track of time, as if everything just stopped,
and the only thing that I could think about was the
activity
Ich verlor den Bezug zur Zeit, als wäre alles stehen
geblieben und das Einzige, woran ich denken
konnte, war die Aktivität
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
683Virtual Reality (2022) 26:669–686
1 3
Declarations
Conflict of interest Thomas Wyssenbach, Melina Zeballos, and Stefan
Loosli were co-founders of the collaborating partner Waimanoo (2020)
at the time of the data collection. The authors declare that there are no
relevant financial or non-financial interests to disclose in this article’s
context.
Ethical approval This study complied with the American Psychological
Association Code of Ethics and was conducted in accordance with the
European General Data Protection Regulation. All participants (psy-
chology students; n = 117) voluntarily participated in this study and
gave informed consent before starting the experiment.
Open Access This article is licensed under a Creative Commons Attri-
bution 4.0 International License, which permits use, sharing, adapta-
tion, distribution and reproduction in any medium or format, as long
as you give appropriate credit to the original author(s) and the source,
provide a link to the Creative Commons licence, and indicate if changes
were made. The images or other third party material in this article are
included in the article’s Creative Commons licence, unless indicated
otherwise in a credit line to the material. If material is not included in
the article’s Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will
need to obtain permission directly from the copyright holder. To view a
copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.
References
Alcañiz M, Parra E, Chicchi Giglioli IA (2018) Virtual reality as an
emerging methodology for leadership assessment and training.
Front Psychol 9:1658. https:// doi. org/ 10. 3389/ fpsyg. 2018. 01658
Allcoat D, von Mühlenen A (2018) Learning in virtual reality: effects
on performance, emotion and engagement. Res Learn Technol
26:1–13. https:// doi. org/ 10. 25304/ rlt. v26. 2140
American Psychological Association (2020) Publication manual of
the American Psychology Association. American Psychologi-
cal Association, Washington, DC
Arvey RD, Campion JE (1982) The employment interview: a summary
and review of recent research. Pers Psychol 35:281–322. https://
doi. org/ 10. 1111/j. 1744- 6570. 1982. tb021 97.x
Barrick MR, Swider BW, Stewart GL (2010) Initial evaluations in the
interview: relationships with subsequent interviewer evaluations
and employment offers. J Appl Psychol 95:1163–1172. https://
doi. org/ 10. 1037/ a0019 918
Beaton DE, Bombardier C, Guillemin F, Ferraz MB (2000) Guidelines
for the process of cross-cultural adaptation of self-report meas-
ures. Spine (Phila Pa 1976) 25:3186–3191. https:// doi. org/ 10.
1097/ 00007 632- 20001 2150- 00014
Blackman M (2017) Using interviewing in selection. In: Goldstein
H, Pulakos E, Passmore J, Semedo C (eds) The wiley black-
well handbook of the psychology of recruitment, selection and
employee retention. John Wiley, Chichester, pp 182–201
Blacksmith N, Willford J, Behrend T (2016) Technology in the employ-
ment interview: a meta-analysis and future research agenda. Pers
Assess Decis 2:12–20. https:// doi. org/ 10. 25035/ pad. 2016. 002
Blascovich J, Loomis J, Beall AC, Swinth KR, Hoyt CL, Bailenson
JN (2002) Immersive virtual environment technology as a meth-
odological tool for social psychology. Psychol Inq 13:103–124.
https:// doi. org/ 10. 1207/ S1532 7965P LI1302_ 01
Bonaccio S, O’Reilly J, O’Sullivan SL, Chiocchio F (2016) Nonverbal
behavior and communication in the workplace: a review and an
agenda for research. J Manag 42:1044–1074. https:// doi. org/ 10.
1177/ 01492 06315 621146
Borkenau P, Liebler A (1995) Observable attributesas manifestations
and cues of personality and intelligence. J Pers 63:1–25. https://
doi. org/ 10. 1111/j. 1467- 6494. 1995. tb007 99.x
Bos N, Olson J, Gergle D, Olson G, Wright Z (2002) Effects of four
computer-mediated communications channels on trust develop-
ment. In: Proceedings of the SIGCHI conference on human fac-
tors in computing systems. Association for Computing Machin-
ery, Minneapolis, MN, pp 135–140
Brown E, Cairns P (2004) A grounded investigation of game immer-
sion. In: Conference on human factors in computing systems.
Association for Computing Machinery, Vienna, Austria, pp
1297–1300
Butler D, Geis F (1990) Nonverbal affect responses to male and female
leaders: implications for leadership evaluations. J Pers Soc Psy-
chol 58:48–59. https:// doi. org/ 10. 1037/ 0022- 3514. 58.1. 48
Campion M, Pursell E, Brown B (1988) Structured interviewing: rais-
ing the psychometric properties of the employment interview.
Pers Psychol 41:25–42. https:// doi. org/ 10. 1111/j. 1744- 6570.
1988. tb006 30.x
Campion M, Campion J, Hudson J Jr (1994) Structured interview-
ing: a note on incremental validity and alternative question
types. J Appl Psychol 79:998–1002. https:// doi. org/ 10. 1037/
0021- 9010. 79.6. 998
Campion M, Palmer D, Campion J (1997) A review of structure in
the selection interview. Pers Psychol 50:655–702. https:// doi.
org/ 10. 1111/j. 1744- 6570. 1997. tb007 09.x
Checa D, Bustillo A (2020) A review of immersive virtual real-
ity serious games to enhance learning and training. Mul-
timed Tools Appl 79:5501–5527. https:// doi. org/ 10. 1007/
s11042- 019- 08348-9
Cheng MT, She HC, Annetta LA (2015) Game immersion experi-
ence: its hierarchical structure and impact on game-based sci-
ence learning. J Comput Assist Learn 31:232–253. https:// doi.
org/ 10. 1111/ jcal. 12066
Chirico A, Gaggioli A (2019) When virtual feels real: comparing
emotional responses and presence in virtual and natural envi-
ronments. Cyberpsychology Behav Soc Netw 22:220–226.
https:// doi. org/ 10. 1089/ cyber. 2018. 0393
Churchill GA, Ford NM, Hartley SW, Walker OC (1985) The deter-
minants of salesperson performance: a meta-analysis. J Mark
Res 22:103–118. https:// doi. org/ 10. 2307/ 31513 57
Cialdini R (2001) Influence: science and practice. Allyn & Bacon,
Boston
Cialdini R (2007) Influence: the psychology of persuasion. Collins,
New York
Cialdini RB, Goldstein NJ (2004) Social influence: compliance and
conformity. Annu Rev Psychol 55:591–621. https:// doi. org/ 10.
1146/ annur ev. psych. 55. 090902. 142015
Cohen J (1988) Statistical power analysis for the behavioral sciences.
Taylor and Francis, Hoboken
Cohn J, Ambadar Z, Ekman P (2007) Observer-based measurement
of facial expression with the facial action coding system. In:
Coan J, Allen J (eds) The handbook of emotion elicitation and
assessment. Oxford University Press, Oxford, pp 203–221
Culbertson SS, Weyhrauch WS, Huffcutt AI (2017) A tale of two
formats: direct comparison of matching situational and behav-
ior description interview questions. Hum Resour Manag Rev
27:167–177. https:// doi. org/ 10. 1016/j. hrmr. 2016. 09. 009
DeGroot T, Gooty J (2009) Can nonverbal cues be used to make
meaningful personality attributions in employment inter-
views? J Bus Psychol 24:179–192. https:// doi. org/ 10. 1007/
s10869- 009- 9098-0
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
684 Virtual Reality (2022) 26:669–686
1 3
DeVellis R (2016) Scale development: theory and applications. Sage,
Los Angeles
Diemer J, Alpers GW, Peperkorn HM, Shiban Y, Mühlberger A (2015)
The impact of perception and presence on emotional reactions: a
review of research in virtual reality. Front Psychol 6:26. https://
doi. org/ 10. 3389/ fpsyg. 2015. 00026
Dipboye R (2017) The selection/recruitment interview: core processes
and contexts. In: Evers A, Anderson N, Voskuijl O (eds) The
Blackwell handbook of personnel selection. Blackwell, Malden,
pp 119–142
Dipboye R, Macan T, Shahani-Denning C (2012) The selection inter-
view from the interviewer and applicant perspectives: can’t have
one without the other. In: Schmitt N (ed) The Oxford handbook
of personnel assessment and selection. Oxford University Press,
Oxford, pp 323–352
Doherty-Sneddon G, Anderson A, O’Malley C, Langton S, Garrod S,
Bruce V (1997) Face-to-face and video-mediated communica-
tion: a comparison of dialogue structure and task performance.
J Exp Psychol Appl 3:105–125. https:// doi. org/ 10. 1037/ 1076-
898X.3. 2. 105
Doll JL (2018) Structured interviews: developing interviewing skills in
human resource management courses. Manag Teach Rev 3:46–
61. https:// doi. org/ 10. 1177/ 23792 98117 722520
Elias ZM, Batumalai UM, Azmi ANH (2019) Virtual reality games
on accommodation and convergence. Appl Ergon 81:102879.
https:// doi. org/ 10. 1016/j. apergo. 2019. 102879
Feurstein M (2019) Towards an integration of 360-degree video
in higher education. Workflow, challenges and scenarios. In:
Schiffner D (ed) Proceedings of DeLFI workshops 2018 co-
located with 16th e-Learning conference of the German com-
puter society (DeLFI 2018). CEUR WS, Frankfurt, Germany,
pp 1–12
Fowler C (2015) Virtual reality and learning: where is the pedagogy?
Br J Educ Technol 46:412–422. https:// doi. org/ 10. 1111/ bjet.
12135
Freina L, Ott M (2015) A literature review on immersive virtual real-
ity in education: state of the art and perspectives. In: The 11th
international scientific conference e-Learning and software for
education. ”Carol I” National Defence University Publishing
House, Bucharest, Romania, pp 214–220
Gallup AC, Vasilyev D, Anderson N, Kingstone A (2019) Conta-
gious yawning in virtual reality is affected by actual, but not
simulated, social presence. Sci Rep 9:294. https:// doi. org/ 10.
1038/ s41598- 018- 36570-2
Georgiou Y, Kyza EA (2017) The development and validation of the
ARI questionnaire: an instrument for measuring immersion in
location-based augmented reality settings. Int J Hum Comput
Stud 98:24–37. https:// doi. org/ 10. 1016/j. ijhcs. 2016. 09. 014
Gifford R, Ng C, Wilkinson M (1985) Nonverbal cues in the employ-
ment interview: links between applicant qualities and inter-
viewer judgments. J Appl Psychol 70:729–736. https:// doi. org/
10. 1037/ 0021- 9010. 70.4. 729
Google (2020a) Google cardboard. https:// arvr. google. com/ cardb
oard. Accessed 31 Aug 2020
Google (2020b) Interactive patterns – controls. https:// desig nguid
elines. withg oogle. com/ cardb oard/ inter active- patte rns/ contr ols.
html. Accessed 31 Aug 2020
Guyer JJ, Briñol P, Petty RE, Horcajo J (2019) Nonverbal behav-
ior of persuasive sources: a multiple process analysis. J
Nonverbal Behav 43:203–231. https:// doi. org/ 10. 1007/
s10919- 018- 00291-x
Hartwell CJ, Johnson CD, Posthuma RA (2019) Are we asking the
right questions? Predictive validity comparison of four structured
interview question types. J Bus Res 100:122–129. https:// doi. org/
10. 1016/j. jbusr es. 2019. 03. 026
Hausknecht JP, Day DV, Thomas SC (2004) Applicant reactions to
selection procedures: an updated model and meta-analysis. Pers
Psychol 57:639–683. https:// doi. org/ 10. 1111/j. 1744- 6570. 2004.
00003.x
Holm S (1979) A simple sequentially rejective multiple test procedure.
Scand J Stat 6:65–70
Huffcutt AI, Conway JM, Roth PL, Klehe UC (2004) The impact of job
complexity and study design on situational and behavior descrip-
tion interview validity. Int J Sel Assess 12:262–273. https:// doi.
org/ 10. 1111/j. 0965- 075X. 2004. 280_1.x
Huffcutt AI, Van Iddekinge CH, Roth PL (2011) Understanding appli-
cant behavior in employment interviews: a theoretical model of
interviewee performance. Hum Resour Manag Rev 21:353–367.
https:// doi. org/ 10. 1016/j. hrmr. 2011. 05. 003
Huffcutt AI, Culbertson SS, Weyhrauch WS (2013) Employment inter-
view reliability: new meta-analytic estimates by structure and
format. Int J Sel Assess 21:264–276. https:// doi. org/ 10. 1111/
ijsa. 12036
Innocenti A, Molinari F, Müller C (2012) Assessing risk perception
and well-being at workplaces: a game based approach. In: 2012
18th international ICE conference on engineering, technology
and innovation. IEEE, Munich, Germany, pp 1–8
Jensen L, Konradsen F (2018) A review of the use of virtual reality
head-mounted displays in education and training. Educ Inf Tech-
nol 23:1515–1529. https:// doi. org/ 10. 1007/ s10639- 017- 9676-0
Jerald J (2016) The VR book: human-centered design for virtual reality.
ACM, New York
Jones B, Kenward M (2015) Design and analysis of cross-over trails.
CRC Press, Boca Raton
Kim HK, Park J, Choi Y, Choe M (2018) Virtual reality sickness ques-
tionnaire (VRSQ): motion sickness measurement index in a vir-
tual reality environment. Appl Ergon 69:66–73. https:// doi. org/
10. 1016/j. apergo. 2017. 12. 016
Kinateder M, Warren WH (2016) Social influence on evacuation
behavior in real and virtual environments. Front Robot AI 3:1–8.
https:// doi. org/ 10. 3389/ frobt. 2016. 00043
Klehe UC, Latham G (2006) What would you do—Really or ideally?
Constructs underlying the behavior description interview and
the situational interview in predicting typical versus maximum
performance. Hum Perform 19:357–382. https:// doi. org/ 10. 1207/
s1532 7043h up1904_3
König CJ, Klehe UC, Berchtold M, Kleinmann M (2010) Reasons for
being selective when choosing personnel selection procedures.
Int J Sel Assess 18:17–27. https:// doi. org/ 10. 1111/j. 1468- 2389.
2010. 00485.x
Kumar R, Sharma G, Kumar L, Chandra S (2017) Effect of immersion
(2D vs. 3D) on attention through virtual reality. Int J Sci Res
6:204–207
Lambert B, Plank RE, Reid DA, Fleming D (2014) A competency
model for entry level business-to-business services salespeople.
Serv Mark Q 35:84–103. https:// doi. org/ 10. 1080/ 15332 969.
2014. 856746
Lanier J (2017) Dawn of the new everything. Bodley Head, London
Larsen RJ, Shackelford TK (1996) Gaze avoidance: personality and
social judgments of people who avoid direct face-to-face contact.
Pers Individ Differ 21:907–917. https:// doi. org/ 10. 1016/ S0191-
8869(96) 00148-1
Liden RC, Martin CL, Parsons CK (1993) Interviewer and applicant
behaviors in employment interviews. Acad Manag J 36:372–386.
https:// doi. org/ 10. 5465/ 256527
Lubeck AJA, Bos JE, Stins JF (2015) Motion in images is essential to
cause motion sickness symptoms, but not to increase postural
sway. Displays 38:55–61. https:// doi. org/ 10. 1016/j. displa. 2015.
03. 001
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
685Virtual Reality (2022) 26:669–686
1 3
Macan T (2009) The employment interview: a review of current stud-
ies and directions for future research. Hum Resour Manag Rev
19:203–218. https:// doi. org/ 10. 1016/j. hrmr. 2009. 03. 006
Markowitz DM, Laha R, Perone BP, Pea RD, Bailenson JN (2018)
Immersive virtual reality field trips facilitate learning about
climate change. Front Psychol 9:2364. https:// doi. org/ 10. 3389/
fpsyg. 2018. 02364
Martín-Gutiérrez J, Mora C, Añorbe-Díaz B, González-Marrero A
(2017) Virtual technologies trends in education. Eurasia J Math
Sci Technol Educ 13:469–486. https:// doi. org/ 10. 12973/ euras ia.
2017. 00626a
McDaniel M, Whetzel D, Schmidt F, Maurer S (1994) The validity
of employment interviews: a comprehensive review and meta-
analysis. J Appl Psychol 79:599–616. https:// doi. org/ 10. 1037/
0021- 9010. 79.4. 599
Niu Y, Wang D, Wang Z, Sun F, Yue K, Zheng N (2019) User experi-
ence evaluation in virtual reality based on subjective feelings and
physiological signals. J Imaging Sci Technol 63:1–11. https:// doi.
org/ 10. 2352/J. Imagi ngSci. Techn ol. 2019. 63.6. 060413
Owen DJ, Slep AM, Heyman RE (2012) The effect of praise, posi-
tive nonverbal response, reprimand, and negative nonver-
bal response on child compliance: a systematic review. Clin
Child Fam Psychol Rev 15:364–385. https:// doi. org/ 10. 1007/
s10567- 012- 0120-0
Pan X, Hamilton AFC (2018) Why and how to use virtual reality to
study human social interaction: the challenges of exploring a
new research landscape. Br J Psychol 109:395–417. https:// doi.
org/ 10. 1111/ bjop. 12290
Papanastasiou G, Drigas A, Skianis C, Lytras M, Papanastasiou E
(2019) Virtual and augmented reality effects on K-12, higher and
tertiary education students’ twenty-first century skills. Virtual
Real 23:425–436. https:// doi. org/ 10. 1007/ s10055- 018- 0363-2
Parsons C, Liden R (1984) Interviewer perceptions of applicant qualifi-
cations: a multivariate field study of demographic characteristics
and nonverbal cues. J Appl Psychol 69:557–568. https:// doi. org/
10. 1037/ 0021- 9010. 69.4. 557
Phillips JF (1992) Predicting sales skills. J Bus Psychol 7:151–160.
https:// doi. org/ 10. 1007/ BF010 13925
Pirker J, Lesjak I, Kopf J, Kainz A, Dini A (2020) Immersive learning
in real VR. In: Magnor M, Sorkine-Hornung A (eds) Real VR –
Immersive digital reality. Springer Nature, Cham, pp 321–336
Powell W, Powell V, Brown P, Cook M, Uddin J (2016) Getting around
in google cardboard – Exploring navigation preferences with
low-cost mobile VR. In: 2016 IEEE 2nd workshop on everyday
virtual reality (WEVR). IEEE, Greenville, SC, pp 5–8
Proost K, Germeys F, Vanderstukken A (2020) Applicants’ pre-test
reactions towards video interviews: the role of expected chances
to demonstrate potential and to use nonverbal cues. Eur J Work
Organ Psychol 30:265–273. https:// doi. org/ 10. 1080/ 13594 32X.
2020. 18179 75
Radianti J, Majchrzak TA, Fromm J, Wohlgenannt I (2020) A sys-
tematic review of immersive virtual reality applications for
higher education: design elements, lessons learned, and research
agenda. Comput Educ 147:103778. https:// doi. org/ 10. 1016/j.
compe du. 2019. 103778
Rentz JO, Shepherd CD, Tashchian A, Dabholkar PA, Ladd RT (2002)
A measure of selling skill: scale development and validation. J
Pers Sell Sales Manag 22:13–21
Rupp MA, Odette KL, Kozachuk J, Michaelis JR, Smither JA, McCo-
nnell DS (2019) Investigating learning outcomes and subjective
experiences in 360-degree videos. Comput Educ 128:256–268.
https:// doi. org/ 10. 1016/j. compe du. 2018. 09. 015
Ryan AM, McFarland L, Shl HB (1999) An international look at selec-
tion practices: nation and culture as explanations for variability
in practice. Pers Psychol 52:359–392. https:// doi. org/ 10. 1111/j.
1744- 6570. 1999. tb001 65.x
Salgado J, Viswesvaran C, Ones D (2004) Predictors used for personnel
selection: an overview of constructs, methods and techniques. In:
Anderson N, Ones D, Sinangil H, Viswesvaran C (eds) Hand-
book of industrial, work and organizational psychology. Sage,
London, pp 375–377
Salmanowitz N (2018) The impact of virtual reality on implicit racial
bias and mock legal decisions. J Law Biosci 5:174–203. https://
doi. org/ 10. 1093/ jlb/ lsy005
Schmidt F, Hunter J (1998) The validity and utility of selection meth-
ods in personnel psychology: practical and theoretical implica-
tions of 85 years of research findings. Psychol Bull 124:262–274.
https:// doi. org/ 10. 1037/ 0033- 2909. 124.2. 262
Schutte NS, Stilinović EJ (2017) Facilitating empathy through vir-
tual reality. Motiv Emot 41:708–712. https:// doi. org/ 10. 1007/
s11031- 017- 9641-7
Schwind V, Knierim P, Haas N, Henze N (2019) Using presence ques-
tionnaires in virtual reality. In: Proceedings of the 2019 CHI
conference on human factors in computing systems. Association
for Computing Machinery, Glasgow, UK, pp 1–12
Sears GJ, Zhang H, Wiesner WH, Hackett RD, Yuan Y (2013) A com-
parative assessment of videoconference and face-to-face employ-
ment interviews. Manag Decis 51:1733–1752. https:// doi. org/ 10.
1108/ MD- 09- 2012- 0642
Slater M (2018) Immersion and the illusion of presence in virtual real-
ity. Br J Psychol 109:431–433. https:// doi. org/ 10. 1111/ bjop.
12305
Slater M, Wilbur S (1997) A framework for immersive virtual envi-
ronments (FIVE): speculations on the role of presence in virtual
environments. Presence Teleoperators Virtual Environ 6:603–
616. https:// doi. org/ 10. 1162/ pres. 1997.6. 6. 603
Suh A, Prophet J (2018) The state of immersive technology research:
a literature analysis. Comput Hum Behav 86:77–90. https:// doi.
org/ 10. 1016/j. chb. 2018. 04. 019
Sutcliffe A, Gault B, Shin JE (2005) Presence, memory and interaction
in virtual environments. Int J Hum Comput Stud 62:307–327.
https:// doi. org/ 10. 1016/j. ijhcs. 2004. 11. 010
Swider B, Barrick M, Harris T (2016) Initial impressions: what they
are, what they are not, and how they influence structured inter-
view outcomes. J Appl Psychol 101:625–638. https:// doi. org/ 10.
1037/ apl00 00077
Taylor PJ, Small B (2002) Asking applicants what they would do ver-
sus what they did do: a meta-analytic comparison of situational
and past behaviour employment interview questions. J Occup
Organ Psychol 75:277–294. https:// doi. org/ 10. 1348/ 09631 79023
20369 712
Tennant M, Youssef GJ, McGillivray J, Clark TJ, McMillan L, McCa-
rthy MC (2020) Exploring the use of immersive virtual reality to
enhance psychological well-being in pediatric oncology: a pilot
randomized controlled trial. Eur J Oncol Nurs 48:101804. https://
doi. org/ 10. 1016/j. ejon. 2020. 101804
The Jamovi project (2020) jamovi (Version 1.2.27.0) [Computer soft-
ware]. https:// www. jamovi. org. Accessed 31 Aug 2020
Theobald EJ, Hill MJ, Tran E etal (2020) Active learning narrows
achievement gaps for underrepresented students in undergraduate
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
686 Virtual Reality (2022) 26:669–686
1 3
science, technology, engineering, and math. Proc Natl Acad Sci
117:6476. https:// doi. org/ 10. 1073/ pnas. 19169 03117
Turnbull PRK, Phillips JR (2017) Ocular effects of virtual reality head-
set wear in young adults. Sci Rep 7:16172. https:// doi. org/ 10.
1038/ s41598- 017- 16320-6
Violante MG, Vezzetti E, Piazzolla P (2019) Interactive virtual tech-
nologies in engineering education: why not 360° videos? Int
J Interact Des Manuf 13:729–742. https:// doi. org/ 10. 1007/
s12008- 019- 00553-y
Viswesvaran C, Ones D (2018) Non-test methods and techniques used
in employee selection. In: Ones D, Anderson N, Viswesvaran C,
Sinangil H (eds) Handbook of industrial, work & organizational
psychology. Sage, Los Angeles, pp 451–473
Voit A, Mayer S, Schwind V, Henze N (2019) Online, VR, AR, Lab,
and In-Situ: comparison of research methods to evaluate smart
artifacts. In: Proceedings of the 2019 CHI conference on human
factors in computing systems. Association for Computing
Machinery, Glasgow, Scotland, UK, pp 1–12
Waimanoo (2020) Survey and education platform. www. waima noo. ch.
Accessed 31 August 2020
Weekley J, Gier J (1987) Reliability and validity of the situational inter-
view for a sales position. J Appl Psychol 72:484–487. https:// doi.
org/ 10. 1037/ 0021- 9010. 72.3. 484
Williamson P, Weber N, Robertson MT (2013) The effect of expertise
on memory conformity: a test of informational influence. Behav
Sci Law 31:607–623. https:// doi. org/ 10. 1002/ bsl. 2094
Wood W (2000) Attitude change: persuasion and social influence.
Annu Rev Psychol 51:539–570. https:// doi. org/ 10. 1146/ annur
ev. psych. 51.1. 539
Wu B, Yu X, Gu X (2020) Effectiveness of immersive virtual reality
using head-mounted displays on learning performance: a meta-
analysis. Br J Edu Technol 51:1991–2005. https:// doi. org/ 10.
1111/ bjet. 13023
Publisher’s Note Springer Nature remains neutral with regard to
jurisdictional claims in published maps and institutional affiliations.
Content courtesy of Springer Nature, terms of use apply. Rights reserved.
1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com
ResearchGate has not been able to resolve any citations for this publication.
Article
Full-text available
Technological innovations, among which the use of video interviews in personnel selection, are welcomed by organizations for reasons such as reductions in cost and time and the ability to reach a more global labour market. The literature to date suggests that applicants do not share this enthusiasm and feel less attracted towards organizations that use video interviews versus face-to-face interviews. In this study, we extended the literature by studying pre- instead of post-test reactions and testing two explanatory mechanisms, namely expected chances to demonstrate potential and to use nonverbal cues. In a first within-subjects experiment among 38 students, the results of two t-tests showed that participants preferred the face-to-face interview above the video interview and that they expected lower chances to demonstrate potential and to use nonverbal cues in the video interview. In a second between-subjects experiment among 121 potential job seekers, these results were partially replicated. The results of a t-test and a mediation bootstrap analysis with PROCESS showed a significant indirect effect of interview medium on organizational attraction via expected chances to demonstrate potential but not via expected opportunities to use nonverbal communication. This study has important practical implications for organizations in the current context of talent shortages.
Article
Full-text available
We tested the hypothesis that underrepresented students in active-learning classrooms experience narrower achievement gaps than underrepresented students in traditional lecturing classrooms, averaged across all science, technology, engineering, and mathematics (STEM) fields and courses. We conducted a comprehensive search for both published and unpublished studies that compared the performance of underrepresented students to their overrepresented classmates in active-learning and traditional-lecturing treatments. This search resulted in data on student examination scores from 15 studies (9,238 total students) and data on student failure rates from 26 studies (44,606 total students). Bayesian regression analyses showed that on average, active learning reduced achievement gaps in examination scores by 33% and narrowed gaps in passing rates by 45%. The reported proportion of time that students spend on in-class activities was important, as only classes that implemented high-intensity active learning narrowed achievement gaps. Sensitivity analyses showed that the conclusions are robust to sampling bias and other issues. To explain the extensive variation in efficacy observed among studies, we propose the heads-and-hearts hypothesis, which holds that meaningful reductions in achievement gaps only occur when course designs combine deliberate practice with inclusive teaching. Our results support calls to replace traditional lecturing with evidence-based, active-learning course designs across the STEM disciplines and suggest that innovations in instructional strategies can increase equity in higher education.
Article
Full-text available
The merger of game-based approaches and Virtual Reality (VR) environments that can enhance learning and training methodologies have a very promising future, reinforced by the widespread market-availability of affordable software and hardware tools for VR-environments. Rather than passive observers, users engage in those learning environments as active participants, permitting the development of exploration-based learning paradigms. There are separate reviews of VR technologies and serious games for educational and training purposes with a focus on only one knowledge area. However, this review covers 135 proposals for serious games in immersive VR-environments that are combinations of both VR and serious games and that offer end-user validation. First, an analysis of the forum, nationality, and date of publication of the articles is conducted. Then, the application domains, the target audience, the design of the game and its technological implementation, the performance evaluation procedure, and the results are analyzed. The aim here is to identify the factual standards of the proposed solutions and the differences between training and learning applications. Finally, the study lays the basis for future research lines that will develop serious games in immersive VR-environments, providing recommendations for the improvement of these tools and their successful application for the enhancement of both learning and training tasks.
Article
With the availability of low‐cost high‐quality head‐mounted displays (HMDs) since 2013, there is a growing body of literature investigating the impact of immersive virtual reality (IVR) technology on education. This meta‐analysis aims to synthesize the findings on the overall effects of IVR using HMDs compared to less immersive desktop virtual reality (DVR) and other traditional means of instruction. A systematic search was carried out on the literature published between 2013 and 2019. Thirty‐five randomized controlled trials (RCTs) or quasi‐experimental studies were identified. We conducted an analysis using the random effects model (REM) to calculate the pooled effect size. The studies were also coded to examine the moderating effects of their characteristics, such as learner stage, learning domain, learning application type, testing format, control group treatment and learning duration, on the outcome measure. The results showed that IVR using HMDs is more effective than non‐immersive learning approaches with a small effect size (ES = 0.24). The key findings of the moderator analysis were that HMDs have a greater impact (a) on K‐12 learners; (b) in the fields of science education and specific abilities development; (c) when offering simulation or virtual world representations; and (d) when compared with lectures or real‐world practices. The meta‐analysis also suggested that HMDs can improve both knowledge and skill development, and maintain the learning effect over time.
Article
Purpose To investigate whether Immersive Virtual Reality (VR) has a greater positive influence on oncology patients’ physical and emotional mood states when compared to an iPad attentional control condition. Our secondary objective was to understand what factors influenced VR effectiveness. Method Participants were 90 oncology inpatients, aged 7–19 years, and their primary parent caregiver. Using a randomized controlled study design patients were allocated to VR (three content groups) or an iPad control condition. Pre-post-intervention self-report state measures were collected using visual analogue scales and an objective measure of physiological arousal (pulse rate). Post-intervention, patients reported on level of immersion, enjoyment and simulator sickness. Results Patients benefited from both Immersive VR and novel iPad intervention with no statistically significant differences found between conditions on child outcomes. However, patients accessing Immersive VR consistently reported greater positive shifts in mood state and reductions in negative symptoms when compared with iPad. No change was observed in physiological arousal levels (pulse rate) in either condition before, during or immediately after intervention. Moderation analysis showed that the degree of child illness (PedsQL), sex, age, and level of immersion were important in influencing the magnitude of differences between the VR and iPad conditions on mood, anxiety and pain. Conclusions These preliminary findings support the use of Immersive VR in clinical oncology settings to improve patient wellbeing. Further studies examining the application of Immersive VR in supporting children adjusting to hospitalization and cancer treatment are therefore warranted. Factors found to moderate VR effectiveness provide important clinical implications.
Chapter
Immersion is an essential factor for successful learning. Immersive learning environments put learners directly into experiences and events related to the learning content. Virtual reality (VR) experiences give us new opportunities and ways for enabling immersive learning processes. In this chapter, we review different learning environments that use (real) VR as a tool to teach in different learning settings and discuss the state-of-the-art design and evaluation techniques and analyze the benefits and limitations of the different methods.
Article
At present, the research on emotion in the virtual environment is limited to the subjective materials, and there are very few studies based on objective physiological signals. In this article, the authors conducted a user experiment to study the user emotion experience of virtual reality (VR) by comparing subjective feelings and physiological data in VR and two-dimensional display (2D) environments. First, they analyzed the data of self-report questionnaires, including Self-assessment Manikin (SAM), Positive And Negative Affect Schedule (PANAS) and Simulator Sickness Questionnaire (SSQ). The result indicated that VR causes a higher level of arousal than 2D, and easily evokes positive emotions. Both 2D and VR environments are prone to eye fatigue, but VR is more likely to cause symptoms of dizziness and vertigo. Second, they compared the differences of electrocardiogram (ECG), skin temperature (SKT) and electrodermal activity (EDA) signals in two circumstances. Through mathematical analysis, all three signals had significant differences. Participants in the VR environment had a higher degree of excitement, and the mood fluctuations are more frequent and more intense. In addition, the authors used different machine learning models for emotion detection, and compared the accuracies on VR and 2D datasets. The accuracies of all algorithms in the VR environment are higher than that of 2D, which corroborated that the volunteers in the VR environment have more obvious skin electrical signals, and had a stronger sense of immersion. This article effectively compensated for the inadequacies of existing work. The authors first used objective physiological signals for experience evaluation and used different types of subjective materials to make contrast. They hope their study can provide helpful guidance for the engineering reality of virtual reality.