Towards Virtual Agents for Supporting Appropriate Small Group Behaviors in Educational Contexts

Abstract

Verbal and non-verbal behaviors that we use in order to effectively communicate with other people are vital for our success in our daily lives. Despite the importance of social skills, creating standardized methods for training them and supporting their training is challenging. Information and Communications Technology (ICT) may have a good potential to support social and emotional learning (SEL) through virtual social demonstration games. This paper presents initial work involving the design of a pedagogical scenario to facilitate teaching of socially appropriate and inappropriate behaviors when entering and standing in a small group of people, a common occurrence in collaborative social situations. This is achieved through the use of virtual characters and, initially, virtual reality (VR) environments for supporting situated learning in multiple contexts. We describe work done thus far on the demonstrator scenario and anticipated potentials, pitfalls and challenges involved in the approach.

Full text

Towards Virtual Agents for Supporting Appropriate
Small Group Behaviors in Educational Contexts
Sahba Zojaji
EECS, KTH-Royal Institute of Technology
zojaji@kth.se
Christopher Peters
EECS, KTH-Royal Institute of Technology
chpeters@kth.se
Abstract—Verbal and non-verbal behaviors that we use in or-
der to effectively communicate with other people are vital for our
success in our daily lives. Despite the importance of social skills,
creating standardized methods for training them and supporting
their training is challenging. Information and Communications
Technology (ICT) may have a good potential to support social and
emotional learning (SEL) through virtual social demonstration
games. This paper presents initial work involving the design of a
pedagogical scenario to facilitate teaching of socially appropriate
and inappropriate behaviors when entering and standing in a
small group of people, a common occurrence in collaborative
social situations. This is achieved through the use of virtual
characters and, initially, virtual reality (VR) environments for
supporting situated learning in multiple contexts. We describe
work done thus far on the demonstrator scenario and anticipated
potentials, pitfalls and challenges involved in the approach.
Index Terms—Virtual Agents, Virtual Reality, Social & Emo-
tional Learning, Politeness, Demonstration Games, Small Groups
I. INTRODUCTION
Group activities are an important part of our daily lives,
foundational to many collaboration and learning activities.
Forming, joining, leaving or engaging others in small groups
may seem simple to adults, but conducting socially-appropriate
behavior is often context-sensitive, subtle and complex. The
stakes are high: failing to do it can cause offense and have
negative repercussions, even if unintended. In order to navigate
complex social landscapes, one must learn the difference
between how others view polite and impolite behaviors. These
competencies are related to the Social and Emotional Learning
(SEL) process, developed by the Collaborative for Academic
Social and Emotional Learning (CASEL) [1]. SEL is a ped-
agogical process for skill development related to how people
feel and show empathy for others, establish and maintain
positive relationships and make responsible decisions. Studies
have shown that students who take courses in SEL show
improvement in social skills and academic performance [1].
There are a number of challenges for SEL in modern
education. Especially, teachers need support for teaching social
competencies to students. These relate not only to standardized
and validated training tasks, but also personnel requirements.
Furthermore, students may feel anxious when engaging in
activities with their peer group. Even in apparently informal
game-play situations, there may be real-world consequences
that may result in anxiety for participants and disengagement
from activities. Information and Communications Technology
(ICT), such as virtual agents and virtual reality (VR), has
the potential to assist teachers in SEL activities. VR can
provide diverse, immersive, stimulating learning environments
which are too expensive, dangerous or simply not possi-
ble in the real world. Virtual environments and objects are
highly customizable with a relatively low cost. Especially
for social demonstration games in which human bodies and
their behaviors are important, virtual agents may be cost-
effective in specific situations. Overall, these potentials provide
an opportunity for situated learning, facilitate observational
learning, learning by doing and learning by teaching (see [2]
for overview) in safe, simulated situations.
Towards this goal, we propose a virtual reality demonstrator
to facilitate SEL in small group scenarios. Specifically, the
demonstrator involves approaching and joining a group in
a polite manner, orienting towards them to make them feel
included in the group and not obstructing or blocking the
view of others by standing in front of them or violating their
personal space. We intend for the demonstrator to be able to
provide examples of a range of polite and impolite behaviors
to reinforce learning goals that reflect the individual nature of
humans across age, gender and culture.
II. RE LATE D WO RK
Expressive virtual agents have been previously applied to
social skills training to enhance social competencies [3]. For
example, Tanaka et al. [4] used virtual agents to assess
listening skills and Jones et al. [5] used them for expressing
social attitudes. See [6] for an overview interactive agents
dedicated to social training. Theoretical background work
involving small group formations, including the definition of
F-formations, o-spaces and personal spaces [7], [8] is a basis
for designing the social demonstrator and have been used to
model and study group joining and leaving behavior [9].
III. SCENARIO
Cartledge et al. [10] proposed a theoretical model followed
by learning activities to teach and develop social skills, specif-
ically to teach students how to join a group in a polite manner
or respecting individual’s personal space which is embedded
in our demonstrator. Based on this model, defining a behavior
to be taught, is the first step. Then, the proficiency of students
in that specific behavior should be assessed before and after
the teaching phase by the teacher. Teaching might be repeated

until receiving satisfactory results from teacher’s perspective.
In a final stage, students generalize and maintain what they
have learned over time.
Our work-in-progress virtual reality demonstrator is being
implemented using Unity 3D and MCS characters1. It consists
of a teacher dashboard (including a camera view into the
VR environment) and the virtual reality training environment
containing an avatar controlled by the student and a small
group of three virtual agents. The task of the student is to
walk up to the small group and to join them. The teacher may
choose a scenario involving the virtual characters, including
their embodiment and social characteristics. They may also
select one of two modes on the demonstrator: training and
test. In the training mode, the environment is annotated for
the student and social information is visualized for them, such
as if they are blocking the line of sight of a group member or
standing too close to them. The test mode is not annotated
with social information and is therefore more challenging.
In this mode, there is less real-time feedback and students
must rely on the motion-captured reactions of group members.
Regardless of the mode, post-demonstration, the teacher may
playback the scenario from multiple perspectives (including
those of the group members) in order to debrief them on
their performance. While the system is intended to give some
automated feedback e.g. personal space violations, standing in
front of others and so on, the main assessment is provided by
the teacher. The purpose of the demonstrator is therefore as a
support tool for them. It should be noted that the teacher does
not need to be present while students are in the demonstration
environment: they are only required prior to scenario start in
order to set the scenario parameters and after the scenario in
order to assess and provide feedback about the outcome.
IV. DISCUSSION
The ability to control and repeat learning tasks in the
demonstrator under a variety of safe social situations can
support generalization before skills are practised in real sit-
uations. For some, virtual environments and agents may also
induce less anxiety than engaging in real social situations and
participants may also be more at ease [11] to participate in
social experimentation and role-play.
More generally, VR provides an opportunity to be exposed
to different situations in a safe manner for participants. There
might be minimized risk of adverse social effects for stu-
dents when engaging in behaviors in virtual environments,
increasing their willingness to participate and investigate social
behaviors that they may be hesitant to do in real-world training
scenarios. However, this needs to be measured against adverse
effects from the equipment, such as motion sickness. Other
benefits include the ability for students to experience their
behaviors from other perspectives in real-time. VR supports
demonstration of features that are not visible in real-life, such
as social distances.
1http://morph3d.com
There are also a number of limitations. Setting up virtual
reality platforms in schools with children is time and energy
consuming and may not be practical for large or even medium
cohorts. Also, teachers must conduct the feedback process
manually, but there may be possibilities to semi-automatize
it through feedback metrics related to degree of personal
space violations, inappropriate body orientations, occlusions
and trajectories in order to better support the teacher. The
social presence of virtual agents is also generally far lower
than humans, although as mentioned previously, this could be
beneficial to reduce initial feelings of social anxiety. However,
an open question is how interactions with such agents may
then generalize to real interactions that involve high degrees
of social presence and how the social presence of virtual agents
may be improved. Overall, this work is a proof-of-concept that
we will use to study these limitations and potentials, with a
view to the use of augmented reality handheld devices.
V. CONCLUSIONS
We presented a pedagogical scenario embedded in a virtual
reality demonstrator as a tool to support teachers. Its purpose
is to facilitate teaching appropriate behaviors when joining
and interacting in small groups through social demonstration.
As a next step, we plan to further develop the demonstrator
and conduct testing to improve it. We will also use the
demonstrator as a basis for designing similar scenarios that
utilize augmented reality via handheld devices.
VI. ACK NOWLEDGEMENTS
This project has received funding from the European Unions
Horizon 2020 research and innovation program under grant
agreement No. 765955.
REFERENCES
[1] J. Axelrod, Collaborative for Academic, Social and Emotional Learning
(CASEL). Boston, MA: Springer US, 2010, pp. 232–233.
[2] D. H. Schunk, Learning theories an educational perspective sixth
edition. Pearson, 2012.
[3] F. Yang, C. Li, R. Palmberg, E. Van Der Heide, and C. Peters,
“Expressive virtual characters for social demonstration games,” in the
9th Int. Conf. on Virtual Worlds and Games for Serious Applications
(VS-Games), Sep. 2017, pp. 217–224.
[4] H. Tanaka, H. Negoro, H. Iwasaka, and S. Nakamura, “Listening skills
assessment through computer agents,” in the Int. Conf. on Multimodal
Interaction. ACM, 2018, pp. 492–496.
[5] H. Jones, M. Chollet, M. Ochs, N. Sabouret, and C. Pelachaud, “Express-
ing social attitudes in virtual agents for social coaching,” in Workshop
Affects, Compagnons Artificiels et Interaction, 2014.
[6] K. Bosman, T. Bosse, and D. Formolo, “Virtual agents for professional
social skills training: An overview of the state-of-the-art,” in Int. Conf.
on Intelligent Technologies for Interactive Entertainment. Springer,
2018, pp. 75–84.
[7] A. Kendon, Conducting interaction: Patterns of behavior in focused
encounters. CUP Archive, 1990, vol. 7.
[8] E. T. Hall, The hidden dimension / Edward T. Hall, [1st ed.] ed.
Doubleday Garden City, N.Y, 1966.
[9] C. Pedica and H. H. Vilhj´
almsson, “Study of nine people in a hallway:
Some simulation challenges,” in the 18th Int. Conf. on Intelligent Virtual
Agents. ACM, 2018, pp. 185–190.
[10] G. Cartledge and J. Kleefeld, Taking Part: Introducing Social Skills to
Children, PreK-grade 3. Research Press, 2009.
[11] G. M. Lucas, J. Gratch, A. King, and L.-P. Morency, “Its only a
computer: Virtual humans increase willingness to disclose,” Computers
in Human Behavior, vol. 37, pp. 94–100, Aug. 2014.