ArticlePDF Available

Abstract and Figures

Virtual environments (VEs) are now becoming a promising new technology to be used in the development of interactive learning environments for children. Perhaps triggered by the success of computer games, VEs are seen as an emergent and engaging new way by which children can learn experimental sciences and other disciplines. Inhabiting these IVEs can be agents or intelligent characters that are responsible for events that happen in the environment and make it not predictive or completely controlled. However, to build such environments, in particular, if populated by synthetic characters, one needs to carefully address the problem of how do the learners respond to the characters in the virtual environment. Do learners like the characters? Do learners identify themselves with characters in virtual environments? This relation between learners and characters in virtual environments can be studied in several perspectives. In this paper, we will focus primarily on the issue of empathy as one desirable aspect of the affective interaction between learners and synthetic characters. In particular, we will defend that in order for such affective relations to happen, characters should be created and designed taking into account what we call the proximity factor. This is based on the fact that children are found to respond more empathically to those that are perceived as similar to the self than those who are perceived as dissimilar (Barnett 19871. Barnett , M. 1987 . Empathy and related responses in children . In Empathy and its Development . Cambridge University Press . View all references). This appears to be the case when similarity is defined in terms of a shared characteristic, such as sex (Bryant 19825. Bryant , K. 1982 . An index of empathy for children and adolescents . Child Development . 53 . View all references), race or in terms of shared personal experiences (Bryant 19825. Bryant , K. 1982 . An index of empathy for children and adolescents . Child Development . 53 . View all references). Thus, designing characters aiming at pedagogical empathic interactions, we should carefully address how close the learner will feel with the synthetic characters developed in terms of situation, behavior or even physical appearance.In order to illustrate this factor in eliciting emotional reactions to synthetic characters, we will present a specific system called FearNot!. FearNot! was developed to address the difficult and often devastating problem of bullying in schools. By using role-playing and synthetic characters in a 3D environment, FearNot! allows children from age 8 to 12 to experience a virtual scenario where they can witness (in a third-person perspective) bullying situations. To build empathy into FearNot, we have considered the following components: agent's architecture, the characters' embodiment, the environment itself, and emotionally charged situations. All these elements were built to allow for a stronger proximity with the user and the system. In this paper, we will focus primarily on this problem and report some results achieved in the evaluation executed with 127 children and 95 adults on the system.
Content may be subject to copyright.
Learning by Feeling: Evoking Empathy with
Synthetic Characters
Ana Paiva, Jo˜
ao Dias, Daniel Sobral
Instituto Superior T´
ecnico and INESC-ID
Av. Prof. Cavaco Silva, IST, Taguspark
Porto Salvo , Portugal
Ruth Aylett
Center for Virtual Environments,
University of Salford, UK
Sarah Woods
Adaptive Systems Research Group,
University of Hertfordshire, UK
Lynne Hall
School of Computing and Technology,
University of Sunderland, Sunderland, UK
Carsten Zoll
Institute of Theoretical Psychology,
University of Bamberg, Germany
Virtual environments are now becoming a promising new technology to be used
in the development of interactive learning environments for children. Perhaps trig-
gered by the success of computer games, VEs are now seen as an emergent and
engaging new way by which children learn experimental sciences and other disci-
plines. Inhabiting these IVEs there can be agents or intelligent characters, that are
responsible for events that happen in the environment and make it not predictive or
completely controlled. However, to build such environments, in particular if pop-
ulated by synthetic characters, one needs to carefully address the problem of how
do the learners respond to the characters in the virtual environment. Do learners
like the characters? Do learners identify themselves with characters in virtual en-
vironments? This relation between learners and characters in virtual environments
can be studied in several perspectives. In this paper, we will focus primarily on
the issue of empathy as one desirable aspect of the affective interaction between
learners and synthetic characters. In particular we will defend that in order for such
affective relations to happen, characters should be created and designed taking into
account what we call the proximity factor. This is based on the fact that children
are found to respond more empathically to those that are perceived as similar to
the self than those who are perceived as dissimilar [2]. This appears to be the case
when similarity is defined in terms of a shared characteristic, such as sex [6], race
or in terms of shared personal experiences [2]. Thus, designing characters aim-
ing at pedagogical empathic interactions, we should careful address how close the
learner will feel with the synthetic characters developed both in terms of situation,
behaviour or even physical appearance.
In order to illustrate this factor in eliciting emotional reactions to synthetic
characters, we will present a specific system called FearNot!. FearNot! was devel-
oped to address the difficult and often devastating problem of bullying in schools.
By using role playing and synthetic characters in a 3D environment, FearNot! al-
lows children from 8 to 12 to experience a virtual scenario where they can witness
(in a third-person perspective) bullying situations. To build empathy into FearNot!
we have considered the following components: agent’s architecture; the charac-
ters’ embodiment; the environment itself and emotionally charged situations. All
these elements were build to allow for a stronger proximity with the user and the
system. In this paper we will focus primarily on this problem and report some
results achieved in the evaluation done with 127 children and 95 adults on the
1 Introduction
Intelligent Virtual Environments (IVEs) bring new challenges to the way we use tech-
nology in educational contexts, promoting and creating new learning experiences where
experimentation and presence are explored. One of the big advantages of IVEs is that
they offer a safe place where learners can explore and understand through experimen-
tation without the dangers or problems of the real situations. Moreover, when IVEs
are augmented with contextual information, questions and activities, they can engage
learners in entertaining and motivating experiences, otherwise often considered as bor-
ing and uninteresting. Like computer games games, IVEs may allow learners to get
immersed and interact in synthetic worlds using a set of interaction facilities, such as
move, talk and specific actions with other characters. Inhabiting these IVEs there can
be agents or intelligent characters, that are responsible for events that happen in the
environment and make it not predictive or completely controlled. Characters can be
given the roles of teacher; helpers, companions, elements in the simulated worlds, or
even friends. They become the part of the environment giving liveness in the interaction
with the learners.
Given these aspects of Intelligent Virtual Environments they have been used suc-
cessfully in science and technology education (see for example [23] [15] [12]), as a
boost to these often scarcely appreciated areas of knowledge. However, when popu-
lated with animated characters IVEs may also offer users a safe environment where
they can explore and learn through experiential and entertaining activities in areas such
as social learning. However, this can only be achieved if the learners feel that the envi-
ronments do meet their expectations in terms of real to life and the synthetic characters
are believable enough to display the appropriate and expected behaviours. Thus, when
considering Social learning using IVEs, believability is perhaps one of the main goals
to attain. A believable character has been defined as a character that gives the illusion
of life and allows the user’s suspension of disbelief [3]. This quest for believability has
indeed been the Holy Grail of the area of synthetic characters for years. However, given
the nature of the concept (believability), several aspects are at stake. One of them is
the character’s appearance. Are more realistic characters more believable? And about
cartoon like characters?. One second factor that leads to belivability is the character’s
autonomy. Again, some results show that the more autonomous may seem more be-
lievable. See for example the case of the tamagochis. However, autonomy is difficult to
achieve in synthetic characters as there are tremendous technological difficulties, such
as for example the speech generation. Often, completely scripted characters lead to
more realistic and believable situations.
One other aspect to consider for believability is the character’s perceivable actions
and expressions. Expressivity is perhaps one of the most challenging problems of
synthetic characters, and work such as [21] [4] or [7] go in that direction. In particular,
the expression of emotions is understood as fundamental to achieve some degree of
believavility. In fact, according to Thomas and Johnston [27], animators from Disney,
there are three important points when expressing emotions: (1) the emotional state
of the character must be clearly defined, in such a way that is undoubtedly perceived
by the viewer; (2) the emotional state affects the reasoning process and consequences
must be perceivably reflected in the actions of the characters; and (3) emotions can be
accentuated or exaggerated, to clearly communicate to the viewer the emotional state of
the character. Another element is personality. A coherent character, that acts according
to its personality will be more believable [19] than a character that has no long term
coherence in its behaviour.
But on a whole, it is not so much one property or another that matters, but rather
the combination of all these factors, that together provide ingredients for building be-
lievability in a Pedagogical Virtual environment with synthetic characters.
However, when we watch a film, or read a book, we do not only suspend our dis-
belief and look at the characters as ”alive”, but we also establish emotional relations
with the characters, even if they are ducks, ants, cartoon or realistic. We feel sad when
they are sad, angry when something unfair is done to our favourite character, disap-
pointed when our character didn’t achieve what we wanted, and so on. That is, we
put ourselves in the shoes of the characters, and feel emotions about what is happen-
ing to them. So, together with emotional expression, autonomy and personality, we
believe that ”empathy” is also an important factor that can lead characters to become
Empathy can be defined in broad terms as ”An observer reacting emotionally be-
cause he perceives that another is experiencing or about to experience an emotion”. An-
other, less broad, definition is given by Wisp´
e that described empathy as ”the process
whereby one person ’feels her/himself into the consciousness of another person” [29].
Although animators and film makers have been doing it for years, creating embod-
ied lifelike autonomous characters that have the power to make the user feel emotional
reactions is still an unexplored research challenge, in particular, for educational pur-
poses. There are several factors to take into account and each one of them is, per se,
a research topic. In this paper we will discuss the role of empathy in the construction
of synthetic characters in VEs for educational purposes, focusing primarily on how to
build characters to evoke emotional responses (empathic responses) .
So, our main problem is:
How can we build synthetic characters that are able to evoke and establish em-
pathic relations with learners in a virtual environment?
To illustrate our approach to this problem we will rely on one particular example of
a pedagogical system, FearNot!, developed for addressing bullying problems in schools
using an interactive virtual storytelling environment. We will describe some of the
issues surrounding the development of FearNot! and the approach taken in terms of
character’s development. By focusing on the proximity factor we also describe the
results achieved by the evaluation of one first prototype of the system. The results show
that children do relate more strongly with the characters than adults, and consider the
characters, the situations and the environment believable. The results also show that
children do exhibit more empathy towards the developed characters than adults.
This paper is organised as follows: first we will review some related work, which
we considered relevant for our problem. Secondly we will describe the application
FearNot! in order to illustrate the problem, the situations and contextualize the research
presented. Then, we will describe our approach to building synthetic agents that were
designed to evoke empathy, not only because of their behaviour and architecture, but
also because of their physical appearance and the situations chosen. Finally, we will
describe an evaluation performed with a small prototype of FearNot! that shows the
relation children had with the characters and draw some directions for future work.
2 Related work
One of the most promising arenas for exploring believability of synthetic character is
education. The characters, when immersed into a learning environment can react to the
learner’s performance, respond adequately to the learner’s actions, help and advice the
learner in a more human like manner.
The benefits of using a character, by contrast to plain learning applications was
well explored by Lester who studied the effect of the presence of a lifelike character
in a interactive learning environment. Using an agent called Herman the Bug [14],
who inhabited a botanical world , through learning session, Herman would provide
guidance and advice to the learner in a quite lively and expressive manner. Herman
used a sequencing engine to dynamically sequence the animated set of behaviours and
thus give the illusion of life.
But perhaps the most influential pieces of work in the area of synthetic characters
in education was the work by J. Rickel [12] with the agent Steve. Steve (Soar Training
Expert for Virtual Environments) is a pedagogical agent that helps students to perform
physical, procedural tasks in a operations room where he had to operate and repair
complex equipment. Learners, whilst interacting with Steve were immersed in a 3D
computer simulated world where their work environment was modelled and learners
were able to practice their stills with the system. In spite of its relevance, this work
did not address carefully the relation established between students and Steve, and the
facial expressions of Steve were somehow quite limited.
Another very interesting example of the presence of children in a VE is the project
NICE, (The Narrative Immersive Constructionist/Collaborative Environments) that was
an exploratory learning environment for children. In such environment, the children are
asked to collaboratively construct, cultivate and maintain a healthy virtual garden [23].
The NICE project combined the ideas of constructionism, narrative, and collaboration
within a single environment. The interactions with the system take place in a CAVE, a
multi-person room-size virtual reality system consisting of three walls and a floor. The
users wear special lightweight stereo-glasses, which allow them to see the virtual and
real world without any transition and carry a special hand-held device for interaction -
called the wand. The children are represented in the virtual world by an avatar, which
interact with the many intelligent guides - who help them to maintain the ecosystem.
Although most of the learning environments with synthetic characters have been
developed in domain areas of science and technology, the role of these characters can
be also explored in social learning setting. One example for this is the well known
Carmen’s Bright IDEAS, an interactive health intervention designed to improve the
problem solving skills of mothers of pediatric cancer patients (Marsella, 2000). The
Bright IDEAS method is a method of social decision-making and problem solving,
applied in clinical settings, via a series of one-on-one sessions with trained counsel-
lors. To provide mothers with a way to explore decision making using the Bright Ideas
Method, an interactive narrative was produced. A professional scriptwriter aided by
the clinical professionals, who conceived and administered the Bright IDEAS method,
developed the story underneath the Carmen’s Bright IDEAS. The story is organized
into three acts: - in the first act, the learner (a mother) is presented with a sequence
of situation vignettes, which show some back-story for the Carmen character. This se-
quence aims at helping the learner to identify and empathise with Carmen. The second
act takes place in the counsellor’s office, where Carmen discusses her problems with
Gina, the counsellor. This discussion is done by first selecting a problem to analyse,
and develops through the evaluation of the possible solutions. The interactivity of this
application varies from act to act, but in general the user is always able to influence
the flow of action within the story and to develop a model of Carmen’s emotional state
that will guide the interactions during the story progression. The story progression in
this phase is highly dependent on Gina’s judgment since she is the one that based on
the thoughts selected, decides if a further elaboration is required or if the current topic
of the Bright IDEAS method is already sufficiently discussed. The application was
tested in a real setting and the results showed that the mothers identified themselves
with Carmen and with Carmen’s problems and found her solutions believable.
3 FearNot!
In order to illustrate our approach to the development of empathic agents in pedagogical
VEs, in particular, the proximity factor, we will present an application, FearNot!, where
empathy is at the center of the interaction between learners and characters.
FearNot! is a computer application being developed to tackle and eventually help
to reduce bullying problems in schools. Bullying behaviour is characterised as ”a
repeated action that occurs regularly over time, and usually involves an imbalance in
strength, either real or perceived” [8]. Bullying has associated with it a wide variety
of behaviours such as hitting, kicking or punching, in the case of direct bullying, or, in
relational bullying, social exclusion or malicious rumor spreading. It is this a repetition
of violent pressure over time (physical or verbal) that distinguishes bullying from other
types of violence, making its consequences so potentially dangerous and enduring.
Figure 1: A first prototype of the fearNot demonstrator
A wide range of anti-bullying initiatives have been developed, focusing on the vic-
tim, the bully or the whole problem. One approach uses live performance to dramatize
the problem, with actors presenting a narrative, followed by workshops in which groups
discuss the story. In an extension of this, Boal’s Forum Theatre [5] allows each group
to take responsibility for one of the characters and to meet with the actors ’in role’
between episodes of the story. Such an approach is however expensive and hard to
organise, while the presence of the group is often intimidatory (some members may be
bullying others) and can even emphasize existent conflicts.
Given the extension of the problem, a EU funded project (VICTEC) was created to
add to the already existing initiatives, a new one, based on Intelligent Virtual Environ-
ments. Through the implementation of a virtual Forum Theatre for education, one can
hope to create a safe environment in which individual children can explore different
perspectives on bullying behaviour. These were the main foundations that forged the
VICTEC project. Using state-of-the-art 3D interactive graphics and synthetic actors
(see Figure 3) , we expect to achieve individual interaction based on creating empathy
with the characters.
Thus, the overall pragmatic objective of the development of FearNot!, was to build
an anti-bullying demonstrator in which children age 8-12 experience a virtual scenario
where they can witness (from a third-person perspective) bullying situations. To avoid
group pressure and enable individualized interaction, the experience is for a single
user. The child acts as an invisible friend to a victimized character, discussing the
problems that arise and proposing coping strategies. Note that in bullying situations
there are quite clear identifiable roles: the bully, the victim,bully-victim (a child that is
sometimes the victim and sometimes the bully) and bystander.
The scenario begins by introducing the child to the school environment and the
characters, providing a starting context (see Figure 3) . This initial presentation pro-
vides the background needed about the characters in the story (a description of who is
the bully, the victim, and so on). Then, the episodes start. The whole session is devel-
oped one episode after another. Within an episode, the child is mostly a spectator of the
unfolding events (the narrative emerges from the actions of the participant characters).
After each episode, however, the victim will seek refuge in a resource room (identified
as a library) where a personalized conversation with the user can occur.
Figure 2: Interacting with FearNot!
Then, the child takes the role of a friend of the victim advising her on what to
do. A short dialogue takes place between the two, where the victim raises the main
events that occurred in the previous episode and asks for the child’s (learner) opinion
and suggestions for future behaviour. The dialogue established between the child user
and the victim character is done based on a set of patterns of response to bullying
situations, however, allowing the children to express the reasons and the expectations
for the advice given to the character victim. Nevertheless, note that the victim is clearly
recognized as a believable self, with its own personality and behaviour, and thus may
decide to reject the child’s suggestions (see Figure 2.
Each dialogue finishes with a decision that influences the character’s behaviour
in future episodes. Thus, episodes are not pre-scripted, and the characters act au-
tonomously, performing their roles in character. To ensure a user-centered experience,
Figure 3: An interaction window between the victim character and the child (for the
physical bullying and relational bullying)
the overall characteristics of each episode are decided by an external entity, a stage
manager (see [25] for more details). This entity selects appropriate places and charac-
ters that potentiate the occurrence of certain events favouring an authored educational
purpose. Nevertheless, the characters autonomously decide their actions.
Bullying, like many of our everyday problems, has no ’magic wand’ solution - only
more or less frequently successful strategies. The only universally accepted message
is that passivity is no solution, and one should never suffer in silence. The purpose of
the system is not to deliver a ’right answer’ but to present a multitude of options to
the child, and allow him or her to explore possible consequences for certain courses
of action. The use of an intelligent virtual environment with characters and emergent
narrative gives us that possibility.
4 Empathy in Pedagogical Lifelike Characters
The term ”empathy” stems from Titchener [28], who derived it from the Greek ”em-
patheia” which means ”passion”, ”passionate affection” or ”to be much affected” (Levy,
1997). Titchener used ”empathy” as a translation of the German term ”Einf¨
which means ”feeling into” somebody. Defined as ”the capacity of participating in or
vicarious experiencing of another’s feeling, volitions, or ideas and sometimes another’s
movements to the point of executing bodily movements resembling his” [1]. This de-
finition implies that, firstly, empathy is an internal state similar to an emotion; and
secondly that emotional state can sometimes be recognised through imitative bodily
movements. As reported by Plutnick [22] empathy is also found in animals and is a
widespread phenomenon in the animal world, involved in a wide variety of behaviour
patterns such as schooling or flocking. All these behaviours involve mimicry and af-
fective communication. In general, empathy refers not to processes between a person
and an object, but to processes between two persons, where one person perceives the
other. The perceiving person, or the persons who ”feels into” the other person, is called
the ”observer”, and the perceived person is called ”target”.
Most contemporary psychologists agree that there are two aspects of empathy that
have to be distinguished. The first one is the mediation of empathy and the second
the outcome of the empathic process. Concerning the mediation of empathy, one can
distinguish two different ways of mediating: (1) via the situation and (2) via emotional
expressions. When empathy is mediated via the situation, the observer concludes the
emotional state of the target from the situation the target is dealing with. For example,
if the observer perceives the target being hit by another, he may think that he would
be very angry in that situation himself. So the target will probably feel angry, too.
Empathy may also be mediated via emotional expressions of the target. This occurs
when the observer interprets the behaviour of the target, as for example, assuming that
when a target smiles he/she is probably happy.
These two aspects give rise to the empathic process, which in turn may have an
outcome. According to Davis [11] empathic process’s outcomes can either be cognitive
or affective. A cognitive outcome involves cognitive activity of the observer, such as
obtaining more information about the target or acting to help the target , whereas an
affective outcome (the one we usually consider as empathy) means that the observer
experiences an emotion because of his/her perception of the target.
Our main focus in this paper is on how to build characters to evoke emotional
responses (empathic responses) from the user. So, our main question is: How do we
build characters that are able to establish empathic relations with the learners in Virtual
To address this problem, we have put forward an hypothesis based on the idea of
proximity. There is evidence in literature that people experience more empathic emo-
tions when the incidents are associated with people with whom they have a communal
relationship (where communal relationships are friendship, romantic love or family
relationship). Also, people who perceive themselves to be similar to another also per-
ceive themselves as having stronger communal relationships with the other, and in turn,
experience more empathic compassion when the other is in need. For example, if the
target is of the same age and with similar features and attitudes. These findings suggest
that one way for the user to feel empathy and put him/herself in the place of a character
is to find similarities with between the user and the character, so that the user feels
similar to the character. So, what we call the proximity factor, tells us that, in order for
a synthetic character to evoke affective and cognitive empathy, users must feel close to
the character. Further, we argue that this is achieved by designing the whole environ-
ment and situations in a way that users feel some degree of familiarity and closeness
with the characters, environment and situations.
Based on this hypothesis in the design and character’s creation of FearNot! we took
several decisions where proximity was taken into the whole design process. Consider-
ing the two ways of mediating empathy we will address the problem considering:
mediation of empathy via the situation which has lead to the creation of episodes,
situations, behaviours and the environment that can lead to empathic reactions
by the users;
mediation of empathy via de emotional expressions has lead to the design and
creation of the behaviours, physical aspects and emotional expressions of the
From the start of the project we have involved children and teachers, and the char-
acters were evaluated by the children from their creation. This evaluation was aimed at
obtaining characters that children relate to and somehow are able to identify with. To
do that, proximity was considered at several levels of design as described.
5 Evoking empathy: The Situations and the Environ-
Bullying is episodic, where a sequence of similar situations - sometimes apparently
innocent on their own - builds into a serious issue that affects the victimized child.
Such a build up of situations is essential for the development of empathy. It is essential
that the child clearly acknowledges the situation as a bullying scenario, thus leading
to an understanding of the unfolding events and to the establishment of a stronger
empathic relation with the victim. To effectively achieve this, the early involvement of
teachers and children in the development of storyboards became an essential task for
the further progress of the project.
5.1 Capturing patterns of bullying behaviour
An essential aspect of the design and implementation of believable and interesting bul-
lying scenarios concerns the profiles and roles designated for each character depicted
within the scenarios. A number of research studies have been carried out to assess bul-
lying profiles and a classification of distinct characteristics are evident for ’pure’ bul-
lies, ’pure’ victim, bully/victim, bully/assistants, defenders and bystanders (e.g. [30]).
The social characteristics of bullying behaviour have also been researched to ensure
that the right contexts are implemented for the scenarios including gender issues, age,
the role of peers in bullying behaviour, and bullying as a group process.
To do that, we used Kar2ouche which is a high fidelity storyboarding tool that
allowed the creation of scenarios, populated with prototypical animated agents, that
aimed to capture both direct and relational bullying behaviour taking into account the
different group roles (bully, victim, bully/victim, defender, bully assistant). The story-
board scripts provided two different stories about direct and relational bullying devised
by experienced psychologists with expertise in bullying research. Each story is com-
prised of a series of episodes/chapters (see Figure 4. The storyboards were used as
an initial means of assessing character preference, engagement and empathy with the
characters and different bullying roles.
Figure 4 illustrates clips from a direct and a relational bullying scenario. In the
first clip we see the entire cast of a direct bullying situation, with a bully (Luke), a
victim (John) and bully assistants in the background. In the second clip we see the
victim (Frances) in a relational bullying situation with the bully (Sarah) and several
assistants. Both stories begin with an introduction and background information about
the protagonists. In the case of the direct bullying scenario this involves Luke (the
Figure 4: Storyboards done in the design of FearNot!
bully) knocking John’s (the victim) pencil case onto the floor in the classroom and then
pushing him off his chair when no one else is looking. Luke then verbally abuses John
and tells him to stop being a wimp and threatens John that he better not tell anybody
about the incident. This happens whilst the bullying assistants are egging Luke on.
The story proceeds to show John trying out a number of different coping mechanisms
including ignoring and trying to avoid Luke, fighting back with Luke and telling the
teacher. The story illustrates ignoring Luke and fighting back as being unsuccessful
strategies. The story ends with John telling the teacher and Luke being warned that if
he did not stop bullying John, he would have to leave the school.
Several patterns of episodes, based on what we obtained from children, were con-
structed in Kartouche. To test their adequacy and truth to life, we performed some
evaluation, where the children watched the direct and relational bullying scenarios and
then completed a questionnaire comprised of both structured and semi-structured ques-
tions. Questions enquired firstly about the direct bullying scenario and secondly the
relational bullying scenario. Questions about the child’s empathic feelings towards
the characters in both the scenarios followed. Direct Bullying Scenario Questions en-
quired about whether physical bullying happened at the respondents’ school, whether
they had experienced victimisation like ’John’ the victim in the scenario and whether
they had bullied anyone like ’Luke’ the bully in the scenario. If children answered yes
to experiencing or carrying out bullying they were asked to explain this in more detail.
Children were asked whether the speech used in the scenario was realistic and similar
to that used in their current school. Children then completed some questions about
coping strategies they would employ if they were in John’s (the victim) position and
were asked to explain why they would select a particular strategy. Children were also
asked what would be the worst thing to do to try and stop Luke bullying John. Finally,
children were asked why they thought Luke bullied John.
Children were also asked whether they felt sorry for any of the characters and if so
which characters and why, whether any of the characters made them feel angry and why
and finally, how they felt overall after watching both the direct and relational bullying
scenarios (very happy, quite happy, neither happy nor sad, quite sad and very sad).
Sample 80 children aged 9-11 with an average age of 9.7 years (SD: 0.66) partici-
pated in the present study involving two schools. One small rural school with children
from middle to upper class socio-economic status, and one larger urban school with
children from predominantly lower to middle social economic status participated. 43
boys and 37 females participated.
Children almost exclusively preferred the victims in the scenarios, with very few
children expressing a preference for any of the three bully characters. However, there
was a clear gender impact on most preferred character, with boys strongest preference
being for the male victim, whilst girls were not as gender specific. Children least
preferred the bullies, with a trend for boys to show least preference for the female
bullying characters. Boys indicated least preference for both the female bully and
the assistant. Girls were more evenly spread between the main bullies (Sarah and
Luke) and few least preferred the bully assistant. This suggests that girls are able
to distinguish between the severity of the role whereas the boys were focused more
on gender rather than character activity. The majority of the participants when asked
about ’prime character’ wished to be a victim, but one that was the same gender as
them. Notably, no boy expressed the desire to be a female bully and only 3 boys
were prepared to be a female character of any sort. This has important consequences
for the design of animated characters aimed at generating empathic relations. There
is a need for focused scenarios to be developed which offers children same gender
animated characters with whom to empathise. The situation seems to be more extreme
with boys, who clearly find it difficult to empathise with a female character. This could
be an effect of age, as in this middle school age group, girls are more socially and
cognitively developed which may enable them to take on both gender perspectives and
the different expressed behaviour patterns. The fact that most children would choose
to be a victim over the bullies may be an indication of the story plot, with the victims
working through to a successful outcome (i.e. the bullying stops).
These results were then used to impact the episodes and the characters built within
5.2 Generating episodes in real time
FearNot! can be seen as sequence of events in time that unfold bullying situations as
a form of episodes. Given that explicitly scripting all the possible situations remains
an unsurmountable task we need to make sure that the events generated by the appli-
cation do follow the structure and patterns detected in the experiments done. Thus, the
FearNot required a narrative control of some kind.
We can generically define a narrative as a sequence of events in time. Thus narra-
tive management is needed to guarantee that the generated situations lead to the em-
pathic relations we desire. We can structure narrative information through a hierarchy
of levels of abstraction. Although we can create as many levels as desired, stemming
from notions in drama arts, we envisioned the concepts of act, scene and beat. An act
is the narrative’s most abstract structure, representing significantly distinct sections of
a narrative (eg., in the case of FearNot!, the final educational message is seen distinc-
tively from the bullying situations). Each act can be seen as a set of scenes. Scenes
within an act hold related content, but represent still distinct narrative moments. Fi-
nally, each scene contains a pattern of beats that are the most basic elements within the
narrative. A beat describes event patterns that are relevant for the narrative within the
scene where they are active. An event signals some change in the world, usually caused
by the actions of the characters. Events are very low-level and are not considered at the
narrative level, although they can satisfy a pattern that constitutes a beat. A beat can
detect either just a simple event (e.g., a specific action) or a complex pattern of events
(e.g., a particular sequence of actions).
Building from these base concepts, bullying situations arise as specific instances of
narrative building blocks (acts, scenes and beats) using specific concepts. Such knowl-
edge is clearly domain-dependent and as such it must be drawn from experts in the field
through the use of knowledge elicitation tools (as we have seen in previous section).
This knowledge consists in specific nomenclature of concepts and the description of
their properties and relations. A victim, as a character role, is an example of a con-
cept of the bullying domain. Narrative information can be described using a simple
rule-based system, therefore easing the use of the elicited knowledge.
Nonetheless, long-term and abstract planning of a whole situation is extremely
complex. In a long-term, to build a complete situation, one would need to integrate
in the narrative all possible behavior traits that characterize each role, otherwise failing
to comply to behaviour believability. Following an agent-based approach, the use of
autonomous characters distributes this complexity thus enacting emergent narratives.
Although the situation is externally prepared, the characters autonomously decide their
actions, performing their roles in character (see next section for more detail on the
characters behaviours). For example, if we wish to potentiate a direct physical bullying
event, we can choose a situation involving the bully and the victim alone in the dressing
room. If some aggression is detected, the episode then halts and the system passes to
the reflection phase (where the child advises the victim).
Taking on ideas from role playing games, we can identify several levels of interven-
tion for narrative management[16]. To provide for a balance between author-induced
content and user’s free-play, narrative control can be carried at the levels of simulation
and presentation [26].
At the simulation level, the narrative only controls the episode (scene) settings by
placing the objects and characters in appropriate locales and sending a play com-
mand, letting the characters play their role within that episode. This is clearly
the solution that enables more interactivity and variability, because its results de-
pend on the character agent architecture and even on the user’s input (if the user
can intervene). To control the execution of the episode, the episode’s beats are
used to detect expected event patterns. When a beat event pattern is detected, its
associated rules are executed or/and other beats are activated to listen for other
event patterns. Associated rules may include the episode’s termination (in a suc-
cessful way) or indicate a counter-measure to avoid unintended pathways. These
counter-measures usually include refining the level of control.
The stage manager can directly control the characters by sending them orders
(for example, to force an action that is necessary for a specific event pattern to
succeed). This is a dangerous level because not only the view actions are not
finely described (i.e., there is still the danger that exists at simulation level of a
lack of control of the presentation), but it also requires a strong knowledge of
the character roles, to avoid unbelievable behaviour. It also makes interactivity
difficult, although not as hard as at the presentation level.
At the presentation level, the narrative can be described as a linear sequence (a script)
of view actions. This is the level that produces the best visual results, but requires
much more work, and is completely inflexible in terms of enabling interactivity.
The presentation level is free of domain, including view actions like ”play ani-
mation”, ”play sound”, ”move object”, ”zoom camera”. This level works as if
the stage manager is directly communicating with the view manager, while the
virtual space is not being used (the character agents are actually paused). In this
case, the stage manager will order (through the execution of an effector) the view
manager to execute a script, in the appropriate view action language.
A particular user experience will consist on a traversal of the narrative content struc-
ture. The stage manager is therefore used to guide such a traversal, using an appropriate
level of intervention. A first introductory act is composed of a single scripted scene,
depicting the introduction of the school environment, the characters and the situation.
Similarly, a final message act displays an educational message. In this case, though,
the particular script presented depends on what happened previously. The impact of
bullying events is greater if the user knows the characters. Since we wish to favour
the development of an empathic relationship, certain situations (mainly initially) must
increase the feeling of proximity. The main act constitutes the bullying scenario itself.
It starts with an initiating bullying episode that introduces the child to the problem that
is occurring. In this episode, a bullying incident is absolutely essential for the rest of
the interaction. Other acts follow, similar in nature but different in function. This entity
uses rule-based authored (by experts in bullying) knowledge, this way selecting appro-
priate places and characters that potentiate the occurrence of certain events favoring
specific authored purposes, drawing on situations that we clearly find in schools.
Each simulated episode within the bullying scenario act defines a set of encoun-
ters that enacts (rather than dictate) bullying situations. Each encounter is emerging,
and is defined in a way that autonomous characters, if designed according to the roles
they play, should effectuate the expected situation, although always in a different way,
according to a multitude of factors. Nevertheless, control mechanisms overlook at the
emerging behaviour and can take counter-measures to force a pedagogically accepted
turn of events. This notion of encounters [16]) led to the creation of the meta-scene,
which is an abstract scene that needs to be instantiated with appropriate data. This
concept was used in the (non)-bullying scene 2, which, according to appropriate initial
facts, can become an episode where no bullying incidents happen or one that such inci-
dents happen. For example, to create a non-bullying episode, we can start without the
bully and/or bully helpers.
5.3 Evoking empathy: The environment
As with the characters we have designed a set of schools (although all are cartoon like)
that resemble to different types of schools the children belong. So, for example we
have urban and countryside schools as well as country specific schools. For example,
Figure 3 shows one character in a urban Portuguese school, and 5 shows one of the
designed classrooms.
Figure 5: One of the characters in a classroom
6 Evoking empathy in FearNot!: The character’s be-
Characters must act in a believable way. So their actions must be generated in a way
that patterns of behaviour allow for children to recognise as common behaviours found
in school’s pupils. Given that we didn’t want to script the actions of the characters,
the architecture developed for our FearNot! synthetic characters allows for a dynamic
generation of actions in a believable and autonomous way.
However, to achieve a degree of believability and empathy neede, the architecture
developed must contain a way to trigger emotional states in the characters (which in
turn will lead to emotional expression- a mediator of empathy). Thus, we developed an
emotional module which is responsible for appraising the situations in a scene and acti-
vate emotional states, which in turn will lead to action tendencies and coping strategies.
The architecture also contains a representation of others, in particular their emotional
state and an action selection mechanism that leads the characters to act according to a
certain emotional state. The main aspects of this architecture are thus:
A model of the world that includes a model of the self with emotions represen-
tation and a model of the other agents (also an affective model);
The emotional model is parameterized for agent based in a personality profile
(see below);
An appraisal component;
An action selection component that depends on action tendencies associated with
the emotions represented;
A coping mechanism;
An affective expression component including body, facial expressions and speech.
Each agent in the world (a character such as Luke or John) perceives the environ-
ment, through a set of sensors (allowing the perception of events, objects, etc. in the
world) and acts on the environment though its effectors, allowing different actions to
be performed. For example, a bully may hit the victim and the victim may cry. The
agent architecture is shown in Figure 6.
Figure 6: Agent Architecture Diagram
Upon receiving a perception (which can, for example, be the presence of another
agent or an object, or even an action from another agent) the agent appraises its sig-
nificance and triggers the appropriate emotions. Additionally, if a goal has become
active, it will add a new intention to achieve the active goal. Intentions map directly
the concept of intentions in a BDI (Beliefs-Desires-Intentions) agent architecture [31].
They represent the agent endeavor to act in order to achieve the desired state.
After the appraisal process, it is necessary to choose the most adequate action. For
successful adaptation to the environment, emotions must have an effect on the actions
of the characters. For example, if our agent is sad it will act differently from a situation
when he is happy. However, the resulting actions are not always of the same type.
Lazarus [13] states that action tendencies are innate biological impulses, while coping
”is a much more complex, deliberate and often planful psychological process”. This
distinction also exists in characters’ actions in FearNot!. For example, if the Victim
character starts to cry when she is bullied, it is not because she has a goal that involves
crying. In fact, after this she will feel ashamed for crying instead of fighting back. The
victim crying is modelled as an innate reaction to a particular distressed emotional state
and the inability for fighting back. On the other hand, other actions, such as talking to
someone is a planned actions resulting from the internal goals of the agent.
Following the same ideas, the action selection mechanism in the FearNot! agents is
composed by two layers. The first one is the schematic layer where a set of action ten-
dencies triggered by particular emotions is defined. For instance, if the bully character
gets very angry, it will tend to kick everything in his path. A second layer is the concep-
tual/coping layer where two kinds of coping are defined. The first is problem-focused
coping, where the character tries to plan and act to achieve his goals; and the second
is emotion-focused coping that works by altering the character’s interpretation of the
environment. For example, an agent that feels distressed for not being able to achieve a
given goal, may generated emotion-focused coping by lowering the goal’s importance
and thus reducing his distress. In this way, emotions will not only influence the agents’
reactive behavior, but also guide the planning process, since emotional focused coping
changes the agents interpretation of its plans.
A description of these processes involved in the agent’s minds is provided next.
6.1 Appraisal
The emotional component of the architecture is based in Ortony, Clore and Collins
Theory of Emotions [20]. The use of OCC for the appraisal, by contrast with other
emotion architectures, allowed us to easily and pragmatically obtain patterns of be-
haviours in the characters (thus allowing for the creation of a bully or a victim in an
easy way). Thus, emotions are seen as valenced reactions to an event in the world.
The character’s emotions are triggered by an appraisal process, which can be seen as a
subjective evaluation of a given event according to the character goals, standards and
Our model uses two of OCC defined goal types, which are the active-pursuit goals
and interest goals. The active-pursuit goals are goals that the characters actively try to
achieve, like going to watch a football much in the school. Interest goals represent goals
that a character has but does not pursue, such as, for instance, wanting his favourite
team to win a match, or avoiding get hurt.
As proposed by Martinho [18], part of the triggering of emotions is handled by
a set of emotional reactions rules. An emotional reaction rule is composed by a do-
main specific construal frame extended with values for some of OCC emotion intensity
variables. The following figure shows three examples of these rules.
As seen on the examples, a reaction rule is composed by the event information and
the values for the appraisal variables. The first part is used to match the rule against the
perceived events, for instance the leftmost rule is selected if the event corresponds to a
cry action made by another character. The second part determines which emotions are
generated and the corresponding intensity.
Emotions can also be triggered by events that affect the plans the agent is pursuing.
However, instead of writing domain specific reaction rules to handle prospect based
reactions (as was done by [18], in this architecture we followed a similar approach to
Figure 7: Emotional Reaction Rules
the one used in the mile System [9] taking advantage of explicitly storing the agent
plans state and intentions into memory. With this approach, prospect based reactions
can be automatically obtained from the plans and goals active in the agent memory.
6.1.1 Emotion Generation
Each time an emotion is created, a potential value for the emotion is determined from
the event appraisal. However, emotions do not become active automatically. Each
character has a set of emotional thresholds and emotional decay rates (one for each
emotion type) according to his personality. The threshold represents the character re-
sistance towards an emotion type. The decay rate represents how fast the emotions of
an emotion type fade out. When the emotion intensity reaches zero it is removed from
the character emotional state. An emotion is added to the character emotional state
only if the emotion potential surpasses the defined threshold.
6.2 Action Selection
As mentioned before, the action selection mechanism is composed by two layers, the
schematic and the coping level. Since the schematic level defines action tendencies,
which represent innate reactions to the environment, they have priority over the Coping
level actions and thus are immediately executed. For example, if the bully most intense
emotion is Gloating at the victim (happy about something bad happening to the victim)
he will mock him (action).
6.2.1 Schematic Layer
The Schematic Layer implements the characters action tendencies. It consists on a set
of actions that are available according to the character’s emotional state.
The action selection mechanism starts by determining which actions can be exe-
cuted, by checking its preconditions. Afterwards it selects the action triggered by the
most intense emotion the character is experiencing. If the emotion intensity is greater
than the specified minimum, the action is executed. If more than one action rule is
selected (triggered by the same emotion), the most specific one is preferred.
6.2.2 Conceptual Layer
A continuous planner [24] that uses partial-ordered-plans builds up the core of the
conceptual layer. After the appraisal process, the planner selects the currently most
intense intention from the intention structure. The selected intention becomes the target
goal which the planner will try to achieve. Afterwards, the continuous planner removes
a flaw or executes an action (if the plan is complete). The resulting plan is stored with
the intention, so that it can be continued later on.
A continuous planner works by incrementally building, executing, and monitoring
a given plan. Since the planner continuously monitors the environment, it detects when
an action is accomplished or fails. It can handle unexpected events that affect future
plans and it can handle serendipity. Suppose that the planner has finished building a
plan to achieve a goal, if some other agent comes in and achieves some precondition
for us, the planner will detect that the condition holds true in the start step and will
remove the action used to achieve such precondition.
As the planner builds a way to achieve the goal, more than one different plan may
be construed. For instance, when removing an open condition, there may be more
than one alternative to achieve the condition (two different actions). Neither of the
alternatives can be forgotten without taking the risk of not finding the best course of
action, or even to find any solution whatsoever. So, instead of one single resulting plan,
it is necessary to store all alternative plans. Taking this into account, the planner must
select one from all alternative plans in order to continue planning or execution.
6.2.3 Plan Selection
In FearNot!, plans representation is based on classical planning and decision theory.
FearNot! generates Partially Ordered Plans which are modeled as a set of operators and
additional constraints. These operators are a slight modification of STRIPS operators,
associating probability values to the effects and represent the actions that an agent may
take in the world.
An action or effect probability is obtained from two sources: the character expe-
rience, whenever the character does some action it remembers the number of times
it was successful; and an interpretational bias. This bias allows the character to use
emotional-focused coping to change the subjective probability that a given effect oc-
After selecting the intention, the planner must also determine what the best plan to
continue the planning process is. We could select the one that seems more probable,
but unfortunately this method would generate a breath-first search on the plan space.
Since adding a step only decreases or maintains the plan probability, the planner will
select first plans with fewer steps (but also farther from the goal). Therefore we would
like that the planner also look at the number of open preconditions (how far is planner
of building a solution?). So it was used a heuristic function that looks at these parame-
ters. The plan chosen for processing is the one with the lowest value of h.
h(plan) = (numberOfSteps +numberOf OpenP r econditions)/P (plan)
The mechanism described so far corresponds to Problem Focused Coping, which
focus on acting on the environment (using planning abilities).
6.2.4 Emotion Focused Coping
Emotion Focused Coping works by altering the character’s interpretation of circum-
stances (for instance the probability bias of a particular effect). Notice, that by chang-
ing the interpretation, the appraisal process will generate distinct or weaker emotions,
thus allowing strong negative emotions do fade out and eventually disappear. Addi-
tionally, this interpretation change will affect the planning process.
Emotion Focused Strategies used in FearNot! are similar to the ones described
in Marsella and Gratch work [17]. The selection of a coping strategy is a two stage
process: first a coping opportunity is identified generating possible coping strategies;
and finally a coping potential is determined and used to decide if the strategy will be
used. All the generated strategies can be combined and applied at the same time. The
several emotion focused strategies will be described next.
Acceptance: Acceptance is the recognition that a goal is not possible to achieve or
protect. This strategy is generated on two distinct situations. When the planner
cannot build a plan to achieve the goal, the intention is removed and the agent ac-
cepts the goal failure. However, this strategy is generated even before the planner
tries all possibilities, if the goal probability goes below a defined threshold, then
an acceptance strategy is triggered. The agent starts to consider the hypothesis
of giving up the goal.
The other situation happens when it is not possible to achieve a goal without
violating an Interest Goal. In this situation the protection constraint is removed
from the plan so that the planner can introduce an action that violates the condi-
tion but achieves a needed precondition. This strategy is applied if the emotion
generated by the failure of the active-pursuit goal is of greater intensity than the
emotion generated by the failure of the interest goal.
Denial / Wishful Thinking Denial works by denying the reality of an event. It is
used when an event/action has an undesirable effect, whether it is an effect that
threatens a causal link, and no promotion or demotion resolves the threat, or an
effect that spoils a protection constraint of an Interest Goal. This strategy works
by lowering the effect probability so that the effect can be ignored. This strategy
is always applied if the effect probability is not too high.
Mental Disengagement When a desired goal seems unachievable, mental disengage-
ment works by reducing the agents ”investment” into the goal, i.e. the goal
importance is reduced. This strategy is selected if the character most intense
emotion is Fear for not achieving the goal. Since the character does not want to
feel Fear, one way of reducing it is to reduce its importance of failure. Given that
the prospect based emotions are automatically determining by the plan proba-
bility and importance, lowering the plan importance will immediately lower the
intensity of the Fear emotion.
Note that these strategies will also indirectly influence the planning process. By
changing goals importance, we may change the next intention to be selected by the
planner. By changing effects probability we are changing plan probability and therefore
a different plan can be chosen next to continue the refinement process. Also, changing
these parameters will lead to reappraisal that will generate different emotions, which
will change the planning process, and thus generating new coping strategies which will
once more lead to a new appraisal, and so forth.
6.3 Illustrative Example
In order to understand the entire architecture, let’s examine a small example. In one of
FearNot’s episodes, John (the victim) is sitting quietly in the classroom before the class
commences. The bully character, Luke, walks in and decides to push the victim books
to the floor, teasing him to get them (see Figure ??). When John gets up to pick his
books, he is pushed by Luke and falls. John starts to cry endlessly while Luke gloats
and threatens him.
Figure 8: Illustrative Example
In terms of what happens in the character’s minds, the first important event, when
the bully enters the classroom, he will receive perceptions from the world showing that
the victim and books are there. These perceptions trigger Luke’s active-pursuit goal of
bullying any of the victim’s props. As the goal becomes active (see previous secions),
an intention to achieve it is added to the conceptual layer. As this goal is quite important
to the bully, the intention draws Luke’s attention by generating a strong Hope emotion,
and therefore, being the most intense intention (and the only one for the moment). This
leads Luke to start building a plan in his ”mind” to accomplish that goal.
The plan is built very quickly, and Luke soon realizes that the plan is very likely
to succeed, thus reinforcing his hope emotion and his intention to do it. After the plan
is complete, Luke starts to execute it: he walks near the victim and pushes the books
on the table. The push event is appraised differently by the two characters. As shown
in the Figure 9, Luke has an emotional reaction rule specifying that such event is very
desirable to him and undesirable to the victim, and also he considers it praiseworthy.
Therefore, he will feel Joy, Gloating, Pride and Gratification. Additionally, the event
is appraised regarding his goals and given the bully goal is achieved, he will also feel
Figure 9: Example
John, on the other hand will feel Distress, Reproach and Anger. But, more impor-
tantly, the event causes an active-pursuit goal to become active. John does not like to
have his things on the floor, so he adds an intention to pick his books up and return
them to the table. So following the same process described above, he will build a small
plan to achieve such end. At the same time John is building his plan, Luke’s emotional
state will trigger an action tendency. He has an action rule specifying that when he
feels satisfied about bullying a victim’s object (pushing it, stealing it, etc) he will tend
to tease the victim. So, Luke will start a Tease ”Speech Act”, saying something like:
”come and get them, you muppet!”.
When John gets up and moves near the books, Luke sees a new opportunity to
bully the victim, so a new active-pursuit goal becomes active. This time, the goal is to
physically harass the victim. Once more the plan is built very quickly (since it is such
a small plan), and generates the same emotions as above. Therefore, at the same time
John is trying to pick his book up, Luke pushes him and makes him fall.
Like before, this event is appraised by the two characters. Luke feels even more
Joy, Gloating, etc and also Satisfaction for achieving his goal. At this moment, the
Gloating emotion has a quite high intensity thus triggering another action tendency.
When the gloating emotion surpasses a defined threshold, the bully starts to perform
Gloat/Mock Speech Acts. On the other hand the victim appraises the event regarding
his goals, and this particular event seems to thwart two of his goals: the active-pursuit
goal of getting his books, and an interest goal of being healthy (or not getting hurt).
Besides feeling disappointed, John distress becomes incredibly high (remember he was
already feeling distressed), and triggers his action tendency to cry.
At this stage, John’s mind can use emotion focused coping to mitigate its negative
emotions. Since the goal of getting his books seems unlikely for the moment, John uses
a mental disengagement strategy to lower the importance of this goal. By lowering the
goal’s importance, the intensity of the Fear emotion (for not achieving the goal) is also
reduced. On the other hand, John uses a denial/wishful strategy to lower the probability
that the push action has the effect of hurting him. Such probability change leads to a
different appraisal of the threat to John interest goal of being healthy, which seems
weaker and more unlikely, and thus reducing his distress. In other words, John starts to
think something like ”Well, he did not hurt me after all.” or ”it did not hurt that much”
and slowly calms down and stops crying.
On the other hand, the crying action is appraised once more by both characters,
repeating the same processes described above. Finally, the episode ends with Luke
threatening John to not tell anyone (or to stop crying) and with poor John feeling very
ashamed for crying and not doing anything.
7 Evoking empathy: The character’s physical aspects
In FearNot! characters are 3D embodied characters, which means that we can use facial
expressions, attitudes, body expressions to convey their emotional states.
In order to design the characters we made some preliminar tests with children and
designed two types of characters: realistic versus cartoon like. Although at first, and
according to the proximity argument, we should adopt realistic characters it was clear
from the studies and results that learners of this age preferred the cartoon characters
(see Evaluation section). Inspired by the very popular characters from a Portuguese
children’s web portal (Cidade da Malta in originally in
2D, we have converted the characters into 3D and adopted them adequately for the age
and gender groups involved.
Furthermore, the characters and the situations for the age groups we are targeting
range a quite distinct set of children’s appearances (see a set of characters for the UK
version in Figure 12) so that children can easily identify with one or another character
.For each country (UK, Portugal and Germany), we designed different characters
given that children in the UK have uniforms and in Portugal and Germany do not.
We also have considered specific situations for both genders (more direct bullying
for boys and relational bullying for girls, see Figure 11).
7.1 Evoking empathy: Conversations and Expressions
Further, and as described earlier, empathy can be mediated in an affective way, so
the characters must be able to express emotions in facial expressions, voice and body
posture. If the user perceives the agent expressing emotions that are adequate to the
displayed situation, believability and empathy should increase. In FearNot! we use
Figure 10: Three of the characters developed for the FearNot! application (John, the
victim, Martinha, the neutral and Luke, the bully)
mainly facial and body expression. A precondition therefore is that the emotional ex-
pression can be recognized by the user correctly. Another possible mode of emotional
expression that avoids the danger of misinterpretation is language. The agent could
inform the user about his emotional state verbally. One should note that the cognitive
component of empathy would be realized if the user has the impression that the virtual
agent ”knows” something about the user’s inner state.
In order to clearly convey the character’s emotional state it was easier and more ef-
fective to adopt cartoon like characters. In fact, tests carried out with children in associ-
ated schools revealed that children preferred the cartoon characters. This also reduces
the importance of using complex and resource intensive real-time facial animation and
lip-sync. Simple textured faces (see Figures 10 and 13) can be very believable (even
more believable than perfectly modelled faces).
As for conversations, we collected from schools several scenarios with typical di-
alogues and even with common aggressive names uttered between children of the tar-
geting age. These dialogues are used as patterns for the language generation system.
Figure 11: Girls characters for the relational bullying scenarios
8 Evaluation and Results
The main focus of this evaluation was to consider the different perspectives and em-
pathic reactions of adult and child populations with the system. The main questions we
were seeking to answer were: Are there differences in the views, opinions and attitudes
of children and adults? And, if there are differences in the empathic reactions to the
system and do these have any important design implications for empathic embodied
8.1 Experimental Design
Using one limited version featuring a single bullying episode of FearNot! already
released and evaluated with several types of users, we have conducted a set of experi-
ments in three different countries: UK, Portugal and Germany. All the main aspects of
the architecture were already in place and the characters built follow the requirements
presented. Children were shown the trailer of the FearNot! which depicts one physical
bullying episode developed by experts in bullying research in conjunction with teachers
and pupils. All characters and animations, places and objects were transposed to 3D
by the team of designers according to the principles just described. The dialogues were
obtained through the recording of real voices. The physical bullying episode contains
3 characters, Luke the bully, John the victim and Martinha the narrator. The trailer
begins with an introduction to the main characters, Luke and John and subsequently
shows Luke knocking John’s pencil case off the table and then kicking him to the floor.
John then asks the user what he should do to try and stop Luke bullying him and arrives
Figure 12: The students developed for the UK schools
Figure 13: Example of Some Facial Expressions in Characters
at 3 possible choices: 1) Ignore Luke, 2) Fight back, 3) Tell someone that he trusts such
as his teacher or parents. Developmental constraints of the application did not allow us
to include the dialogue phase in the first trailer developed. Nonetheless, the importance
of the dialogue phase for the overall success of the application required us to include it
in the demonstrator (as briefly stated).
A questionnaire applicable for children and adults was designed in order to eval-
uate aspects of FearNot!, the VICTEC bullying demonstrator. The questionnaire was
divided into 7 main sections and was predominantly measured according to a 5 point
Likert scale. Table 1 illustrates the main sections of the questionnaire and the nature of
the questions within each section.
Section Nature of question
1 Preference for cartoon or
realistic characters
2 Characters attributes
- voice believability
- likeableness
- conversational content
(believable to unbelievable)
- conversation interest (interesting or boring)
- realism of characters (true to life to false)
3 Character Movement
- movement believability (believable to
- realism to movement (realistic to unrealistic)
- smoothness of movement (smooth or jerky)
4 Appearance of the school environment
5 Bullying Storyline
- storyline believability (believable to
- unbelievable)
- storyline lenght
6 Character preference
(character likes most and character liked least)
7 Empathy towards characters
Feeling sorry for characters
(and if yes which character)
Feeling angry towards the characters
(and if yes which character)
Two hundred and twenty five questionaires were done, out of which 128 by children
from schools in England and Portugal. The remaining were done by adults (teachers
and experts). These are some of the results (more on these results can be found in [10])
that show partially the significance of the proximity factor.
8.2 School Environment
Significant differences were revealed between child’s, expert’s and teacher’s views of
the appearance of the school environment in the trailer. Post hoc tests showed that
these significant differences were between the child and expert views and between the
child and teacher views for the attractiveness of the school environment, indicating that
the children viewed the school environment more positively than experts and teachers.
Furthermore, there were also significant differences between child, expert and teacher
views in relation to the match between the environment and the characters. These sig-
nificant differences lay between the teacher and the child, where children were signifi-
cantly more positive towards the match between the school environment and characters
compared to teachers. We found this result quite positive as we wanted children to feel
as close as possible with the environment.
8.3 Character Movement
Concerning the character’s movement, there were significant differences between the
stakeholder groups and views of the believability of character movement. Children
thought that the character movement was significantly more believable than teachers,
which again is quite positive. Overall, no significant gender differences were revealed
for the believability of character movement, however, when age was taken into account,
female children found character movement significantly more believable compared to
female adults who found the character movement least believable. Again this show
that our design was able to inspire more belivability to the right target users. Signifi-
cant differences emerged for views of the realism of character movement where chil-
dren thought that the character movement was significantly more realistic compared to
teachers and experts. An independent samples T-test revealed significant gender dif-
ferences for the realism of character movement. Females found character movement
significantly more realistic than males. When age was considered, female children
found the character movement significantly more realistic compared to male children,
male adults and female adults.
8.4 Conversation and Storyline
Significant differences were found in the views of the true-to-lifeness of character con-
versation, where teachers found the character conversation significantly more false and
less true to life compared to children. Given our effort in obtaining real children di-
alogues and scenarios, this again is a quite positive result. Furthermore, significant
differences were found between groups for views of the believability of the storyline
where children found the storyline significantly more believable than teachers.
8.5 Affective and Cognitive Empathy
Significant differences were found between children, experts and teachers for affective
empathy. Significantly more children (80%) expressed feeling sorry for the characters
compared to teachers and experts (70%). Affective empathy was only expressed for
Luke and John, and not for Martinha. Significant differences were uncovered for age
and gender, (x=15.02, N=213, df=3, p=0.002) where significantly more female children
(95%) expressed affective empathy compared to male adults (67%).
Significant differences were found between children, experts and teachers for cog-
nitive empathy. Significantly more children (71%) expressed cognitive empathy to-
wards characters compared to experts (47%) and teachers (28%). Significantly more
experts expressed anger towards John (the victim) compared to children and teachers
and significantly more teachers expressed anger towards Martinha compared to experts
and children. Interesting significant age and gender differences emerged, where sig-
nificantly more female children expressed anger towards the characters compared to
adults. This anger was almost exclusively directed at Luke (90%), the bully , which
again is a very positive result because it shows that FearNot! can evoke emotions to
children of the age that we are targeting.
9 Final Discussion
In this paper we have provided a discussion on some of the features needed to build
characters that are able to establish an empathic relation with users. One of the issues
here described is what we call the proximity factor that tells us that, in order for a syn-
thetic character to evoke affective and cognitive empathy, users must feel close to the
character. Further, we argue that this is achieved by designing the whole environment
and situations in a way that users feel some degree of familiarity and closeness with
the characters, environment and situations.
To illustrate these issues we have presented a system FearNot! that has been devel-
oped to address bullying problems in schools using empathic synthetic characters. We
have described some design decisions of FearNot! and discussed the results attained
with the trailer, in particular the aspects associated with the proximity of the characters
with the user.
10 Acknowledgements
The work here reported is part of the VICTEC project. We would like to thank all the
partners in the project for their contributions in some of the issues here reported.
[1] Webster’s Third New International Dictionary. 1971.
[2] M. Barnett. Empathy and related responses in children. In Empathy and its
Development. Cambridge University Press, 1987.
[3] J. Bates. The role of emotion in believable agents. Technical Report CMU-CS-
94-13, Carnegie Mellon University, 1994.
[4] R. Bindiganavale, W. Schuler, J. Allbeck, N. Badler, A. Joshi, and M. Palmer.
Dynamically altering agent behaviors using natural language instructions. In Pro-
cedings of the International Conference on Autonomous Agents (AA 2000). ACM
Press, 2000.
[5] A. Boal. Legislative theatre: Using performance to make politics. Routledge,
[6] K. Bryant. An index of empathy for children and adolescents. Child Development,
53, 1982.
[7] J. Cassell. Nudge nudge wink wink: Elements of face-to-face conversation for
embodied conversational agents. In e. a. J. Cassell, editor, Embodied Conversa-
tional Agents. MIT Press, Cambridge, MA, 1999.
[8] K. Dautanhahn and S. Woods. Possible connections between bullying behaviour,
empathy and imitation. Technical report, 2003.
[9] J. Gratch. Emile: Marshalling passions in training and education. In Autonomous
Agents 2000. ACM Press, 2000.
[10] L. Hall, S. Woods, K. Dautenhahn, D. Sobral, A. Paiva, D. Wolke, and L. Newall.
Designing empathic agents: Adults vs. kids. In Intelligent Tutoring Systems.
Springer, 2004.
[11] M. H.Davis. Empathy: a social psychological approach. Dubuque: Brown and
Benchmark Publishers, 1994.
[12] R. J. and J. L. Integrating pedagogical capabilities in a virtual environment agent.
In L. Johnson and B. Hayes-Roth, editors, Autonomous Agent ’97. ACM Press,
[13] R. Lazarus. Emotion and Adaptation. Oxford University Press, 1991.
[14] J. Lester, S. Converse, S. Kahler, T. Barlow, and R. Bhoga. The persona effect.
In Proceedings of Computer Human Interaction’97. ACM Press, 1998.
[15] J. Lester, S. Towns, C. Callaway, J. Voerman, and P. FitzGerald. Deictic and
emotive communication in animated pedagogical agents. In C. J., P. S., and S. J.,
editors, Embodied Conversational Agents, Cambridge, USA, 2000. MIT Press.
[16] S. Louchart and R. Aylett. Narrative theory and emergent interactive narrative. In
Proceedings of 2nd International Workshop on Narrative and Interactive Learn-
ing Environments, 2002.
[17] S. Marsella and J. Gratch. Modeling coping behavior in virtual humans: don’t
worry, be happy. In AAMAS 2003, pages 313–320. ACM Press, 2003.
[18] C. Martinho and A. Paiva. Underwater love: Building tristao and isolda’s person-
alities. In M. Wooldridge and M. Veloso, editors, Artificial Intelligence Today.
Springer, 1999.
[19] R. Moffat. Personality parameters and programs. In Creating Personality for
Synthetic Actors. Springer, 1997.
[20] A. Ortony, G. Clore, and A. Collins. The Cognitive Structure of Emotions. Cam-
bridge University Press, New York, reprinted 1994 edition, 1988.
[21] C. Pelachaud and I. Poggi. Interacting through different modalities: Multimodal
communication between synthetic agents. In Proceedings of the working confer-
ence on Advanced visual interfaces, 1998.
[22] R. Plutchik. Evolutionary bases of empathy. In N. Eisenberg and J. Strayer,
editors, Empathy and its Development. Cambridge University Press, 1987.
[23] M. Roussos. Issues in the design and evaluation of a virtual reality learning envi-
ronment (msc). Technical report, University of Illinois at Chicago, 1997.
[24] S. Russell and P. Norvig. Artificial Intelligence: A Modern Approach. Prentice
Hall, 1995.
[25] D. Sobral, I. Machado, and A. Paiva. Managing authorship in plot conduction. In
G. Goos, H. Hartmanis, and J. Leeuwen, editors, Virtual Storytelling. Springer,
[26] D. Sobral, I. Machado, and A. Paiva. Managing authorship in plot conduction. In
Virtual Storytelling: Using Virtual Reality Technologies for Storytelling. Springer,
[27] F. Thomas and O. Johnston. The Illusion of Life: Disney Animation. Walt Disney
Productions, 1981.
[28] E. Titchener. A textbook of psychology. New York: Mcmillan, 1924.
[29] L. Wisp´
e. History of the concept of empathy. In N. Eisenberg and J. Strayer,
editors, Empathy and its Development. Cambridge University Press, 1987.
[30] D. Wolke, S. Woods, H. Schultz, and K. Stanford. Bullying and victimisation
of primary school children in soth england and south germany: Prevalence and
school factors. British Journal pf Psychology, 92, 2001.
[31] M. Wooldridge. An Introduction to Multi-Agent Systems. John Wiley and Sons,
Ltd, 2002.
... De plus, la réalisation de la représentation humaine doit être menée avec précaution pour éviter la vallée de l'étrange et la mauvaise appréciation des utilisateursMori et al., 2012. Les caractéristiques d'apparence visuelle comme le genre ou l'ethnie ont des effets positifs sur les médiateurs relationnels(Paiva et al., 2005) lorsqu'ils sont utilisés de manière à mimer le genre ou l'ethnie de l'utilisateur(Qiu et Benbasat, 2010) et que celui-ci parvient à s'identifier à l'agent. Cependant, il existe une contradiction dans la littérature sur la caractéristique du genre puisque certaines études font état d'une préférence pour les agents de même genre et d'autres rapportant l'inverse(Qiu et Benbasat, 2010 ;Nowak et Rauh, 2005).Powers et al. (2005) pensent que cette contradiction provient du rôle important que jouent les persona dans une interaction humain-Agent. ...
Ce travail de thèse a été consacré à l’exploration de l’usage de deux technologies qui vont à terme transformer notre quotidien : les assistants virtuels et la voiture autonome.Les assistants virtuels ont déjà une place importante dans nos vies et sont en train de révolutionner notre façon d’interagir avec les systèmes en nous proposant une interaction vocale. Nous pouvons désormais, par une simple phrase, obtenir des informations sur la météo ou mettre de la musique. Les voitures autonomes pour leur part, bien qu’elles ne soient pas encore accessibles au public, portent la promesse d’améliorer le confort de la conduite, de réduire les accidents et de fluidifier la circulation sur les routes. Cependant, l’adoption d’une telle technologie nécessite de la confiance de la part des utilisateurs. Il semble que les assistants virtuels, par la nature même de leur interface anthropomorphique, puissent jouer un rôle dans ce contexte. Nous explorons donc le potentiel des assistants virtuels pour augmenter la confiance dans la conduite autonome.Les questions principales abordées dans ce travail concernent d’une part les choix adaptés pour concevoir un assistant virtuel afin que celui-ci soit perçu comme anthropomorphique et digne de confiance. D’autre part, elles concernent l’impact que peut avoir une telle interface dans un cockpit de véhicule autonome sur la perception d’anthropomorphisme et la confiance des utilisateurs. Pour répondre à ces questions, nous avons, dans un premier temps, choisi l’apparence visuelle de l’assistant en évaluant l’impact de différentes représentations visuelles sur la perception d’anthropomorphisme et de confiance. Notre choix s’est porté sur une représentation de type Automate-Humain. Puis, nous avons implémenté cette représentation en trois dimensions et intégré le résultat dans un simulateur de conduite sous la forme d’un hologramme. Pour évaluer l’assistant virtuel, nous avons conduit une expérimentation pour comparer une interface de référence sans assistant virtuel avec deux interfaces intégrant deux versions de l’assistant virtuel. Les résultats montrent que la perception d’anthropomorphisme ne s'accroît pas avec le niveau d’anthropomorphisme. Une corrélation significative vient confirmer l’impact de la perception d’anthropomorphisme sur la confiance. D’autres résultats plus surprenant concernant l’impact de l’assistant virtuel sur la performance ou encore l’impact de l’expérience acquise sur la confiance sont discutés.
... Also, research on CyberBullet (Mikka-Muntuumo et al., 2018) and Conectado (Calvo-Morata et al., 2018) presented its design and development as a single player game, whereas we propose a multiplayer game. The latter also presented a single game session, such as other games like FearNot! (Paiva et al., 2005), whereas we propose multiple sessions. However, like with other games, such as the The #StopBully app (Neo et al., 2018), the Inn Inoue game (Higashino et al 2019), the Bully Book (De Troyer et al., 2016), Cyberhero Mobile Safety game (Hswen, et al., 2014), and Cooperative Cybereduca 2.0 (Garaigordobil & Martínez-Valderrey, 2018), research provided no information regarding CyberBullet's effectiveness in promoting prosocial and empathic behavior (Mikka-Muntuumo et al., 2018), which we present in this study regarding our game. ...
... (C) Using and expressing emotions: utilize the measured emotions and current state of mind to drive one's thoughts, take action, choose responses, empathize, and express appropriate emotions using verbal and nonverbal cues. Recently, there have been several studies that investigate incorporating empathy in social robots [24], [25], [26], [27]. This is mainly due to advances in emotion recognition in different modalities. ...
Full-text available
This paper presents our recent research on integrating artificial emotional intelligence in a social robot (Ryan) and studies the robot's effectiveness in engaging older adults. Ryan is a socially assistive robot designed to provide companionship for older adults with depression and dementia through conversation. We used two versions of Ryan for our study, empathic and non-empathic. The empathic Ryan utilizes a multimodal emotion recognition algorithm and a multimodal emotion expression system. Using different input modalities for emotion, i.e. facial expression and speech sentiment, the empathic Ryan detects users' emotional state and utilizes an affective dialogue manager to generate a response. On the other hand, the non-empathic Ryan lacks facial expression and uses scripted dialogues that do not factor in the users' emotional state. We studied these two versions of Ryan with 10 older adults living in a senior care facility. The statistically significant improvement in the users' reported face-scale mood measurement indicates an overall positive effect from the interaction with both the empathic and non-empathic versions of Ryan. However, the number of spoken words measurement and the exit survey analysis suggest that the users perceive the empathic Ryan as more engaging and likable.
... Empathy is an essential part of building rapport and creating a bond, or a working alliance, between the patient and therapist to improve patient adherence and satisfaction (Bennett et al. 2011). Empathy has been defined as 'an observer reacting emotionally because he perceives that another is experiencing or about to experience an emotion' (Paiva et al. 2005), (p.4), or 'the process whereby one person feels her/himself into the consciousness of another person' (Wispé 1987). Empathy is expressed through a wide range of nonverbal behaviours, such as mirroring head nods, and verbal behaviours, such as the choice of words in social dialogues, the use of greetings and farewell rituals and the art of bringing continuity in the conversation (Laver 2011). ...
Embodied Conversational Agents (ECAs) are virtual agents that exhibit humanlike verbal and non-verbal behaviours. When it comes to eHealth, ECAs can provide vital support to patients by being more reachable. In order to make ECAs more effective, humanlike empathy expressed during conversation through relational cues is essential. Empathy revolves around a wide range of verbal and non-verbal behaviours that include, for example, the choice of words in social dialogues. Owing to the COVID-19 situation, there was an opportunity to record online consultations in the Incontinence Clinic and Sleep Clinic at the Children’s Hospital at Westmead in Sydney, Australia. The present study analysed these human dialogues using qualitative research methods to understand the role of empathic dialogue used by the medical team. The qualitative analysis of the live calls used psychology-based relational cues derived from conversational characteristics of humans to build a coding framework around the most relevant themes. Statistical analysis was used to compare relational cue usage between healthcare roles. Finally, using the framework dialogues of the medical team and two ECAs known as Dr Evie (eVirtual Agent for Incontinence and Enuresis) and SAM (Sleep Adherence Mentor) were compared to provide recommendations for health practitioners and future ECA dialogue development.
Full-text available
Giftedness is a phenomenon that has from time to time occupied the research community in terms of its definition, its characteristics and the factors that contribute to its manifestation. Research has shown that charisma not only corresponds to a high mental ability but also includes other concepts such as that of consciousness. In the pyramid of charisma, in fact, consciousness is at the top of the hierarchy. The present study was based on articles in scientific journals, in order to investigate some of the factors that enhance the manifestation of giftedness. The results showed that Information and Communication Technologies (ICT), environmental factors (family, school) and other complementary factors play an important role in the concept of giftedness.
Full-text available
Araştırmada organizasyonlarda eşitlik kuramı kapsamı içerisinde yer alan eşitlik algısı çerçevesinde; çalışma ortamı memnuniyeti, empati, uyum ve duygu yönetimi kavramları ortak unsurlar açısından değerlendirilmeye çalışılmıştır. Çalışma ortamında bireylerin eşitlik algıları bu kavramlar üzerinde etkili olabilir. Yani eşitlik algısı, bireyin çalışma ortamından memnun olmasına, diğer çalışma arkadaşlarına yönelik empati kurabilmesine, çalışma ortamına uyum sağlamasına ve çalışma arkadaşları ile olan diyaloglarında duygu yönetimine önemli katkılar sağlayabilir.
Virtual reality (VR) has widely been implemented in a variety of industries, with immersive storytelling now gaining popularity in the field of journalism. In this regard, there are many important questions about which direction modern journalism should take as a whole. To test the feasibility of immersive storytelling, this study developed a cognitive experience model containing paths for presence, flow, credibility, empathy, understanding, and enjoyment. A total of 131 participants were divided into three groups and exposed to either VR, 360-degree videos, or two-dimensional (2D) videos, then asked to rate their experiences. With the exception of understanding, results showed that the VR and 360-degree videos were more highly evaluated in each path when compared with 2D videos. In this case, technical attributes did not greatly affect the study variables. However, presence played a vital role in news promotion, while flow was positively affected by presence; flow also affected other variables when combined with presence. Finally, a structural equation model was tested and the weights of the different influences on the enjoyment impact were presented. In sum, this study found that immersive storytelling improved both presence, flow, and credibility, which jointly and positively affected enjoyment. Understanding negatively affected enjoyment. Empathy had little effect on enjoyment in the model.
1. Introduction The study of emotion Types of evidence for theories of emotion Some goals for a cognitive theory of emotion 2. Structure of the theory The organisation of emotion types Basic emotions Some implications of the emotions-as-valenced-reactions claim 3. The cognitive psychology of appraisal The appraisal structure Central intensity variables 4. The intensity of emotions Global variables Local variables Variable-values, variable-weights, and emotion thresholds 5. Reactions to events: I. The well-being emotions Loss emotions and fine-grained analyses The fortunes-of-others emotions Self-pity and related states 6. Reactions to events: II. The prospect-based emotions Shock and pleasant surprise Some interrelationships between prospect-based emotions Suspense, resignation, hopelessness, and other related states 7. Reactions to agents The attribution emotions Gratitude, anger, and some other compound emotions 8. Reactions to objects The attraction emotions Fine-grained analyses and emotion sequences 9. The boundaries of the theory Emotion words and cross-cultural issues Emotion experiences and unconscious emotions Coping and the function of emotions Computational tractability.
Conference Paper
Believability is one of the key concerns when developing synthetic characters in intelligent virtual environments. To achieve believability, the virtual characters must behave consistently with their assumed or perceived personality. This paper describes a model for the construction of believable emotional characters with synthetic personae in intelligent virtual environments. The model assumes that personality and emotions are essentially the same mechanism and allows the definition of the characters personality by a set of emotional reactions. The system designer defines a set of concepts, which structure is inspired by Ortony, Clore and Collins's theory of emotions, that will then automatically integrated in the IVE character cycle, inspired by Fridja's theory of emotions. Then, the system designer implements the final IVE specific characteristics around those concepts. The methodology was applied to a real time IVE, S3A, developed in the context of the last world exposition of the century, EXPO'98, which featured two pathematic dolphins, Tristão and Isolda, who lived in the synthetic estuary of the river Sado during the four months of the EXP0'98, and were displayed to more than one million visitors. The project showed that the followed approach is a viable solution for the creation of the synthetic personae of believable emotional agents in intelligent virtual environments.
This study presents the development and validation of an index of empathy for use with children and adolescents. 56 first graders, 115 fourth graders, and 87 seventh graders were studied. Item means, item-total correlations, test-retest reliabilities, correlations testing the relationship of empathy to aggressiveness and acceptance of individual differences, correlations testing the relationship of this adapted index of empathy to other existing measures of empathy as well as to social desirability response set and reading achievement formed the basis of internal, discriminant, convergent, and general construct validation. The measure demonstrated satisfactory reliability and preliminary construct validity. The study of a subset of items controlling for same-sex versus cross-sex stimulus figures provided the basis for investigating developmental aspects of empathic arousal toward peers of different sexes. Overall, the availability of comparable forms of a measure of empathy for use with children, adolescents, and adults will be useful for exploring the developmental antecedents and conditions surrounding the expression of emotional empathy.