Content uploaded by Michael Leonard Walters
Author content
All content in this area was uploaded by Michael Leonard Walters
Content may be subject to copyright.
The Negative Attitudes towards Robots Scale and
Reactions to Robot Behaviour in a Live Human-Robot
Interaction Study
Dag Sverre Syrdal, Kerstin Dautenhahn, Kheng Lee Koay, Michael L. Walters,1
ABSTRACT
This paper describes the use of the Negative Attitudes Towards
Robots Scale (NARS) to explain participants' evaluations of robot
behaviour styles in a Human-Robot Interaction (HRI) study. Twenty-
eight participants interacted with a robot in two experimental
conditions in which the robot’s behaviour was varied. Reliability
analysis and a PCA was performed on the NARS items, creating three
new subscales. Correlations between the subscales and other
evaluations of the robot's behaviour found meaningful results,
supporting the use of the NARS in English speaking samples.
1 INTRODUCTION
The LIREC (LIving with Robots and intEractive Companions)
project aims to investigate theoretical aspects of artificial
companions and test these across a wide range of
embodiments in different contexts and environments [1].
While research in the LIREC project has a strong
technological component, the development of tools for
standardised measurements of the quality of interactions,
perceptions of robots and agents and pre-existing attitudes
towards such agents, is very important in order to establish a
means of objectively comparing the success of agent and
robot behaviour across a wide range of interactions and
embodiments.
Dautenhahn [2] suggests that current work in human-robot
interaction(HRI) is characterised by heterogeneity, both in
terms of methodologies and measurements used to study
technologies and their impact. This allows the field of HRI to
accumulate a wide range of information regarding the use and
perceptions of different robots in different contexts, but also
limits the replicability of results across the field as a whole. It
is understood that the technology driven nature of the field,
and as such, the need to evaluate specific technologies in
specific contexts, is a motivating factor for conducting such
research. It is, however, important to note that this may
become a problem for the HRI community in the long-term as
the lack of common benchmarks and measures may hamper
communication and application of results across different
research groups and projects, and thus the advancement of the
field as a whole.
The multidisciplinary nature of the LIREC consortium ,as
well as the qualitative differences between agent embodiments
and use-contexts across the project, suggests that this issue is
not only of particular interest, but also provides a unique
opportunity to study the efficacy of measures intended to
measure attitudes to robots and their behaviours across diverse
situations.
2 THE NEGATIVE ATTITUDES TOWARDS
ROBOTS SCALE
In order to establish an understanding of how the behaviour
and embodiment factors of a robot are perceived and
responded to by potential users, such tools will also be needed
to establish an understanding of the idiosyncratic factors in the
individual user that may impact on behaviour, as well as
subsequent evaluations of a given interaction. We have
previously [2-5] suggested that individual differences in terms
of underlying personality, gender and demographics may play
an important role. However, these effects may not necessarily
translate into analogous behaviour across different
interactions. Also, measures regarding experience with
computers and robots have been used [3, 6]. The results from
Walters et al. [6] in particular, suggests that the relationship
between individual differences and evaluation of robots and
their behaviours can be quite complex. While these measures
may provide possible answers to explain participant responses
to robots, sometimes they are heavily influenced by the
context of use, as suggested by Mutlu & Forlizzi [7]. They
found that while overall computer use was not relevant, using
computers for playing games did have an impact. As such,
pre-existing biases and attitudes towards robots are difficult to
extrapolate purely from demographics, personality and a
history of technology usage.
One such scale is the Negative Attitudes towards Robots Scale
(NARS). The NARS was developed using a lexical method, in
which its developers created a scale based on free-form
responses from participants regarding anxieties towards robots
(Nomura and Kanda 2003). This later formed the NARS
1Adaptive Systems Research Group, School of Computer Science,
AL10 9AB Email: (d.s.syrdal, k.dautenhahn, k.l.koay, m.l.
walters, }@herts.ac.uk.
Table 1 NARS Items with Subscales
Item No. Questionnaire Item Sub-Scale
1 I would feel uneasy if robots really had emotions. S2
2 Something bad might happen if robots developed into living beings. S2
3 I would feel relaxed talking with robots* S3
4 I would feel uneasy if I was given a job where I had to use robots. S1
5 If robots had emotions I would be able to make friends with them.* S3
6 I feel comforted being with robots that have emotions.* S3
7 The word “robot” means nothing to me. S1
8 I would feel nervous operating a robot in front of other people. S1
9 I would hate the idea that robots or artificial intelligences were making judgements about things. S1
10 I would feel very nervous just standing in front of a robot. S1
11 I feel that if I depend on robots too much, something bad might happen. S2
12 I would feel paranoid talking with a robot. S1
13 I am concerned that robots would be a bad influence on children. S2
14 I feel that int the future society will be dominated by robots. S2
(*inverse item)
Scale, and was used successfully to explain differences in
participants' behaviour in live Human-Robot Interaction (HRI)
studies [8, 9]. Nomura et al. [10] also examined the
relationship between the NARS scale and the Robot Anxiety
Scale (RAS) with participants' behaviour in a HRI trial. The
NARS has also been proposed as a means of gauging changes
in attitudes towards robots over time as a result of prolonged
interactions [11]. These possible applications make the use of
such a scale interesting for both general HRI research across
different robot types, as well as studying how prolonged
relationships with robot companions may influence potential
users' attitudes towards robots.
The items in the NARS are presented in Table 1, along with
the sub-scales they are assigned to. The sub-scales are as
follows:
Sub-scale 1: Negative Attitudes toward Situations
and Interactions with Robots
Sub-scale 2: Negative Attitudes toward Social
Influence of Robots,
Sub-scale 3: Negative Attitudes toward Emotions in
Interaction with Robots
An English translation of the items in the scale has been
created using appropriate methods of translation and
backwards translation to achieve a linguistically valid scale.
However, this translation has primarily been used for the
purpose of evaluating cultural differences in attitudes towards
robots [12]. Findings from these studies have been counter-
intuitive, and suggest that Western participants are more well-
disposed towards robots than Japanese. Studies using other
means of measuring such as Implicit Association Tests and
non-standardised Likert-scale questionnaires have provided
conflicting results suggesting that Western participants find
robots more threatening than their Japanese counterparts [13].
While this issue could be seen as a threat to the overall
validity of using the NARS with a non-Japanese population,
researchers should also consider some of the inherent dangers
of using standardised questionnaires for such cross-cultural
evaluations. In the field of individual differences, there exists
a body of research that suggests that comparing culturally
different samples using only participants' scores on such a
scale is problematic. It appears more appropriate to investigate
how differences in behaviour or related attitudes within the
samples can be explained by such scores [13]. Secondly,
while the NARS translation into English is valid from a
linguistic perspective, cross-cultural differences not related to
language may alter the internal reliability of both the scale as
a whole as well as its sub-scales [15]. As such, reliability
analysis of both the scale and its sub-scales would be useful
when applied to a non-Japanese sample.
This paper describes the use of the NARS in a live HRI study,
in order to examine the internal validity of the scale and its
sub-scales, and to ascertain if it can account for differences in
reactions to a robot's behaviour amongst participants.
Table 2 Robot Behaviour Styles:
Behaviour Robot A Robot B
Path Straight Circuitous Route
with respect to
participant’s pose
Speed Fast Slow when close
to participant
Camera Static and Forward-
facing
Moving and
Tracking
Negotiation of Space “Excuse me”, and
continuing as soon
as possible
“After you”,
continues after
participant has
moved away
Initiative in Bringing
Pen
Does not wait for
participant
Waits for
participant to look
for /ask for pen
Initiative in Delivering
Pen
Bringing basket
with pen to side of
table, close to
participant putting it
down
Waiting in front of
table facing
participant,
waiting for the
participant’s
notice, then
putting basket
down.
3 METHOD
Twenty eight (14 male, 14 female; aged between 18-55)
participants were recruited for the study from students and
staff at the University of Hertfordshire from a variety of
disciplines. These participants took part in two interaction
sessions with a robot. The sessions took part in a seminar
room that was transformed into a simulated ‘living room’ for
the purpose of this study (Fig. 1). In both interaction sessions,
the participants were asked to perform a task which involved
moving in a shared space with the robot as well as requiring a
pen which was brought to the seated participant by the robot.
The robot's behaviour would differ between the two sessions.
These behaviours were labelled Socially Ignorant (A) and
Socially Interactive (B). The main differences between these
behaviours can be found in table 2, both in terms of shared
spaces as well as other interactional differences. The two
behaviour styles were defined by the research team in terms of
how much the robot adjusted its behaviour to the participant,
rather than treating her as any other obstacle in the
environment.
Participants were invited to evaluate the robot's behaviour
after each interaction session, as well as rate the robot on a
personality scale.
The whole experimental session, including the questionnaires
took about 45 minutes per participant.
Fig. 1: A participant interacting with a robot, a Peoplebot
robot (ActivMedia Robotics).
4 RESULTS
Results from these trials not related to the NARS scale can be
found in [16], which discusses the relationship between
participant personality traits and those attributed to the robot.
NARS Analysis
After administering the NARS scale to the participants, a
reliability analysis using Cronbach's Alpha was performed on
the participant's responses, and three items were removed
from the analysis. These items were:
•The word 'robot' means nothing to me (S2)
•I would feel nervous operating a robot in front of others
(S1)
•I feel that in the future, society will be dominated by robots
(S2)
After these items were removed, the revised NARS scale had
a Cronbach's Alpha of .80, supporting the notion of the scale
as measuring a uni-dimensional construct.
However, [15] suggests that there were some cultural
differences in how the scale functioned for a Western
European sample. This would make an exploratory Factor
Analysis using the Principal Components Analysis (PCA)
method appropriate for investigating how attitudes towards
robots would load within this sample.
The results from the PCA using the Varimax rotation method
are in Table 3.
•Future/Social influence - had clear similarities with the
Sub-scale 2 reported by Nomura et al.[9]
•Relational attitudes - which included items from all of the
original Japanese sub-scales.
•Actual interactions and situations - shared characteristics
with Sub-scale 1 in the original Japanese version (as
opposed to the largely hypothetical aspects of the two other
sub-scales highlighted by the double loading on one of the
items).
Table 3: Subscale loadings
Item Subscale
1 2 3
I feel that if I depend on robots too
much, something bad might happen
0.86
I am concerned that robots would be a
bad influence on children
0.65
I would hate the idea that robots or
artificial intelligences were making
judgements about things
0.54
I would feel uneasy if robots really had
emotions
0.81
I feel comforted being with robots that
have emotion*
0.79
I would feel relaxed talking with
robots*
0.75
If robots had emotions I would be able
to make friends with them*
0.64
I would feel paranoid talking with a
robot
0.44
I would feel very nervous just standing
in front of a robot
0.79
I would feel uneasy if I was given a job
where I had to use robots
0.67
Something bad might happen if robots
developed into living beings
0.48 -0.51
*inverse item
While there were similarities between these sub-scales and the
original sub-scales reported by Nomura et al. [9], the
differences between the two versions were considered large
enough to merit the use of these sub-scales in this study.
This use should be considered tentative and was primarily
done in order to understand the relationship between
responses on the NARS and responses to robot behaviour
styles within this sample of participants. It should be
considered a response to the results from the Reliability
Analysis which suggested differences in how the scale
performed when compared to the results from Nomura et al.
[9].
The revised NARS, as well as it sub-scales, was then used in a
series of correlations in order to examine how they impacted
evaluations of the robot's behaviour in each condition across
the different types of interactions that took place in the
experiment.
Participant Evaluation of Robot Behaviour Style
The evaluations the participants were asked to make were
reported ratings on 5-point Likert scales when:
•Comfort for approaching or being approached by
the robot.
•Comfort when physically close to the robot.
•Comfort moving in the same room as the robot.
•Comfort interacting with the robot whilst seated at
the table.
•Reported overall enjoyment of the interaction.
A series of paired t-tests found that overall there were no
significant differences between the two robot behaviour styles
in how they were viewed by the sample.
Also, participants were invited to rate the robot according to
12 different traits suggested by Eysenck [17], anxiety, tension,
shyness, emotional vulnerability, sociability, general activity
level, assertiveness, excitement-seeking, dominance), and
aggressiveness, impulsiveness, creativity. Note that some
traits were removed from those originally suggested as they
were considered unsuitable for describing robot behaviour. In
addition, autonomy, controllability, predictability and
considerateness were additional traits added. This was based
on research that suggested that mental models of robots, while
incorporating aspects of how we view humans, also include
aspects that are defined by the robots' mechanical nature [18].
The Impact of the NARS
The focus of the subsequent analysis was to investigate if the
NARS could be used to differentiate participant evaluation of
the robot, both in terms of how participants evaluated the
robot’s behaviours and how participants attributed personality
to the robot.
Evaluating Robot Behaviour Styles
In order to investigate the relationship between NARS scores
and post-experimental evaluations of robot behaviour, a series
of correlations were run between the NARS and its sub-scales
with participant evaluations of the robot's behaviour styles.
The most important trait for evaluating robot behaviour styles
within the interaction was the third tentative sub-scale: Actual
Interactions. In terms of Robot A (Socially Ignorant) it
impacted on both Comfort when being physically close to the
robot (r(28)=.448, p=.02) as well as comfort when
approaching or being approached by the robot (r(28)=.464,
p=.01). For Robot B (Socially Interactive), the Actual
Interactions sub-scale also impacted Comfort when being
physically close to the robot (r(28)=.442, p=.02) as well as
comfort when approaching or being approached by the robot
(r(28)=.466, p=.01). It also impacted Comfort when moving
in the same room as the robot (r(28)=.462, p=.01) and the
Overall enjoyment of the interaction (r(28)=.393, p=.04)
The Overall NARS scores had no significant relationships
with evaluations of Robot A's behaviour. There were however
significant relationships between the NARS and Comfort
when interacting with the robot while seated at the table
(r=(28)=.425, p=..02) and Overall enjoyment of the
interaction (r(28)=.383,p=.04).
Note that for all these significant correlations a higher score
on the NARS or its sub-scale, suggests a less favourable
evaluation of the interaction.
Table 4 Correlation between trait attributions and
NARS scores for Robot A (Socially Ignorant)
Trait Overall
Nars
Social/Future
Implications
Emotional
Attitudes
Actual
Interac.
Anxiety r=-.476,
p=.010
r=-.436,
p=.02
Tension
Shyness r=-.441,
p=.02
Emotional
Vulnerability
Sociability
General
Activity Level
Assertiveness
Excitement
Seeking
r=-.430,
p=.022
r=-.394,
p=.04
Dominance
Aggressiveness
Impulsiveness
Creativity r=-.377,
p=.05
Autonomy r=.389,
p=.04
Controllability
Predictability
Considerateness
These results suggest that the NARS differentiated between
participant responses to the two different robot behaviour
styles. Participants with a higher score on the NARS scale
found Robot B's behaviour less comfortable across a wider
range of interaction sequences.
Attributing Traits to the Robot
In order to investigate the relationship between participants'
NARS scores and how traits were attributed to the robot, a
series of tests were run, correlating the NARS and its sub-
scales with the different traits. The results from these
correlations are shown in tables 4 and 5.
Table 5 Correlations between trait attributions and
NARS scores for Robot B (Socially Interactive)
Trait Overall
NARS
Social/Future
Implications
Emotional
Attitudes
Actual
Interac.
Anxiety r= -.491,
p=.01
Tension
Shyness r= -.434,
p=.02
Emotional
Vulnerability
Sociability
General
Activity Level
Assertiveness
Excitement
Seeking
r= -.446,
p=.02
Dominance
Aggressiveness
Impulsiveness
Creativity
Autonomy r=.407,
p=.03
Controllability r= -.402,
p=.03
r= -395,
p=.04
Predictability r= -.384,
p=.04
Considerateness r= -.457,
p=.01
Summarising these results presented in table 4 and 5, the
NARS and it sub-scales correlate significantly with the traits
attributed to both Robot A and Robot B. However, the overall
picture emerging from these correlations is less clear than that
for the evaluation of the interactions.
In general, it seems that Negative Attitudes Towards Robots
Scale and its sub-scales are associated for both robots in terms
of seeing the robot as the less anxious, less shy and less
excitement seeking. What is more interesting for this
particular investigation are the correlations between NARS
scores and the robot specific traits. For these traits, the NARS
sub-scale of Actual Interactions serves to differentiate
between how participants attribute traits to the two different
robots. Participants having higher scores on the Actual
Interaction sub-scale tend to rate Robot B as more
autonomous, less predictable and less considerate. Also higher
Overall NARS and Emotional Attitudes were associated
with seeing Robot B as less Controllable. This relationship is
not seen for Robot A.
5 DISCUSSION
The results suggest that using the English translation of the
NARS is an appropriate method of investigation prior
attitudes towards robots that may impact participant
evaluations of robot behaviour styles. After assessing the
NARS using the Cronbach's Alpha as a measure of internal
consistency, and removing three items, it had a high degree of
internal consistency in a sample recruited at a British
University. As suggested by Auer et al.[15], when using
standardised measures across cultures, certain artefacts that
originate in particulars of a given culture may impact both
internal consistency as well as the validity of such a measure
when applied to other cultures. It is possible that the three
items removed may originate in such artefacts, specific to
Japanese culture. This can also may serve as a possible
explanation as to the differences in how the PCA performed in
the responses from this sample described the sub-scales and
those suggested by Nomura et al. [9]. We propose that such
artefacts, rather than actual differences in cultural attitudes
towards robots, may be the cause for the divergence of results
from this study with those performed on Japanese samples.
MacDorman et al. [13] suggests that these differences are not
as pronounced as they are often believed to be.
More importantly however, is the utility of the NARS to
explain, and possibly predict, other aspects of how people
view and evaluate robot behaviour styles. In terms of
differentiating between the two types of robot behaviours in
this study, use of the NARS and its sub-scales differentiated
between robot behaviour styles, which over the sample as a
whole were not evaluated differently. Of particular interest is
that higher scores on the NARS and the Actual Interactions
sub-scale were associated with a more negative evaluation of
the behaviour of Robot B, which was actually intended to act
in a more socially appropriate manner.
There may be several reasons for this. One reason may be that
this robot was seen as more socially sophisticated and that
participants scoring high on the NARS as well as the sub-
scale may be more wary of robots displaying a higher degree
of sophistication.
Another explanation can be found in relating the trait
attributions to these evaluations. It appears that participants
with higher scores in the Actual Interactions sub-scale were
more likely to rate Robot B as more autonomous, and less
predictable. This may have been caused by the robot's
behaviour. The behaviours by the socially interactive robot,
could by some participants be considered more intrusive.
Some of the behaviours of the robot, such as the movement of
the camera, responding to the participants presence in terms of
movement, waiting for participants to respond before leaving
the pen, could have drawn attention to the robot as an
autonomous agent within the scenario to a larger extent than
Robot B's behaviour. This effect may be analogous as to that
reported by Rickenberg & Reeves [19] when examining the
impact of the behaviour of an animated character, in which
participant evaluation of two differend behavioural styles
varied dramatically depending on the participants' locus of
control.
6 CONCLUSIONS
These results suggest that the Negative Attitudes towards
Robots Scale may be susceptible to cultural differences. This
may necessitate that research using this scale on a population
outside of Japan may need to re-validate the scale and its sub-
scales. However, our research also validates the value of the
NARS as a means of explaining variance within a given
sample in terms of evaluations of robot behaviour styles in a
live HRI trial. Both the NARS and its sub-scales had an
impact, not only on how participants evaluated their
interactions with the robot, but also had some power to
explain how participants differentiated between the two
different robot behaviour styles. Negative Attitudes towards
robots tended be associated with more negative evaluations of
the behaviour of robot B (Socially Interactive behaviour
style).
Of particular interest here, is that the sub-scale
that had the strongest relationship with
evaluations, Actual Interactions, might be considered to
be related to the notion of robot anxiety as described in [11] as
these items do refer to anxieties in actual interactions.
We would however, like to qualify the results from the PCA,
as the number of participants was quite low due to resource
constraints when running a live HRI experiments (the current
study already took 2 months with daily HRI trials). However,
this does not invalidate the meaningful relationships between
at least one of the sub-scales and participant evaluations of the
robots behaviour that were found.
7 ACKNOWLEDGEMENTS
We would like to thank our colleague Christina Kaouri for her
help in setting up and running the experiments. The work
described in this paper was conducted within the EU
Integrated Projects, COGNIRON (“The Cognitive Robot
Companion”) and LIREC (LIving with Robots and integrated
Companions). Funded by the European Commission under
FP6-IST and FP7-ICT under contracts FP6-002020 and FP7-
215554.
8 REFERENCES
[1] LIREC, http://www.lirec.org, 2008.
[2] K. Dautenhahn, "Methodologies and Themes of
Human-Robot Interaction - A Growing Research Field,"
International Journal of Advanced Robotics Systems, vol.
4, pp. 103-108, 2007.
[3] D. S. Syrdal, K. Dautenhahn, S. Woods, M. Walters,
and K. L. Koay, "Looking Good? Appearance Preferences
and Robot Personality Inferences at Zero Acquaintance,"
Multidisciplinary Collaboration for Socially Assistive
Robotics: Papers from the AAAI Spring Symposium
-Technical Report SS-07-07, pp. 86-92, 2007.
[4] D. S. Syrdal, K. Dautenhahn, S. Woods, M. L.
Walters, and K. L. Koay, "‘Doing the right thing wrong’ –
Personality and tolerance to uncomfortable robot
approaches," Proc. 15th IEEE International Workshop on
Robot and Human Interactive Communication(RO-MAN
2006), pp. 183-188, 2006.
[5] D. S. Syrdal, K.-L. Koay, M. L. Walters, and K.
Dautenhahn, "A personalised robot companion? The role
of individual differences on spatial preferences in HRI
scenarios.," IEEE International Symposium on Robot and
Human Interactive Communication(Ro-man), Jeju Island,
Korea, 2007.
[6] M. Walters, D. S. Syrdal, K. Dautenhahn, R. T.
Boekhorst, K. L. Koay, and S. Woods, "Avoiding the
Uncanny Valley – Robot Appearance, Personality and
Consistency of Behavior in an Attention-Seeking Home
Scenario for a Robot Companion," Autonomous Robots,
vol. 24, pp. 159-178, 2008.
[7] B. Mutlu, J. Forlizzi, and J. Hodgins, "A Storytelling
Robot: Modeling and Evaluation of Human-like Gaze
Behavior," presented at Humanoid Robots, 2006 6th
IEEE-RAS International Conference on, 2006.
[8] T. Nomura, T. Kanda, T. Suzuki, and K. Kato,
"Psychology in Human-Robot Communication: An
Attempt through Investigation of Negative Attitudes and
Anxiety toward Robots," Proc. 2004 IEEE International
Workshop on Robot and Human Interactive Interactive
Communication, Kurashiki, Okayama Japan, September
20-22, 2004, pp. 35-40, 2004.
[9] T. Nomura, T. Kanda, and Y. Suzuki, "Experimental
Investigation into Influence of Negative Attitudes toward
Robots on Human-Robot Interaction," AI&Society, vol.
20, 2006.
[10] T. Nomura, T. Suzuki, T. Kanda, and K. Kato,
"Altered attitudes of people toward robots: Investigation
through the Negative Attitudes toward Robots Scale,"
Proc. AAAI-06 Workshop on Human Implications of
Human-Robot Interaction, pp. 29-35, 2006.
[11] T. Nomura, T. Kanda, T. Suzuki, and K. Kato,
"Prediction of Human Behavior in Human--Robot
Interaction Using Psychological Scales for Anxiety and
Negative Attitudes Toward Robots," Robotics, IEEE
Transactions on, vol. 24, pp. 442, 2008.
[12] C. Bartneck, T. Nomura, T. Kanda, T. Suzuki, and K.
Kennsuke, "A cross-cultural study on attitudes towards
robots.," Proceedings of the HCI International 2005, Las
Vegas., 2005.
[13] K. F. MacDorman, S. K. Vasudevan, and C.-C. Ho,
"Does Japan really have robot mania? Comparing
attitudes by implicit and explicit measures," AI & Society,
vol. 23, pp. 485-510, 2009.
[14] G. Matthews, I. J. Deary, and M. C. Whiteman,
Personality Traits. Cambridge, UK: Cambridge
University Press, 2003.
[15] S. Auer, H. Hampel, H.-J. Möller, and B. Reisberg,
"Translations of Measurements and Scales: Opportunities
and Diversities," International Psychogeriatrics, vol. 12,
pp. 391-394, 2000.
[16] S. Woods, K. Dautenhahn, C. Kaouri, R. te
Boekhorst, K. L. Koay, and M. Walters, "Are Robots Like
People? - The Role of Subject and Robot Personality
Traits in Human-Robot Interaction Trials," Interaction
Studies., vol. 8, pp. 281-305, 2007.
[17] Eysenck, H. J., & Eysenck, M. W. (1985).
Personality and individual differences: A natural science
approach. New York, USA: Plenum.
[18] S. Kiesler and J. Goetz, "Mental Models and
Cooperation with Robotic Assistants," Proceedings of the
ACM CHI’02 Conference on Human Factors in
Computing Systems, pp. 576–577., 2002
[19] R. Rickenberg and B. Reeves, "The Effects of
Animated Characters on Anxiety, Task Performance, and
Evaluations of User Interfaces.," Proceedings of the ACM
CHI 2000 Human Factors in Computing Systems
Conference., pp. 49-56., 2000.
