How to approach humans?: strategies for social robots to initiate interaction.

Full text

Page 1

How to Approach Humans?
–Strategies for Social Robots to Initiate Interaction

Satoru Satake, Takayuki Kanda, Dylan F. Glas, Michita Imai, Hiroshi Ishiguro, Norihiro Hagita
ATR Intelligent Robotics and Communication Laboratories
2-2-2 Hikaridai, Seika-cho, Souraku-gun, Kyoto, Japan
{satoru, kanda, dylan, michita, ishiguro, hagita}@atr.jp
ABSTRACT
This paper proposes a model of approach behavior with which a
robot can initiate conversation with people who are walking. We
developed the model by learning from the failures in a simplistic
approach behavior used in a real shopping mall. Sometimes
people were unaware of the robot’s presence, even when it spoke
to them. Sometimes, people were not sure whether the robot was
really trying to start a conversation, and they did not start talking
with it even though they displayed interest. To prevent such
failures, our model includes the following functions: predicting
the walking behavior of people, choosing a target person,
planning its approaching path, and nonverbally indicating its
intention to initiate a conversation. The approach model was
implemented and used in a real shopping mall. The field trial
demonstrated that our model significantly improves the robot’s
performance in initiating conversations.
Categories and Subject Descriptors
H.5.2 [Information Interfaces and Presentation]: User
Interfaces-Interaction styles
General Terms
Algorithms, Performance, Design, Human Factors
Keywords
Approaching behavior, Position-based Interaction, Anticipation
1. INTRODUCTION
Social robots have started to move from laboratories to real
environments [11,17,27,28,32,34], where, a robot interacts with
ordinary people who spontaneously interact with it. For such a
robot, initiating interaction is one of its most fundamental
capabilities. In previous studies, many robots were equipped with
the capability to encourage people to initiate interaction
[14,15,16,22,25]. These robots wait for people to approach them,
which is one strategy for robots to initiate interaction.
Alternatively, a “mobile” robot can approach people (Fig. 1) to
initiate interaction. This way of providing services is more
proactive than waiting, since it enables robots to help people who
have potential needs. For instance, imagine a senior citizen who
has gotten lost in a mall. If a robot were placed in the mall for
providing route direction, it could wait until the senior citizen
approaches it and asks for help; but he might not know what the
robot can do, or he might hesitate to ask for help. Instead, it is
more appropriate that the robot approaches and offers help.
Fig. 1 What’s wrong? “Unaware” failure in approach
The robot’s capability to approach people enables a number of
applications. We believe that one promising application is an
invitation service: a robot offers shopping information and invites
people to visit shops, while giving people the opportunity to
interact with the robot since robots are very novel to them.
2. RELATED WORKS
Hall, who classified human interactions based on a concept of
distance, coined the following terms. “Public distance” refers to
situations in which people give a speech, and “social distance”
characterizes situations in which people talk to each other for the
first time [10]. Our approach is related to public and social
distances. The robot needs to find a person with whom to talk, to
start approaching that person at a public distance, and to initiate
the conversation at a social distance.
2.1 Finding a Person for Interaction
To find a person for interaction, first, robots need to locate people.
In robotics, many traditional studies are related to obstacle
avoidance and path planning, and researchers have recently
started to study methods for finding and tracking people. Vision
as well as distance sensors onboard robots have been used
[2,3,4,6,20]. Moreover, researchers have started to use sensors
embedded in environments [5,23,29,35] that enable a robot to
recognize people from a distance.
After finding people, the robot needs to identify a person with
whom to interact. There are previous studies about human
behaviors related to this. For example, Yamazaki et al. analyzed
how elderly people and caregivers start conversations. They found


(b) The robot started to speak

(c) Turned away from robot (d) Left without glancing at it

Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies
are not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy
otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee.
HRI’09, March 11–13, 2009, La Jolla, California, USA.
Copyright 2009 ACM 978-1-60558-404-1/09/03...$5.00.
(a) Robot approached a man looking at a map
109

Page 2

that to identify elderly people who require help, a caregiver
displays his/her availability non-verbally with body orientation,
head direction, and gaze [18]. Fogarty et al. analyzed human
interruptibility in an office environment and demonstrated that
even simple silence detectors could significantly estimate
interruptibility [13].
In HRI, Michalowski et al. classified the space around a robot to
distinguish such human levels of engagement as interacting and
looking [22]. Bergström et al. classified people’s motion toward a
robot and categorized people into four categories: interested,
indecisive, hesitating, and not interested [25]. Tasaki et al.
developed a robot that chooses a target person based on distance
[33]. Finke et al. developed a robot that chooses a target person
based on motion [20]. Although these studies focused on people’s
behaviors directed to robots, such information is not available for
a robot that approaches people from a distant place. Instead, the
robot needs to observe people’s behavior, such as their way of
walking, to estimate the possibility to talk.
2.2 Interaction at Public Distance
When people are at a public distance, it is too far for them to talk;
but they can recognize each other’s presence. At such a distance,
interaction is mainly achieved by changing body position and
orientation. Sisbot et al. developed a path-planning algorithm that
considers people’s positions and orientation to avoid disturbances
[6,7]. Pacchierotti et al. studied passing behavior and developed a
robot that waits to make room for a passing person [9]. Gockley et
al. found the merits of a direction-following strategy for a robot
when following a person [26].
These robots only use people’s current position; however, since
human-robot interaction is dynamic and quick, prediction and
anticipation are crucial. Hoffman and Breazeal demonstrated the
importance of anticipation in a collaborative work context [12]. In
contrast, there are few studies about the anticipation of people’s
positions. Bennewitz et al. utilized such a prediction of position
[19], but it was only used for helping a robot avoid people, not for
allowing interaction with them. In a previous study, we
anticipated people’s positions for letting a robot approach them
and demonstrated the importance of anticipating positions [31].
But it lacks a path-planning process, which is important for
notifying the target person of the robot’s presence.
2.3 Initiating Conversation at Social Distances
After entering a social distance, a robot initiates conversation with
its target. Humans start conversations with greetings. Goffman
suggested that social rules exist for accepting/refusing approaches,
including eye-contact, which is a ritual that mutually confirms the
start of a conversation [8]. Kendon suggested that friends
exchange greetings twice, first nonverbally at a far distance and
again at a close distance by smiling [1].
Several previous HRI studies have addressed the greeting process.
Dautenhahn et al. studied the comfortable direction of an
approach [15] as well as the distance to talk [21]. Greeting
behavior initiates human-robot conversation [27,30]. Yamamoto
and Watanabe developed a robot that performs a natural greeting
behavior by adjusting the timing of its gestures and utterances[24].
These studies assume that the target person intends to talk with
the robot; however, in reality people are often indecisive about
whether to talk when they see a robot for the first time. A few
studies have been conducted on the first-time-meeting situation
and making robots nonverbally display a welcoming attitude
[22,25]; but these passive robots only waited for a person to
engage in conversation. Although such a passive attitude is fine
for some situations, many situations require a robot to engage in
an active approach. Our study aims to allow a robot to actively
approach a person to initiate conversation.
“An Approach from a robot” is not an easy problem since the
robot’s approach needs to be acknowledged nonverbally in
advance; otherwise, the approached person might not recognize
that the robot is approaching him/her or would be surprised by the
robot’s impolite interruption. Humans do this well with eye gaze
[1,8], but in a real environment it is too difficult for a robot to
recognize human gazes. Instead, we use the body orientation of
the target and the robot for non-verbal interaction.
3. Environment, Hardware, and Task
This study, which focuses on the spontaneous initiation of human-
robot interaction, requires a realistic scenario in a real field. We
chose a shopping mall for the environment and “advertising about
shops” as the robot’s task. The robot approaches a person to
recommend a shop in the mall to increase interest in visiting it.
3.1 Environment: Shopping Mall
The robot was placed in a shopping mall located between a
popular amusement park, Universal Studios Japan, and a train
station. The primary visitors of the mall are groups of young
people, couples, and families with children. The robot moved
within a 20 m section of a corridor (Fig. 1). Clothing and
accessories shops are on one side and an open balcony is on the
other.
3.2 Hardware
3.2.1 Robot
We used Robovie, a communication robot, who is characterized
by its human-like physical expressions. It is 120 cm high and 40
cm in diameter and equipped with basic computation resources as
well as WiFi communication. Its locomotion platform is Pioneer3
DX. We set it to move at a velocity of 300 mm/sec (approx. 1.0
km/h) forward and 45 degree/sec for turns. The platform can
navigate the robot faster (up to 1600 mm/sec) than these
parameters, but we chose a lower velocity for safety.
3.2.2 Sensors
To approach people, the robot needs to robustly recognize their
positions and its own position, even in distant places. We decided
to use sensors embedded in the environment for tracking human
and robot positions. Six SICK LMS-200 laser range finders were
positioned around the area’s perimeter. Laser range finders were
set to a maximum detection range of 80 m with a nominal
precision of 1 cm, and each scanned an angular area of 180° at a
resolution of 0.5°, providing readings every 26 ms.
For detection and tracking of people and robots, a technique
based on the algorithm described by Glas [5] was used. In this
algorithm, particle filters are used for estimating people's
positions and velocities, and a contour-analysis technique
estimates the direction in which a person is facing. This
orientation angle helps determine whether the robot should
approach a person. This system covers 20 m x 5 m area and
concurrently detects over 20 people’s locations.
3.3 Task
The robot’s task was “advertising shops.” It was placed in a
corridor of the mall. The robot was designed to approach visitors
110

Page 3

and to recommend one of the 24 shops in the mall by providing
such shop information as, “It’s really hot today, how about an
iced coffee at Seattle’s Best Coffee?” or “It’s really hot today,
how about an ice cream shop?” Then the robot pointed at the shop.
When the robot was neither approaching nor talking, it roamed on
a pre-defined route.
Visitors freely interacted with the robot and could quit anytime.
For safety, our staff monitored the robot from a distant place; thus,
from the visitors view, it seemed as if the robot moved around and
approached them without assistance from human experimenters.
4. Modeling of Approach Behavior
We define the term "approach behavior" as a sequence of the
following activities: (1) selecting a target, (2) approaching the
target, and (3) initiating conversation from a close distance.
In the beginning, we implemented a “simple” approach behavior,
where (1) the robot selects the closest person, (2) takes the
shortest path to that person, and (3) greets him/her when it arrives
within the social distance (3 m); however, such a simplistic
approach behavior was too simple because it often failed to
initiate conversation. In this section, we describe the lessons
learned from failures and introduce our model for more efficient
and polite “approach behavior.”
4.1 Lessons Learned
Many people ignored the robot. These failures, which reflected
many problems in the “simple” approach behavior, were analyzed
and categorized into four categories: unreachable, unaware,
unsure, and rejected. Table 1 summarizes the failure categories,
which we introduce in this subsection and discuss how the robot
can avoid them. We classified failures by watching videos and
position data.
Table 1 Classification of failures
Category What happened
unreachable - The robot did not get close to target person.
unaware
- The person did not look at robot.
- The person did not listen to it.
unsure
- The person recognized its presence and reacted
(e.g., checked its reactions); but the robot did not
respond correctly in time.
rejected
- The person recognized its presence and its
greeting behavior, but did not start a conversation.

Unreachable
One typical failure is a case where the robot failed to get close
enough to a target person. This failure happened because (a) the
robot was slower than the target person, and/or (b) the robot chose
a person who was leaving.
Unaware
When unaware of the robot, a person does not recognize its action
as initiating conversation, even when the robot speaks to him/her.
Figure 1 shows one such failure. In this case, a man was looking
at a map on a wall when the robot spoke to him (Fig. 1(b)), but he
wasn’t listening (Fig. 1(c)) and left without even glancing at the
robot (Fig. 1(d)). He probably did not hear the robot because the
mall was quite noisy; perhaps he heard but he did not recognize
that it was directed at him; he might have recognized it but simply
ignored it.
Figure 2 shows another example where two women were walking
together (Fig. 2(a)). The robot started approaching one of them
from the front and seemed to be within her sight (Fig. 2(b)). When
the robot reached the distance to talk, it approached her right side
(Fig. 2(c)). Unfortunately, since she wasn’t looking at the robot
but at a shop, she ignored the robot as if nothing happened and
walked on.
To avoid this type of failure, the robot needs to improve its
notifying behavior.

(a) Robot approached the woman (b) Robot seemingly in her sight but she
paid no attention

(c) She didn’t see the robot while it
approached her right side
Fig. 2 “Unaware” failure: a person is walking and talking to
another person
(d) She left
Unsure
We labeled another type of failure as “unsure.” Sometimes,
although people were aware of the robot, it failed to initiate
conversation. They observed the robot’s behavior and recognized
its utterances. However, they did not stop since they seemed
unsure whether the robot intended to talk to them. Some people
even “tested” the robot’s reaction after its greeting, but since the
robot was not prepared to react to such “testing” behaviors, it
failed to provide an appropriate reaction. Thus, the robot failed to
initiate conversation.
Figure 3 shows one such failure. A woman and a man entered the
environment (Fig. 3(a)). The robot approached and greeted her.
She stopped walking and performed a kind of “test” by reaching
her hand to the robot's face (cp. Fig. 3(c)). The robot, however,
did not respond, so the woman left a few seconds later.
To avoid this type of failure, the robot must establish mutual
understanding with the target with whom they are going to engage
in a conversation. In addition, it should quickly establish
contingency with the person (e.g., facing the person, re-orienting
its body to the person, etc.) when it is going to initiate a
conversation.
Rejected
Some people are not interested in conversation with the robot,
presumably because they are too busy. These people avoided the
robot at a glance and rejected to talk to it, although they were
aware of it and knew that the robot was trying to talk to them. We
called this failure “rejected.” These people should simply not be
approached.

A
A Ap p pp p prrro o oa a accch h h ttta a arrrg g geeettt
111

Page 4

(a) Robot approached a person (b) She stopped when robot started to
speak

(c) She observed robot’s reaction (d) She left when robot did not
immediately react
Fig. 3 “Unsure” failure: woman unsure whether robot
intended to speak to her
4.2 Modeling
By summarizing the lessons learned from the failures in the
“simple” approach behavior, we developed a model for a more
efficient and polite “approach behavior,” as shown in Table 2. It
consists of three phases.
Table 2 Model of approach behavior
Finding an interaction target
The first phase is “finding an interaction target.” The robot needs
to predict how people walk and estimate who can be reached with
its locomotion capability. It also needs to anticipate those who
might be willing to interact with it; this requirement is really
difficult, but at least it can avoid choosing busy people who are
probably unwilling to talk with it.
Interaction at public distance
The second phase is “interaction at a public distance,” where the
robot notifies its presence to the target at a public distance by
approaching from his/her front. The difficulty here is that the
robot must predict the target’s walking course to position itself
within his/her sight before starting the conversation.
Initiating conversation at a social distance
The last phase is “initiating conversation at a social distance.”
One might argue that this can be done simply by such greetings as
“hello;” however, greeting strangers is not easy. People are
sometimes unaware of the robot’s utterance. People sometimes do
not recognize that its greeting is directed to them. Instead, we
focused on using nonverbal behavior to indicate the robot’s
intention to initiate a conversation. When the target is about to
change her course, the robot faces her so that she can clearly
recognize that the robot is trying to interact with her. We assume
that, if she stops, she is accepting interaction with the robot. After
receiving such an acknowledgement, the robot starts a
conversation.
5. Implementation
We implemented approach behavior with 4 functions: (1) tracking
people, as described in section 3.2.2, (2) anticipating people’s
behavior, (3) planning path to approach, and (4) initializing
conversation. These functions were executed in this order.
5.1 Anticipating people’s behavior
In a shopping mall, most of the people have rather simple
intentions, such as passing to a destination or walking for window
shopping. Consequently people’s walking behavior (trajectories
and speed) are similar to each other. A busy passenger tries to
follow the shortest path with high velocity while a window
shopper prefers to pass a side of a shop with slow velocity. In our
previous study [31], we anticipate people’s future behavior by
gathering and clustering a tens thousand of people’s trajectories.
To anticipate people’s future behavior, we applied a SVM
(support vector machine) to classify 2-seconds of each trajectory
into four behavior classes: fast-walking, idle-walking, wandering,
and stopping. This classification is based on features of shapes of
trajectory and velocity.

(a) Fast-walk (b) Idle-walk (c)Wandering (d) Stop
Fig. 4 Classification of trajectories to behaviors
A clustering algorithm with a DP matching method was applied to
a whole length of gathered trajectories. As the result of the
clustering, we got 300 clusters, some clusters represented “busy
person” patterns (Fig.5 (a)), and some cluster represented
“window shopper” patterns (Fig. 5 (b)). In Fig. 5, blue color
represents a location where trajectories in a cluster were usulally
classfied as fast-walking behavior. Green represents a location
where trajectories were ussulally classifed as idle-walk behavior
Gathered trajectories in a cluster enable to anticipate people’s
future location and behavior. From the trajectories, we derived
likelihood values of each behavior at each grid in a cluster at any
time after a person whose trajectory belongs to the cluster appears.
We manage these likelihood values as an “expectation map”.

(a) A “busy person” pattern (b) A “window shopper” pattern
Fig. 5 Extraction of typical behaviors
A predicted position of a person is the expectation value of the
position. If a trajectory of a person is observed from tracking
people function, we search the nearest 5 clusters to the trajectory
and merge likelihood of each exception map to anticipate the
person. Fig. 6 shows expectation maps for various time
increments of a person. The red circles represent the positions of
the person walking through the space. The expected behaviors can
be seen tracing a path through the corridor, heading toward the
upper right. In fact, this course was correctly predicted, and the
person followed that general path.
Phase Robot’s behavior
Failures to be
moderated
Finding an
interaction target
Interaction at
public distance
Initiating
conversation at
social distance
Selecting a likely
interaction target
Announcing its presence
and intention to talk
Unreachable/
Rejected
Unaware
Nonverbally showing
intention to interact
Unsure
112

Page 5

t1 + 5 t1 + 10 t1 + 15
Fig. 6 Example of anticipation

Fig. 7 Approach from frontal direction
5.2 Planning path to approach
Fig. 7 shows overview of our path planning strategy. The robot
predicts the trajectory of its target person (the dashed line from
the person in Fig. 7) and finds the turning point where the robot
has enough time to correct its orientation in order to approach
from frontal direction after the robot arrives. In the figure, three
possible approach paths are described. A robot plans to arrive to
a point named turning point, and orient its body direction to
approach to the person from the frontal direction.
The system searches for the most promising approach path based
on the algorithm shown in the Table 3. In the table, i represents
the person’s id, t0 represents the current time, and t represents the
future time. In Fig. 7, t1, t2, and t3 also represent future time
(t0<t1< t2 < t3). Pos(i, t) represents the expected position of the
person i at time t, provided by an expectation map described in
section 5.1. The estimation of the success of the approach is given
by Papproach(i,t), which is computed from 4 probabilities: Pack(i,t),
Pfront(i,t), Pgaze(i,t), and Uncertanty(t) (Fig. 8). Each of these
probabilities is estimated as follows:
Table 3 The algorithm to choose the target person and point

Fig. 8 Probability functions for the person at Fig. 7
Pack(i,t)
It represents a probability whether target person might be willing
to interact. It is difficult to accurately estimate this; instead, as we
discussed in 4.2, our strategy is to choose a person whose future
behavior class is idle-walking rather than busy-walking. Thus,

likelihood value of “idle-walking” of Pos(i,t) at the future time t is
used as Pack(i,t).
Pfront(i,t)
It represents a probability that the robot approach the target
person from frontal direction. To approach from frontal direction,
the robot needs to appear in advance to the place where the person
will come. We calculated this based on the largeness of margin
time to change robot’s orientation. The following algorithm is
applied. To notify the robot’s presence at public distance, we must
choose an approach plan that has high Pfront(I,t).
1.
Calculate tarrive , the time the robot arrives at the Pos(i,t)
2.
Calculate tmargin, that is, t - tarrive. If tmargin< 0, Pfront(i,t)=0
3.
Pfront(i,t)= tmargin / threshold. If tmargin >threshold, Pfront(i,t)=1
In Fig. 7 tmargin at t2 is less than tmargin at t3 , therefore Pfront(i,t2) is
less than Pfront(i,t3) (Fig. 8). In Fig. 7, tmargin is less than 0 and
Pfront(i,t1) equals 0 (Fig. 8).
Pgaze(i,t)
It represents a probability that the robot orient its head to the
target person during its approach so that the target person could
find the robot looking at him/her. To notify the robot’s intention
at public distance, this gazing behavior is needed. Pgaze(i,t) is
calculated based on the angular difference between the robot’s
head orientation and the direction of locomotion. The robot has
mechanical constraint on the head direction, thus it can only look
at the direction within a certain range. Moreover, it is uncertainty
in the prediction of the target motion. If the angular difference is
small, it is highly possible for the robot to orient its head to the
target person continuously.
Uncertanty(t)
There is a large uncertainty in the prediction of target person’s
trajectory in the future, although the robot plans approach path
based on the prediction. If the system tries to plan approach path
in the far future, the likelihood of error of the prediction is larger.
Thus, Uncertanty(t) is bigger when t is larger.
5.3 Initializing conversation
Initializing conversation has two purposes: (1) nonverbally
showing the robot’s intention to interact, and (2) recognizing an
acknowledgment to start interaction from its target. It decides the
1 For each person i, for each t (t0 < t < t0+20), calculate
Papproach(i,t) = Pack(i,t) ·Pfront(i,t) ·Pgaze(i,t) ·Uncertatinty(t)
2 Find i’ and t’ which satisfy
Papproach(i’,t’) = max( Papproach(i, t) )
3 Approach path is set to Pos(i’, t’)
robot’s behavior based on human behavior within the social
distance.
Table 4 shows the strategy of this function. It classifies human
behavior into 4 categories: approaching, stopping, avoiding, and
leaving. When a person changes his/her course, his/her behavior
is categorized as “avoiding” and the robot faces its body
orientation to him/her to show its intention to talk to him/her.
When a person stops, the robot starts a conversation. When a
person leaves the robot, the robot gives up starting a conversation.
The person
The robot
tmargin
tarrive
Pos(i,t1)
Turning point
Pos(i,t3)



t0
Pos(i,t2)
t
Probability
Pgaze(i,t) Pfront(i,t)
Uncertanty(t)
Pack(i,t)
t1 t2 t3 t0
person 1 here
113

Page 6

Table 4 The strategy of initializing conversation
Human
behavior
Robot behavior
approaching Approach
stopping Start conversation
Show intention to talk
strongly
Give up having a
conversation
avoiding
leaving

We transform human trajectory to robot coordinate system and
apply SVM. SVM classified 1-second of each transformed
person’s trajectory (Fig. 9). This classification used features of
shapes of trajectory, velocity, and direction. We used 45
trajectories for learning and its recognition rate was 88.9 % tested
with leave-one-one method.

(a) approaching (b) avoiding (c) leaving
Fig. 9 Classification of trajectories at initializing conversation
phase to behaviors
6. Field Trial
We conducted a field trial at a shopping mall1 to evaluate the
effectiveness of the “proposed” approach behavior. The robot’s
task was to approach visitors to provide shopping information.
The details of the environment and task are described in Section 3.
6.1 Procedure
We compared the proposed method with the “simple” approach
behavior to evaluate the effectiveness of the “proposed” method.
The “simple” approach behavior is described in Section 4. In the
“proposed” approach behavior, the robot follows the model
reported in Section 4.2 and uses all the implemented techniques
reported in Section 5.
For comparisons, we ran the trials for several sessions to eliminate
environmental effects, such as the time of the trial. Each session
lasted about 30 minutes. The two conditions, “simple” and
“proposed,” were assigned into sessions whose order was
counterbalanced. In total, we ran the experiment for two hours for
each condition, and about the same number of approach behaviors
were conducted in both conditions.
6.2 Improvement of Success Rate
Fig. 10 shows the comparison results. The approach behavior was
defined as successful when the robot’s approach target stopped
and listened to the robot’s utterance at least by the end of its
greeting. In this section, we defined the term “trials” to mean
actual approaches toward people, and not simply the number of
people passing through the area.

1In this study, we obtained approval from the shopping mall
administrators for this recording under the condition that the
information collected would be carefully managed and only
used for research purposes. The experimental protocol was
reviewed and approved by our institutional review board.
With the proposed method, the robot was successful in 33
approaches out of 59 trials (252 people passed through). On the
other hand, with the simple method the robot was only successful
20 times out of 57 trials (221 people passed through). A chi-
square test revealed significant differences among conditions
(χ2(1) =5.076, p<.05). Residual analysis revealed that in the
proposed method, “successful approach” is significantly high
(p<.05) and “failed approach” is significantly low (p<.05). Thus,
the experimental result indicates that the proposed method
contributed to better success in approach behavior.
0
5
10
15
20
25
30
35
40
SuccessFail
The total number of trial
ProposedSimple

Fig. 10 Comparing results of field trial
6.3 Detailed Analysis of Failures
To reveal why failures decreased in the “proposed” approach, we
classified the failures based on the criteria of Table 1. This
analysis was conducted by a third person, who didn’t know the
purpose of the research. He classified failures by watching videos
and position data during the field trial.

Fig. 11 Calculating failure rate at each step
These failures are consequentially related: “unaware” failure only
happened when the robot “reached” the person, and “unsure”
failure only happened when the person was “aware” of the robot.
Only a “sure” person would “reject” the approach. Thus, we can
model these processes as a probabilistic state transition. Fig. 11
summarizes the calculations at each failure category.
Table 5 shows the failure rate at each step in each condition. In
table 5, the denominators used for calculating each failure are
different; “unreachable” used number of approach trials, but
“unaware” used number of “reached” people. Overall, we believe
that the proposed method affects each step of the approach
behavior. It shows a trend where the proposed approach behavior
less often failed than the simple approach behavior at any of the
steps. Unreachable and unaware failures were largely decreased.
Table 5 Failure rate at each step

proposed
3%
4%
18%
27%
simple
25%
14%
24%
29%
unreachable
unaware
unsure
rejected
Target
Unreachable
Reached
Unaware
Aware Sure
Unsure Rejected
Accepted
Unreachable
rate
Unaware rate Unsure rate
Rejected rate
person
person
person
robot
robot
robot
114

Page 7

7. Discussion
7.1 Applications
A couple of possible applications are enabled by a robot with
better approach capabilities. As demonstrated in this paper,
providing advertisement information is one possible application.
During the field trial, we noted revealing observations about the
robot’s effect on advertisement. For example, the robot
approached a young couple and said, “There’s a nice restaurant
named Restraint Bay in this mall. You can see Osaka Bay from it.
The view is beautiful!” The women said to the man, “He says the
restaurant has a good view. How about visiting it?” A similar
interaction happened with a child who hoped to get some ice
cream from his mother. These examples show that a robot can
help people by providing information.
Moreover, approach capability enables a robot to proactively
serve people who are not aware of its presence or of its
capabilities, e.g., a robot can provide route direction for a person
who is lost. Since people sometimes hesitate to ask for help, a
proactive way of serving is also helpful for such reluctant people.
In our study, people could nonverbally reject these services if they
wanted; we believe that this functionality is also useful to politely
provide such a proactive service. The proposed approach model,
however, is not limited to information-providing tasks; it can also
be applied to such functions as porter, shop salesperson,
receptionist, and security guard.
7.2 Generalizability and Future Works
To apply our model to such different applications, the approach
model must be improved. The “finding an interaction target”
phase especially needs to be adjusted for each of the applications.
For example, a route-guide robot needs to find a person who got
lost, which might affect observations of a person’s trajectory to
find certain patterns. A porter robot might need to be able to
observe a piece of baggage to find a person who potentially needs
such a service. We also attack cultural difference of our model.
The “interaction at a public distance” phase might depend on the
robot’s speed. In our case, we assume that robots move slower
than people. We believe that this option is safe for using robots in
a crowded city environment; but around fewer people, robots
could move faster than people. If this is the case, our path-
planning algorithm needs to be refined.
The “initiating conversation at a social distance” is currently
limited by its sensing capability. The robot failed to recognize
people’s gazes and other behaviors such as hand movements.
However, gaze has an important role in the phase[1,8]. Our
implementation was limited by the current technology; if gaze-
recognition is developed to work robustly in a real environment,
this phase could be greatly improved because the robot could
engage in eye contact to decide whether to initiate interaction.
8. Conclusion
The research aim of this paper might seem simple: “how to let a
robot approach a person.” One might argue that approaching is
easy, “just send the robot to the person and let it speak!” This was
in fact what we thought. However, it was not successful at all.
People walked so much faster than the robot. We were surprised
that people were sometimes unaware of the robot; sometimes they
failed to start conversations because they were not sure whether
the robot intended to do so. Studying these failures, we developed
a model of approach behavior. The robot selects a target person
who seems approachable and willing to interact. The robot starts
to approach on its path planned for notifying its presence. When it
reaches the social distance to the person, it nonverbally shows its
intention to interact.
The results of the field trial demonstrated the effectiveness of the
proposed model. The success rate of approaches significantly
increased. The proposed model was successful in 33 out of 59
approaches; the simplistic approach was only successful 20 out of
57 approaches. Moreover, during the field trial, we observed that
people enjoyed receiving information from the robot, suggesting
the usefulness of a proactive approach in initiating services from a
robot. The applications of approach behavior will be various.
They are not limited to simple advertisement services where a
robot just recommends shops, but will be connected to other
services for helping people with both physical services (e.g.,
transporting baggage) and information-providing services. Our
interesting future work will connect other services with the
proposed model of approach behavior.
9. ACKNOWLEDGMENTS
We wish to thank the administrative staff at Sumisho Urban
Kaihatsu Co. Ltd. for their participation. We also wish to thank Dr.
Akimoto, Dr. Miyashita, Mr. Kakio, and Mr. Kurumizawa for
their help. This research was supported by the Ministry of Internal
Affairs and Communications of Japan.
10. REFERENCES
[1] A. Kendon, Features of the structural analysis of human
communicational behavior. In Walburga von Raffler Engel,
ed., Aspects of Nonverbal Communication. 1980.
[2] A. Scheidig, S. Mueller, C. Martin, and H.-M. Gross,
Generating Persons Movement Trajectories on a Mobile
Robot, IEEE Int. Workshop on Robot and Human Interactive
Communication (Ro–Man 2006), pp. 747–752, 2006.
[3] B. Fransen et al., Using Vision, Acoustics, and Natural
Language for Disambiguation, In Proc. Int. Conf. on Human-
robot interaction (HRI2007), pp. 73-80, 2007.
[4] B. Jensen et al., The interactive autonomous mobile system
RoboX, In Proc. Int. Conf. Intelligent Robots and Systems
(IROS2002), 2002.
[5] D. Glas et al., Laser Tracking of Human Body Motion Using
Adaptive Shape Modeling, In Proc. Int. Conf. Intelligent
Robots and Systems (IROS2007), pp. 602-608, 2007.
[6] E. A. Sisbot et al., Implementing a Human-Aware Robot
System, IEEE Int. Workshop on Robot and Human
Interactive Communication, pp. 727-732, 2006.
[7] E. A. Sisbot et al., “Navigation in the Presence of Humans,”
IEEE-RAS International Conference on Humanoid Robots,
2005
[8] E. Goffman, Behavior in public place: Notes on the Social
Organization of Gatherings, 1963.
[9] E. Pacchierotti, H. I. Christensen, and P. Jensfelt, Design of
an office guide robot for social interaction studies, In Proc.
Int. Conf. Intelligent Robots and Syste , pp. 4965-4970, 2006.
[10] E. T. Hall, The Hidden Dimension, 1966.
[11] F. Tanaka et al., Socialization between toddlers and robots at
an early childhood education center, Proceedings of the
115

Page 8

National Academy of Sciences of the USA, 104(46), pp.
17954-17958, 2007.
[12] G. Hoffman and C. Breazeal, Effects of Anticipatory Action
on Human-Robot Teamwork, In Proc. Int. Conf. on Human-
Robot Interaction (HRI2007), pp. 1-8, 2007.
[13] J. Fogarty, S. E. Hudson, C. G. Atkeson, D. Avrahami, J.
Forlizzi, S. Kiesler, J. C. Lee, and J. Yang, Predicting
Human Interruptibility with Sensors, ACM Trans. on
Computer-Human Interaction, 12(1), pp. 119-146, 2005.
[14] J. Schulte, C. Rosenberg, and S. Thrun, Spontaneous,
short-term interaction with mobile robots, IEEE Int. Conf. on
Robotics and Automation (ICRA’99), pp. 658-663, 1999.
[15] K. Dautenhahn et al., How may I serve you?: a robot
companion approaching a seated person in a helping context,
In Proc. Int. Conf. on Human-robot interaction (HRI2006),
pp. 172–179, 2006.
[16] K. Hayashi, D. Sakamoto, T. Kanda, M. Shiomi, S. Koizumi
H. Ishiguro, T. Ogasawara, and N. Hagita, Humanoid robots
as a passive-social medium - a field experiment at a train
station -, ACM/IEEE 2nd Annual Conference on Human-
Robot Interaction (HRI2007), pp. 137-144, 2007.
[17] K. Wada, T. Shibata, T. Saito, and K. Tanie, Analysis of
Factors that Bring Mental Effects to Elderly People in Robot
Assisted Activity, IEEE/RSJ Int. Conf. on Intelligent Robots
and Systems, 2002.
[18] K. Yamazaki et al., Prior-to-request and request behaviors
within elderly day care: Implications for developing service
robots for use in multiparty settings, Proceedings of the
Tenth European Conference on Computer Supported
Cooperative Work, pp. 61-78, 2007.
[19] M. Bennewitz, W. Burgard, G. Cielniak, and S. Thrun,
Learning Motion Patterns of People for Compliant Robot
Motion, Int. J. of Robotics Research, 24(1), pp. 31-48, 2005.
[20] M. Finke, Kheng Lee Koay, and Kerstin Dautenhahn, Hey,
I’m over here – How can a robot attract people’s attention,
IEEE Int. Workshop on Robot and Human Interactive
Communication (Ro-Man 2005), pp. 7 – 12, 2005.
[21] M. L. Walters, D. S. Syrdal, K. L. Koay, K. Dautenhahn, R.
te Boekhorst, Human Approach Distances to a Mechanical-
Looking Robot with Different Robot Voice Styles, IEEE Int.
Symposium on Robot and Human Interactive
Communication (Ro-Man 2008), pp. 707-712, 2008.
[22] M. P. Michalowski, S. Sabanovic, and R. Simmons, A
Spatial Model of Engagement for a Social Robot, IEEE
International Workshop on Advanced Motion Control
(AMC’06), pp. 762-767, 2006.
[23] M. Shiomi, T. Kanda, H. Ishiguro, and N. Hagita, Interactive
Humanoid Robots for a Science Museum, IEEE Intelligent
Systems, vol. 22, no. 2, pp. 25-32, 2007.
[24] M. Yamamoto and T. Watanabe, Timing Control Effects of
Utterance to Communicative Actions on Embodied
Interaction with a Robot, IEEE Int. Workshop on Robot and
Human Interactive Communication, pp. 467-472, 2004.
[25] N. Bergström, T. Kanda, T. Miyashita, H. Ishiguro, and N.
Hagita, Modeling of Natural Human-Robot Encounters, Int.
Conf. Intelligent Robots and Systems (IROS2008), 2008.
[26] R. Gockley, J. Forlizzi, and R. Simmons, Natural person-
following behavior for social robots, In Proc. Int. Conf. on
Human-robot interaction (HRI2007), pp. 17-24, 2007.
[27] R. Gockley et al., Designing robots for long-term social
interaction, In Proc. Int. Conf. Intelligent Robots and
Systems (IROS2006), pp. 1338- 1343, 2006.
[28] R. Siegwart et al., Robox at expo.02: A Large Scale
Installation of Personal Robots, Robotics and Autonomous
Systems, 42(3), pp. 203-222, 2003.
[29] S. Kagami, S. Thompson, Y. Nishida, T. Enomoto, and T.
Matsui. Home robot service by ceiling ultrasonic locator and
microphone array, IEEE International Conference on
Robotics and Automation (ICRA2006), pp. 3171-3176, 2006.
[30] S. Li, B. Wrede, and G. Sagerer, A dialog system for
comparative user studies on robot verbal behavior, IEEE Int.
Symposium on Robot and Human Interactive
Communication (Ro-Man 2006), pp.129-134, 2006.
[31] T. Kanda, D. F. Glas, M. Shiomi, H. Ishiguro, and N. Hagita,
Who will be the customer?: A social robot that anticipates
people’s behavior from their trajectories, Int. Conference on
Ubiquitous Computing (UbiComp 2008), 2008.
[32] T. Kanda et al., Interactive Robots as Social Partners and
Peer Tutors for Children: A Field Trial, Human Computer
Interaction, 19(1-2), pp. 61-84, 2004.
[33] T. Tasaki, S. Matsumoto, H. Ohba, M. Toda, K. Komatani, T.
Ogata, and H. G. Okuno, Dynamic Communication of
Humanoid Robot with multiple People Based on Interaction
Distance, IEEE Int. Workshop on Robot and Human
Interactive Communication (Ro-Man 2004), pp.71-76, 2004.
[34] W. Burgard et al., The interactive museum tour-guide robot,
Proc. of National Conference on Artificial Intelligence, pp.
11-18, 1998.
[35] X. Shao et al., Detection and tracking of multiple pedestrians
by using laser range scanners, Int. Conf. on Intelligent
Robots and Systems (IROS 2007), pp. 2174-2179, 2007.

116