Conference PaperPDF Available

POMDP-based long-term user intention prediction for wheelchair navigation

Authors:
  • Dubai Futue Labs

Abstract and Figures

This paper presents an intelligent decision-making agent to assist wheelchair users in their daily navigation activities. Several navigational techniques have been successfully developed in the past to assist with specific behaviours such as "door passing" or "corridor following". These shared control strategies normally require the user to manually select the level of assistance required during use. Recent research has seen a move towards more intelligent systems that focus on forecasting users' intentions based on current and past actions. However, these predictions have been typically limited to locations immediately surrounding the wheelchair. The key contribution of the work presented here is the ability to predict the users' intended destination at a larger scale, that of a typical office arena. The systems relies on minimal user input - obtained from a standard wheelchair joystick - in conjunction with a learned Partially Observable Markov Decision Process (POMDP), to estimate and subsequently drive the user to his destination. The projection is constantly being updated, allowing for true user- platform integration. This shifts users' focus from fine motor- skilled control to coarse control broadly intended to convey intention. Successful simulation and experimental results on a real wheelchair robot demonstrate the validity of the approach.
Content may be subject to copyright.
POMDP-based Long-term User Intention Prediction for Wheelchair
Navigation
Tarek Taha, Jaime Valls Mir´
o and Gamini Dissanayake
Abstract This paper presents an intelligent decision-making
agent to assist wheelchair users in their daily navigation ac-
tivities. Several navigational techniques have been successfully
developed in the past to assist with specific behaviours such as
“door passing” or “corridor following”. These shared control
strategies normally require the user to manually select the level
of assistance required during use. Recent research has seen a
move towards more intelligent systems that focus on forecasting
users’ intentions based on current and past actions. However,
these predictions have been typically limited to locations im-
mediately surrounding the wheelchair. The key contribution of
the work presented here is the ability to predict the users’
intended destination at a larger scale, that of a typical office
arena. The systems relies on minimal user input - obtained from
a standard wheelchair joystick - in conjunction with a learned
Partially Observable Markov Decision Process (POMDP), to
estimate and subsequently drive the user to his destination. The
projection is constantly being updated, allowing for true user-
platform integration. This shifts users’ focus from fine motor-
skilled control to coarse control broadly intended to convey
intention. Successful simulation and experimental results on a
real wheelchair robot demonstrate the validity of the approach.
I. MOTIVATIO N
The world’s aging population and the large number of
people effected by motor disabilities has motivated past
researchers to develop assistive technologies to give other-
wise immobile people freedom of movement, dramatically
increasing independence and improving the quality of life of
those affected. Systems such as robotic walkers [1], smart
blind sticks [2] and robotic wheelchairs [3]–[8] have been
developed with this goal in mind. Out of these, robotic elec-
tric wheelchairs are particularly desirable given their social
acceptance and ubiquity. Yet depending on the users’ type
of disability, safely and effectively driving the wheelchair
may be difficult in general, for example people with severe
tremors. Furthermore, wheelchairs are also large in com-
parison to the passageways typical of indoors environments
such as offices, nursing homes, hospitals and the home
environment. This means that for some users, apparently
trivial tasks such as passing through a doorway or navigating
a hallway may be quite challenging. This work proposes
an intelligent driving system designed to assist these users
This work is supported by the Australian Research Council (ARC)
through its Centre of Excellence programme, and by the New South Wales
State Government. The ARC Centre of Excellence for Autonomous Systems
(CAS) is a partnership between the University of Technology Sydney, the
University of Sydney and the University of New South Wales.
All authors are with the ARC Centre of Excellence
for Autonomous Systems (CAS), Faculty of Engineering,
University of Technology Sydney (UTS), NSW2007, Australia
{t.taha,j.vallsmiro,g.dissanayake}@cas.edu.au
by understanding and complying with their intentions. The
interaction takes place transparently to the user, demanding
a minimal input from them to automatically perform the
required fine motion control for the given situation.
II. INT RODU CTI ON AN D RELATE D WORK
If a truly user-machine integrated system is to be devel-
oped, the type of cooperation between user and machine
must be comparable to the cooperation between a horse
and its rider [9]: the rider navigates, while the horse avoids
(small) dangerous obstacles on the ground. To achieve this
level of user-machine integration the machine must grow and
learn with the user so that a relationship may form (such as
that between a horse and its rider) and so that the machine
can predict the users intention and autonomously enact the
intention with only minimal corrective input from the user.
The fundamental component of this relationship is inten-
tion recognition. The primary considerations for an inten-
tion recognition system are whether an accurate representa-
tion/model of the environment is required and whether the
system is going to be reactive or deliberative. Reactive refers
to systems that do not use a representation of the environment
and therefore are usually weak in decision making and long-
term prediction. They rely on local or temporal information
collected on-line which might not be sufficient to develop
correct long-term plans. They establish a direct link between
the perceptions obtained by their sensors and their effectors;
the control doesn’t comply to a model but simply happens as
a low level response to the perception. Systems with limited
resources like processing power, memory and communica-
tion mediums often use a reactive system but the scope
of their possible applications and intelligence is limited.
Several wheelchair platforms such as Rolland III [4], Bre-
men Autonomous Wheelchair, Sharioto [3], RobChair [5],
Senario [6], VAHM [7], Wheelesley [8], and Navchair [10]
employ reactive control algorithms, limited by either the set
of operating modes that the user must select (e.g. manual,
fully autonomous or semi-autonomous), and/or by the limited
scope of their navigation algorithms, reduced to a local scale.
In the last few years some wheelchair assistive reactive
techniques have emerged which overcome some of these
restrictions by capturing the users local intentions in order
to facilitate a limited set of tasks such as avoiding obstacles,
following a wall, entering a room, going towards an object
or other local area of interest. These algorithms are based on
systems that can act intelligently but not think intelligently.
In other word, they try to make the link between perception
and action as direct as possible by combining decisions
made by both the human and the machine [9], [10]. Even
though such techniques appear to work well in predicting the
user’s intentions at a local scale (same room or same open
space), they lack the cognitive capabilities to autonomously
recognize them; rather, those must be manually specified
prior to system operation. Moreover, they lack the markedly
higher scope of assistance that would support specific user’s
activities with the appropriate sequence of actions beyond
the local boundaries, such as going to the bathroom or out
the main door.
This paper presents an intention recognition and goal
prediction assistance strategy that uses environmental knowl-
edge to plan and interact with the user in a deliberate
manner, but at a larger scale. Typically, disabled users
requiring wheelchair assistance have a known set of target
locations that they go to during their daily activities, such
as the bathroom, kitchen or T.V. room. Through monitoring
a wheelchair user through his daily routine/activities, the
technique hereby proposed first determines the locations
of interest that the user regularly frequents and builds
knowledge about these locations using machine learning
techniques. This knowledge is then used to predict the users
intended destination from sensed inputs, as derived from a
POMDPs model. In this scheme of things, the wheelchair is
considered an intelligent agent with an internal representation
of the environment which effectively translates these target
destinations into a plan of action to reach them in the
presence of uncertainties. An intelligent controller subse-
quently performs the lower level navigational tasks such as
local path planning, collision avoidance and actuating motion
control. It is important to emphasize that whilst in motion
the user remains in complete control of the system, providing
continuous (or discrete) action/course correcting feedback to
the system through the intention recognition algorithm.
III. THE POMDP MODEL
Partially Observable Markov Decision Processes
(POMDP) provide a general framework for sequential
decision making in environments where states are hidden
and actions are stochastic. POMDPs were recently used in
assistive applications and they proved to be reliable and
efficient when modelled properly [11], [12]. A POMDP
model represents the dynamics of the environment, such
as the probabilistic outcomes of the actions (the transition
function T), the reward function R, and the probabilistic
relationships between the agents observations and the
states of the environment (the observation function O). In
POMDP terminology, system states are typically referred to
as “hidden states”, since they are not directly observable by
the agent. The POMDP framework is a systematic approach
that uses belief states to represent memory of past actions
and observations. Thus, it enables an agent to compute
optimal policies using the MDP framework, as depicted in
Fig. 1.
A POMDP model is defined by < S, A, T, R, Z, γ, O >,
a seven tuple where:
S: A set of states that represents the state of the system
at each point in time.
A: A set of actions that an agent can take (can depend
on the current state).
T:A×S×S[0,1]: The state transition function,
which maps each state action pair into a probability
distribution over the state space. The next distribution
over the state space depends only on the current state
action pair and not on the previous state action pairs.
This requirement ensures the Markovian property of
the process. We define T(s, a, s0)as the probability that
an agent took action afrom state sand reached state
s0.
R:S×A→ < : The immediate reward function which
indicates the reward for doing an action in some state.
Z: A set of observations.
γ: A discount factor used to reduce the award given to
future (and more uncertain) steps.
O:A×S×Z[0,1]: A function that maps the action at
time t-1 and the state at time tto a distribution over the
observation set. We define O(s0, a, z)as the probability
of making observation zgiven that the agent took action
aand landed in state s0.
The belief is a sufficient statistic for a given history and it
is updated at each time step according to 1, where P r(o|a, b)
is a normalizing constant [13], [14].
b0(s0) = O(s0, a, o)PsST(s, a, s0)b(s)
P r(o|a, b)(1)
Given a POMDP model, the goal is to find a sequence
of actions, or policy, a0, ...., at that maximizes the expected
sum of rewards E[PtγtR(st, at)]. Since the states are not
fully observable, the goal is to maximize the expected reward
for each belief [15]. The function V(s)that solves the
Bellman equation (2) is called the value function, and its
associated optimal policy can be formulated using (3).
V(s) = maxa[R(s, a) + γX
s0S
T(s, a, s0)V(s0)] (2)
π
t=argmaxa[R(s, a) + γX
s0S
T(s, a, s0)V
t1(s0)] (3)
IV. POMDP PROB LE M SP EC I FIC ATI ON
Within this POMDP framework, our intention recogni-
tion problem is transferred into a planning problem where
the wheelchair is transformed into a decision maker agent
required to find the best plan (optimal policy) that repre-
sents the user’s intention by reducing the uncertainty in
the belief state, categorized by the Destination the user
is trying to reach. The state space is described by the
cross product of two features, the W chairLocation =
{s1, ..., sx}and the Destination ={d1, ..., dy}resulting
in a StateSpace ={s1d1, s2d1, ..., sxdy}. The wheelchair
starts from a known position and the plan finishes when
the W chairLocation is the same as the Destination.
The W chair can have one of the following actions:
Fig. 1. A POMDP agent is made up of two main components. The state
estimator module receives observations from the environment, the last action
taken and the previous belief state and produces an updated belief state. The
policy module maps belief state to an action.
{N orth, South, East, W est, DoN othing}indicating the
global direction of travel. A reward of -1 is given for each
motion step and +100 reward is given when the Wchair
performs an action that leads to Destination. It is assumed
the W chairLocation is fully observable via a localizer,
but not the destination, and the effect of an action has a
predictable deterministic effect as the example described
by (4):
P r(W chair =Sx|W chair =Sy, S outh) = 1 (4)
The position of the Destination is unobservable until
the wheelchair reaches its destination. At each state the
joystick input is observed and is represented by a set of
discrete states = {U p, Down, Right, Lef t, NoInput}, and
the uncertainty in the user’s input is taken into consideration
while generating the observation model (further explained in
section V-C).
V. POM DP GE NER ATI ON
To obtain a efficient POMDP system, we need to have
proper T ransition,Observation and StateSpace models.
Our model generation consists of three major parts as de-
picted in Fig. 2. These three steps will be explained in the
subsections below:
A. State Space
In our assistive system, we want the user to be able to
navigate in a high level topological manner. This means that
the user should be focusing on driving the wheelchair from
one room to another, or from one spatial location to another
without having to worry about the intermediate steps that
comes in between (planning wise). In order for us to do
so, only significant spatial feature are considered, such as a
hallway intersection, a door opening or a room.
The ability to learn tasks and represent environments
[16], [17] is essential in our application as it creates the
bases for the long term intention recognition and prediction.
This is done by simplifying the encapsulation of spatial
and activity information. For this to happen, the wheelchair
should have the ability to represent the spatial information
Fig. 2. The POMDP model generation architecture. The map topology
together with the training data are used to determine the transition model.
The training data is also used to determine the observation model of
the POMDP. User’s Joystick calibration determines the uncertainty in the
observations.
TABLE I
LIS T OF TASKS RECORDED FROM THE USERS ACTIVITIES
Start End Path
Task1 Lab Office 26/D - 25/L - 24/L - 22/D - 23/N
Task2 Office Meeting 42/U - 40/L - 43/U - 44/N
Task3 Office Bathroom 3/D - 4/L - 5/D - 6/N
of the environment in a simplistic topological manner that
can make it easier to store, extract and update information.
For our POMDP platform, the state space consists
of two features: the W chairLocation and the intended
Destination. The cross product of the above two feature
will form the StateSpace ={s1d1, s2d1, ..., sxdy}, these
features are separately extracted in two different steps de-
scribed below:
1) Spatial States: The spatial representation we are using
is based on the topological graph representation of the
environment, where vertices are locations in the environment
and edges represent a viable path connecting two locations
as a result of performing an action. In our research we are
mainly targeting indoor office or home environments. For
such environments there has been a lot of research done
on how to build maps and extract topological representation
accurately. For simplicity, we assume that the maps are
already available and that the topological map representation
is hand coded and programmed. It might be more convenient
in the future to consider a complete system that can build
maps and extract topological representations simultaneously
but this is out of the scope of the current research. The
map topology will be represented by a graphical tree of
nodes and connections (segments), where the set of nodes
W chairLocation ={s1, ..., sx}represents a location in
the map and the connection represents a physical path that
connects two locations. The hand coded spatial configuration
of the domain used for planning illustrated in Fig. 3.
2) Destinations States: Identifying places of interest is not
an easy task and there is no direct method to achieve this as
it is an application and environment dependent problem. For
the prediction problem we are trying to solve, it’s sufficient
Fig. 3. The map topology used for our intention recognition. Circles represent intersections and cannot be a destination while squares represent rooms or
open spaces and can be considered as a possible destination. The numbers inside the circles represent the state number and is used to build the transition
model. Gray shaded rectangles represent learned destinations.
to think about the place of interest as a spatial location in
the environment where the user spends significantly most
of his/her time. After observing the user’s activities we can
determine the time that the user need to stay in the same
place for it to be considered as a place of interest. In general
staying few minutes in a certain location can nominate that
location to the place of interest set. For POMDP model
generation purposes we log the activities of the user over
a period of time, then in that log we determine the locations
of interest Destination ={d1, ..., dy}based on the above
criteria.
B. Transition Model
Transition model specifies the translation from one state
to another given a certain action T(s, a, s0). In our model
specifications, the actions are global navigation commands
{N orth, South, East, W est, Stop}and determines the spa-
tial node that we will end up at if we are in location sand
executed the action a. The transition mode is built directly
from the map topology. This transition is deterministic and
independent of the intention, i.e., it is derived regardless
of where the user wants to go. The result of executing an
action in the same location will be the same. For exam-
ple T(s3d1, N orth, s2d1) =T(s3d2, North, s2d2) = 1 in
Fig. 3.
C. Observation Model
The observation model defines the probability of observing
zgiven that the wheelchair took an action aand landed in
state s0O(s0, a, z). To generate a proper observation model
that correctly models the user’s intention and activities, we
use a training data from that particular user. In an indoor
environment, the wheelchair users usually perform a repet-
itive set of tasks that represents navigating from one place
to another. A task can be for example going from the living
room to the bathroom or to the kitchen. This set of tasks
can be defined by the user himself or extracted from a set of
data recorded by monitoring the user’s activities. The tasks
are defined by a starting location, intermediate locations, end
location and the joystick inputs/observation that the user gave
at each location as described in Table I where the path is
represented by numbers corresponding to the states’ numbers
and the letters corresponding to the observation in each state
(L=Left, R=Right, U=Up and D=Down).
The user in many cases might be unable to give a proper
joystick input due to disability or a disease causing a shaky
hand for example. To best customize the POMDP model
for this user, a joystick calibration is required to determine
the uncertainties in the user’s inputs. This uncertainty will
be a set of nprobabilities describing the user’s inability
to give the right joystick input, where nis the number of
JoystickInputs ={U p, Down, Right, Lef t, N oInput}.
Having obtained the training data and the uncertainty,
the observation model is then generated by adding the
uncertainty to the frequency probability (the probability of
obtaining a certain observation in a state).
VI. ON-L I NE AS SIS TAN CE
Once the planning problem is formulated, we solve the
POMDP to get the optimal policy π. While predicting
on-line, we start with an initial belief state bt. Since we
Fig. 4. The POMDP driver assistance architecture. The user’s input together
with the current location generate an observation that helps in updating the
belief in the destination. The appropriate action will be selected based on
that belief, and the next state will then be determined and given to the
navigator to drive the wheelchair to the next state.
know our current location from our localizer, the initial
belief is limited to those states in the StateSpace with
our current location, and we will end up with a belief set
size equivalent to the available destinations. For example,
if our Destination =K itchen, Bathroom, T .V Room and
we know where we are, then our initial belief is distributed
among these destinations and is equal to 1/3. Based on our
initial belief, we execute the optimal policy action for that
belief state π(st), calculate the reward rtfor taking that
action, get an observation zt+1 and update our belief bt+1,
then repeat the procedure. This is described in Procedure 1
and illustrated in Fig. 4.
Procedure 1 On-line Navigation
1. Initial belief: bt.
2. Execute the action from the optimal policy: π(st).
3. Calculate the reward: rt.
4. Get an observation : zt+1.
5. Update the belief: bt+1.
6. Repeat until destination reached.
VII. EXPERIMENTAL RES ULTS
To validate the proposed intention recognition architecture
we simulated a training data that represents the activities of a
user in the environment shown in Fig. 3. The destinations are
represented by the gray shaded squares and they form the set
Destination ={s1d1, s6d2, s26d3, s30d4, s31d5, s38d6}.
The POMDP was generated using a simulated training data
with uncertainty added to the observations to represent
the user’s ability to control the joystick (in this example
uncertainty on Up=10%, Down=5%, Right=15%, Left=10%
and Nothing=20%). The generated POMDP problem was
then solved using zmdpSolver [18] and the optimal policy
was obtained.
The generated policy and model were tested against the
tasks in the training data. For each task in the training data
Fig. 5. The result of a real experiment. The wheelchair starts in state 22
and tries to predict where the user is going to based on his joystick inputs
(observations). The wheelchair in this case successfully takes the user’s
joystick inputs and decides on the correct actions that take the user to state
30.
Fig. 6. The result of a real Wheelchair experiment showing the path (dashed
line) and observations (arrows). The wheelchair starts in state 2 and drives
the user successfully to state 26 by updating the belief at each step from
the obtained observation.
we start with a known location (the first state in the task) but
unknown destination (equal belief among destinations) then
we take observations from that task one by one, update the
belief based on these observations, select an action based on
the optimal policy and execute that action to get to the next
state. This procedure is repeated until we reach the end of
the observations in the task. If the end state reached after the
last observation is the same as the intended destination (the
last state in the task), then the test is considered successful,
otherwise it fails. The test was successful in all of the 289
tasks in this experiment producing a 100% success rate.
An example of a navigation task on a real wheelchair
platform is shown in Fig. 5. The wheelchair used was
the one described in [19] and it measures 1.2x0.7m. The
wheelchair’s size is considered large compared to the envi-
ronment and driving it in such a constrained environment
can be a challenging task for inexperienced users or users
with severe tremors. In this example, the user was giving
observations at each state to indicate where he wants to go.
Initially, the user can be going to any of the pre-determined
destinations, therefore the belief is uniformly distributed
among them. With the first observation, the belief is updated
and the next state is determined based on the appropriate
selected action and the wheelchair navigates to that state
autonomously. This is repeated until the user reaches his
destination.
A longer real wheelchair navigation example can be seen
in Fig. 7. The path followed and the observations obtained
Fig. 7. The results of the navigation experiment depicted in Fig. 6.
are those illustrated in Fig. 6. The same procedure as the
one described above is used and again the sequence of
observations help the system to successful drive the user to
his/her destination.
VIII. CONCLUSION
In this paper we have presented a new method for
wheelchair assistance that considers the wheelchair as a
smart robotic agent, interacting with the user with the
aid of a sequential decision making algorithm (POMDP).
Unlike most of the currently available assistive methods
that are based on semi-autonomous systems which merge
wheelchair’s perception and user’s control with some added
heuristics, our method tries to predict where the wheelchair’s
user is trying to go, and takes him there without any extra
mode or behavioural selection. POMDP was chosen because
it provides a good platform for planning and predicting under
uncertainty for human-robot interaction, as we have shown
in this paper. The results we have obtained so far from the
simulated and real platform tests are promising and they
validate our method. Our efforts in the future will be devoted
to further enhance the capabilities and the intelligence of
the system through automated activity monitoring and tasks
extraction.
REFERENCES
[1] M. Alwan, P. J. Rajendran, A. Ledoux, C. Huang, G. Wasson, and
P. Sheth. Stability margin monitoring in steering-controlled intelligent
walkers for the elderly. In Proceedings of AAAI Symposium, 2005.
[2] S. Kang, Y. Ho, and I. Hyuk Moon. Development of an intelligent
guide-stick for the blind. Proceedings of IEEE International Confer-
ence on Robotics and Automation, 4:3208–3213, 2001.
[3] D. Vanhooydonck, E. Demeester, M. Nuttin, and H. V. Brussel. Shared
control for intelligent wheelchairs: an implicit estimation of the user
intention. In Proceedings of the International Workshop on Advances
in Service Robotics, 2003.
[4] C. Mandel, K. Huebner, and T. Vierhuff. Towards an autonomous
wheelchair: Cognitive aspects in service robotics. In Proceedings of
Towards Autonomous Robotic Systems, pages 165–172, 2005.
[5] G. Pires, R. Araujo, U. Nunes, and A. T. de Almeida. Robchair-a
powered wheelchair using a behaviour-based navigation. Proceedings
of International Workshop on Advanced Motion Control, pages 536–
541, 1998.
[6] P. Beattie and J. M. Bishop. Localization of the senario wheelchair.
In Proceedings of MobiNet Symposium Mobile Robotics Technology
for Health Care Services, pages 287–293, 1997.
[7] A. Pruski, M. Ennaji, and Y. Morere. Vahm: a user adapted intelligent
wheelchair. Proceedings of the International Conference on Control
Applications, 2:784–789, 2002.
[8] H. A. Yanco. Development and testing of a robotic wheelchair system
for outdoor navigation. In Proceedings of the Conference of the
Rehabilitation Engineering and Assistive Technology Society of North
America, 2001.
[9] K. A. Tahboub. Intelligent human-machine interaction based on dy-
namic bayesian networks probabilistic intention recognition. Journal
of Intelligent and Robotic Systems, 45(1):31–52, 2006.
[10] E. Demeester, A. Huntemann, D. Vanhooydonck, G. Vanacker,
A. Degeest, H. V. Brussel, and M. Nuttin. Bayesian estimation of
wheelchair driver intents: Modeling intents as geometric paths tracked
by the driver. International IEEE/RSJ Conference on Intelligent Robots
and Systems, pages 5775–5780, Oct. 2006.
[11] M. Lopez, L. M. Bergasa, R. Barea, and M. S. Escudero. A navi-
gation system for assistant robots using visually augmented pomdps.
Autonomous Robots, 19(1):67–87, 2005.
[12] J. Hoey, A. V. Bertoldi, P. Poupart, and A. Mihailidis. Assisting
persons with dementia duing handwashing using a partially observ-
able markov decision process. In Proceedings of the International
Conference on Vision Systems (ICVS), Biefeld, Germany, 2007.
[13] L. P. Kaelbling, M. L. Littman, and A. R. Cassandra. Planning
and acting in partially observable stochastic domains. Artificial
Intelligence, 101:99–134, 1998.
[14] J. Pineau and G. Gordon. Pomdp planning for robust robot control. In
Proceedings of International Symposium on Robotics Research (ISRR),
2005.
[15] J. Pineau, G. Gordon, and S. Thrun. Point-based value iteration:
An anytime algorithm for pomdps. In Proceedings of International
Joint Conference on Artificial Intelligence (IJCAI), pages 1025–1032,
August 2003.
[16] T. Spexard, S. Li, B. Wrede, J. Fritsch, G. Sagerer, O. Booij,
Z. Zivkovic, B. Terwijn, and B. Krose. Biron, where are you? enabling
a robot to learn new places in a real home environment by integrating
spoken dialog and visual localization. Intelligent Robots and Systems,
2006 IEEE/RSJ International Conference on, pages 934–940, Oct.
2006.
[17] M. Heerink, B. J. A. Krose, B. J. Wielinga, and V. Evers. Human-robot
user studies in eldercare: Lessons learned. June 2006.
[18] T. Smith. zmdpsolver. [http://www.cs.cmu.edu/˜trey/zmdp/].
[19] T. Taha, J. V. Miro, and D. Lui. An efficient path planner for large
mobile platforms in cluttered environments. In Proceedings of IEEE
Conference on Robotics, Automation and Mechatronics, pages 225–
230, 2006.
... When the state space is finite, frameworks for both user's intent estimation and planning of supporting actions in assistive robotic applications are proposed using a POMDP formulation [13]. Other applications include the importance of future action prediction for interactive humanoid robots in manipulation [14] and trajectory prediction [15]. ...
Preprint
This paper addresses the problem of human operator intent recognition during teleoperated robot navigation. In this context, recognition of the operator's intended navigational goal, could enable an artificial intelligence (AI) agent to assist the operator in an advanced human-robot interaction framework. We propose a Bayesian Operator Intent Recognition (BOIR) probabilistic method that utilizes: (i) an observation model that fuses information as a weighting combination of multiple observation sources providing geometric information; (ii) a transition model that indicates the evolution of the state; and (iii) an action model, the Active Intent Recognition Model (AIRM), that enables the operator to communicate their explicit intent asynchronously. The proposed method is evaluated in an experiment where operators controlling a remote mobile robot are tasked with navigation and exploration under various scenarios with different map and obstacle layouts. Results demonstrate that BOIR outperforms two related methods from literature in terms of accuracy and uncertainty of the intent recognition.
... Various probabilistic models have been proposed and used to estimate the PWC user's driving intent from joystick inputs. The most common models used in this context are based on Markov decision processes [14], Bayesian networks [15], and Gaussian mixture models (GMMs) [16]. Similar probabilistic decision models were proposed and used to estimate vehicle drivers' intent (e.g., to change lane) [17], [18]. ...
... It is not uncommon in human-robot interaction and assistive teleoperation studies that the robot is assumed to know the human intention [24][25][26][27][28][29][30][31][32]. In some other studies, it is assumed that the human is following one of a predefined goals or paths, and then a classifier is used to decide the human goal [33][34][35][36][37][38][39]. In many real-world scenarios, explicit goal specification may not be possible, or is undesirable [3]. ...
Article
Full-text available
Human-in-the-loop robot control systems naturally provide the means for synergistic human–robot collaboration through control sharing. The expectation in such a system is that the strengths of each partner are combined to achieve a task performance higher than that can be achieved by the individual partners alone. However, there is no general established rule to ensure a synergistic partnership. In particular, it is not well studied how humans adapt to a nonstationary robot partner whose behavior may change in response to human actions. If the human is not given the choice to turn on or off the control sharing, the robot–human system can even be unstable depending on how the shared control is implemented. In this paper, we instantiate a human–robot shared control system with the “ball balancing task,” where a ball must be brought to a desired position on a tray held by the robot partner. The experimental setup is used to assess the effectiveness of the system and to find out the differences in human sensorimotor learning when the robot is a control sharing partner, as opposed to being a passive teleoperated robot. The results of the four-day 20-subject experiments conducted show that 1) after a short human learning phase, task execution performance is significantly improved when both human and robot are in charge. Moreover, 2) even though the subjects are not instructed about the role of the robot, they do learn faster despite the nonstationary behavior of the robot caused by the goal estimation mechanism built in.
... providing assistance for a specific task) also were introduced. Finally learning techniques such as using Partially Observable Markov Decision Processes [TMD08], or Leaning from Demonstration [GDA13] were investigated for long term intention prediction. All intention prediction mechanisms described above are highly deterministic and do not have the capability of estimating user intention uncertainty which may cause much discomfort to the user if the wheelchair does not behave as intended. ...
Thesis
Les premiers documents attestant l'utilisation d'une chaise à roues utilisée pour transporter une personne avec un handicap datent du 6ème siècle en Chine. À l'exception des fauteuils roulants pliables X-frame inventés en 1933, 1400 ans d'évolution de la science humaine n'ont pas changé radicalement la conception initiale des fauteuils roulants. Pendant ce temps, les progrès de l'informatique et le développement de l'intelligence artificielle depuis le milieu des années 1980 ont conduit inévitablement à la conduite de recherches sur des fauteuils roulants intelligents. Plutôt que de se concentrer sur l'amélioration de la conception sous-jacente, l'objectif principal de faire un fauteuil roulant intelligent est de le rendre le plus accessible. Même si l'invention des fauteuils roulants motorisés ont partiellement atténué la dépendance d'un utilisateur à d'autres personnes pour la réalisation de leurs actes quotidiens, certains handicaps qui affectent les mouvements des membres, le moteur ou la coordination visuelle, rendent impossible l'utilisation d'un fauteuil roulant électrique classique. L'accessibilité peut donc être interprétée comme l'idée d'un fauteuil roulant adaptée à la pathologie de l'utilisateur de telle sorte que il / elle soit capable d'utiliser les outils d'assistance. S'il est certain que les robots intelligents sont prêts à répondre à un nombre croissant de problèmes dans les industries de services et de santé, il est important de comprendre la façon dont les humains et les utilisateurs interagissent avec des robots afin d'atteindre des objectifs communs. En particulier dans le domaine des fauteuils roulants intelligents d'assistance, la préservation du sentiment d'autonomie de l'utilisateur est nécessaire, dans la mesure où la liberté individuelle est essentielle pour le bien-être physique et social. De façon globale, ce travail vise donc à caractériser l'idée d'une assistance par contrôle partagé, et se concentre tout particulièrement sur deux problématiques relatives au domaine de la robotique d'assistance appliquée au fauteuil roulant intelligent, à savoir une assistance basée sur la vision et la navigation en présence d'humains. En ciblant les tâches fondamentales qu'un utilisateur de fauteuil roulant peut avoir à exécuter lors d'une navigation en intérieur, une solution d'assistance à bas coût, basée vision, est conçue pour la navigation dans un couloir. Le système fournit une assistance progressive pour les tâches de suivi de couloir et de passage de porte en toute sécurité. L'évaluation du système est réalisée à partir d'un fauteuil roulant électrique de série et robotisé. A partir de la solution plug and play imaginée, une formulation adaptative pour le contrôle partagé entre l'utilisateur et le robot est déduite. De plus, dans la mesure où les fauteuils roulants sont des dispositifs fonctionnels qui opèrent en présence d'humains, il est important de considérer la question des environnements peuplés d'humains pour répondre de façon complète à la problématique de la mobilité en fauteuil roulant. En s'appuyant sur les concepts issus de l'anthropologie, et notamment sur les conventions sociales spatiales, une modélisation de la navigation en fauteuil roulant en présence d'humains est donc proposée. De plus, une stratégie de navigation, qui peut être intégrée sur un robot social (comme un fauteuil roulant intelligent), permet d'aborder un groupe d'humains en interaction de façon équitable et de se joindre à eux de façon socialement acceptable. Enfin, à partir des enseignements tirés des solutions proposées d'aide à la mobilité en fauteuil roulant, nous pouvons formaliser mathématiquement un contrôle adaptatif partagé pour la planification de mouvement relatif à l'assistance à la navigation. La validation de ce formalisme permet de proposer une structure générale pour les solutions de navigation assistée en fauteuil roulant et en présence d'humains.
Article
With the substantial growth of logistics businesses the need for larger warehouses and their automation arises, thus using robots as assistants to human workers is becoming a priority. In order to operate efficiently and safely, robot assistants or the supervising system should recognize human intentions in real-time. Theory of Mind (ToM) is an intuitive human conception of other humans’ mental state, i.e., beliefs and desires, and how they cause behavior. In this paper we propose a ToM based human intention estimation algorithm for flexible robotized warehouses. We observe human’s, i.e., worker’s motion and validate it with respect to the goal locations using generalized Voronoi diagram based path planning. These observations are then processed by the proposed hidden Markov model framework which estimates worker intentions in an online manner, capable of handling changing environments. To test the proposed intention estimation we ran experiments in a real-world laboratory warehouse with a worker wearing Microsoft Hololens augmented reality glasses. Furthermore, in order to demonstrate the scalability of the approach to larger warehouses, we propose to use virtual reality digital warehouse twins in order to realistically simulate worker behavior. We conducted intention estimation experiments in the larger warehouse digital twin with up to 24 running robots. We demonstrate that the proposed framework estimates warehouse worker intentions precisely and in the end we discuss the experimental results.
Article
Human-Robot Collaboration (HRC) studies how to achieve effective collaborations between human and robots to take advantage of the flexibility from human and the autonomy from robots. Many applications involving HRC, such as joint assembly manufacturing systems and advanced driver assistance systems, need to achieve high-level tasks in a provably correct manner. These applications motivate the requirements of HRC to have the performance guarantee to assure the task completion and safety of both human and robots. In this paper, a correct-by-design HRC framework is built to enable a performance guaranteed HRC. To model the uncertainties from human, robots and the environment, partially observable Markov decision process (POMDP) is used as the model. Based on the POMDP modeling, a supervisory control framework is applied and designed to be adaptive to modeling uncertainties. To reduce the model checking complexity involved in the supervisor synthesis process, an abstraction method for POMDP is integrated to find a quotient system with a smaller size of state space. Based on the abstraction method, a verification adaptation technique is developed with simulation relation checking algorithms to deal with possible online model changing. If the verification adaptation indicates the necessity to update the supervisor, supervisor adjustment methods are given. Altogether, it leads to a semi-online adaptation approach for system model changing. Examples are given to illustrate this framework.
Preprint
Full-text available
Cognitive systems are highly adaptable and flexible, such that action and perception capabilities can be achieved with the body in various ways, and incorporate features of the environment and nonbiological tools. Perceptual learning refers to enduring changes to a system’s ability to perceive and respond to environmental stimuli. Here I present an integrative framework for understanding how such capabilities occur in human-machine systems comprised of brain-body-tool-environment interactions. Central to this work is the claim that the capacity for high degrees of adaptation, flexibility, and learning are possible because human-machine systems are soft-assembled systems, that is, systems whose material constitution is not rigidly constrained so as to achieve goals via a variety of configurations. I begin by presenting the foundations of the framework on offer: the concepts, methods, and theories of ecological psychology, embodied cognition, dynamical systems theory, and machine intelligence. Next, I apply the framework to the case of visually-guided action. I conclude by explaining how this framework provides the explanatory and investigative tools to understand human-machine perceptual systems as soft-assembled systems that span brains-bodies-tools-environments.
Conference Paper
Full-text available
This paper presents some of the current research work in the field of mobile robotics rehabilitation technology. Specifically, it reports some projects currently working toward “intelligent” powered wheelchair prototypes. All these projects aim to assist physically and mentally handicapped persons to steer and control powered wheelchairs. RobChair, a project running in our Institute, is also described. Mainly we describe the part of the project concerned with the integration of local obstacle avoidance strategies used to ensure the safety of the user and to improve the wheelchair mobility
Article
Full-text available
This paper emphasizes the importance of assessing stability index for steering-controlled three-wheeled walkers. The paper describes a stability computation model that can be used to generate a reference input to the intelligent shared-control algorithm. The model can be used to evaluate the instantaneous stability margin of the walker-user system. This knowledge of the online stability of the walker will enable the shared controller to intelligently decide on the most appropriate time for the activation of the control to minimize the likelihood of jeopardizing the system stability as a result of the system's control actions, and possibly prevent falls due to control actions. The results of a stability computation model, based on the force-angle stability measure, are presented for different walker-assisted navigational scenarios including: walking straight, making soft turns, making sharp turns, and control fighting. The results showed that assisted steering enhanced the user's stability when the user's and the walker's intents were aligned. However, the results also indicated that a conflict between the user's intent and the walker's control actions (or walker's intent) in so called intelligent walkers could jeopardize the stability of the walker-user system, and hence that of the walker's user.
Article
Full-text available
This paper describes our experiences in collecting user data on human-robot interaction in nursing homes for the elderly. Learnings from two experiments were used to develop guidelines to support human-robot user studies with elderly users, in particular for experiments in an eldercare institution. Our experiences show that this demands a very strict organization, full cooperation by nursing personnel and extreme attention to informing the participants both before and during the experiment. Furthermore, first analysis of data from the studies suggests that social abilities in a robotic interface contribute to feeling comfortable talking to it and invite elders to be more expressive.
Article
Full-text available
Elderly and disabled people can experience consid-erable difficulties when driving an electric wheelchair, especially if they do not possess the fine steering capac-ities that are required to perform 'common' manoeu-vres, like avoiding obstacles, docking at tables, driv-ing through doors, etc. This paper describes a possi-ble approach to equip the wheelchair with an intelli-gent controller that performs low-level assistance, so wheelchair control is shared between the user and this controller. For proper operation, the controller should have a good idea of what the user wants in a particu-lar situation, or in other words what his/her intention could be. This text focuses on an 'implicit' estimation of the user's intention.
Conference Paper
Full-text available
Many elderly and disabled people today experience difficulties when manoeuvring an electric wheelchair. In order to help these people, several robotic assistance platforms have been devised in the past. In most cases, these platforms consist of separate assistance modes, and heuristic rules are used to automatically decide which assistance mode should be selected in each time step. As these decision rules are often hard-coded and do not take uncertainty regarding the user's intent into account, assistive actions may lead to confusion or even irritation if the user's actual plans do not correspond to the assistive system's behavior. In contrast to previous approaches, this paper presents a more user-centered approach for recognizing the intent of wheelchair drivers, which explicitly estimates the uncertainty on the user's intent. The paper shows the benefit of estimating this uncertainty using experimental results with our wheelchair platform Sharioto
Conference Paper
Full-text available
This paper presents a one step smooth and efficient path planning algorithm for navigating a large robotic platform in known cluttered environments. The proposed strategy, based on the generation of a novel search space, relies on non-uniform density sampling of the free areas to direct the computational resources to troubled and difficult regions, such as narrow passages, leaving the larger open spaces sparsely populated. A smoothing penalty is also associated to the nodes to encourage the generation of gentle paths along the middle of the empty spaces. Collision detection is carried out off-line during the creation of the configuration space to speed up the actual search for the path, which is done on-line. Results prove that the proposed approach considerably reduces the search space in a meaningful and practical manner, improving the computational cost of generating a path optimised for fine and smooth motion
Article
A robotic wheelchair system must be able to navigate outdoors as well as indoors. This paper describes the outdoor navigation system for our robotic wheelchair, Wheelesley, which uses a vision system to avoid obstacles and to stay centered on the current path. User tests were performed on 7 able-bodied subjects using single switch scanning on an outdoor course. The outdoor navigation system reduced user effort by 74% and reduced the time needed to navigate the test course by 20%. BACKGROUND The goal of this research is to provide people unable to drive a standard powered wheelchair with a mobility device that does not require the assistance of a caregiver. Robotic wheelchairs must be able to navigate in indoor and outdoor environments. A survey of powered and manual wheelchair users found that 56.6% used their wheelchair only outside, 33.3% used their wheelchair both inside and outside, and 10% used their wheelchair only inside (1). Most prior work on robotic wheelchairs has only addressed the problem of indoor navigation. This research project has developed a robotic wheelchair system that provides navigation assistance in indoor and outdoor environments, allowing its user to drive more easily and efficiently. Acoustic and vision based sensors are used to provide assistance. The wheelchair system is semi- autonomous, which takes advantage of the intelligence of the chair's user by allowing the user to plan the general route while taking over lower level control such as obstacle avoidance and centering on a path. The developed system has an easily customizable user interface that has been tested with eye tracking and with single switch scanning. This paper addresses the development and testing of the outdoor navigation system; for a full report on the wheelchair system, see (2). RESEARCH QUESTION Can assisted navigation in an outdoor environment improve driving performance when using single switch scanning as an access method? METHODS Sensors used in indoor environments, such as sonar and infrared, are not very effective in outdoor environments. Assistive navigation in an outdoor environment is accomplished using computer vision. We use a STH-V1 Stereo Head from Videre Designs mounted on the front of the wheelchair's tray to capture images of the world in front of the wheelchair. Disparities of points1 in the image are used to compute obstacle boundaries for obstacle avoidance. The navigation system also computes the location of the edges of the current path to provide path following. Obstacle avoidance takes priority over path following. The robot will only follow the command of the path following module if there are no obstacles detected. To detect obstacles, the disparity image is scanned horizontally and vertically, looking for changes in disparity between adjacent points that exceed a specified threshold, indicating a likely 1 Disparity measures the difference of the location of a point in the left and the right image of a stereo pair. Disparity is greater for closer objects and smaller for objects in the distance. You can experience this by looking ahead at a scene with some close and some far obstacles. Alternate closing your left and right eyes. Close objects will appear to move more from one image to the next than far objects do.
Article
As the origin of this work comes from the ar-eas of Artificial Intelligence and Cognitive Ro-botics we basically describe the sensorimotor ca-pabilities of an autonomous robot i. e. an auto-mated wheelchair. Beginning with the technical aspects of sensing the environment and extracting its essential features, we relate these techniques to the terminology of Cognitive Science. We con-tinue this practice in describing the way the robot represents its local neighbourhood as well as the global workspace. By describing algorithms for global localisation, navigation, and a natural lan-guage interface to the overall system, we conclude the description of our state of affairs in the devel-opment of a fully autonomous wheelchair.
Article
In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains. We begin by introducing the theory of Markov decision processes (mdps) and partially observable MDPs (pomdps). We then outline a novel algorithm for solving pomdps off line and show how, in some cases, a finite-memory controller can be extracted from the solution to a POMDP. We conclude with a discussion of how our approach relates to previous work, the complexity of finding exact solutions to pomdps, and of some possibilities for finding approximate solutions.