Conference PaperPDF Available

Wheelchair Driver Assistance and Intention Prediction Using POMDPs

Authors:
  • Dubai Futue Labs
Conference Paper

Wheelchair Driver Assistance and Intention Prediction Using POMDPs

Abstract and Figures

Electric wheelchairs give otherwise immobile people the free- dom of movement, they significantly increase independence and dramatically increase quality of life. However the physical control systems of such wheelchair can be prohibitive for some users; for example, people with severe tremors. Sev- eral assisted wheelchair platforms have been developed in the past to assist such users. Algorithms that assist specific behaviors such as door − passing, f ollow − corridor, or avoid − obstacles have been successful. Recent research has seen a move towards systems that predict the users intentions, based on the users input. These predictions have been typically limited to locations immediately surrounding the wheelchair. This paper presents a new assisted wheelchair driving system with large scale intelligent intention recognition based on POMDPs (Partially Observable Markov Decision Processes). The systems acts as an intelligent agent/decision-maker, it relies on minimal user input; to predict the users intention and then autonomously drives the user to his destination. The prediction is constantly being updated as new user input is received allowing for true user/system integration. This shifts the users focus from fine motor-skilled control to coarse control intended to convey intention.
Content may be subject to copyright.
Wheelchair Driver Assistance and Intention
Prediction using POMDPs
# Tarek Taha, Jaime Valls Mir´
o and Gamini Dissanayake
ARC Centre of Excellence for Autonomous Systems
Mechatronics and Intelligent Systems Group
University of Technology Sydney
NSW2007, Australia
{t.taha, j.vallsmiro, g.dissanayake}@cas.edu.au
Abstract
Electric wheelchairs give otherwise immobile people the free-
dom of movement, they significantly increase independence
and dramatically increase quality of life. However the physical
control systems of such wheelchair can be prohibitive for
some users; for example, people with severe tremors. Sev-
eral assisted wheelchair platforms have been developed in
the past to assist such users. Algorithms that assist specific
behaviors such as door passing,follow corridor, or
avoid obstacles have been successful. Recent research has
seen a move towards systems that predict the users intentions,
based on the users input. These predictions have been typically
limited to locations immediately surrounding the wheelchair.
This paper presents a new assisted wheelchair driving system
with large scale intelligent intention recognition based on
POMDPs (Partially Observable Markov Decision Processes).
The systems acts as an intelligent agent/decision-maker, it
relies on minimal user input; to predict the users intention
and then autonomously drives the user to his destination.
The prediction is constantly being updated as new user input
is received allowing for true user/system integration. This
shifts the users focus from fine motor-skilled control to coarse
control intended to convey intention.
1. INTRODUCTION
The world’s aging population and the large number of people
effected by motor disabilities has motivated past researchers
to develop assistive technologies in the hope of improving
the quality of life of those affected. Systems such as robotic
walkers [1], smart blind sticks [2] and robotic wheelchairs [3]–
[8] have been developed with this goal in mind. Out of these,
robotic electric wheelchairs are particularly desirable as they
greatly benefit the most needy users but problems exist when
they are used manually. For instance, depending on the users
type of disability; safely and effectively driving the wheelchair
may be difficult in general. Furthermore, such wheelchairs
are also large in comparison to the passageways typical in
indoors environments such as offices, nursing homes, hospitals
and houses. This means that for some users tasks such as
passing through a doorway or navigating a hallway may be
quite challenging.
For those who experience these considerable difficulties
when driving an electric wheelchair, an intelligent system
designed to assist driving would increase their mobility and
increase their quality of life. This intelligent system must
perform the fine motion control required in the presence
of obstacles or in tight spaces. Further to this, the system
should be intelligent enough to understand the users intentions
and to comply with them whilst demanding the minimal
input from the user. The interaction must be transparent to
the user and should rely on the least possible number of
communication devices. In this paper we are proposing a
strategy to accomplish this.
2. RELATED WOR K
If a truly user-machine integrated system is to be developed,
the type of cooperation between user and machine must be
comparable to the cooperation between a horse and its rider
[9]; the rider navigates, while the horse avoids (small) dan-
gerous obstacles on the ground. To achieve this level of user-
machine integration the machine must grow and learn with the
user so that a relationship may form (such as that between a
horse and its rider) and so that the machine can predict the
users intention and autonomously enact the intention with only
minimal corrective input from the user.
The fundamental component of this relationship is intention
recognition. The primary considerations for an intention recog-
nition system are whether an accurate representation/model of
the environment is required and whether the system is going
to be reactive or deliberative. Reactive refers to systems that
do not use a representation of the environment and therefore
are usually weak in decision making and long term prediction
as they rely on local or temporal information collected online
which might not be sufficient to make correct long term plans.
These reactive systems rely on directly reacting to the changes
in the real world by linking the perception obtained by their
sensors to their effectors; the control doesn’t comply to a
model but simply happens as a low level response to the
perception. Systems with limited resources like processing
power, memory and communication mediums often use a
reactive system but the scope of their possible applications
and intelligence is limited. On the other hand, having a rep-
resentation of the environment and processing the perception
through an internal model reduces the ability of the robot to
act quickly in new environments.
Several wheelchair platforms that help people in their
daily navigation tasks have been previously developed: Rol-
land III [3], Bremen Autonomous Wheelchair, Sharioto [4],
RobChair [5], Senario [6], VAHM [7], Wheelesley [8], and
Navchair [10]. The majority of these algorithms are reactive
control algorithms limited by either sets of operating modes
that the user must select (manual, fully autonomous or semi-
autonomous) or by the set of states they try to predict in a
local scale. The drawback of such systems is their limitation
to manual mode selection and the limited scope of their
navigation algorithms and intention recognition.
In the last few years some wheelchair assistive reactive
techniques have emerged based on capturing the users local
intentions and assisting them with a limited set of motion states
or tasks like avoiding obstacles, following a wall, entering a
room, going to an object or a local location of interest. These
algorithms are based on systems that can act intelligently
but not think intelligently. In other word, they try to make
the link between perception and action as direct as possible
by combining decisions made by both the human and the
machine [9], [10]. Even though such techniques appear to
work well in predicting the user’s intentions in a local scale
(same room or same open space) these algorithms lack the
intelligence to autonomously recognize the user’s places of
interest; rather, the usually visited places/locations must be
manually specified prior to system operation. They also lack
the larger scope of assistance that would help the user to
achieve specific goal such as going to the bathroom or to
the kitchen using his wheelchair. Disabled users require an
intelligent system that can help achieve such tasks with as
little feedback as possible whilst allowing them to remain in
complete control of the system.
Unlike previous prediction methods, this paper presents
a larger scale intention recognition and goal prediction as-
sistance strategy that uses environmental knowledge to plan
and interact deliberately. Typically, users requiring wheelchair
assistance have a known set of target locations that they go to
during their daily activities: locations like the bathroom, the
kitchen or the T.V. room. Through monitoring a wheelchair
user through his daily routine/activities the system presented
in this paper determines the locations of interest that the use
regularly frequents and builds knowledge about these locations
using machine learning techniques. This knowledge is then
used to predict the users intended destination and to assist
the user in driving the wheelchair to this destination. The
wheelchair is considered an intelligent agent with an internal
representation of the environment and the intention prediction,
derived via a POMDPs model of the system, is translated
into a planning method in the presence of uncertainties. The
intelligent controller then performs low-level navigation tasks
such as local path planning, collision avoidance and actuating
the fine motion control of the wheelchair to assist the user in
reaching his goal position. Whilst in motion the user performs
high level planning by providing action/course correcting
feedback to the system through the intention recognition
algorithm. This correcting feedback can be inputted through
different kinds of interfaces (e.g. a traditional joystick or a
voice interface) and is not necessarily a continuous input as
with shared autonomy.
3. INTRODUCTION TO POMDPS
Partially Observable Markov Decision Processes (POMDP)
provides a general framework for sequential decision making
in environments where states are hidden and actions are
stochastic. POMDPs were recently used in assistive technolo-
gies and they proved to be successfull and applicable to wide
range of assistive applications [11], [12]. A POMDP model
represents the dynamics of the environment, such as the prob-
abilistic outcomes of the actions (the transition function T), the
reward function R, and the probabilistic relationships between
the agents observations and the states of the environment(the
observation function O). In POMDP the model and the states
of the environment are typically referred to as “hidden states”,
since they are not directly observable by the agent. The
POMDP framework is a systematic approach that uses belief
states to represent memory of past actions and observations.
Thus, it enables an agent to compute optimal policies using
the MDP framework. A POMDP model is defined by a six
tuple < S, A, T, R, γ, Z, O >:
S: A set of states that represents the state of the system
at each point in time.
A: A set of actions that an agent can take ( can depend
on the current state).
T:A×S×S[0,1]: The state transition function, which
maps each state action pair into a probability distribution
over the state space. The next distribution over the state
space depends only on the current state action pair and
not on the previous state action pairs. This requirement
ensures the the Markovian property of the process. We
write T(s, a, s0)for the probability that an agent took
action afrom state sand reached state s0.
R:S×A→ < : The immediate reward function which
indicates the reward for doing an action in some state.
Z: A set of observations.
γ: A discount factor used to reduce the award given to
far future steps.
O:A×S×Z[0,1]: A function that maps the action at
time t-1 and the state at time tto a distribution over the
observation set. We write O(s0, a, z )for the probability
of making observation zgiven that the agent took action
aand landed in state s0.
The belief is a sufficient statistic for a given history and it’s
update at each time step is represented by Equation 1 where
P r(o|a, b)is a normalizing constant [13], [14].
b0(s0) = O(s0, a, o)PsST(s, a, s0)b(s)
P r(o|a, b)(1)
Fig. 1: A POMDP agent is made up of two main components. The state
estimator module receives observations from the environment, the last action
taken and the previous belief state and produces an updated belief state. The
policy module maps belief state to an action.
The goal of POMDP is to find a sequence of actions
a0, ...., at that maximizes the expected sum of rewards
E[PtγtR(st, at)]. Since the states are not fully observable,
the goal is to maximize the expected reward for each be-
lief [15]. The value function is calculated using the Bellman
Equation 2 and the optimal policy can be obtained using
Equation 3.
Considering our intention recognition problem as a planning
problem, the wheelchair is transformed into an agent and a
decision maker that needs to drive the wheelchair user to his
destination based on a certain optimal policy (plan). The agent
estimates the state of the environment as the input and generate
an action based on that as depicted in Figure 1.
V(s) = maxa[R(s, a) + γX
s0S
T(s, a, s0)V(s0)] (2)
π
t=argmaxa[R(s, a) + γX
s0S
T(s, a, s0)V
t1(s0)] (3)
4. POMDP PROBLE M SPECIFI CATI ON
Using the POMDP framework, our prediction is transferred
into a planning problem in which we are trying to find
the best plan that represents the user’s intention by re-
ducing the uncertainty in the belief state, categorized by
the Destination we are trying to reach. In our POMDP
formulation, the wheelchair moves stochastically according
to a fixed policy. The state space is described by the
cross product of two features, the W heelchairLocation =
{s1, ..., sx}and the Destination ={d1, ..., dy}resulting
in a StateSpace ={s1d1, s2d1, ..., sxdy}. The wheelchair
starts from a known postion and the plan finishes when
the W heelchairLocation is the same as the Destination.
The W heelchair can have one of the following ac-
tions: {N orth, South, East, W est, DoNothing }indicating
the global direction of travel. A reward of -1 is given for each
motion step and +100 reward is given when the Wheel chair
performs an action that leads to Destination. Throughout
Fig. 2: The POMDP model generation architecture. The map topology
together with the training data are used to determine the transition model. The
training data is also used to determine the observation model of the POMDP.
User’s Joystick calibration determines the uncertainty in the observations.
navigation, the W heelchairLocation is fully observable but
not the destination, and the effect of an action has a predictable
deterministic effect as the example described in Equation 4:
P r(W heelchair =Sx|W heelchair =S y, South) = 1 (4)
The position of the Destination is completely unobserv-
able until the wheelchair reaches its destination. At each
state the joystick input is observed and is represented by a
set of discrete states = {U p, Down, Right, Lef t, N oInput},
uncertainty in the user’s input is taken into consideration while
generating the observation model and will be explained in
section 5-C.
5. POMDP GEN ERATION
To obtain a efficient POMDP system, we need to have
proper T ransition,Observation and StateSpace models.
Our model generation consists of three major parts as depicted
in Figure 2. These three steps will be explained in the
subsections below:
A. State Space
In our assistive system, we want the user to be able to navigate
in a high level topological manner. This means that the user
should be focusing on driving the wheelchair from one room to
another, or from one spatial location to another without having
to worry about the intermediate steps that comes in between
(planning wise). In order for us to do so, only significant
spatial feature are considered, such as a hallway intersection,
a door opening or a room.
The ability to learn tasks and represent environments [16],
[17] is essential in our application as it creates the bases for
the long term intention recognition and prediction. This is
done by simplifying the encapsulation of spatial and activity
information. For this to happen, the wheelchair should have the
ability to represent the spatial information of the environment
in a simplistic topological manner that can make it easier to
store, extract and update information.
TABL E 1: LIS T OF TASKS RECORDED FROM THE USERS ACTIVITIES
Start End Path
Task1 Lab Office 26/D - 25/L - 24/L - 22/D - 23/N
Task2 Office Meeting 42/U - 40/L - 43/U - 44/N
Task3 Office Bathroom 3/D - 4/L - 5/D - 6/N
For our POMDP platform, the state space consists of
two features: the W heelchairLocation and the intended
Destination. The cross product of the above two feature will
form the StateSpace ={s1d1, s2d1, ..., sxdy}, these features
are separately extracted in two different steps described below:
1) Spatial States: The spatial representation we are us-
ing is based on the topological graph representation of the
environment, where vertices are locations in the environment
and edges represent a viable path connecting two locations
as a result of performing an action. In our research we are
mainly targeting indoor office or home environments. For such
environments there has been a lot of research done on how to
build maps and extract topological representation accurately.
For simplicity, we assume that the maps are already available
and that the topological map representation is hand coded and
programmed. It might be more convenient in the future to
consider a complete system that can build maps and extract
topological representations simultaneously but this is out of
the scope of the current research. The map topology will
be represented by a graphical tree of nodes and connections
(segments), where the set of nodes W heelchairLocation =
{s1, ..., sx}represents a location in the map and the connec-
tion represents a physical path that connects two locations.
The hand coded spatial configuration of the domain used for
planning illustrated in Figure 3.
2) Destinations States: Identifying places of interest is not
an easy task and there is no direct method to achieve this as
it is an application and environment dependent problem. For
the prediction problem we are trying to solve, it’s sufficient
to think about the place of interest as a spatial location in
the environment where the user spends significantly most
of his/her time. After observing the user’s activities we can
determine the time that the user need to stay in the same place
for it to be considered as a place of interest. In general staying
few minutes in a certain location can nominate that location
to the place of interest set. For POMDP model generation
purposes we log the activities of the user over a period of
time, then in that log we determine the locations of interest
Destination ={d1, ..., dy}based on the above criteria.
B. Transition Model
Transition model specifies the translation from one state
to another given a certain action T(s, a, s0). In our model
specifications, the actions are global navigation commands
{N orth, South, East, W est, Stop}and determines the spa-
tial node that we will end up at if we are in location sand
executed the action a. The transition mode is built directly
from the map topology. This transition is deterministic and
independent of the intention, so regardless where we want
to go. The result of executing an action in the same loca-
Fig. 4: The POMDP driver assistance architecture. The user’s input together
with the current location generate an observation that helps in updating the
belief in the destination. An appropriate action will be executed based on that
belief driving the wheelchair to the next destination.
tion will be the same. For example T(s3d1, North, s2d1) =
T(s3d2, N orth, s2d2) = 1 in the Figure 3.
C. Observation Model
The observation model defines the probability of observing
zgiven that the wheelchair took an action aand landed in
state s0O(s0, a, z). To generate a proper observation model
that correctly models the user’s intention and activities, we
use a training data from that particular user. In an indoor
environment, the wheelchair users usually perform a repetitive
set of tasks that represents navigating from one place to
another. A task can be for example going from the living
room to the bathroom or to the kitchen. This set of tasks
can be defined by the user himself or extracted from a set
of data recorded by monitoring the user’s activities. The tasks
are defined by a starting location, intermediate locations, end
location and the joystick inputs/observation that the user gave
at each location as described in Table 1 where the path is
represented by numbers corresponding to the states’ numbers
and the letters corresponding to the observation in each state
(L=Left, R=Right, U=Up and D=Down).
The user in many cases might be unable to give a proper
joystick input due to disability or a disease causing a shaky
hand for example. To best customize the POMDP model
for this user, a joystick calibration is required to determine
the uncertainties in the user’s inputs. This uncertainty will
be a set of nprobabilities describing the user’s inability
to give the right joystick input, where nis the number of
JoystickInputs ={U p, Down, Right, Lef t, NoInput}.
Having obtained the training data and the uncertainty, the
observation model is then generated by adding the uncertainty
to the frequency probability (the probability of obtaining a
certain observation in a state).
6. ONLINE ASSI STANCE
Once the planning problem is formulated, we solve the
POMDP to get the optimal policy π. While predicting online,
we start with an initial belief state bt. Since we know our
currently location from our localizer, the initial belief is limited
Fig. 3: The map topology used for our intention recognition. Circles represent intersections and cannot be a destination while squares represent rooms or
open spaces and can be considered as a possible destination. The numbers represent the state number and is used to build the transition model. Gray shaded
rectangles represent learned destinations.
to those states in the StateSpace with our current location,
and we will end up with a belief set size equivalent to the
available destinations. For example, if our Destination =
Kitchen, B athroom, T.V Room and we know where we are,
then our initial belief is distributed among these destinations
and is equal to 1/3. Based on our initial belief, we execute
the optimal policy action for that belief state π(st), calculate
the reward rtfor taking that action, get an observation zt+1
and update our belief bt+1, then repeat the procedure. This is
described in Procedure 1 and illustrated in Figure 4.
Procedure 1 Online Navigation
1. Initial belief: bt.
2. Execute the action from the optimal policy: π(st).
3. Calculate the reward: rt.
4. Get an observation : zt+1.
5. Update the belief: bt+1.
6. Repeat until destination reached.
7. EVAL UATIO N IN A SIMUL ATED ENVIRONMENT
To validate the proposed intention recognition architecture we
simulated a training data that represents the activities of a user
in the environment shown in Figure 3. The destinations are
represented by the gray shaded squares and they form the set
destinations ={s1d1, s6d2, s26d3, s30d4, s31d5, s38d6}.
The POMDP was generated using a simulated training data
with uncertainty added to the observations to represent the
user’s ability to control the joystick (in this example uncer-
tainty on Up=10%, Down=5%, Right=15%, Left=10% and
Nothing=20%). The generated POMDP problem was then
solved using zmdpSolver [18] and the optimal policy was
obtained.
The generated policy and model were tested against the
tasks in the training data. For each task in the training data
we start with a known location (the first state in the task) but
unknown destination (equal belief among destinations) then
we take observations from that task one by one, updated the
belief based on the takes observation, select an action based
on the optimal policy and execute that action to get to the
next state. This procedure is repeated until we reach the end
of the observations in the task. If the end state reached after
the last observation is the same as the intended destination
(the last state in the task), then the test for this particular
test is considered successful, otherwise it fails. The test was
successful in all of the 289 tasks in this experiment producing
a 100% success rate.
An example of a single task experiment is shown in Fig-
ure 5. The wheelchair starts in spatial state s42 in the map
shown in Figure 3 and tries to predict where the user is trying
to go based on his/her inputs. Initially the user’s destination
can be any of the six destinations extracted from the training
data, then after the first observation, our belief is now higher
in 5 of those destinations. At state s29, the obtained “Right”
observation increases the belief that we are going to state s31
or s38 and reduces that of going to state s30. At state s32, the
system is very confident that the user is going to state s38.
Fig. 5: The result of a simulated experiment. The wheelchair starts in state 42 and tries to predict where the user is going to based on his joystick inputs
(observations). The wheelchair in this case successfully takes the user’s joystick inputs and decides on the correct actions that take the user to state 38.
8. CONCLUSION
In this paper we have presented a new method for wheelchair
assistance that considers the wheelchair as a smart robotic
agent, interacting with the user with the aid of a sequential
decision making algorithm (POMDP). Unlike most of the
currently available assistive methods that are based on semi-
autonomous systems which merge wheelchair’s perception and
user’s control with some added heuristics, our method tries to
predict where the wheelchair’s user is trying to go, and takes
him there without any extra mode or behavioral selection.
POMDP was chosen because it provides an good platform
for planning and predicting under uncertainty for human-robot
interaction, as we have shown in this paper. The results we
obtained so far are promising and represent a realistic outcome
for the smart assistance technology we are trying to develop.
The methodology is currently being implemented on a real
wheelchair platform, and future efforts are being devoted to
design a more sophisticated “activity extrator” strategy.
ACKNOWLEDGMENTS
This work is supported by the Australian Research Council
(ARC) through its Centre of Excellence programme, and by
the New South Wales State Government. The ARC Centre of
Excellence for Autonomous Systems (CAS) is a partnership
between the University of Technology Sydney, the University
of Sydney and the University of New South Wales.
REFERENCES
[1] M. Alwan, P. J. Rajendran, A. Ledoux, C. Huang, G. Wasson, P. Sheth,
“Stability Margin Monitoring in Steering-Controlled Intelligent Walkers
for the Elderly”, AAAI Symposium, November 2005.
[2] S.J. Kang, Y. Ho, I. H. Moon, “Development of an intelligent guide-
stick for the blind”, IEEE International Conference on Robotics and
Automation, 2001, Vol.4, pp. 3208-3213.
[3] C. Mandel, K. Huebner, T. Vierhuff, “Towards an Autonomous
Wheelchair: Cognitive Aspects in Service Robotics”, In Proceedings
of Towards Autonomous Robotic Systems, 2005, pp. 165-172.
[4] D. Vanhooydonck, E. Demeester, M. Nuttin, H. Van Brussel, “Shared
Control for Intelligent Wheelchairs: an Implicit Estimation of the User
Intention”, Proceedings of the ASER ’03 1st International Workshop on
Advances in Service Robotics, Bardolino Italy, 13-15 March 2003.
[5] G. Pires, R. Araujo, U. Nunes, A. T. de Almeida, “RobChair-a powered
wheelchair using a behaviour-based navigation”, International Work-
shop on Advanced Motion Control, Coimbra., 29 June - 1 July 1998,
pp. 536-541.
[6] P. Beattie, J. M. Bishop, “Localization of the SENARIO wheelchair”,
Mobile Robotics Technology for Health Care Services, Proceedings of
the 1st MobiNet Symposium, Athens Greece, pp. 287-293.
[7] A. Pruski, M. Ennaji, Y. Morere, “VAHM: a user adapted intelligent
wheelchair”, International Conference on Control Applications, Vol.2,
Iss., 2002, pp. 784-789.
[8] A. Yanco Holly, “Development and Testing of a Robotic Wheelchair
System for Outdoor Navigation”, In Proceedings of the 2001 Conference
of the Rehabilitation Engineering and Assistive Technology Society of
North America, RESNA Press, 2001.
[9] K. A. Tahboub, “Intelligent Human-Machine Interaction Based on Dy-
namic Bayesian Networks Probabilistic Intention Recognition”, Journal
of Intelligent and Robotic Systems, Volume 45, Number 1, pp. 31-52.
[10] E. Demeester, A. Huntemann, D. Vanhooydonck, G. Vanacker, A.
Degeest, H. V. Brussel, M. Nuttin, “Bayesian Estimation of Wheelchair
Driver Intents: Modeling Intents as Geometric Paths Tracked by the
Driver”, International IEEE/RSJ International Conference on Intelligent
Robots and Systems, Beijing China, October 2006.
[11] E. L´
opez, L. M. Bergasa, R. Barea y M. Escudero, “A navigation system
for assistant robots using visually augmented POMDPs”, Autonomous
Robots, vol. 19, no. 1, July 2005.
[12] J. Hoey, A. V. Bertoldi, P. Poupart, A. Mihailidis, “Assisting Per-
sons with Dementia during Handwashing Using a Partially Observable
Markov Decision Process”, In Proceedings of the International Confer-
ence on Vision Systems, Biefeld Germany, 2007.
[13] L. P. Kaelbling, M. L. Littman, A. R. Cassandra, “Planning and Acting
in Partially Observable Stochastic Domains”, Artificial Intelligence,1998
, Vol. 101, pp. 99-134.
[14] J. Pineau, G. Gordon, “POMDP Planning for Robust Robot Control”.
International Symposium on Robotics Research (ISRR), San Francisco
USA, 2005.
[15] J. Pineau, G. Gordon, S. Thrun, “Point-based value iteration: An anytime
algorithm for POMDPs”, International Joint Conference on Artificial
Intelligence (IJCAI), August 2003, pp. 1025-1032.
[16] T. Spexard, S. Li, B. Wrede, J. Fritsch, G. Sagerer, O. Booij, Z. Zivkovic,
B. Terwijn,B. Kr´
ose, “Biron, where are you? - enabling a robot to learn
new places in a real home environment by integrating spoken dialog and
visual localization, In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots
and Systems. IEEE, October 2006.
[17] M. Heerink, B. J. A. Kr´
ose, B. J. Wielinga, V. Evers, “Human-robot
user studies in eldercare: Lessons learned”, In Proc. Int. Conf. on Smart
Homes and Health Telematics, Belfast Northern Ireland, June 2006.
[18] T. Smith,“zmdpSolver”, [http://www.cs.cmu.edu/˜trey/zmdp/].
... In recent years, both national and international scholars have begun to research the influence of driver's physiological and psychological factors on traffic safety. Taha [3] recognized that operator's intentions typically followed a Markov Decision Process and applied the results to the development of a new assistant wheelchair driving system. Kamaruddin et al. [4]studied the recognition of emotion in a driver's voice while driving and established an emotional space model that can be utilized in vehicle active safety warning systems. ...
... The blue line represents the measured value, magenta line (simulation 1) represents the results of the simple psychological test method, red line (simulation 2) represents the results of the obvious prediction method, and the cyan line (simulation 3)represents the results of the model developed in this paper Simulation results of headway. The blue line represents the measured value, magenta line (simulation 1) represents the results of the simple psychological test method, red line (simulation 2) represents the results of the obvious prediction method, and the cyan line (simulation3) represents the results of the model developed in this paper Simulation results of acceleration. The blue line represents the measured value, magenta line (simulation 1) represents the results of the simple psychological test method, red line (simulation 2) represents the results of the obvious prediction method, and the cyan line (simulation3) represents the results of the model developed in this paper ...
... The blue line represents the measured value, magenta line (simulation 1) represents the results of the simple psychological test method, red line (simulation 2) represents the results of the obvious prediction method, and the cyan line (simulation3) represents the results of the model developed in this paper Simulation results of acceleration. The blue line represents the measured value, magenta line (simulation 1) represents the results of the simple psychological test method, red line (simulation 2) represents the results of the obvious prediction method, and the cyan line (simulation3) represents the results of the model developed in this paper ...
Article
Full-text available
Driver's propensity describes the physiological and psychological states of a vehicle driver and can be used as a dynamic measurement of driver characteristics and an important index for determining factors that may affect driver characteristics such as emotional state and decision preference. Drivers with different propensities behave differently under the same traffic situations; therefore, it is an important concept for studying traffic flow theory and developing new vehicle warning systems. In this study we focused on the prediction of driver's propensity, and an overall consideration of the physiological and psychological indexes which influence the prediction were investigated. Psychological measurements, simulated driving, and vehicle tests were conducted and a prediction model for driver's propensity was established. Validation results showed that the method in this paper was a better predictor of driver's propensity than previous methods, and the new model provides a basis for further study of dynamic characteristics and recognition of driver's propensity.
... POMDP model is intensively used by the researchers in artificial intelligence, machine learning and computer engineering. The applications include, quality controlling in a production system (Ben-Zvi and Grosfeld-Nir, 2013;Grosfeld-Nir, 2007), robot navigation (Simmons and Koenig, 1995), aiding disabled people (Taha et al., 2007;Hoey et al., 2010). Littman (2009) has given a brief tutorial of POMDP for behavioural scientists. ...
... 1.4. STATE OF ART AND THE PRESENT THESISquality controlling in a production system(Ben-Zvi and Grosfeld-Nir, 2013;Grosfeld-Nir, 2007), robot navigation(Simmons and Koenig, 1995), aiding disabled people(Taha et al., 2007;Hoey et al., 2010).Littman (2009) has given a brief tutorial of POMDP for behavioural scientists. Zhang (2011) has used POMDP for investigating the optimal cancer screening policies. ...
... Research related to wheelchair path planning can be classified into three categories: the shared control approach, local path planning to avoid collision, and global path planning to a destination or waypoint. In previous studies where shared control is applied, estimating the user's intention is most important because global path planning is conducted by the user; the wheelchair only assists in local path planning [6]- [11]. In [9], [10], the user model was constructed as a Bayesian model using the user's driving data. ...
Article
Full-text available
A novel path planning algorithm for an autonomous electric wheelchair that can be used in hospitals is proposed and verified. The objective of this study is to develop an autonomous path planning algorithm that satisfies body acceleration constraints for patient safety and follows paths suitable for hospitals. The wheelchair dynamics and equations of motion, including caster dynamics and the user’s influence on the wheelchair, are newly derived in this research. Since the proposed algorithm is based on two-wheel robot kinematics, it is applicable to most two-wheel actuated systems. When a wheelchair’s destination is set, a series of waypoints is computed and then the desired speed & attitude at each waypoint of the path are designed. Then, a speed profile is designed to minimize the travel time while adaptively changing the angular rate gain to minimize excessive lateral motion. As a result, the linear speed command of each actuated wheel is computed. Additionally, the hybrid reciprocal velocity obstacle algorithm is updated to guarantee that the body acceleration constraints are satisfied for applications with complicated dynamics. The performance of the proposed algorithm, with the full dynamics of the wheelchair, is demonstrated by extensive numerical simulations with conventional toy problems and then evaluated in two hospitals. The simulation results demonstrate that the proposed algorithm satisfies body acceleration constraints while following paths which are suitable for hospitals and that the proposed autonomous electric wheelchair could be applied in hospitals.
... In [58,59], for instance, an intelligent decision making agent is presented for driver's intention detection in uncertain local environments based on the Partially Observable Markov Decision Process (POMDP). Using the same methodology, a global intention recognition model is also presented in [60] for autonomous wheelchair navigation. Besides, a multihypothesis approach is considered in [61,62] to predict the driver's intention and provide collaborative control, by adjusting the steering signal to avoid observable risks during navigation. ...
Article
Full-text available
In spite of the presence of powered wheelchairs, some of the users still experience steering challenges and manoeuvring difficulties that limit their capacity of navigating effectively. For such users, steering support and assistive systems may be very necessary. To appreciate the assistance, there is need that the assistive control is adaptable to the user’s steering behaviour. This paper contributes to wheelchair steering improvement by modelling the steering behaviour of powered wheelchair users, for integration into the control system. More precisely, the modelling is based on the improved Directed Potential Field (DPF) method for trajectory planning. The method has facilitated the formulation of a simple behaviour model that is also linear in parameters. To obtain the steering data for parameter identification, seven individuals participated in driving the wheelchair in different virtual worlds on the augmented platform. The obtained data facilitated the estimation of user parameters, using the ordinary least square method, with satisfactory regression analysis results.
... In this study, only two Autonomous Navigation platforms are involved, and it is assumed that a natural hierarchy exists between them [13]. By hierarchy, we mean that one platform (the leader) makes its decision and then advertises this decision to the other platform (the follower) [14][15][16]. One platform is clearly designed to act as the leader when a motion deadlock is detected [17,18]. ...
... When a human is part of two agent team, most applications focus on the decision problem. For the wheelchair collaborative control applications, the intention of the driver is predicted based on the navigation context given by the on-board sensory-based systems [9][10][11][12]. By estimating the wheelchair user intention, an appropriate navigation mode is selected. ...
... Autonomous navigation plans and executes a path to the user-defined location; a cited example of an autonomous navigating smart-wheelchair is TetraNauta [Balcells1998]. A more recent example [Taha2007] based on Partially Observable Markov Decision Processes (POMDPs) attempts to predict the wheelchair user's intention and navigates to where the user is trying to go without any other information from its user. Autonomous operating smartwheelchairs however, are usually unable to navigate unknown areas or adjust for unplanned obstacles. ...
Conference Paper
Human intention recognition is one of the most important research areas in the human-robot interaction in order to accomplish more natural and intelligent interaction. In the public service area, the basic four human intentional actions, `approach', `depart', `bypass', and `stop' were investigated by the human's movement pattern. In order to recognize those four human intentional actions, the mobile robot measured 360 degree distance between a robot and a human by two IR-scanner sensors, and three problems were solved: 1) human detection and tracking by improved K-means clustering method, 2) absolute human velocity estimation by extended Kalman filter, and 3) inferring intentional actions by HMM with position-dependent observation model. The result showed as quite good performance that inferred the probabilities of those four intentional actions.
Conference Paper
Full-text available
This paper presents some of the current research work in the field of mobile robotics rehabilitation technology. Specifically, it reports some projects currently working toward “intelligent” powered wheelchair prototypes. All these projects aim to assist physically and mentally handicapped persons to steer and control powered wheelchairs. RobChair, a project running in our Institute, is also described. Mainly we describe the part of the project concerned with the integration of local obstacle avoidance strategies used to ensure the safety of the user and to improve the wheelchair mobility
Article
Full-text available
This paper emphasizes the importance of assessing stability index for steering-controlled three-wheeled walkers. The paper describes a stability computation model that can be used to generate a reference input to the intelligent shared-control algorithm. The model can be used to evaluate the instantaneous stability margin of the walker-user system. This knowledge of the online stability of the walker will enable the shared controller to intelligently decide on the most appropriate time for the activation of the control to minimize the likelihood of jeopardizing the system stability as a result of the system's control actions, and possibly prevent falls due to control actions. The results of a stability computation model, based on the force-angle stability measure, are presented for different walker-assisted navigational scenarios including: walking straight, making soft turns, making sharp turns, and control fighting. The results showed that assisted steering enhanced the user's stability when the user's and the walker's intents were aligned. However, the results also indicated that a conflict between the user's intent and the walker's control actions (or walker's intent) in so called intelligent walkers could jeopardize the stability of the walker-user system, and hence that of the walker's user.
Article
Full-text available
This paper describes our experiences in collecting user data on human-robot interaction in nursing homes for the elderly. Learnings from two experiments were used to develop guidelines to support human-robot user studies with elderly users, in particular for experiments in an eldercare institution. Our experiences show that this demands a very strict organization, full cooperation by nursing personnel and extreme attention to informing the participants both before and during the experiment. Furthermore, first analysis of data from the studies suggests that social abilities in a robotic interface contribute to feeling comfortable talking to it and invite elders to be more expressive.
Article
Full-text available
Elderly and disabled people can experience consid-erable difficulties when driving an electric wheelchair, especially if they do not possess the fine steering capac-ities that are required to perform 'common' manoeu-vres, like avoiding obstacles, docking at tables, driv-ing through doors, etc. This paper describes a possi-ble approach to equip the wheelchair with an intelli-gent controller that performs low-level assistance, so wheelchair control is shared between the user and this controller. For proper operation, the controller should have a good idea of what the user wants in a particu-lar situation, or in other words what his/her intention could be. This text focuses on an 'implicit' estimation of the user's intention.
Conference Paper
Full-text available
Many elderly and disabled people today experience difficulties when manoeuvring an electric wheelchair. In order to help these people, several robotic assistance platforms have been devised in the past. In most cases, these platforms consist of separate assistance modes, and heuristic rules are used to automatically decide which assistance mode should be selected in each time step. As these decision rules are often hard-coded and do not take uncertainty regarding the user's intent into account, assistive actions may lead to confusion or even irritation if the user's actual plans do not correspond to the assistive system's behavior. In contrast to previous approaches, this paper presents a more user-centered approach for recognizing the intent of wheelchair drivers, which explicitly estimates the uncertainty on the user's intent. The paper shows the benefit of estimating this uncertainty using experimental results with our wheelchair platform Sharioto
Article
A robotic wheelchair system must be able to navigate outdoors as well as indoors. This paper describes the outdoor navigation system for our robotic wheelchair, Wheelesley, which uses a vision system to avoid obstacles and to stay centered on the current path. User tests were performed on 7 able-bodied subjects using single switch scanning on an outdoor course. The outdoor navigation system reduced user effort by 74% and reduced the time needed to navigate the test course by 20%. BACKGROUND The goal of this research is to provide people unable to drive a standard powered wheelchair with a mobility device that does not require the assistance of a caregiver. Robotic wheelchairs must be able to navigate in indoor and outdoor environments. A survey of powered and manual wheelchair users found that 56.6% used their wheelchair only outside, 33.3% used their wheelchair both inside and outside, and 10% used their wheelchair only inside (1). Most prior work on robotic wheelchairs has only addressed the problem of indoor navigation. This research project has developed a robotic wheelchair system that provides navigation assistance in indoor and outdoor environments, allowing its user to drive more easily and efficiently. Acoustic and vision based sensors are used to provide assistance. The wheelchair system is semi- autonomous, which takes advantage of the intelligence of the chair's user by allowing the user to plan the general route while taking over lower level control such as obstacle avoidance and centering on a path. The developed system has an easily customizable user interface that has been tested with eye tracking and with single switch scanning. This paper addresses the development and testing of the outdoor navigation system; for a full report on the wheelchair system, see (2). RESEARCH QUESTION Can assisted navigation in an outdoor environment improve driving performance when using single switch scanning as an access method? METHODS Sensors used in indoor environments, such as sonar and infrared, are not very effective in outdoor environments. Assistive navigation in an outdoor environment is accomplished using computer vision. We use a STH-V1 Stereo Head from Videre Designs mounted on the front of the wheelchair's tray to capture images of the world in front of the wheelchair. Disparities of points1 in the image are used to compute obstacle boundaries for obstacle avoidance. The navigation system also computes the location of the edges of the current path to provide path following. Obstacle avoidance takes priority over path following. The robot will only follow the command of the path following module if there are no obstacles detected. To detect obstacles, the disparity image is scanned horizontally and vertically, looking for changes in disparity between adjacent points that exceed a specified threshold, indicating a likely 1 Disparity measures the difference of the location of a point in the left and the right image of a stereo pair. Disparity is greater for closer objects and smaller for objects in the distance. You can experience this by looking ahead at a scene with some close and some far obstacles. Alternate closing your left and right eyes. Close objects will appear to move more from one image to the next than far objects do.
Article
As the origin of this work comes from the ar-eas of Artificial Intelligence and Cognitive Ro-botics we basically describe the sensorimotor ca-pabilities of an autonomous robot i. e. an auto-mated wheelchair. Beginning with the technical aspects of sensing the environment and extracting its essential features, we relate these techniques to the terminology of Cognitive Science. We con-tinue this practice in describing the way the robot represents its local neighbourhood as well as the global workspace. By describing algorithms for global localisation, navigation, and a natural lan-guage interface to the overall system, we conclude the description of our state of affairs in the devel-opment of a fully autonomous wheelchair.
Article
In this paper, we bring techniques from operations research to bear on the problem of choosing optimal actions in partially observable stochastic domains. We begin by introducing the theory of Markov decision processes (mdps) and partially observable MDPs (pomdps). We then outline a novel algorithm for solving pomdps off line and show how, in some cases, a finite-memory controller can be extracted from the solution to a POMDP. We conclude with a discussion of how our approach relates to previous work, the complexity of finding exact solutions to pomdps, and of some possibilities for finding approximate solutions.
Conference Paper
The goal of this research is the creation of a complete robotic wheelchair system to be used by people unable to drive standard powered wheelchairs. A complete robotic wheelchair system must be able to navigate in-door and outdoor environments and should switch au-tomatically between navigation modes. For the system to be useful, it must be easily customized for the specific access methods required for each user. This abstract fo-cuses on the vision system being developed for outdoor navigation and the method for selecting whether to use the indoor navigation mode or the outdoor navigation mode. A report of the indoor navigation mode and user interface can be found in (Yanco In press). Navigation is divided into two classes: indoor and outdoor. In both navigation modes, the user gives a high-level command ("forward," "left," "right," and "backward") through the graphical user interface which has been customized to accommodate the user's access method. The system carries out the user's command using common sense constraints such as obstacle avoid-ance. Since the user must be able to successfully navi-gate novel environments immediately, the system does not use maps or environment modification. The navigation problem in an indoor environment is simplified by regularities such as level floors and walls. A vision system can be used in an indoor environment, but it is not necessary. Robots navigating indoors can use sonar and infrared sensors to avoid obstacles. Outdoor navigation is not feasible using only sonar and infrared sensors. To facilitate navigation in outdoor environments, a vision system with one camera is be-ing added to the wheelchair. Initial image processing is aimed at finding and navigating sidewalks, crosswalks, curb cuts and handicapped ramps. To stay on sidewalks and in crosswalks, the vision system will detect long straight lines that head towards the vanishing point. Ground-plane constraint or optical flow using succes-sive frames will be used to approximate the distance of