Conference Paper

A demonstration of a conversationally guided smart wheelchair

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

There is a substantial population of wheelchair users who do not have the motor capabilities needed to efficiently operate a power wheelchair on their own. Various interfaces have been devised including some simple voice controlled chairs that can understand simple commands. However, such systems are awkward and slow to use. This demonstration shows operation of a smart wheelchair through a spoken conversational interface. By using a more capable dialogue, rather than a simple command paradigm, the chair can leverage off of the user's perceptual capabilities in order to process natural, high-level commands such as take me to the desk, which initiates a conversation with the chair to determine which desk and -if it is not immediately detected by the chair's sensors - where the desk is located.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... The wheelchair presented by Hockey and Miller appears to have been used as a proof-of-concept demonstration within a limited domain [22]. As such, it has not been empirically evaluated, and there are scarce details about how it works algorithmically. ...
... Finally, NL-enabled wheelchairs should accept commands to go to objects and locations they have not already visited; a feature exhibited only by Duvallet et al. [26], Hockey and Miller [22], and ourselves [32], and must use belief modeling and episodic memory for the better resolution of ambiguous references. ...
... Wheelchairs able to issue NL commands not requiring perception The next group of wheelchairs are those that can navigate relative to local environmental features such as walls and elevators. All projects in this category ([22,23,24]; 16-18 inTable 6) use powered wheelchair bases controllable by joystick. In addition, the wheelchair presented by Pineau et al. can be controlled by a touch screen[23]. ...
Article
Full-text available
Natural language is a flexible and powerful control modality which can transform a wheelchair from a vehicle into a genuine helper. While autonomous wheelchairs are increasingly designed to use natural language for control, most of them only handle a small number of rigid commands. To establish the state-of-the-art in language-enabled wheelchairs and determine how to improve natural language capabilities, we introduce a framework for analyzing and classifying properties of language-enabled wheelchairs. We then apply the framework to the twenty-four most recent natural language-enabled wheelchair projects, in order to compare their achievements and identify areas for improvement.
... While such NL-enabled wheelchairs have existed for nearly forty years (e.g., [10]), even many of the most recently presented NLenabled wheelchairs have only limited capabilities, e.g., the ability to be commanded to go forward, left, right, backwards and to stop [2,3,6,19,20,22,23,31,32,34,38,40,39,44,45]. There have, however, been a small number of recent wheelchairs also capable of following walls [30], entering elevators [25], traveling to nearby objects [14], or traveling to named objects and locations [13,24,26,29,33,42]. ...
Conference Paper
Full-text available
Vulcan and DIARC are two robot architectures with very different capabilities: Vulcan uses rich spatial representations to facilitate navigation capabilities in real-world, campus-like environments, while DIARC uses high-level cognitive representations to facilitate human-like tasking through natural language. In this work, we show how the integration of Vulcan and DIARC enables not only the capabilities of the two individual architectures, but new syner-gistic capabilities as well, as each architecture leverages the strengths of the other. This integration presents interesting challenges, as DIARC and Vulcan are implemented in distinct multi-agent system middlewares. Accordingly, a second major contribution of this paper is the Vulcan-ADE Development Environment (VADE): a novel multi-agent system framework comprised of both (1) software agents belonging to a single robot architecture and implemented in a single multi-agent system middleware, and (2) " Dual-Citizen " agents that belong to both robot archi-tectures and that use elements of both multi-agent system middlewares. As one example application, we demonstrate the implementation of the new joint architecture and novel multi-agent system framework on a robotic wheelchair, and show how this integration advances the state-of-the-art for NL-enabled wheelchairs.
... Introduction aux personnes handicapées (voir (Hockey et Miller, 2007) qui présentent le pilotage d'un fauteuil électrique par le biais d'un dialogue oral entre l'utilisateur et la machine) ou aux personnes âgées (voir l'adaptation du système d'information LET'S GO ! aux personnes âgées (Raux et al., 2003)). ...
Article
Full-text available
Spoken dialog systems enable users to interact with computer systems via natural dialogs, as they would with human beings. These systems are deployed into a wide range of application fields from commercial services to tutorial or information services. However, the communication skills of such systems are bounded by their spoken language understanding abilities. Our work focus on the spoken language understanding module which links the automatic speech recognition module and the dialog manager. From the user's utterance analysis, the spoken language understanding module derives a representation of its semantic content upon which the dialog manager can decide the next best action to perform. The system we propose introduces a stochastic approach based on Dynamic Bayesian Networks (DBNs) for spoken language understanding. DBN-based models allow to infer and then to compose semantic frame-based tree structures from speech transcriptions. First, we developed a semantic knowledge source covering the domain of our experimental corpus (MEDIA, a French corpus for tourism information and hotel booking). The semantic frames were designed according to the FrameNet paradigm and a hand-craft rule-based approach was used to derive the seed annotated training data.Then, to derive automatically the frame meaning representations, we propose a system based on a two decoding step process using DBNs : first basic concepts are derived from the user's utterance transcriptions, then inferences are made on sequential semantic frame structures, considering all the available previous annotation levels. The inference process extracts all possible sub-trees according to lower level information and composes the hypothesized branches into a single utterance-span tree. The composition step investigates two different algorithms : a heuristic minimizing the size and the weight of the tree ; a context-sensitive decision process based on support vector machines for detecting the relations between the hypothesized frames. This work investigates a stochastic process for generating and composing semantic frames using DBNs. The proposed approach offers a convenient way to automatically derive semantic annotations of speech utterances based on a complete frame hierarchical structure. Experimental results, obtained on the MEDIA dialog corpus, show that the system is able to supply the dialog manager with a rich and thorough representation of the user's request semantics
... De fait, la parole est généralement utilisée comme modalité d'entrée, ou pour des applications à petit vocabulaire, comme le pilotage de fauteuil électrique. Dans certains cas, le pilotage du fauteuil peut même conduire à un minidialogue oral entre l'utilisateur et la machine (Hockey et Miller, 2007). ...
Article
Full-text available
This paper presents a survey of the state of the art in NLP techniques dedicated to alternative and augmentative communication for disabled people. We show that these techniques have mostly evolved during the two last decades from knowledge based approaches to data oriented approaches. Evaluation results give an indication on the efficiency of these prediction techniques as well as on the real aid provided to the user. As a conclusion, we show that the question of text prediction concerns on the whole any text entry
Article
Power wheelchair users often use and carry multiple mobile computing devices. Many wheelchair users experience motor impairments that affect their hands, arms, neck, and head. Additionally, a power wheelchair user's ability to interact with computing technology may be physically restricted by the wheelchair's frame, which can obstruct movement or limit reach. We believe these accessibility issues can be overcome by designing new systems to take advantage of the physical abilities of the user while leveraging the wheelchair's frame to create assistive systems. We also hope to make progress toward limiting social barriers that hinder technology adoption. This could potentially improve the overall benefit of emerging technologies for people with varying abilities. With this research I aim to develop an interactive wheelchair-based computing platform designed to support the unique and dynamic needs of power wheelchair users while also creating new interaction possibilities.
Conference Paper
Full-text available
Much robotics research is carried out using either PICs and processors that are a decade or more out of date The alternative is custom built electronics that is expensive and/or must be reinvented every time a new project is begun. The XBC is a new design for a robot controller merging a modern ARM processor with an FPGA that allows high performance - especially in vision processing and motor control - for a cost similar to controllers with a fraction of its capabilities. Additionally, the XBC uses a new, and still free, software development system, already in wide use. The XBC is being mass produced (at least in research hardware terms) so it is readily available and does not require computer hardware or electronics skills in order to be obtained. This paper describes the system, its capabilities and some potential applications.