Page 1

Control of Sensory Perception for Discrete Event Systems

by

Geir E. Hovland

ISBN: 1-58112-070-2

DISSERTATION.COM

1999

Page 2

Copyright © 1999 Geir E. Hovland

All rights reserved.

ISBN: 1-58112-070-2

Dissertation.com

1999

www.dissertation.com/library/1120702a.htm

Page 3

Control of Sensory Perception for

Discrete Event Systems

Geir Edvin Hovland

Siv.Ing. (NTNU Norway)

August 1997

A thesis submitted for the degree of Doctor of Philosophy

of The Australian National University

Department of Engineering

Faculty of Engineering and Information Technology

The Australian National University

Page 4

Foreword

About the author

Geir Edvin Hovland was born in Stavanger, Norway, in 1970. He received his MSc

degree in Control Engineering from the Norwegian University of Science and Technol-

ogy, Trondheim, Norway in 1993. His MSc thesis was titled “Passivity based velocity

observer for robot control”. He received his PhD in 1998 from the Engineering Depart-

ment of the Australian National University, Canberra, Australia. Since 1997 he has

been a senior research scientist at ABB Corporate Research in Oslo, Norway and ABB

Robotics in V¨ aster˚ as, Sweden. His current interests include identification and control of

lightweight and elastic industrial manipulators, force-controlled industrial manipulation

and automation of assembly/disassembly. (Contact email: geir hovland@ieee.org).

Some of the reviewers comments on the thesis

This thesis is concerned with the application and development of discrete event systems

modeling to the problem of the control of sensory perception. The subject and approach

taken by the candidate is indeed novel and differs substantially in philosophy and intent

from previous approaches to this problem. The work is a significant contribution to

the general area of robotics and sensory control.

The notion of discrete sensing states and transitions between states is what distinguishes

the approach taken in this thesis from the more continuous-time, information-theoretic

methods. The thesis contains significant advances in the state of the art in on-line

monitoring of sensor-based robot actions. The emphasis is on monitoring of contact

changes and estimation of the contact situation after the transitions, during repetitive

execution.

2

Page 5

Page 6

Declaration

The work contained in this thesis, unless explicitly stated, is original research whose

major portion was done by the author. The work has not been submitted for a degree

at any other university or institution. The results presented in the thesis have been

published or submitted to journals and conferences as listed below.

Journal Papers:

[J1] G.E. Hovland and B.J. McCarragher, Control of Sensory Perception in Discrete

Event Systems Using Stochastic Dynamic Programming, ASME Journal of Dy-

namic Systems, Measurement and Control, Vol. 121, No. 2, June 1999.

[J2] G.E. Hovland and B.J. McCarragher, Control of Sensory Perception in a Mobile

Navigation Problem, The International Journal of Robotics Research, Vol. 18,

No. 2, Feb. 1999, pp. 201-212, ISSN: 0278-3649.

[J3] G.E. Hovland and B.J. McCarragher, Hidden Markov Models as a Process Mon-

itor in Robotic Assembly, The International Journal of Robotics Research, Vol.

17, No. 1, Feb. 1998, pp. 153-168, ISSN: 0278-3649.

Conference Papers:

[C1] G.E. Hovland and B.J. McCarragher, The Control of Sensory Perception for

Discrete Event Systems, Proceedings of the 1998 IEEE International Conference

on Systems, Man and Cybernetics (SMC’98), San Diego, USA, 11-14 October

1998, pp. 776-781.

[C2] G.E. Hovland and B.J. McCarragher, Controlling Sensory Perception for Indoor

Navigation, The 1998 IEEE International Conference on Robotics and Automa-

tion, Leuven, Belgium, 17-21 May 1998, pp. 2211-2216.

i

Page 7

Declarationii

[C3] G.E. Hovland and B.J. McCarragher, Sensitivity Analysis of a Sensory Perception

Controller, Proceedings of Control 97, Sydney, 20-22 October 1997.

[C4] H. Bruyninckx, G.E. Hovland and B.J. McCarragher, Robust Sensing for Force-

Controlled Assembly, Workshop Text for the IEEE/RSJ International Conference

on Intelligent Robots and Systems (IROS’97), Grenoble, 8-13 September 1997.

[C5] G.E. Hovland and B.J. McCarragher, Combining Force and Position Measure-

ments for the Monitoring of Robotic Assembly, Proceedings of the IEEE/RSJ

International Conference on Intelligent Robots and Systems (IROS’97), Greno-

ble, 8-13 September 1997.

[C6] G.E. Hovland and B.J. McCarragher, Dynamic Sensor Selection for Robotic Sys-

tems, Proceedings of the 1997 IEEE International Conference on Robotics and

Automation, Albuquerque, 20-25 April 1997, pp. 272-277.

[C7] G.E. Hovland and B.J. McCarragher, Control of Sensory Perception Using Stochas-

tic Dynamic Programming, Proceedings of the 1st Australian Data Fusion Sym-

posium, Adelaide, 21-23 November 1996, pp. 196-201.

[C8] G.E. Hovland and B.J. McCarragher, Frequency-Domain Force Measurements

for Discrete Event Contact Recognition, Proceedings of the 1996 IEEE Interna-

tional Conference on Robotics and Automation, Minneapolis, 22-28 April 1996,

pp. 1166-1171.

[C9] G.E. Hovland and B.J. McCarragher, State Transition Recognition in Robotic

Assembly Using Hidden Markov Models, Proceedings of the 1995 National Con-

ference of the Australian Robot Association, Melbourne, 5-7 July 1995, pp.75-86.

[C10] G.E. Hovland and B.J. McCarragher, A Hidden Markov Approach to the Mon-

itoring of Robotic Assembly, Proceedings of the 6th Irish DSP and Control Col-

loquium, Queen’s University Belfast, Belfast, 19-20 June 1995 (F. Gaston and G.

Dodds eds).

In addition to the above papers, the following papers whose contents do not directly

relate to the material in the thesis were published as joint work with other members of

the Automated Systems Laboratory Group in the Department of Engineering, ANU.

[E1] B.J. McCarragher, G. Hovland, P. Sikka, P. Aigner and D. Austin, Hybrid Dy-

namic Modelling and Control of Constrained Manipulation Systems, IEEE Robotics

and Automation Magazine, June 1997 Special Issue on Discrete Event Systems.

Page 8

Declarationiii

[E2] G.E. Hovland, P. Sikka and B.J. McCarragher, Skill Acquisition from Human

Demonstration Using a Hidden Markov Model, Proceedings of the 1996 IEEE

International Conference on Robotics and Automation, Minneapolis, 22-28 April

1996, pp. 2706-2711.

Canberra, August 1997.

Geir Edvin Hovland

Department of Engineering

Faculty of Engineering and Information Technology

The Australian National University

Canberra ACT 0200, AUSTRALIA.

Page 9

Acknowledgements

First I give my thanks to Brenan McCarragher for his contributions to the research

presented in this thesis. My colleagues Peter Aigner and David Austin were endless

sources of humour and enthusiasm. In the early stages of the project I was lucky to work

with Pavan Sikka and his friendship helped me to adapt to a new country and working

environment. I had the best possible sounding-board for my research and source of ideas

from former and current members of the Automated Systems Laboratory, in particular

Peter Aigner, Pudji Astuti, David Austin, Aidan Cahill, Werner Kraus, Paul Logothetis

and Pavan Sikka. Other people who made my stay in the department interesting and

enjoyable were Paul Compston, Peter Dower, Deshan Lu, Rajmohan Madhavan and

Andrew Matlakowski. A big thank you to you all.

I want to thank the members of my thesis committee, Jon Kieffer and Iven Mareels.

Also, I acknowledge the financial support from the Research Council of Norway which

allowed me to spend these rewarding years in Australia.

G.E.H

August 1997

iv

Page 10

Abstract

The problem of controlling sensory perception for use in discrete event feedback control

systems is addressed in this thesis. The sensory perception controller (SPC) is formu-

lated as a sequential Markov decision problem. The SPC has two main objectives; 1) to

collect perceptual information to identify discrete events with high levels of confidence

and 2) to keep the sensing costs low. Several event recognition techniques are available

where each of the event recognisers produces confidence levels of recognised events.

For a discrete event control system running in normal operation, the confidence levels

are typically large and only a few event recognisers are needed. Then, as the event

recognition becomes harder, the confidence levels will decrease and additional event

recognisers are utilised by the SPC. The final product is an intelligent architecture

with the ability to actively control the use of sensory input and perception to achieve

high performance discrete event recognition.

The discrete event control framework is chosen for several reasons. First, the theory of

discrete event systems is applicable to a wide range of systems. In particular, manufac-

turing, robotics, communication networks, transportation systems and logistic systems

all fall within the class of discrete event systems. Second, the dynamics of the sensing

signals used by the event recognisers are often strong and contain a large amount of

information at the occurrence of discrete events. Third, because of the discrete nature

of events, feedback information is not required continuously. Hence, valuable process-

ing time is available between events. Fourth, the discrete events are a natural common

representational format for the sensors. A common sensor format aids the decision

process when dealing with different sensor types. Fifth, the sensing aspect of discrete

event systems has often been neglected in the literature. In this thesis we present a

unique approach to on-line discrete event identification.

The thesis contains both theoretical results and demonstrated real-world applications.

The main theoretical contributions of the thesis are 1) the development of a sensory

v

Page 11

Abstractvi

perception controller for the dynamic real-time selection of event recognisers. The

proposed solution solves the Markov decision process using stochastic dynamic pro-

gramming (SDP). SDP guarantees cost-efficiency of the real-time SPC by solving a

sequential constrained optimisation problem. 2) A sensitivity analysis method for the

sensory perception controller has been developed by exploring the relationship between

Markov decision theory and linear programming. The sensitivity analysis aids in the

robust tuning of the SPC by finding low sensitivity areas for the controller parameters.

Two real-world applications are presented. First, several event recognition techniques

have been developed for a robotic assembly task. Robotic assembly fits particularly well

in the discrete event framework, where discrete events correspond to changes in contact

states between the workpiece and the environment. Force measurements in particular

contain a significant amount of information when the contact state changes. Second, the

sensory perception control theory and the sensitivity analysis have been demonstrated

for a mobile navigation problem. The cost-efficient use of sensory perception reduces

the need for mobile robots to carry heavy computational resources.

Page 12

Contents

Declaration i

Acknowledgements iv

Abstractv

Nomenclaturexi

1 Introduction1

1.1Motivation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .1

1.2 Contributions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .3

1.3 Organisation of the Thesis . . . . . . . . . . . . . . . . . . . . . . . . . .4

2 Formulation of the Sensory Perception Control Problem7

2.1 Discrete Event Formalism . . . . . . . . . . . . . . . . . . . . . . . . . .7

2.2 Literature Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .10

2.2.1 Open-Loop Sensor Planning Solutions . . . . . . . . . . . . . . . 10

2.2.2 On-Line Dynamic Sensor Selection . . . . . . . . . . . . . . . . . 11

2.3 Discounted Markov Decision Process . . . . . . . . . . . . . . . . . . . .15

3Stochastic Dynamic Programming Solution for Real-Time Implemen-

tation 18

3.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .18

3.2 Sensory Perception Controller . . . . . . . . . . . . . . . . . . . . . . . .20

vii

Page 13

Contentsviii

3.2.1Stochastic Dynamic Programming Model . . . . . . . . . . . . . 21

3.2.2Stochastic Dynamic Programming Algorithm . . . . . . . . . . .26

3.2.3Limitation on Total Cost. . . . . . . . . . . . . . . . . . . . . . 27

3.2.4 Data Fusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .29

3.2.5 Properties of the Sensory Perception Controller . . . . . . . . . .32

3.3 Sensory Perception Control Example . . . . . . . . . . . . . . . . . . . .33

3.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .38

3.4.1 Advantages . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .39

3.4.2 Limitations . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40

3.5Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .40

4Linear Programming Solution to Sensitivity Analysis 42

4.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .42

4.2 Linear Programming Solution . . . . . . . . . . . . . . . . . . . . . . . .43

4.2.1 Linear Programming Equations . . . . . . . . . . . . . . . . . . .43

4.2.2 Linear Program in Standard Form . . . . . . . . . . . . . . . . .45

4.3 Sensitivity Analysis . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .46

4.4 Sensitivity Analysis Example . . . . . . . . . . . . . . . . . . . . . . . .49

4.4.1Sensitivity of the Discount Factor α . . . . . . . . . . . . . . . .50

4.4.2Sensitivity of the State Reward Constant K . . . . . . . . . . . .52

4.4.3 Sensitivity of the State Rewards r(si,a1) . . . . . . . . . . . . . . 54

4.4.4Sensitivity of the Monitor Rewards li. . . . . . . . . . . . . . . . 55

4.5 Discussion and Conclusion . . . . . . . . . . . . . . . . . . . . . . . . . .56

5 Case Study I: Control of Sensory Perception in Robotic Assembly 58

5.1Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

5.2Literature Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

5.3Discrete Event Model of Robotic Assembly. . . . . . . . . . . . . . . .61

Page 14

Contents ix

5.4 Discrete Event Control Architecture . . . . . . . . . . . . . . . . . . . .63

5.5 Process Monitors . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .65

5.5.1 Geometric Modelling . . . . . . . . . . . . . . . . . . . . . . . . .65

5.5.2Event Detection . . . . . . . . . . . . . . . . . . . . . . . . . . .67

5.6 Experiments . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .67

5.7 Discussion and Conclusions . . . . . . . . . . . . . . . . . . . . . . . . .74

6 Case Study II: Control of Sensory Perception in Mobile Navigation76

6.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .76

6.2 Literature Survey . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .77

6.3 Discrete Event Model of Navigation Problem . . . . . . . . . . . . . . .79

6.4 Process Monitoring Techniques . . . . . . . . . . . . . . . . . . . . . . .82

6.5 Experiments on Sensory Perception Control . . . . . . . . . . . . . . . .83

6.5.1 Experiments with Closed Doors . . . . . . . . . . . . . . . . . . .86

6.5.2Experiments with Open Doors . . . . . . . . . . . . . . . . . . .88

6.6 Experiments on Sensitivity Analysis . . . . . . . . . . . . . . . . . . . .91

6.6.1 Sensitivity of the Discount Factor α . . . . . . . . . . . . . . . .91

6.6.2Sensitivity of the State Reward Constant K . . . . . . . . . . . .92

6.7Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 94

7Conclusions and Further Research 95

7.1Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .95

7.2 Major Conclusions and Results . . . . . . . . . . . . . . . . . . . . . . .95

7.3Further Research . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . .97

7.3.1Imperfect Event Recognition . . . . . . . . . . . . . . . . . . . .97

7.3.2Extension to More Complex Tasks . . . . . . . . . . . . . . . . .98

7.3.3Continuous Control of Sensory Perception . . . . . . . . . . . . . 99

7.3.4 Analytical Research . . . . . . . . . . . . . . . . . . . . . . . . . 99

Page 15

Contentsx

7.3.5Parallel Decision Making. . . . . . . . . . . . . . . . . . . . . . 99

References101

A Process Monitoring Using Hidden Markov Models 108

A.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 108

A.2 Observation Symbols . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 110

A.3 Length of Observation Window . . . . . . . . . . . . . . . . . . . . . . . 112

A.4 Training . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 113

A.5 Event Recognition . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 116

A.6 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

B Process Monitoring Using Distance Functions 119

B.1 Distance Functions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

B.2 Classification of Contact States . . . . . . . . . . . . . . . . . . . . . . . 120

B.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 122

C Process Monitoring Using a Multilayer Perceptron123

C.1 Multilayer Perceptron Network . . . . . . . . . . . . . . . . . . . . . . . 123

C.2 Training Algorithm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124

C.3 Real-Time Operation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125

C.4 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 126

D Process Monitoring Using Qualitative Reasoning128

D.1 Qualitative Reasoning . . . . . . . . . . . . . . . . . . . . . . . . . . . . 128

D.2 Confidence Levels . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 129

D.3 Evaluation . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 130

Page 16

Nomenclature

Acronyms

DEC

DES

DF

HMM

LP

MLP

QR

SDP

SPC

: Discrete Event Controller

: Discrete Event System

: Distance Functions

: Hidden Markov Model

: Linear Programming

: Multilayer Perceptron

: Qualitative Reasoning

: Stochastic Dynamic Programming

: Sensory Perception Controller

Symbols

A

ak∈ A

an(z,si)

b

c

C

ei

e∗(n)

(Fx,Fy,Mz)

(∆Fx,∆Fy)

K

ki

li

N

Nm

Pn,ij(z,a)

: LP constraint matrix in standard form

: Markov action space

: Optimal action function

: LP constraint vector in standard form

: LP cost vector

: Confidence level

: Discrete event

: Recognised event by the SPC after the nth action

: Planar force/torque measurements

: Change of planar force measurements

: SDP state reward constant

: Individual monitor costs

: Individual monitor rewards

: Submatrix of A corresponding to non-basic LP variables

: Number of available process monitors

: SDP state transition probability

xi

Page 17

Nomenclature xii

(Px,Py,θ)

Rn(z,si,ak)

r(si,a1)

si∈ S

Vn(z,si)

x

x∗

xB

xN

z

z∗

: Planar position measurements

: Markov reward function

: Markov state rewards

: Markov state space

: Optimal value function

: LP state vector in standard form

: Optimal basic feasible LP solution

: LP basic variable vector

: LP non-basic variable vector

: Permutation of process monitors

: Optimal permutation of process monitors

α

γi

γ∗

κ

λ(si,ak)

µ

σ(n)

τ

φ

Φ

ψ

: Discount factor

: State describing DES

: Final recognised DES state

: Total accumulated SDP cost

: LP state variables

: Confidence level discretisation function

: State of SPC after the nth action

: Map from discrete events to DES state

: Edges of geometric model

: Sensitivity analysis parameter

: Surfaces of geometric model

Page 18

Chapter 1

Introduction

1.1 Motivation

Perceptual capabilities are often the main bottleneck for successful operation of dis-

crete event autonomous systems. As the degree of system uncertainty and ambiguity

increases, one sensor alone may not provide sufficient information. Research in the

area of multi-sensor fusion has increased the robustness of systems operating in uncer-

tain environments. However, these solutions often require high computational power

by utilising all the sensors continuously. As the tasks become more complex and more

numerous, the number of relevant features of the environment quickly exceeds the sens-

ing and processing resources that are feasible to supply to real-world control systems.

Hence, there is an increasing need for controlling sensory perception. An intelligent

sensory perception system is able to perform time-constrained on-line analysis of au-

tonomous systems. Typically, in normal operation only a few sensors are needed. Then,

as the quality of the sensing decreases, more sensors are utilised. The design of a sen-

sory perception controller is an important contribution to the development of discrete

event autonomous systems operating in uncertain and unpredictable environments.

The control of sensory perception fits particularly well in the discrete event frame-

work. The traditional approach to the control of sensory perception in feedback con-

trol systems requires heavy continuous computations during execution. The continuous

computations often outweigh the actual sensing costs and the practical value of such

solutions is limited. In the discrete event formalism, the control of sensory perception

is only needed at the occurrence of the events. Hence, valuable processing time is

available between the occurrence of events for other components of the discrete event

1

Page 19

1.1. Motivation 2

control architecture.

A wide range of different sensor types have been used in autonomous control systems.

The sensory perception models and sensor selection strategies often depend on the

specifics of the sensor types used in each application. Hence, transferring these methods

to other systems with different sensor types is often difficult. For example, active vision

research has focussed on planning using vision sensors, such as cameras, range finders

and illuminators. A host of other sensors such as tactile sensors, three-dimensional

range sensors, force-torque sensors and acoustic sensors are currently being used in, for

example, robotics and manufacturing applications. Tarabanis, Allen and Tsai (1995)

surveyed the area of vision-based systems and write: Further work needs to be done to

properly integrate these sensors and their unique constraints into the overall planning

system.

One advantage of the discrete event framework is the common representational format

provided for different sensor types. The natural common format is the discrete events

occurring during execution. The general sensory perception control theory presented

in this thesis is applicable to a wide range of sensor types in different discrete event

control systems. In particular, the discrete event framework has proven successful in

manufacturing problems, robotic assembly, mobile robot navigation, communication

networks, transportation systems and logistic systems.

The control of discrete event systems has received a significant amount of attention in

recent years, see for example Baccelli, et.al. (1992), Cassandras (1993), Cassandras,

Lafortune and Olsder (1995), Rubinstein and Shapiro (1993). Traditionally, perfect

sensing of discrete events has been assumed. In other work, for example Kumar and

Garg (1995),¨Ozveren and Willsky (1990), the requirement of perfect event recognition

is reduced to a sub-set of all possible discrete events. State ambiguities are allowed

to develop, but these must be resolvable after a bounded interval of events. For most

practical systems the assumption of perfect event sensing is unrealistic. To increase

applicability of discrete event theory, there is a need for dealing with the sensing aspects

of discrete events. Discrete event identification is one of the main difficulties with

interfacing continuous-time systems with discrete event controllers.

In this thesis we present a new and unique approach to discrete event identification. A

sensory perception controller actively selects different event monitors to identify discrete

events as they occur with high levels of confidence and low sensing costs. The method

improves the applicability of discrete event theory by increasing the event recognition

rates compared to single sensor systems. The total sensing costs are low compared to

Page 20

1.2. Contributions3

multi-sensor fusion techniques where all the sensors are used for every event occurrence.

Hence, the proposed method is well suited to event identification in real-time feedback

control systems operating in uncertain environments.

1.2 Contributions

The thesis contributes in the following areas:

i. The sensory perception controller (SPC) is formulated as a sequential Markov

decision problem in a discrete event feedback control system. The Markov for-

mulation allows for fast-real time solutions and provides a facility for sensitivity

analysis. The discrete event formalism offers advantages in providing a com-

mon representational format for different sensor types, highlighting sensing at

the occurrence of discrete events where the signal dynamics are often strong and

reducing the sensing costs by avoiding continuous control of sensory perception.

ii. A fast real-time SPC algorithm has been developed solving the Markov decision

process using stochastic dynamic programming (SDP). SDP is used to determine

the optimal sequence of sensing strategies. The SDP algorithm provides a cost-

efficient solution to the sensory perception control problem. A low-cost SPC is

required to improve the performance compared to existing multi-sensor fusion

algorithms.

iii. A thorough sensitivity analysis of the controller has been developed. The sensitiv-

ity analysis aids in the robust tuning of the SPC by finding low sensitivity areas

for the model parameters. The sensitivity analysis is performed by exploring the

relationship between discounted Markov decision problems and linear program-

ming. Linear programming allows for a thorough sensitivity analysis without

having to solve the problem from scratch for each new model parameter.

iv. The last decade has seen little effort in applying Markov decision theory to practi-

cal systems. In this thesis the sensory perception controller has been successfully

implemented in two applications; robotic assembly and mobile navigation. The

cost of sensing is reduced compared to multi-sensor systems where all the sensors

are used all the time. The experiments also demonstrate that the sensory percep-

tion controller achieves higher event recognition rates than any individual sensor.

Moreover, the experiments show how the sensing system is able to recover from

undetected discrete events.

Page 21

1.3. Organisation of the Thesis 4

v. The applications required the development and implementation of several pro-

cess monitoring techniques. The appendices present new and unique techniques

for discrete event recognition based on force/torque and position measurements.

Hidden Markov Models (HMMs) have proven to be a powerful tool for discrete

event recognition. The method 1) allows for dynamic force/torque measurements,

2) accounts for sensor noise and friction and 3) exploits the fact that the amount

of force information is large at the occurrence of the events.

A monitor using position measurements has been developed. The position based

monitor is sensitive to geometrical model uncertainties, but still adds useful infor-

mation to the sensory perception controller. The main advantage of the method

is its computational efficiency.

A multilayer perceptron (MLP) network using both force/torque and position

measurements for event recognition has been implemented. One advantage of

this method compared to the other solutions presented is the fact that it models

both dynamic and static behaviour. The MLP is able to achieve relatively large

successful event recognition rates compared to other methods.

1.3 Organisation of the Thesis

The thesis contains both theoretical results and two demonstrated real-world applica-

tions. It is comprised of the following chapters.

Chapter 2 formulates the problem of sensory perception control. The discrete event

framework is used and the sensory perception controller is formulated as an ac-

tive element of a feedback control structure. The chapter contains a literature

survey of research relevant to the sensory perception problem and concludes with

a formulation of the SPC as a sequential Markov decision process.

Chapter 3 presents the stochastic dynamic programming solution to the Markov deci-

sion process. The solution provides a fast algorithm for use in real-time feedback

control systems. Several different data fusion methods are incorporated and a

sensory perception control example is given. The chapter ends with a discussion

of some advantages and limitations of the proposed method.

Chapter 4 examines the sensitivity analysis of the controller developed in Chapter 3.

Due to the iterative nature of dynamic programming, a direct sensitivity approach

Page 22

1.3. Organisation of the Thesis 5

is difficult. The sensitivity analysis is performed by exploring the relationship

between discounted Markov decision problems and linear programming.

Chapter 5 demonstrates the control of sensory perception for a planar robotic assem-

bly task. The assembly task is modelled as a constrained motion system and

the discrete events correspond to changes in the motion constraints. The con-

trol of sensory perception is demonstrated using four different event recognition

techniques.

Chapter 6 demonstrates the control of sensory perception in mobile navigation. The

navigation problem is modelled as a discrete event system, where the discrete

events correspond to changes in the mobile unit motion constraints. The sensory

perception problem is demonstrated for a planar model containing three rooms,

fixed walls and open or closed doors.

Chapter 7 brings the conclusions of the thesis. Open problems and areas for further

research are discussed.

The appendices present several process monitors which allow us to demonstrate

the control of sensory perception for robotic assembly and mobile navigation in

Chapter 5 and 6.

Appendix A presents a process monitor based on Hidden Markov Models (HMMs).

Each discrete event is modelled by a HMM which represents a stochastic,

knowledge-based system. The HMMs are trained off-line on planar force/torque

measurements. In real-time operation all HMMs corresponding to possible events

are evaluated. The event with the highest model score is chosen and the associ-

ated confidence level is also calculated from the HMM model scores.

Appendix B presents a process monitor based on position measurements and distance

functions. The monitor is based on a geometrical model of the world and calcu-

lates the nearest distances between all relevant surfaces and edges. The monitor

recognises the world states and calculates confidence levels based on the distance

functions.

Appendix C describes a multilayer perceptron process monitor.

trained off-line on force/torque and position measurements. Each network output

corresponds to a discrete event. In real-time operation the measured forces/torques

and positions are used as the network inputs. The event with the highest corre-

sponding network output is chosen and the associated confidence level is calcu-

lated from the network outputs.

The network is

Page 23

1.3. Organisation of the Thesis6

Appendix D describes a process monitor based on qualitative reasoning. The control

of sensory perception requires each monitor to produce confidence level outputs

of the recognised discrete events. The monitor based on qualitative reasoning was

developed by McCarragher and Asada (1993). The original contribution in this

appendix is the incorporation of confidence levels to the process monitor.

Page 24

Chapter 2

Formulation of the Sensory

Perception Control Problem

2.1 Discrete Event Formalism

The sensory perception control problem addressed in this thesis is formulated in the

discrete event control framework. Ramadge and Wonham (1989) defined a discrete

event system (DES) as a dynamic system that evolves in accordance with the abrupt

occurrence, at possibly unknown irregular intervals, of physical events.

γ1

γ2

γ3

γ4

γ5

γ6

γ7

γ8

γ(t)

Time

Events

e1

e2

e3

e4

e5

e6

e7

e8

t1

t2

t3

t4

t5

t6

t7

t8

Figure 2.1: Sample path of a discrete event system.

7

Page 25

2.1. Discrete Event Formalism8

Figure 2.1 shows a sample path of a discrete event system. The DES state variable,

denoted γ(t), is piece-wise constant and can only change value at the occurrence of a

discrete event. The sensory perception controller (SPC) has two main objectives; 1) to

collect perceptual information to identify discrete events with high levels of confidence

and 2) to keep the sensing costs low. The general sensory perception control problem

for discrete event systems is formulated as follows.

Problem Statement:

tors such as to minimise the cost related to the event recognition error plus the costs of

obtaining the monitor outputs, ie. find the optimal V

Given the occurrence of a discrete event, consult event moni-

V = minf(e∗− e) +

?

i∈I

g(ki)(2.1)

The term?g(ki) represents the cost of obtaining the monitor outputs, where kiare the

individual monitor costs. The individual monitor costs are usually fixed and determined

off-line. Only the event monitors actually consulted by the SPC are contained in the set

I. The term f(e∗−e) represents the cost related to the event recognition error, where

e∗is the recognised event from the SPC. In general, the correct event e is unknown.

Hence, the event recognition error e∗−e is also unknown. In this thesis, each individual

event monitor produces a confidence level C ∈ [0,1] which is used to estimate the event

recognition error. A large confidence level indicates a low recognition error. A trade-

off has to be made between the event recognition errors and the monitor costs. Low

monitor costs often result in large average event recognition errors, while large monitor

costs often result in low average event recognition errors.

Figure 2.2 shows the block diagram of the discrete event control structure. The percep-

tual capabilities of the discrete event system consist of several process monitors. Process

monitor i recognises a discrete event ei(tk) occurring at time tk. Due to noisy measure-

ments, model uncertainties or world unpredictability, the recognised event ei(tk) may

not correspond to the actual physical event e(tk) that occurred. A very important fea-

ture of a process monitor is its ability to indicate the confidence level of the recognised

event. A good process monitor produces low confidence levels for events recognised

incorrectly and large confidence levels for events recognised correctly.

When a discrete event occurs, the sensory perception controller (SPC) has the option

of consulting any of the process monitors. The SPC has two main objectives. First,

the SPC must use the recognised events ei(tk) and the corresponding confidence levels

Ci(tk) efficiently to correctly recognise the actual discrete events. Even when some of