ArticlePDF Available

A simple and robust hierarchical control system for a walking robot

Authors:

Abstract and Figures

The present work applies a control architecture proposed by W.T. Powers [3, 4], to several problems in robotics, and suggests that it may have wide practical applicability. The architecture is called (Hierarchical) Perceptual
Content may be subject to copyright.
A SIMPLE AND ROBUST HIERARCHICAL CONTROL SYSTEM FOR A WALKING ROBOT
J R Kennaway
University of East Anglia, U.K.
INTRODUCTION
The present work applies a control architecture proposed
by W.T. Powers [3, 4], to several problems in robotics,
and suggests that it may have wide practical applicabil-
ity. The architecture is called (Hierarchical) Perceptual
Control Theory, or HPCT, and was proposed by Powers
as a possible organisation for living control systems.
EXAMPLE: THE INVERTED PENDULUM
We shall introduce this approach by way of a simple ex-
ample, designed by Powers. Consider the inverted pendu-
lum shown in Figure 1. We assume that the cart travels on
Figure 1: An inverted pendulum
a frictionless track, the rigid pendulum rod swivels freely
at its base, and that there is an actuator which applies any
specified horizontal force to the cart. To move the pen-
dulum bob to a specified horizontal position by means of
this actuator is a complicated task. Nevertheless, it can be
achieved by breaking the matter down into simpler tasks,
as follows.
If we had an actuator that could set the bob immediately
to any desired position, no control system would be nec-
essary. We don’t have such an actuator; but if we had
one which could set the pendulum’s horizontal velocity,
we could use this to control the position: set the veloc-
ity equal to k0(rbb)for some constant k0, where rbis
the demanded position and bis the current position. We
don’t have such a velocity actuator, but if we had an actu-
ator that set the bob’s acceleration, we could control the
velocity ˙
bto approach a reference value r˙
bby applying
an acceleration k1(r˙
b˙
b). The acceleration is propor-
tional to the pendulum angle, which is proportional to
o=bc, where cis the position of the cart. So we can
set the acceleration by setting o. We cannot set odirectly,
but we could control oif we could set the cart’s velocity,
by setting ˙c=k2(rcc), where rcis the reference cart
position. We cannot set ˙cdirectly, but we could control
it if we could set the cart’s acceleration: ¨c=k3(r˙c˙c),
where r˙cis the demanded cart velocity. Finally, we can
set the cart’s acceleration by applying a force to the cart,
which by hypothesis we are able to do.
The resulting arrangement of four proportional con-
trollers is shown in Figure 2. For suitably chosen values
of the gain parameters, it is found to work very stably
and robustly (although it is not able to swing the pendu-
lum up from the straight down position). Although the
construction has been described above on the assumption
of linearity, which fails when the pendulum angle departs
too far from the vertical, the non-linearities are controlled
against in the same way as external disturbances. The
physical simulation (from which Figure 1 is a screen shot)
uses the true differential equations, valid for all pendulum
angles.
Figure 2: Control hierarchy for inverted pendulum
ANALYSIS OF A 2-LEVEL CONTROLLER
As an example of the mathematical analysis of such a sys-
tem, we illustrate a two-level controller, consisting of the
bottom two levels of the pendulum controller. The po-
sition xis controlled by setting a reference for ˙x, which
is controlled by setting a force, which determines ¨xby
Newton’s second law. Letting rxbe the reference posi-
tion and r˙xthe reference velocity, the equations are:
r˙x=k(rxx)m¨x=k0(r˙x˙x)
and therefore
m¨x+k0˙x+kk0x=kk0rx
This is identical to the equation of damped harmonic mo-
tion, although there are no physical springs involved. If
we write ρ=k0/mk (the ratio of the time constant 1/k
of the upper controller to m/k0, that of the lower con-
troller), then the roots of the characteristic equation are
1
2k(ρ±p(ρ2)24). For large ρ, these tend to k
and k(ρ1). For ρ= 4, the roots coincide at 2k,
and as ρapproaches zero, they describe circular arcs in
the complex plane as in Figure 3. The fastest response is
Figure 3: Root locus of two-level control system
obtained for ρequal to 4 or a little more (depending on
the precise definition of response time).
Cascade control
The above control scheme closely resembles a standard
configuration in process control known as cascade con-
trol, although the motivation is somewhat different. In
cascade control, where a single controller produces unac-
ceptable performance, due to the chosen actuator having
a slow effect on the controlled variable, a second con-
troller is introduced which controls some variable (called
the secondary variable) which has a more rapid effect on
the primary controlled variable. The output of the pri-
mary controller connects to the secondary reference (set-
point) input, and the secondary output connects to the ac-
tuator. Cascades of three or more controllers are possible,
but typical practice employs just two controllers.
We find it more instructive to consider each controller as
providing a virtual actuator to the next controller up, as
suggested by our description of the inverted pendulum
controller. In addition, the hierarchical arrangement is ca-
pable of much wider application, which we will demon-
strate with the walking robot which forms the main ex-
ample of this paper.
The value of 4 that we found above for the ratio of upper
to lower level time constant agrees with a standard rule
of thumb for cascade design, that the secondary controller
should have a response 4 or 5 times as fast as the primary.
A FOUR-LEGGED WALKING ROBOT
We have constructed a physical simulation of a walking
robot (see Figure 4) with four or more legs in which there
are two levels of the hierarchy, six controllers at the upper
level, and 12 controllers at the lower level. All of these
controllers are of the PID type. At the upper level there is
one controller for each degree of freedom of the robot’s
body. We assume that the robot can perceive the height of
its body above the ground; its other five degrees of free-
dom are assumed to be defined relative to the positions
of its feet on the ground (calculated relative to the body
from the joint angles by forward kinematic calculations).
At the lower level, there is a controller for each of the
three degrees of freedom of each leg: one at the knee and
two (pitch and yaw) at the shoulder. Each of the lower-
level controllers controls the rate of change of joint angle;
its output is the torque applied to the joint.
Figure 4: Four-legged robot. The blobs are virtual food
particles.
Each top level controller’s output is directed to some sub-
set of the lower level controllers’ reference inputs. Thus
the reference of each lower level controller is a weighted
sum of the outputs of the upper level controllers. The
weightings are given by a 6 by 12 linkage matrix, in
which all of the elements are 1, 0, or 1. (The behaviour
of the robot is experimentally found to be insensitive to
the exact values.) For this example, it is sufficient to
choose the weights by straightforward physical intuition.
For the robot to lift its body higher, it must decrease the
pitch angle at each shoulder. To swing its body to the left,
it must swing each shoulder joint to the right. To sway the
body towards the right, it must decrease the knee angles
on the right, and increase those on the left. And similarly
for the other three degrees of freedom: pitch, roll, and
forward sway.
With this control system, the robot is able to stand up
and balance on uneven terrain, and resist random exter-
nal forces. The architecture works equally well for six,
eight, or more legs (the linkage matrix having a general
definition that is uniform in the number of pairs of legs),
and the robot can continue to stand and resist disturbing
forces even when a leg is removed. The values of the
linkage matrix are not critical. One can even replace a
few of them by random values and obtain a system that
controls almost as well. Provided that there are sufficient
degrees of freedom at the lower level, and that different
body controllers do not both try to use the same set of
signals to the lower level in order to control different per-
ceptions, it is possible for all of the top level controllers
to simultaneously achieve good control, despite their in-
teractions.
It is important to the functioning of the robot that the body
controllers do not try to set the joint angles, but only their
velocities. Computing the angles required to produce a
given posture of the body requires complicated inverse
kinematic calculations and sensing of the terrain, and er-
rors in the data on which these calculations depend would
result in errors of comparable size in the body parame-
ters. The control scheme described avoids this problem
by having the body controllers demand certain rates of
change of the joint angles. The negative feedback action
of the controllers ensures that the joint angles will arrive
at whatever values are required, with residual errors de-
pending only on the tuning of the controllers and the ac-
curacy with which the controled variables are measured.
As an indication of the simplicity of the control scheme,
in a physical construction (which would dispense with
the physics simulation code), the most complicated cal-
culations would be the forward kinematic computation
of the body position relative to the feet. There is no mo-
tion planning, inverse kinematics, learning, adaptation, or
modelling by the robot of its environment. The robot is
described in more details in [2].
Analysis of a simplified robot
To mathematically illustrate, in a simpler setting, the op-
eration of a hierarchy such as we have described, we will
consider a greatly simplified robot with two legs and one
degree of freedom in each leg (Figure 5). Each leg is a
Figure 5: Two-legged two d.o.f. robot
linear actuator which has a length xor y, and exerts a
vertical force fxor fy. The body has two degrees of free-
dom of movement: height (h) and pitch (p). We assume
the centre of the body is constrained to a vertical line, and
that the pitch remains small, so that we can approximate
the kinematics and dynamics by linear equations:
h= (x+y)/2p= (yx)/l
¨
h= (fx+fy)/m ¨p= (fyfx)l/I
To simplify things, we choose units so that the mass m
of the body, its length l, and its moment of inertia Iare
all 1. There are four proportional controllers, arranged
according to the network of Figure 6. The lower level
controllers control ˙xand ˙y, and the upper level controls
hand p. For simplicity we will take the reference inputs
rhand rpto be zero (measuring h,x, and yrelative to
some convenient point above the ground). The resulting
Figure 6: Control hierarchy
equations are:
r˙x=khh+kpp r ˙y=khhkpp
fx=kx(r˙x˙x)fy=ky(r˙y˙y)
By symmetry it is reasonable to choose kh=kpand
kx=ky. By rescaling of variables we can choose kh=
kp= 1. Writing kfor kxand ky, the resulting equations
for hand pare:
¨
h+ 2k˙
h+ 2kh = 0 ¨p+ 2k˙p+ 2kp = 0
which again is the equation for damped harmonic mo-
tion for each variable. The optimal value for kis 2;
this is equivalent to the value of 4 found for the two-
controller hierarchy, taking into account that the linkage
matrix maps each top-level output to both the bottom-
level references.
It is instructive to consider what happens if we change the
linkage matrix. If we replace the equation for r˙xby
r˙x=khαh +kpp
(that is, replacing 1by αin Figure 6 on the line from the
hcontroller to the ˙xcontroller) then the equations for h
and pbecome:
¨
h+ 2k˙
h+k(α+ 1)h= 0
¨p+ 2k˙p+ 2kp =k(α1)h
When α= 1, this is the original system. For α= 0, the
height and pitch control interact, but both still reach their
reference value. As αapproaches 1, the response time
of the height controller becomes longer and longer. When
α=1, the linkage matrix is singular, which means that
the height and pitch controllers are attempting to control
different variables by means of identical actions. The re-
sult is that disturbances to the height of the robot are not
controlled. For α < 1, the system is unstable.
Walking and navigation
To make the robot walk, all that is required is for it to
repeatedly lift up some subset of its legs, swing them for-
wards, and put them down again. The controller for for-
wards position relative to the footprint will then pull the
body forwards. On uneven terrain, the body pitch and roll
controllers will keep the body aligned with the plane of
the footprint. Similarly, to turn anticlockwise, it repeat-
edly lifts a subset of legs, swings them anticlockwise, and
puts them down, letting the body heading controller bring
the body into alignment with the new footprint.
By these means, the robot is able to walk and turn on
uneven terrain, and go up and down (shallow) stairs.
Adding rudimentary senses to detect the direction of a
landmark enables it to navigate towards it by varying the
magnitude of the walk and turn actions. The robot has
been implemented in a simulation that can be run from
http://www.cmp.uea.ac.uk/jrk/Archy/Archy.html.
The simulation includes an implementation of the dy-
namics of a rigid body (the robot body excluding the
legs), acted on by forces exerted by the legs between the
body and the ground. The dynamics of legs lifted from
the ground has not been modelled; nevertheless, the con-
trol problem, although simplified from reality, is a com-
plex control problem in its own right, which the con-
trol architecture we have described is empirically able to
solve.
TWO MORE CASE STUDIES
A backhoe excavator
A backhoe excavator, such as that of Figure 7, has three
joints, each actuated by a hydraulic cylinder. If one
wishes to drive the bucket in a straight horizontal line,
keeping it in a constant orientation, one must operate
all three actuators in a rather complex way. A con-
trol system whose controlled variables are the reach,
lift, and inclination of the bucket can provide the oper-
ator with the ability to directly drive the bucket straight
forwards, backwards, up, and down. It is straightfor-
ward to apply the architecture described above to the
task, and we have done so using a simulation based on
the Vortex physical simulation library1. Some screen-
captured movies of the resulting simulations are available
at http://www.cmp.uea.ac.uk/jrk/Robotics/digger.
The control architecture is essentially the same as that of
the 2-degree of freedom robot, but with three controllers
on each level. On the top level there is one to control
each degree of freedom of the bucket, and on the bottom
level there is one to control the velocity of each joint. The
actuators are assumed to supply a specified torque to the
joint. (We have not modelled actual hydraulic actuators.)
The linkage matrix connecting the top-level outputs to
the bottom-level references is more complicated than for
1Vortex is produced by CM Labs, http://www.cm-labs.com.
Figure 7: Backhoe excavator (JCB model 803)
the robot, as it depends on the current state of the robot.
When the bucket is far, increasing the dipper angle will
raise it, but when it is close, increasing the dipper angle
lowers it (Figure 8). When the dipper is vertical, it has no
effect on bucket height. The routing of the output from
the height controller to the reference input of the dipper
controller must therefore depend on the current configu-
ration. For each top-level variable x, and each joint y,
we have chosen the corresponding element of the linkage
matrix to be ∂x/∂y. That is, the more effect an actua-
tor has on a top-level controlled variable, the more that
actuator will be used to control it. As with the 4-legged
robot, the precise values are not critical. In the resulting
simulation, the bucket closely follows the reference point
and reference inclination as the references are moved by
the operator.
Figure 8: Dependence of linkage matrix on configuration
An obvious generalisation of this design is to a backhoe
with more joints in its arm. With four joints, as in Fig-
ure 9, there is an extra degree of freedom which is not
constrained by the position and orientation of the bucket.
A simple way of fixing the extra degree of freedom is
to add another top-level controller whose controlled vari-
able is a3a2(the difference between the exterior an-
gles at the second and third joints), or more generally,
a3ka2for some constant k. This forces joints 2 and 3
to have similar exterior angles, avoiding either of the ex-
treme configurations shown in the Figure. While a four-
jointed backhoe may not necessarily be a more practical
excavation machine than the standard three-jointed con-
figuration, there are applications to robotic arms requir-
ing extra joints to reach into confined spaces. Arms with
five or more joints can be controlled similarly, using ex-
tra equalisation controllers to fix all the extra degrees of
freedom. Such arms may be beyond the capacity of a hu-
man controller to operate effectively by directly driving
the joints.
Figure 9: Four-joint backhoe
A two-dimensional biped robot.
The physical simulation of our four-legged robot deliber-
ately simplifies reality in order to alleviate the program-
ming task. Using the Vortex library, we have begun to
develop more faithful simulations with the goal of simu-
lating the robot with sufficient physical fidelity to justify
actual construction. Initially, we have constructed a two-
legged robot constrained to a vertical plane, capable of
standing and controlling the three degrees of freedom of
its body: height, pitch, and sway (Figure 10). As with
Figure 10: Two-legged two-dimensional robot
the backhoe, legs with more joints than necessary can be
used, with the extra degrees of freedom taken up by con-
trollers keeping consecutive joint angles equal. There is
one further degree of freedom unaccounted for: this is the
force between the two feet tending to splay them apart or
draw them together. An extra controller can be added
to maintain the splay force near zero, by adding suitable
amounts to the rate of change of the torques at the leg
joints.
FUTURE WORK ON WALKING
The four-legged robot simulation described above omit-
ted both the physics and the control problem of control-
ling a leg whose foot is off the ground. A controller
for foot position can be designed along similar lines to
the backhoe control system (assuming the other legs are
meanwhile providing sufficient support for the body).
This will require switching each leg between being used
either to control foot position or to control body posi-
tion. A third-level walking controller would accomplish
this by altering the linkage matrix connecting the foot and
body controllers to the joint angle velocity controllers.
RELATED APPROACHES
Besides cascade control, described above, there is another
approach to the design of robotic systems which bears
a superficial resemblance to the present proposal, called
subsumption. This is an architecture originally devised
by Brooks [1], in which the control problem is, as for
HPCT, broken down into a hierarchical arrangement of
simpler agents. There are two fundamental differences
with HPCT. Firstly, in a subsumption architecture, the
agents are not necessarily conceived of as controllers,
that is, agents which attempt to produce a certain input
by means of their outputs. Secondly, the main principle
of the subsumption architecture, for which it is named,
is that all of the controllers at all levels act directly on
the actuators, controllers at higher levels suspending the
actions of controllers at lower levels as necessary, the
lower level resuming its operation when the higher level
has completed its task. Thus some of the actions taken
to balance a legged robot in a standing posture are sus-
pended when a higher-level agent for walking needs to
lift some legs off the ground; an agent for walking in a
straight line will be suspended by an agent for collision
avoidance, and so on. In HPCT, only the bottom-level
controllers send signals to the actuators. Higher level
controllers send their outputs only to the reference inputs
of controllers at the next level down. In subsumption,
higher-level agents operate instead of lower level agents;
in HPCT, higher-level controllers operate by means of
lower level controllers.
In principle, an HPCT controller could act not only by al-
tering the references of lower level controllers, but also,
for example, by altering parameters of their output func-
tions, or the linkage matrix connecting them to their de-
scendants. However, there is never any skipping of lev-
els. As mentioned above, we intend to employ the lat-
ter scheme of modifying the linkage matrix to implement
walking, to switch each leg between the roles of support-
ing the body and moving the foot to a new position.
References
[1] R. Brooks. A robust layered control system for a mo-
bile robot. IEEE Journal of Robotics and Automa-
tion, 2(1):14–23, March 1986.
[2] J. R. Kennaway. Control of a multi-legged robot
based on hierarchical perceptual control theory. J.
Perceptual Control Theory, 1(1), 1999.
[3] W. T. Powers. Behavior: The Control Of Perception.
Benchmark Publications, 1998. Originally published
by Wildwood Press, 1974.
[4] W. T. Powers. Making Sense of Behavior: The Mean-
ing of Control. Benchmark Publications, 1999.
... As mentioned above, other PCT theorists have developed computer simulations (E.g., Powers, 1979aPowers, , 1979bPowers, , 1989Powers, , 1999Marken, 1986Marken, , 1989Marken, , 1992Kennaway, 1999) (Powers, 2008). ...
Research
Full-text available
Background: Many key researchers have noted the importance of fully understanding the mechanism of psychological change, as it is undoubtedly one of the most important aspects of psychology in need of investigation. Such an investigation however, has proven challenging both methodologically and scientifically, as no coherent and comprehensive explanation of such a mechanism is available in the literature. This study intends to overcome these challenges and provide evidence for the process of reorganization as a mechanism of psychological change, as suggested by Perceptual Control Theory (PCT). Aims: To provide tangible evidence towards a definition of the mechanism of psychological change according to the principles of PCT, and demonstrate that higher level reorganization is more effective than lower level reorganization within a control hierarchy, which is a key assumption in the implementation of the cognitive psychotherapeutic approach which draws on PCT; the Method of Levels. Method: Computer software programmed according to the guiding principles of PCT was used to simulate control in a 3 tier hierarchy of control systems. This hierarchy was then assigned 20 sets of randomised connection strengths (each set constituting one ‘participant’) which were then subjected to repeated trials over three conditions. These conditions were; (a) Reorganization disabled, (b) Tier 2 –Mid Level Reorganization, and (c) Tier 3 – Higher Level Reorganization. The data collected was then subjected to a 2- Way repeated measures ANOVA to investigate the difference in error reduction in each individual ‘participant’ across all three conditions. Various secondary statistical tests were also conducted to verify the findings of the ANOVAs. Results: The analyses implemented revealed that (a) the control system was capable of reducing error at a minor level even in the absence of reorganization, (b) mid level reorganization led to greater error reduction than the ‘reorganization disabled’ condition, and (c) that in accordance with the main hypothesis of the study, higher level reorganization led to the greatest reduction in error in comparison with both mid level reorganization and no reorganization. Conclusions: Though there were some limitations to the computer software being used, the results of the statistical analysis indicate that the hypothesis that higher level reorganization leads to improved error reduction in comparison to lower level reorganization is accurate. Future studies may wish to address the limitations outlined in this study and further investigate this hypothesis, as consistent findings across numerous studies will improve the case for a mechanism of psychological change.
... As mentioned above, other PCT theorists have developed computer simulations (E.g., Powers, 1979aPowers, , 1979bPowers, , 1989Powers, , 1999Marken, 1986Marken, , 1989Marken, , 1992Kennaway, 1999) (Powers, 2008). ...
... As mentioned above, other PCT theorists have developed computer simulations (E.g., Powers, 1979aPowers, , 1979bPowers, , 1989Powers, , 1999Marken, 1986Marken, , 1989Marken, , 1992Kennaway, 1999) (Powers, 2008). ...
... Perceptual control theory (and HPCT) appears to offer a number of interesting features for modelling the behaviour of living systems [71], and should thus be a serious contender for implementing control in artificial systems such as robots (as envisaged by Powers [72,73,74,75]). A few attempts have been made in this area by PCT practitioners [76,77,78]. However, since a large proportion of robotocists are control engineers who take negative feedback systems as a given, they have not seen the particular benefits of using (H)PCT. ...
Article
Recent years have witnessed increasing interest in the potential benefits of `intelligent' autonomous machines such as robots. Honda's Asimo humanoid robot, iRobot's Roomba robot vacuum cleaner and Google's driverless cars have fired the imagination of the general public, and social media buzz with speculation about a utopian world of helpful robot assistants or the coming robot apocalypse! However, there is a long way to go before autonomous systems reach the level of capabilities required for even the simplest of tasks involving human-robot interaction - especially if it involves communicative behaviour such as speech and language. Of course the field of Artificial Intelligence (AI) has made great strides in these areas, and has moved on from abstract high-level rule-based paradigms to embodied architectures whose operations are grounded in real physical environments. What is still missing, however, is an overarching theory of intelligent communicative behaviour that informs system-level design decisions in order to provide a more coherent approach to system integration. This chapter introduces the beginnings of such a framework inspired by the principles of Perceptual Control Theory (PCT). In particular, it is observed that PCT has hitherto tended to view perceptual processes as a relatively straightforward series of transformations from sensation to perception, and has overlooked the potential of powerful generative model-based solutions that have emerged in practical fields such as visual or auditory scene analysis. Starting from first principles, a sequence of arguments is presented which not only shows how these ideas might be integrated into PCT, but which also extend PCT towards a remarkably symmetric architecture for a needs-driven communicative agent. It is concluded that, if behaviour is the control of perception, then perception is the simulation of behaviour.
Article
A new architecture for controlling mobile robots is described. Layers of control system are built to let the robot operate at increasing levels of competence. Layers are made up of asynchronous modules that communicate over low-bandwidth channels. Each module is an instance of a fairly simple computational machine. Higher-level layers can subsume the roles of lower levels by suppressing their outputs. However, lower levels continue to function as higher levels are added. The result is a robust and flexible robot control system. The system has been used to control a mobile robot wandering around unconstrained laboratory areas and computer machine rooms. Eventually it is intended to control a robot that wanders the office areas of our laboratory, building maps of its surroundings using an onboard arm to perform simple tasks.
Making Sense of Behavior: The Meaning of Control
  • W T Powers
W. T. Powers. Making Sense of Behavior: The Meaning of Control. Benchmark Publications, 1999.