Content uploaded by Daan Marinus Pool

Author content

All content in this area was uploaded by Daan Marinus Pool on Sep 16, 2014

Content may be subject to copyright.

Modeling Human Multimodal Perception and

Control Using Genetic Maximum Likelihood

Estimation

P.M.T. Zaal,∗D.M. Pool,†

Q.P. Chu,‡M.M. van Paassen,§M. Mulder¶and J.A. Mulderk

Delft University of Technology, Delft, The Netherlands

This paper presents a new method for estimating the parameters of

multi-channel pilot models that is based on maximum likelihood estima-

tion. To cope with the inherent nonlinearity of this optimization problem,

the gradient-based Gauss-Newton algorithm commonly used to optimize

the likelihood function in terms of output error is complemented with a

genetic algorithm. This signiﬁcantly increases the probability of ﬁnding

the global optimum of the optimization problem. The genetic maximum

likelihood method is successfully applied to data from a recent human-in-

the-loop experiment. Accurate estimates of the pilot model parameters and

the remnant characteristics were obtained. Multiple simulations with in-

creasing levels of pilot remnant were performed, using the set of parameters

found from the experimental data, to investigate how the accuracy of the

parameter estimate is aﬀected by increasing remnant. It is shown that only

for very high levels of pilot remnant the bias in the parameter estimates

∗Ph.D. student, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058,

2600GB Delft, The Netherlands; p.m.t.zaal@tudelft.nl. Student member AIAA.

†Ph.D. student, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058,

2600GB Delft, The Netherlands; d.m.pool@tudelft.nl. Student member AIAA.

‡Associate Professor, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058,

2600GB Delft, The Netherlands; q.p.chu@tudelft.nl. Member AIAA.

§Associate Professor, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058,

2600GB Delft, The Netherlands; m.m.vanpaassen@tudelft.nl. Member AIAA.

¶Associate Professor, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058,

2600GB Delft, The Netherlands; m.mulder@tudelft.nl. Member AIAA.

kProfessor, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058, 2600GB

Delft, The Netherlands; j.a.mulder@tudelft.nl. Member AIAA.

1 of 29

is substantial. Some adjustments to the maximum likelihood method are

proposed to reduce this bias.

Nomenclature

AState matrix

AdDisturbance sinusoid amplitude deg

AtTarget sinusoid amplitude deg

BInput matrix

COutput matrix

DFeedthrough matrix

eTracking error signal deg

fProbability density function -

fdDisturbance forcing function deg

ftTarget forcing function deg

H(s) Transfer function

H(jω) Frequency response function

HnRemnant ﬁlter

Hnm Neuromuscular dynamics

Hpe Pilot visual response

Hpθ Pilot motion response

Hsc Semicircular canal dynamics

Hθ,δeControlled pitch dynamics

jImaginary unit -

LLikelihood function -

KmMotion perception gain -

KnRemnant ﬁlter intensity -

KvVisual perception gain -

Kδe,u Pitch stick gain -

MΘΘ Fisher information matrix

mNumber of samples -

NRemnant Fourier transform -

NeOrder of the error response state matrix -

nPilot remnant signal deg

ndDisturbance frequency integer factor -

ntTarget frequency integer factor -

SSpectrum

2 of 29

sLaplace variable -

Tlead Visual lead time constant s

Tlag Visual lag time constant s

TmMeasurement time s

Tsc1,Tsc2Semicircular canal time constants s

tTime s

uPilot control signal deg

¯xState vector

Symbols

αLine search parameter -

δeElevator deﬂection deg

ǫPrediction error deg

ζnRemnant ﬁlter damping -

ζnm Neuromuscular damping -

Θ Parameter vector

θPitch angle deg

¨

θPitch acceleration deg s−2

µMean

σStandard deviation

τmMotion perception time delay s

τvVisual perception time delay s

φSinusoid phase shift rad

ωFrequency rad s−1

ωmMeasurement base frequency rad s−1

ωnRemnant ﬁlter break frequency rad s−1

ωnm Neuromuscular frequency rad s−1

Subscripts

epilot error response

ddisturbance

ttarget

θpilot pitch response

3 of 29

I. Introduction

In manual control of aircraft, pilots combine information from cockpit instruments, their

view of the outside world and physical motion sensations to achieve a suitable control action.

Knowledge on how pilots use these diﬀerent types of motion cues not only gives more insight

into human motion perception processes, but it is also crucial to the design of manual control

systems and the tuning of ﬂight simulators.1,2

Many researchers applied multi-channel pilot models in their eﬀorts to explain and quan-

tify the eﬀects of diﬀerent modalities on pilot control behavior.3–9 Human manual vehicle

control behavior is an inherently nonlinear and time-varying closed-loop process. For care-

fully designed control tasks, however, the control behavior of well-trained individuals can be

accurately described with quasi-linear pilot models.10 Such models consist of a linear part

that describes a pilot’s response to all perceived variables in terms of control-theoretical

elements, supplemented with a remnant signal that accounts for all nonlinear contributions

to the observed control behavior. For tasks where pilots perceive information from multiple

modalities – e.g., visual and motion – these linear pilot models generally have a multi-channel

structure, where the response to the diﬀerent modalities is separated. The characteristics

of such linear pilot responses are deﬁned by the model parameters, such as weighing gains

and time delays. Values for these parameters, which can be determined from experimental

data using mathematical identiﬁcation techniques, help to explain the eﬀects of diﬀerent

perceptual modalities on pilots’ control behavior in control-theoretical terms.

Parameter estimation techniques currently employed to estimate the parameters of a

multi-channel pilot model use either Fourier transforms11 or linear time-invariant models8to

obtain non-parametric pilot describing functions in the frequency domain. In a second step,

the parameters of a multi-channel pilot model are then optimized to yield an optimal ﬁt to

these frequency responses. These methods have two main disadvantages. First, the accuracy

of the parameter estimate is aﬀected by biases that originate from both identiﬁcation steps.

In addition, for these identiﬁcation methods to give accurate results, highly speciﬁc demands

on the design of the control task – and speciﬁcally the adopted forcing functions – need to

be met.8

An alternative to these frequency-domain parameter estimation methods are time-domain

identiﬁcation procedures, which allow for estimation of model parameters from time-domain

data directly. For such time-domain identiﬁcation techniques, the requirements to ensure

the identiﬁability of a model are signiﬁcantly less stringent.12 Few studies are described

in the literature that investigate the application of time-domain identiﬁcation techniques to

pilot modeling.13–18 These studies only considered very simple pilot models and provide little

detail of the identiﬁcation procedure.

4 of 29

Maximum likelihood estimation is an example of a statistical time-domain identiﬁca-

tion method that, for instance, has been successfully applied to the identiﬁcation of aircraft

stability and control derivatives from ﬂight test data19–21 and of air- and spacecraft struc-

tural modes.22, 23 This study focuses on the application of a maximum likelihood parameter

estimation algorithm to the problem of multi-channel pilot model identiﬁcation. For the de-

velopment of this algorithm, an output-error structure is assumed for the quasi-linear pilot

model. Of particular interest are the steps that need to be taken to make the identiﬁcation

technique suitable for coping with the inherent nonlinearity and many local minima of the

cost function. To cope with these nonlinearities and local minima, the maximum likelihood

method is enhanced with a genetic algorithm. Data from a recent human-in-the-loop ex-

periment,2which was performed in the SIMONA Research Simulator at Delft University

of Technology, is used to test the genetic maximum likelihood algorithm. To evaluate the

performance of this parameter estimation method with increasing levels of pilot remnant,

the average bias and standard deviation of the parameter estimates are assessed using pilot

model simulations.

The structure of the paper is as follows. First, the parameter estimation problem for

multi-channel pilot models will be discussed. Then, the genetic maximum likelihood estima-

tion procedure will be explained in detail and parameter estimation results from experimental

data will be given. The paper ends with a discussion and conclusions.

II. The Parameter Estimation Problem

A manual vehicle control task that was investigated in a recent human-in-the-loop exper-

iment will be considered as an example of a typical multi-channel pilot model identiﬁcation

problem in this paper. The objective of the experiment was to investigate the eﬀects of ro-

tational and vertical motion during aircraft pitch attitude control in a disturbance-rejection

task.2It was found that the presence of rotational pitch motion signiﬁcantly aﬀected pi-

lot control behavior, making aircraft pitch attitude control a clear example of a multi-loop

control task.

ftu

−

e

θ

n

Hpe

Hpθ

−

pilot

Kδe,u Hθ,δe

θ

fd

δe

+ + ++

+

Figure 1. Multi-loop representation of a closed-loop aircraft pitch control task.

5 of 29

II.A. The Multi-Loop Control Task

In Figure 1, a schematic representation of the pitch attitude control task that is studied in

this paper is depicted. As can be seen in this ﬁgure, the pilot acts as a feedback controller on

the pitch dynamics of an aircraft, Hθ,δe. For this experiment, the controlled pitch dynamics

were those of a Cessna Citation II aircraft as given by:

Hθ,δe(s) = −10.6189 s+ 0.9906

s(s2+ 2.756s+ 7.612).(1)

In addition to the tracking error e, which is the diﬀerence between the current and

the desired pitch attitude as perceived from the compensatory visual display, information

about the aircraft pitch attitude can be perceived via physical rotational motion. Hence,

the total pilot response consists of the contributions of two linear response functions, Hpe

and Hpθ. A remnant signal nis added to the linear model output representing the nonlinear

behavior. The gain Kδe,u in Figure 1 represents the scale factor between sidestick and

elevator deﬂection. Two forcing functions, a disturbance fdand target ft, are used to excite

the combined pilot-aircraft system. The target signal gives the desired pitch attitude; the

disturbance signal can be considered as turbulence acting on the aircraft, perturbing the

pitch angle.

II.B. The Multi-Channel Pilot Model

The linear response functions are parametrized by gains and time constants, the pilot equal-

ization; time delays and neuromuscular dynamics, the pilot limitations; and vestibular dy-

namics, the pilot sensor dynamics. Based on McRuer’s precision model10 and Van der Vaart’s

multi-channel pilot model,6appropriate models for the linear response functions Hpe and Hpθ

for control of the pitch dynamics deﬁned by Eq. (1) are given by:2

Hpe (jω) = Kv

(1 + jωTlead)2

(1 + jωTlag )e−jωτvHnm (jω),(2)

and

Hpθ (jω) = (jω)2Hsc (jω)Kme−jωτmHnm (jω).(3)

In Eq. (2), which describes the response to visual motion cues, Kvis the visual perception

gain, Tlead the visual lead time constant, Tlag the visual lag time constant and τvthe visual

perception time delay. Rotational pitch motion is mainly perceived with the semi-circular

canals of the vestibular system, which are sensitive to pitch acceleration.7The model for

the pitch motion perception channel Hpθ (Eq. (3)) therefore includes the dynamics of the

6 of 29

semi-circular canals Hsc, the motion perception gain Kmand a motion perception time delay

τm.

As can be veriﬁed from the presence of Hnm in both Eq. (2) and Eq. (3), the total

linear pilot response is attenuated by the dynamics of the neuromuscular system. The

neuromuscular system is modeled as a second-order mass-spring-damper system, of which

the damping ζnm and natural frequency ωnm are parameters to be estimated:

Hnm (jω) = ω2

nm

ω2

nm + 2ζnmωnm jω + (jω)2.(4)

Finally, the dynamics of the semi-circular canals, which are part of the pitch motion

perception channel of the pilot model, are given by:

Hsc (jω) = 1 + jωTsc1

1 + jωTsc2

,(5)

with Tsc1= 0.11 and Tsc2= 5.9 seconds. These values are taken from previous research7and

are assumed ﬁxed when modeling pilot control behavior for the pitch control task deﬁned in

Figure 1. The parameter vector Θ, with a total of eight parameters to be estimated, is given

by:

Θ = [KvTlead Tlag τvKmτmζnm ωnm]T.(6)

II.C. Forcing Functions

For reliable identiﬁcation of both the pilot visual and motion responses with previously used

parameter estimation methods, two independent forcing function signals are required. For

the control task described in Figure 1, both the target and disturbance forcing function sig-

nals (ftand fd) were deﬁned as sums of ten sine waves with diﬀerent frequencies, amplitudes

and phase shifts:

ft(t) =

10

X

k=1

At(k) sin (ωt(k)t+φt(k)) ,(7)

fd(t) =

10

X

k=1

Ad(k) sin (ωd(k)t+φd(k)) .(8)

To allow for use of spectral methods in the analysis of the experimental data, the forcing

function sine wave frequencies were all deﬁned as integer multiples of the experimental mea-

surement time base frequency, ωm= 2π/Tmwith Tm= 81.92 seconds. The corresponding

integer factors ntand ndare listed in Table 1, together with the target and disturbance

7 of 29

signal frequencies, amplitudes and phases.

Table 1. Multi-sine forcing function properties.

disturbance, fdtarget, ft

k, – nd, – ωd, rad s−1Ad, deg φd, rad nt, – ωt, rad s−1At, deg φt, r ad

1 5 0.383 0.344 -0.269 6 0.460 0.698 1.288

2 11 0.844 0.452 4.016 13 0.997 0.488 6.089

3 23 1.764 0.275 -0.806 27 2.071 0.220 5.507

4 37 2.838 0.180 4.938 41 3.145 0.119 1.734

5 51 3.912 0.190 5.442 53 4.065 0.080 2.019

6 71 5.446 0.235 2.274 73 5.599 0.049 0.441

7 101 7.747 0.315 1.636 103 7.900 0.031 5.175

8 137 10.508 0.432 2.973 139 10.661 0.023 3.415

9 171 13.116 0.568 3.429 194 14.880 0.018 1.066

10 226 17.334 0.848 3.486 229 17.564 0.016 3.479

The target signal was deﬁned to have only a quarter of the power of the disturbance

signal to make the task primarily a disturbance-rejection task.2

II.D. Parameter Estimation Methods

Estimation of multi-channel pilot model parameters from measurement data is a highly

nonlinear optimization problem. First of all, a linear model is ﬁt on data that is inherently

nonlinear. The fact that the models are nonlinear in their parameters further increases the

nonlinearity of the optimization problem. Due to these nonlinearities, the resulting cost

function contains many local minima in addition to the global minimum.

In addition, it is apparent from the model structure given in Eq. (2) and Eq. (3) that

the lead term in the visual perception channel and the lead resulting from the integrating

action of the semicircular-canal dynamics in the vestibular channel are more or less inter-

changeable. If multiple terms in a model can cause the same overall response, the model is

overdetermined, which adds to the number of local minima. It is found in previous research

that the occurrence of these local minima is highly dependent on the controlled dynamics,

the type of forcing function and even the subject who performed the experiment.24

Many diﬀerent techniques can be applied to estimate the parameters of a multi-channel

pilot model from measurement data. Figure 2 illustrates that two groups can be distin-

guished, parameter estimation in the frequency domain and parameter estimation in the

time domain.

II.D.1. Frequency-Domain Techniques

Parameter estimation in the frequency domain requires an additional step, to transform

the measured time-domain data to the frequency domain, as illustrated in Figure 2. In

this additional step, Fourier coeﬃcients (FC) or linear time-invariant (LTI) models (e.g.,

8 of 29

autoregressive exogeneous (ARX) models) are used to estimate non-parametric frequency

response functions.8, 11 These non-parametric frequency responses are then used in a second

step, in which a multi-channel pilot model is ﬁt by adjusting its parameters. For estimating

the pilot model parameters a frequency-domain criterion is used, which results in a nonlinear

optimization problem.8, 9, 25

time

domain

frequency

domain

non-parametric

identiﬁcation

parameter

estimation

parameter

estimation

frequency-domain

techniques

time-domain

techniques

pilot model parameters

measured time traces

Figure 2. Comparison of pilot model parameter estimation methods.

With some exceptions, in the ﬁeld of human operator modeling, parameter estimation

has mainly been performed in the frequency domain.3, 4, 6–9, 11, 25, 26 An advantage of this

technique is that the ﬁrst step serves as a data reduction step, which makes the parameter

optimization eﬃcient in terms of computational power. Another advantage of this method

is that the non-parametric frequency response functions provide an indication of the pilot

model structure required for describing the pilot dynamics.

One of the disadvantages of using two steps to estimate the pilot model parameters is that

the bias and variance of the non-parametric frequency responses can cause increased bias

and variance in the obtained parameter estimates. A second disadvantage is that in order to

estimate multiple frequency response functions in the ﬁrst step, an equal number of forcing

functions needs to be inserted into the closed-loop system to assure that the inputs to the

pilot model are uncorrelated (see Figure 1).8, 11 Inserting multiple forcing functions may yield

an unrealistic piloting task as compared to real ﬂight and makes the data less comparable to

classical single forcing function target-following and disturbance-rejection tasks used in many

experiments described in literature.6, 7, 10, 26 When using Fourier coeﬃcients or LTI models

for estimating the frequency response functions, the forcing functions need to be carefully

designed keeping in mind the requirements for identiﬁability of the frequency responses and

the pilot limitations.8

9 of 29

II.D.2. Time-Domain Techniques

Time-domain parameter estimation techniques, such as the least-squares method and max-

imum likelihood estimation,27 directly ﬁt a parameter model on the time-domain data (see

Figure 2). As time-domain signals generally have signiﬁcantly more data points than the

frequency response functions used in the frequency-domain methods, ﬁtting the parameter

model on the time-domain data requires more computational power. Another disadvantage

is that it is diﬃcult to determine the correct model structure beforehand and an incorrect

model structure can not be easily detected by comparing the model output with the measured

pilot output.

One of the advantages of a time-domain method is that the parameter model is directly ﬁt

using only one step, reducing the bias and variance of the estimated parameters compared to

the frequency-domain methods. Also, there are fewer constraints when designing the forcing

functions, as the frequency content of the signals has less inﬂuence on the identiﬁability of

the parameters. There is also no explicit requirement for uncorrelated input signals of the

pilot model as is the case using the frequency-domain methods.

In experiments, the diﬀerences in pilot control behavior for diﬀerent experimental condi-

tions are often small in magnitude. The increased accuracy of the estimated parameters for

time-domain techniques could increase the signiﬁcance of the results in these experiments.

Also, as time-domain methods allow for more design freedom of the forcing functions, tasks

which are more comparable to real piloting tasks can be used in experiments. This will

increase the insight into multimodal perception and control for such tasks.

III. Genetic Maximum Likelihood Estimation

Maximum likelihood estimation (MLE) is a time-domain parameter estimation technique

that has many applications.19–23, 28 One of the main reasons for this is that maximum likeli-

hood estimates have some attractive statistical properties. They are consistent and eﬃcient,

which means that the parameter estimate converges to the true parameter set and that the

variance reduces to the Cram´er-Rao lower bound as the sample size increases.29 Furthermore,

the errors in the resulting parameter estimates have an unbiased Gaussian distribution.

In this section a procedure for estimating the parameters of multi-channel pilot models

that is based on the principle of maximum likelihood will be described. The structure of

such pilot models warrants the use of an output-error model structure, which results in a

simpliﬁcation of the full maximum likelihood estimation problem. To increase the probability

of ﬁnding the global minimum of this nonlinear optimization problem, a genetic algorithm

is used to determine the initial parameter estimates for a classical gradient-based parameter

estimation procedure.

10 of 29

III.A. The Maximum Likelihood Method

The pilot model considered in this paper incorporates separate pilot visual and motion

responses, as depicted in Figure 1. The pilot’s control output uis considered to be the sum

of a response to visual and motion cues (eand θ, respectively) and a remnant signal n.

Maximum likelihood estimation procedures generally require models that are written in

state-space form. For the multi-channel pilot model of Figure 1, a discrete time state-space

representation can be given by:

˙

¯x(k+ 1) =

Ae(Θ) 0

0Aθ(Θ)

¯x(k) +

Be(Θ) 0

0Bθ(Θ)

e(k)

θ(k)

(9)

u(k) = hCe(Θ) −Cθ(Θ) i¯xs(k) + hDe(Θ) −Dθ(Θ) i

e(k)

θ(k)

+n(k).

In Eq. (9), Θ represents the vector of unknown pilot model parameters as deﬁned by

Eq. (6). The subscripts eand θdivide the total state-space system into the state-space

models of the error response function Hpe and the pitch response function Hpθ , respectively.

These state-space representations are easily obtained by converting the transfer function

models deﬁned in Equations (2) and (3) using the controller canonical form, as illustrated

for Ae(Θ) by:

Ae(Θ) =

0 1 0 ··· 0

0 0 1 ....

.

.

.

.

..........0

0 0 ··· 0 1

Ae(Ne,1) Ae(Ne,2) ··· Ae(Ne, Ne−1) Ae(Ne, Ne)

,(10)

where Neis the order of Ae(Θ). For the conversion to a state-space representation, the pilot

model time delays are approximated by using Pad´e approximations of typically ﬁfth order,

to ensure accurate description of the high-frequency phase roll-oﬀ. Especially due to these

Pad´e approximations, the matrices of the state-space model as deﬁned by Eq. (9) contain

coeﬃcients that are a highly nonlinear function of the pilot model parameters. For example,

Equations (11) and (12) give the ﬁrst and last coeﬃcients in the ﬁnal row of Ae(Θ) for a

ﬁfth order Pad´e approximation of the visual time delay τv.

11 of 29

Ae(Ne,1) = −30240ω2

nm

Tlagτ5

v

(11)

Ae(Ne, Ne) = −2Tlag τ5

vζnmωnm −30Tlag τ4

v−τ5

v

Tlagτ5

v

(12)

The state-space pilot model given in Eq. (9) only has an additive noise term in the output

equation, i.e., no process noise is assumed. This output-error structure signiﬁcantly reduces

the complexity of the MLE procedure. The pilot remnant n, which is accounted for by

the measurement noise in the chosen model structure, is modeled as an additive zero-mean

Gaussian white noise signal, whose properties are deﬁned as:

E{n(k)}= 0; En(k)nT(k)=σ2

n.(13)

Maximum likelihood estimation techniques attempt to ﬁnd an estimate ˆ

Θ of the pa-

rameter vector Θ that maximizes the likelihood function. The likelihood function L(Θ) is

deﬁned as the joint conditional probability density function of the prediction error for m

measurements of u(k):

L(Θ) = f(ǫ(1), ǫ(2),...,ǫ(k),...,ǫ(m)|Θ).(14)

The prediction error, indicated as ǫ(k) in Eq. (14), is deﬁned as the diﬀerence between

the measured pilot control signal u(k) and the modeled pilot control signal ˆu(k) at dis-

crete instants. Given the properties of the remnant as deﬁned in Eq. (13), the conditional

probability density function for one measurement of ǫ(k) is given by:

f(ǫ(k)|Θ) = 1

p2πσ2

n

e−ǫ2(k)

2σ2

n.(15)

The set of parameters that maximizes the likelihood function is the maximum likelihood

estimate of the parameter vector Θ. For the MLE method it is common practice to minimize

the negative natural logarithm of the likelihood function instead of maximizing L(Θ), as

this results in a more straightforward optimization problem. When a global minimum of the

negative log-likelihood is attained, the resulting parameter vector is the maximum likelihood

estimate, indicated with ˆ

ΘML. By combining Eq. (14) and Eq. (15) and considering the fact

that the pilot model contains a single output, the following expression for this maximum

likelihood estimate can be obtained:

12 of 29

ˆ

ΘML = arg min

Θ

−ln L(Θ) = arg min

Θ"m

2ln σ2

n+1

2σ2

n

m

X

k=1

ǫ2(k)#.(16)

Eq. (16) summarizes the parameter estimation problem that is investigated for the estima-

tion of multi-channel pilot model parameters in this paper. Similar to the frequency-domain

methods, this is a highly nonlinear optimization problem, as described in Section II.D. To

cope with this problem, a strategy is chosen here that uses a genetic algorithm to identify a

solution close to the global minimum.

III.B. Genetic Likelihood Optimization

Genetic algorithms are commonly used to ﬁnd approximate solutions to nonlinear optimiza-

tion problems, and have been successful in many applications.18, 30 The problem solving

capabilities of genetic algorithms are based on the principle of “survival of the ﬁttest” as

considered in evolutionary biology. In such an algorithm, a population of candidate solu-

tions to the optimization problem is subjected to random genetic functions such as selection,

mating, crossover and mutation. These functions cause the population to evolve toward in-

creasingly better, i.e., more ﬁt, solutions. The inherent randomness of genetic algorithms and

the fact that a large number of diﬀerent solutions are evaluated results in a high probability

of ﬁnding the global minimum of an optimization problem.

Zaychik and Cardullo18 used a genetic algorithm as the principal estimation procedure

for identifying the parameters of the Hess operator model5from simulation data. However,

in this research, maximum likelihood estimation was not used for the formulation of the

optimization problem. Abutaleb30 describes an application of a genetic algorithm to a max-

imum likelihood estimation problem. The negative logarithm of the likelihood function is

used as the ﬁtness criterion and the parameter sets that produce the lowest log-likelihood

are deﬁned to be the ﬁttest members of the population.

In the maximum likelihood estimation method developed in this paper, a genetic algo-

rithm is used to ﬁnd sets of pilot model parameters that minimize the likelihood function

deﬁned by Eq. (14). At the start of a genetic optimization process a random population of

parameter sets is generated within a lower and upper bound for all parameters. The ﬁtness

of all the members of the population is calculated. Next, iterations are performed with the

following steps:

1. First, a random selection of members within the population is made. The members

with the highest ﬁtness will have the highest probability of being selected. The selected

members will be used for mating.

2. The parameters of each member are coded into genes given their lower and upper

13 of 29

bound. A gene is a binary representation of the parameter value. For example, if

the current value of a parameter is 1.6 and the lower bound and upper bound are

respectively 0 and 2, a binary representation using 8 bits will be 11001100. The lower

and upper bound will be 00000000 and 11111111, respectively. The higher the amount

of bits used for the binary presentation, the higher the resolution of possible parameter

values between the lower and upper bound.

3. For each parameter set two solutions are picked at random from the pool created in

step 1. These two solutions will mate and produce oﬀspring.

4. During the mating process, crossover between the genes will occur with a certain

probability, usually set to 0.7. The crossover location is chosen randomly. For example,

if we have two genes, 11001100 and 10101010, and a crossover location of 3, the two

oﬀspring will have genes given by 11001010 and 10101100.

5. Next, every bit in the genes of the oﬀspring will mutate with a certain mutation

probability. If a bit mutates, it changes from 0 to 1 or from 1 to 0. The mutation

probability is typically very low, e.g., 0.01.

6. The oﬀspring is decoded from binary codes into real values and their corresponding

ﬁtness is calculated.

7. A new population is chosen based on the ﬁtness of the oﬀspring and the original

population.

The genetic optimization is terminated when a speciﬁed number of iterations of steps 1

to 7 has been performed.

Due to the limited gene size that is commonly selected for encoding of parameters in

genetic algorithms to reduce the computational burden, the resulting parameter estimates

tend to be relatively inaccurate. In addition, genetic algorithms are not deterministic and

will give diﬀerent results every time they are run. Therefore, a genetic algorithm is believed

to be less suitable to be used as the sole estimation method for estimating pilot model

parameters from measurement data.

III.C. Unconstrained Gauss-Newton Optimization

The parameter sets that are obtained from the genetic likelihood optimization have a high

probability of being close to the global optimum of the optimization problem. To further

reﬁne these parameter estimates, the solutions of the genetic algorithm are used as the initial

parameter estimates for an unconstrained Gauss-Newton optimization. This gradient-based

14 of 29

optimization method is the classical approach to solving maximum likelihood estimation

problems.23, 29 The iterative parameter update equation for the Gauss-Newton optimization

is given by:

ˆ

Θ (i+ 1) = ˆ

Θ (i)−α(i)M−1

ΘΘ(ˆ

Θ (i))∂L(ˆ

Θ (i))

∂Θ.(17)

The gradients of the likelihood function with respect to all parameters, ∂L/∂Θ, can be

evaluated using the Jacobians of all state-space matrices with respect to the parameter vector

Θ. The Fisher information matrix, indicated with the symbol MΘΘ in Eq. (17), is given by:

MΘΘ =1

σ2

n

m

X

k=1 ∂ǫ(k)

∂Θ2

.(18)

The Fisher information matrix is symmetrical and should be positive deﬁnite, i.e., full

rank, as it needs to be inverted for the parameter update (Eq. (17)). The inverse of the

Fisher information matrix yields the Cram´er-Rao lower bound (CRLB), i.e., the minimum

achievable variance of the parameter estimate. Both the Fisher information matrix and the

ﬁrst-order gradient of the likelihood function vary with the magnitude of the estimation error

ǫ, yielding parameter update steps of larger magnitude if estimation errors are larger. In

some instances, this causes the Gauss-Newton optimization to become unstable if initial pa-

rameter errors are large: in that case the ﬁrst parameter update may force the unconstrained

algorithm to a region of meaningless and inaccurate solutions from which it cannot recover.

Using the solution of a genetic algorithm as proposed here as the initial parameter es-

timate for the Gauss-Newton optimization reduces the occurrence of such undesired eﬀects

signiﬁcantly. A second measure that can be taken to avoid this behavior is by considering

the line-search parameter αin Eq. (17). This parameter, which typically varies between 0

and 1, is determined at each iteration before the actual parameter update to ensure optimal

(i.e., most rapid) minimization of the likelihood function. This is illustrated in Figure 3,

where a typical variation of the likelihood Lis depicted as a function of α. For the example

shown in Figure 3 it is clear that α= 0.8 yields a superior parameter set after the update

step than would be obtained for α= 1.

The resolution that is considered for αduring the optimization – indicated with circles

in Figure 3 – can be selected freely and is highly dependent on the application. For the pilot

model estimation problem considered here, it was found that for the ﬁrst iterations a rather

coarse spacing of α, i.e. 0.1, already yielded enough added stability to the optimization

algorithm. When parameter estimates came closer to their optimal values, it was found that

small improvements could still be made for very small values of α. Therefore, a ﬁner spacing

of α, i.e. 0.01, was considered below 0.2 (see Figure 3).

15 of 29

α, -

-ln(L), -

0 0.2 0.4 0.6 0.8 1

500

600

700

800

900

1000

Figure 3. Typical likelihood variation along search vector.

IV. Results

In this section, the genetic maximum likelihood method will be evaluated using experi-

mental data. In addition, the parameter estimation results from the experimental data will

be used in multiple pilot model simulations with increasing remnant levels to investigate the

robustness of the method to increasing pilot remnant (measurement noise).

IV.A. Application to Experiment Data

The experiment was performed in the SIMONA Research Simulator of the Delft University

of Technology, Faculty of Aerospace Engineering. The objective of the experiment was to

investigate the eﬀects of pitch rotational motion and heave motion on multimodal pilot

control behavior in an aircraft pitch control task. The experiment had a full factorial design

with eight conditions with varying motion cues. In every condition, pitch rotational motion

and two distinct components of heave motion could be either on or oﬀ.

The genetic maximum likelihood method was used to estimate the parameters of the

pilot model using the experimental data. In this section, the performance results of the

genetic MLE method will be given for data of one subject for one of the experimental

conditions. In this condition, rotational pitch motion was available in addition to the cues

from a compensatory display. The subject for which the performance data is shown was

chosen randomly.

IV.A.1. Experimental Measurements

Seven subjects participated in the experiment. All had experience with similar manual

control tasks from previous human-in-the-loop experiments. Two subjects have additional

experience as aircraft pilots. The subjects ages ranged from 25 to 46 years old. Before

starting the experiment, the subjects received an extensive brieﬁng on the scope and objective

16 of 29

e, deg

(a) Error signal

0 10 20 30

-2

-1

0

1

2

u, deg

(b) Control signal

0 10 20 30

-8

-4

0

4

8

t, s

θ/ft, deg

(c) Pitch signal

pitch attitude θ

target ft

0 10 20 30

-3.0

-1.5

0.0

1.5

3.0

t, s

¨

θ, deg s−2

(d) Pitch acceleration signal

0 10 20 30

-40

-20

0

20

40

Figure 4. Average time traces of various loop signals (sub ject 1).

See, deg2/(rad s−1)

(a) Error spectrum

10-1 100101

10-8

10-6

10-4

10-2

100

102

Suu, deg2/(rad s−1)

(b) Control spectrum

spectrum

disturbance freq.

target freq.

10-1 100101

10-6

10-3

100

103

ω, rad s−1

Sθθ , deg2/(rad s−1)

(c) Pitch spectrum

10-1 100101

10-8

10-6

10-4

10-2

100

102

ω, rad s−1

S¨

θ¨

θ, deg2s−2/(rad s−1)

(d) Pitch acceleration spectrum

10-1 100101

10-6

10-4

10-2

100

102

104

Figure 5. Average loop signal spectra (subject 1).

17 of 29

of the experiment. The main instruction they received before the experiment was to attempt

to minimize the pitch tracking error, i.e., the signal that was presented on the visual display,

as best as possible.

The experiment started with a considerable number of training runs. The training phase

ended when a constant level of performance was achieved. After the training phase, 5

repetitions of every condition were performed in a random order, of which the data was used

in the ﬁnal results. Each individual experiment run lasted 110 seconds, of which the last

81.92 seconds were used as the measurement data. Participants generally need some time

to stabilize the disturbed aircraft model after the start of a run, therefore the data from the

ﬁrst 28.08 seconds of each run were discarded. Data was logged at a frequency of 100 Hz.

Further details of the experiment setup can be found in Reference 2.

Time traces of all signals depicted in Figure 1 were recorded during the experiment. To

reduce the noise content in the signals and improve pilot model parameter estimates, the time

traces of the ﬁve repetitions performed by each subject were averaged for the identiﬁcation

procedure.8Examples of averaged time traces of the error signal e, the control signal u, the

pitch attitude θand pitch acceleration ¨

θare depicted in Figure 4. The time trace of the

pitch target signal ftis shown alongside that of the pitch attitude for reference.

Note from Figure 4 that the subjects successfully made the aircraft pitch attitude follow

the target signal ft. Most of the additional oscillations in pitch attitude can be attributed

to the presence of the disturbance signal.

The power spectral densities of the time traces given in Figure 4 are given in Figure 5.

The input frequencies of the forcing functions, as given in Table 1, are indicated with circles

and triangles for fdand ft, respectively. The signal-to-noise ratio at the input frequencies is

high for all the signals in the control loop.

IV.A.2. Algorithm Performance

The error, pitch acceleration and control signals given in Figure 4 are used to estimate the

pilot model parameters with the genetic maximum likelihood method. Before the Gauss-

Newton optimization, 100 iterations with the genetic algorithm are performed. The parame-

ter lower bounds were set to zero and the upper bounds were taken positive and large enough

to account for all relevant solutions of the estimation problem (see Table 2). For the param-

eter conversion to binary strings, 20 bits are used. Values for the crossover and mutation

probabilities were set to 0.7 and 0.01, respectively. The solution of the Genetic algorithm

is used as the initial parameter set for the Gauss-Newton optimization. The Gauss-Newton

optimization is terminated if the derivative of the parameters with respect to the search

vector is smaller than 10−6.

Typically, 100 repetitions of the algorithm are performed on a single data set, of which

18 of 29

Table 2. Lower and upper parameter bounds used in the genetic algorithm.

KvTlead Tlag Kmτvτmωnm ζnm

- s s - s s rad s−1-

lower bound 0.0 0.0 0.0 0.0 0.0 0.0 5.0 0.0

upper bound 5.0 10.0 10.0 10.0 1.0 1.0 20.0 1.0

iteration #

-ln(L), -

(a) Total likelihood variation

Genetic Gauss-

Newton

Genetic optimization

Gauss-Newton optimization

0 25 50 75 100 125

0

2000

4000

6000

8000

(b) Gauss-Newton likelihood variation

iteration #

-ln(L), -

Genetic Gauss-

Newton

1×parameter set 2

9×parameter set 1

95 100 105 110 115 120

0

500

1000

1500

2000

2500

Figure 6. Changes in negative logarithm of the likelihood during optimization (10 repetitions,

subject 1).

the ten with the lowest likelihood are further evaluated using Gauss-Newton optimization.

The negative logarithm of the likelihood for these ten repetitions is given in Figure 6. It is

clear from Figure 6b that the solution of the genetic algorithm is diﬀerent for all repetitions.

After reﬁnement by the Gauss-Newton algorithm, two diﬀerent sets of parameters are found.

The solution with the lowest negative logarithm of the likelihood is found nine times. In

one instance, the optimization ends up in a diﬀerent (local) minimum, which clearly has a

higher likelihood.

This illustrates the nonlinearity of the optimization problem. For data from other sub-

jects, none to up to three of the obtained parameter sets were found to represent local minima.

Note that a local minimum was never attained more often than the global minimum.

To illustrate the signiﬁcance of this result, pilot model parameters were also estimated by

using only the Gauss-Newton maximum likelihood algorithm. To illustrate the large inﬂuence

of the initial parameter estimate on the results of this gradient-based optimization, a large

number of initial conditions were evaluated. For all parameters, a number of diﬀerent values

were selected between the upper and lower bounds deﬁned for the genetic algorithm. The

full factorial of all these initial parameter values yielded 40000 diﬀerent initial parameter

sets. Figure 7 depicts the likelihood of the ﬁnal parameter estimates obtained with the

Gauss-Newton algorithm for all these diﬀerent initial estimates, in ascending order.

The leftmost shaded area in Figure 7 indicates the set of initial conditions for which the

19 of 29

1436×parameter set 1

843×parameter set 2

initial parameter set #

-ln(L), -

0 1000 2000 3000 4000 5000 6000 7000

0

2000

4000

6000

8000

10000

Figure 7. Negative log-likelihood of Gauss-Newton parameter estimates for 7000 diﬀerent

initial parameter sets, sorted in ascending order (subject 1).

Gauss-Newton algorithm attained parameter estimate 1, as depicted in Figure 6; the second

shaded area, which corresponds to a solution with a higher likelihood, are the occurrences

for which parameter set 2 was found. All parameter sets that are outside either of the shaded

areas in Figure 7 represent initial estimates for which the algorithm was unable to converge.

Note that the Gauss-Newton algorithm only converged to the global minimum (parameter set

1) for around 20% of all tested initial conditions. For all other initial conditions, the Gauss-

Newton parameter estimation procedure would either converge to an incorrect minimum,

or fail to converge to a stable solution. As illustrated by Figure 6, the addition of the

genetic algorithm ensures reliable estimation of the parameter set that represents the global

minimum.

IV.A.3. Pilot Describing Functions

Figure 8 gives the linear pilot response functions Hpe and Hpθ for the two diﬀerent solutions

found by the algorithm depicted in Figure 6b. The Fourier coeﬃcients, which can be analyt-

ically calculated from the measured time-domain data, coincide with parameter set 1. This

is a further indication that parameter set 1 represents the correct solution of the parameter

estimation problem. The parameter values and the square root of the diagonal of the CRLB

resulting from the Gauss-Newton optimization are given in Table 3. Note that the CRLB is

found to be higher for nearly all parameter estimates of parameter set 2. From the param-

eter values, it can be seen that the gain for the vestibular channel Kmis much higher for

parameter set 1. In parameter set 2 the lower gain for the vestibular channel is compensated

for by the higher values for the lead and lag time constants. The vestibular time delay is

higher than the visual time delay for parameter set 2, which disagrees with ﬁndings from

previous research.2,6, 7 Finally, the neuromuscular frequency found for parameter set 1 is

20 of 29

higher than that obtained for parameter set 2.

|Hpe(jω)|, -

(a) Visual, magnitude

FC at ωd

FC at ωt

parameter set 1

parameter set 2

10-1 100101

10-1

100

101

|Hpθ(jω)|, -

(b) Motion, magnitude

10-1 100101

10-3

10-2

10-1

100

101

ω, rad s−1

6Hpe(jω), deg

(c) Visual, phase

10-1 100101

-540

-360

-180

0

180

ω, rad s−1

6Hpθ(j ω), deg

(d) Motion, phase

10-1 100101

-540

-360

-180

0

180

Figure 8. Bode plot of the pilot visual and motion response for parameter sets 1 and 2 (subject

1).

Table 3. Values and standard deviation for parameter sets 1 and 2 (subject 1).

parameter parameter set 1 parameter set 2

ˆ

ΘqCRLB( ˆ

Θ) ˆ

ΘqCRLB( ˆ

Θ)

Kv, - 0.938 6.77·10−30.932 8.91·10−3

Tlead, s 0.278 6.47·10−30.337 9.21·10−3

Tlag, s 0.502 1.92·10−20.774 3.08·10−2

Km, - 0.692 1.29·10−20.147 1.11·10−2

τv, s 0.268 2.79·10−30.134 2.41·10−3

τm, s 0.175 1.94·10−30.679 1.29·10−2

ωnm, rad s−111.560 6.41·10−29.775 7.15·10−2

ζnm, - 0.175 4.60·10−30.168 9.56·10−3

The measured pilot control signal is compared with the model output for the two param-

eter solutions in Figure 9. It can be seen that the diﬀerence between both solutions in the

time domain is only very small. The variance accounted for (VAF), which is a measure of

the correlation of two time-domain signals,8is only marginally higher for parameter set 1.

Figure 10 gives the pilot frequency response functions corresponding to the optimal pa-

rameter estimate for all subjects for the experimental condition used in this paper. From

21 of 29

t, s

u, deg

measurement

parameter set 1: VAF = 85.30%

parameter set 2: VAF = 82.05%

0 2 4 6 8 10

-2

-1

0

1

2

Figure 9. Comparison of model control signals for parameter sets 1 and 2 (subject 1).

this ﬁgure it can be seen that all subjects give consistent results. The response functions

vary slightly between subjects as every individual has a diﬀerent control strategy, which is

commonly seen in this type of experiments.

IV.A.4. Pilot Remnant

The pilot remnant is determined by the diﬀerence between the model output of the linear

pilot model and the measured pilot control signal, as depicted in Figure 9. A time trace of

the pilot remnant for the model with parameter set 1 is given in Figure 11a. The power

spectrum of this signal is given in Figure 11b. It can be seen that the spectrum has the

approximate characteristics of a third-order low-pass ﬁlter with a damping coeﬃcient:

Hn(jω) = Knω3

n

((jω)2+ 2ζnωnjω +ω2

n) (jω +ωn),(19)

with Kn= 2.6 the remnant intensity and ζn= 0.26 and ωn= 12.7 rad/s the remnant

ﬁlter damping factor and break frequency, respectively. Similar characteristics are found in

previous research.8

The probability density function of the pilot remnant is given in Figure 12. This ﬁgure

clearly shows that the remnant signal has a distribution that is almost perfectly Gaussian

with a zero mean, as is assumed in Eq. (13). The low-pass remnant spectrum has no eﬀect

on the normality of the signal. The fact that the remnant is distinctly non-white, as shown

in Figure 11b, may cause extra bias in the pilot model parameter estimates, which has been

further investigated using oﬀ-line simulations.

IV.B. Oﬀ-line Simulations

To investigate the robustness of the genetic MLE method to increasing levels of pilot remnant,

parameter set 1 is used to perform multiple simulations of the closed-loop control task with

22 of 29

|Hpe(jω)|, -

(a) Visual, magnitude

10-1 100101

10-2

10-1

100

101

102

|Hpθ(jω)|, -

(b) Motion, magnitude

10-1 100101

10-2

10-1

100

101

102

ω, rad s−1

6Hpe(jω), deg

(c) Visual, phase

subject 1

subject 2

subject 3

subject 4

subject 5

subject 6

subject 7

10-1 100101

-540

-360

-180

0

180

ω, rad s−1

6Hpθ(j ω), deg

(d) Motion, phase

10-1 100101

-540

-360

-180

0

180

Figure 10. Bode plot of the pilot visual and motion response for all experiment subjects.

t, s

n, deg

(a) Remnant time history

0 10 20 30 40 50 60 70 80

-4

-2

0

2

4

ω, rad s−1

|N|, dB

(b) Remnant spectrum

remnant spectrum

remnant at ωd

remnant at ωt

remnant model

10-1 100101

-80

-60

-40

-20

0

Figure 11. Pilot remnant characteristics (sub ject 1).

23 of 29

n, deg

f(n), -

measured remnant f(n)

normal f(n|µ= 0, σ= 0.936 deg)

-4 -3 -2 -1 0 1 2 3 4

0

500

1000

1500

Figure 12. Remnant probability density function (subject 1).

increasing levels of remnant. The remnant ﬁlter break frequency ωnand damping factor ζn

are kept constant and the remnant intensity Knis gradually increased to achieve the desired

ratio between control signal power and remnant power. The ratio is increased from 0 to 0.5,

while the value found for the averaged experimental data is approximately 0.15 for parameter

set 1. For each remnant level, eighty simulations are performed, each with a diﬀerent noise

realization.

Figure 13 shows the average parameter bias found by applying the genetic MLE procedure

to the data of each simulation. Also, the 99% conﬁdence interval for each parameter is

indicated in this ﬁgure. It can be seen that for increasing levels of pilot remnant, the average

bias of the parameter estimates increases. For all parameters, the average bias remains

within the 99% conﬁdence interval, which indicates that the bias is not signiﬁcant. At a

remnant-control signal power ratio of 0.15, found experimentally, the bias is relatively small

for all parameters.

V. Discussion

This paper presents a new strategy for the estimation of multi-channel pilot model pa-

rameters from time-domain data based on maximum likelihood estimation. The classical

MLE method, with only a gradient-based algorithm to estimate the parameters, is very

dependent on the initial parameter guess. This is highly undesirable given the fact that

the cost function is highly nonlinear and contains many local minima. For this reason, a

genetic algorithm was used in combination with the gradient-based Gauss-Newton optimiza-

tion procedure. The genetic algorithm does not require an initial parameter guess and is far

less sensitive to local minima, as a population of possible solutions is kept until the algo-

rithm terminates. The subsequent use of a Gauss-Newton algorithm increases the accuracy

of the parameter estimate, as gradient information is included. The method was successfully

24 of 29

bias Kv, -

σ2

n/σ2

u, -

(a) visual perception gain

0 0.1 0.2 0.3 0.4 0.5

-1.0

-0.5

0.0

0.5

1.0

bias Tlead, s

σ2

n/σ2

u, -

(b) visual lead constant

0 0.1 0.2 0.3 0.4 0.5

-0.2

-0.1

0.0

0.1

0.2

bias Tlag , s

σ2

n/σ2

u, -

(c) visual lag constant

0 0.1 0.2 0.3 0.4 0.5

-0.6

-0.4

-0.2

0.0

0.2

0.4

0.6

bias τv, s

σ2

n/σ2

u, -

(d) visual perception time delay

0 0.1 0.2 0.3 0.4 0.5

-0.1

-0.1

0.0

0.1

0.1

bias Km, -

σ2

n/σ2

u, -

(e) motion perception gain

0 0.1 0.2 0.3 0.4 0.5

-2.0

-1.0

0.0

1.0

2.0

bias τm, s

σ2

n/σ2

u, -

(f ) motion perception time delay

0 0.1 0.2 0.3 0.4 0.5

-0.04

-0.02

0.00

0.02

0.04

bias ζnm, -

σ2

n/σ2

u, -

(g) neuromuscular damping

0 0.1 0.2 0.3 0.4 0.5

-0.3

-0.2

-0.1

0.0

0.1

0.2

0.3

bias ωnm, rad s−1

σ2

n/σ2

u, -

(h) neuromuscular frequency

99 % conf.int.

mean bias

0 0.1 0.2 0.3 0.4 0.5

-4.0

-2.0

0.0

2.0

4.0

Figure 13. Average parameter bias for diﬀerent remnant levels.

25 of 29

applied to data from an experiment investigating the inﬂuence of pitch and heave motion

cues on pilot control behavior.

Applying the genetic maximum likelihood algorithm to the experiment data multiple

times for all subjects resulted in the occurrence of solutions other than the global optimum.

These occurrences were infrequent, however, and the corresponding solutions always have a

higher likelihood than the global minimum. Hence, the optimization should be performed

several times for each condition from an experiment to ensure that the global optimum

solution is found. The often subtle diﬀerences between the parameter sets of a local or

a global optimum, and the equivalence of the resulting time-domain ﬁts make it diﬃcult

to distinguish between these solutions based on this information alone. Although, Fourier

coeﬃcients can provide an indication of the frequency response of the global optimum, these

can only be computed when multi-sine signals are used as forcing functions.

Simulations have been performed to investigate the robustness of the MLE method to

increasing levels of remnant. It was found that the average bias of the parameters increases

for increasing levels of remnant power. The bias remained within the 99% conﬁdence in-

terval of the parameter estimate, which indicates that the bias is not signiﬁcant. For the

parameter estimation on averaged time-domain data of ﬁve runs (σ2

n/σ2

u≈0.15) the bias of

the parameter estimates is relatively small. When individual runs (σ2

n/σ2

u≈0.30) are used

in the parameter estimation procedure, the bias becomes more substantial, however.

As maximum likelihood estimates have the property to be unbiased for inﬁnite mea-

surement times, the increase in bias for higher levels of remnant is an unexpected result.

Although this bias is partly caused by the ﬁnite measurement times, the fact that the rem-

nant is colored rather than purely Gaussian white noise may cause an additional bias. Also,

because of a correlation between the pilot model input and output in the closed loop – the

remnant, which is part of the output of the pilot model, travels around and is also present

in the input of the pilot model – an additional bias may be present. A solution for both

problems would involve inclusion of process noise into the state-space representation of the

pilot model, which requires a prediction-error algorithm to be added to the MLE method.

This allows for the properties of the remnant ﬁlter to be estimated, and could reduce the bias

in the parameter estimates at higher levels of remnant. The Kalman ﬁlter that is needed

to solve this problem, however, will increase the complexity of the parameter estimation

procedure and will signiﬁcantly increase the computational burden.

In this paper, the genetic MLE method is evaluated with data from an experiment where

multi-sine signals were used as forcing functions. Parameter estimation in the time domain

also allows the use of novel types of input signals, however, such as ramp or step signals that

are often considered in other system identiﬁcation problems. These signals can be used as

target signal and may yield a control task that is much more similar to real piloting tasks. An

26 of 29

investigation on the use of these signals is already performed,24 but more research is needed to

evaluate the performance of the MLE method with these signals. The maximum likelihood

parameter estimation algorithm can also be used to identify multi-channel pilot models

when only one forcing function is present, as long as all the inputs to the pilot model are

suﬃciently excited. The method has already been successfully applied to measurement data

from human-in-the-loop experiments where pure target-following tasks were considered.31, 32

Finally, the use of a genetic algorithm allows for the introduction of nonlinear elements

into the pilot model, as no gradient information is required. An example can be the addition

of motion perception thresholds. This will also be a subject of future research.

VI. Conclusions

A pilot model parameter estimation technique, based on the maximum likelihood method,

was introduced and enhanced with a genetic algorithm to considerably improve performance.

The new method was successfully applied to data from an experiment investigating the role

of pitch and heave motion during pitch control of an aircraft. The parameters of the multi-

channel pilot model and the properties of the remnant could be accurately estimated from

the time-domain signals. Using multiple iterations of the algorithm on a single data set,

the global optimum solution was found for 90% of the cases. Using multiple simulations

with diﬀerent levels of pilot remnant, it was shown that there is no signiﬁcant bias in the

parameters for lower remnant levels. For high levels of pilot remnant bias is more substantial,

caused by the fact that remnant tends to be colored rather than purely white noise.

Acknowledgments

This research was supported by the Technology Foundation STW, the applied science

division of NWO, and the technology program of the Ministry of Economic Aﬀairs. The

authors would like to thank Tom Berger, currently aﬃliated with the NASA Ames Research

Center, for his initial investigation into maximum likelihood estimation of pilot model pa-

rameters.

References

1Hosman, R. J. A. W., “Are Criteria for Motion Cueing and Time Delays Possible?” Proceedings of the

AIAA Modelling and Simulation Technologies Conference and Exhibit, Portland (OR), No. AIAA-1999-4028,

1999.

2Zaal, P. M. T., Pool, D. M., De Bruin, J., Mulder, M., and Van Paassen, M. M., “Use of Pitch and

Heave Motion Cues in a Pitch Control Task,” Journal of Guidance, Control and Dynamics, Accepted for

27 of 29

publication, November 2008.

3Stapleford, R. L., McRuer, D. T., and Magdaleno, R., “Pilot Describing Function Measurements in a

Multiloop Task,” IEEE Transactions on Human Factors in Electronics, Vol. HFE-8, No. 2, 1967, pp. 113–

125.

4Stapleford, R. L., Peters, R. A., and Alex, F. R., “Experiments and a Model for Pilot Dynamics with

Visual and Motion Inputs,” Tech. Rep. NASA CR-1325, NASA, 1969.

5Hess, R. A., “Model for Human Use of Motion Cues in Vehicular Control,” Journal of Guidance,

Control, and Dynamics, Vol. 13, No. 3, 1990, pp. 476–482.

6Van der Vaart, J. C., Modelling of Perception and Action in Compensatory Manual Control Tasks,

Doctoral dissertation, Faculty of Aerospace Engineering, Delft University of Technology, 1992.

7Hosman, R. J. A. W., Pilot’s Perception and Control of Aircraft Motions, Doctoral dissertation,

Faculty of Aerospace Engineering, Delft University of Technology, 1996.

8Nieuwenhuizen, F. M., Zaal, P. M. T., Mulder, M., van Paassen, M. M., and Mulder, J. A., “Modeling

Human Multichannel Perception and Control Using Linear Time-Invariant Models,” Journal of Guidance,

Control, and Dynamics, Vol. 31, No. 4, July–Aug. 2008, pp. 999–1013.

9Ellerbroek, J., Stroosma, O., Mulder, M., and van Paassen, M. M., “Role Identiﬁcation of Yaw

and Sway Motion in Helicopter Yaw Control Tasks,” Journal of Aircraft, Vol. 45, No. 4, July–Aug. 2008,

pp. 1275–1289.

10McRuer, D. T. and Jex, H. R., “A Review of Quasi-Linear Pilot Models,” IEEE Transactions on

Human Factors in Electronics, Vol. HFE-8, No. 3, 1967, pp. 231–249.

11Van Paassen, M. M. and Mulder, M., “Identiﬁcation of Human Operator Control Behaviour in

Multiple-Loop Tracking Tasks,” Proceedings of the Seventh IFAC/IFIP/IFORS/IEA Symposium on Anal-

ysis, Design and Evaluation of Man-Machine Systems, Kyoto Japan, Pergamon, Kidlington, Sept. 16–18

1998, pp. 515–520.

12Stepner, D. E. and Mehra, R. K., “Maximum Likelihood Identiﬁcation and Optimal Input Design for

Identifying Aircraft Stability and Control Derivatives,” Tech. Rep. NASA CR-2200, National Aeronautics

and Space Administration, Washington, D. C., 1973.

13Taylor, L. W., “A Comparison of Human Response Modeling in the Time and Frequency Domains,”

Third Annual NASA-University Conference on Manual Control, 1967, pp. 137–156.

14Shirley, R. S., “A Comparison of Techniques for Measuring Human Operator Frequency Response,”

Proceedings of the Sixth Annual Conference on Manual Control, 1970, pp. 803–870.

15Phatak, A. V., R. K. Mehra, R., and Day, C., “Application of System Identiﬁcation to Modeling

the Human Controller under Stress Conditions,” IEEE Transactions on Automatic Control, Vol. 20, No. 5,

October 1975, pp. 657 – 664.

16Ninz, N. R., “Parametric Identiﬁcation of Human Operator Models,” Proceedings of the Sixteenth

Annual Conference on Manual Control, 1980, pp. 137–145.

17Schmidt, D. K., “Time Domain Identiﬁcation of Pilot Dynamics and Control Strategy,” Eighteenth

Annual Conference on Manual Control, 1982, pp. 19–40.

18Zaychik, K. B. and Cardullo, F. M., “Genetic Algorithm Based Approach for Parameters Estimation

of the Hess Operator Model,” Proceedings of the AIAA Modeling and Simulation Technologies Conference

and Exhibit, Hilton Head (SC), No. AIAA-2007-6893, 2007.

28 of 29

19Maine, R. and Iliﬀ, K., “AGARD Flight Test Techniques Series Volume 3 on Identiﬁcation of Dynamic

Systems – Applications to Aircraft Part 1: The Output Error Approach,” AGARDograph No. 300 AGARD-

AG-300-VOL.3 PART I, North Atlantic Treaty Organization (NATO), 1986.

20Chu, Q. P., Mulder, J. A., and van Woerkom, P. T. L. M., “Modiﬁed Recursive Maximum Likelihood

Adaptive Filter for Nonlinear Aircraft Flight-Path Reconstruction,” Journal of Guidance, Control, and

Dynamics, Vol. 19, No. 6, Nov.–Dec. 1996, pp. 1285–1295.

21Milne, G., Mulder, J. A., Soijer, M. W., Juliana, S., and Hermansyah, “Maximum Likelihood Stability

& Control Derivative Identiﬁcation of a Cessna Citation II,” Proceedings of the AIAA Atmospheric Flight

Mechanics Conference and Exhibit, Montreal (Ca), No. AIAA-2001-4013, 2001.

22Mehra, R. K. and Prasanth, R. K., “Time-Domain System Identiﬁcation Methods for Aeromechanical

and Aircraft Structural Modeling,” Journal of Aircraft, Vol. 41, No. 4, July-August 2004, pp. 721–729.

23Chu, Q. P., Maximum Likelihood Parameter Identiﬁcation of Flexible Spacecraft, Ph.D. thesis, Delft

University of Technology, 1987.

24Zaal, P. M. T., Pool, D. M., Mulder, M., and van Paassen, M. M., “New Types of Target Inputs for

Multi-Modal Pilot Model Identiﬁcation,” Proceedings of the AIAA Modeling and Simulation Technologies

Conference and Exhibit, Honolulu (HI), No. AIAA-2008-7106, 18–21 Aug. 2008.

25L¨ohner, C., Mulder, M., and van Paassen, M. M., “Multi-Loop Identiﬁcation of Pilot Central Visual and

Vestibular Motion Perception Processes,” Proceedings of the AIAA Modeling and Simulation Technologies

Conference and Exhibit, San Francisco (CA), No. AIAA-2005-6503, 15–18 Aug. 2005.

26Pool, D. M., Mulder, M., van Paassen, M. M., and van der Vaart, J. C., “Eﬀects of Peripheral Visual

and Physical Motion Cues in Roll-Axis Tracking Tasks,” Journal of Guidance, Control, and Dynamics,

Vol. 31, No. 6, 2008, pp. 1608–1622.

27Ljung, L., System Identiﬁcation Theory for the User, Prentice Hall, Inc., 2nd ed., 1999.

28Berger, T., Zaal, P. M. T., Mulder, M., and van Paassen, M. M., “Time Domain Pilot Model Identi-

ﬁcation Using Maximum Likelihood Estimation,” Proceedings of the AIAA Modeling and Simulation Tech-

nologies Conference and Exhibit, Honolulu (HI), No. AIAA-2008-7109, 18–21 Aug. 2008.

29Mulder, J. A., Design and Evaluation of Dynamic Flight Test Manoeuvres, Ph.D. thesis, Delft Uni-

versity of Technology, 1986.

30Abutaleb, A. S., “A Genetic Algorithm for the Maximum Likelihood Estimation of the Parameters of

Sinusoids in a Noisy Environment,” Circuits Systems Signal Processing, Vol. 16, No. 1, 1997, pp. 69–81.

31Pool, D. M., Zaal, P. M. T., Mulder, M., van Paassen, M. M., and Mulder, J. A., “Parameter Estimation

of Multimodal Pilot Models for Manual Target-following Tasks,” Proceedings of the 27th European Annual

Conference on Human Decision-Making and Manual Control, June 12–13 2008.

32Zaal, P. M. T., Mulder, M., van Paassen, M. M., and Mulder, J. A., “Maximum Likelihood Estimation

of Multi-Modal Pilot Control Behavior in a Target-Following Task,” Proceedings of the IEEE International

Conference on Systems, Man and Cybernetics, 2008., Oct. 12–15 2008.

29 of 29