Content uploaded by Umberto Saetti
Author content
All content in this area was uploaded by Umberto Saetti on Feb 18, 2020
Content may be subject to copyright.
Probabilistic Assessment of Pilot-Vehicle System Performance and Perceived
Vehicle Handling Qualities
Umberto Saetti
Postdoctoral Fellow
School of Aerospace Engineering
Georgia Institute of Technology
Atlanta, GA 30313
Jonathan Rogers
Associate Professor
School of Aerospace Engineering
Georgia Institute of Technology
Atlanta, GA 30313
ABSTRACT
This paper proposes a novel mathematical methodology for probabilistic handling qualities evaluation using the Koop-
man operator. The Koopman operator is used to quantify the impact of parametric uncertainty in a pilot model to
specific pilot-vehicle system performance specifications and to perceived vehicle handling qualities. The considered
specifications include the Hover Mission Task Element (MTE), broken loop gain and phase margins, and pilot opinion
ratings of workload. The application to each specification is illustrated through an example involving the UH-60 heli-
copter. Results demonstrate the utility of the proposed method in mapping uncertainty in the pilot model parameters
to uncertainty in handling qualities evaluations.
INTRODUCTION
A key requirement in the certification of a new or modified
rotorcraft is that the aircraft meet specific handling qualities
(HQ), performance, and stability requirements. In a typical
certification process, evaluation of these criteria involves the
use of pilot opinion ratings of the aircraft according to the
Cooper-Harper rating scale (Ref. 1) for various Mission Task
Elements (MTEs) (Ref. 2). However, inter/intra pilot vari-
ability in opinion rating exists due to pilot physiological dif-
ferences and differing levels of aggressiveness. Mathematical
pilot models that account for such differences have been de-
veloped in the past (Refs. 3–5). Particularly, the parameter-
ized pilot model proposed in Ref. 5accounts for differences
in both level of aggressiveness and central nervous and neu-
romuscular systems. When combined with uncertainty quan-
tification (UQ) techniques, these models may be exploited to
assess how the uncertainty in level of aggressiveness and pilot
physiological characteristics propagates to the pilot-vehicle
system performance and perceived vehicle handling qualities.
In this manner, assessments can be made as to the probability
that a vehicle conforms to specific handling qualities, given
the uncertainty in the pilot-vehicle closed-loop dynamics.
Pilot-vehicle system performance variability and pilot rating
variability have been the subject of a significant body of prior
work (Refs. 3,6–8). Likewise, some prior research investi-
gating probabilistic handling qualities evaluation techniques
has been documented (Refs. 9–12). However, to the au-
thors’ knowledge there has been no prior attempt to bridge
the gap between variability in pilot models and variability
in pilot-vehicle system performance and/or perceived vehicle
Presented at the 2020 Rotorcraft Handling Qualities Technical Meet-
ing, Huntsville, AL, Feb 19–19, 2020. Copyright c
2020 by the
Vertical Flight Society. All rights reserved.
handling qualities. In light of this, there is a need for computa-
tionally efficient methodologies that can map the uncertainty
in the pilot model parameters to the resulting uncertainty in
handling qualities evaluations. Further, all the cited research
efforts investigating the effects of parametric uncertainty on
handling qualities use uncertainty quantification methods that
suffer from poor scalability. In the UQ domain, explicit uncer-
tainty quantification approaches, particularly the Frobenius-
Perron (FP) Operator (Ref. 13), are used to map a probability
density explicitly through a nonlinear transformation with im-
proved scalability characteristics with respect to Monte Carlo.
Recently, Meyers et al. (Ref. 14) applied the FP opera-
tor and its adjoint, the Koopman operator, to various optimal
decision-making problems under uncertainty, demonstrating
the scalability benefits of these explicit techniques compared
to Monte Carlo. For this reason, the Koopman operator is
chosen as the preferred method for this study.
This paper presents a novel methodology for computing prob-
abilistic handling qualities performance assessments using the
Koopman operator. The algorithm begins with a discretiza-
tion of the uncertainty space comprised of the uncertain pilot
model parameters. Each discretized point is mapped through
relevant transformations to the handling qualities evaluation
domain, resulting in a handling qualities performance assess-
ment for each discretized point (similar to Monte Carlo, ex-
cept usually with significantly less points). Using this map-
ping, the handling qualities evaluation is pulled back to the
initial uncertainty domain using the Koopman operator. The
expected value of the handling qualities performance is then
computed through an expected value integral with the initial
joint PDF, yielding a probabilistic assessment with respect
to relevant specifications. Following a detailed discussion of
the methodology and its computational implementation, ex-
1
(a)
(b)
Figure 1: (a) Forward Mapping of Uncertainty Domain to
HQ/Performance Domain (b) Pull-Back of HQ Evaluation
Score to Uncertainty Domain Using Koopman Operator.
amples are presented for a modified UH-60 helicopter model.
The paper begins with an explanation of the mathematical
methodology behind probabilistic handling qualities evalua-
tion using the Koopman operator. In the second section, the
mathematical framework for a quasi-nonlinear rotorcraft sim-
ulation model is defined. The third section describes the im-
plementation of a multi-axis structural pilot model for holding
position and heading in hover. The fourth and fifth sections
applies the Koopman operator to the propagation of the un-
certain parameters in the pilot model to specific pilot-vehicle
system performance specifications and to perceived vehicle
handling qualities including the Hover MTE, broken loop gain
and phase margins, and pilot opinion ratings of workload. For
each type of specification, the relevant state transformation
and cost function definition(s) is identified. Each is illustrated
through an example involving the UH-60 helicopter. These
results demonstrate the utility of the proposed method in map-
ping uncertainty in the pilot model parameters to uncertainty
in handling qualities evaluations. Final remarks summarize
the overall findings of the study and identify areas for future
work.
METHODOLOGY
Define a closed-loop pilot-rotorcraft system, with state vector
~xand parameterized by ~p∈Rpaccording to:
˙
~x=~
f(~x,~p,~u)(1)
where ~uis the vector of controls. A new augmented state
is defined by combining the rigid body state with uncertain
model parameters, and any other components needed to eval-
uate HQ ratings. Define the set of uncertain model parameters
as ~pu∈Rdwhere ~pu⊂~pand dis the dimension of the uncer-
tainty space. Likewise, ~pk∈Rp−dis the vector of known pa-
rameters. The augmented state is then given as ¯x={~x,~pu}T.
Suppose that a map exists that such that, for each unique ini-
tial state vector ¯x0, a state vector at a final time ¯xfis generated
that may be used to evaluate handling qualities:
¯xf=S(¯x0)(2)
In Eq. 2, the transformation Smay represent various types
of algebraic or dynamic transformations. For instance, in the
evaluation of ADS-33E performance standards, Mission Task
Elements (MTEs) are used to categorize flight performance
into Desired, Adequate, or Inadequate performance (or equiv-
alently, into Level 1, Level 2, or Level 3 performance). In
this case, evaluation of this specification requires executing
the MTE from the prescribed initial condition using a se-
lected value of ~p. Thus, Smay involve propagation of a flight
dynamic model from the prescribed initial condition for the
MTE to the prescribed final condition, with the final state
being ¯xf. Other handling qualities, performance, or stabil-
ity metrics may involve other types of dynamic or algebraic
transformations Sto obtain a vector ¯xf. In subsequent discus-
sion, this final state vector will be referred to as ¯xeval =¯xffor
clarity, to represent the state vector on which the HQ ratings
can be evaluated.
Given this mapping, a “score” function gis defined that as-
signs a numerical score based on ¯xeval representing the han-
dling qualities or performance specification associated with
the forward mapping of the augmented state ¯x0. This score
function value may be “pulled back” to the initial condition
domain of the augmented state through use of the Koopman
operator defined as,
USg=USg(¯xeval) = g(S(¯x0)) (3)
The only restriction on gis that it must be finite, i.e., ||g∞|| <
∞. The score or cost function value is associated with a
certain handling qualities criterion – for instance, a value of
g(¯xeval ) = 1 may be assigned for ¯xeval in a particular region
associated with Desired handling qualities, g(¯xeval ) = 2 for
regions associated with Adequate handling qualities, and so
on. Note that the Koopman operator USin Eq. 3is defined
specific to a particular transformation S.
This overall process is illustrated in Fig. 1. In Fig. 1(a),
each discretized point in the uncertainty domain is mapped to
a specific numerical value in the HQ or performance domain
using the forward mapping S. In Fig. 1(b), the score/cost
function associated with each forward-mapped point is pulled
back to the original uncertainty domain, yielding USg. With
this pulled-back score function, the expected value of the han-
dling qualities assessment function gcan be computed given
the joint probability density function on the uncertain param-
eters ~puaccording to:
E[g(~x,~p)] = ZΩ
~
fpu(~pu)USg(¯xeval )d~pu(4)
where Ωis the support of the joint probability density of the
uncertain parameters, ~
fpu(~pu). This expected value represents
2
the expected handling qualities evaluation score for this par-
ticular metric signified by the cost or score function g. For
instance, in the example shown in Fig. 1, calculation of Eq.
4would provide a value between 0 and 1, representing the
probability of the vehicle achieving the Desired level of per-
formance for the particular handling qualities or performance
metric. Note that by defining several binary score functions
for each handling qualities ranking, probability values can be
generated that specify the probability that the vehicle will con-
form to each level. This represents a generalization of the
chance constraint techniques detailed in Refs. 14,15.
ROTORCRAFT MODEL
This study uses an 14-state model representative of the flight
dynamics of a UH-60 in hover. The stability and control
derivatives and trim data are taken from Ref. 16. The deriva-
tives and trim data are combined with nonlinear equations of
motion to produce a quasi-nonlinear simulation model (Ref.
17). Consider the dynamics of the flight-identified linear
model from Ref. 16:
˙
x6
˙
xH=A6A6H
AH6 AHH x6
xH+B6
BHu(5)
where:
x6are the states relative to the rigid-body dynamics,
xHare the nHstates relative to the higher-order dynamics,
A66 contains the rigid-body stability derivatives,
A6H contains the higher-order to rigid-body stability deriva-
tives,
AH6 contains the rigid-body to higher-order stability deriva-
tives,
AHH contains the higher-order stability derivatives,
B6contains the rigid-body control derivatives,
BHcontains the higher-order control derivatives, and
uis the control vector.
The two portions of the state vector are:
xT
6=uvwpqr(6a)
xT
H=β1cβ1s(6b)
where:
u,v,ware the body velocities,
p,q,rare the body angular rates, and
β1c,β1sare the longitudinal and lateral flapping angles.
The control vector is:
u=δlat δlon δcol δped(7)
where:
δlat,δlon are the lateral and longitudinal cyclic inputs,
δcol is the collective input, and
δped is the pedal input.
In order to calculate the aerodynamic perturbation forces and
moments, it is necessary to obtain the state and control pertur-
bations:
∆x6=X6−[X6]0(8a)
∆xH=XH−[XH]0(8b)
∆u=U−U0(8c)
where:
X6are the nonlinear simulation rigid-body states,
[X6]0are the trim rigid-body states in hover,
XHare the nonlinear simulation higher-order states,
[XH]0are the trim higher-order states in hover,
Uis the nonlinear simulation control vector, and
U0is the trim control vector in hover.
It is worth noting that the rigid-body perturbation vector does
not include Euler angles or the position since the aerodynamic
forces and moments do not depend explicitly on the Euler an-
gles or the position (Ref. 17). The aerodynamic perturbation
forces and moments are:
∆Faero
∆Maero=M(A66 ∆x6+A6H∆xH+B6∆u)(9)
where:
M=
m
m
m
Ixx −Ixz
Iyy
−Ixz Izz
(10)
is the mass matrix containing the mass mand inertia terms
Ixx,Iyy,Izz,Ixz for the helicopter. The aerodynamic trim forces
and moments, based on the trim Euler angles Φ0,Θ0in hover,
are:
Faero0=
gsinΘ0
−gcosΘ0sinΦ0
−gcosΘ0cosΦ0
m(11)
The nonlinear simulation gravitational forces, which are non-
linear with respect to the Euler angles, are:
Fgrav =
−gsinΘ
gcosΘsinΦ
gcosΘ0cosΦ
m(12)
The aerodynamic perturbation forces and moments are
summed with the trim and gravitational forces and moments
to obtain the total aerodynamic forces and moments:
F
M=∆Faero +Faero0+Fgrav
∆Maero (13)
These forces and moments are incorporated in the nonlinear
equations of motion in the body-fixed frame to provide the
fuselage 6-DOF dynamics:
˙
U=RV −QW +Fx
m(14a)
3
˙
V=PW −RU +Fy
m(14b)
˙
W=QU −PV +Fz
m(14c)
˙
P=Izx(Ixx −Iyy +Izz)PQ −(I2
zz −IyyIzz +I2
xz)QR +IzzL+IxzN
IxxIzz −I2
xz (14d)
˙
Q=(Izz −Ixx)PR −Ixz(P2−R2) + M
Iyy
(14e)
˙
R=(I2
xx −IxxIyy +I2
xz)PQ −Izx(Ixx −Iyy +Izz)QR +IxzL+Ixx N
IxxIzz −I2
xz (14f)
˙
Φ=P+QsinΦtanΘ+RcosΦtanΘ(14g)
˙
Θ=QcosΦ−RsinΦ(14h)
˙
Ψ=QsinΦsecΘ+RcosΦsecΘ(14i)
˙
X=UcosΘcosΨ+V(sinΦsinΘcosΨ−cosΦsinΨ)
+W(cosΦsinΘcosΨ+sinΦsinΨ)(14j)
˙
Y=UcosΘsinΨ+V(sinΦsinΘsinΨ+cosΦcosΨ)
+W(cosΦsinΘsinΨ−sinΦcosΨ)(14k)
˙
Z=−UsinΘ+Vsin Φcos Θ+Wcos Φcos Θ(14l)
The higher-order dynamics are given by:
˙
B1c
˙
B1s=˙
xH=AH6x6+AHH xH+BH∆u(15)
STRUCTURAL PILOT MODEL
The human pilot is modeled using a Structural Pilot Model
taken from Ref. 4. The closed-loop pilot-rotorcraft system for
a multi-loop tracking task is shown in Fig. 2. The individual
blocks in Fig. 2are given by:
YP
n=ω2
n
s2+2ζnωn+ω2
n
(16a)
Yf=K1s
s+1
T1
(16b)
Ym=K2s
s+1
T2k−1(16c)
where:
Table 1: Structural pilot model inner loop parameters.
Axis ωc[rad/s] T2[sec] Ke
Lateral 2.5 0.246 36.69 [in/rad]
Longitudinal 2 1.534 64.28 [in/rad]
Directional 1 3.238 17.02 [in/rad]
Vertical 1 3.572 -1.331 [in/ft]
YP
nis the open-loop neuromascular dynamics,
Yfand Ymemulate the dynamics associated with muscle
spindles, Golgi tendon organs, and higher-level signal pro-
cessing,
ωnand ζnare the natural frequency and damping ratio as-
sociated with the open-loop neuromuscular dynamics,
Ycis the aircraft dynamics,
YPis the pilot dynamics,
Kedefines the pilot level of aggressiveness,
τ0is the delay in visual process and motor nerves conduc-
tion times,
Kyand TLdescribe the outer-tracking loop pilot compensa-
tion, and
K1,K2,T1,T2,k, are other parameters defined in Ref. 4.
It is worth noting that the pilot workload, for which a measure
is the root mean square (RMS) of the ratio between the output
of Ymand the pilot gain Ke(i.e. um/Ke), can be related through
the Cooper-Harper rating scale to pilot opinion ratings (Refs.
4,18).
A multi-axis pilot model is created around the unaugmented
(i.e. no automatic flight control system) aircraft. Inner atti-
tude and outer position loops are implemented for the roll and
pitch axes to track lateral and longitudinal positions, respec-
tively. Single loop strategies are implemented for the direc-
tional and vertical axes to track heading and vertical position,
respectively. The structural pilot model parameters are found
using the procedure in Ref. 4. The inner loop crossover fre-
quencies ωcchosen for the open-loop pilot-vehicle system are
reported in Table 1. To achieve crossover model characteris-
tics in the region of crossover for all axes, lead compensation
is needed (i.e. k=2 according to Ref. 4). Again, according to
Ref. 4, if k=2, then K1=1, K2=10, T1=2.5 sec, τ0=0.15
sec, ζn=0.707, and ωn=10 rad/s. T2is chosen to ensure
K/s-like pilot-vehicle system characteristics around the inner
loop crossover frequency. For this reason, T2for each axis is
taken as the opposite of the inverse of the damping derivative
for such axis (e.g., for the roll axis T2=−1/Lp). Values for
T2are reported in Table 1. Next, the pilot gain for each axis
is computed to ensure that the desired crossover frequency is
obtained:
Ke=1
|YP(jωc)|
1
|Yc(jωc)|(17)
The pilot gain values for each axis are reported in Table 1. For
the lateral and longitudinal outer position loops, the crossover
frequencies are reduced by a factor of four with respect to the
inner loop crossover frequencies. Values of the lead time TL
and outer loop gains Kyare found in Table 2.
4
Figure 2: A pilot-rotorcraft system for a multi-loop tracking task.
Table 2: Structural pilot model outer loop parameters.
Axis ωc[rad/s] TL[sec] Ky[rad/ft]
Lateral 0.625 3 0.0074
Longitudinal 0.5 3 -0.0066
Figure 3shows the response of the closed-loop pilot-rotorcraft
system for a step in the longitudinal position. Specifically,
Fig. 3(a) illustrates that the closed-loop system is able to
track the command while holding lateral position and altitude.
Figure 3(b) shows that the heading is successfully regulated
throughout the maneuver.
PILOT-VEHICLE SYSTEM PERFORMANCE
Time-Domain Specifications
This section illustrates the mapping of parametric uncertainty
in the pilot model to uncertainty in quantitative handling qual-
ities. Consider the case in which a handling qualities metric is
defined over a transformed augmented state vector. ADS-33E
(Ref. 2) defines the Hover MTE in which a vehicle must main-
tain hover with certain performance requirements on the ve-
hicle state throughout the maneuver. Suppose a pilot-vehicle
system simulation model with state ~xand uncertain parame-
ters ~puis used to evaluate performance against the assigned
MTE performance criteria. Since central nervous and neu-
romuscular systems properties differ from pilot to pilot, and
since each pilot may fly the rotorcraft with different levels of
aggressiveness, let the pilot lateral and longitudinal gains Keφ,
Keθ, and the neuromuscular delay τ0be uncertain parameters
with joint probability fKeφ,Keθ,τ0(Keφ,Keθ,τ0). The MTE can
be simulated from an initial condition ~x0and specific set of
parameters to obtain a time history of ~xthroughout the ma-
neuver. Key values from this time history needed to evaluate
the MTE metrics, such as maximum position and heading ex-
cursions, may be extracted from this time history to produce
(a)
(b)
Figure 3: Closed-loop pilot-rotorcraft response to a step in
commanded longitudinal position: (a) position (b) attitude.
5
Table 3: Maximum position and heading excursions as
defined in the ADS-33E Hover MTE (Ref. 2)
Performance X[ft] X[ft] Z[ft] Ψ[deg]
Desired ±3±3±2±5
Adequate ±6±6±4±10
an evaluation state ¯xeval. Maximum position and heading ex-
cursions for different performance levels of the Hover MTE
are shown Table 3. The mapping S, which maps ¯x0to ¯xeval is
thus the dynamic simulation model of the helicopter. Given
¯xeval, a cost or score function g(¯xeval )may be defined based on
the MTE definition which categorizes performance into De-
sired, Adequate, and Inadequate based on the specific thresh-
olds listed in ADS-33E. Once the simulation model is used
to compute USg(¯xeval)over a discretized set of points in the
initial augmented state space, the expected score can be com-
puted as:
E[g(¯xeval)] =
ZΩ
fKeφ,Keθ,τ0(Keφ,Keθ,τ0)USg(¯xeval)d~xd~pu(18)
As an alternative, suppose three different score functions are
used to assess performance. Each of these is defined as an
indicator function such that:
gDes(¯xeval) = (1 if ¯xeval ∈(¯xeval)Des
0 else (19)
where ¯xeval ∈(¯xeval)Des is the set of evaluation vectors that is
defined to give Desired performance according to the MTE
definition. Analogous score function definitions can be cre-
ated for the Adequate and Inadequate categories. Then, the
probability that the system satisfies the Desired, Adequate,
and Inadequate specifications can be computed as:
E[gDes(¯xeval] =
ZΩ
fKeφ,Keθ,τ0(Keφ,Keθ,τ0)USgDes(¯xeval)d~xd~pu(20)
In practice, each uncertain parameter dimension is discretized
into np=8 evenly-spaced grid points spanning µ±4σ, where
µand σare the mean and standard deviation of each parame-
ter. Since the number of uncertain parameters is 3, this leads
to a total of n3
p=512 data points. The number of grid points
npis chosen to ensure that the numerical solution of the inte-
grals defined on the uncertainty domain (Eqs. 18,20) is suffi-
ciently accurate. Table 4shows mean and standard deviation
values of the uncertain parameters. While mean and standard
deviation for the neuromuscular time delay τ0are taken from
Ref. 5, the standard deviations for the lateral and longitudinal
pilot gains are selected somewhat arbitrarily for the purposes
of this initial study.
A Control Equivalent Turbulence Input (CETI) model is used
to simulate turbulence (Ref. 19). The CETI model is a
Table 4: Uncertain parameters statistics.
Parameter Mean, µStandard Deviation, σ
Keφ36.69 [in/rad] 5% ¯
Keφ[in/rad]
Keθ64.28 [in/rad] 5% ¯
Keθ[in/rad]
τ00.15 [sec] 0.025 [sec]
Table 5: CETI turbulence model parameters (Ref. 20).
Turbulence Level U0[ft/s] σv[ft/s] σw[ft/s]
Moderate 60.0 8.5 6.5
Light 30.0 4.3 3.3
Ambient 13.3 1.9 1.5
hover/low-speed turbulence model that simulates the effects
of atmospheric turbulence by injecting turbulence equivalent
inputs into the control system of an aircraft. The CETI model
used in this simulation study is:
δ0
lat,lon
ξ(s) = 0.2783σ0.9912
wrU0
πL1
s+αw(21a)
δ0
col
ξ(s) = 0.0682σ0.5490
wr3U0
πL1+10.2αw
(s+0.53αw)(s+1.5αw)
(21b)
δ0
ped
ξ(s) = 0.5014σ0.7475
wrU0
πL1
s+αw(21c)
where:
(.)0denotes CETI inputs to the control system,
ξis white noise,
U0is the mean wind speed,
σv,σware the wind speed standard deviations in the lateral
and vertical directions,
L=53.7 ft is the UH-60 main rotor diameter, and
αw=2U0/L.
The CETI model parameters are taken from Ref. 20 and re-
ported in Table 5for different turbulence levels.
For each data point, to which corresponds a particular pilot
model, twenty 30 sec hover simulations are run. The maxi-
mum position and heading excursions for each data point are
taken as the average maximum excursions across the twenty
simulations. If the explicit uncertainty propagation problem
was to be treated rigorously, white noise would have to be ap-
proximated deterministically using a Karhunen-Lo`
eve (KL)
expansion (Ref. 21). This is because the Koopman operator
approach is limited to parametric uncertainty only as it does
not model diffusion of the augmented state PDF due to pro-
cess noise. The KL expansion of white noise is:
ξ(t)≈r2
T
Nt
∑
i=1
ζi(ω)cosi−1
2πt
T(22)
where:
6
tis the current simulation time,
Tis the total simulation time,
Ntis the number of terms used for the expansion, and
ζi(ω)∼N(0,1)are the independent and identically dis-
tributed (i.i.d.) random variables (or uncertain Fourier coeffi-
cients) characterizing the white noise frequency content.
The i.i.d. random variables would have to be included in the
uncertain parameter vector, thus increasing significantly the
order of the problem. Another approach would be to fix such
variables and thus excite the system with a deterministic sig-
nal. It is worth noting that if the i.i.d. variables are fixed,
Eq. 22 reduces to a sum of cosines. Future investigations will
consider these approaches.
Once each data point in the uncertainty domain is mapped to
Desired, Adequate, or Inadequate performance, the integrals
of Eqs. 18,20 are solved numerically to find the mean perfor-
mance level, and the probability that the pilot-vehicle system
will fall within each performance level. The results for the
Hover MTE are shown for ambient turbulence level in Fig.
4(a), and light turbulence level in Fig. 4(b). The mean perfor-
mance of the pilot-vehicle system falls within the Adequate
performance level for both ambient and light turbulence lev-
els. However, the performance is shown to degrade, proba-
bilistically speaking, with increasing turbulence level. In fact,
the probability to have Adequate performance for ambient tur-
bulence level is about 79%, whereas the probability to have
Adequate performance for light turbulence level is about 55%.
It is worth noting that the proposed methodology is applied to
a pilot-vehicle system where the pilot model closes the loop
around an unaugmented aircraft. However, the methodology
could readily be applied to the case where the pilot model
closes the loop around an augmented aircraft, provided that
the pilot model parameters are derived from the augmented
aircraft dynamics.
Frequency-Domain Specifications
The goal of this section is to compute the longitudinal axis
expected broken loop gain margin (GM) and phase mar-
gin (PM) (Ref. 10). This leads to the natural definition of
two cost functions, gGM(Keφ,Keθ,τ0) = GM(Keφ,Keθ,τ0)and
gPM(Keφ,Keθ,τ0) = PM(Keφ,Keθ,τ0). For this particular ex-
ample, the transformations Sdescribed above are the iden-
tify transformations, since the cost functions are defined di-
rectly as functions of the uncertain parameters (rather than on
a transformed set of states and parameters). This means that
USis the identity operator. Once gGM and gPM are computed
over a discretized domain of (Keφ,Keθ,τ0), the expected gain
margin can be found as:
E[GM] =
ZΩ
fKeφ,Keθ,τ0(Keφ,Keθ,τ0)gGM(Keφ,Keθ,τ0)dKeφdKeθdτ0
(23)
where Ωis the support of fKeφ,Keθ,τ0. An analogous equation
can be created for the expected phase margin. As an alterna-
tive, suppose three different score functions are used to assess
(a)
(b)
Figure 4: Uncertain pilot-vehicle system Hover MTE
performance assessment for a) ambient and b) light
turbulence levels.
performance. Each of these is defined as an indicator function
such that:
gDes(Keφ,Keθ,τ0) = (1 if GM(Keφ,Keθ,τ0)∈GMDes
0 else
(24)
where GM(Keφ,Keθ,τ0)∈GMDes is the set of evaluation vec-
tors that is defined to give Desired performance. Analogous
score function definitions can be created for the Adequate and
7
Table 6: Broken loop gain and phase margins requirements
(Ref. 22).
Performance GM [dB] PM [deg]
Desired 6 45
Adequate 3 35
Figure 5: Uncertain pilot-vehicle system gain and phase
margins performance assessment.
Inadequate categories, as well as for phase margin Desired,
Adequate, and Inadequate performance. Then, the probabil-
ity that the system satisfies the Desired specifications can be
computed as:
E[gDes(Keφ,Keθ,τ0)] =
ZΩ
fKeφ,Keθ,τ0(Keφ,Keθ,τ0)gDes(Keφ,Keθ,τ0)dKeφdKeθdτ0
(25)
Although gain and phase margins specifications do not exist
for a pilot-vehicle system, the stability margins requirements
were taken from SAE AS94900 (Ref. 22). These stability
margins requirements are shown in Table 6. Figure 5shows
the realizations of the discretized points in the uncertainty do-
main in the pilot-vehicle system gain margin and phase mar-
gin domains, using the uncertainty distributions defined in the
previous section. The pilot-vehicle system mean performance
is Adequate. The analysis suggests that the probability to have
Desired, Adequate, and Inadequate performances are, respec-
tively, 1%, 79%, and 20%.
PERCEIVED VEHICLE HANDLING
QUALITIES
This section illustrates the mapping of parametric uncertainty
in the pilot model to uncertainty in the perceived vehicle han-
Table 7: Pilot opinion rating vs mean square value of pilot
control activity (data extrapolated from Ref. 4)
Pilot Rating RMS(um/Ke)
2 0.0010
3 0.0262
4 0.1459
5 0.2901
6 0.5572
7 1.0703
8 2.1277
dling qualities (or pilot opinion ratings). Consider the frame-
work illustrated in the previous section. Assume that it is pos-
sible to relate quantitative metrics from the time history of the
augmented state to pilot opinion ratings. Then, a similar ap-
proach can be used to map uncertainty in the pilot model un-
certainty to variability in pilot ratings. Define a score function
such that:
gPRi(¯xeval) = (1 if ¯xeval ∈(¯xeval)PRi
0 else (26)
where ¯xeval ∈(¯xeval)PRiis the set of evaluation vectors that is
defined to give ith pilot rating. Then, the probability that the
vehicle will receive a particular pilot rating can be computed
as:
E[gPRi(~xeval,~pu)] =
ZΩ
fKeφ,Keθ,τ0(Keφ,Keθ,τ0)USgPRi(¯xeval)d~xd~pu(27)
For instance, the pilot workload, which is defined as the root
mean square (RMS) of um/Ke, can be related through the
Cooper-Harper rating scale (Ref. 1) to a pilot rating for a pre-
cision hovering task, according to Refs. 4,18. It follows that,
for each hover simulation described in the previous section, a
particular mean square value of pilot control activity um/Ke
can be computed. Such a value corresponds to pilot opinion
ratings through the relation shown in Fig. 6and Table 7. Once
each data point in the uncertainty domain is mapped to the pi-
lot rating domain, the integrals of Eqs. 26,27 can be solved
numerically to find the mean pilot rating for the vehicle and
the probability that the pilot rating will fall within each rating
level.
The results for the precision hovering task are shown for ambi-
ent turbulence level in Fig. 7(a), and for light turbulence level
in Fig. 7(b). The mean pilot rating is shown to degrade from
PR=3 to PR=4 with increasing turbulence level (i.e. ambient
to light). This result is in accord with the literature (Ref. 19).
The approach used in this paper may be used in future investi-
gations to tune pilot model parametric uncertainty such that it
matches pilot rating uncertainty obtained through (physically)
piloted simulations and/or flight testing.
8
Figure 6: Pilot opinion rating vs mean square value of pilot
control activity (re-created from Ref. 4).
CONCLUSIONS
A mathematical methodology for probabilistic handling qual-
ities evaluation using the Koopman operator was developed.
Results were obtained using a quasi-nonlinear rotorcraft sim-
ulation model and a multi-axis structural pilot model. The
Koopman operator was applied to the propagation of the un-
certain parameters in the pilot model to specific pilot-vehicle
system performance specifications and to perceived vehicle
handling qualities, including the Hover MTE, broken loop
gain and phase margins, and pilot opinion ratings of work-
load. For each type of specification, the relevant state trans-
formation and cost function definition(s) were identified and
numerical examples were presented. Based on this work, the
following conclusions can be reached:
1. The Koopman operator provides a convenient framework
for the propagation of pilot model uncertainty to pilot-
vehicle system performance specifications and to per-
ceived vehicle handling qualities. Specifically, the Koop-
man operator enables evaluation functions that extract
HQ ratings to be pulled back to the uncertainty domain,
wherein the expected value integral can be easily com-
puted.
2. According to probabilistic assessments of the pilot-
vehicle system performance, the mean performance for
a Hover MTE was shown to be Adequate for both ambi-
ent and light turbulence levels. However, the mean per-
formance is shown to degrade with increasing turbulence
level.
3. According to a probabilistic assessment of the broken
loop gain and phase margins specifications, the mean
performance of the pilot-vehicle system was shown to
be Adequate. The analysis suggested that the probability
to have Desired, Adequate, and Inadequate performance
was, respectively, 1%, 79%, and 20%.
(a)
(b)
Figure 7: Pilot model uncertainty mapped to mean square
value of control activity and pilot opinion ratings for a)
ambient and b) light turbulence levels.
4. Uncertainty in the pilot model parameters resulted in sig-
nificant variability in pilot-vehicle system performance
both for frequency- and time-domain specifications. This
result indicates the need for probabilistic assessments in
which it may be stated that the pilot-vehicle system con-
forms to specific guidelines with a certain probability,
given the uncertainty in the pilot-vehicle closed-loop dy-
namics.
5. Uncertainty in the pilot model was propagated to the pi-
lot control activity, a measure of pilot workload, and re-
lated to pilot opinion ratings for a precision hover task.
The mean pilot rating was shown to degrade from PR=3
to PR=4 with increasing turbulence level (ambient to
9
light), which is in accord with the literature.
6. Although the proposed methodology is applied to a pilot-
vehicle system where the pilot model closes the loop
around an unaugmented aircraft, the same methodology
could be readily extended to the case where the pilot
model closes the loop around an augmented aircraft, pro-
vided that the pilot model parameters are derived from
the augmented aircraft dynamics.
The approach used in this paper may be used in future investi-
gations to tune pilot model parametric uncertainty such that it
matches the pilot rating uncertainty obtained through (physi-
cally) piloted simulations and/or flight testing. The methodol-
ogy will be extended to different MTEs and a larger number of
uncertain parameters in the pilot model. Comparative studies
will be conducted to assess the computational advantages of
the proposed technique with respect to Monte Carlo analysis.
REFERENCES
1. Cooper, G. R., and Harper, R. P., The use of pilot rating
in the evaluation of aircraft handling qualities, NASA TN
D-5153, National Aeronautics and Space Administration,
Washington, DC, 1969.
2. Anon, Aeronautical Design Standard Performance Spec-
ification, Handling Qualities Requirements for Military
Rotorcraft, ADS-33E-PRF, USAAMCOM, 2000.
3. Doman, D., Optimal control pilot modeling for resolving
Cooper-Harper rating discrepancies, 24th Atmospheric
Flight Mechanics Conference, Portland, OR, August 9-
11, 1999.
4. Hess, R. A., Theory for aircraft handling qualities based
upon a structural pilot model, Journal of Guidance, Con-
trol, and Dynamics, Vol 12, No. 6, pp. 792-797, 1989.
5. Hess, R. A., Structural Model of the Adaptive Human Pi-
lot, Journal of Guidance and Control, Vol. 3, No. 5, pp.
416-423, 1980.
6. Wilson, D., and Riley, D., Cooper-Harper pilot rating
variability, 16th Atmospheric Flight Mechanics Confer-
ence, Boston, MA, August 14-16, 1989.
7. Riley, D., and Wilson D., More on Cooper-Harper pi-
lot rating variability, 17th Atmospheric Flight Mechanics
Conference, Portland, OR, August 20-22, 1990.
8. Perhinschi, M., and Prasad., J. V. R., Analytical investi-
gation of helicopter handling qualities criterion using a
structural pilot model, 20th Atmospheric Flight Mechan-
ics Conference, Baltimore, MD, August 7-10, 1995.
9. Mavris, D., DeLaurentis, D., and Soban, D., Probabilistic
Assessment of Handling Qualities Characteristics in Pre-
liminary Aircraft Design, 36th Aerospace Sciences Meet-
ing and Exhibit, Reno, NV, January 12-15, 1998.
10. Tischler, M. B., Berger, T., Ivler, C. M., Mansur, M.
H., Cheung, K. K., and Soong, J. Y., Practical Methods
for Aircraft and Rotorcraft Flight Control Design: An
Optimization-Based Approach, AIAA Education Series,
VA, April 2017.
11. Saetti, U., J., Horn, J. F., Lakhmani, S., Lagoa, C.,
and Berger, T., Design of Dynamic Inversion and Explicit
Model Following Control Laws for Quadrotor Inner and
Outer Loops, Vertical Flight Society 74th Annual Forum
Proceedings, Phoenix, AZ, May 14-17, 2018.
12. Cooper, J., DeVore, M. D., Reed, A., and Klyde, D.,
Handling Quality Analysis and Control Design Using
Polynomial Chaos Model-based Uncertainty Methods,
Vertical Flight Society 75th Annual Forum Proceedings,
Philadelphia, PA, May 13-16, 2019.
13. Halder, A., and Bhattacharya, R., Dispersion Analy-
sis in Hypersonic Flight During Planetary Entry Using
Stochastic Liouville Equation, Journal of Guidance, Con-
trol, and Dynamics, Vol. 34, No. 2, pp. 459-474, 2011.
14. Meyers, J. J., Leonard, M., Rogers J. D., Gerlach, A. R.,
Koopman Operator Approach to Optimal Control Selec-
tion Under Uncertainty, 2019 American Control Confer-
ence, Philadelphia, PA, July 10-12, 2019.
15. Leonard, L., Rogers, J., and Gerlach, A., Koopman Op-
erator Approach to Airdrop Mission Planning Under Un-
certainty, Journal of Guidance, Control, and Dynamics
(2019): 1-17.
16. Fletcher, J. W., A model structure for identification of
linear models of the UH-60 helicopter in hover and for-
ward flight, NASA Technical Memorandum 110362, US-
AATCOM Technical Report 95-A-008, Moffett Field,
CA, 1995.
17. Tobias, E. L. and Tischler, M. B., A Model Stitching Ar-
chitecture for Continuous Full Flight-Envelope Simula-
tion of Fixed-Wing Aircraft and Rotorcraft from Discrete-
Point Linear Models, U.S. Army AMRDEC Special Re-
port RDMR-AF-16-01, April 2016.
18. Miller, D. P., and Vinje E. W., FIXED-BASE FLIGHT
SIMULATOR STUDIES OF VTOL AIRCRAFT HAN-
DLING QUALITIES IN HOVERING AND LOW-SPEED
FLIGHT, UACRL-F910482-12, United Aircraft Research
Laboratories, East Hartford, CT, January 1968.
19. Lusardi, J. A., Control Equivalent Turbulence Input
Model for the UH-60 Helicopter, Doctoral Dissertation,
University of California, Davis, CA, 2004.
20. Blanken, C. L., Lusardi, J. A., Ivler, C. M., Tis-
chler, M. B., H¨
ofinger, M. T., Decker, W. A., Berger,
T., and Tucker, G. E., An Investigation of Rotorcraft
Stability-Phase Magin Requirements in Hover, Ameri-
can Helicopter Society 65th Annual Forum Proceedings,
Grapevine, TX, May 27-29, 2009.
10
21. Dutta, P., Halder, A., and Bhattacharya, R., Uncertainty
Quantification for Stochastic Nonlinear Systems using
Perron-Frobenius Operator and Karhunen-Lo`
eve Expan-
sion, 2012 IEEE International Conference on Control Ap-
plications, Dubrovnik, Croatia, October 3-5, 2012.
22. Anon, Aerospace - Flight Control Systems - Design, In-
stallation and Test of Piloted Military Aircraft, General
Specifications For, AS94900, SAE International, Warren-
dale, PA, 2007.
11
