Comparing Multimodal Pilot Pitch Control Behavior Between Simulated and Real Flight

Abstract

To improve the tuning process of flight simulator motion cueing filters and support the development of objective simulator motion cueing requirements, a better understanding of how multimodal pilot control behavior is affected by simulator motion fidelity is required. To this end, an experiment is conducted in which seven pilots perform a pitch target-following disturbance-rejection task in a simulator under four different motion cueing settings. In addition, the task is performed in a real aircraft, which serves as the baseline condition. Differences between the simulator and aircraft experiment setup are minimized. Small remaining differences in the display and the sidestick setup slightly affected the experiment dependent measures. However, the effects introduced by the motion cueing settings are far more apparent. When motion fidelity improves to full aircraft motion, pilots are able to improve performance in attenuating the disturbance signal significantly. In addition, for improved motion fidelity, a change in multimodal pilot control behavior is observed by a decrease in pilot visual lead, while visual and vestibular perception delays increase. Pilot performance and control behavior in the simulator condition with full pitch motion and filtered pitch and center of gravity heave motion is most similar to the in-flight condition.

Full text

Comparing Multimodal Pilot Pitch Control
Behavior Between Simulated and Real Flight
P.M.T. Zaal,
∗
D.M. Pool,
†
M.M. van Paassen,
‡
and M. Mulder
§
Delft University of Technology, Delft, The Netherlands
In order to improve the tuning process of flight simulator motion cueing
filters and support the development of objective simulator motion cu ein g
requirements, a better understanding of how multimodal pilot control be-
havior is affected by simulator motion fidelity is required. To this end,
an experiment is conducted where seven pilots performed a pitch target-
following disturbance-rejection task in a simulator under four different mo-
tion cueing s etti ngs . In addition, the task is performed in a real airc ra ft,
which serves as the baseline condition. Differences between the simulator
and aircraft experiment setup are minimized. Small remaining differences
in the display and the sidestick setup slightly affected the experiment de-
pendent measures. However, the effects i ntro du c ed by the motion cueing
settings are far more apparent. When motion fidelity improves to full air-
craft motion, pilots are able to improve performance in attenuating the
disturbance signal significantly. In addition, for improved motion fidelity,
a ch an ge in multimodal pilot control behavior is observed by a decrease
in pilot visual lead, while visual and vestibular perception delays increase.
Pilot performance and control behavior in the simulator condition with full
pitch motion and filtered pitch and center of gravity heave motion is most
similar to the in-flight condition.
∗
Ph.D. student, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058,
2600GB Delft, The Netherlands; p.m.t.zaal@tudelft.nl. Student member AIAA.
†
Ph.D. student, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058,
2600GB Delft, The Netherlands; d.m.po ol@ tu de lf t.n l. Student member AIAA.
‡
Associate Professor, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058,
2600GB Delft, The Netherlands; m.m.vanpaassen@ tu d elf t. nl. Member AIAA.
§
Professor, Control and Simulation Division, Faculty of Aerospace Engineering, P.O. Box 5058, 2600GB
Delft, The Netherlands; m.mulder@tudelft.nl. Senior Memb er AIAA.
1 of 36
Peter M. T. Zaal, Daan M. Pool, Marinus M. van Paassen and Max Mulder,
Comparing Multimodal Pilot Pitch Control Behavior Between Simulated and Real
Flight (2012), in: Journal of Guidance, Control, and Dynamics, 35:5(1456-1471)

Nomenclature
A
d,t
sinusoid amplitude, V, deg
a
z
pilot st at i o n acceleration , m s
−2
a
z
c.g
c.g. accelerat i on , m s
−2
a
z
θ
pitch acceleration, m s
−2
e tracking error signal, deg
f
d
disturbance forcing
function, V
f
t
target forci n g function, deg
H
a
z
cg
,δ
e
aircraft h e ave dynamic s
H
c
controlled dynamics
H
fbw
fly-by-wire dynamics
H
mf
motion filter
H
ol
open-loop dynamics
H
pa
z
pilot h eave response
H
pe
pilot vi sual response
H
pp
prepositioning fi l t er
H
pθ
pilot p i t ch response
H
θ,δ
e
aircraft p i t ch dynamics
j imaginary unit
K
h,δ
e
c.g. heave dynamics gain
K
m
pilot m ot i on gain
K
mf
motion filter gain
K
s
stick gain
K
v
pilot vi sual gain
K
θ,δ
e
pitch dynamics gain
k sinusoid index
l distance between aircraft
c.g. and pilot station, m
n pilot r em n a nt signal, deg
n
d,t
forcing fu n ct i on frequency
integer factor
s Laplace variable
T
A
1
,A
2
amplitude filter
time con st ants, s
T
h
2
,h
3
c.g. heave dynamics
time con st ants, s
T
lead
pilot l ead time constant, s
T
lag
pilot l ag time constant, s
T
m
measurement time, s
T
θ
2
pitch dynamics time constant, s
t time, s
u pilot co ntrol signal, V
Symbols
δ
e
elevator deflection, deg
θ pitch angle, deg
σ
2
variance
ϕ
m
phase mar gi n , deg
φ
d,t
sinusoid phase shift, rad
τ
ds
display time delay, s
τ
m
pilot m ot i on delay, s
τ
ms
motion sy st em time delay, s
τ
v
pilot vi sual delay, s
ω frequency, rad s
−1
ω
b
mf
motion fi l t er break frequency, rad s
−1
ω
b
pp
prepositioning fi l t er
break frequency, rad s
−1
ω
c
crossover frequency, rad s
−1
ω
d,t
sinusoid frequency, rad s
−1
ω
m
measurement time
break frequency, rad s
−1
ω
nm
neuromuscular frequency, rad s
−1
ω
n
mf
motion fi l t er
natural frequency, rad s
−1
ω
n
pp
prepositioning fi l t er
natural frequency, rad s
−1
ω
sp
short-period frequency, rad s
−1
ζ
mf
motion fi l t er dampin g ratio
ζ
nm
neuromuscular damping ratio
ζ
pp
prepositioning fi l t er dampin g ratio
ζ
sp
short-period damping ratio
2 of 36

I. Int roduction
In o r d er for a high fidelity flight simulator to be an effective tool for research and train-
ing, a pilot performing a task in this simulator should behave as in the real aircraft.
1
How-
ever, it has been shown that for skill-based aircraft control tasks, p i l ot performance and
control behavior are significantly affected by simulator motion cueing settings.
2–5
Current
technology-centered fidelity metrics do not refl ect to what extent a simulator is able to induce
real-flight pilot behavior, as they do not incorporate knowledge about human perception and
control processes. This warrants t he develop m e nt of a new fidelity metric that determines
the simulator’s ability to induce real-flight pilot control behavior.
6, 7
When developing such a behavioral simulator fidel i ty met ri c, the first step is t o determine
systematically h ow pilot control behavior is affected by the limited physical motion stimuli
that are typically provided in a simulator. This is accomplished by i d entifying pilot con-
trol behavior in a simulator under different motion cueing conditions, and comparing this
behavior to the baseline control behavior determined i n real flight. Next, this knowledge
can be u s ed to trace behavioral discrepancies back to the way motion stimuli are presented
in the simulator. Standards and metrics for behavioral fidelity can be developed, and, fi-
nally, m ot i on cuein g algorith m s optimi zed to improve simulator behavioral fidelity based on
objectively-defined targets.
6
Pilot control behavior can be identified by estimating the parameter values of quasi-
linear pilot models. In previous studi es, this approach was used to compare pilot control
behavior between real and simulated flight.
8–14
However, in all these s t u d i es only a single pilot
response function was identified, wit h out distinguishing between the contributions of different
perceptual modalities, for example, visual and vestibular. In a multi-sensory environment,
such as a motion-base flight simulator, this may conceal the pilot’s ability to adopt a differ ent
control strategy by a different use of perceptual modalities. Ther e for e, to compare control
behavior between real and simulated fl i ght adequately, multimodal pilot models need to be
identified that a r e able to distin g u i sh the pilot’s u se of his modalities separately. This paper
presents the results of a study in which multimodal pilot control behavior in a pitch atti tude
control task is comp ar ed between real and simulated flight for the first time.
The identification of multimodal pilot models requi r es a combined target-following distur-
bance-rejection task, as multiple forcing functions need to be insert ed at differ ent locations
in the control loop, to allow for accurate estimati on of the model parameters.
3, 15, 16
To
facilitate this task in real flight in the current study, an experimental fly-by-wire (FBW)
system is developed an d installed into a Cessna Citation II laboratory aircraft.
17
Using this
FBW sy st em a di st u r b a n ce forcin g function is added to the pilot control signal to allow for
a determinist i c and independent disturbance of the a ir cr a ft motion, whil e a target forcing
3 of 36

function is visualized on a vi sual displ ay i n the cockpit. In addition to performing the task
in the aircraft, pilots perform the task in the SIMONA Research Simulator (SRS) under
four different motion cueing conditions, to investigate the effect of limited simulator motion
fidelity on pilot performance and multimoda l control behavior.
In Section II, the pitch attit u de co ntrol task for the identification of multimodal pilot
control behavior in real and simulated flight is discussed. Next, in Section III, the experiment
setup is given for both the simulator and th e in-flight measurements. This is followed by a
comparison of pilot performance an d multimodal pilot model identification results from the
aircraft and simulator data in Section IV. The paper ends with a discussion and conclu si on s .
II. Pitch Attitude Control Task
In order to com p ar e multimodal pilot control behavior between real and simulated flight,
a pitch attitude tr acking task is performed in both a Cessna Cit at i on II laboratory aircraft
and the S RS. This section describes the pitch tra cking task, the controlled aircraft dynamics,
and the pilot model used to qua ntify multimod al pilot control behavior.
II.A. Pitch and Heave Motion Cues
As visualized in Figure 1, in a Cessna Citation II – as in most conventional fixed-wing
aircraft – pilots ar e seated a considerable distance in front of the center of gravity (c.g.).
When performing a pitch attitude control task, pilots are su bject to both rot at i on a l pitch
and linear heave m ot i on . The rotational pitch motion
¨
θ is a direct resu l t of the change in
aircraft pitch attit ude. The total linear heave motion at the pilot station is a combination of
pitch h eave a
z
θ
and c.g. heave motion a
z
cg
.
4, 5
Pitch heave is the linear acceleration induced
by the pitch rotation of the aircraft and the relative position of the pilot station in front of
the c.g.. Center of gr avity heave results from relatively slow changes in aerodynamic lift due
to the change in aircraft angle of at t a ck while pitching. Figure 1 shows t h a t the total heave
at the pilot station can be descr ibed by:
a
z
= a
z
cg
+ a
z
θ
= a
z
cg
− l
¨
θ (1)
where l is the distance between the aircraft c.g. and the pilot st at i on . For the Cessna
Citation II, this distance is approximately 3.2 m.
Pitch rotational motion and both types of heave motion are directly correlated with
the change in aircraft pitch attitude and , when perceived by pilot s, allows them to close
additional feedback loops around the controlled aircraft dynamics, allowing for an increase
in performance.
18
In conventional flight simulators, linear accelerations result i n g from the
4 of 36

c.g.
pilot station
¨
θ
l
a
z
cg
a
z
cg
+ a
z
θ
Figure 1. Aircraft motion at the center of gravity and pilot station during a pitch maneuver.
aircraft model need to be severely attenuated to keep the simulator cab within th e limits of
the motion system. Classical li n ea r washout filters are most commonly used for attenuation
in magnitude and phase.
19
For the compensatory pitch tracking task considered in this paper,
and for most flight simulation applications in general, the c.g. heave component is the m ost
problematic for m ot i on cueing due to its low-frequency but high-amplitude characteristics.
In preparation t o the experiment described in the current paper, two experiments have
been performed in the SRS to investigate the effects of limited motion cues on pilot per-
formance and control behavior in a pitch attitude control task. In the first experiment,
described in Ref. 4, t h e effects of rotational pitch and the two heave motio n components
have been investiga t ed . Ref. 5 describes the sec on d experiment in which the effects of
different heave motion filter set t i n g s were investigated.
It was not possible to represent c.g. heave one-to-one in these experiments, as this
drives the SRS motion system beyond its physical limits. Hence, there was no baseline
condition with full aircraft m o ti o n to investigate truly the effects of attenuating the motion
in a simulator. The experiment described in the current paper is designed to compare pilot
performance and multimodal control behavior in different si mulator motion con d i t i on s to an
in-flight baseline condition, th at is, one-to-one pitch and full heave motion. The experimental
conditions in this experiment are kept as simil a r as possible to t h e previous experiments
described in Refs. 4 and 5 to allow for a direct comparison of the results.
II.B. Compensatory Tracking Task
Figure 2 shows the compensatory target-following disturbance-rejection task used in this
study. This type of co ntrol task has been considered in many previous investigations into
multimodal human manual control behavior
3, 4, 20
and techniques for the identification of
control behavior in such tasks are well established.
15, 21, 22
The figure shows a pilot performing
the task in a motion-base simulator.
In this compensatory tracking task, the objective of the pilot is to minimi ze actively the
deviation of the aircraft pitch attitude θ fro m a desired pitch att it ude, the forcing function
f
t
. In addition, the pilot needs to minimize a disturbance acting on the aircraft , induced by a
5 of 36

H
pe
(s)
H
pθ
(s)
n
e
δ
e
θ
f
t
f
d
u
θ
controlled dynamics, H
c
(s)
pilot
– –
τ
ds
τ
ms
K
s
display
FBW
system
H
fbw
(s)
H
θ,δ
e
(s)
sidestick
H
mf
(s)τ
ms
motion
system
H
pa
z
(s)
a
z
a
z
c.g. heave
dynamics
H
a
z
cg
,δ
e
(s)
pitch
dynamics
e
θ
s
a
z
s
s
2
l
a
z
cg
a
z
θ
–
Figure 2. Simulator compensatory tracking task.
e
Figure 3. Compensatory display.
disturbance forcing function f
d
. The deviation from the tar ge t for ci n g function is visualized
by t h e error e on the compensatory display depicted in Figure 3. The error is the only task
variable available on the vi su a l display; that is, as opposed to a pursuit display, there is no
visual information on the actual pitch attitude of the air cr aft .
The pilot controls the aircraft dynamics using a sidestick with a gain K
s
(Figure 2). The
summation of the d i st urbance forcing fun ct i on f
d
and the stick output serves as the control
input for a mod el of the aircraft’s FBW system H
fbw
(s). This model provides the aircraft
dynamics with elevat o r control inputs δ
e
. The aircraft pitch and c.g. heave responses a r e
given by the transfer functions H
θ,δ
e
(s) and H
a
z
cg
,δ
e
(s), respectively. The controlled dynamics
H
c
(s) are a combination of the FBW system dynamics and the pitch response of the aircraft.
The total heave at the pilot station (a
z
) is t h e summation of c.g. heave (a
z
cg
) and
pitch heave (a
z
θ
), as defined i n Eq. (1). Pitch and heave accelerations serve as inputs to
the simulator motion system. For the purpose of this experiment, the dynamics of both
the disp l ay and motion systems
23
are appr oximated by a pure time delay, τ
ds
and τ
ms
,
6 of 36

respectively. In the h eave m ot i on channel a motion filter is incorporated with dynamics
H
mf
(s).
The simulator display and m o t io n systems provide the pilot with visual cues e , and
physical motion cues
¨
θ
s
and a
z
s
, respectively (Figure 2). The pilot’s responses to these
different cues can be modeled by linear response functions. H
pe
(s) is the linear response
to vi su a l error cues. H
pθ
(s) describes the response to rotational accelerations using the
semicircular canals, while H
pa
z
(s) describes the response to linear accelerations using the
otoliths.
24, 25
The pilot control output u is a summation of the outputs of the linear response
functions and a rem n a nt signal n that accounts for nonlinear behavior. This pilot control
output results in anot h er control input through th e sidestick, effectively cl os ing t h e control
loop.
The control diagram of th e compensatory pitch tar g et -fol l owing disturbance- r eject io n
task performed in the aircraft is similar to the dia gr am of the task performed in t h e simulator,
depicted in Figure 2. In the aircraft control diagram, the FBW system and aircraft dynam i c
models are replaced by the actual FBW system and the aircraft. No motion system dynamics
(H
mf
(s) and τ
ms
) are present in the aircraft control diagram, as the aircraft has no mot i on
system. However, display dynamics are still present in the form of a time delay (τ
ds
).
II.C. Controlled Dynamics
The total controlled dynamics are a combination of the FBW system an d the aircraft pitch
dynamics. A n on l i n ea r dynamic model of the FBW control system is implemented in t h e
SRS.
26
The FBW system is a nonlinear system in part due to several li m i t s on internal control
signals. These limits are either physical limits of system components or limits introduced
for safety reasons. When the FBW system is operated within these limits, the dynamics of
the FBW pitch channel H
fbw
(s) can be simplified to a gain of 0.045 and a time delay of 60
ms as identified using experimental data in Ref. 27.
In the SRS, the dynamics of the Cessna Citation II aircraft are represented by linear
transfer functions. This is justified by the fact that there are only small deviations from the
trim condition during an experiment run. Before the simulator part of the experiment is
conducted, the aircraft pitch and c.g. heave d y n am i cs are identified using data from a test
flight. The dynamics are identified for an indicated airspeed of 160 kt and an altitude of
17, 000 ft. Th i s is dete r m ined to be the optimal flight condition based on the torque limit
switching prop er t i es of the FBW system,
17
weather con ditions, and availability of controlled
airspace.
A cost function is minimized in the fr eq u ency domain to fit t h e transfer function of a two
degree-of-freedom short-period approximation to an identified frequen cy res ponse.
28
This
frequency response i s an autoreg r essi ve exogeneous (ARX) model estimate calculated usin g
7 of 36

flight test data of the elevator δ
e
, pitch angle θ, and c.g. acceleration a
z
cg
. The identified
transfer fu n ct i ons are given by:
H
θ,δ
e
(s) = K
θ,δ
e
T
θ
2
s + 1
s

s
2
ω
2
sp
+
2ζ
sp
ω
sp
s + 1

= 1.64
0.90s + 1
s

s
2
5.49
+ 0.41s + 1

(2)
H
a
z
cg
,δ
e
(s) = K
h,δ
e
(T
h
2
s + 1) (T
h
3
s + 1)
s
2
ω
2
sp
+
2ζ
sp
ω
sp
s + 1
= 157.39
(0.15s + 1)(−0.16s + 1)
s
2
5.49
+ 0.41s + 1
(3)
The controlled pitch dynamics are a single integrator for lower frequ encies up to 1/T
θ
2
=
1.12 rad/s. Th en, up to a peak resul t ing from the short-period eigenmode at ω
sp
=
√
5.49 =
2.34 rad/s, the aircraft pitch dynamics are a gain. Fi n al l y, for frequencies above the short-
period eigenfrequency, the system behaves like a double integrator. Figure 14 depicts the
frequency responses of the aircraft dynamics.
II.D. Forcing Functions
As d epicted in Figure 2, disturbance and tar get forcing function signals f
d
and f
t
are used
for inducing pilot control activity. Both the target and disturbance forcing functio n s are
constructed as quasi- r an dom sums of 10 sinusoids, according to:
f
d,t
(t) =
10
X
k=1
A
d,t
(k) sin [ω
d,t
(k)t + φ
d,t
(k)] (4)
Tracking runs are defi n ed to last a total of 90 s, of which the final 81.92 s are considered as
the measurement interval T
m
. Sinusoid frequencies ω
d,t
(k) are distributed over the frequency
range of interest (0.1–20 rad/s) and ar e defined to be integer multiples of the measurem ent
time base frequency ω
m
= 2π/T
m
: ω
d,t
(k) = n
d,t
(k)ω
m
. The integer factors n
d
and n
t
are
chosen such that f
d
and f
t
have power at interleaving frequencies ( Table 1) to allow for
multimodal pilot model identification using spectral methods.
3
To yield forcing function si gn a ls with reduced power at the higher frequencies, the am-
plitude distributions A
d,t
(k) are defined using the low-pass filter d escr i bed in Ref. 4:
A
d,t
(k) =




(1 + T
A
1
jω
d,t
(k))
2
(1 + T
A
2
jω
d,t
(k))
2




(5)
The time constants T
A
1
and T
A
2
in Eq. (5) are set to 0.1 and 0.8 s, respectively. The
forcing function signal phases φ
d,t
(k) are chosen randomly, though care wa s taken not to end
up with signals with severe cresting or peaking due to phase overlap.
29
The target forcing
function signal is scaled to have a time-domain variance of 0.4 d e g
2
. The disturbance forcing
function amplitudes as defined by Eq. (5) ar e pre-filtered with the inverse of the combined
8 of 36

aircraft and FBW syst em dynami cs (see Figure 2) and scaled to yield a low-pass disturbance
of the pitch att i t u d e θ wit h a tim e-domain variance of 0.4 deg
2
. As the FBW system requ i r es
control inputs in Volts, the resulting disturbance signal also has that unit.
The numerical valu es of all multi-sine forci n g function signal parameters defined in Eq. (4)
are listed in Table 1. Power spectra of both forcing functions are depicted in Figure 4, clearly
showing the amplit u d e distributions. Figure 5 depicts the first 30 seconds of both forcing
function signals.
Table 1. Experiment forcing function properties.
k, –
Disturbance, f
d
Target, f
t
n
d
, – ω
d
, rad s
−1
A
d
, V φ
d
, rad n
t
, – ω
t
, rad s
−1
A
t
, deg φ
t
, rad
1 5 0.383 0.026 1.145 6 0.460 0.698 1.288
2 11 0.844 0.035 5.336 13 0.997 0.489 6.089
3 23 1.764 0.021 0.802 27 2.071 0.220 5.507
4 37 2.838 0.017 7.390 41 3.145 0.119 1.734
5 51 3.912 0.021 8.326 53 4.065 0.080 2.019
6 71 5.446 0.028 5.398 73 5.599 0.049 0.441
7 101 7.747 0.038 -1.349 103 7.900 0.031 5.175
8 137 10.508 0.053 0.128 139 10.661 0.023 3.415
9 171 13.116 0.071 0.696 194 14.880 0.018 1.066
10 226 17.334 0.109 0.916 229 17.564 0.016 3.479
S
ff
ω, rad s
−1
S
f df d
(ω), V
2
/rad s
−
1
S
f tf t
(ω), deg
2
/rad s
−
1
S
f df d
(ω
d
), V
2
/rad s
−
1
S
f tf t
(ω
t
), deg
2
/rad s
−
1
10
-1
10
0
10
1
10
-20
10
-15
10
-10
10
-5
10
0
10
5
Figure 4. Forcing function power spectra.
f
d
, V
t, s
disturbance, f
d
target, f
t
trim pitch, θ
0
f
t
, θ
0
, deg
0 5 10 15 20 25 30
-6
-4
-2
0
2
4
6
-0.3
-0.2
-0.1
0
0.1
0.2
0.3
Figure 5. Forcing function time traces.
Figures 4 an d 5 show a low-frequen cy target forcing function f
t
and a much more h i gh -
frequency f
d
, which is a result of the pre-shaping fil t er . Furthermore, note that a fade-in is
applied to the disturbance signal so that f
d
gently increases to its full amplitude over the
first 5 seconds of the run-in time of each tracking run. Figure 5 further shows that the tar g et
forcing func t io n that needed to be tracked by the pilo ts is defined with respect t o the trim
pitch attitude of the aircraft. During the in- fl i ght experiments, this θ
0
is determined from
the actual aircraft pitch at the start of each run. For the simulator experiments, θ
0
is the
fixed aircraft model trim pitch that r esulted from the selected trim co n d i t i on and which is
also used for pitch attitude motion cueing (see Secti on III.A.3).
9 of 36

n
–
e
θ
K
v
(1+sT
lead
)
2
(1+sT
lag
)
K
m
e
−sτ
v
e
−sτ
m
s
ω
2
nm
ω
2
nm
+2ζ
nm
ω
nm
s+s
2
H
pe
(s)
H
pθ
(s)
equalization
limitations
u
˙
θ
u
e
u
θ
Figure 6. The multimodal pilot model structure.
II.E. Multimodal Pilot Model
Human operators subconsciously adjust their control behavior to the controlled dynamics
in such a way that th e pilot-aircraft open-loop dynam i cs in the region of crossover can be
described by integrator dynamics and a t i m e delay.
30
Figure 6 provides an appropriate model
structure to model multimodal pilot control behavior in the pitch attitu d e control t ask g iven
in Figure 2.
24, 25
The model consists of two inputs e and θ and two channels H
pe
(s) and
H
pθ
(s) to model the pilot’s visual and vestibular modalities. A remnant signal n, which can
be characterized by a low-pass filtered white noi se signal, is added to the output of the linear
channels to account for nonlinearities in the pil ot control output.
The mod el only consists of two channels as opposed to the t h r ee channels in Figure 2. It
has been shown in previous experiments that both the semicircu la r canals and the otoliths
provide additional pilo t lead – that is, a response to rotational velo ci ty – in parallel to the
lead generated by the vis u al response.
24, 31
Due to the similar contribution of t h ese two
channels to th e overall pilot response, the total model would be overdetermin ed, decreasing
the accuracy of the model parameters. Therefore, the contributions of the vestibular pitch
and heave motion channels are combined in a single vestibular response channel that takes
the form of a pure lead on θ with a time delay.
5
With the aircraft pitch dynamics provided by Eq. (2) (Figure 14) and typical crossover
frequencies between 2 and 4 rad/s for th i s type of control task, the pilot needs to generate
lag around 1/T
θ
2
to compensate for the gain dy n am i cs before the short-period frequency.
Subsequently, a qua d r at i c lead t er m i s nee d ed arou nd ω
sp
to compensate for the lag and
achieve the required lead compensation for the double integrator dynamics above the short-
period frequency.
32
This r es u l t s in the pilot equalizatio n given in Figure 6. Note that there
is no d ouble lead generated at higher frequencies, as the model might suggest. The quadratic
lead term is preceded by a lag term, resulting in single lead at h i gher frequencies.
10 of 36

The visual perception cha n n el H
pe
(s) contains the vi sual gain K
v
, visual lead time con-
stant T
lead
, visual lag time constant T
lag
, and visual time delay τ
v
. The vestibular channel
H
pθ
(s) includ es the vestibular gain K
m
and a vestibular time delay τ
m
. In both channels, the
control action of the pilot is limited by the neur omuscular system dynamics characterized
by the neuromuscular damping ratio ζ
nm
and natural frequency ω
nm
. These eight param-
eters are estimated in a time-domain m a xi mum likelihood estimation (ML E) procedure to
quantify changes in pilot control strategy in the different experimental conditions.
22
In the freq u ency domain, pilot performan ce in attenuating the target and disturban ce
signals is determ i n ed by the crossover frequencies and phase mar gi n s of the target and
disturbance open-loop responses, respectively.
2
Using the control scheme in Figure 2 and
the pilot response fun ct i on s given in Figure 6, the distu r b a n ce and target open-loop responses
are deter m i n ed by:
H
ol,d
(s) =
U(s)
δ
e
(s)
= [H
pe
(s) + H
pθ
(s)] K
s
H
c
(s) (6)
H
ol,t
(s) =
E(s)
θ(s)
=
H
pe
(s) K
s
H
c
(s)
1 + H
pθ
(s) K
s
H
c
(s)
(7)
The disturbance and target cro ssover frequencies, ω
c,d
and ω
c,t
, are the frequencies where
the magnitude of the disturbance and target open-loop responses have a magnitude of 1.0.
The corresponding phase margins, ϕ
m,d
and ϕ
m,t
, are the phase d i ff er en ces from −180 degrees
at these crossover frequencies.
II.F. Variables Affecting the Pilot-Vehicle System
The experiment described in this paper compares pil ot control behavior in two different
environments: a real in-flight cockpit and a motion-base flight simulator. As pointed out
by McRuer and Jex,
18
human operator behavior for a certain control task i s affected by
a multitude of variables. These variables c an be divided into four groups: environmental
variables, operator-centered variab l es, procedur al variables, and task variables. Differences
in pilot control behavior resulting from differ en ces in variables other than th e change i n
motion fidelity complicate the interpretation of the effect of motion cueing on pilot behavior
as made in this paper a n d should be minimized as much as possible.
Task variables – such as, forcing functions and controlled dynamics – and procedural
variables – for example, i n st r u ct i ons and training – are relatively easy to match between the
aircraft and the simulator with good experiment design. However, environmental variables
– temperature, ambient lighting conditions, etc. – and operator-centered variables – such
as, motivation and workload – are much more difficult to control. Anoth er major operator-
centered variab l e that can affect the results in th e current experiment is the psychological
11 of 36

Figure 7. The SIMONA Research
Simulator (SRS).
Figure 8. The SRS cockpit setup.
effect of controlling an actual aircraft a s opposed to controlling an aircraft model in the
relatively safe environment of a flight simulator.
III. Experiment
The experiment setup in both the si mulator and the aircraft are discussed in the following
two subsections. Next, some remaining experimental considerations, the dependent measures
and hyp ot heses of the overall experiment, are discussed.
III.A. Simulator Experiment Setup
III.A.1. Apparatus
The flight simulator part of the experiment is performed in the SRS (Figure 7). The SRS
has a hydraulic six degree- of- fr eed om hexapod motion system, which is u sed to su pply pil ot s
with pitch and heave motion. The time delay of the SRS motion cues is τ
ms
≈ 30 ms in all
axes.
33
The pilots are seated in the right p i l ot seat in the S R S cockpit. The compensatory display
(Figure 3) is shown on the primary flight display (PFD) directly in front of them. No other
visual cues, fo r instance from the outside visual system, are provided. The time delay of the
image gener a t io n on the PFD has been measured to be in the order of τ
ds
≈ 25 ms.
34
The pil o t s use a Moog FCS Ecol-8000 electrical sidestick to ma ke control inputs. The
characteristics of thi s control loaded sidest i ck can be fully adjusted. The sidestick is config-
ured to have the same characteristics a s the Citation force stick. As opposed to a deflect i on
stick, a force stick has very h i gh stiffness and no manipul a to r deflections ar e required for
making a control in p u t . Rather, the control input made by the pilot is proportional t o the
force applied to the m an i p u l a to r . The stick is configured to have a linear force-voltage r el a-
tion with a gradient of 23 N/V, limit ed at ±2.5 V. Th e stick gain K
s
(see Figure 2) is set to
0.3.
12 of 36

III.A.2. Independent Variables
Four experimental conditions are evaluated in the SRS (see Table 2). In the remainder of
this paper, these conditions are referre d to using the symbols in the first co l u m n of Table 2.
A reference condit i on (S0) is performed without any moti on cueing. In the remai n i n g motion
conditions (S1–S3), rotational pitch motion is presented 1-to-1 (no washout). In addition to
the rotational pitch motion , conditions S1 and S2 present only the pitch heave compo n ent of
the total aircraft heave mot i on, whi l e the total aircraft heave motion is presented in con d i t i on
S3. For attenuating the total aircraft heave motion so that it can be p re sented using th e SRS
motion base, the heave wa sh out filter also used in Ref. 4 is applied for condition S3. The
same filter is used for presenting fi l t er ed pitch he ave motion in condition S1. These motion
conditions represent a subset of the conditions previously evaluated for a very similar pitch
control task in Refs. 4 and 5.
Table 2. Simulator experimental conditions.
Condition Apparatus Environment
Motion
Pitch Pitch heave C.g. heave
S0 SRS on ground – – –
S1 SRS on ground full filtered –
S2 SRS on ground full full –
S3 SRS on ground full filtered filtered
III.A.3. Simulator Motion Cueing
By im p l em enting additi o n al heave and surge motion, the axis ar ound which rotational pitch
motion cues are presented is moved to the same position rel at i ve to the pilot head position
as it is measured to be in the Citation. Furthermore, using a prepositioning filter, the
SRS is moved to the trim pitch attitude θ
0
before the start of each measurement run.
35
This prepositioning is also done for the no-motion condition S0. The prepositioning filter is
defined as:
H
pp
(s) =
s
2
s
2
+ 2ζ
pp
ω
n
pp
s + ω
2
n
pp
·
s
s + ω
b
pp
(8)
The parameter values of Eq. (8) can be found in Table 3, in addition to the trim pitch
attitude θ
0
to which the simulator is prepositioned. Figure 9 depicts the first part of the time
trace of the simulator pitch motion response for a typical experiment run. It clear ly shows
the prepositioning filter response for t < 0 and the simulator pitch motion r es u l t ing from
the pitch control task for t > 0. Due to the prepositioning and the moderate magnitude of
the pitch excursions attained during the control task, the simulator pitch attitude is always
positive during t he whole experiment.
13 of 36

Table 3. Motion cueing parameters.
Filter Parameter Value Unit
H
pp
ω
n
pp
1.0 rad s
−1
ζ
pp
1.0 –
ω
b
pp
2.0 rad s
−1
θ
0
4.34 deg
H
mf
K
mf
0.6 –
ω
n
mf
1.25 rad s
−1
ζ
mf
0.7 –
ω
b
mf
0.3 rad s
−1
t, s
θ, deg
simulator pitch, θ
trim pitch, θ
0
-10 0 10 20 30 40 50
0
1
2
3
4
5
6
7
Figure 9. Time trace of simulator pitch attitude.
a
z
cg
a
z
θ
a
z
, m s
−2
t, s
(a) aircraft heave accelerations
20 22 24 26 28 30
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
a
z
, m s
−2
t, s
(b) filtered heave ac celerat ion s
20 22 24 26 28 30
-2.0
-1.5
-1.0
-0.5
0.0
0.5
1.0
1.5
2.0
Figure 10. Pilot station heave accelerations.
For two of the SRS condition s (S1 and S3) a heave moti on filter is u sed for presenting
heave motion cues (Table 2). The filter used for these conditions is the sam e third-order
high-pass filter als o utilized in Ref. 4:
H
mf
(s) = K
mf
s
2
s
2
+ 2ζ
mf
ω
n
mf
s + ω
2
n
mf
·
s
s + ω
b
mf
(9)
The values for the filter corner frequen cie s, ω
n
mf
and ω
b
mf
, and dampi n g factor ζ
mf
can
be found i n Table 3. Figures 10(a) and 10(b) show the heave acceleration components in a
typical experiment run before and after filtering, respectively. By comparing the two figures,
it can be seen that especially the magn i t u d e of the c.g. heave is heavily affected by the
motion filter.
14 of 36

Figure 11. The Cessna Citation II
laboratory aircraft.
Figure 12. The Cessna Citation II
cockpit setup.
III.A.4. Experimental Procedures
All pilots performed the simulator part of the experiment before the i n - fl i ght part in the
Citation laboratory aircraft. First, pilots performed a number of training runs – typically
4-6 repetitions of each experimental condition – until their proficiency in performing the
tracking task stabilized. After all the trainin g runs are performed, five repetitions of each
experimental conditio n are collected as the measurement data. The motion conditions list ed
in Table 2 (S0-S3) are presented in random order (Latin square) throughout both the training
and measu r em ent phases of the experiment. Breaks are taken regularly to avoid fatigue.
After the simulator is prepositioned to the aircraft trim pitch attitude, the experimenter
counts down from three and starts the run. Directly after a run ends the simulator is tilted
back to zero pitch attitude, after which participants are asked to give a subjective judgment
of motion fidelity for the run they just completed (see Section III.D). After this subjective
evaluation, pilots are informed of their t r acking score, defined a s the root mean square (RMS)
of the recorded error signal e.
III.B. Aircraft Experiment Setup
III.B.1. Apparatus
The Cessna Citation II laboratory aircraft depicted in Figure 11 is a twin-jet busin ess aircraft.
The aircraft is equipped with a custom Flight Test Instrumentation System (FTIS).
17
The
experiment setup in the laborat o ry aircraft is very simil ar to the setup in Ref. 27. A FBW
control system is especially d e veloped and installed into the aircraft for the purpose of this
experiment and is ext ensivel y described in Ref. 17. The aircraft is equipped with a nose
boom with alpha and beta vanes, which allows for accurate monitoring of the angle of attack
and side sli p, respectively.
As can be verified from Figure 12, the experiment setup in the Citation cockpit is very
similar to the se t u p in the SRS cockpit. The experiment is performed from the right pilot
15 of 36

seat. A programmable LCD display i s installed in front of the origi n al instrument panel on
which the compensatory display shown in Figure 3 is presented. Using a custom visual delay
measurement system,
34
the latency of the disp l ay (including the projection) i s measured to
be τ
ds
≈ 25 ms. This is approximately the same as t h e delay foun d for the display in the
simulator.
Figure 12 further shows the sidestick manipulator that is installed at the right pil ot seat
in the Citation cockpit. Due to the limited space available in the cockpit, a BG Systems
JFf force joystick is used to make control inputs during the in-flight tracking tasks. The
output voltage of the stick is lim i t ed between ±2.5 V. The relation between applied force
and out p u t voltage is determined from static measurements, where known weights are used
to induce known stick forces. Th e resulting output voltage is then measured. The force-
voltage character i st i c of the Citation force stick is found to be approximately linear with a
gradient of 23 N/V over the full range of th e output voltage.
As explained in detail in Refs. 17 and 27, the side s t ick commands and the dist u r bance
forcing function sign al are fed to a custom experiment computer, which in tur n uses the
actuators of the aircraft’s aut om a t ic flight control system to control the control surfaces.
Note that due to the fact that the automatic control system makes use of the mechanical
control architecture, both side stick i n p u t s and the disturbance forcing function signal result
in a moving control column (see Figure 12).
III.B.2. Independent Variables
The three different conditio n s evaluated in the in-flight par t of the experiment are listed in
Table 4. These conditions are referred to using the symbols in the first c ol u m n of Table 4.
To determine the effect of limited motion cues on pi l ot control behavior, the cond i t i on s per-
formed in the SRS are compared with the in-flight condition CF. For th is baseline condition
the full unrestri ct ed aircraft pitch and heave motion are present.
Table 4. In-flight experimental conditions.
Condition Ap paratu s Environment
Motion
Pitch Pitch heave C.g. heave
C0 Citation on ground – – –
C0F Citation in flight – – –
CF Citation in flight full full full
Table 4 lists two additional conditions that are performed in the Citat i on laboratory
aircraft: C0 and C0F. For these two conditions the pitch control task is performed using the
experimental setup utilized for the in-flight m ea su r em ents of CF, but pilots are controlling
the same model of the Citation and FBW system dynamics as used in the SRS conditions
rather than the aircraft itself. Condition C0 is performed on the ground and ther e for e yields
16 of 36

a condition that should be equival ent t o the simulator condition with o u t any motion, S0.
Condition C0F is the same as C0, but the measureme nts for this condition were taken d u r ing
flight, at the same altitude and velocity as selected for CF.
Even though extreme care is taken t o ensure the important ele m ents of the experimental
setups in the Citation and SRS are as equ al as possible, these a d d i t i on al conditions are
performed to quantify possible discrepancies in control behavior due to remaining differences
in the experimental set u p and the environment in which pilot control behavior is measured.
III.B.3. Experimental Procedures
As in th e simulator part of th e experiment, participants performed a number of training runs
until their proficiency in performing the tra cking task stabi l ize d . Then five more repetitions
of each experimental condition are collected as the measurement data. The three conditions
of the in-flight part of the exper i m ent (see Table 4) are always performed in the same order.
Before take-off, the measurements for t h e C0 condition ar e taken. The main reason for
taking these measurements first is that it allows for initial refamiliarization with the control
task in a more controlled environment than duri n g flight. This refamiliarization is necessary
as the aircraft part of the experiment was conducted 5 months after the simulator par t of
the experiment. In addition, it reduces the number of training runs required for the i n - fl i ght
measurements of condition CF. During flight, both pilots performed the in-flight aircraft
control task CF first, after which condition C0F is evaluated.
During the in-flight measurements, two pilots are always required for each flight. The
pilot in the left seat functions as the safety pil o t and is responsible for monitoring t h e
aircraft during the ex periment and ensures the aircraft is in t h e desired trim state (V = 160
kt, h = 17, 000 ft) before each run. The other pil ot , the experiment pilot, performs the
experiment.
To avoid that the moving control column and pilots’ view of the outside world affect
their control strategy during the pitch tracking task, the experiment pilot is required to
wear a hood ( se e Figu re 13) t hat limited the field of v iew t o the LCD display during the
measurements for conditions CF and C0F. For condition C0, participants di d not wear th e
hood.
For conditions C0 and C0F the experimenter in the main cabin of the aircraft initiates
the star t of a run after counting down from three , as is also done during the simulator part
of the exp er i m ent. For the CF condition, where the actual aircraft is co ntrolled, th e run is
initiated by the experiment pilot using the pi l ot interface shown above t h e LCD screen in
Figure 12. For more details on th i s pilot interface, refer to Refs. 17 and 27.
After completion of a run for condition CF, the FBW system disengages itself, after
which t h e aircraft slightly drifts from the t ri m condition. The safety pilot then takes control
17 of 36

Figure 13. Experiment pilot wearing the hood during the in-flight ex periment.
of the aircraft to bring it back to the desired trim state for the next run of the experiment.
The experimenter not i fi e s the exp e ri m ent p il ot of his performan ce for the las t run, defined
as the RMS of the error sign a l e.
III.C. Pilots
Seven pilots performed the pitch attitude tracking task under the four simulator conditions
listed i n Table 2 and th e three in-flight conditions listed in Table 4. All par t i cipants are
active Cessna Citatio n I I pilots and all except one have experience with similar control tasks
from previous simulator and in-flight experiments.
13
The participants’ flight experiences
range from 1,500-14,000 hrs on a multitude of different aircraft. Their ages range from 34
to 72 years.
III.D. Dependent Measures
Several dependent measures are considered for this experiment to evaluate the effects of the
variation in motion fidelity on pilot t r acking performance and multimodal control behavior.
First of all, during the SRS part of the experiment, participants ar e asked to give a subjective
indication of the motion fidelity. Participants rate the motion fidelity by drawing a vertical
line through a horizontal bar of a motion fidelity rating scale for each tracking run. Zero
percent (the left side of the bar) indicates that th e experienced simulator motion is not like
the real aircraft motion at all, wh i le 100 % (the right side of the bar) indicates that t h e
motion is perceived to be exactly the same as experienced in the aircraft. These subjective
ratings merely serve as a reference for the objective measures of the experiment.
In addition, a large number of objective measures of pilot tracking performance, control
activity, and control behavior are considered. Tracking performance an d control activity are
evaluated from the time-domain variances of th e recorded time traces of the pitch tracking
error e and pilo t control signal u, respectively. Using a spectral method described in Ref. 2,
18 of 36

the contributions of the target and disturbance forcing functions and pil ot remnant to these
signal variances ar e evalu at ed .
The most important dependent measures that are considered , however, are the parame-
ters of the multimodal pilot model introduced in S ect i on II.E. These pi l ot model p ar a m et er s
are identified from measur em ents of e, u, and
˙
θ (the latter only for cond it i ons with motion
cues) using th e time-domai n MLE procedure described in Ref. 22. Before being used for
model identification, the high-frequen cy noise present in these sign al s – that is, the noise
above 30 rad/s, well above the highest frequen cy sinusoids i n f
t
and f
d
(Table 1) – is filtered
out. The five rep e ti t i ons of these signals are then averaged to yield one identificat i o n data
set for each condition and participant. Finally, us ing the calculated open-loop frequency
responses, pilot- vehicle system crossover frequencies and phase margins are determined.
III.E. Hypotheses
To assess possible differences in control strategy that result from the difference in experimen-
tal setup and environmental factors, the C0 and C0F no-motion conditions from the aircraft
part of the experiment are compared directly to th e equivalent S0 no-motion condition from
the SRS. Extreme care is taken to ensure th a t the conditions under which the in-flight and
simulator experiments are performed are as similar as possible. Hence, differences in con -
trol strategy observed between the S0 , C0, and C0F conditio n s are expected to be minor
compared to the effect of the variation in motion cueing.
The variation in simulator motion cueing evaluated for the comparison wit h in-flight
measurements of pilot control behavior consists of cueing conditions also evaluated in pre-
vious simulator experiments.
4, 5
Based on the findings of these two experiments and many
additional experiments performed in the past,
36
pilot tracking performance is expected to
improve and control activity will increase for improved pitch heave motion fidelity. However,
performance is expected to degrade when c.g. heave motion is present. Consistent with the
improvement in performance, an increase in d i st u r b a n ce crossover frequ e n cy and decrease in
disturbance phase margin is expected.
A change in pilot control strategy for improved motion fideli ty will be observed by an
increase in visual and vestibular gains, a decrease in visu a l lead and lag, and an increase
in visual and vestibular time delays, as observed in the previous experiments. However, as
found in Ref. 4, an increase in visual l ead is expected when c.g. heave is present. Due to
this effect of c.g. heave, it is hypothesized that the SRS condition S3 will yield pilot control
behavior that is closest to that observed in flight for condition CF.
Finally, the experiment described here allows for a direct comparison of subjective and
objective measurements of flight simulator motion fidelity for th e cond i t io n s evaluated in
the SRS. It is expected that both these sets of d ependent measures will show similar general
19 of 36

trends. The objective measurements of pilot behavior, however, are expected to allow for
better evaluat i on of the differences in fidelity between the mot io n conditio n s S1-S3.
IV. Results
This section presents the combined results of the seven p i l ot s who participated in the ex-
periment. In figu r es displaying da ta from all experimental conditions, the baseline condition
CF is marked by a gray horizontal line for reference. The error bar plot s are corrected for
between-subject variability by normalizing the subject means.
37
A repeated-measures multivariate analysis of variance (MANOVA) is perform ed first to
investigate if an overall significant effect exists in the calculated dependent measur es . The
analysis indicated that a significant effect is indeed present in the data. This allows for
the analysis of the dependent measures separately using a repeated-measures univariate
analysis of variance (ANOVA) to reveal any significant g r ou p differences. First, a MANOVA
and ANOVA are performed on condition s C0, C0F, and S0 – that is, the three conditi ons
without physical motion – to determine th e effects induced by the experiment environment or
apparatus (see Tables 2 and 4). Second, the analyses are performed on conditions S0, S1, S2,
S3, and CF, to reveal the effects of improved motion fidelity. In every ANOVA, Mauchly’s
test of sphericity is performed t o test if the assumption of sph er i ci ty is met (p > 0.05).
37
If
the assumption is not met, the d egr ees of freedom are corrected u si n g Greenhouse-Geisser
estimates of spherici ty.
IV.A. Aircraft Dynamics
To valida t e that pilo ts controlled the same aircraft dynam i cs in both the simulator and the
aircraft parts o f the experiment, the controlled pitch dynamics are identified using data from
both apparatus. An MLE procedure is used to estimate the parameter s of a two degree-
of-freedom short-period ap p r oximation of the aircraft dynamics.
28
The mean frequency
responses of the dynamic model im p l em e nted in the SRS are calcul at ed using the data of
conditions S0-S3 and all pilots, the Citation m ea n frequency responses are calculated using
the data of condition CF and all pilots. The frequency responses of the aircraft pitch and
c.g. heave dynamics, H
θ,δ
e
(jω) an d H
a
z
cg
,δ
e
(jω), ar e depicted in Figure 14.
In the Citation part of the experiment, there are sli ght deviations from the desired trim
condition for every experiment run. However, F ig u r e 14 shows that the mean responses for
the pitch and c.g. heave dynamics in both the SRS and the Citation are ap p r oximately
equal, which means that these slight deviations only had a negligible effect on the controlled
aircraft dynamics. Given a certain input, the variance accounted for (VAF) is a measure
of how well the output of the identified model describes the measured time-domain data.
15
20 of 36

ω
sp
(a) pitch magnitude
|H
θ,δ
e
|, -
ω, rad s
−1
SRS
Citation
10
-1
10
0
10
1
10
-2
10
-1
10
0
10
1
10
2
ω
sp
(b) c.g. heave magnitu d e



H
a
z
cg
,δ
e



, -
ω, rad s
−1
10
-1
10
0
10
1
10
1
10
2
10
3
(c) pitch phase
6
H
θ,δ
e
, deg
ω, rad s
−1
10
-1
10
0
10
1
-180
-90
0
(d) c.g. heave phase
6
H
a
z
cg
,δ
e
, deg
ω, rad s
−1
10
-1
10
0
10
1
-180
-90
0
Figure 14. Mean frequency responses of the aircraft dynamics (V = 160 kt an d h = 17, 000 ft).
For the short-period appr oximations identified here, the VAF is generally above 99% for the
pitch dynamics and above 96% for the c.g. heave dynamics, indicating th at the accuracy of
the identified models is very high.
IV.B. Subjective Evaluations
Figure 15 pr ovides an error bar plot of the simulator motion fidelity rating results. A
significant trend can be observed [F(3, 18) = 37.960, p < 0.01]. The figure also depicts the
mean data from individual pilot s. Th e results indicate that p i l ot s r at e t he condition wi t h o u t
simulator motion (S0) the lowest. However, one pilot consi st ently rated this condition with a
significantly higher value compared to the other pilots. All th e motion conditions (S1-S3) are
rated approximately equal, around 65%. However, the simulator condition with fi l te r ed c.g.
heave (S3) shows a slightly larger error bar, in dicating that pilots rate this motion condition
less consist ently, as can a l so be observed from the individual pilot data.
IV.C. Pilot Performance and Control Activity
Pilot performance and control activity are evaluated u si n g the variance of the err o r and pilot
control signals, respectively. The disturbance forcing function, target forcing function, and
21 of 36

motion fidelity rating, %
condition
7 pilots
mean data
S0 S1 S2 S3
0
20
40
60
80
100
Figure 15. Subjective motion fidelity ratings.
Table 5. ANOVA results of pilot performance, where ∗∗ is highly significant (p < 0.01), ∗ is
significant (0.01 ≤ p < 0.05), and − is not significant (p ≥ 0.05).
Indep en dent
Dependent measures
variables
σ
2
(e) σ
2
(e
d
) σ
2
(e
t
) σ
2
(e
n
)
Factor df F Sig. df F Sig. df F Sig. df F Sig.
Environment (S0,C0,C0F) 2,12 7.355 ∗∗ 2,12 1.000 − 2,12 5.716 ∗ 2,12 6.906 ∗
Motion (S0,S1,S2,S3,CF) 4,24 5.694 ∗∗ 4,24 13.651 ∗∗ 1.9,11.3 4.746 ∗ 2,24 2.188 −
remnant portions of the variance are determined using t he power spectral densities of the
signals at the input frequencies of the forcing functions.
2
The calculated variances for the error signal are d ep i ct ed in Figure 16(a) and the cor-
responding ANOVA results are given in Table 5. Figu r e 16(a) also depicts the means and
95% confidence intervals o f the total variance. Note that both the disturbance and target
forcing functions are scaled to induce an error variance of 0.4 deg
2
(see Section II.D). Figur e
16(a) in dicates that pilots can attenuate a sign i fi ca ntly higher percentage of the disturbance
forcing function as compared to the targ et forcing function, as the disturbance portions of
the variance bars are smaller than the target portions. This result is also found in previous
experiments.
16, 27
The ANOVA of the total variance of the err or in the no-motion conditions (Table 5)
indicates that overall performance is significantly better in the simulator no-motion cond i t i on
(S0) as compared to the aircraft no-motion conditions ( C0 and C0F). The ANOVA further
indicates that the disturbance component of the error variance is not significantly affected by
the experiment environment. However, in the simulator no-motion conditio n , the target and
remnant components of the variance are significantly lower than in the aircraft no-motion
conditions. This implies that the i m p r ovement in overa ll pilot performance in th e simulator
is caused by a better attenuation of the target signal.
The level of motion fidelity (conditions S0-S3, and CF) is also found to have a significant
effect on overall performance. Figure 16(a) indicates a slight improvement in performa n ce
22 of 36

σ
2
(e), deg
2
condition
(a) error signal
target
disturbance
remnant
S0 C0 C0F S0 S1 S2 S3 CF
0.00
0.05
0.10
0.15
0.20
0.25
0.30
0.35
σ
2
(u), V
2
condition
(b) control signal
S0 C0 C0F S0 S1 S2 S3 CF
0.0
0.2
0.4
0.6
0.8
1.0
Figure 16. Variance decomposition of pilot performance and control activity.
Table 6. ANOVA results of pilot control activity, where ∗∗ is highly significant (p < 0.01), ∗ is
significant (0.01 ≤ p < 0.05), and − is not significant (p ≥ 0.05).
Indep en dent
Dependent measures
variables
σ
2
(u) σ
2
(u
d
) σ
2
(u
t
) σ
2
(u
n
)
Factor df F Sig. df F Sig. df F Sig. df F Sig.
Environment (S0,C0,C0F) 2,12 0.080 − 2,12 0.048 − 2,12 0.115 − 2,12 0.076 −
Motion (S0,S1,S2,S3,CF) 1.3,8.0 1.869 − 4,24 4.126 ∗ 1.2,7.3 1.313 − 1.4,8.5 1.907 −
(decreasing error variance) when the fi delity of pitch heave is improved and a degrading
performance (increasing variance) when filter ed c.g. heave is present. This is also observed
in the experiment describ ed in Ref. 4. The disturbance component significantly decreases
when the level of motion fidelity is improved from S0 to CF, indicating that pilots are
better able to attenuate the d i st u r bance si gn a l . The ma in decrease in disturb a n ce variance
is pr es ent between the no-motion and th e motion conditions. In the aircraft condition CF,
the target component is significantly bigger than in the simulator condi t io n s S0- S 3. The
remnant component is not sign i ficantly affected by the m ot i on fidelity.
Figure 16(b) depicts the variances of the control sign al . The corresponding ANOVA
results are given in Table 6. No significant effects are found, except for the disturbance
component of the control signal in the motion fidelity conditions. In simulator condit io n
S3 the variance of the di st u r bance com ponent is significantly higher compared to th e other
conditions. Although not significant, an increase in total variance of the control signal can
be observed for the simulator conditions for i m p r oved motion fidelity, followed by a decrease
of control activity in the in - fl i ght condition CF.
23 of 36

(a) visual magnitude
|H
pe
|, -
ω, rad s
−1
10
-1
10
0
10
1
10
-2
10
-1
10
0
10
1
(b) vestibular magnitude
|H
pθ
|, -
ω, rad s
−1
10
-1
10
0
10
1
10
-2
10
-1
10
0
10
1
(c) visual phase
6
H
pe
, deg
ω, rad s
−1
10
-1
10
0
10
1
-360
-270
-180
-90
0
90
180
(d) vestibular phase
6
H
pθ
, deg
ω, rad s
−1
MLE, VAF=82.4%
FC
10
-1
10
0
10
1
-360
-270
-180
-90
0
90
180
Figure 17. Frequency responses of pilot visual and vestibular modalities (CF, pilot 4).
IV.D. Pilot Control Behavior
IV.D.1. Frequency Responses
The eight parameters of the multimodal pilot model depicted in Figure 6 are estimate d using
an MLE procedure on averaged time-domain data from the SRS and Citation con d i t i on s.
22
For the condition s without motion, the pilot model onl y contains t h e visu al channel, and the
number of parameters to be identified is reduced from eight to six. For the MLE procedure,
the pilot model is converted to a state-space rep r esentation in which the time delays are
implemented using fifth order Pad´e approximations.
Fifty iterations are perfor m ed with a genetic MLE algorithm, after which the solutions
are further optimized using a gradient-based Gauss-Newton optimi zat i on. The solution with
the lowest likelihood that occurs multiple times – indicating, a consistent global minimum
in the parameter search space – is chosen as the final parameter set. For all conditions of
every subject a stable global minimum is found.
Figure 17 depicts the frequency responses of the pilot model visual and vestibular channels
for condition CF performed by pilot 4. The Fou r i er coefficients (FC) of th e pil ot res ponse
functions can be calculated from the measured time-domain data independent of t h e selected
model structure and are also given in the figure.
15
The estimated pilot model responses
24 of 36

V AF , %
condition
S0 C0 C0F S0 S1 S2 S3 CF
60
65
70
75
80
85
90
95
100
Figure 18. Pilot model variance accounted
for.
σ
2
(u
e
)/σ
2
(u
θ
), %
condition
S0 S1 S2 S3 CF
0
5
10
15
20
25
30
35
40
Figure 19. Control variance fraction.
closely follow t h e calculated FC frequency responses, indicating a high accuracy of the model
in the frequency domain. The data presented in Figure 17 are rep r esentative for all conditions
and all pilots.
IV.D.2. Variance Accounted For
The VAF of the pilot model output is calculated as a measure of accuracy in the ti m e domain.
Figure 18 depicts th e means and 95% confidence intervals of the pilot model VAFs. The
total VAFs for all conditions are in the order of 85% a n d no significant trends are foun d for
changes in experiment environment an d apparatus, and motion fidelity [F(2, 12) = 1.874 ,
p > 0.05 and F(1.7, 10.1) = 0. 681 , p > 0.05]. This indicates that the multimodal pilot
model can accuratel y describe the pilot co ntrol signal data measured in both th e SRS and
the aircraft equally well. Also, in both t h e SRS and the aircraft, 85% of t h e averaged pilot
control outpu t can be described by linear response functions, revealing that pilots controlled
with th e same level of lin ea ri ty in both apparatus.
Figure 19 depicts the fraction between th e variances of the pilot visu al (u
e
) and the
pilot motion (u
θ
) contributions to the total pilot control si gn a l ( u) , see Figur e 6. In the
conditions without motion the contribution o f th e mot i on channel is zero and hence the
fraction between the two cont r ibutions is zero. In the condi t i on s with motion, the variance
of the vestibular contribution is around 20% of the variance of the vi su a l contribution to
the control signal. This indicates that pilots are mostly relyin g on their visual modality. A
significant effect is found in t he fraction between the variance contributions of both modalities
[F(1.9, 11.2) = 4.095, p < 0.05]. When the fidelity of pitch heave is improved from S1 to
S2, the fraction between the variances increases, in d i cat i ng that pilots rely more on motion
cues. However, for th e simulator cond it i on with filt er ed c.g. heave (S3) it decreases to a
value similar to the value in the aircraft condition (CF).
25 of 36

Table 7. ANOVA results of pilot model equalization parameters, where ∗∗ is highly significant
(p < 0.01), ∗ is significant (0.01 ≤ p < 0.05), and − is not significant (p ≥ 0.05).
Indep en dent
Dependent measures
variables
K
v
K
m
T
lead
T
lag
Factor df F Sig. df F Sig. df F Sig. df F Sig.
Environment (S0,C0,C0F) 2,12 0.680 − 2,12 4.675 ∗ 2,12 0.661 −
Motion (S0,S1,S2,S3,CF) 4,24 4.430 ∗∗ 1.3,7.9 2.764 − 4,24 60.957 ∗∗ 4,24 26.232 ∗∗
IV.D.3. Parameters
Means and 95% confidence intervals of the estimated multimodal pilot model par am et er s
are depicted in Figure 20. The ANOVA results for the equalization parameters of the pi lo t
model are summarized in Table 7. Figure 20(a) depicts t h e data for the visual gain K
v
.
This parameter is not sign ificantly affected by the experiment environment. However, there
is a significant effect of motion fidelity. An increase in visual gain with increasing simulator
motion fidelity can be observed, followed by a decrease for the in-flight motion condition.
Figure 20(b) depicts t h e data for the vestibular gain of the pilot model. This p ar am et er
is on l y estimated for the conditions where physical motion is present. The ANOVA on the
motion conditions indicates that the vestibular gain is not significantly affected by motion
fidelity.
Data for the visual le ad and lag time constants, T
lead
and T
lag
, are dep i ct ed in Figures
20(c) and 20( d ) , respectively. For the control of aircraft pitch dynamics, these two equaliza-
tion parameters are strongly coupled – that is, show the same trend – as can be observed
from the figure.
4, 5
The visual lead time constant is significantly affected by the experi-
ment environment. A slightly lower lead time constant can be observed for the in-flight
no-motion condition C0F. However, the visual lag time constant is no t s ig n i fi cantly affected
by experiment environment.
Both T
lead
and T
lag
show a highly significant decrease with the introduction of motion,
indicating a reduction in the amount of visual lead eq u al i za t io n . This effect is mainly present
between the no motion condition S0 and the r em ai ning motion conditions S1-CF. A slight
increase in both the l ea d and lag time constants can be ob s er ved when c.g. heave is present
(S3 and CF). This increase is also observed in the exper i m ent described in Ref. 4. Note that
for the conditions with motion (S1-CF), the visual lead an d lag time constants approximate
the characteristic time constants of the controlled dynamics, 1/ω
sp
and T
θ
2
, respectively.
32
This shows evid en ce that pilots adjust their visual equali zat i on to the controlled aircraft
dynamics much more accurately when simulator motion is present.
The ANOVA results for the limitation parameters of the pilot m odel are given i n Table 8.
The visual time delay is depicted in Figure 20(e). This parameter is significantly affect ed by
the experiment environment. In t h e simulator, the visual time delay is significantly lower.
26 of 36

K
v
, -
condition
(a) visual gain
S0 C0 C0F S0 S1 S2 S3 CF
0.6
0.8
1.0
1.2
1.4
1.6
1.8
2.0
K
m
, -
condition
(b) vestibular gain
S0 C0 C0F S0 S1 S2 S3 CF
0.14
0.18
0.22
0.26
0.30
0.34
ω
−1
sp
= 0.427 s
T
lead
, s
condition
(c) visual lead
S0 C0 C0F S0 S1 S2 S3 CF
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
T
θ
2
= 0.895 s
T
lag
, s
condition
(d) visual lag
S0 C0 C0F S0 S1 S2 S3 CF
0.0
0.5
1.0
1.5
2.0
2.5
3.0
τ
v
, s
condition
(e) visual time delay
S0 C0 C0F S0 S1 S2 S3 CF
0.15
0.17
0.19
0.21
0.23
0.25
τ
m
, s
condition
(f) vestibular time delay
S0 C0 C0F S0 S1 S2 S3 CF
0.02
0.04
0.06
0.08
0.10
0.12
0.14
ζ
nm
, -
condition
(g) neuromuscular dampin g
S0 C0 C0F S0 S1 S2 S3 CF
0.12
0.14
0.16
0.18
0.20
0.22
0.24
ω
nm
, rad s
−1
condition
(h) neuromuscular frequency
S0 C0 C0F S0 S1 S2 S3 CF
7.5
8.0
8.5
9.0
9.5
10.0
10.5
11.0
Figure 20. Multimodal pilot model parameters.
27 of 36

Table 8. ANOVA results of pilot model limitation parameters, where ∗∗ is highly significant
(p < 0.01), ∗ is significant (0.01 ≤ p < 0.05), and − is not significant (p ≥ 0.05).
Indep en dent
Dependent measures
variables
τ
v
τ
m
ζ
nm
ω
nm
Factor df F Sig. df F Sig. df F Sig. df F Sig.
Environment (S0,C0,C0F) 2,12 4.317 ∗ 2,12 5.601 ∗ 2,12 17.286 ∗∗
Motion (S0,S1,S2,S3,CF) 4,24 22.693 ∗∗ 1.3,8.1 6.295 ∗ 4,24 1.033 − 4,24 2.985 ∗
Table 9. ANOVA results of cros sover frequencies and phase margins, where ∗∗ is highl y
significant (p < 0.01), ∗ is significant (0.01 ≤ p < 0.05), and − is not significant (p ≥ 0.05).
Indep en dent
Dependent measures
variables
ω
c,d
ω
c,t
ϕ
m,d
ϕ
m,t
Factor df F Sig. df F S ig. df F Sig. df F Sig.
Environment (S0,C0,C0F) 2,12 0.650 − 2,12 0.650 − 2,12 0.724 − 2,12 0.724 −
Motion (S0,S1,S2,S3,CF) 4,24 6.211 ∗∗ 4,24 13.710 ∗∗ 4,24 3.790 ∗ 1.8,10.8 10.069 ∗∗
In additi o n , motion fide li ty introduces a highly si gnificant effect. This effect refl ec t s an
increase in visual time delay for increasing levels of motion fidelity, as is also observed i n a
previous experiment.
5
The biggest incr ea se is present between the simulator conditions and
the aircraft condi t io n CF. Figure 20(f) depicts the results for the vestibular time delay. The
vestibular delay increases for increasing motion fidelity, a significant effect. There is mainly
an increas e between the conditions without (S1-S2) and with c.g. heave motion (S3-CF).
The neuromuscular dampin g ζ
nm
and frequency ω
nm
– characterizing the neuromuscular
dynamics – are dep i ct ed in Figures 20(g) and 20(h), respect i vely. Both the neuromuscular
damping and frequ en c y are affected by the experiment environment. For the no-m ot i on
conditions in the aircr aft , ζ
nm
and ω
nm
are lower. In addition, the neuro muscular frequency
is significantly affected in the motion fidelity conditions. It first inc r eases for higher pitch
heave motion fide li ty levels, but then decr e ases when c.g. heave motion is present.
IV.E. Crossover Frequencies and Phase Margins
The crossover frequencies and phase mar gi n s for the di st u r bance and target open-loop re-
sponses – characterizing pilot-vehicle performance and stability in attenuating the distur-
bance and target forcing functions – a re depicted in Figure 21. The corresponding ANOVA
results are given in Table 9. Note that the disturbance and target o pen-loop responses,
and thus disturban ce and target crossover frequencies and phase margins, are equal for the
conditions without motion (H
pθ
= 0) as can be der i ved from Equations 6 and 7.
The disturbance and target crossover frequencies are depicted in Figures 21(a) and 21(b),
respectively. The crossover frequencies are not affected by the experiment environment. The
disturbance crossover fr equency shows an increase for increasing motion fidelity, a significant
28 of 36

ω
c,d
, rad s
−1
condition
(a) distu rb an ce crossover frequency
S0 C0 C0F S0 S1 S2 S3 CF
1.0
1.5
2.0
2.5
3.0
3.5
4.0
ω
c,t
, rad s
−1
condition
(b) target crossover frequency
S0 C0 C0F S0 S1 S2 S3 CF
1.0
1.5
2.0
2.5
3.0
3.5
4.0
replacemen
ϕ
m,d
, deg
condition
(c) distu rb an ce phase margin
S0 C0 C0F S0 S1 S2 S3 CF
20
30
40
50
60
70
80
90
ϕ
m,t
, deg
condition
(d) target phase margin
S0 C0 C0F S0 S1 S2 S3 CF
20
30
40
50
60
70
80
90
Figure 21. Crossover frequencies and phase margins.
effect. This effect is mainly present between the condition without motion S0 and the
remaining motion conditions. A significant decrease in target crossover frequency can be
observed for increasing mot io n fidelity. An increase in distu r b a n ce crossover frequency and
decrease in target crossover fr eq u en c y when add i n g pitch motion is also observed in p r ev i ou s
experiments.
4, 5
Both the disturbance and t ar g et phase margins (Figures 21(c) and 21(d), respectively) are
not significantly affected by experiment environment. The improvement in motion fidelity
yields a s ig n i fi cant effect for the disturb a n ce phase margin. For the increase in disturb ance
crossover frequency a slight decrease in corresponding dist u r b ance phase margin can be
observed. Consistent with th e decrease in target crossover frequency shown in Figure 2 1( b )
a signifi cant increase in phase margin is visi b l e in Figure 21(d).
V. Discussion
Seven experienced pilots participated in an experiment where, for the first tim e, multi-
modal pilot pitch control behavior is compared between real flight and fou r different motion
conditions in a motion-base flight simulator. The purpose of the experiment is to deter-
29 of 36

mine th e effects of simulator motion fidelity on pilot performance and multimodal control
behavior.
V.A. Experiment Environment and Apparatus
To isolat e the effects of limited simulator m o t io n , it is extremely important that all the
remaining experimental variables – such as sidestick dynamics, display characteristics, and
instructions to pilots – do not affect the comparison of measurements taken in the simulator
and in the aircraft (Section II.F). Considerable effort is put into matching these experimental
variables in the two apparatus as closely as possible. However, some differences are known
to be present. Some examples of differences are that pilots are wearing a hood in the aircraft
but not in the simulator, much more ambient light is present in the aircraft as compared to
the simulator, and the displ ay in the aircraft is slightly tilted com p ar e d to the display in the
simulator (see Figures 8 and 12).
To investigate if these additional d iff er en ce s in experimental variables have an effect on
performance and control behavior, t hree conditions are performed where pilots controlled
the same aircraft dynamic model without any motion feed b ack. S0 is performed in the
simulator, C0 in the aircr a ft on the ground, and C0F in the aircraft in flight. The dependent
measures from these three conditions are analyz ed using an ANOVA to determine if the
different experiment environment or apparatus introduces any significant effects. Despite
our efforts to minimize differences between the experimental setups, significant effects are
found in some of the dependent measures.
Pilot performance in the simulator is slightly better as compared to the aircraft. Th i s
is d ue to a better target error reduction in the simulator. As the target error is directly
visible on the display, this is most likely the result of better visibility of the target error
on the simulator display. In addition, the remnant component of the error is smaller in the
simulator. Differences in ambient light condit io n s, the sli ghtly tilted screen in the simulator,
or the difference i n graphics processing hardware between th e two setups can be the cause of
this. The slightly higher pilot model visual time delay in the aircraft also indicates that more
time is needed to process the information from the aircraft visual display. Note, however,
that the latency in the display hardware is measured to be the same in both the aircraft and
the simulator.
Finally, the pilot model neuromuscular dam ping ratio and natural frequency in the ai r -
craft are significantly lower than in the simulator. Althou gh the force-voltage characteristics
of both sticks are exactly the same, small differences in other stick characteristics or the rel-
ative position of the pilo t with respect to the sid e st ick probably caused this significant effect.
The sidestick in the simulator is an active stick t h a t is set up to be completely fixed during
30 of 36

the experiment. The force sidestick in the aircraft is a passive stick that moves slightly when
force is applied.
Overall, mor e highly significant effects are found between the m ot i on fidelity conditions.
In addition, the effects found between conditions S3 and CF are general ly a continuation of
the trend for impr oving motion fidelity in th e simulator (S0-S3). Therefor e, the observed
effects in the de pendent measures between the simulator conditions S0-S3 and the in-flight
condition CF are expected to be mainly caused by the change in motion fidelity and are only
slightly affected by the differences in experimental setup.
However, cross coupling between motion fidelity and other differences in experimental
variables can also be present. Fo r example, t h e hood pilots wore in the aircraft can have no
effect in the in-flight aircraft cond i t io n without motion, but can have had a significant effect
in the aircraft in full motion due to its momentum. These cross-coupling effects cannot be
examined using th e current experiment setup.
Based on pilot comments, there is no indication of a significant psychological effect of
controlling the actual aircraft in flight as opposed to controlling in th e relatively safe en-
vironment of a flight simulator. However, a significant smaller visual perception gain is
observed for the in-flight full-motion condition, possibly indicating that pilots control with
more caut i on in the real aircraft.
To eliminate fully the experimental effects other than the effects of motion fidelity, the
entire experiment should be conducted in a single environment and apparatus. This requi r es
a flight simulator with a very large mo t io n space, to allow for the simulation of fu l l unfiltered
aircraft motion. Another possibility i s to utilize the experimental FBW system in the Cita-
tion II laboratory aircraft to simulate filtered aircraft motion in flight. However, without the
possibility of direct lift control in the current aircraft con fi g u r at i on , pitch and heave motion
will be coupled.
V.B. Motion Fidelity
Pilot performance, determined by the total variance of the error, is significantly aff ect ed wh en
motion fidelity is improved to full air cr a ft motion in the aircraft. A significant improvement
in performance is observed when pitch heave moti on fidelity is improved, followed by a
reduction in performance when c.g. heave is present. Th e reduction in overall tracking
performance with c.g. heave shows that this heave component acts as a disturbance on
the aircraft pitch rotational and pitch heave motion. The pilot is less ab l e to perceive the
motion components directly related to pitch, decreasing the pilots’ use of motion cues to
improve overall performance.
4
This indicates that adequate simulation of c.g. heave motion
is requir e d to not let pilots perform better in the simulator as compared to real flight.
31 of 36

An improvem ent in disturbance-rejection performance for improved motion fideli ty is
observed by a significant reduction of the distu r b an c e variance component of the error. In
addition, a significant incr ease in disturbance crossover frequency and a reduction in the
disturbance phase margin are observed fo r improved motion fidelity. This re veals that pilots
are able to compensate better for the disturbance as motion fidelity is impr oved.
Multimodal pilot control behavior is significantly affected by the level of motion fidelity.
The most significant effects ar e found for the visual lead and lag time constants, and th e
visual and motion perception time delays. Visual lead significantly decreas es wh en t he level
of pitch heave motion fidelity is improved. In addition, a small increase in visual lead can be
observed when c.g. heave motion is present. This is consistent with previous resea r ch and
again sh ows that c.g. heave reduces the pilots’ use of physical motion cues in exchange for
visual lead.
4
A significant decrease in the variance fractio n between vestibular and visual
contributions to the pilot control signal for condit i on s with c.g. heave motion also supports
this finding. The lag time const ant, which is strongly coupled with t h e lead time constant,
shows similar effects. For the conditions with motion (S1-CF) the lead and l a g time constants
approximate the characteristic time constants of the aircraft pitch dynamics, indicating that
pilots can adjust their visual equalization to th e controlled dynamics much more accurately
when moti on is present.
The visual perception time delay increases significantly by i m prov ing motion fidelity. A
remarkable result is the significa ntly higher visual time delay for the in-flight full- m ot i on
condition, as compar ed to the simulator conditions. Differences between the display in the
aircraft and the si mulator might have contributed to this effect. However, as the differences
in visual time delays between the no-motion condi ti o n s performed in the aircraft and the
simulator are much smaller, this effect is mos t likely al so caused by the level of motion
fidelity. Finally, the vestibular time delay shows a significant increase when c.g. heave
motion is present.
An increase in perceptual time delays for improved simulator motion fidelity has also been
observed in previous experiments where a combined target-following disturbance-rejection
task is used.
4, 5
In this type of task, the visual and physical moti on cues provide conflicting
information (see Figure 2). As motion fidelity improves, the conflict between the two cues
increases, resulting in an increase in perceptual time d el ays, as more ti m e is needed to process
the infor m at i on .
When su bjective and objective measures are compared, the same overall trend between
no-motion and motion conditions can be observed. For example, the rating between the
simulator no-mot i on and motion con d i t i on s increases, while t h e pilot model lead an d lag
time constants decrease. However, t h e objective measures are clearly affected between the
different simulator motion conditions, whereas pil ot s rate these conditions equally. Although,
32 of 36

to gain pilot acceptance, subjective comments and ratings should still be an important part
of the motion filter tuning process, this indicates that the presented objective measures have
more potential to determine simulator motion fidelity.
Based on the four motion conditions evaluat ed in the current experiment, the cond i t i on
with full pitch rotati on al motion and filtered pitch and c.g. heave (S3) induces per for m an c e
levels and pilot control behavior most comparable to the baseline full-motion condition in the
aircraft. However, more research is needed to determine a set of optimal simulator motion
cueing settings that induce multimodal pil ot control behavior that best approximates in-
flight pilot control behavior.
Finally, it should be noted that aircraft size is an important factor in t h i s discussion.
This expe ri m ent was conducted in a Cessna Citation II, which is relatively small. In large
commercial airliners, the pilot position is much further in front of the c.g., increasing the
relative strength of the pitch heave component. This most likely changes how pilot control
behavior is affected by different levels of heave motion fidelity and suggests that t h e results
presented here cannot be directly extended to larger aircraft. Some prelimin ar y research on
this topic has been perfo rmed recently.
38
However, mor e research is req u i r ed to address this
topic fu ll y.
VI. Conclusions
To investigat e how multimodal pilot control behavior is affected by simulator motion
fidelity, seven pilots per for m ed a pitch target-following disturbance-rejection task in a sim-
ulator under different motion cueing settings. In addition, they performed the exact same
task in an aircraft in flight, the baselin e condition. Pilots significantly change their control
strategy for different levels of simulator motion fi d el i ty and also compared to the aircraft.
This change in control strategy is apparent fr om a change in the estim at ed multimodal pi-
lot model parameters. When improving m ot i on fidelity, visu a l and vestibular time delays
increase, whil e the visual lead time constant decreases, allowing for improved pilot perfor -
mance. Multi m odal pilot control behavior in the simulator motion con d i t io n with full pit ch
motion and filtered pitch and c.g. heave motion best app r oximates in- fl i ght pilot control
behavior.
Acknowledgments
The aut h or s would like to tha n k all members of the Flight Department of the Delft
University of Technology Faculty of Aerospace Engineering for their help in preparing the
in-flight experiments. We would also like to thank the Netherlands’ National Aerospace
33 of 36

Laboratory (NLR) for their support in p r ep a r ing and conducting the experiments. Finally,
we would like to express our gratitude to the pilots wh o participa t ed in the experiment.
This project was su p ported by the Dutch Technology Foundation (STW), the applied
science d i vi si o n of the Netherlands Organization for Scientific Research (NWO), and the
technology program of the Mini st r y of Economic Affairs.
References
1
Feddersen, W. E., “The Role of Motion Information and its Contribution to Simulation Validity,”
Tech. Rep. D228-429-001, Bell Helicopter Company, April 1962.
2
Jex, H. R., Magdaleno, R. E., and Junker, A. M., “Roll Tracking Effects of G-vector Tilt and Various
Types of Motion Washout,” Fourteenth Annual Conference on Manual Control, University of Southern
California, Los Angeles (CA), April 25–27 1978, pp. 463–502.
3
Stapleford, R. L., Peters, R. A., and Alex, F. R., “Experiments and a Model for Pilot Dynamics with
Visual and Motion Inputs,” NASA Contractor Report NASA CR-1325, NASA, 1969.
4
Zaal, P. M. T., Pool, D. M., de B r ui n, J., Mulder, M., and Van Paassen, M. M ., “Use of Pitch and
Heave Motion Cues in a Pitch Control Task,” Journal of Guidance, Control, and Dynamics, Vol. 32, No. 2,
March–April 2009, pp. 366–377.
5
Pool, D. M., Zaal, P. M. T., Van Paassen, M. M., and Mulder, M., “Effects of Heave Washout Settings
in Aircraft Pit ch Disturbance Rej ect ion, ” Journal of Guidance, Control, and Dynamics, Vol. 33, No. 1,
Jan.–Feb. 2010, pp. 29–41.
6
Van den Berg, P., Zaal, P. M. T., Mulder, M., and Van Paassen, M. M. , “Conduct in g multi-modal pilot
model identification - Results of a simulator experiment,” Proceedings of the AIAA Modeling and Simulation
Technologies Conference and Exhibit, Hilton Head (SC), No. AIAA-2007-6892, 20–23 Aug. 2007.
7
Hess, R. A. and Marchesi, F., “Analy ti cal Assessment of Flight Simu lator Fidelity Using Pilot Models,”
Journal of Guidance, Control, and Dynamics, Vol. 32, No. 3, May–June 2009, pp. 760–770.
8
Mooij, H. A., “In-Flight Measured Human Pilot Describing Function and Remnant f or Pitch Atti-
tude Control,” Proceedings of the Ninth Annual Conference on Manual Control, Massachusetts Institute of
Technology, Cambridge (MA), May 23–25 1973, pp. 311–318.
9
van Gool, M. F. C. and Mooij, H. A., “A Comparison of In-Flight and Gr oun d- Base d Pitch Attitude
Tracking Experiments,” Proceedings of the Twelfth Annual Conference on Manual Control, University of
Illinois, Urbana (IL), May 25–27 1976, pp. 443–454.
10
Newell, F. D. and Pietrzak, P. E., “In-Flight Measurement of Human Response Characteristics,”
Journal of Aircraft, Vol. 5, No. 3, May–June 1968, pp. 277–284.
11
Newell, F. D. and Smith, H. J., “Human Transfer Characteristics in Flight and Ground Simulation
for a Roll Tracking Task,” Tech. Rep. NASA TN D-5007, NASA, 1969.
12
Hess, R. A. and Mnich, M. A., “Identification of Pilot-Vehicle Dynamics from In-Flight Tracking
Data,” Journal of Guidance, Control, and Dynamics, Vol. 9, No. 4, July–Aug. 1986, pp. 433–440.
13
Steurs, M., Mulder, M., and Van Paassen, M. M., “A Cybernetic Approach to Assess Flight Sim-
ulator Fidelity,” Proceedings of the AIAA Modelling and Simulation Technologies Conference and Exhibit,
Providence (RI), No. AIAA-2004-5442, 16–19 Aug. 2004.
34 of 36

14
Ferguson, S. W., Clement, W. F., Cleveland, W. B., and Key, D. L., “Assessment of Simulation Fidelity
Using Measurements of Piloting Technique in Flight,” Proceedings of the 40th American Helicopter Society
Annual Forum, Arlington (VA), May 16–18 1984.
15
Nieuwenhuizen, F. M., Zaal, P. M. T., Mulder, M., Van Paassen, M. M., and Mulder, J. A., “Modeling
Human Multichannel Perception and Control Using Linear Time-Invariant Models,” Journal of Guidance,
Control, and Dynamics, Vol. 31, No. 4, July–Aug. 2008, pp . 999–1013.
16
Zaal, P. M. T., Pool, D. M., Mulder, M., and Van Paassen, M. M., “Multimodal Pilot Control Behavior
in Combined Target-Following Disturbance-Rejection Tasks,” Journal of Guidance, Control, and Dynamics,
Vol. 32, No. 5, Sept.–Oct. 2009, pp. 1418–1428.
17
Zaal, P. M. T., Pool, D. M., Postema, F. N., In ’t Veld, A. C., Mulder, M., Van Paassen, M. M., and
Mulder, J. A., “Design and Certific ation of a Fly-By-Wire System with Minimal Impact on the Original
Flight Controls,” Proceedings of the AIAA Guidance, Navigation, and Control Conference and Exhibit,
Chicago (IL), No. AIAA-2009-5985, 10–13 Aug. 2009.
18
McRuer, D. T. and Jex, H. R., “A Review of Quasi-Linear Pilot Models,” IEEE Transactions on
Human Factors in Electronics, Vol. HFE-8, No. 3, 1967, pp. 231–249.
19
Schmidt, S. F. an d Conrad, B., “Motion Drive Signals for Piloted Flight Simulators,” Tech. Rep.
NASA CR-1601, Ames Research Center, National Aeronautics and Space Adminis tr ati on, 1970.
20
Kaljouw, W. J., Mulder, M., and Van Paassen, M. M., “Multi-loop Identification of Pilot’s Use of
Central and Peripheral Visual Cues,” Proceedings of the AIAA Modelling and Simulation Technologies Con-
ference and Exhibit, Providence (RI), No. AIAA-2004-5443, 16–19 Aug. 2004.
21
Van Paassen, M. M. and Mulder, M., “Identification of Human Operator Control Be haviour in
Multiple-Loop Tracking Tasks,” Proceedings of the Seventh IFAC/IFIP/IFORS/IEA Symposium on Anal-
ysis, Design and Evaluation of Man-Machine Systems, Kyoto Japan, Pergamon, Kidlington, Sept. 16–18
1998, pp. 515–520.
22
Zaal, P. M. T., Pool, D. M., Chu, Q. P. , Van Paassen, M. M., Mulder, M., and Mulder, J. A., “Modeling
Human Multimodal Perception and Control Using Genetic Maximum Likelihood Estimation,” Journal of
Guidance, Control, and Dynamics, Vol. 32, No. 4, July–Aug. 2009, pp. 1089–1099.
23
Nieuwenhuizen, F. M., Paassen, M. M. V., Mulder, M., and B¨ulthoff, H. H., “Implementation and
Validation of a Model of the MPI Stewart Platform,” Proceedings of the AIAA Guidance, Navigation, and
Control Conference and Exhibit, Toronto (ON), Canada, No. AIAA-2010-8217, 2–5 Aug. 2010.