ArticlePDF Available

Response times and gaze behavior of truck drivers in time critical conditional automated driving take-overs

Article

Response times and gaze behavior of truck drivers in time critical conditional automated driving take-overs

Abstract and Figures

The desire to enable conditional automated driving (CAD) in the near future, entails the challenge to manage drivers’ safe transitions from automation back to manual control. Several factors have been considered in recent years in the passenger car context, while the truck has largely been disregarded. For the first time take-over behavior of heavy-duty truck drivers in time critical take-overs is considered in CAD research. This study analyzes the effect of non-driving related tasks, CAD duration, take-over situations and number of take-overs on reaction times of truck drivers. Gaze behavior was tracked with a remote eye-tracker; reaction times and driver interaction during CAD drives was recorded and analyzed. Two different non-driving related tasks were presented in nine unique take-over situations, while also controlling for the duration of CAD. Contrary to assumption, no influence of non-driving related tasks or CAD duration on reaction times is found. Notably, different reaction times are recorded due to the nine unique take-over situations. Finally, it is shown that our take-over times decrease over the course of the experiment and are far lower than other published reaction times (M = 1.35 s) in the passenger car context. The findings are discussed and implications with regard to other published studies are drawn.
Content may be subject to copyright.
1 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
1
Alexander Lotz a, b * rene_alexander.lotz@daimler.com
2
Nele Russwinkel b nele.russwinkel@tu-berlin.de
3
Enrico Wohlfarth a
4
5
a Daimler AG, TP/VES HPC T332, 70546 Stuttgart, Germany
6
b Technische Universität Berlin, Marchstr. 23, Sekr. MAR 3-2, 10587 Berlin, Germany
7
8
* Corresponding author Tel.:+49 711 17 59925
9
10
Abstract
11
The desire to enable conditional automated driving (CAD) in the near future, entails the challenge to manage drivers’
12
safe transitions from automation back to manual control. Several factors have been considered in recent years in
13
the passenger car context, while the truck has largely been disregarded. For the first time take-over behavior of
14
heavy-duty truck drivers in time critical take-overs is considered in CAD research. This study analyzes the effect of
15
non-driving related tasks, CAD duration, take-over situations and number of take-overs on reaction times of truck
16
drivers. Gaze behavior was tracked with a remote eye-tracker; reaction times and driver interaction during CAD
17
drives was recorded and analyzed. Two different non-driving related tasks were presented in nine unique take-over
18
situations, while also controlling for the duration of CAD. Contrary to assumption, no influence of non-driving related
19
tasks or CAD duration on reaction times is found. Notably, different reaction times are recorded due to the nine
20
unique take-over situations. Finally, it is shown that our take-over times decrease over the course of the experiment
21
and are far lower than other published reaction times (M=1.35 sec) in the passenger car context. The findings are
22
discussed and implications with regard to other published studies are drawn.
23
24
25
Keywords Conditional Autonomous Driving; Driving simulator; Non-driving related tasks; CAD duration; Take-over
26
situations; Truck;
27
28
29
30
31
Response Times and Gaze Behavior of Truck Drivers in Time
Critical Conditional Automated Driving Take-overs
2 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
Response Times and Gaze Behavior of Truck Drivers in Time Critical
32
Conditional Automated Driving Take-overs
33
34
1 Introduction
35
Current advanced driver assistance systems (ADAS) generate the possibility of automated driving. While the
36
vehicle is assigned to safely navigate through the environment for this driving scenario, conditional automated
37
driving (Level 3; SAE J3016, 2018) requires the driver to transition back to full control of the vehicle once
38
system limits are reached. Due to the complexity of sharing the driving task, car manufacturers are currently
39
investigating the implementation of Level 3 on highway and interstate roads. Although driving on a highway
40
may occur at much greater average speeds, the roads and behavior are extremely regulated, making the
41
technical controllability simpler than in urban scenarios. Higher automation levels (Level 3 & 4) allow drivers
42
to shift their attention away from the driving task (Gasser, et al., 2012) and potentially focus on non-driving
43
related tasks (NDRT). This transition phase between the driving and non-driving related task creates a time
44
and safety critical challenge. Generally, the human perception of the environment and relevant factors requires
45
time and can cause higher error rates (Allport, et al., 1994). Prolonged reaction times have also been identified
46
specifically for increased automation levels in vehicles (Young, et al., 2007). The question that arises when
47
designing a Level 3 system is; how much time does the driver need to process information in Level 3 and
48
regain control of the vehicle safely (Gold, et al., 2013)? Different aspects of take-over behavior have recently
49
been investigated for the qualitative and quantitative description of take-overs. These aspects can broadly be
50
categorized into four classes, driver, environment, vehicle and human-machine interaction (HMI), according to
51
Vogelpohl et al. (2016), which will be described in section 1.1.
52
Typically, if the Level 3 technical system is unable to solve a driving situation and the vehicle is travelling with
53
any speed, take-over is time critical due to obstacles in its path. At higher speeds, obstacles or humans in the
54
environment and/or dangerous roads also makes situations safety critical. Two different approaches can be
55
adopted to define adequate take-over times. Either a globally valid fixed time is chosen for all take-over
56
situations, or conditional times depending on the abovementioned factors driver, environment, vehicle and HMI
57
are assigned. For the second approach, the driver and environment factors have to be monitored continuously
58
to adjust the adequate take-over time accordingly, as vehicle and HMI factors usually remain constant within
59
one vehicle. This is a paradigm shift in the traditional human-machine relationship in which the human no
60
longer needs to monitor the state of the vehicle or machine, but the vehicle has to monitor the environment
61
and the driver, roles are reversed. Some of the driver factors can be measured through remote measurement
62
systems, while most cognitive factors cannot or require on-body sensors. Such determinable factors
63
incorporate gaze behavior, physical movements of the driver and NDRT interaction (Ohn-Bar, et al., 2014).
64
Since understanding of the influencing factors on the driver’s reaction is needed, better measurement
65
techniques and algorithms are required that potentially predict the right time interval for secure human
66
behavior (Lotz, et al., 2019). The second approach for the definition of adequate take-over times is a more
67
elegant, however, technically more complex due to additional sensors and algorithms necessary for driver
68
observation.
69
The aims of the study are to investigate influencing factors which possibly cause varying behavioral driver
70
reactions. The independent variables chosen for this study are categorized as environmental factors and
71
incorporate the temporal duration of automation and type of NDRT. In addition, it is our goal to identify if
72
learning behavior can be established in the obtained data. The study aims also include the analysis with regard
73
to take-over quality in temporal critical take-over situations. Gaze behavior during take-over will be consulted
74
for the determination of visual reaction times as well as the construction of situation awareness as a measure
75
of take-over quality. Within this study situation awareness refers to the perception, comprehension and
76
projection of environmental objects according to Endsley (1995). Based on the findings, applicability of the
77
abovementioned second approach is discussed.
78
Notably, driver availability is not just an absolute description of the current driver state but a relative
79
assessment that is influenced by the situational demands” (Marberger, et al., 2017). The underlying question
80
that needs to be answered when defining take-over time reserves is, which factors influence reaction times,
81
as previously stated by Gold et al. (2013). While a global fixed take-over time might cause safety issues due
82
to extensive complex situations, variable take-over times could also have a negative effect, by warning too
83
late or misjudging situations leading to safety critical take-over. The identification of relevant factors to allow
84
for variable warning times is paramount. Additionally, this study should reproduce some of the findings
85
regarding the influencing factors automation duration and NDRT from the passenger car for the truck domain,
86
closing the gap of research between domains. Strongly differentiating NDRT (e.g. reading on a fixed screen in
87
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 3
the cockpit and playing an instrument) surely present different reaction times, due to different modalities
88
being required for both tasks and objects causing physical hindrance in the take-over behavior. Due to the
89
amount of possibilities, conducting validation of each feasible NDRT and its influence on take-over is
90
impossible. It would be favorable if observation of temporal short behavioral driver operations, e.g. cascade
91
of fixations or movements, before take-over situations exhibit evidence about take-over times. These smaller
92
behavioral operations would increment to a large NDRT with operations being exchangeable between different
93
tasks, i.e. fixation of a screen and touching the surface. Previous work has shown that information is present
94
in the data, however, misclassification rates are too high for safe predictions of take-over times (Lotz, et al.,
95
2019).
96
To investigate the four groups of factors by Vogelpohl et al. (2016), a simulator study was designed to
97
determine if behavioral factors can be identified for differences in take-over behavior. Three relevant take-over
98
times according to Damböck (2013) will be reported in the following sections. The take-over procedure
99
typically incorporates the shift of visual attention towards the driving scene, forming motor readiness and
100
showing a reaction at the steering wheel and/or ac-/decelerator pedals. This study presents results from the
101
analysis of the four main influencing factors; NDRT, situation type, trial and CAD duration, to investigate the
102
possibility of designing variable warning times. Eriksson et al. (2017) give an overview of 25 publications
103
investigating take-over behavior in automated vehicles. Reaction times at the steering wheel or pedals vary
104
between 1.14 15 seconds, with the majority of reaction times measuring between 2 3.5 seconds.
105
Vogelpohl et al. (2016) cite a reaction time bandwidth of 3 8 seconds. These references will be consulted
106
for comparison to the results obtained in this study to draw the bridge between the truck and passenger car
107
domain. Table 1 holds acronyms utilized in this study.
108
Table 1: Acronyms within publication for further reference.
109
Acronym
Definition
ADAS
Advanced Driver Assistance System
CAD
Conditional Automated Driving (Level 3)
HMI
Human-machine interaction
NDRT
Non-Driving Related Task
RtI
Request to Intervene (a.k.a. Take-over Request TOR)
TTC
Time to Collision
TTEoR
Time to Eyes on Road
TTFR
Time to First Reaction
TTHoS
Time to Hands on Steering
110
1.1. Factors influencing take-over behavior
111
As mentioned previously, Vogelpohl et al. (2016) define four classes of influencing factors on take-over
112
behavior that will be introduced and discussed in the following.
113
1.1.1. Driver Factors
114
Firstly, driver factors are a prime subject matter when analyzing Level 3. Included in this factor class are
115
behavioral reactions drivers demonstrate while driving or engaging in NDRT. The driver factors entail all driver
116
knowledge and personal characteristics that can vary over long- or short-term. Fatigue or situational
117
awareness are part of this first class. Most of these characteristics are limited to internal abstract
118
psychological constructs, e.g. trust in automation, or anthropological measures that can influence take-over.
119
These factors are difficult to measure and manipulate objectively due to their level of abstraction. Several
120
studies have investigated widely accepted constructs in the passenger car context. Specifically, fatigue does
121
not seem to negatively affect the drivers’ take-over capabilities (Weinbeer, et al., 2017). Although task-related
122
fatigue can be induced through NDRT and measured through eye closure and head movement changes
123
(Schmidt, et al., 2016; Jarosch, et al., 2017), recent findings also present results in which engaging and
124
activating NDRT can manage and reduce driver drowsiness (Weinbeer, et al., 2019). Typically, the train of
125
thought when conducting a Level 3 experiment, is to have very few take-over situations to exclude the
126
4 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
possibility of learning. Instead of trying to exclude any possibility of learning, this study will present multiple
127
take-over situations. Therefore, learning effects are expected to reduce the time to first reaction, as defined
128
in section 2.1 (Hypothesis 1).
129
1.1.2. Environmental Factors
130
The second class, environmental factors, are widely considered by previous published studies regarding take-
131
over behavior during Level 3, e.g. Damböck (2013), Merat, et al., (2014), Gold, et al., (2016). These factors
132
are primarily investigated as they are easily manipulated in experimental designs and can be measured
133
objectively. Included in this class are factors such as type of driving environment, criticality of take-over
134
situations, CAD duration and NDRT. As the effects of a NDRT may influence a driver, but are induced through
135
the environment, we chose to define NDRT as an environmental factor, altering the original definition of
136
Vogelpohl et al. (2016). Most current Level 3 studies are limited to highway scenarios and Gold et al. (2016)
137
find a negative influence of higher traffic densities on take-over performance in the passenger car context.
138
Also, higher complexities of take-over situations seem to elongate take-over times (Damböck, 2013).
139
Prolonged take-over times for high situation complexity can be explained cognitively through the multitude of
140
objects that need to be perceived and processed, leading to more complex planning processes and reactions
141
as proposed by Endsley (1995). While cognitive processing of the environment cannot be monitored
142
objectively, the surroundings that are processed by the driver can be measured and described through camera
143
systems, radars and lidars in a mathematical sense. However, a clear description method of vehicle
144
surroundings in take-over situations has not been generated thus far (Schneider, 2009), making it difficult to
145
assign environmental complexities to differentiating take-over performance.
146
Feldhütter et al. (2016) examined CAD durations of five and twenty minutes, findings showed temporal
147
differences in reaction times of first gaze movement. Typically CAD durations in current publications focus on
148
times lower than five minutes, extremely quick CAD durations being only a few seconds (Louw, et al., 2015).
149
CAD duration seems to have a negative effect on take-over as glances to the driving scene shorten (Feldhütter,
150
et al., 2016) as well as leading to fatigue (Neubauer, et al., 2012). This effect is explained through the loss of
151
situational awareness, leading to higher effort and time necessary to perceive the environment. Additionally,
152
autonomous driving appears to have an effect on the quality of take-over after motoric, visual and cognitive
153
control has been obtained (Brandenburg, et al., 2014). Multiple studies also address the effect of different
154
NDRT as independent variables, as drivers tend to reallocate attention away from driving-related activities and
155
focus on a NDRT variant (Naujoks, et al., 2016). Typically, these studies require participants to perform
156
standardized abstract NDRT [i.e. n-back task, Surrogate Reference Task (Radlmayr, et al., 2014), Tracking Task
157
(Damböck, et al., 2012), Twenty-Question-Task (Petermann-Stock, et al., 2013)]; entertainment NDRT [i.e.
158
watching video, playing games, listening to music (Zeeb, et al., 2016)]; vehicle related NDRT,[i.e. interaction
159
with navigation system (Wulf, et al., 2013) and speaking on the phone (Naujoks, et al., 2017)]. The abstract
160
tasks of the abovementioned list of NDRT can be scaled and controlled in their complexity, e.g. n-back task
161
by increasing the ‘n’ (Kirchner, 1958). Findings display contradicting results when analyzing the effect of NDRT
162
on reaction times, sometimes displaying an effect of type of NDRT (Petermann-Stock, et al., 2013) and
163
sometimes not (Radlmayr, et al., 2014; Gold, et al., 2016). Comparison between the studies is difficult, as
164
other parameters are varied throughout the experimental setup between studies. It seems that the execution
165
of NDRT in general lead to longer take-over times as opposed to take-overs in which drivers continuously
166
monitor the driving task (Vogelpohl, et al., 2016). Zeeb et al. (2015) find no effect of visual distraction on
167
motor readiness of hands at the steering wheel. Nomadic handheld devices are not examined by Zeeb et al.
168
(2015), as neither this study will, due to the fact that these devices can reduce motor readiness if the hands
169
are not free and take-over warnings are difficult to synchronize on nomadic devices.
170
As environmental factors are the prime group of influencing factors being analyzed in current studies, this
171
generates a large network of complexity. For the truck context none of these factors have been investigated
172
and a comparison to the passenger car context should be formed. The present study will consider two different
173
NDRT which are expected to generate differences in take-over times as presented by Petermann-Stock et al.
174
(2013) (Hypothesis 2). Three different CAD durations are also examined while keeping traffic densities
175
constant. It is expected that different CAD durations have an effect on reaction times (Hypothesis 3). To
176
examine the possibility of different take-over situation types, multiple highly temporal situations will be
177
explored to exclude the possibility of learning reactions for one specific situation when repeated. A study from
178
Walch et al. (2015) presented results in which reaction times were investigated with regard to warning time
179
criticality as well as take-over situations, however, with only a significant effect between some warning types
180
and situations. Take-over situations in the present study are categorized in three groups, i.e. obstacle
181
avoidance, stabilization and braking & stabilization scenarios. Resulting from these three groups and the
182
variation in driver responses needed to perceive and plan the take-over reaction, significantly different take-
183
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 5
over times will be observed for different take-over situations (Hypothesis 4). A hypothesis with regard to which
184
type of situation generates the quickest take-over times is not postulated.
185
1.1.3. Vehicle Factors
186
The third factor class, vehicle factors, include technical ADAS versions, fallback systems, and visual field of
187
view from the cockpit within a vehicle. These factors have not been explicitly regarded as independent
188
variables in the abovementioned publications, but are inherently different between experiments. Therefore, a
189
comparison between the scientific discoveries needs to consider the difference in vehicle parameters. It is
190
unrealistic to compare all of the vehicles utilized in the listed publications regarding take-over behavior in Level
191
3, however, especially fallback systems possibly affect behavior considerably. To the best of the authors’
192
knowledge no current study compares different reaction types in different vehicles. This class of factors is
193
held constant in this experiment and no hypotheses are generated. By utilizing only one truck, i.e. vehicle
194
factors are held constant, the abovementioned environmental factors can be analyzed and a comparison can
195
be drawn between the group of all passenger car studies which also had constant vehicle factors
196
1.1.4. Human-Machine Interaction Factors
197
The fourth class, human-machine interaction factors, regards the warning modalities, interaction design and
198
NDRT layout within the vehicle (Naujoks, et al., 2017). This set of parameters is not investigated explicitly in
199
the present study and is held constant throughout experimentation. While iteration of these parameters can
200
affect take-over behavior (Naujoks, et al., 2014), vehicle interior design always needs to be taken into account
201
when arranging a HMI. As HMI concepts are different among different vehicles of one automobile manufacturer
202
and are changed with new models, influencing factors also vary. To reduce any effect of HMI factors, this study
203
will only regard one HMI setup and keep all interaction identical between participants.
204
1.2. Truck context
205
Predominantly driven by a trained group of professionals, as opposed to occasional drivers in the car context,
206
truck drivers are responsible for vehicles with a higher mass that potentially cause higher safety critical
207
situations if mismanaged. German legislation passed a law forcing new trucks with a mass greater than eight
208
tons to have an emergency brake assist as of 2015, reducing the current speed by 10 km/h (Deutscher
209
Bundestag, 2016), similar actions are discussed within the European Union. Automation in the truck context
210
thus far always incorporates a human in-the-loop, who is assisted in certain situations through specific ADAS,
211
e.g. lane departure warning systems or automatic transitions.
212
213
Figure 1: Influencing factors for the current experimental setup including measurement techniques.
214
When considering reactions in take-over behavior between truck and car contexts, behavior may vary between
215
drivers of the two vehicle types. Accordingly, a multitude of factors of truck driving may influence the take-
216
over behavior differently when compared with drivers of passenger cars. Firstly, driver factors, e.g. experience
217
or trust in vehicle, and intrinsic driving motivation are essentially different. In Germany and Britain an average
218
of roughly 39 minutes are spent in passenger cars (Streit, et al., 2014; Department for Transport, 2015), while
219
professional truck drivers are allowed to drive up to ten hours according to German law. This effect is also
220
recorded by the total distance passenger cars and trucks cover annually in Germany; cars: 13,341 km, trucks:
221
98,809 km (Kraftverkehrsstatistik, 2016). Similar statistical differences are documented in other countries
222
(U.S. Department of Transportation Federal Highway Administration, 2014; Royal Automobile Club Foundation
223
for Motoring, 2017). In addition, a larger portion of time is spent on highways by truck drivers, 50% of annual
224
mileage in Germany (Statistisches Bundesamt, 2013; Bueringer, 2007). This allows professional truck drivers
225
to practice and learn driving maneuvers over an extensive period of time with a higher reoccurring frequency.
226
Secondly, the environmental factors are largely similar for truck drivers compared to passenger car drivers. A
227
difference being, that the situations truck drivers experience are different as a majority of their travel is limited
228
to the right lane on highways. An additional hypothesis is therefore generated, truck drivers will take back
229
6 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
control quicker than passenger car drivers due to more experience and training (Hypothesis 5). Thirdly, when
230
regarding vehicle factors, depending on the truck model, drivers have a field of view approximately two meters
231
higher above tarmac than those of passenger cars. Vision is more confined for close ranges in trucks, due to
232
a large blind spot imminently in front of the bumper and to the rear of the cabin. Accelerations and velocities
233
are lower, however, sharp decelerations must be avoided to reduce the risk of damaged cargo or rear-ending
234
of following vehicles. Fourthly, different vehicles also present different HMI factors, which may influence
235
drivers differently in the two vehicle types. Without a controlled experimental comparison between different
236
setups and interaction modalities a comparison is difficult and has thus far not been conducted.
237
When taking all four factor classes into account, Vogelpohl et al. (2016) conclude that complexity of take-over
238
situations, warning modalities and the warning times before take-over affect take-over behavior
239
predominantly. All current published studies conducted on time critical take-over behavior of Level 3
240
automated driving, i.e. CAD, were performed in the passenger car context, investigating different
241
environmental and HMI factors as independent variables (Damböck, 2013; Feldhütter, et al., 2016; Louw, et
242
al., 2015; Radlmayr, et al., 2014; Vogelpohl, et al., 2016). Take-over behavior in the truck context has thus far
243
been investigated by Zhang et al. (2017) in platooning scenarios with no time criticality. Different reaction
244
times were measured by controlling for the interaction with the truck and allowing for an NDRT. Unfortunately
245
these findings are not comparable to our scope, as no time criticality was introduced. Similarly the workload
246
and trust in automation has been investigated in platooning scenarios (Hjälmdahl, et al., 2017). Interestingly,
247
Hjälmdahl et al. (2017) found that drivers tend to overestimate their situational awareness in platooning
248
scenarios. Further research focusing on the design of an HMI in the truck context has been conducted by
249
Richardson et al. (2018) and for military vehicles by Baltzer et al. (2016). Socio-economic factors and the
250
scope of NDRTs on drivers and hauliers was investigated by Janson et al. (2017). The lateral partial automation
251
of a truck was tested by Schermers et al. (2004), identifying that drivers assisted were less likely to change
252
lanes compared to manual drivers in critical take-over situations which required braking. While the truck
253
context has been regarded in several aspects of automated driving, research focusing on time critical take-
254
over scenarios in Level 3 with NDRT has not been addressed.
255
1.3. Hypotheses
256
The scope of this study is to record behavioral data of truck drivers currently operating a semi-truck with an
257
activated Level 3 automation function, SAE Level 3. This incorporates the data collection of reaction times and
258
eye-tracking data. The measurements of the body posture are not included in this publication but are
259
considered in Lotz et al. (2019). A moving-based simulator will be used to manipulate environmental factors,
260
while HMI and vehicle factors are kept constant. The variation of said factors will be measured through
261
different camera-based systems and referenced on reaction times. This type of experimental setup is thus far
262
novel within the truck context and will present data on take-over behavior for highly trained drivers that can
263
be compared to established Level 3 research. The comparison can thus far only be generated superficially to
264
other publications in the field. Two different NDRT are chosen which are hypothesized to distract drivers over
265
longer periods of time, making it easier to compare trials in the beginning and end of the experiment as
266
attention should be held almost constant. When presenting many take-overs, drivers will likely display attentive
267
behavior towards the driving task after multiple take-over repetitions and will try and uphold higher situation
268
awareness, with less attention being allocated towards NDRT. Typically, studies with Level 3 only present
269
between one to four take-overs which bears the danger of learning behavior influencing reaction times in early
270
trials. In order to observe learning behavior, short CAD durations were chosen for this study which is coherent
271
with typical CAD durations of less than five minutes (Nilsson, et al., 2017).
272
To exclude the possibility of participants recognizing identical take-over situations and learning situation
273
specific behavior, nine different take-over situation types were defined for each of the nine take-overs in the
274
experimental study.
275
Based on the description of previous research within the Level 3 domain, five primary hypotheses are
276
generated. The first four hypotheses are expected to have a direct effect on reaction times for time critical
277
take-over situations. If an influence is found, this could lead to the possibility of adaptive warning times.
278
1. Learning behavior will be observed due to first time use of the automation function. Reaction times
279
will decrease as interaction for first time users of the automation function will adapt behavior.
280
2. NDRT game will prolong take-over times compared to video, as definite visual attention is required to
281
successfully solve tasks. The video presents information in two modalities (auditory and visual), while
282
the game can only be solved if visual and motoric modalities is allocated. Both tasks require cognitive
283
resources additionally.
284
3. Longer CAD durations will prolong take-over times for TTEoR as well as TTHoS and TTFR, as argued by
285
Feldhütter et al. (2016).
286
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 7
4. Due to the complexity of the surrounding and the take-over situations different take-over times are
287
expected.
288
5. Average reaction times will be lower than those reported of passenger car drivers. This hypothesis
289
can only be compared quantitatively to published studies as no pure passenger car drivers were
290
examined. Additionally, the driver, environmental, HMI and vehicle factors of published studies may
291
differ to this study.
292
2 Materials & Methods
293
2.1. Reaction Time Definition
294
The following reaction times are defined for a comparison of independent variables (Damböck, 2013).
295
TTEoR: The Time To Eyes on Road is a measure for cognitive processing, as a visual and auditory request to
296
intervene (RtI) has to be processed by subjects and eyes have to be moved towards the road. Times can vary
297
significantly due to the visual path subjects choose to address. Theoretically, if information regarding take-
298
over is searched for in the instrument cluster before the environment is focused on, TTEoR rises drastically
299
due to the additional saccade and fixation towards the instrument cluster.
300
TTHoS: Time To Hands on Steering expresses the motoric reaction time necessary for subjects to regain
301
lateral control of the vehicle. It is not possible to distinguish whether subjects begin with motoric reaction
302
after eyes fixate the road or if motoric reaction is initialized simultaneously with eye movement.
303
TTFR: Time To First Reaction measures the time between RtI and a steering wheel input greater 1 Nm or 5%
304
accelerator/decelerator pedal position change. At RtI the trajectory and speed of the automation function was
305
kept constant until one of the abovementioned thresholds were activated. This caused the automation function
306
to deactivate. It could be possible, that the TTFR is lower than TTHoS, as reactions at the pedals are also an
307
input for TTFR.
308
309
Figure 2: Nine different take-over situations categorized into three basic maneuvers that were required to circumvent a crash. Times to
310
collision (TTC) are displayed above each situation.
311
2.2. Participants
312
The sample size for this experiment consisted of 95 participants, four female and 91 male. This gender
313
distribution is similar to the German truck driver population, with 1.7% professional female drivers employed
314
in Germany (Statista, 2017). Age distribution was controlled for in the sampling and reflects the demographic
315
distribution of truck drivers in Germany. A total of 88 participants completed the course and were considered
316
for statistical analysis, no one was involved in a crash. Exclusion of 7 participants occurred due to technical
317
synchronization failures of sensors. The primary vehicle driven in everyday use by 71 of the 88 participants
318
was a Mercedes truck. Mean age of the participants (n=88) was M = 42.6 years (SD = 9.61 years) with a range
319
from 23 to 62 years. All participants had an annual minimal mileage of 20,000 km/year and a mean of
320
M = 73,823.8 km/year (SD = 45,378.5 km). A total of 38 participants had corrected vision, wearing contact
321
lenses or glasses, participants with progressive lenses did not take part in the experiment. Overall, 55% of
322
participants were familiar and used speedometers daily (Adaptive Cruise Control = 34%). As personal
323
preferences of ADAS usage varies, even for professional drivers, this factor could not be considered during
324
sampling. To improve controllability for the Level 3 automation function, interaction was identical to that of
325
Mercedes-Benz speedometers. An influence of Mercedes familiarity was not found in the data, as the
326
automation function was completely new for all participants. Physical wellbeing was addressed and queried
327
prior to the start of the experiment and monitored during procedure.
328
8 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
2.3. Experimental Design
329
All participants were presented with nine different take-overs (trials). Each trial consisted of one of the nine
330
situation types, which required either an obstacle avoidance (i.e. Fog & Warning Triangle, Stranded Vehicle,
331
Lost Cargo, Lane Narrowing and Construction), stabilization of the vehicle (i.e. Loss of Lane) or braking (i.e.
332
Cut in Left, Cut in Right and Lead Braking), the situations types can be found in Figure 2. Ten different variants
333
of the sequence of situation types were designed, of which each participant drove one, see Figure 3. In each
334
variant the nine take-over situations were divided into three blocks. Within each block the trials had a CAD
335
duration of each 30, 120 or 240 seconds, levels were randomized within each block. Two different tasks, video
336
or game were presented during the experiment. During one of the three blocks (3 take-over situations) the
337
video was shown, during the other two blocks (6 take-over situations) a geography game was played, which is
338
described in section 2.4.3. Procedure requires all participants that are involved in a collision to terminate the
339
experiment. In order to reduce the amount of take-over collisions, we chose to keep the left lane immediately
340
before and during take-over idle to reduce crash risk at RtI.
341
This results in an experimental design with the unbalanced within-subject factors CAD
342
duration [30, 120, 240 sec] and NDRT [video, game] as well as the balance within-in subject factors trial [1 - 9]
343
and situation types [9 different situations]. The comparatively high amount of independent variables and
344
factors were chosen to address the study aims of identifying influencing factors on the take-over behavior.
345
This resulted in a non-orthogonal experimental design. A full iteration of all variants would have required
346
9! = 362880 variants and comparability is limited to this extent.
347
348
Figure 3: Experimental procedure of the experiment with three exemplary variants. Nine trials were driven by each participant that had
349
different factors of the independent variables.
350
Each of the nine trials started when the automation function of the vehicle was activated through the driver
351
and the Level 3 commenced. Due to individual temporal activation of the automation function, CAD duration
352
was not fixed, but was distributed around 30 seconds (M=37.0 sec; SD=7.0 sec), 120 seconds (M=121.4 sec;
353
SD=7.5 sec) and 240 seconds (M=243.6 sec; SD=8.4 sec). Simultaneously, the NDRT was activated on the
354
mounted tablet in the central console. Due to the occurrence of one of nine take-over situations, a take-over
355
signal was displayed for three seconds after which the tablet was blackened and locked. This take-over signal
356
or RtI consisted of visual and auditory warnings in the instrument cluster and tablet. The visual icon displayed
357
the outline of a set of hands holding a steering wheel colored red with the text “Hands On” written beneath.
358
Simultaneously, a sound was triggered at 1200 Hz and 200 bpm until a TTFR was measured. During manual
359
driving, after the RtI, the display was locked and darkened. The vehicle held the current steering angle and
360
speed at the RtI until driver took back control. After take-over a 50 second manual drive commenced until
361
drivers were asked to activate the automation function again, initiating the next trial. Within this manual drive,
362
after 20 seconds, a verbal question was presented to the participants through loudspeakers in the cabin with
363
regard to the perceived temporal criticality of the prior take-over. The experiment was designed to verbally
364
inquire criticality in temporal proximity to the RtI. In order to shape the query as non-invasive and simple as
365
possible, subjects were asked in regard to the temporal effect of the RtI. The additional aspect of non-fluent
366
native speakers was also taken into account and the question was phrased with regard to take-over time. The
367
question requested a categorization of temporal criticality: “Was the time for take-over sufficient, short or much
368
too short?” Graphical representation of the experimental design can be found in Figure 3.
369
Dependent variables consisted of take-over times on the basis of Damböck (2013), time to eyes on road
370
(TTEoR), time to hands on steering (TTHoS) and time to first reaction (TTFR). The TTFR is the main measure of
371
reaction as it represents the first oversteering of the automation function by the driver.
372
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 9
2.4. Apparatus
373
2.4.1. Driving Simulator and Track
374
Venue of this experiment was the Mercedes-Benz Moving-Base-Simulator, allowing 360° simulation of driving
375
related situations with realistic forces and accelerations of complete vehicles. A Mercedes Benz Actros truck
376
cabin was mounted within the simulator dome, which was connected to a hexapod on a carriage enabling
377
translatory motion as well as limited pitch, yaw and roll. All human-machine-interaction and actuator inputs
378
from within the cabin are translated with a realistic engine to movements of the composite dome, carriage
379
and visual display of the 360° projection screens. The interior of the cabin was equipped with a capacitive
380
steering wheel designated to record the TTHoS.
381
The activation of the Level 3 system was managed through the SET-button on the right-hand side of the
382
steering wheel. Through activation with the SET-button on the steering wheel the automation function and
383
therefore Level 3 was initiated, this was displayed at all times in the instrument cluster with a blue icon. During
384
Level 3 the driver could overrule lateral or longitudinal control at any time without deactivation of the
385
automation function. As soon as overruling was completed and all controls were let go, automation reactivated
386
itself. An exception being if the brake pedal was pressed, in these instances the automation needed resetting.
387
2.4.2. Eye-Tracker
388
A Smart Eye Pro eye-tracker was used as a means to record gaze behavior and was mounted on the dashboard
389
of the truck cabin. Four lenses were dispersed within the cockpit along the top of the dashboard to allow a
390
large field of view in which the eye-tracker is capable to record data. The eye-tracking cameras were calibrated
391
in an automated procedure for each participant prior to the drive. Data obtained was utilized for the calculation
392
of TTEoR and for the analysis of visual attention during and immediately after take-over.
393
2.4.3. Non-driving related tasks (NDRT)
394
Two different NDRT were analyzed, however, experimental design dictated which NDRT was available. The
395
NDRT was presented in blocks of three trials with each trial within these blocks consisting of a different CAD
396
durations. Rather than choosing NDRT that are abstract and could lead to a loss of attention over time, it was
397
decided to present tasks that were common for everyday use, at best for longer periods of time and would
398
allow comparison between early and late trials. Data analysis showed a constant visual attention over all trials
399
towards the NDRT and confirms the hypothesis, that engagement through realistic tasks was constant.
400
The experimental block in which the video was shown allows comparability to published studies with this
401
NDRT, e.g. (Carsten, et al., 2012). Theme of our video was a short documentary about truck driving. This NDRT
402
presented auditory and visual information to participants.
403
The second NDRT was available during two of the three experimental blocks. This task was an interactive
404
geography quiz also presented on the fixed tablet in the central console. Participants could see a contour map
405
of Germany on the tablet and were asked to locate certain cities. By touch the participants could set a point
406
on the map and received feedback as to the actual location of the city. Feedback consisted of a distance
407
measure between the participants guess and the actual position. An analysis of the game was not conducted.
408
The only measures drawn were the number of taps on the tablet and their coordinates. The task was chosen
409
due to the hypothesis that professional truck drivers have obtained a high geographical knowledge and are
410
motivated to perform well in such a quiz. Notably, if no NDRT is available or the task is mundane,
411
disengagement can lead to a state of passive fatigue (Marberger, et al., 2017). Demonstrating typical tasks
412
that could be allowed by legislation and offer comparability to other studies in which video NDRT was examined
413
(Naujoks, et al., 2017). Visual attention and motoric manipulation was necessary to successfully execute this
414
task, as no acoustic interaction was required. All text was written in German.
415
2.4.4. Take-over situations
416
Nine independent non-reoccurring take-over situations in the environment caused the event of nine RtI. Prior
417
to RtI the automation function followed a leading convoy of five trucks with a distance of 55 m to the last truck
418
in the convoy. Participants were instructed to set the speed of the automation function to 80 km/h during
419
Level 3. Due to the requirement of manual driving after RtI and natural speed variation through drivers, the
420
convoy could drive away, however, speed of the convoy decreased in these instances to assure that
421
participants would catch up again.
422
Time to collision (TTC) represents the time drivers are given between the RtI and a collision with the object
423
causing take-over. Take-over situations were presented within a near proximity of the ego-vehicle to guarantee
424
quick take-over behavior. If situations were to be presented with TTC of larger magnitude, participants
425
reactions would not need to be as quick. Full observation of the environment and updating of situation
426
awareness could occur before an input at the steering wheel or accelerator/brake pedals would be necessary.
427
This is also a prime aspect that makes comparability of publications with different TTC difficult. The take-over
428
situations all required a rapid reaction from the driver in order to overt an accident by either holding the lane,
429
10 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
changing the lane or holding the lane and decelerating, see Figure 2 and Appendix B. The TTC varied due to
430
the fluctuation of surrounding traffic needed to initiate take-over, see Figure 2, especially in take-over
431
situations in which moving objects caused take-over (i.e. Cut in Right, Cut in Left). For take-over situation Fog
432
& Warning Triangle TTC was less than four seconds, however, the warning triangle was not situated in the path
433
of the truck’s trajectory. Furthermore, for take-over Lane Narrowing TTC is calculated to the beginning of the
434
cones, actual lane narrowing commenced after an additional two seconds. Further conditions in Appendix B.
435
2.5. Procedure
436
Participants were welcomed by an experimenter and were asked to fill out consent forms, bank details to
437
receive their financial compensation and a demographic questionnaire. A written description of the experiment
438
was presented in which the automation function was explained. The occurrence and explicit investigation of
439
take-over behavior was not mentioned, however, take-over functionality for RtI was explained during the
440
detailed explanation of the automation function. The experimenter additionally explained the activation and
441
display of the Level 3 ADAS after participants had read instructions. This was to ensure that all participants
442
had understood basic functionality, as some participants were not fluent in German. It was never mentioned
443
to the participants, that take-overs were the focus of the study, rather user feedback on the new ADAS
444
automation function was being evaluated.
445
Following the introduction, participants were led to the Actros cabin through a gangway without visually seeing
446
the dome. It was explained, that functionality of the truck was identical to any truck on the road and that all
447
German road traffic rules apply. The automation buttons on the steering wheel were revised and the calibration
448
of the eye-tracker was conducted. An introductory drive including a verbal tutorial of the automation function
449
(activation, overriding of functions, take-over and deactivation) was driven for 10 minutes after which the
450
experimental drive started by activation of the tablet. The acclimatization phase was conducted to ensure,
451
that participants could control the Level 3 ADAS and were not instructed to react in any way. Participants
452
were asked to follow a leading convoy (five trucks) at 80 km/h during manual and automated drive without
453
overtaking. A total of 38 km were driven within approximately 28.5 minutes after the acclimatization phase.
454
Additionally, the tutorial was conducted within the first section of highway of approximately 12 km length.
455
Each trial consisted of a phase in which the Level 3 automation function was activated (CAD duration 30, 120
456
or 240 sec) with availability of an NDRT. Following RtI a 50 sec long manual drive phased was performed
457
during which a verbal question was presented to participants asking for a subjective rating of the amount of
458
time given for take-over.
459
3 Results
460
3.1. Reaction times - Hypotheses 1, 2, 3, 4
461
A total of 768 take-overs were successfully conducted by the 88 participants. Due to early identification of
462
take-over situations by the participants, 24 take-overs were not regarded. TTFR displayed a positively skewed
463
distribution with a mean of 1.35 seconds (SD=0.49 seconds). Visual representation of mean TTFR for the
464
independent variables CAD duration and NDRT are displayed in Figure 4, with the slowest reaction of TTHoS
465
and TTFR at 6.3 seconds. On average a take-over was conducted sequentially by focusing the eyes on the road
466
followed by taking control of the vehicle and finally showing a first reaction. Due to movement of the driver
467
and possible loss of the cornea, quality of the gaze direction data is not constant. To classify eye-tracking as
468
conclusive a qualitative comparison of TTEoR, TTHoS and TTFR was conducted. If the sequence of reaction
469
times did not match a video of the participants during take-over, eye-tracking was ruled inconclusive for the
470
period of the take-over. Reaction times for all three dependent measures were: TTEoR=0.34 seconds
471
(SD=0.43) (291 trials of TTEoR were disregarded due to inconclusive eye-tracking data during take-over,
472
leading to 477 trials for TTEoR); TTHoS=1.20 seconds (SD=0.48); TTFR=1.35 seconds (SD=0.49). Due to
473
frequent control glances at the road, in 217 of 477 regarded TTEoR trials, drivers were in the midst of a control
474
glance leading to short TTEoR. This aspect will be discussed in more detail in section 3.2. The formulated
475
hypotheses are investigated through statistical procedure leading to an adjusted alpha-level of aadj = 0.002,
476
see Table 2. Three full-factorial within-subject ANOVA were calculated with the four abovementioned
477
independent variables, i.e. CAD duration, NDRT, trial and situation type on TTFR
478
(3 [30, 120, 240 sec] x 2 [video, game] x 9 [1-9] x 9 [9 different situations, see Figure 2]). Due to the
479
experimental design, some combinations are non-existent and factor-levels are not considered. A full-factorial
480
ANOVA was chosen for statistical analysis, as the calculation is mathematically identical to multiple one-way
481
ANOVA and additionally offers the possibility of analyzing interactions.
482
Two main effects of trial F(8,710) = 22.381, p<<0.001, see Figure 5, and of situation type F(8,710) = 11.91,
483
p<<0.001, see Figure 6, were identified, while no interaction was determined for TTFR. Therefore, our results
484
are in accordance with Hypothesis 1 and Hypothesis 4. The classification of take-over situations according to
485
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 11
Figure 2 was not aligned with results of TTFR. All take-over situations exhibit learning behavior, even if different
486
average reaction times were observed as seen in Figure 5. The results of the ANOVA also leads to a rejection
487
of Hypothesis 2, F(1,710) = 0.520, p=0.471 and a rejection of Hypothesis 3, F(2,710) = 2.103, p=0.123.
488
The second full-factorial within-subject ANOVA is calculated on TTHoS. Again two main effects are identified
489
for trial F(8,710) = 15.67, p<<0.001 and of situation type F(8,710) = 5.60, p<<0.001. The results also present
490
a high correlation between the two dependent measures of TTFR and TTHoS (r=0.817). While statistical
491
significant differences were observed for the independent variable situation types which shows accordance of
492
the results with Hypothesis 4, section 3.2. will display reasoning why this conclusion is false.
493
A full-factorial within-subject ANOVA on TTEoR, see Table 2, for the reduced dataset was calculated and two
494
main effects for trial F(8,419) = 5.41, p<<0.001 and of situation type F(8,419) = 3.841, p=0.0002 were
495
identified. In combination with the results of the full-factorial ANOVAs for TTHoS and TTFR, this leads to a
496
rejection of Hypothesis 3.
497
3.2. Take-over situation comparison Hypothesis 4
498
To have a more accurate look at different behavioral reaction times, the dataset of 477 trials with conclusive
499
eye-tracking was divided into two parts, see Table 3. The first subset in Table 3 contains only those 217 trials
500
in which the eyes were fixated at the road at RtI. The primary cause for these fixations were control glances.
501
Naturally, the TTEoR equaled 0 seconds while TTHoS=0.94 seconds (SD=0.51) and TTFR=1.10 seconds
502
(SD=0.52). The second subset contains all other trials in which the eyes were not fixated on the road at RtI.
503
The values of Subset 2 were calculated as TTEoR=0.62 seconds (SD=0.40), TTHoS=1.29 seconds (SD=0.34)
504
and TTFR=1.44 seconds (SD=0.37). If the temporal difference between TTFR and TTHoS are compared within
505
the two subsets, both values equal 160ms and 150ms. Therefore, almost no difference is made between the
506
motoric response once hands are place on the steering wheel in dependence to visual information at RtI.
507
Similarly, when comparing TTHoS and TTFR between the subsets, both values are almost identical with 350ms
508
and 340ms, confirming the findings of Zeeb, et al., (2015) that motoric readiness is isolated from visual
509
information. This means, that visual information of the take-over cause at RtI generates a benefit in motoric
510
response prior to hands touching the steering wheel, without altering the temporal action sequence once
511
hands or feet are placed on controls for the first reaction.
512
513
Figure 4: Reaction times Time To Eyes on Road (TTEoR), Time To Hands on Steering (TTHoS) and Time To First Reaction (TTFR) for
514
independent variables CAD duration (left) and secondary task (right).
515
Minimal reaction times (lower whiskers) are an indication for premature visibility of take-over situations, if
516
participants chose to observe the driving situation in these instances through control glances. As seen in
517
Figure 6 three take-over situations (Fog & Warning Triangle, Lane Narrowing and Cut in Right) are mainly
518
responsible for TTEoR=0 seconds. Subset 1 and 2 are therefore subdivided further to differentiate between
519
situation types that display the possibility of premature visibility. Premature visibility is defined in this context
520
as the possibility of observing objects in the environment that can be identified as causing a take-over prior
521
to any warning. When comparing the second subset (all trials where no eyes were on the road at RtI) with the
522
fourth and sixth subset (partition of Subset 2 with regard to different situation types), similarity of the mean
523
times of the data is apparent. As the eyes were not on the road at RtI for these three subsets, behavioral
524
procedure at take-overs seems to be identical. However, the third dataset representing take-over situations
525
with the three distinct situations (Fog & Warning Triangle, Lane Narrowing and Cut in Right) and
526
TTEoR=0 seconds, displays much quicker TTHoS and TTFR. This benefit can be attributed to the timely
527
perception of the take-over situation. Trials of Subset 3 made it possible for the driver to perceive and plan a
528
reaction prior to RtI, generating the identical benefit as long TTC would, with the difference that immediate
529
action was required in the situations presented in this study. The fifth subset entails the take-over situations
530
12 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
(Lost Cargo, Construction, Loss of Lane, Cut in Left, Lead Braking and Stranded Vehicle) at which
531
TTEoR=0 seconds, TTHoS=1.19 seconds (SD=0.50) and TTFR=1.38 seconds (SD=0.50) were calculated. A
532
comparison of Subset 3 and Subset 5, under the assumption that situations were perceived equally beginning
533
at TTEoR=0 seconds, displays the benefit premature perception generates. The benefit of 530ms for TTFR was
534
observed in the data. The results of the ANOVAs calculated in section 3.1. regarding the significant results for
535
situation type are not confirmed by the results in Table 3. As the comparison by means of a one-way ANOVA
536
of Subset 4 with Subset 5 and 6 displays, the different situations do not cause differences in TTHoS
537
F(1,361) = 0.347, p=0.556, partial ² = 0.00006 or TTFR F(1,361) = 0.021, p=0.885, partial ² = 0.001 as
538
found by the ANOVAs in section 3.1. Significant results obtained in section 3.1. are due to the fact that
539
Subset 3 is included in the data. The conclusions drawn with regard to Hypothesis 4 are contradictory. Taking
540
all aspects into account, Hypothesis 4 is rejected.
541
542
Figure 5: Chronological visualization of reaction times Time To Eyes on Road (TTEoR), Time To Hands on Steering (TTHoS) and Time To
543
First Reaction (TTFR). For each of the abovementioned reaction times all trials are displayed from left to right (first to last).
544
545
Figure 6: Reaction times Time To Eyes on Road (TTEoR), Time To Hands on Steering (TTHoS) and Time To First Reaction (TTFR) for all take-
546
over situations as displayed in Figure 2.
547
3.3. Manipulation check
548
A manipulation check was not conducted during the experimental procedure to validate engagement in the
549
NDRT. However, for the game the amount of tablet taps were monitored during the experiment. Of the 517
550
trials in which the game was presented, 459 participants had touched the tablet at least once within
551
10 seconds prior to RtI. On average participants tapped the tablet M=2.85 times within 10 seconds, M=1.40
552
times within 5 seconds and M=0.54 times prior to RtI. A one-way ANOVA was conducted on all trials in which
553
the game was presented. The boolean independent variable whether a tap occurred within 10 seconds prior
554
to RtI was investigated for the dependent variable TTFR, no significant effect was found F(1,515) = 0.29,
555
p=0.59, partial ² = 0.00056. Investigation of engagement with the video NDRT is more difficult as no motoric
556
response was necessary. Control glances directed at the driving task were measured during the complete
557
experiment with respect to the frequency and duration of gazes within the windscreen. The average frequency
558
of control glances occurred every 2.46 sec (SD = 3.44), having a duration of 0.43 sec (SD = 0.20) and last
559
occurrence prior RtI being 4.63 sec (SD=11.99). Control glances at the road can be compared between the
560
two NDRT with three one-way ANOVAs. A total of 23 observations were disregarded due to no control glances
561
being performed and set to the CAD duration in these instances. No significant differences were found for the
562
time since the occurrence of the last control glance F(2,451) = 1.83, p=0.161, partial ² = 0.008 average
563
control glance duration F(2,451) = 0.252, p=0.778, partial ² = 0.0011 or average control glance frequency
564
F(2,451)= 3.886, p=0.021, partial ² = 0.0169.
565
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 13
Table 2: ANOVA of dependent variable TTFR and TTHoS
566
Dependent Variable
Measure
Sum of Squares
Mean Square
F
Pr (>F)
Partial 2
Time To First
Reaction
NDRT
0.09
0.091
0.52
0.471
0.000732
Trial
31.23
3.904
22.381
< 2e-16
0.201000
Situation Type
16.62
2.077
11.910
4.44e-16
0.118000
CAD Duration
0.73
0.367
2.103
0.1229
0.005890
NDRT * Trial
1.57
0.196
1.124
0.3449
0.012500
NDRT * Situation Type
2.18
0.273
1.562
0.1325
0.017300
Trial * Situation Type
3.41
0.310
1.778
0.0540
0.026800
NDRT * CAD Duration
0.84
0.418
2.396
0.0918
0.006700
Trial * CAD Duration
0.96
0.161
0.921
0.4788
0.007730
Situation Type * CAD Duration
0.75
0.752
4.313
0.0382
0.006040
NDRT * Trial * Situation Type
0.25
0.126
0.723
0.4859
0.002030
Time To Hands on
Steering
NDRT
0.07
0.0678
0.368
0.5444
0.000518
Trial
23.12
2.8894
15.67
< 2e-16
0.150000
Situation Type
8.26
1.0325
5.600
6.7e-07
0.059400
CAD Duration
0.76
0.3794
2.058
0.1285
0.005760
NDRT * Trial
3.03
0.3788
2.054
0.0381
0.022600
NDRT * Situation Type
4.12
0.5149
2.793
0.0048
0.030500
Trial * Situation Type
2.93
0.2664
1.445
0.1481
0.021900
NDRT * CAD Duration
0.77
0.3871
2.099
0.1233
0.005880
Trial * CAD Duration
1.67
0.2784
1.510
0.1721
0.012600
Situation Type * CAD Duration
1.47
1.4745
7.997
0.0048
0.011100
NDRT * Trial * Situation Type
0.30
0.1486
0.806
0.4472
0.00227
Time To Eyes on Road
NDRT
1.06
1.0591
6.611
0.0105
0.015500
Trial
6.93
0.8668
5.411
1.69e-06
0.093600
Situation Type
4.92
0.6153
3.841
0.0002
0.068300
CAD Duration
0.22
0.1120
0.699
0.4977
0.003320
NDRT * Trial
1.48
0.1851
1.155
0.3251
0.021600
NDRT * Situation Type
2.29
0.2858
1.784
0.0783
0.033000
Trial * Situation Type
2.38
0.2165
1.351
0.1937
0.034300
NDRT * CAD Duration
0.09
0.0427
0.267
0.7660
0.001270
Trial * CAD Duration
0.77
0.1280
0.799
0.5713
0.011300
Situation Type * CAD Duration
0.10
0.0992
0.619
0.4318
0.001480
NDRT * Trial * Situation Type
0.06
0.0311
0.194
0.8237
0.000926
14 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
Table 3: Comparison of reaction times for all situations.
567
Subset 1
Subset 2
Subset 3
Subset 4
Subset 5
Subset 6
Situation types
All
All
Fog & Warning
Triangle
Lane Narrowing
Cut in Right
Fog & Warning
Triangle
Lane Narrowing
Cut in Right
Lost Cargo
Construction
Loss of Lane
Cut in Left
Lead Braking
Stranded Vehicle
Lost Cargo
Construction
Loss of Lane
Cut in Left
Lead Braking
Stranded Vehicle
Number of
take-overs
217
260
114
65
103
195
Sum take-overs
∑477
∑477
Eyes on Road at
RtI?
Yes
No
Yes
No
Yes
No
Time To Eyes on
Road [sec]
0
0.62
(SD=0.40)
0
0.59
(SD=0.36)
0
0.63
(SD=0.42)
Time To Hands
on Steering
[sec]
0.94
(SD=0.51)
1.29
(SD=0.34)
0.71
(SD=0.39)
1.29
(SD=0.36)
1.19
(SD=0.50)
1.30
(SD=0.33)
Time To First
Reaction [sec]
1.10
(SD=0.52)
1.44
(SD=0.37)
0.85
(SD=0.40)
1.43
(SD=0.32)
1.38
(SD=0.50)
1.45
(SD=0.39)
568
3.4. Quality of take-over
569
A central question regarding take-over from a highly automated vehicle is whether drivers had enough time to
570
conduct the take-over safely. This is based on the introduced discussion how warning times have to be
571
designed, fixed or variable, to allow for adequate take-over times see section 1. Taking back control from an
572
automated vehicle always brings with it an aspect of safety criticality, due to the temporal facet of the vehicle
573
being in motion. As warning times during Level 3 should never provoke drivers to safety critical behavior,
574
different measures can be consulted to investigate the quality of take-over, such as accelerations (Petermann-
575
Stock, et al., 2013). Therefore, a full-factorial ANOVA was conducted for the maximum acceleration in
576
longitudinal and lateral direction in the 10 seconds after RtI, to identify if safety critical behavior was promoted
577
through certain factors of the independent variables. No significant main or interaction effects were found,
578
see Appendix C. With regard to Hypothesis 4 no significant effects of lateral and longitudinal accelerations
579
were found for the independent variable situation type. Therefore, even if different trajectories were required
580
to circumvent obstacles, the situation itself didn’t cause any difference in swerving as neither did any other
581
independent variable.
582
As a further measure of take-over quality the subjective temporal criticality for each take-over situation was
583
also recorded by presenting a verbal question to the participants. Subjective temporal criticality decreased
584
over the course of the experiment (from first to ninth trial). Analogously to the reaction times, subjective
585
temporal criticality was calculated with a full factorial ANOVA displaying a main effect for trial F(8,710)=7.48,
586
p<<0.001, see Table 4. This shows that participants learned how to react in critical situations and possibly
587
became more familiar with the human-machine-interaction (HMI) over the course of the experiment.
588
As a final measure of take-over quality we analyzed the Percentage Gaze into the left rear-view mirror
589
immediately after RtI. De Winter, et al. (2014) argue, that “for the shorter take-over request time of 5 s, the
590
drivers were less likely to gaze into the mirrors and over the shoulders […]”. The environment behind the ego-
591
vehicle is generally addressed with less visual attention in take-over situations, as obstacles are typically
592
presented in front of participants. Figure 7 displays the gaze percentage towards the left mirror within a
593
timeframe of one, two and five seconds after RtI. While keeping in mind that average take-over times (TTFR)
594
was calculated as 1.35 seconds, we can see only a few instances in which gaze behavior towards the rear was
595
recorded in this timeframe. This behavior is presented even though a lane switch was required for certain
596
situations and participants reacted within 1.35 seconds on average, oncoming traffic on the left lane was only
597
considered between 2-5 seconds. Additionally, merely take-over situations requiring a lane change and
598
orientation towards the left lane, as seen in Figure 2, present elongated visual attention towards the rear within
599
5 seconds. The overall analysis of quality measures shows that the independent variables caused no significant
600
different steering or braking behavior. However, gaze allocation towards the rear was delayed when compared
601
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 15
to the reaction times. Drivers did not assign visual attention towards the rear view mirror on average, even if
602
a prompt analysis of the oncoming traffic was relevant for certain situations. A conclusion drawn from this
603
data could be, that due to temporal criticality, the drivers had not enough time to safely manage the situation
604
even if reaction times were very quick. Self-reported subjective temporal criticality decreased over the course
605
of the drive significantly, substantiating the results on reaction times of Hypothesis 1
606
607
Figure 7: Percentage of Gaze towards left mirror for each take-over situation and NDRT. (Top) Within a timeframe of 1 second. (Middle)
608
Within a timeframe of 2 seconds. (Bottom) Within a timeframe of 5 seconds.
609
Table 4: ANOVA of dependent variable Subjective Criticality
610
Dependent Variable
Measure
Sum of Squares
df
Mean Square
F
Pr (>F)
Partial 2
Subjective
Criticality
NDRT
0.30
1
0.304
0.696
0.405
0.00098
Trial
26.14
8
3.268
7.486
1.29e-09
0.07785
Situation Type
3.60
8
0.450
1.031
0.4107
0.0115
CAD Duration
1.32
2
0.661
1.514
0.2208
0.0043
NDRT * Trial
2.16
8
0.270
0.617
0.7637
0.0069
NDRT * Situation Type
8.03
8
1.004
2.300
0.0195
0.0252
Trial * Situation Type
7.39
11
0.672
1.538
0.1131
0.0233
NDRT * CAD Duration
0.08
2
0.039
0.090
0.9143
0.0002
Trial * CAD Duration
3.14
6
0.523
1.199
0.3047
0.0100
Situation Type * CAD Duration
0.01
1
0.006
0.013
0.9096
2e-05
NDRT * Trial * Situation Type
0.33
2
0.166
0.380
0.6839
0.0011
611
4 Discussion
612
Focus of this simulator study was to investigate the impact of NDRT, CAD duration, situation types and effect
613
of trials (multiple take-overs) on the take-over behavior of professional truck drivers. A generic automation
614
function was implemented in a commercially available semi-truck cabin, allowing participants to activate Level
615
3 under controlled conditions of a moving-base simulator. Insights were gained for our hypotheses:
616
1. Learning behavior will be observed due to first time use of the automation function.
617
o Our results are in accordance with this hypothesis, with learning behavior found for the first
618
three trials and a significant main effect. Reaction times converged after the third trial.
619
2. NDRT game will prolong take-over times compared to video.
620
o Our results did not support this hypothesis, as no significant differences in reaction times
621
were found.
622
16 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
3. Longer CAD durations will prolong take-over times.
623
o Our results did not support this hypothesis, as no significant differences in reaction times
624
were found.
625
4. Due to the complexity of the surrounding different take-over times are expected.
626
o Our results did not support this hypothesis, due to the results in section 3.2. for situation
627
type.
628
5. Average reaction times will be lower than those reported of passenger car drivers.
629
o Without the respect of independent variables, the reaction times of this study are lower than
630
those of reported passenger car publications.
631
4.1. Learning Hypothesis 1
632
Learning behavior is observed in our data with reaction times approximately 0.5 seconds slower in the first
633
trial compared to the quickest trial (eight), see Figure 5. Petermann-Stock et al. (2013) also find that reaction
634
time reduces between the first and second trial in their results. Due to the fact that nine take-overs took place
635
in the present study, convergence is detected. It is likely, that this time reduction is caused by the effect of
636
practice, leading to the proceduralization of knowledge (Anderson, 1982). Which reaction is desired when
637
take-over signals are presented and the reduction of scan-paths, can lead to significant time savings that
638
account for the experienced reduction of reaction times. Due to the large number of take-over situations a
639
complete balanced iteration at each chronological trial was not possible. Each take-over situation was,
640
however, presented at least at two different positions in the trial sequence. General learning behavior is
641
observed for each take-over situation type within the first three trials. Reaction times seemingly increase for
642
certain trials, specifically the ninth trial, this is however a misconception as the take-over situation ‘Loss of
643
Lane’ was presented in half of all instances in the ninth trial, resulting in longer take-over times.
644
4.2. NDRT Hypothesis 2
645
The null hypothesis of Hypothesis 2 is rejected. This finding of no NDRT effect on TTFR correlates with those
646
of Radlmayr, et al., (2014) and Gold, et al., (2016) raising the question if this is a universal finding for the car
647
and truck context or whether certain NDRT on non-nomadic devices show reactional differences as reported
648
by Petermann-Stock et al. (2013). A possibility for the different findings is that the take-over scenario was
649
implemented during a traffic congestion at 35 km/h. A plethora of NDRT could have been investigated in the
650
present study. While telephoning, social media interaction and similar NDRT might provide a realistic test case
651
for professional drivers, we considered these NDRT not engaging enough over multiple trials or assumed
652
difficulties in controllability (social media). Additionally, visual tasks were required to initiate visual inattention
653
towards the driving task.
654
The visual attention towards the NDRT as well as the frequency, last occurrence and duration of control
655
glances remained constant throughout the experiment, showing that visual behavior was not adapted and was
656
not influenced by the learning effect. Furthermore, the duration of inattention towards the road did not affect
657
the reaction time (TTFR).
658
4.3. CAD Duration Hypothesis 3
659
The data displayed in section 3 Results does not confirm a correlation between TTFR and CAD duration. The
660
null hypothesis is rejected. The results in Figure 4 (left) exhibit that duration effects of automation do not
661
affect our results in this specific experimental setting, this finding confirms the finding of Feldhütter et al.
662
(2016) in the truck context. A possible reason that no effect was found can relate to the fact that CAD duration
663
did not exceed four minutes. While Feldhütter et al. (2016) did not find a significant effect for reaction times
664
of short and longer drives (20 min), other research suggests that fatigue influences take-over significantly.
665
Vogelpohl et al. (2017) find that Level 3 should not exceed 15 minutes without NDRT, as fatigue influences
666
drivers’ take-over as if drivers were heavily distracted. The excitement of driving in a moving-based simulator
667
and being monitored during driving may have influenced our participant sample with regard to concentration
668
and involvement. Again, internal driver factors as discussed in the introduction, cannot be monitored with
669
sufficient meaningfulness and are a prerequisite for many drowsiness experiments and herein a limitation (c.f.
670
Goncalves, et al., 2016). When comparing the participant samples on experience, truck drivers may have a
671
higher tolerance for task induced drowsiness compared to the non-professional drivers. Investigating short
672
timeframes is by far not trivial, as it is not possible to guarantee that a Level 3 system will run for multiple
673
minutes before generating a RtI. Unexpected environmental situations, which the technical automation
674
function is not design for can arise at any instance. Another possible reason for missing variation in reaction
675
times due to CAD duration could be that the decay of situation awareness had no effect on take-over, as
676
situations were sudden and surrounding vehicles were not in direct periphery. Gold et al. (2013) also draw this
677
conclusion stating, “[w]ith shorter TOR-time, the subjects come to a decision more quickly, reacting faster,
678
but the quality is generally worse”. Under this assumption, the influence of reduced situation awareness could
679
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 17
be investigated if correct reactions of participants would be more confined and the possibility of crashing were
680
higher. Within this proposed approach drivers with a higher situation awareness would not be prone to crashing
681
as much as those whose situation awareness has decayed more rapidly.
682
4.4. Situation Types Hypothesis 4
683
The null hypothesis of Hypothesis 4 is rejected, see section 3.2. Different situation types generated
684
significantly different TTFR although traffic density was held constant. We have concluded that the
685
combination of situational complexity and premature visibility during control glances seems to affect take-
686
over times massively. As the analysis in section 3.2. shows, the significant effects on TTFR and TTHoS are due
687
to situations in which drivers observed the environment during RtI leading to quicker take-overs. A quicker
688
reaction time was only achieved for three situation types, while no effect was established for the other six
689
situations. The influence of these three situations generated significant results of ANOVAs in section 3.1. This
690
finding is beneficial, as some environments with complexities similar to those investigated here might allow
691
us to disregard factors such as CAD duration and NDRT to some extent. Validity of this conclusion needs
692
investigation in future work. An environmental model of the truck driver’s surrounding would potentially
693
generate added value to the problem, due to the possibility of being able to model visual attention to relevant
694
objects. Such a model expressing the continuous complexity of a situation could allow drivers to engage in
695
more stimulating tasks if low environmental complexities would preside, without the need of systems to
696
monitor cognitive states. This would simplify the problem of guaranteeing adequate take-over times. Schneider
697
(2009) addresses the question that no clear definition of traffic situations exists. First attempts in this regard
698
have been initiated, however, relying on subjective criteria for the description of situations and generic object
699
frequencies (von Benda, et al., 1983; Ohn-Bar, et al., 2016; Gold, et al., 2017).
700
4.5. Comparing truck to passenger car drivers Hypothesis 5
701
A quantitative comparison to literature on reaction times in the passenger car context is conducted for
702
Hypothesis 5. On average the time between RtI and time until eyes fixated within the windscreen (TTEoR) was
703
0.34 seconds. This time was, as stated above, dependent on situations. When compared to similar published
704
experiments, the reaction times published by Damböck (2012), i.e. approx. 0.8-1.2 seconds, are far slower
705
than our TTEoR. If only those situations are taken into account, in which participants were not in the midst of
706
a control glance, our TTEoR increases to approximately 0.6 seconds, see Table 3. This value is far closer to
707
the reported times in the literature. Quickness in response could have been influenced by professional
708
experience of our participants, when comparing values to the previously reported 0.8 seconds (Damböck,
709
2013). Our reaction times of TTHoS and TTFR were also at the lower end of the scale when compared to
710
literature values analyzed (Eriksson, et al., 2017). The quickest reported TTFR by Zeeb et al. (2015), Radlmayer
711
et al. (2014) and Gold et al. (2014) range from 1.14 sec to 1.67 sec and are comparable to our findings.
712
Parallel to other studies, either the experimental setup or participant sample seems to have an influence on
713
reaction times, as the average reaction time in our study undercut the upper scale of reported reaction times
714
of Level 3 scenarios of 10 sec (Merat, et al., 2014). Gold et al. (2017) state, “when interested in the maximum
715
driver performance within take-over situations, demanding scenarios are needed”. Hence, some studies, e.g.
716
Merat et al. (2014), in which uncritical take-over situations are not comparable to our data. Our findings also
717
show, that the visual attention towards the road at RtI allowed drivers to react (TTHoS and TTFR) circa
718
0.3 seconds quicker. The procedure of motoric readiness was not affected, confirming previous findings (Zeeb,
719
et al., 2015). Two factors limit the applicability of these findings. Firstly, simulator studies are limited in their
720
comparability to real world driving due to underestimation of speed in virtual settings (Bellem, et al., 2016).
721
Secondly, drivers knew that they were in a simulation setup and monitored, possibly intrinsically motivating
722
participants to peak performance. However, based on the limited comparison, Hypothesis 5 is failed to be
723
rejected, as truck drivers seem to present quicker reaction times compared to passenger car drivers. Future
724
research will focus on a study in which truck drivers and passenger car drivers are compared directly.
725
4.6. Generalization of reactions
726
A statistical evaluation of our reaction times showed that when calculating a -interval for the TTFR, which
727
depicts 99.7% of the data, we receive a reaction time interval of 0 seconds to 2.82 seconds. Reaction times
728
of 0 seconds (n=5) were achieved if participants directly engaged in take-over when RtI was presented. A total
729
of six take-overs are not included in the 3σ interval. Five of these six take-overs occurred within the first trial,
730
while one was the Loss of Lane situation, requiring no immediate reaction as no obstacle needs to be obverted.
731
It is likely that prolonged reaction times can be attributed to inexperience with the warning cascade and not
732
knowing how to react. The slowest reaction time out of all trials was 6.3 seconds, a first take-over. When
733
investigating TTFR for Level 3 on a general level, currently it is not possible to define a universal time reserve.
734
This study depicts a small subset of take-over scenarios for truck drivers on the multilane highway. While
735
higher automation functions are in discussion for urban driving areas, research on Level 3 is mainly limited to
736
18 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
highways. Without taking the human driver into account, modern radar systems allow for an environmental
737
perception of up to 250m on a straight road (Continental, 2017). When travelling at 80km/h this provides
738
information 11.25 seconds before a possible collision, without taking computation and sensor fusion times
739
into account. Therefore, from a technical standpoint the current maximal time available for a driver to take
740
back control of a vehicle is quite limited. Restricted to the take-over situations and NDRT on non-nomadic
741
devices in this study, this leaves enough time for drivers to regain control of the vehicle. However, inferring
742
the cognitive state of the drivers as well as stress was not investigated.
743
A quick take-over does not guarantee a safe take-over, as seen in Figure 7. No effects of any independent
744
variable on lateral or longitudinal accelerations were found. We assume that the driving trajectory and evasion
745
of an obstacle is based on a large part on the strategy and experience of a driver. For future studies, a higher
746
number of surrounding vehicles should be considered around the ego-vehicle. The estimation of distances
747
with long vehicles is difficult through rearview mirrors, especially during take-over situations that require the
748
perception of dynamic objects (Nilsson, et al., 2017). Vehicles in direct periphery would limit the amount of
749
correct reactions, possibly leading to a higher collision rate, but also clarifying behavioral differences between
750
successful and unsuccessful take-overs. The data shows that within the first two seconds of take-over drivers
751
are not capable of reacting to the RtI, perceiving the forward environment as well as the rear environment,
752
even if this is required for a safe and adequate take-over, see Figure 7. Although reaction times suggest that
753
take-overs can be conducted quickly and with a high consistency under three seconds, sufficient visual
754
attention is not observed, leading us to conclude that take-over was not safely managed on average.
755
4.7. Quality/Validity of the experiment
756
All possible influencing factors cannot be mentioned or observed in such studies. It should be clear, however,
757
that our experiment presented highly critical take-over situations, TTC between 0.92 seconds and 5 seconds
758
(M=4.10 seconds), which left little time to avoid obstacles. The results presented in this paper describe
759
minimal reaction times to take-over situations for heavy duty trucks in combination with NDRT, similarly to the
760
declared objective of Damböck et al. (2012). Comparable published studies of Level 3 within the truck domain
761
do not exist to the best of our knowledge, leaving only room to speculate as to how these participants would
762
have reacted in a passenger car and if significant differences in reaction times would present themselves.
763
Lower speeds and accelerations, different fields of view and driver experiences are factors that can only be
764
considered if a comparison study were to be conducted. When extrapolating the findings to the real world in
765
which the professional driver is highly familiar with all vehicle factors, we hypothesize that similar TTFR would
766
occur. However, no empirical data is currently available of such critical take-overs in real world scenarios.
767
With regard to our study aim of identifying factors that enable a variable warning time for Level 3, the results
768
show high variance. Primary differences in TTFR occurred due to visibility of objects in the environment prior
769
to RtI. Based on the results collected the intra- and inter-individual differences in reaction times could not be
770
pinpointed to controlled independent variables. It is likely that the variance of reaction times would increase
771
outside of the controlled experimental settings, as drivers would gain more trust in the system and be
772
immersed within their known environment. However, based on the data collected, it is not possible to allow
773
for variable warning times.
774
5 Conclusions
775
A large empirical simulator study with professional truck drivers investigating Level 3 take-over behavior has
776
not been published thus far, as most studies focus on passenger car behavior or manual driving aspects of
777
truck driving.
778
Results obtained lead us to conclude that multiple findings of influencing environmental factors in the
779
passenger car context are transferable to the truck context. This includes that different NDRT and CAD
780
duration did not alter reaction times significantly. It could be possible that the examined CAD durations were
781
not long enough to induce significant differences in behavior and that long-term effects of driving related
782
inactivity are experienced after extensive periods of time, as suggested in some publications (Brandenburg, et
783
al., 2014). However, Level 3 systems in current development might require take-overs more often than not,
784
due to technical limitations, forcing drivers to regain control of the vehicle in short intervals. Additionally, we
785
observed clear learning behavior during our experiment leading to quicker reaction times. This is either due to
786
the fact that participants learned how to interact with the novel ADAS, or that they became more alert
787
throughout the drives. However, as the visual attention towards the NDRT did not reduce, it seems that the
788
first explanation is more likely. The in-depth analysis of the situation types presented that premature visibility
789
of take-over situations allows the drivers to react significantly quicker, see Table 3. An important factor that
790
could have caused a major influence on the results of this work is the effect of warning times, hence the TTC.
791
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 19
Walch, et al., (2015) did not find any effect in these times, however, this was not controlled for in this study
792
and could have taken an influence.
793
An important finding of the present study is that take-over quality can only be investigated if take-over
794
situations are more confining. Allowing drivers to respond with a variety of reactions with minimal risk of
795
crashing, as long as the obstacle causing take-over is obverted, does not produce enough precise information
796
on take-over quality. We found that although drivers obverted critical obstacles with low TTCs, the gaze
797
behavior analysis offered indications that the take-overs were not of sufficient quality. In the future it could be
798
possible to restrict the amount of correct responses to reduce the amount of crash avoidance. This possibility
799
would also allow for in depth analysis of gaze behavior and the reconstruction of situation awareness at take-
800
over.
801
Quick take-overs do not automatically guarantee a safe take-over, for which we found indicators such as
802
missing mirror gazes. This could cause severe accidents. It is possible that different strategies are in place if
803
TTC are low, either leading drivers to react quickly without respect for lane markings or safety, with higher
804
percentage of braking maneuvers. Distinguishing between strategies is difficult without subjective input from
805
participants and has to be considered in future studies. In general however, truck drivers seem to present the
806
capability of showing quicker reaction times when compared to published results on passenger car drivers.
807
Based on these findings, a time in which the driver can safely regain control of a vehicle could not be
808
determined reliably. This leads us to question whether Level 3 can or should be made available for drivers.
809
Daimler Trucks recently announced that the benefits of Level 3 do not outweigh the costs of such a system
810
and will skip development of this level (Daimler AG, 2019).
811
The investigation of influencing factors on take-over behavior has shown that especially the external
812
environment determines a large part on how quick a driver will react. As this environment can be monitored
813
with sensors, our underlying question regarding individual warning times based on influencing factors seems
814
to be possible to some extent. Further research will consider external traffic models and monitoring to
815
implement a first prototypic varying warning time.
816
817
Acknowledgements
818
We thank our colleagues for continuous technical support, for assistance in carrying out the empirical study
819
and the complete team of the Mercedes-Benz driving simulator.
820
20 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
Appendix A
821
Experimental Factor
Value
Number of lanes
2 Lanes
Minimal radius of curves
500m
Length of drive
Approx. 50km including tutorial
Width of lanes
3.5m
Color of lane markings
White
Take-over situation Lane narrowing - yellow on right boundary
Maximum speed of traffic
180km/h
Minimal speed of traffic
65km/h
Speed of ego-vehicle during Level 3
80km/h
Distance to lead vehicle
55m
Number of vehicles in environment
14 vehicles/km randomly dispersed and appearing 500m behind ego-vehicle
Time to Collision calculation
From take-over request to collision object (including acceleration/deceleration
of objects e.g. Cut in Left)
822
Appendix B
823
Take-over
Situation
Description
Fog & Warning
Triangle
Weather gradually worsened starting 1min before take-over until visibility was reduced to 60m at which
a warning triangle was situated at the side of the road within ego lane. The warning triangle was located
on the right lane marking.
Stranded Vehicle
A stranded vehicle situated half on ego lane and half on the hard shoulder which was circumnavigated
by the lead truck abruptly only then making it visible. The lead truck was traveling at 80 km/h and was
55 m ahead of the ego-vehicle.
Lost Cargo
Lost wooden box situated on the side of ego lane which was circumnavigated by lead truck abruptly only
then making it visible.
Lane Narrowing
Pylons and concrete barriers narrowing lane. Warning was display approx 1 sec before first pylon.
Construction
Lead truck switched lane abruptly to give way to a construction site. Ongoing traffic gave way to drivers
who needed to perform a lane change.
Stabilization
Washed out lane markings in a curve representing a system boundary. TTC was calculated as the time to
which the vehicle would hit crash barriers if trajectory were kept. Note, the hard shoulder was not
present in this condition to reduce TTC.
Cut in Left
A vehicle cutting in from the left and decelerating immediately in front of ego vehicle.
Cut in Right
A vehicle cutting accelerating from the hard shoulder immediately in front of ego vehicle. Distance to
lead truck was increased gradually before take-over to 85 m.
Lead Braking
Abrupt deceleration of lead truck from 80 km/h to 45 km/h within 2 sec.
824
825
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 21
Appendix C
826
827
Dependent Variable
Measure
Sum of Squares
df
Mean Square
F
Pr (>F)
Lateral Acceleration
NDRT
1.10
1
1.077
0.492
0.483
Trial
15.2
8
1.898
0.866
0.545
Situation Type
11.7
8
1.460
0.666
0.721
CAD Duration
1.90
2
0.967
0.442
0.643
NDRT * Trial
13.3
8
1.657
0.756
0.642
NDRT * Situation Type
19.5
8
2.440
1.114
0.351
Trial * Situation Type
22.8
11
2.075
0.947
0.494
NDRT * CAD Duration
9.20
2
4.586
2.093
0.124
Trial * CAD Duration
19.9
6
3.309
1.510
0.172
Situation Type * CAD Duration
0.00
1
0.025
0.011
0.915
NDRT * Trial * Situation Type
0.80
2
0.398
0.182
0.834
Longitudinal
Acceleration
NDRT
0.33
1
0.326
1.103
0.294
Trial
1.57
8
0.196
0.662
0.726
Situation Type
0.66
8
0.083
0.280
0.972
CAD Duration
0.62
2
0.309
1.045
0.352
NDRT * Trial
1.54
8
0.192
0.650
0.735
NDRT * Situation Type
2.95
8
0.369
1.248
0.268
Trial * Situation Type
2.36
11
0.214
0.724
0.716
NDRT * CAD Duration
0.20
2
0.102
0.346
0.708
Trial * CAD Duration
2.51
6
0.419
1.417
0.205
Situation Type * CAD Duration
0.12
1
0.118
0.399
0.528
NDRT * Trial * Situation Type
0.18
2
0.091
0.307
0.736
828
829
22 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
830
Bibliography
Allport, D. A., Styles, E. A. and Hsieh, S. 1994. Shifting Intentional Set: Exploring the Dynamic Control of
Tasks. Cambridge : MIT Press, 1994.
Anderson, J. R. 1982. Acquisition of cognitive skill. Psychological Review. 89, 1982, pp. 369-406.
Baltzer, Marcel, et al. 2016. Cooperative Guidance and Control in Highly Automated Convoys - StrAsRob.
[book auth.] Neville Stanton, et al. Advances in Human Aspects of Transportation. Florida, USA : Springer
Verlag, 2016.
Bellem, Hanna, et al. 2016. Can We Study Autonomous Driving Comfort in Moving-Base Driving
Simulators? A Validation Study. Human Factors. 2016.
Brandenburg, S. and Skottke, E. M. 2014. Switching from manual to automated driving and reverse: Are
drivers behaving more risky after highly automated driving? IEEE 17th International Conference on Intelligent
Transportation Systems (ITSC). 2014, pp. 2978-2983.
Bueringer, Helmut. 2007. Entwicklung des Straßenverkehrs in Baden-Wuerttemberg - Jahresfahrleisungen
mit Kraftfahrzeugen. Stuttgart : Statistisches Monatsheft Baden-Wuerttemberg, 2007. pp. 37-41.
Carsten, O, et al. 2012. Control task substitution in semiautomated driving: does it matter what aspects
are automated? Human Factors. 2012, Vol. 54, 5, pp. 747-761.
Continental. 2017. Continental Automotive. Advanced Driver Assistance Systems. [Online] 2017. [Cited: 08
01, 2018.] https://www.continental-automotive.com/en-gl/Passenger-Cars/Chassis-Safety/Advanced-
Driver-Assistance-Systems/Radars/Long-Range-Radar/Advanced-Radar-Sensor-ARS441.
Daimler AG. 2019. Daimler. Daimler Trucks invests half a billion Euros in highly automated trucks. [Online]
January 07, 2019. [Cited: April 15, 2019.]
https://media.daimler.com/marsMediaSite/en/instance/ko.xhtml?oid=42188247.
Damböck, D. 2013. Automationseffekte im Fahrzeug - von der Reaktion zur Übernahme. München :
Technische Universität München, Lehrstuhl für Ergonomie, 2013. Dissertation.
Damböck, D., et al. 2012. Übernahmezeiten beim hochautomatisierten Fahren. Tagung Fahrerassistenz. 15,
2012, 16.
de Winter, Joost C.F., et al. 2014. Effects of adaptive cruise control and highly automated driving on
workload and situation awareness: A review of the empirical evidence. Transportation Research Part F. 27,
2014, pp. 196-217.
Department for Transport. 2015. The pathway to driverless cars: summary report and action plan. 2015.
Deutscher Bundestag. 2016. Vermeidung von LKW-Auffahrunfällen auf Bundesautobahnen durch
Notbremssysteme. WD 5 - 3000 - 069/16, 2016.
Endsley, Mica R. 1995. Toward a Theory of Situation Awareness in Dynamic Systems. Human Factors
Journal 37(1). 1995, pp. 32-64.
Eriksson, Alexander and Stanton, Neville A. 2017. Take-over time in highly automated vehicles: non-
critical transitions to and from manual control. Human Factors. 2017.
Feldhütter, Anna, Gold, Christian and Bengler, Klaus. 2016. How the Duration of Automated Driving
Influences Take-Over Performance and Gaze Behavior. Kongress der Gesellschaft für Arbeitswissenschaft.
2016.
Gasser, T. M., et al. 2012. Rechtsfolgen zunehmender Fahrzeugautomatisierung. Berlin : Berichte der
Bundesanstalt für Straßenwesen. Unterreihe Fahrzeugtechnik (83), 2012.
Gold, C., et al. 2013. Take over! How long does it take to get the driver back into the loop? Proceedings of
the Human Factors and Ergonomics Society Annual Meeting. 2013, Vol. I, Vol. 57, pp. 1938-1942.
Gold, Christian, et al. 2016. Taking Over Control From Highly Automated Vehicles in Complex Traffic
Situations - The Role of Traffic Density. Human Factors. 2016, Vol. 58, 4, pp. 642-652.
Gold, Christian, et al. 2017. Testing Scenarios for Human Factors Research in Level 3 Automated Vehicles.
[book auth.] Neville A. Stanton. Advances in Human Aspects of Transportation: Proceedings of the AHFE 2017
International Conference on Human Factors in Transportation, July 17-21. The Westin Bonaventure Hotel, Los
Angeles, California, USA : Springer, 2017.
Goncalves, J., Happee, R. and Bengler, K. J. 2016. Drowsiness in conditional automation: Proneness,
diagnosis and driving performance effects. Proceedings of the IEEE 19th International Conference on
Intelligent Transportation Systems, ITSC 2016. 2016, 10.1109/ITSC.2016.7795658, pp. 873-878.
Happee, Riender, et al. 2017. Take-over performance in evasive manoeuvres. Accidnet Analysis and
Prevention. 106, 2017, pp. 211-222.
Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs 23
Hjälmdahl, Magnus, Krupenia, Stas and Thorslund, Birgitta. 2017. Driver behaviour and driver
experience of partial and fully automated truck platooning - a simulator study. European Transport Research
Review. 2017, Vol. 9, 1.
Janson, Amelie and Lindqvist, Caroline. 2017. Working in a Highly Automated Truck - Who, What and Why?
Stockholm : KTH Industrial Engineering and Management Machine Design, 2017. MMK 2017:112 MCE 361.
Jarosch, O., et al. 2017. It's Out of Our Hands Now! Effects of Non-Driving Related Tasks During Highly
Automated Driving on Drivers' Fatigue. Proceedings of the Ninth International Driving Symposium on Human
Factors in Driver Assessment, Training and Vehicle Design. 2017, pp. 319-325.
Kirchner, W. K. 1958. Age differences in short-term retention of rapidly changing information. Journal of
experimental psychology. 1958, Vol. 55, 4.
Kraftverkehrsstatistik. 2016. Kraftverkehr- Güterverkehr und Fahrleistung: Verkehr in Kilometer. Flensburg :
Kraftfahrt-Bundesamt, 2016.
Lotz, A. and Weissenberger, S. 2019. Predicting take-over times of truck drivers in conditional
autonomous driving. Advances in Intelligent Systems and Computing. Advances in Human Aspects of
Transportation, 2019, pp. 329-338.
Louw, T., Merat, N. and Jamson, H. 2015. Engaging with Automation: To be or not to be in the loop?
Proceedings of the 8th International Driving Symposium on Human Factors in Driver Assessment, Training and
Vehicle Design. 2015.
Marberger, Claus, et al. 2017. Understanding and Applying the Concepf of "Driver Availability" in
Automated Driving. [book auth.] Neville A. Stantion. Advances in Human Aspects of Transportation:
Proceedings of the AHFE 2018 International Conference on Human Factors in Transportation July 17-21. The
Westin Bonaventure Hotel, Los Angeles, California, USA : Springer, 2017.
Merat, N., et al. 2014. Transition to manual: Driver behaviour when resuming control from a highly
automated vehicle. Transportation reasearch part F: traffic psychology and behaviour, 27. 2014, pp. 274-282.
Naujoks, F., Mai, C. and Neukum, A. 2014. The Effect of Urgency of Take-Over Requests During Highly
Automated Driving Under Distraction Conditions. Advances in Human Aspects of Transportation. Part I, 2014,
Vol. 431, 7.
Naujoks, Frederik and Neukum, Alexandra. 2017. A review of non-driving-related tasks used in studies
on automated driving. International Confreence on Applied Human Factors and Ergonomics. 2017, Vol. 597,
pp. 525-537.
Naujoks, Frederik, Purucker, C. and Neukum, A. 2016. Secondary task engagement and vehicle
automation - comparing the effects of different automation levels in an on-road experiment. Transportation
Research F. 38, 2016, pp. 67-82.
Neubauer, Catherine and Matthews, Gerald. 2012. The Effects of Cell Phone Use and Automation on
Driver Performance and Subjective State in Simulated Driving. Proceedings of the Human Factors and
Ergonomics Society Annual Meeting. 56, 2012.
Nilsson, Peter, Laine, Leo and Jacobson, Bengt. 2017. A Simulator Study Comparing Characteristics of
Manual and Automated Driving During Lane Changes of Long Combination Vehicles. Transactions of
Intelligent Transportation Systems. 18, 2017, Vol. 9, pp. 2514-2524.
Ohn-Bar, Eshed and Trivedi, Mohan M. 2016. What Makes an On-road Object Important? Intl. Conference
on Pattern Recognition. 23, 2016, pp. 3392-3397.
Ohn-Bar, Eshed, et al. 2014. Head, Eye, and Hand Patterns for Driver Activity Recognition. 22nd IEEE
International Conference on Pattern Recognition. 2014.
Petermann-Stock, I., et al. 2013. Wie lange braucht der Fahrer - eine Analyse zu Übernahmezeiten aus
verschiedenen Nebentätigkeiten während einer hochautomatisierten Staufahrt. 6. Tagung
Fahrerassistenzsysteme. Der Weg zum automatischen Fahren. 2013.
Radlmayr, J., et al. 2014. How Traffic Situations and Non-Driving Related Tasks Affect the Take-Over
Quality in Highly Automated Driving. Proceedings of the Human Factors and Ergonomics Society Annual
Meeting. 2014, Vol. I, Vol. 58, pp. 2063-2067.
Royal Automobile Club Foundation for Motoring. 2017. Road Traffic Estimates: Great Britain 2016.
Gillingham : Department of Transport, 2017.
Schermers, G., Malone, K. M. and Van Arem, B. 2004. Dutch Evaluation of Chauffeur Assistant (DECA):
Traffic flow effects of implementation in the heavy goods vehicles sector. Nagoya, Aichi : ITS World Congress,
2004.
Schmidt, Jürgen, et al. 2016. Driver Drowsiness and Behavior Detection in Prolonged Conditionally
Automated Drives. IEEE Intelligent Vehicles Symposium. Gothenburg, Sweden, 2016.
24 Response Times and Gaze Behavior of Truck Drivers in Time Critical Conditional Automated Driving Take-overs
Schneider, J. H. 2009. Modellierung und Erkennung von Fahrsituationen und Fahrmanövern für
sicherheitsrelevante Fahrerassistenzsysteme. Technische Universität Chemniz, Chemniz : Fakultät für
Elektrotechnik und Informationstechnik, 2009.
Shen, S. and Neyens, D. M. 2014. Assessing drivers' performance when automated driver support systems
fail with different levels of automation. Proceedings of the Human Factors and Ergonomics Society Annual
Meeting (Vol. 58, No.1). SAGE Publications 2014, pp. 2068-2072.
Standard, SAE International. 2018. Taxonomy and Definitions for Terms Related to Driving Automation
Systems for On-Road Motor Vehicles. J3016 2018.
Statista. 2017. Das Statistik-Portal. Höhen des Frauenanteils bei den sozialversicherungspflichtig
beschäftfigten Berufskraftfahrern in Deutschland in den Jahren 2012 bis 2016. [Online] 2017. [Cited: 12 12,
2017.] https://de.statista.com/statistik/daten/studie/202938/umfrage/hoehe-des-frauenanteils-bei-den-
kraftfahrzeugfuehrern-in-deutschland/.
Statistisches Bundesamt. 2013. Verkehr auf einen Blick. Wiesbaden, Germany : s.n., 2013. 0080006-
13900-1.
Streit, Tatjana, et al. 2014. Deutsches Mobilitätpanel (MOP) - Wissenschaftliche Begleitung und Auswertung
Bericht 2012/2013: Alltagsmobilität und Fahrleistungen. Karlsruhe, Germany : Karlsruher Institut für
Technologie (KIT) Institut für Verkehrswesen, 2014. Forschungsprojekt FE-NR. 70.0864/2011.
Tofetti, A., et al. 2009. CityMobil. Transportation Research Record: Journal of the Transportation Research
Board. 2009, 2110(1), pp. 1-8.
U.S. Department of Transportation Federal Highway Administration. 2014. Annual Vehicle Distance
Traveled in Miles and Related Data. Washington D.C., USA : Policy and Governmental Affairs Office of
Highway Policy Information, 2014. VM-1.
Vogelpohl, Tobias and Vollrath, Mark. 2017. Tiredness and level 3 - automated driving. Berlin : GDV The
German Insurers, 2017. Compact accident research 70.
Vogelpohl, Tobias, et al. 2016. Übergabe von hochautomatisiertem Fahren zu manueller Steuerung. Berlin :
Gesamtverband der Deutschen Versicherungswirtschaft e.V., 2016.
von Benda, Helga, Hoyos, Carl and Schaible-Rapp, Agnes. 1983. Klassifikation und Gefährlichkeit von
Strassenverkehrssituationen. Germany : Bundesanstalt für Strassenwesen, Bereich Unfallforschung, 1983.
Walch, Marcel, et al. 2015. Autonomous Driving: Investigating the Feasibility of Car-driver Handover
Assistance. Proceedings of the 7th International Conference on Automotive User Interfaces and Interactive
Vehicular Applications. Automotive UI '15, 2015, pp. 11-18.
Weinbeer, V., et al. 2017. Highly automated driving: How to get the driver drowsy and how does
drowsiness influence various take-over-aspects? [book auth.] M. Lienkamp. Proceedings of the Tagung
Fahrerassistenz. München : Volume 8, 2017.
Weinbeer, V., Muhr, T. and Bengler, K. 2019. Automated Driving: The Potential of Non-driving-Related
Tasks to Manage Driver Drowsiness. [book auth.] S. Bagnara, et al. Proceedings of the 20th Congress of the
International Ergonomics Association (IEA 2018). Advances in Intelligent Systems and Comuting. Florence :
Springer, 2019, pp. 179-188.
Wulf, Felix, et al. 2013. Effects of Human-Machine Interaction Mechanisms on Situation Awareness in
Partly Automated Driving. 16th International IEEE Conference on Intelligent Transportation Systems. 2013.
Young, M.S. and Stanton, N.A. 2007. Back to the future: brake reaction times for manual and automated
vehicles. Ergonomics. 50, 2007, Vol. 1, pp. 46-58.
Zeeb, K., Buchner, A. and Schrauf, M. 2015. What determines the take-over time? An integrated model
approach of driver take-over after automated driving. Accident Analysis & Prevention, 78. 2015, pp. 212-221.
Zeeb, Kathrin, Buchner, Axel and Schrauf, Michael. 2016. Is take-over time all that matters? The impact
of visual-cognitive load on driver take-over quality after conditionally automated driving. Accident Analysis
and Prevention. 2016, pp. 230-239.
Zhang, Bo, et al. 2017. Driver Response Times when Resuming Manual Control from Highly Automated
Driving in Truck Platooning Scenarios. Conference: RSS2017 - Road Safety & Simulation international
conference. 2017, The Hague, Netherlands.
... A paper [7] evaluated the influence of drivers when reading news and emails, watching a video clip, and engaging with a tablet. Another paper [8] used video and tablet gaming NDRTs to evaluate driving behavior in a critical conditional takeover. The authors concluded that there was no influence of NDRTs on reaction time. ...
... At present, the link of gameplay to awareness has not been explored; most inquiries are directed to the resumption of control and not on the development of awareness using games. Conventional approaches utilize quiz games or touch interfaces inside consoles or tablets to assess ToR [8,35]. The setup is predominantly an eyes-off-the-road setup that negatively affects situational awareness. ...
... This suggests that the AR-Game driver has a consistent RT compared to other tasks that fluctuate with attention shifts. The overall recognition time for all subjects yielded a reaction time of 2.9 s, which agrees with the findings of other researchers [8,45]. The findings suggest that a driver engaging in an AR-Game would not be impaired by the gaming elements in recognizing threatening scenarios. ...
Article
Full-text available
Background: As Automated Driving Systems (ADS) technology gets assimilated into the market, the driver’s obligation will be changed to a supervisory role. A key point to consider is the driver’s engagement in the secondary task to maintain the driver/user in the control loop. This paper aims to monitor driver engagement with a game and identify any impacts the task has on hazard recognition. Methods: We designed a driving simulation using Unity3D and incorporated three tasks: No-task, AR-Video, and AR-Game tasks. The driver engaged in an AR object interception game while monitoring the road for threatening road scenarios. Results: There was a significant difference in the tasks (F(2,33) = 4.34, p = 0.0213), identifying the game-task as significant with respect to reaction time and ideal for the present investigation. Game scoring followed three profiles/phases: learning, saturation, and decline profile. From the profiles, it is possible to quantify/infer drivers’ engagement with the game task. Conclusion: The paper proposes alternative monitoring that has utility, i.e., entertaining the user. Further experiments with AR-Games focusing on the real-world car environment will be performed to confirm the performance following the recommendations derived from the current test.
... Experiments can be carried out even in high-risk situations where accidents may occur. For these reasons, related studies mainly used driving simulators [30][31][32][33][34]. To provide real vehicle experiences in the driving simulator, visual, auditory, and tactile cues and a vehicle motion generation system are important [35]. ...
... This software provides solutions for virtual test driving, including basic vehicle systems such as dynamics and powertrain, driving scenario generators, autonomous driving sensor models, and realistic visual graphics. [30][31][32][33][34]. To provide real vehicle experiences in the driving simulator, visual, auditory, and tactile cues and a vehicle motion generation system are important [35]. ...
Article
Full-text available
According to SAE J3016, autonomous driving can be divided into six levels, and partially automated driving is possible from level three up. A partially or highly automated vehicle can encounter situations involving total system failure. Here, we studied a strategy for safe takeover in such situations. A human-in-the-loop simulator, driver–vehicle interface, and driver monitoring system were developed, and takeover experiments were performed using various driving scenarios and realistic autonomous driving situations. The experiments allowed us to draw the following conclusions. The visual–auditory–haptic complex alarm effectively delivered warnings and had a clear correlation with the user’s subjective preferences. There were scenario types in which the system had to immediately enter minimum risk maneuvers or emergency maneuvers without requesting takeover. Lastly, the risk of accidents can be reduced by the driver monitoring system that prevents the driver from being completely immersed in non-driving-related tasks. We proposed a safe takeover strategy from these results, which provides meaningful guidance for the development of autonomous vehicles. Considering the subjective questionnaire evaluations of users, it is expected to improve the acceptance of autonomous vehicles and increase the adoption of autonomous vehicles.
... A paper [7] evaluated the influence of drivers in news and email reading, watching a video clip, and engaging with a tablet. Another paper [8] used video and a tablet gaming NDRT to evaluate driving behavior in a critical conditional take-over. The authors concluded that there was no influence of NDRT on reaction time. ...
... The overall recognition time for all subjects yielded a reaction time of 2.9 s, which agrees with the findings of other researchers [8,40] . Recognition time is a general interest for automobile system designers and researchers alike in intention studies. ...
Preprint
Full-text available
As Automated Driving Systems (ADS) technology gets assimilated into the market, the driver’s obligation will be changed to a supervisory role. A key point to consider is the driver’s engagement in the secondary task to maintain the driver/user in the control loop. The paper’s objective is to monitor driver engagement with a game and identify any impacts the task has on hazard recognition. We designed a driving simulation using Unity3D and incorporated three tasks: No-task, AR-Video, and AR-Game tasks. The driver engaged in an AR object interception game while monitoring the road for threatening road scenarios. From the results, there was less than 1 second difference between the means of gaming task (mean = 2.55s, std = 0.1002s) to no-task (mean = 2.55s, std = 0.1002s). Game scoring followed three profiles/phases: learning, saturation, and decline profile. From the profiles, it is possible to quantify/infer drivers’ engagement with the game task. The paper proposes alternative monitoring that has utility, i.e., entertaining the user. Further experiments AR-Game focusing on real-world car environment will be performed to confirm the performance following the recommendations derived from the current test.
... Kinesthetic: Kinesthetic input comprises the position and movement of body parts and activity of muscles and joints [272], to which we also attribute the facial muscle activity needed for emotion and facial expressions. For example, body position classifiers were used to recognize driver distraction [390], and posture tracking allows inference of situation awareness and reaction times [233]. The body pressure applied to the seat was also analyzed to inform optimal seat vibration points [181]. ...
Article
Full-text available
Automotive user interfaces constantly change due to increasing automation, novel features, additional applications, and user demands. While in-vehicle interaction can utilize numerous promising modalities, no existing overview includes an extensive set of human sensors and actuators and interaction locations throughout the vehicle interior. We conducted a systematic literature review of 327 publications leading to a design space for in-vehicle interaction that outlines existing and lack of work regarding input and output modalities, locations, and multimodal interaction. To investigate user acceptance of possible modalities and locations inferred from existing work and gaps unveiled in our design space, we conducted an online study (N=48). The study revealed users' general acceptance of novel modalities (e.g., brain or thermal activity) and interaction with locations other than the front (e.g., seat or table). Our work helps practitioners evaluate key design decisions, exploit trends, and explore new areas in the domain of in-vehicle interaction.
... For fatigued drivers, short TTB can lead to more hasty takeover behaviours such as the rapid braking behaviour described in Section 4.2.2, and hasty takeovers are not necessarily safe ones. During hasty takeovers, drivers do not check the rear view mirror or perform other avoidance behaviours, which can lead to collisions [92]. Moreover, some researchers have found that shorter TTB worsened the driver's vehicle control after takeover and resulted in shorter TTC, longer maximum lateral or longitudinal acceleration, and greater standard deviations in lane position and steering angle [93]. ...
Article
Full-text available
With advancements in automated driving technology, Level 3 vehicle automation has been proliferating. In Level 3, drivers can turn their attention to things unrelated to driving and only takeover in cases of emergency. However, the transition to the main driving task can be affected by different physical and psychological changes in drivers such as fatigue and distraction. Although existing research in the field of manual driving has demonstrated the effects of fatigue on driver performance, there have been relatively few studies on fatigue in the field of automated driving. Fatigue decreases drivers’ alertness, which can prevent drivers from conducting necessary emergency takeover manoeuvres in a safe and timely manner. This systematic literature review was conducted to establish relationships between driving fatigue and takeover behaviour in the existing literature on Level 3 automated driving systems. Different from existing reviews of driving fatigue, this paper focuses on the commonly used and most effective evaluation indicators of driving fatigue and further discusses the methods to improve the takeover efficiency of fatigued drivers. This is the first detailed systematic review of general relationships between fatigue in automated driving and drivers’ takeover behaviours when the vehicle sends a takeover request. Researchers are invited to expand on the findings here to further clarify the findings on the associations between driver fatigue indicators in automated vehicle driving and drivers’ takeover response times.
... In the present study, the drivers' takeover performance in critical situations was measured by their reaction time and abnormal performances, the SDLP, the lane departure probability, and the minTTC. Specifically, the brake reaction time under the deceleration situation is defined as the duration between the onset of lead vehicle deceleration and the ego vehicle's brake pedal reaching 10% of the entire stroke (Frederik, Höfling, Christian, & Zeeb, 2018;Lotz et al., 2019). Meanwhile, the steering reaction time under the lane change situation is defined as the duration between the onset of the lead vehicle's lane change and the ego vehicle's steering wheel rotation reaching 2 • (steering data = 2/180 = 0.011) (Perlman et al., 2019), as shown in Fig. 4. ...
Article
Mixed control by driver and automated system will remain in use for decades until fully automated driving is perfected. Thus, drivers must be able to accurately regain control of vehicles in a timely manner when the automated system sends a takeover request (TOR) at its limitation. Therefore, determining the factors that affect drivers’ takeover quality at varying levels of automated driving is important. Previous studies have shown that visually distracting secondary tasks impair drivers’ takeover performance and increase the subjective workload. However, the influence of purely cognitive distracting secondary tasks on drivers’ takeover performance and how this influence varies at different levels of automation are still unknown. Hence, a 5 (driving modes) × 3 (cognitive secondary tasks) factorial design with the within-subject factors was adopted for this driving simulator experiment. The sample consisted of 21 participants. The participants’ subjective workloads were recorded by the NASA-Task Load Index (NASA-TLX). Results showed that compared to manual driving conditions, the drivers’ subjective workloads were significantly reduced in both partially and highly automated driving conditions, even with a TOR, confirming the benefit of the automated driving system in terms of reducing the driving workload. Moreover, the drivers exhibited a lower takeover behavior quality at high levels of automation than manual driving in terms of increased reaction time, abnormal performance, standard deviation of lane position, lane departure probability, and reduced minimum of time to collision. However, at the highly automated driving condition, the drivers’ longitudinal driving safety and ability to follow instructions improved when performing a highly cognitive secondary task. This phenomenon possibly occurred because automated driving conditions lead to an underload phenomenon, and the execution of highly cognitive tasks transfers drivers into moderate load, which helps with the drivers’ takeover performance.
... Green's much-cited review [38] declared that the PbRT of 2.50 s adopted in [25] is a reasonable guess for the 90% to 95% general population. Predominantly driven by a trained group of professionals, as opposed to occasional drivers in the car context, truck drivers will response or take back control quicker than passenger car drivers [54]. Because the separate PbRT for trucks and passenger drivers is not generally used in highway design [25], the PbRT of truck drivers in L0 − is still 2.50 s. ...
Article
Full-text available
Truck automation is emerging as an innovative technology with benefits in traffic safety and the economy to revolutionize freight traffic. Despite these benefits, the potential negative or positive effects of different driving automation levels (from no automation to full automation) on highway geometry remained to be determined. In this study, differences related to sight distance characteristics among varied automation levels were firstly discussed and calibrated. Then, seven analysis scenarios of typical levels were proposed. Based on each level with tailored characteristics, the current models of geometric design elements including the required stopping sight distance, horizontal sight line offset, and lengths of vertical curves were revised. Finally, impacts of each level on computed values of those elements were evaluated. Results show that high or full driving automation could substantially lower the requirements of geometric design. Active safety systems have a similar role but with less significant effects. Differently, the driver assistance and partial or conditional automation systems put a higher demand on the road geometric design in terms of driving safety. Outcomes of this study can be used to design real-world geometry of dedicated lanes and provide a methodological basis for the operation of different driving automation features.
Article
Exploring the influence of secondary task immersion duration on the take-over process is an important task in the research on conditional automated driving (SAE, L3). In this paper, with the tool of driving simulation, two different secondary task types and take-over request (TOR) times were designed for the ramp area take-over scenario. Forty-two participants were recruited to conduct driving simulation experiments. Based on the K-means algorithm, different immersion durations of secondary tasks were clustered. Three driving indicators, namely speed, lateral offset, and accelerator pedal depth, were selected to characterize the take-over performance. Then the influence of immersion duration on the take-over response time and take-over performance was analyzed. The effect of different immersion durations on the section traffic operation during the take-over process was then investigated with the help of VISSIM software. The results showed that the TOR time had no significant effect on the take-over response time, and the take-over response time was positively correlated with the immersion duration. At the beginning of the take-over process, the take-over performance of drivers between different immersion durations was different, for example, the average speed of the vehicle is lower when the immersion duration is low. In the take-over process, the driver’s take-over speed and accelerator pedal operation intensity are negatively correlated with take-over urgency. The immersion duration of secondary tasks has different degrees of influence on the efficiency of section traffic flow. With the improvement of immersion duration level, the average vehicle delay on the road section also increases to a certain extent.
Article
Cooperative driving of human driver and automated system can effectively reduce the necessity of extremely accurate environment perception of highly automated vehicles, and enhance the robustness of decision-making and motion control. However, due to the two players' different intentions, severe conflicts may exist during the cooperation, which often result in negative consequences on driving safety and maneuverability. This paper presents an indirect shared control method to model the situation and improve the driving performance, which focus on the affine input nonlinear vehicle dynamic system for shared controller design under the framework of non-zero sum differential game. The Nash equilibria strategy indicates the best response for the automated system, which can guide the automated controller to act more safely and comfortably. Aimed to obtain fast solution for practical application, approximate dynamic programming is utilized to find the Nash equilibria, which is represented by deep neural networks and solved iteratively. Driver-in-the-loop tests on a driving simulator were conducted to verify the performance of the proposed method under highway driving scenarios. The results show that the designed controller is able to reduce the driving workload and ensure the driving safety.
Article
Drowsiness and distraction are major factors of road crashes and responsible of>35% of road fatalities. Automated driving could solve or minimize their impact, yet it is also in itself a way to promote them. Previous literature reviews and meta-analysis regarding take-overs during automated driving primarily focused on distraction rather than drowsiness. We thus present a systematic and meta-analysis literature review focused on the effect of distraction and drowsiness on take-over performance. From an initial selection of 1896 articles from databases, we obtained by applying systematic review methodology a total of 58 articles with 42 articles dedicated to distraction and 17 articles related to drowsiness. According to our analysis, we demonstrated that distraction and drowsiness increased the take-over request reaction time (TOR-RT), which could also lead to a reduction of the quality of take-overs. In addition, this longer reaction time was even more important in the case of handheld non-driving related tasks, which is important to consider as phone use is among the most frequent tasks done during automated driving. On a more methodological aspect, we also demonstrated that take-over time budget had a significant effect on TOR-RT. However, it is difficult to estimate to what extend distraction and drowsiness could impact the take-over quality, even if several elements supported safety issues. We underpinned several limits of the current methodologies applied in the study of distraction and drowsiness such as (i) the lack of additional measures related to the take-over quality (e.g., accelerations, collision rate), (ii) the many different methodologies applied to the determination of the TOR-RT (e.g., deactivation by the steering wheel, pedals, button), (iii) the high frequency of take-over requests which can lead to habituation effects, (iv) the lack of control conditions, (v) the fact that the level of drowsiness was relatively low in most studies. We thus highlighted recommendations for a better estimation of the effect of distraction and drowsiness on take-over performance. Further studies should adopt more standardized measures of TOR-RT and additional take-over quality measures, try minimizing the number of take-over requests, and carefully consider the time budget available for the use case since it influences the TOR-RT. Regarding distraction, researchers should consider the impact of tasks requiring handholding items. Concerning drowsiness, further protocols should consider the non-linearity of drowsiness and presence of micro sleeps and favor take-over requests based on drowsiness level protocols rather than on fixed duration protocols.
Conference Paper
Full-text available
In the context of highly automated driving (HAD) the driver state drowsiness is becoming increasingly important. For assessing the usefulness of different strategies to manage drowsiness during HAD, appropriate test methods are needed. To determine whether a right-hand-drive vehicle (RHDV) is a suitable method, a study with 31 participants was conducted on the motorway A9 in Germany. Two investigators evaluated the drowsiness level (DL) of the participants during the test drive. Depending on the participant's DL Requests to Intervene (RtI) were triggered. There was no statistically significant influence of drowsiness on take-over time aspects. Our results indicate that extremely drowsy drivers are still able to perceive and to react to a RtI. However, it should be considered that the take-over scenario used in this study was rather simple and quality aspects could not be assessed by the RHDV-setting. This study demonstrates that it is possible to induce and enhance drowsiness by controlling several influencing factors such as caffeine and atmosphere. Thus, this approach can be helpful for future studies when evaluating the effectiveness of different reactivation or transition strategies to manage drowsiness during HAD.
Conference Paper
Full-text available
The paper proposes a taxonomy for testing scenarios used in human factors research of Level 3 automated vehicles. Therefore, the literature was reviewed and testing scenarios were extracted. To categorize these scenarios, the four factors urgency, predictability, criticality and complexity of the driver response are introduced and defined. Furthermore, testing scenarios are suggested in dependence of the most important human factors research questions in Level 3 automated driving. The taxonomy thereby serves as a guidance and framework for the scenario selection and design of experiments in the context of Level 3 automated vehicle guidance.
Article
Full-text available
We investigated after effects of automation in take-over scenarios in a high-end moving-base driving simulator. Drivers performed evasive manoeuvres encountering a blocked lane in highway driving. We compared the performance of drivers 1) during manual driving, 2) after automated driving with eyes on the road while performing the cognitively demanding n-back task, and 3) after automated driving with eyes off the road performing the visually demanding SuRT task. Both minimum time to collision (TTC) and minimum clearance towards the obstacle disclosed a substantial number of near miss events and are regarded as valuable surrogate safety metrics in evasive manoeuvres. TTC proved highly sensitive to the applied definition of colliding paths, and we prefer robust solutions using lane position while disregarding heading. The extended time to collision (ETTC) which takes into account acceleration was close to the more robust conventional TTC. In line with other publications, the initial steering or braking intervention was delayed after using automation compared to manual driving. This resulted in lower TTC values and stronger steering and braking actions. Using automation, effects of cognitive distraction were similar to visual distraction for the intervention time with effects on the surrogate safety metric TTC being larger with visual distraction. However the precision of the evasive manoeuvres was hardly affected with a similar clearance towards the obstacle, similar overshoots and similar excursions to the hard shoulder. Further research is needed to validate and complement the current simulator based results with human behaviour in real world driving conditions. Experiments with real vehicles can disclose possible systematic differences in behaviour, and naturalistic data can serve to validate surrogate safety measures like TTC and obstacle clearance in evasive manoeuvres.
Article
Full-text available
IntroductionThis paper builds our knowledge of truck driver behaviour in and experience of automated truck platooning, focusing on the effect of partially and fully automated truck platoons on driver workload, trust, acceptance, performance, and sleepiness. Methods Twenty-four male drivers experienced three conditions in a truck driving simulator, i.e., baseline, partial automation, and full automation: the baseline condition was driving with standard cruise control; partial automation was automated longitudinal control ten metres behind the truck in front, with the driver having to steer; and full automation was automated longitudinal and lateral control. Each condition was simulated in three situations: light traffic, heavy traffic, and heavy traffic plus fog. ResultsThe experiment demonstrated that automation affects workload. For all workload measures, partial automation produced higher workload than did the full-automation or baseline condition. The two measures capturing trust, i.e., the Human Trust in Automated Systems Scale (HTASS) and Cooper–Harper Scales of Workload, Temporal Load, Situation Awareness, and Trust, were consistent and indicated that trust was highest under the baseline condition, with little difference between partial and full automation. Driver acceptance of both levels of automation was lower than acceptance of baseline. Drivers rated their situation awareness higher for both partial and full automation than for baseline, although both levels of automation led to higher sleepiness. Conclusions Workload was higher for partial than for full automation or the baseline condition. Trust and acceptance were generally highest in the baseline condition, and did not differ between partial and full automation. Drivers may believe that they have more situation awareness during automated driving than they actually do. Both levels of automation led to a higher degree of sleepiness than in the baseline condition. The challenge when implementing truck platooning is to develop a system, including human–machine interaction (HMI), that does not overburden the driver, properly addresses driver sleepiness, and satisfies current legislation. The system also must be trusted and accepted by drivers. To achieve this, the development of well-designed HMI will be crucial.
Article
Full-text available
Objective: The aim of this study was to review existing research into driver control transitions and to determine the time it takes drivers to resume control from a highly automated vehicle in noncritical scenarios. Background: Contemporary research has moved from an inclusive design approach to adhering only to mean/median values when designing control transitions in automated driving. Research into control transitions in highly automated driving has focused on urgent scenarios where drivers are given a relatively short time span to respond to a request to resume manual control. We found a paucity in research into more frequent scenarios for control transitions, such as planned exits from highway systems. Method: Twenty-six drivers drove two scenarios with an automated driving feature activated. Drivers were asked to read a newspaper, or to monitor the system, and to relinquish, or resume, control from the automation when prompted by vehicle systems. Results: Significantly longer control transition times were found between driving with and without secondary tasks. Control transition times were substantially longer than those reported in the peer-reviewed literature. Conclusion: We found that drivers take longer to resume control when under no time pressure compared with that reported in the literature. Moreover, we found that drivers occupied by a secondary task exhibit larger variance and slower responses to requests to resume control. Workload scores implied optimal workload. Application: Intra- and interindividual differences need to be accommodated by vehicle manufacturers and policy makers alike to ensure inclusive design of contemporary systems and safety during control transitions.
Chapter
Conditional autonomous driving requires the description of sufficient time reserves for drivers in take-over situations. The definition of this time reserve has not been addressed for the truck context thus far. Through the observation of physiological measures, the possibility of estimating reaction times is considered. Driver data is collected with a remote eye-tracker and body posture camera. Empirical data from a simulator study is utilized to train and compare four machine learning algorithms and generate driver features. The estimation of take-over times is defined as a classification problem with four reaction time classes, leading to a misclassification rate of a linear support vector machine (SVM) of 38.7%. Utility of driver features for reaction time estimation are discussed.