ArticlePDF Available

Abstract and Figures

Objective: The aim of this study was to review existing research into driver control transitions and to determine the time it takes drivers to resume control from a highly automated vehicle in noncritical scenarios. Background: Contemporary research has moved from an inclusive design approach to adhering only to mean/median values when designing control transitions in automated driving. Research into control transitions in highly automated driving has focused on urgent scenarios where drivers are given a relatively short time span to respond to a request to resume manual control. We found a paucity in research into more frequent scenarios for control transitions, such as planned exits from highway systems. Method: Twenty-six drivers drove two scenarios with an automated driving feature activated. Drivers were asked to read a newspaper, or to monitor the system, and to relinquish, or resume, control from the automation when prompted by vehicle systems. Results: Significantly longer control transition times were found between driving with and without secondary tasks. Control transition times were substantially longer than those reported in the peer-reviewed literature. Conclusion: We found that drivers take longer to resume control when under no time pressure compared with that reported in the literature. Moreover, we found that drivers occupied by a secondary task exhibit larger variance and slower responses to requests to resume control. Workload scores implied optimal workload. Application: Intra- and interindividual differences need to be accommodated by vehicle manufacturers and policy makers alike to ensure inclusive design of contemporary systems and safety during control transitions.
Content may be subject to copyright.
Take-over time in highly automated vehicles: non-critical transitions to and from manual control
Alexander Eriksson, Neville A Stanton
Transportation Research Group, Faculty of Engineering and the Environment, University of
Southampton, Boldrewood campus, SO16 7QF, UK
Cite as: Eriksson, A & Stanton, N. A. (2017) Take-over time in highly automated vehicles: non-critical
transitions to and from manual control, Human Factors, DOI: 10.1177/0018720816685832
Corresponding author: Alexander Eriksson Transportation Research Group, Faculty of Engineering
and the Environment, University of Southampton, Boldrewood campus, SO16 7QF, UK. Email:
Alexander.eriksson@soton.ac.uk
Acknowledgements: This research has been conducted as a part of the European Marie Curie ITN
project HFAuto - Human Factors of Automated driving (PITN-GA-2013-605817)
Abstract
Objective: The aim of this study was to review existing research into driver control transitions
and to determine the time it takes drivers to resume control from a highly automated vehicle in
non-critical scenarios.
Background: Contemporary research has moved from an inclusive design approach to only
adhering to mean/median values when designing control transitions in automated driving.
Research into control transitions in highly automated driving has focused on urgent scenarios
where drivers are given a relatively short time span to respond to a request to resume manual
control. We found a paucity in research into more frequent scenarios for control transitions,
such as planned exits from highway systems.
Method: Twenty six drivers drove two scenarios with an automated driving feature activated.
Drivers were asked to read a newspaper, or to monitor the system, and to relinquish, or resume,
control from the automation when prompted by vehicle systems.
Results: Significantly longer control transition times were found between driving with and
without secondary tasks. Control transition times were substantially longer than those reported
in the peer-reviewed literature.
Conclusion: We found that drivers take longer to resume control when under no time-pressure
compared to that reported in the literature. Moreover, we found that drivers occupied by a
secondary task exhibit larger variance, and slower responses to requests to resume control.
Workload scores implied optimal workload.
Application: Intra- and inter-individual differences need to be accommodated by vehicle
manufacturers and policy makes alike to ensure inclusive design of contemporary systems and
safety during control transitions.
Keywords: Automation, Automated Driving, Control Transitions, Take-Over Requests, Driving
Performance, Task Regulation
Précis: This study reviews the literature for non-critical control transitions in highly automated
driving and contrasts the reported results with driver-paced control transitions. The results show
increased response times compared to the literature, and when engaged in secondary tasks
compared with no task engagement. The study also reports on transition times from manual to
automated driving for the first time.
Introduction
Highly automated vehicles are becoming an engineering reality and will become commonplace on
our roads in the very near future (Walker et al., 2015). For example, Tesla released its Autopilot
feature in 2015, with BMW, Mercedes and Audi quickly following with similar technologies (Audi,
2014; BMW, 2013; safecarnews.com, 2015). It is a common misconception that these features are
‘highly automated’ when they are in fact classified as conditional driving automation (SAE Level 3,
'SAE International, 2016). This means that they come with limitations, such as the features may only
be intended for use under certain Operational Design Domains, for example, on highways, as well as
requiring driver monitoring and intervention (Stanton et al., 1997; Wolterink et al., 2011).
When using a driver assistance system that is able to automate the driving task to such an extent
that hands- and feet-free driving is possible (SAE Level 3, 'SAE International, 2016), the driver
becomes decoupled from the operational and tactical levels of control (Michon, 1985; Stanton &
Young, 2005), leaving the high level strategic goals to be dealt with by the driver (until the point of
resuming manual control). This is a form of “driver-initiated automation”, where the driver is in
control of when the system becomes engaged or disengaged (Banks & Stanton, 2015, 2016; Lu & de
Winter, 2015). Indeed, according to Bainbridge (1983), two of the most important tasks for humans
in automated systems are monitoring the system to make sure it performs according to expectations
and to be ready to resume control when the automation deviates from expectation (Stanton &
Marsden, 1996). Research has shown that vehicle automation has a negative effect on mental
workload and situation awareness (Endsley & Kaber, 1999; Kaber & Endsley, 1997; Stanton et al.,
1997; Stanton & Young, 2005; Young & Stanton, 2002), and that reaction times increase as the level
of automation increases (Young & Stanton, 2007). This becomes problematic when the driver is
expected to regain control when system limits are exceeded, as a result of a sudden automation
failure. Failure-induced transfer of control has been extensively studied (see Desmond et al., 1998;
Molloy & Parasuraman, 1996; Stanton et al., 1997; Stanton et al., 2001; Strand et al., 2014; Young &
Stanton, 2007). In one failure-induced control-transition-scenario, Stanton et al. (1997) found that
more than a third of drivers failed to regain control of the vehicle following an automation failure
whilst using Adaptive Cruise Control. Other research has shown that it takes approximately one
second for a driver manually driving to respond to an unexpected and sudden braking event in traffic
(Summala, 2000; Swaroop & Rajagopal, 2001; Wolterink et al., 2011). Young and Stanton (2007)
report brake reaction times of 2.13±0.55 seconds for drivers using Adaptive Cruise Control (SAE
Level 1), and brake reaction times of 2.48±0.66 seconds for drivers with Adaptive Cruise Control and
Assistive Steering (SAE Level 2). By contrasting the results from Young and Stanton (2007) where
drivers experienced an automation failure whilst a lead vehicle suddenly braked, with Summala
(2000) it seems like it takes an additional 1.1-1.5 seconds to react to sudden events requiring braking
whilst driving with Driver Assistance Automation (SAE Level 1) and Partial Driving Automation (SAE
Level 2). This increase, in combination with headways as short as 0.3 seconds (Willemsen et al.,
2015) coupled with evidence that drivers are poor monitors (Molloy & Parasuraman, 1996), could
actually cause accidents. Evidently, automating the driving task seem to have a detrimental effect on
driver reaction time (Young & Stanton, 2007). Therefore, as Cranor (2008) and Eriksson and Stanton
(2016) proposed, the driver needs to receive appropriate feedback if they are to successfully re-
enter the driving control loop. Recent research efforts have been made to determine the optimal
Take-Over-Request lead time (TORlt: the lead-time from a ‘take-over request’ (TOR) to a critical
event, such as a stranded vehicle) and Take Over reaction time (TOrt: the time is takes the driver to
take back control of the vehicle from the automated system when a TOR has been issued) with times
varying from 0-30 seconds for TORlt and 1.14-15 seconds for TOrt as shown in Table 1. A total of 25
papers reported either TORlt, or TOrt and were included in the review (see Table 1).
Table 1. Papers included in the review. Modalities for the Take-over request is coded as: A =
Auditory, V = Visual, H = Haptic and B = Brake Jerk.
Paper
TORlt
TOrt
Modality
1
Gold et al. (2016)
7
2.47-3.61
-
2
Körber et al. (2015)
3
-
A
3
Louw et al. (2015b)
6.5
2.18-2.47
A
4
Zeeb et al. (2016)
2.5, 4
-
V
5
Damböck et al. (2012)
4, 6, 8
-
A
6
Kerschbaum et al. (2015)
7
2.22-3.09
V A
7
Belderbos (2015)
10
5.86-5.87
V A H
8
Walch et al. (2015)
4, 6
1.90-2.75
V A
9
Lorenz et al. (2014)
7
2.86-3.03
V A
10
Merat et al. (2014)
0
10-15
-
11
Naujoks et al. (2014)
-
2.29-6.90
V A
12
Schömig et al. (2015)
12
-
V A
13
Louw et al. (2015a)
3
-
V A
14
Zeeb et al. (2015)
2.5, 3, 3.5, 12
1.14
V A
15
Mok et al. (2015)
2, 5, 8
-
A
16
Gold et al. (2014)
5
1.67-2.22
V A
17
Radlmayr et al. (2014)
7
1.55-2.92
V A
18
Dogan et al. (2014)
3
-
V A
19
Gold et al. (2013)
5, 7
2.06-3.65
V A
20
van den Beukel and van der Voort (2013)
1.5, 2.2, 2.8
-
A
21
Melcher et al. (2015)
10
3.42-3.77
V A B
22
Naujoks and Nekum (2014)
0, 1, 2, 3, 4
-
V A
23
Feldhütter et al. (2016)
6
1.88-2.24
A
24
Payre et al. (2016)
2, 30
4.30-8.70
V A
25
Körber et al. (2016)
7
2.41-3.66
A
The review showed that the mean TORlt was 6.37±5.36 seconds (Figure 1) with a mean reaction time
of 2.96±1.96 seconds. The most frequently used TORlts tended to be; 3 seconds with a mean TOrt of
1.14±0.45 [studies 2, 13, 14, 18, 22], 4 seconds with a mean TOrt of 2.05±0.13 [studies 4, 8, 22], 6
seconds with a mean TOrt of 2.69±2.21 [studies 5, 8, 23], and 7 seconds with a mean TOrt of
3.04±1.6 [studies 1, 6, 9, 17, 19, 25] as shown in Figure 2.
Figure 1. The TORlt used in the reviewed papers. Several papers used a multitude of TOR lead
times and thus contributed on several points of the graph.
Take Over reaction times stay fairly consistent around 2-3.5 seconds in most control transitions, with
a few outliers, as seen in Figure 2. Belderbos (2015), Merat et al. (2014), Naujoks et al. (2014) and
Payre et al. (2016) show longer TOrt compared to the rest of the reviewed papers. Merat et al.
(2014) and Naujoks et al. (2014) had the control transition initiated without any lead time whereas
Belderbos (2015) and Payre et al. (2016) did. Merat et al. (2014) showed that there is a 10-15 second
time lag between the disengagement of the automated driving system and resumption of control by
the driver. Notably, the control transition was system initiated and lacked a pre-emptive TOR which
may have caused the increase in TOrt. Similarly, Naujoks et al. (2014) observed a 6.9 second TOrt
from when a TOR was issued and the automation disconnected until the driver resumed control in
situations where automation became unavailable due to missing line markings, the beginning of a
work zone or entering a curve. Based on personal communication with the author, the vehicle would
have crossed the lane markings after approximately 13 seconds, and would have reached the faded
lane markings approximately 10 seconds after the TOR. The velocity in Naujoks et al. (2014) was 50
Kph, which is fairly slow compared to most other TOR studies that use speeds over 100 Kph [studies
1, 3, 4, 6, 9, 10, 12, 14, 17, 19, 21, 23, 24, 25] and may have had an effect on the perceived urgency.
Figure 2. Take Over reaction time averages for all the conditions in the reviewed studies. Some
studies had more than one take over event and is therefore featured multiple times.
Belderbos (2015) showed TOrt’s of 5.86±1.57 to 5.87±4.01 when drivers were given a TORlt of 10
seconds during unsupervised automated driving. Payre et al. (2016) utilised two different TORlt’s, 2,
and 30 seconds. These TORlt’s produced significant differences in TOrt, the 2 second TORlt produced
a TOrt of 4.3±1.2 seconds and the two scenarios that used the 30 second TORlt produced TOrt’s of
8.7±2.7 seconds and 6.8±2.5 seconds respectively. The shorter TOrt of the two 30 second TOR
events occurred after the 2 second emergency TOR and could have been affected by the urgency
caused by the short lead time in the preceding, shorter TOR.
Merat et al. (2014) concluded, based on their observed TOrt, that there is a need for a timely and
appropriate notification of an imminent control transition. This observation is in line with the current
SAE-guidelines which state that the driver Is receptive to a request to intervene and responds by
performing dynamic driving task fallback in a timely manner(SAE International, 2016, p. 20). In
initial efforts to determine how long in advance the driver needs to be notified before a control
transition is initiated, Damböck et al. (2012) and Gold et al. (2013) explored a set of TOR lead times.
Damböck et al. (2012) utilised three TORlt’s, 4, 6, and 8 seconds and found that given an 8 second
lead time, drivers did not differ significantly from manual driving. This was confirmed by Gold et al.
(2013) who reported that drivers need to be warned at least 7 seconds in advance of a control
transition to safely resume control. These findings seem to have been the inspiration for the TORlt of
some recent work utilising timings around 7 seconds [studies 1, 6, 9, 17].
A caveat of a number of the reviewed studies is that the lead time given in certain scenarios such as;
disappearing lane markings, construction zones, and merging motorway lanes is surprisingly short,
from 0 to 12 seconds (c.f. Table 1), and will likely be longer in on road use cases [studies 4, 5, 11, 14,
15, 21]. The reason for this is the increasing accuracy of contemporary GPS hardware and associated
services, such as Google Maps. Such services are already able to direct lane positioning whilst driving
manually, as well as notifying drivers of construction zones and alternate, faster routes. Thus, there
is no evident gain of having short lead times in such situations.
Several of the studies reviewed have explored the effect of TOR’s in different critical settings by
issuing the TOR immediately preceding a time critical event [studies 1, 2, 3, 4, 6, 7, 8, 9, 13, 16, 17,
19, 20, 23, 24, 25]. These studies have explored how drivers manage critical situations in terms of
driving behaviour, workload, and scanning behaviour. Whilst it is of utmost importance to know how
quickly a driver can respond to a TOR and what the shortest TOR-times are in emergencies, there is a
paucity of research exploring the time it takes a driver to resume control in normal, non-critical,
situations. We argue that if the design of normal, non-critical, control transitions are designed based
on data obtained in studies utilising critical situations, there is a risk of unwanted consequences such
as: drivers not responding optimally due to too short lead time (suboptimal responses are
acceptable in emergencies as drivers are tasked with avoiding danger), drivers being unable to fully
regain situation awareness, and sudden, dramatic, increases in workload. Arguably, these
consequences should not be present in every transition of control as it poses a safety risk for the
driver, as well as other road users. Therefore, the aim of this study is to establish driver take-over
time in normal traffic situations when, e.g. the vehicle is leaving its Operational Design Domain as
these will account for most of the situations (Nilsson, 2014; SAE International, 2016). We also
explore how TOR take-over time is affected by a non-driving secondary task, as this was expected to
increase the reaction time (Merat et al., 2012).
Moreover, none of the papers included in the review mentioned the time it takes drivers to
transition from manual to automated driving. Gaining an understanding on the time required to
toggle an automated driving system on is important in situations such as: entering an area dedicated
to automated vehicles or engaging the automated driving mode in preparation for joining a platoon
as proposed by the SARTRE project (Robinson et al., 2010). Therefore, the aim of this study was to
establish the time it takes a driver to switch to automated driving when automated driving features
become available. Ultimately, this research aims to provide guidance about the lead-time required
to get the driver back into, and out of, the manual vehicle control loop.
Method
Participants
Twenty-six participants (10 females, 16 males) between 20 and 52 years of age (M = 30.27 SD = 8.52)
with a minimum one year and an average 10.57 years (SD = 8.61) of driving experience were asked
to take part in the trial. Upon recruiting participants, their informed consent was obtained. The
study complied with the American Psychological Association Code of Ethics and had been approved
by the University of Southampton Ethics Research and Governance Office (ERGO number 17771).
Equipment
The experiment was carried out in a fixed based driving simulator located at the University of
Southampton. The simulator was a Jaguar XJ 350 with pedal and steering sensors provided by
Systems Technology Inc. as part of STISIM Drive® M500W Version 3
(http://www.stisimdrive.com/m500w) providing a projected 140° field of view. Rear view- and side-
mirrors were provided through additional projectors and video cameras. The original Jaguar XJ
instrument cluster was replaced with a 10.6” Sharp LQ106K1LA01B Laptop LCD panel connected to
the computer via a RTMC1B LCD controller board to display computer generated graphics
components for TOR’s. The default configuration of the instrument cluster is shown in Figure 3.
Figure 3. The instrument cluster in its default configuration
When a TOR was issued the engine speed dial was hidden and the request was shown in its place.
The symbol asking for control resumption is shown in Figure 5 and the symbol used to prompt the
driver to re-engage the automation is shown in Figure 4.
Figure 4. The icon shown when the automation becomes available. The icon was coupled with a
computer generated voice message stating “automation available”.
Figure 5. The take-over request icon shown on the instrument cluster. The icon was coupled with a
computer generated voice message stating "please resume control”.
The mode switching human machine interface was located on a Windows tablet in the centre
console, consisting of two buttons, used either to engage, or to disengage the automation. To enable
dynamic dis-engagement and re-engagement of the automation, bespoke algorithms were
developed and are reported elsewhere (c.f. Eriksson et al., 2016).
Experiment Design
The experiment had a repeated-measures, within-subject design with three conditions; Manual,
Highly Automated Driving (HAD) and Highly Automated Driving with a secondary task. The conditions
were counterbalanced to counteract order effects. For the automated conditions, participants drove
at 70 mph on a 30 kilometre, three lane highway with some curves, with oncoming traffic in the
opposing three lanes separated by a barrier and moderate traffic conditions. The route was mirrored
between the two automated conditions to reduce familiarity effects whilst keeping the roadway
layout consistent.
In the secondary task condition, drivers were asked to read (in their head) an issue of National
Geographic whilst the automated driving system was engaged in order to remove them from the
driving (and monitoring) task. During both conditions, drivers were prompted to either resume
control from, or relinquish control to, the automated driving system. The control transition requests
were presented as both a visual cue (c.f. Figure 4 and Figure 5) and an auditory message, in line with
previous research [studies 6,7,8,9,11,12,13,14,16,17,18,19,22,24], in the form of a computer
generated, female voice stating “please resume control” or “automation available”. No haptic
feedback was included in this study, despite the findings from Petermeijer et al. (2016) and Scott and
Gray (2008) showing shorter reaction times when vibrotactile feedback was used. The motivation for
excluding the haptic modality was that it was under-represented in the review, with only 1 paper in
the review utilising a form of haptic feedback. Furthermore, Petermeijer et al. (2016) concluded that
haptic feedback is best suited for warnings, and as the current experimental design explored non-
critical warnings, no motivation for including haptics could be found. The interval in which these
requests were issued ranged from 30-45 seconds, thus allowing for approximately 24 control
transitions of which half were to manual control.
Procedure
Upon arrival, participants were asked to read an information sheet, containing information regarding
the study, the right to at any point abort their trial without any questions asked. After reading the
information sheet the participants were asked to sign an informed consent form. They were also told
that they were able to override any system inputs via the steering wheel, throttle or brake pedals.
Drivers were reminded that they were responsible for the safe operation of the vehicle regardless of
its mode (manual or automated), and thus needed to be able to safely resume control in case of
failure. This is in accordance with current legislation (United Nations, 1968) and recent amendments
to the Vienna Convention of Road Traffic. They were informed that the system may prompt them to
either resume or relinquish control of the vehicle, and that when such a prompt was issued they
were required to adhere to the instruction, but only when they felt safe doing so. This instruction
was intended to reduce the pressure on drivers to respond immediately and to reinforce the idea
that they were ultimately responsible for safe vehicle operation.
At the end of each driving condition, participants were asked to fill out the NASA-RTLX (Byers et al.,
1989). They were also offered a short break before continuing the study. Reaction time data were
logged for each transition to and from manual control.
Dependent variables
The following metrics were collected for each condition per participant.
Reaction time to the control transition request was recorded from the onset of the TOR. The control
transition request was presented in the instrument cluster coupled with a computer generated voice
to initiate a change in mode to and from manual control and was recorded in milliseconds.
Driving performance as measured by Standard Deviation of Steering Angular rate (degrees / second).
Subjective workload scores were collected via the NASA-TLX sub-scales at the end of each driving
condition. Overall workload score was calculated through the summation of sub-scales (Byers et al.,
1989; Hart & Staveland, 1988).
Analysis
The dependent measures were tested for normal distribution using the Kolmogorov-Smirnov test,
which revealed that the data was non-normally distributed. To assess driving performance after
control was handed back to the driver, a measure of the standard deviation of the absolute steering
angular rate was used to capture corrective steering actions (Fisher et al., 2011 ch 40, pp 10).
Furthermore, as the TOrt data is reaction time data, the median TOrt values for each participant was
calculated after which Wilcoxon signed-rank test was used to analyse the time and workload data.
The box plots in Figure 6 and Figure 8 had their outlier thresholds adjusted to accommodate the log-
normal distribution of the TOrt data by using the LIBRA library for MatLab (Verboven & Hubert,
2005) and its method for robust boxplots for non-normally distributed data by Hubert and
Vandervieren (2008). Effect sizes were calculated as: r = abs(Z/√N).
Results
The results showed that it took approximately 4.2-4.4±1.96-1.80 seconds (median) to switch to
automated driving, see Table 2. No significant differences between the two conditions could be
found when drivers transitioned from manual to automated driving (Z = -0.673, p = 0.5, r = 0.13).
Control transition times from manual to automated driving in the two conditions is shown in Figure 6
and individual transition times for each participant are available in the supplementary material.
Figure 6. Adjusted box-plot of control transition times from manual driving to Automated Driving.
The dashed horizontal line indicates the max/min values assuming a normal distribution.
Figure 7. A distribution plot of TOrt when drivers were prompted to engage the automation. The
asterisk* marks the median value, the X axis contains 160 bins.
The results showed a significant increase in control transition time of ~1.5 seconds when drivers
were prompted to resume control whilst engaged in a secondary task (Z = -4.43, p < 0.01, r = 0.86). It
took drivers approximately 4.46±1.63 seconds to resume control when not occupied by a secondary
task, and 6.06±2.39 seconds to resume control when engaged in a secondary task as shown in Table
2.
Table 2. Descriptive statistics of the control transition times (in milliseconds) from Automated
Driving to manual control, and from manual control to Automated Driving as well as descriptive
statistics from the presented TOrt’s from the reviewed articles.
Meta-review
From Automated to Manual
From Manual to Automated
No secondary task
Secondary task
No secondary task
Secondary task
Median
2470 ms
4567 ms
6061 ms
4200 ms
4408 ms
IQR
1415 ms
1632 ms
2393 ms
1964 ms
1800 ms
Min
1140 ms
1975 ms
3179 ms
2822 ms
2926 ms
Max
15000 ms
25750 ms
20994 ms
23884 ms
23221 ms
Figure 8. Adjusted boxplot of the Take Over reaction time when switching from automated to
manual control in the two experimental conditions contrasted with the TOrt of the reviewed
papers.
Figure 9 A distribution plot of TOrt when drivers were prompted to resume manual control. The
asterisk* marks the median value, the X axis contains 160 bins. The amplitude of the reviewed
papers is caused by the low number of values provided by the reviewed papers.
The results from analysing the driving performance data from 0 to 18 seconds post control transition
showed non-significant differences in Standard Deviation of Absolute Steering Angular Rate (Table 3)
between the two task conditions.
Table 3. Standard deviation of Angular Rate (Degrees/second) for the two task conditions from 0-
18 seconds post take-over.
No secondary task
With secondary task
time
M (SD)
M (SD)
Z
p
r
0-3s
0.08 (0.06)
0.08 (0.06)
-0.11
0.91
0.02
3-6s
0.03 (0.03)
0.03 (0.03)
-0.14
0.89
0.03
6-9s
0.02 (0.02)
0.06 (0.17)
-1.48
0.13
0.29
9-12s
0.02 (0.02)
0.11 (0.44)
-0.98
0.33
0.19
12-15s
0.02 (0.02)
0.11 (0.45)
-0.39
0.69
0.08
15-18s
0.03 (0.07)
0.13 (0.51)
-0.42
0.67
0.08
The analysis of the subjective ratings for driver mental workload showed that the secondary task
condition has marginally higher scores overall, as shown in Table 4. Only temporal demand had a
statistically significant difference (Z= -3.11, p < 0.05, r = 0.61), with higher rated demand in the
secondary task condition as shown in Figure 10.
Table 4. Overall workload scores as well as individual workload ratings for the two conditions. ** =
Significant at the 0.01 level.
Without secondary
task
With secondary task
Variable
Median (IQR)
Median (IQR)
Z
p
r
Overall Workload
5 (7.33)
6.2 (6.5)
-1.953
0.051
0.38
Mental demand
7.5 (10)
10.5 (9)
-1.41
0.16
0.28
Physical demand
4 (5)
5.5 (6)
-1.93
0.054
0.38
Temporal demand
3 (6)
8.5 (8)
-3.11
0.00**
0.61
Performance
6 (7)
4.5 (5)
-0.47
0.63
0.09
Effort
5 (9)
7 (8)
-0.23
0.82
0.05
Frustration
4 (9)
6.5 (7)
-1.04
0.3
0.2
Figure 10. Boxplot of Subjective estimations of workload in the two conditions.
Discussion
Relinquishing control to automation
In this study we subjected drivers to multiple control transitions between manual and automated
control in a highway scenario. Upon reviewing the literature, no mention of how long the driver
takes to engage an automated driving system was found, making this study a first-of-a-kind. We
found that drivers take between 2.82-23.8 seconds (Median = 4.2-4.4) to engage automated driving
when the system indicates that the feature is available. No significant differences between the two
conditions was found, but as Figure 6 shows, there was large range in the time it takes to relinquish
control. It is clear from Figure 7 that designing for the median, or average driver effectively exclude a
large part of the user group, which could have severe implications for drivers who fall outside of the
mean or median. It has been common practice in Human Factors and Anthropometrics to design for
90% of the population, normally through accommodating the range between the 5th percentile
female and the 95th percentile male (Porter et al., 2004). Thus, it is important that vehicle
manufacturers are made aware of the intra-individual differences, as such differences have a large
effect on the larger traffic system if drivers are expected to toggle Automated Driving systems within
a certain time frame. An example of potential situations where the driver would need to toggle the
automated driving system could be in HAD-dedicated areas. Moreover, it may be that the time it
takes to engage the automated driving system depends on external factors such as, perceived safety,
weather conditions, traffic flow rates, presence of vulnerable road users, roadworks, and so on. If
the driver deems a situation unsafe, or has doubts as to how well the automation would perform in
a situation, the driver may hold off on completing a transition until the driver feels that the system
can comfortably handle the situation.
Resuming control from automation
Previous research was reviewed and it was found that most studies utilised system paced
transitions, where the automated driving system warns in advance of failure or reduced automation
support with relatively short lead times, from 3 seconds [studies 2, 13, 14, 18, 22] to 7 seconds
[studies 1, 6, 9, 17, 19, 25]. It has previously been shown that whilst it takes approximately
2.47±1.42 seconds on average, it can take up to 15 seconds to respond to such an event (Merat et
al., 2014). We argue that this use case, albeit important, does not reflect the primary use case for
control transitions in highly automated driving. When comparing the range of TOrt in the literature
to the user paced (no secondary task) condition in this study, a great deal of overlap can be seen.
The observed values in the current study are closer to the higher values observed by Merat et al.
(2014) whilst the median range of 4.56-6.06s is closer to the range of times suggested by Gold et al.
(2013) and Damböck et al. (2012). It is evident that there is a large spread in the TOrt, which when
designing driving automation should be considered, as the range of performance is more important
than the median or mean, as these exclude a large portion of drivers.
When subjecting drivers to TOR’s without time restrictions we found that drivers take between 1.97-
25.75 seconds (Median = 4.56) to resume control from automated driving in normal conditions, and
between 3.17-20.99 seconds (Median = 6.06) to do so whilst engaged in a secondary task preceding
the control transition. This shows that there is a median 2 second difference in control transition
times in the reviewed manuscripts compared to the user paced control transitions. There was a large
effect of secondary task engagement on TOrt, showing an increase in driver control resumption
times when engaged in a secondary task. This might be explained in part by the nature of the
secondary task, as the driver had to allocate time to put down the magazine they were asked to read
whilst the automated driving feature was activated. It could also be partly attributed to driver task
adaptation, by holding off transferring control until they have had time to switch between the
reading task and driving task. This is supported by research indicating that drivers tend to adapt to
external factors such as traffic complexity to allow for more time to make decisions (Eriksson et al.,
2014), by for example slowing down when engaged in secondary tasks (Cooper et al., 2009) or the
expectation of resuming control (Young & Stanton, 2007). In light of these results, there is a case for
“adaptive automation” that modulates TORlt by, for example, detecting whether the driver gaze is
off road for a certain time-period, providing the driver with a few additional seconds before
resuming control.
Furthermore, the 1.5 second increase of control resumption time when engaged in a reading task is
similar to the reaction time increase caused by the introduction of automated driving features
observed by Young and Stanton (2007) compared to reaction time in manual driving (Summala,
2000). Therefore, a further increase of reaction times when drivers are engaged in other tasks will
have to be expected, but measures must be taken to reduce the increase in reaction time, for
example through the addition of informative displays, to reduce the risk of accidents (Cranor, 2008;
Eriksson & Stanton, 2016).
There was a significant increase in perceived temporal demand when drivers were tasked with
reading whilst the automation was engaged. This increase in perceived temporal demand may have
been caused by the TOR, and the driver not being fully sure as to how long the vehicle could manage
before a forced transition would occur (this was not a possibility in the current experiment). This
increase in perceived temporal demand could also be attributed to the pace of the experiment, and
that the drivers were required to pick up, and put down the magazine whenever a control transition
was issued. Overall there are little differences in workload, and the median workload in both
conditions was approximately at the halfway point on the scale, implying optimal loading (Stanton et
al., 2011).
In light of these results, combined with the non-significant, small effect of task condition on driving
performance indicated that the drivers were able to self-regulate the control transition process by
adapting the time needed to resume control (Eriksson et al., 2014; Kircher et al., 2014) and therefore
maintaining optimal levels of workload, minimizing the severity of the after-effects observed in
studies by e.g. Gold et al. (2013).
Conclusions
Relinquishing control to automation
The literature on control transitions in highly automated driving is absent in research reports on
transitions from manual to automated vehicle control. In a first-of-a-kind study, we found that it
takes drivers between 2.8 and 23.8 seconds to switch from manual to automated control. This
finding has some implications for the safety of drivers merging into automated driving dedicated
lanes, or other infrastructure whilst in manual mode. Such an event may require certain adaptations
for traffic already occupying such a lane. Adaptations may include increasing time-headway or
reducing speed to accommodate the natural variance in human behaviour to avoid collisions, or
discomfort for road users in such a lane. Moreover, it may be that part of the variance could be
reduced by designing merging zones on straight, uncomplicated road sections as drivers may
otherwise hold off transferring control to the automated driving system until the driver feels it safe
to hand control to the automated driving system.
Resuming control from automation
A review of the literature found that most papers tend to report the mean Take Over reaction time
and often fail to report standard deviation and range (c.f. Figure 2), thus the variance in control
transition times remain unknown (median, and interquartile range for each participant in this study
can be found in the supplementary material). Additionally, the reviewed papers tended to give
drivers a lead time of between 0 and 30 seconds between the presentation of a TOR and a critical
event with the main part of the reviewed papers using a 3 or 7 second lead time. In this study we
found that the range of time in which drivers resume control from the automated driving system
was between 1.9 and 25.7 seconds depending on task engagement. The spread of TOrt in the two
conditions in this study indicates that mean, or median values do not tell the entire story when it
comes to control transitions. Notably, the distribution of TOrt approaches platykurtic (c.f. Figure 9)
when drivers are engaged in a secondary task. This implies that vehicle manufacturers must adapt to
the circumstances, providing more time to drivers who are engaged in secondary tasks, whilst in
highly automated driving mode to avoid excluding drivers at the tail of the distribution. In light of
this, designers of automated vehicles should not focus on the mean, or median, driver when it
comes to control transition times. Rather they should strive to include the larger range of control
transitions times so they do not exclude users that fall outside the mean or median. Moreover,
policy-makers should strive to accommodate these inter- and intra-individual differences in their
guidelines for “sufficiently comfortable transition times”. When drivers were allowed to self-
regulate the control transition process, little differences could be found in both driving performance
and workload between the two conditions. This lends further support to the argument for designing
for the range of transition-times rather than the mean or median in non-critical situations.
Lastly, based on the large decrease in TOrt kurtosis when drivers were engaged in a secondary task,
it may also be the case that future automated vehicles need to adapt the TORlt to account for
drivers engaged in other, non-driving, tasks and even adapt TORlt to accommodate external factors
such as traffic density and weather.
Key Points
Large differences between control transitions times reported in the literature and the no-
secondary task condition were found.
Drivers take longer to resume control from automation when engaged in a secondary task prior
to the control transition.
An inclusive design approach is needed to accommodate the observed variance as the mean or
median response times are not sufficient when it comes to designing control transitions in
automated driving.
References
Audi. (2014). Audi piloted driving. from
http://www.audi.com/com/brand/en/vorsprung_durch_technik/content/2014/10/piloted-
driving.html
Bainbridge, L. (1983). Ironies of Automation. Automatica, vol 19, no 6, pp 775-779.
Banks, V. A., & Stanton, N. A. (2015). Contrasting models of driver behaviour in emergencies using
retrospective verbalisations and network analysis. Ergonomics, vol 58, no 8, pp 1337-1346.
Banks, V. A., & Stanton, N. A. (2016). Keep the driver in control: Automating automobiles of the
future. Appl Ergon, vol 53 Pt B, pp 389-395.
Belderbos, C. (2015). Authority Transition Interface: A Human Machine Interface for Taking over
Control from a Highly Automated Truck. TU Delft, Delft University of Technology.
BMW. (2013). ConnectedDrive.
Byers, J. C., Bittner, A., & Hill, S. (1989). Traditional and raw task load index (TLX) correlations: Are
paired comparisons necessary. Advances in industrial ergonomics and safety I, pp 481-485.
Cooper, J. M., Vladisavljevic, I., Medeiros-Ward, N., Martin, P. T., & Strayer, D. L. (2009). An
investigation of driver distraction near the tipping point of traffic flow stability. Human
Factors: The Journal of the Human Factors and Ergonomics Society, vol 51, no 2, pp 261-268.
Cranor, L. F. (2008). A framework for reasoning about the human in the loop. Paper presented at the
Proceedings of the 1st Conference on Usability, Psychology, and Security.
Damböck, D., Bengler, K., Farid, M., & Tönert, L. (2012). Übernahmezeiten beim
hochautomatisierten Fahren. Tagung Fahrerassistenz. München, vol 15, pp 16.
Desmond, P. A., Hancock, P. A., & Monette, J. L. (1998). Fatigue and automation-induced
impairments in simulated driving performance. Human Performance, User Information, and
Highway Design, no 1628, pp 8-14.
Dogan, E., Deborne, R., Delhomme, P., Kemeny, A., & Jonville, P. (2014). Evaluating the shift of
control between driver and vehicle at high automation at low speed: The role of anticipation.
Paper presented at the Transport Research Arena (TRA) 5th Conference: Transport Solutions
from Research to Deployment.
Endsley, M. R., & Kaber, D. B. (1999). Level of automation effects on performance, situation
awareness and workload in a dynamic control task. Ergonomics, vol 42, no 3, pp 462-492.
Eriksson, A., Lindström, A., Seward, A., Seward, A., & Kircher, K. (2014). Can user-paced, menu-free
spoken language interfaces improve dual task handling while driving? Human-Computer
Interaction. Advanced Interaction Modalities and Techniques (pp. 394-405): Springer.
Eriksson, A., Spychala, A., F., D. W. J. C., & Stanton, N. A. (2016). A Highly Automated Driving
Algorithm Implementation on STISIM. Paper presented at the AHFE 2016.
Eriksson, A., & Stanton, N. A. (2016). The Chatty Co-Driver: A Linguistics Approach to Human-
Automation-Interaction. Paper presented at the IEHF2016.
Feldhütter, A., Gold, C., Schneider, S., & Bengler, K. (2016). How the Duration of Automated Driving
Influences Take-Over Performance and Gaze Behavior. Paper presented at the Arbeit in
komplexen Systemen - Digital, vernetz, human?! 62. Kongress der Gesellschaft für
Arbeitswissenschaft.
Fisher, D. L., Rizzo, M., Caird, J., & Lee, J. D. (2011). Handbook of driving simulation for engineering,
medicine, and psychology: CRC Press.
Gold, C., Damböck, D., Lorenz, L., & Bengler, K. (2013). “Take over!” How long does it take to get the
driver back into the loop? Paper presented at the Proceedings of the Human Factors and
Ergonomics Society Annual Meeting.
Gold, C., Korber, M., Lechner, D., & Bengler, K. (2016). Taking Over Control From Highly Automated
Vehicles in Complex Traffic Situations: The Role of Traffic Density. Hum Factors.
Gold, C., Lorenz, L., & Bengler, K. (2014). Influence of automated brake application on take-over
situations in highly automated driving scenarios. Paper presented at the Proceedings of the
FISITA 2014 World Automotive Congress.(in print).
Hart, S. G., & Staveland, L. E. (1988). Development of NASA-TLX (Task Load Index): Results of
empirical and theoretical research. Advances in psychology, vol 52, pp 139-183.
Hubert, M., & Vandervieren, E. (2008). An adjusted boxplot for skewed distributions. Computational
statistics & data analysis, vol 52, no 12, pp 5186-5201.
Kaber, D. B., & Endsley, M. R. (1997). Out-of-the-loop performance problems and the use of
intermediate levels of automation for improved control system functioning and safety.
Process Safety Progress, vol 16, no 3, pp 126-131.
Kerschbaum, P., Lorenz, L., & Bengler, K. (2015). A transforming steering wheel for highly automated
cars. Paper presented at the Intelligent Vehicles Symposium (IV), 2015 IEEE.
Kircher, K., Larsson, A., & Hultgren, J. A. (2014). Tactical driving behavior with different levels of
automation. IEEE Transactions on Intelligent Transportation Systems, vol 15, pp 158-167.
Körber, M., Gold, C., Lechner, D., & Bengler, K. (2016). The influence of age on the take-over of
vehicle control in highly automated driving. Transportation Research Part F: Traffic
Psychology and Behaviour, vol 39, pp 19-32.
Körber, M., Weißgerber, T., Kalb, L., Blaschke, C., & Farid, M. (2015). Prediction of take-over time in
highly automated driving by two psychometric tests. Dyna, vol 82, no 193, pp 195-201.
Lorenz, L., Kerschbaum, P., & Schumann, J. (2014). Designing take over scenarios for automated
driving How does augmented reality support the driver to get back into the loop? Paper
presented at the Proceedings of the Human Factors and Ergonomics Society Annual
Meeting.
Louw, T., Kountouriotis, G., Carsten, O., & Merat, N. (2015a). Driver Inattention During Vehicle
Automation: How Does Driver Engagement Affect Resumption Of Control?
Louw, T., Merat, N., & Jamson, H. (2015b). Engaging with Highly Automated Driving: To be or Not to
be in the Loop? Paper presented at the 8th International Driving Symposium on Human
Factors in Driver Assessment, Training and Vehicle Design, At Salt Lake City, Utah, USA.
Lu, Z., & de Winter, J. C. F. (2015). A Review and Framework of Control Authority Transitions in
Automated Driving. Procedia Manufacturing, vol 3, pp 2510-2517.
Melcher, V., Rauh, S., Diederichs, F., Widlroither, H., & Bauer, W. (2015). Take-Over Requests for
Automated Driving. Procedia Manufacturing, vol 3, pp 2867-2873.
Merat, N., Jamson, A. H., Lai, F. C., & Carsten, O. (2012). Highly automated driving, secondary task
performance, and driver state. Human Factors: The Journal of the Human Factors and
Ergonomics Society, vol 54, no 5, pp 762-771.
Merat, N., Jamson, A. H., Lai, F. F. C. H., Daly, M., & Carsten, O. M. J. (2014). Transition to manual:
Driver behaviour when resuming control from a highly automated vehicle. Transportation
Research Part F: Traffic Psychology and Behaviour, vol 26, Part A, pp 1-9.
Michon, J. A. (1985). A critical view of driver behavior models: What do we know, what should we
do? In L. Evans & R. C. Schwing (Eds.), Human behavior and traffic safety. New York: Plenum
Press.
Mok, B., Johns, M., Lee, K. J., Miller, D., Sirkin, D., Ive, P., et al. (2015). Emergency, Automation Off:
Unstructured Transition Timing for Distracted Drivers of Automated Vehicles. Paper
presented at the Intelligent Transportation Systems (ITSC), 2015 IEEE 18th International
Conference on.
Molloy, R., & Parasuraman, R. (1996). Monitoring an automated system for a single failure: Vigilance
and task complexity effects. Human Factors, vol 38, no 2, pp 311-322.
Naujoks, F., Mai, C., & Nekum, A. (2014, 19-23 July). The effect of urgency of take-over requests
during highly automated driving under distracted conditions. Paper presented at the 5th
International Conference on Applied Human Factors and Ergonomics AHFE 2014, Krakow,
Poland.
Naujoks, F., & Nekum, A. (2014). Timing of in-vehicle advisory warnings based on cooperative
perception. Paper presented at the Human Factors and Ergonomics Society Europe Chapter
Annual Meeting.
Nilsson, J. (2014). Safe Transitions to Manual Driving From Faulty Automated Driving System:
Chalmers University of Technology.
Payre, W., Cestac, J., & Delhomme, P. (2016). Fully Automated Driving: Impact of Trust and Practice
on Manual Control Recovery. Hum Factors, vol 58, no 2, pp 229-241.
Petermeijer, S. M., de Winter, J. C. F., & Bengler, K. J. (2016). Vibrotactile Displays: A Survey With a
View on Highly Automated Driving. IEEE Transactions on Intelligent Transportation Systems,
vol 17, no 4, pp 897-907.
Porter, J. M., Case, K., Marshall, R., Gyi, D., & Sims neé Oliver, R. (2004). ‘Beyond Jack and Jill’:
designing for individuals using HADRIAN. International Journal of Industrial Ergonomics, vol
33, no 3, pp 249-264.
Radlmayr, J., Gold, C., Lorenz, L., Farid, M., & Bengler, K. (2014). How Traffic Situations and Non-
Driving Related Tasks Affect the Take-Over Quality in Highly Automated Driving. Proceedings
of the Human Factors and Ergonomics Society Annual Meeting, vol 58, no 1, pp 2063-2067.
Robinson, T., Chan, E., & Coelingh, E. (2010). Operating platoons on public motorways: An
introduction to the sartre platooning programme. Paper presented at the 17th world
congress on intelligent transport systems.
SAE International. (2016). Taxonomy and Definitions for Terms Related to Driving Automation
Systems for On-Road Motor Vehicles, J3016_201609: SAE International.
safecarnews.com. (2015). Intelligent Drive Concept for new Mercedes-Benz GLC. from
http://safecarnews.com/intelligent-drive-concept-for-new-mercedes-benz-glc_ju6145/
Schömig, N., Hargutt, V., Neukum, A., Petermann-Stock, I., & Othersen, I. (2015). The interaction
between highly automated driving and the development of drowsiness.
Scott, J., & Gray, R. (2008). A comparison of tactile, visual, and auditory warnings for rear-end
collision prevention in simulated driving. Human Factors: The Journal of the Human Factors
and Ergonomics Society, vol 50, no 2, pp 264-275.
Stanton, N. A., Dunoyer, A., & Leatherland, A. (2011). Detection of new in-path targets by drivers
using Stop & Go Adaptive Cruise Control. Appl Ergon, vol 42, no 4, pp 592-601.
Stanton, N. A., & Marsden, P. (1996). From fly-by-wire to drive-by-wire: Safety implications of
automation in vehicles. Safety Science, vol 24, no 1, pp 35-49.
Stanton, N. A., Young, M., & McCaulder, B. (1997). Drive-by-wire: The case of driver workload and
reclaiming control with adaptive cruise control. Safety Science, vol 27, no 2-3, pp 149-159.
Stanton, N. A., & Young, M. S. (2005). Driver behaviour with adaptive cruise control. Ergonomics, vol
48, no 10, pp 1294-1313.
Stanton, N. A., Young, M. S., Walker, G. H., Turner, H., & Randle, S. (2001). Automating the Driver’s
Control Tasks. INTERNATIONAL JOURNAL OF COGNITIVE ERGONOMICS.
Strand, N., Nilsson, J., Karlsson, I. C. M., & Nilsson, L. (2014). Semi-automated versus highly
automated driving in critical situations caused by automation failures. Transportation
Research Part F: Traffic Psychology and Behaviour.
Summala, H. (2000). Brake Reaction Times and Driver Behavior Analysis. Transportation Human
Factors, vol 2, no 3, pp 217-226.
Swaroop, D., & Rajagopal, K. R. (2001). A review of Constant Time Headway Policy for Automatic
Vehicle Following. Paper presented at the IEEE Intelligent Transportation Systems
Conference Proceedings, Oakland (CA), USA.
United Nations. (1968). Convention on road traffic, Done at Vienna on 8 November 1968.
Amendment 1. Retrieved from
http://www.unece.org/fileadmin/DAM/trans/conventn/crt1968e.pdf.
van den Beukel, A. P., & van der Voort, M. C. (2013). The influence of time-criticality on Situation
Awareness when retrieving human control after automated driving. Paper presented at the
Intelligent Transportation Systems-(ITSC), 2013 16th International IEEE Conference on.
Verboven, S., & Hubert, M. (2005). LIBRA: a MATLAB library for robust analysis. Chemometrics and
intelligent laboratory systems, vol 75, no 2, pp 127-136.
Walch, M., Lange, K., Baumann, M., & Weber, M. (2015). Autonomous driving: investigating the
feasibility of car-driver handover assistance. Paper presented at the Proceedings of the 7th
International Conference on Automotive User Interfaces and Interactive Vehicular
Applications.
Walker, G. H., Stanton, N. A., & Salmon, P. M. (2015). Human Factors in Automotive Engineering and
Technology: Ashgate: Aldershot.
Willemsen, D., Stuiver, A., & Hogema, J. (2015). Automated Driving Functions Giving Control Back to
the Driver: A Simulator Study on Driver State Dependent Strategies. Paper presented at the
24th International Technical Conference on the Enhanced Safety of Vehicles (ESV).
Wolterink, W. K., Heijenk, G., & Karagiannis, G. (2011). Automated Merging in a Cooperative
Adaptive Cruise Control (CACC) System. Paper presented at the Fifth ERCIM Workshop on
eMobility, Vilanova i la Geltrú, Catalonia, Spain.
Young, M. S., & Stanton, N. A. (2002). Malleable attentional resources theory: a new explanation for
the effects of mental underload on performance. Hum Factors, vol 44, no 3, pp 365-375.
Young, M. S., & Stanton, N. A. (2007). Back to the future: brake reaction times for manual and
automated vehicles. Ergonomics, vol 50, no 1, pp 46-58.
Zeeb, K., Buchner, A., & Schrauf, M. (2015). What determines the take-over time? An integrated
model approach of driver take-over after automated driving. Accident Analysis & Prevention,
vol 78, pp 212-221.
Zeeb, K., Buchner, A., & Schrauf, M. (2016). Is take-over time all that matters? The impact of visual-
cognitive load on driver take-over quality after conditionally automated driving. Accid Anal
Prev, vol 92, pp 230-239.
... The authors concluded that reduced road glances result in increased TOT and individual reaction times affect TOT. Similarly, in [22], the authors concluded that TOT varies with different drivers who have different reaction times and levels of driving experience. Accordingly, [10] concluded that TOT varies with scenario complexity, prior driving experience with automated driving technologies, and skill. ...
... In addition, from the new results in this contribution and compared to existing results from other studies [2], [12], [18], [21], [22], [29], the variables for budgeting TOR time online considering the defined conditions in the studied scenarios are summarized in Fig. 15 as an extension of Fig. 2. Specifically, more time should be budgeted for the FIGURE 15: Variables for TOR time budget for scenarios first drive of every scenario irrespective of whether the driver is familiar with the process of switching back to manual driving. In other words, the time related to the ordinal should be subtracted from the TOR time budget in a second or subsequent experience of a scenario. ...
... More so, individual driving experience [km/wk] defined and utilized in this contribution covaries with TOT which and should be integrated into the TOR time budget. Combining driving experience with individual stimulus response time concluded from previous studies [21], [22], constitute individual reaction time. Altogether, the TOR time budget can therefore also be expressed as a combination of the mean and standard deviation of TOT Therefore, substituting the mean and standard deviation of TOT values from Table 4 in (3), the MRT for scenarios S1|II, S3|II, and S4|II can be calculated as in (4), (5), and (6). ...
Article
Full-text available
An automated driving system notifies a fallback-ready human driver to resume driving when critical operational and functional limits have been or are about to be exceeded. The point between notification and critical limit is the time budget. Previous studies indicate that interdependencies exist between takeover variables and the time budget leading to performance variations and sometimes accidents. It is known that drivers may delay or respond inadequately depending on the time budget. This contribution focuses on utilizing these interdependencies to evaluate the suitability of the time budget for specific scenarios. A 7 s time budget, eight scenarios, and three secondary tasks were studied in a driving simulator with 70 participants aged between 19 Yrs and 41 Yrs. The results indicate that drivers prioritize takeover effort in decreasing order of relative speed, traffic agents, and junctions. Furthermore, 7 s is suitable at a vehicle speed of 80 Km/h to 130 km/h, maximum two traffic agents and three junctions, and handsfree tasks but too high for lower complexities. Generally, the time budget is a sum of the takeover time and maneuver (e.g., lane change) response time. These results are relevant to safety and adaptive variation of the time budget for successful takeover.
... One possible reason can be that the driver is required to follow a process of adaptation in this transition from a state of low situational awareness to a higher one (Morales-Alvarez, Sipele, Léberon, Tadjine, & Olaverri-Monreal, 2020; Russell et al., 2016). Thus, drivers' ability to safely resume manual control is one of the primary safety concerns in automated driving (Cayeux et al., 2021;Eriksson & Stanton, 2017). ...
Article
Full-text available
The emergence of automated vehicles (AV) has been occurring rapidly as these vehicles have the potential to reduce/eradicate human driving faults and related collisions. To enhance AV safety, the NHTSA recommends the continuous presence of a backup human driver that has a reasonable understanding of AV technologies to ensure disengagement when manual overtake is required. However, due to several AV-related traffic crashes during roadway testing and extensive media interest, AV safety has become a critical issue. This study collected 255 crash reports filed by different manufacturers testing AVs in California from September 2014 to April 2020. The crash dataset was analyzed using two data mining algorithms (association rule mining and text network analysis) to identify the key AV-related crash attributes and their associations based on the vehicle's prior driving mode (conventional or automated). The results show that the manner of collision and the prior movement of the testing vehicle are strongly connected with prior driving mode. For example, AV crashes in a manual driving mode often result in a sideswipe collision during moving status, whereas AV crashes in an automated driving mode are highly associated with rear-end collisions when AVs are stopped in traffic. The findings of this study can help policymakers and AV engineers improve AV deployment strategies to support the adoption of AVs and promote potential safety benefits for AV technologies.
... Among all control transitions in highway driving, we selected non-critical, driver-initiated AP disengagements to manual control (henceforth referred as AP disengagement; step C in Figure 1). We investigated this type of disengagements-in contrast to many studies that focused on critical, system-initiated transitions-because they are the most common and they likely provide a reference on aspects of normative driver behavior with AP [see also 11]. The AP disengagements were selected according to the following criteria: (i) AP was used for at least 30 s prior the disengagement without being overridden by accelerating or steering; (ii) There were at least 10 s of manual driving after disengagement; (iii) Road class was highway; (iv) Speed was above 30 mph; (v) Longitudinal acceleration was less than -3 m/s 2 ; (vi) no advanced driver assistance system (ADAS) warnings were issued (e.g., forward collision warning). ...
... The time budget for the takeover to be addressed by the method should be at least below 10 s (most literature on takeover reaction times in L3 assume time budgets between 3 and 7 s; see [5] ). Furthermore, the method should be also able to answer Level 2 (L2) system-related research questions with regard to HMI. ...
Article
Full-text available
The presented method describes a standardized test procedure for the evaluation of takeover performance of drivers during automated driving. It was primarily developed to be used for evaluating Level 3 systems (conditional automated driving). It should be applied in a driving simulator environment during the development phase of a system. The method consists of a test course on a three-lane highway with 12 test scenarios where the driver repeatedly has to take over control from the automated system. Each scenario requires the driver to build up an adequate situation awareness in order to take the decision for the correct action. The method was explicitly designed to map the four relevant steps in the takeover process of automated driving: Perception – Awareness – Decision – Action and is therefore called PADA-AD Test for automated driving. The method description contains guidelines with regard to the specification of the test course and the included test scenarios, the design and duration of the automated drives, the non-driving related task to be performed during the automated drives, the instructions to be given to the subjects and finally the measures for evaluating takeover performance of the drivers. • A test procedure for the evaluation of takeover performance of drivers during automated driving was developed for usage in a driving simulator during the development phase of a system/HMI • The test course enables the assessment of the driver's takeover performance in various test scenarios including higher cognitive processes • The method is highly standardized and thus replicable through use of a predetermined test course with clearly defined scenarios, reduced environmental conditions and "popping up” of situational elements
... For the non-critical scenarios, the literature on the takeover issue of L3 automation vehicles is limited [8]. Eriksson and Stanton [14] performed a takeover experiment for non-critical scenarios without time pressure, suggesting that designers of automated vehicles should take a wider range of takeover times into consideration rather than just focusing on the mean or median values. Pampel et al. [7] investigated the impact of unplanned and planned TOR on the takeover quality in non-critical situations by a simulator experiment. ...
Article
Full-text available
The objective of this study is to examine the effects of visibility and time headway on the takeover performance in L3 automated driving. Both non-critical and critical driving scenarios were considered by changing the acceleration value of the leading vehicle. A driving simulator experiment with 18 driving scenarios was conducted and 30 participants complete the experiment. Based on the data obtained from the experiment, the takeover reaction time, takeover control time, and takeover responses were analyzed. The minimum Time-To-Collision (Min TTC) was used to measure the takeover risk level and a binary logit model for takeover risk levels was estimated. The results indicate that the visibility distance (VD) has no significant effects on the takeover control time, while the time headway (THW) and the acceleration of the leading vehicle (ALV) could affect the takeover control time significantly; most of the participants would push the gas pedal to accelerate the ego vehicle as the takeover response under non-critical scenarios, while braking was the dominant takeover response for participants in critical driving scenarios; decreasing the TCT and taking the appropriate takeover response would reduce the takeover risk significantly, so it is suggested that the automation system should provide the driver with the urgency of the situation ahead and the tips for takeover responses by audio prompts or the head-up display. This study is expected to facilitate the overall understanding of the effects of visibility and time headway on the takeover performance in conditionally automated driving.
... Takeover times in non-critical handover situations are reviewed in [14]. Under noncritical conditions, drivers needed 1.9 to 25.7 seconds to take back control. ...
Article
Self-driving technologies introduced new challenges to the control engineering community. Autonomous vehicles with limited automation capabilities require constant human supervision, and human drivers have to be able to take back control at any time, which is called handover. This is a critical process in terms of safety, thus appropriate handover modeling is fundamental in design, simulation and education related to self-driving cars. This article reviews the literature of handover processes, situation awareness and control-oriented human driver models. It unifies the psychological and physiological control theory models to create a parameterized engineering tool to quantify the handover processes.
Article
With the development of automated driving systems and V2I (vehicle-to-infrastructure) communications, soft-safety driver alerts can be implemented to supplement imminent driver alerts. This type of alert improves drivers’ situational awareness of emerging risks over a more extended period with more detailed incident information and longer response time. However, compared to the large number of studies focusing on imminent risks, there are insufficient studies on the evaluation and effectiveness of soft-safety alerts. This study proposed an innovative metric to assess the comfort and safety of soft-safety driver alerts by constructing an ideal speed profile and calculating the deviation between the actual and ideal profile. We select the highway end-of-queue event as the experiment scenario, which is a leading cause of fatal highway crashes. Human subjects’ experiments are conducted in the driving simulator to validate the proposed metrics. The results have proved that the proposed metrics have good potential to assess driving comfort objectively. We also found that soft-safety alerts tend to improve driving comfort. However, there is insufficient evidence to conclude statistically about the prototype soft-safety alerts implemented in the experiments.
Chapter
Autonomous vehicles (AVs) are expected to play an increasingly important role in future transportation systems as a promising means of improving road safety and efficiency by eventually replacing human-driven vehicles. Semi-autonomous vehicles (semi-AVs; SAE Level 2 and Level 3) feature automatic lateral and longitudinal control of the vehicle with human drivers required to supervise the system at all times (Level 2) or prepared to resume control when requested (Level 3). As these definitions reveal, semi-AVs still require human oversight and intervention to fully ensure safety. Humans are required to monitor and be ready to take over control when the vehicle fails to recognize or respond to hazardous events. Thus, it is essential to ensure effective human-automation interaction and collaboration for semi-AVs. This book chapter will discuss the critical challenges for effective human-automation interaction for semi-autonomous driving, including communicating potential risks to human drivers and maintaining proper driver trust in the semi-AV. Risks in the current context are moving or stationary objects and road environments that impose imminent threats to drivers, including overt hazards such as road obstacles, a pedestrian crossing the road, and an intruding vehicle, or covert hazards such as a pedestrian that is about to cross but is occluded by a parked truck or a roadway structure. We discuss the design of effective risk communication mechanisms to convey these risks to the human driver, which helps maintain the driver’s situation awareness and facilitate the driver’s actions when needed. In addition, the effectiveness of this risk communication can be influenced not only by the characteristics of the driver and the semi-AV, but also their interaction. Finally, we will discuss factors that affect drivers’ trust in semi-AVs and subsequently how it affects effective risk communication in semi-AV driving.KeywordsSemi-autonomous drivingTrustRisk communication
Article
Background Meta studies on factors contributing to take-over performance did not include the design of take-over request (TOR) signals, other than the modality at which TORs are presented. A detailed understanding of the influence of TOR design on take-over performance is therefore lacking. Objective To gain an overview of the level of detail with which TOR designs are reported in academic literature, by using and evaluating a novel classification framework. In this framework TORs are classified in terms of modalities, classes, and underlying attributes. Furthermore, the framework involves classification of potentially competing background signals, as well as the setting in which a study is performed. Method A systematic review was performed on articles written in English that were published between 2014 and 2021 using Web of Science, as well as articles retrieved from two previous TOR classification studies and three meta studies on take-over performance. Studies were considered for subsequent analysis if they involved a downward transition of the level of automation following a TOR, resulting in a sample of 391 TORs found in 189 studies. Results No predominant TOR design was found, and a considerable part of the available design space has not yet been explored. Studies reported less information on TOR designs when examining TOR designs at an increased level of detail. On average, attribute information was reported for half of the TORs per class. Conclusions More attention towards a detailed description of TOR implementations is needed and how this can impact experimental findings. The classification framework and the corresponding coding sheet could support systematic reporting and subsequent meta-analysis in future work. This way, a better understanding about the impact of TOR design on take-over performance can be gained, which in turn can support implementation of safe and effective TORs in (automated) vehicles.
Conference Paper
Full-text available
In future automated driving systems, drivers will be free to perform other secondary tasks, not needing to stay vigilant in monitoring the car’s activity. However, there will still be situations in which drivers are required to take-over control of the vehicle, most likely from a highly distracted state. While highly automated vehicles would ideally accommodate structured takeovers, providing ample time and warning, it is still very important to examine how drivers would behave when they are subjected to an unstructured emergency transition of control. In this study, we observed how participants (N=30) in a driving simulator performed after they experienced a loss of automation. We tested three transition time conditions, with an unstructured transition of control occurring 2 seconds, 5 seconds, or 8 seconds before the participants encountered a road hazard that required the drivers’ intervention. Participants were given a passive distraction (watching a video) to do while the automated driving mode was enabled, so they needed to disengage from the task and regain control of the car when the transition occurred. Few drivers in the 2 second condition were able to safely negotiate the road hazard situation, while the majority of drivers in the 5 or 8 second conditions were able to navigate the hazard safely. Similarly, drivers in the 2 second condition rated the vehicle to be less trustworthy than drivers in the 5 and 8 second conditions. From the study results, we are able to narrow down a minimum amount of time in which drivers can take over the control of vehicle safely and comfortably from the automated system in the advent of an impending road hazard.
Conference Paper
Full-text available
The takeover of the driving task in highly automated vehicles at system limits is subject to latest research in ergonomics and human-machine-interaction. Most studies focus on driving simulator studies, examining the takeover performance mainly after short periods of automated driving, although takeover requests may not occur such frequently in future automated vehicles. This study tries to close this gap and compares driving performance and reaction times of a takeover after 5 and 20 minutes of automated driving. Further, the gaze behavior in the beginning and end of the 20 minutes period is compared. While the duration of automated driving did not show to influence the takeover performance, gaze behavior changed within the 20 minutes of automated driving. The SuRT and the 20 minutes automation period induced slower reactions, but no significant changes regarding accelerations and time to collision.
Book
Offering a unique perspective on vehicle design and on new developments in vehicle technology, this book seeks to bridge the gap between engineers, who design and build cars, and human factors, as a body of knowledge with considerable value in this domain. The work that forms the basis of the book represents more than 40 years of experience by the authors. Human Factors in Automotive Engineering and Technology imparts the authors' scientific background in human factors by way of actionable design guidance, combined with a set of case studies highly relevant to current technological challenges in vehicle design. The book presents a novel and accessible insight into a body of knowledge that will enable students, professionals and engineers to add significant value to their work.
Conference Paper
In the near future, highly automated driving will almost certainly be available in commercial vehicles. Concerning the human-machine interface in such cars, two main issues have to be addressed. First, the detrimental effects of automation have to be avoided. Second, cars should provide an interface that allows the driver to utilize the time while driving highly automated. We conducted a driving simulator study to investigate the concept of geometrical transformation of the steering wheel to address both issues. Therefore, we implemented a prototype steering wheel which changes its shape depending on the current driving mode to improve mode awareness and comfort when driving highly automated. The study was focused on possible negative effects of the mechanical transformation in front of the driver during the take-over process. Results indicate that on average participants reacted faster and took over control later. The number of lane change errors, for example changing lanes without looking into the mirror, even somewhat decreased when using the transforming steering wheel. Furthermore, participants mainly rated the proposed concept as usable without problems during the take-over process.
Article
A sufficient take-over time is essential to ensure the controllability of highly automated vehicles. Because this time budget is limited by factors like the sensor range, this paper verifies in a driving simulator experiment whether an automated brake application which lengthens the time budget by reducing vehicles’ speed is suitable for improving the controllability of highly automated vehicles. In the dynamic driving simulator of the BMW Group Research and Technology, which is equipped with a full vehicle mock-up, 48 subjects experienced four different take-over scenarios. The subjects were divided into three groups of 16 subjects each - one group with a brake application of 5.0 m/s², one with 3.5 m/s² and a third without automated braking, representing the baseline condition. The time budget for regaining control and avoiding the critical situation was five seconds in each group, plus the extension obtained through the automated brake application (6.09s / 5.50s). To ensure that subjects were unable to identify the system limit before the take-over request occurred, the limit emerged suddenly and the subjects were visually and motorically distracted by the ISO-standardized SuRT task. Vehicle dynamics, gaze behaviour and video data were recorded. Furthermore, the subjects were interviewed after each situation and completed a questionnaire to assess their subjective evaluation. No statistical evidence was found that automated brake application enhances take-over quality or shortens take-over time. Nevertheless, the study of outcomes like time to collision, lateral accelerations and accident probability indicate that a positive influence may exist, and that experimental restrictions merely prevented its detection.
Article
Currently, development of conditionally automated driving systems which control both lateral and longitudinal vehicle guidance is attracting a great deal of attention. The driver no longer needs to constantly monitor the roadway, but must still be able to resume vehicle control if necessary. The relaxed attention requirement might encourage engagement in non-driving related secondary tasks, and the resulting effect on driver take-over is unclear. The aim of this study was to examine how engagement in three different naturalistic secondary tasks (writing an email, reading a news text, watching a video clip) impacted take-over performance. A driving simulator study was conducted and data from a total of 79 participants (mean age 40 years, 35 females) were used to examine response times and take-over quality. Drivers had to resume vehicle control in four different non-critical scenarios while engaging in secondary tasks. A control group did not perform any secondary tasks. There was no influence of the drivers' engagement in secondary tasks on the time required to return their hands to the steering wheel, and there seemed to be only little if any influence on the time the drivers needed to intervene in vehicle control. Take-over quality, however, deteriorated for distracted drivers, with drivers reading a news text and drivers watching a video deviating on average approximately 8-9cm more from the lane center. These findings seem to indicate that establishing motor readiness may be carried out almost reflexively, but cognitive processing of the situation is impaired by driver distraction. This, in turn, appears to determine take-over quality. The present findings emphasize the importance to consider both response times and take-over quality for a comprehensive understanding of factors that influence driver take-over. Furthermore, a training effect in response times was found to be moderated by the drivers' prior experience with driver assistance systems. This shows that besides driver distraction, driver-related factors influencing take-over performance exist.