Content uploaded by Jing Feng
Author content
All content in this area was uploaded by Jing Feng on Oct 23, 2017
Content may be subject to copyright.
INTRODUCTION
According to the U.S. Department of Transportation
(2013), in 2012, over 33 thousand people died in motor vehi-
cle crashes, and 2.36 million people were injured in the United
States. The industry has been tirelessly working towards creat-
ing a safer driving environment, and autonomous vehicle
technology has been proposed as a potential solution. Compa-
nies like Volvo, Nissan and even NASA have issued state-
ments with the desire to implement an autonomous vehicle by
2020 (Los Angeles Times, 2015; Nissan, 2015). Due to this
accelerated interest, the National Highway Traffic Safety Ad-
ministration (NHTSA) has made autonomous vehicle technol-
ogy research a primary point of focus (NHTSA, 2013, 2014)
and has described 5 total levels for vehicle automation
(NHTSA, 2013).
It is estimated that autonomous vehicles could be in-
troduced as early as 2020. However, this technology may not
be fully adopted by the majority of the population until ap-
proximately 2060 due to vehicle cost and selective location
availability (Littman, 2015). Given the fast pace of technology
innovation, human factors research is needed to understand
driver behavior when interacting with each level of autono-
mous driving.
The NHTSA has placed an importance in research on
autonomous vehicle technology, specifically the human fac-
tors elements that are incorporated in take-overs
such that
“drivers can safely transition between automated and non-
automated vehicle operation” (NHTSA, 2013). According to
the NHTSA (2013) a warning should be provided to the driver
in an autonomous vehicle prior to any transition or take-over.
A sufficient warning to take-over interval is essential to allow
the driver to fully assess the situation and re-engage in the
driving task. It is important to design the driver-vehicle inter-
face to support seamless transition. A few assistive systems
are being developed, including vehicle route planning and path
mapping, placement of sensors, vehicle display units, collision
avoidance (Bergholz, Timm, & Weisser, 2000), long range
radars and GPS tracking (Los Angeles Times, 2015), vehicle
to vehicle communication (NHTSA, 2014), and gesture con-
trol (BMW Group, 2015).
A systematic understanding of driver behavior during
take-over period is highly necessary. Research in other do-
mains (e.g. aviation) can broaden our understanding about
automation and can provide insight to the potential and hin-
drances of the technology. For example, previous research
identified important considerations on system error (Sarter &
Woods, 1997), over-reliability (Parasuraman & Riley, 1997)
under-reliability (Endsley & Kiris, 1995, Kaber, Onal, &
Endsley, 1995), and communication errors (Hollands & Wick-
ens, 1999).
A few pioneering projects have started looking at
driver engagement and disengagement using eye tracking
(Merat, Jamson, Lai, Daly, & Carsten, 2014) as well as path
mapping via eLane measuring standard deviation of lane posi-
tion, mean and minimum longitudinal velocity, steering wheel
angle input, time headway and time to contact (Merat, Jamson,
Lai, & Carsten, 2014).
As a first step to understand driver behavior during
take-over in semi-autonomous driving, this research focuses
on answering two questions: 1) how do drivers react after they
have been disengaged from the task of driving in a semi-
autonomous vehicle (e.g., level 3); 2) how do drivers adapt
and learn with accumulating experience in transition. This
study aims to observe driver behavior during the take-over
(the brief period of time after driver regains control from the
vehicle) between manual driving and an artificially designed
and simplified level 3 autonomous driving. In addition, this
study also explores how drivers learn to take over with in-
creasing experience.
METHOD
We simulated partial-autonomous driving in a driving
simulator. Take-over scenarios were activated when the vehi-
cle approached a construction zone (thus the driving environ-
ment became much more complex and human operation was
needed). This study aims to explore how participants handle a
take-over during partially automated driving. Moreover, the
study was also designed to examine whether warnings of two
different warning to take-over intervals (i.e., how much in
advance was the warning provided) differ in terms of their
effectiveness, as well as how drivers’ take-over performance
would change over time.
Participants
This is an ongoing study. By the time of writing up
this proposal, we have tested five participants (all male, un-
dergraduate students, age range (19-21). These participants
Semi-Autonomous Vehicles: A Closer Look at the Take-Over
A pivotal area of research focuses on the human factors elements incorporated with autonomous vehi-
cles, specifically, understanding how drivers adopt and interact with this technology, identifying the
challenges that a driver may face during the interaction and considering these issues and challenges in
the design of the driver-vehicle interface are critical to the success of this advanced technology. Previ-
ous analyses suggested that take-over – the time in which an individual regains control of the vehicle –
is one driving period that could raise critical safety concerns. This on-going study aims to observe how
drivers perform during the critical take-over period. In addition, this study also explores the effective-
ness of two warning to take-over intervals (warning given 7.5 seconds or 4.5 seconds before take-over).
Some trends in the preliminary results are emerging; particularly a possible improved performance over
three sections of drives (4 take-overs in each section). Findings and future directions are discussed.
were gathered from the University’s Experimetrix website
which is where students may sign up for participants and re-
ceive course credit in exchange for participation. In this pro-
posal, we present some preliminary results from the observa-
tions of the five participants.
Materials
Simulated driving. Driving simulation was run on a console
version of STISIM Drive 3 (Figure 1). The simulation was
displayed on three adjacent 42-inch television screens. The
graphics were presented at 1920 x 1080.The simulator is con-
sisted of a steering wheel, and driving pedals. Participants
were allowed to adjust the seat according to their personal
preferences. The simulator collected various driving perfor-
mance measures and at a rate of 60Hz. Two auditory notifica-
tions were used to indicate the upcoming transition from au-
tomation to manual driving (i.e., driver taking over) and from
manual driving to automation (i.e., manual driving will end
and automation will be activated). The warning for an upcom-
ing take-over was two beeps of first 400 Hz and then a succes-
sive beep of 350 Hz. The notification of the upcoming activa-
tion of autonomous control was two beeps of first 350Hz and
then 400 Hz.
Figure 1: STISIM Drive 3 Simulator
Questionnaire measuring opinions concerning autonomous
driving. We have adapted a questionnaire (Schoettle and Si-
vak, 2014) for measuring opinions of drivers concerning au-
tonomous vehicles to assess our participants’ familiarity to
autonomous driving and opinions of the driving situations that
we developed. Participants completed this self-report ques-
tionnaire before and after completing simulated driving. By
collecting opinions from every participant twice, we may be
able to identify the potential changes in their opinions regard-
ing partially autonomous driving with accumulating experi-
ence.
Procedure
Participants first completed an online version of the
questionnaire. They then participated in the experiment ses-
sion in the lab. Participants were provided a consent form as
well as a brief demographic survey.
Once they had completed all of the surveys, the par-
ticipants were provided instructions about the simulated driv-
ing task. The notification sounds (warning for driver take-
over, warning for automation about to start) were played to the
participants and explanations were provided for both. Partici-
pants then practiced simulated driving during a practice drive
which was based off of the experimental drives. During this
drive, participants experienced a simple driving route with
stop signs, and also two transitions from computer controlled
drive (partially autonomous) to manual and vice-versa. Sound
notifications of the state changes were provided in advance
before a transition took place. Participants were allowed to
repeat the practice if they desired additional time. Prior to the
practice, participants were encouraged to direct any questions
to the experimenter. Participants then completed the experi-
ment session.
The experiment session consisted of three drives. The
vehicle was centered in a rural highway environment with
construction zones placed at specific intervals. Each drive was
a total of 12.52 km, and the set speed limit during the autono-
mous portion was 67.5 km/h, and that same speed limit re-
mained the advised speed throughout the manual drives. Each
drive contained four manual zones (designed as construction
zones) which expanded a constant distance of 0.94 km, and
four autonomous driving sections were presented in each
drive. Each autonomous driving sections was of either 1.81
km or 2.41 km. These distances were selected based on time,
with each autonomous drive will last 1.5 minutes (1.81 km) or
2 minutes (2.41 km). Two distances of autonomous driving
were used instead of one, in order to induce disengagement
and also to reduce the repetitiveness and predictability of the
take-over timing. An overview of the procedure is illustrated
in Figure 2. Given greater familiarity to imperial units than to
the metric units of our participants, we presented imperial
units in our scenarios. We converted these measures into met-
ric units for the presentation of this proposal.
Design
The drives were designed to resemble a 4 lane rural
highway, with various trees placed outside of the roadway.
The autonomous portion contained oncoming vehicles. During
this portion, the system was designed to control both the speed
and lane position. Although the scenario attempted to mimic a
level 3 autonomous vehicle (limited self-driving capabilities)
within the capability of our simulator, we are very aware of
the limitations of our scenario to study partially autonomous
Figure
2: An Overview of the Research Procedure
driving. The computerized controls were in fact level 2 (two
advanced functions, including a cruise control and a lane posi-
tion control, operating simultaneously). To be successful in
our mimicry, we did not include any sudden objects on the
road during these intervals, thus the vehicle appeared to drive
itself during the autonomous drive and no input was required
from participants. In addition, participants were instructed that
the vehicle would drive it self during some period, but transi-
tion between autonomous drive and manual drive would take
place after warning sounds were presented and the vehicle
would initiate these transitions. By these designs, the partici-
pants were in a driving situation that involved more than level
2 autonomous driving, more similar to level 3.
The manual driving portions were designed as con-
struction zones (Figure 3). During manual driving, participants
controlled all aspects of the vehicle (e.g., lateral control, longi-
tudinal velocity control). There were six different construction
zones that were designed and each was presented twice and in
a randomized order in the three drives.
Figure 3: Entrance to one manual zone.
Before take-over, participants were notified using an
auditory warning. This warning was presented at two warning
to take-over intervals: one being .15 km (7.5 seconds) before
the take-over took place, and the other being .09 km (4.5 sec-
onds) before the take-over happened. After each take-over,
participants had .12 km to re-engage in driving (i.e., resume
full control of the vehicle) before a construction zone started.
RESULTS
The measures collected for analysis include drivers
speed (average speed, and the minimum speed), standard de-
viation of lane position, brake input and throttle input. These
measures were collected for the entire manual driving portion,
however to single out the take-over, we specifically analyzed
the 0.12 km zone that occurred immediately following the
take-over, this distance will be called the take-over zone.
There were 6 repetitions of each warning to take-over interval
condition within participants during which these measures
were collected. In addition to analyzing the two warning inter-
vals, we were interested in examining the difference among
the three drives. Although a sample size of 5 is much smaller
than the desired number to properly detect any effect that we
were interested in, given ANOVA is in general a robust test,
we did a 3 by 2 repeated measures ANOVA (drive by warning
interval) for each of the measures listed above to examine the
trend of potentially emerging results. In the following results
sections, we only noted effects with a p value smaller than .30
which may be suggesting some trends. The main focus of
analysis at this stage is to examine the patterns as indicated by
the mean values of each measure across conditions.
Longitudinal Velocity
The participant’s average speeds (km/h) over the en-
tire manual drive for both warning interval conditions are il-
lustrated in Figure 4. As seen, there is a general trend of a de-
crease in speed during take over. In the analysis, we examined
both average speed and minimum speed during the task over
period.
Figure 4: Average Longitudinal Velocity
To analyze this decrease in speed, a t-test was con-
ducted for each drive and condition (comparing the minimum
speed to the preset 67.5 km/h). A possible trend of a difference
between minimum speed and the set speed emerged at take-
over, in particular for Condition 1.
The overall means on average speed during take-over
showed some difference between the three drives, with a gen-
eral trend of increasing average speed across the three drives
(Mdrive
1
= 48.73, Mdrive
2
= 57.70, Mdrive
3
= 59.82) during
the take-over zone for the drives. There was a trend observed
in significance testing, F(2, 8) = 2.48, p = .145. Not much dif-
ference was observed on average speed during take-over be-
tween the two warning interval conditions (M 7.5s warning =
56.31, M 4.5s warning =54.52) and in general no trend of in-
teraction. Similarly, the means of minimum speed looked dif-
ferent with a trend of increasing minimum speed from Drive 1
to Drive 3 (Mdrive
1
= 40.39, Mdrive
2
= 48.33, Mdrive
3
=
Table 2
Results from T-Test for Minimum Speeds in Take-Over Zone
M t df p
Condition
1
Drive 1 39.88 -2.46 4 .069
Drive 2 45.52 -2.31 4 .082
Drive 3 56.68 -2.22 4 .090
Condition
2
Drive 1 40.89 -2.02 4 .113
Drive 2 51.14 -2.01 4 .115
Drive 2 51.45 -2.01 4 .115
54.07), with a very weak trend in significance testing, F(2,8) =
1.60, p = .26. Not much trend was found about the difference
on minimum speed during take-over between the two warning
intervals (M 7.5s warning = 47.36, M 4.5s warning =47.83),
or any interaction between drive and warning interval. The
change of means across conditions are illustrated in Figure 5
(minimum speed), Figure 6 (average speed).
Figure 5: Minimum Speeds by Condition over Drives. Condition 1
the warning is provided .15 km before take-over and Condition 2 the
warning is provided .09 km before take-over.
Figure 6: Average Speed by Condition over Drives. Condition 1 the
warning is provided .15 km before take-over and Condition 2 the
warning is provided .09 km before take-over.
Standard Deviation of Lane Position
In addition to longitudinal velocity, lane position of
the vehicle during take-over could be another driving perfor-
mance measure showing the smoothness of take-over. No
trend emerged for means during the take-over zone of the
drives (Mdrive
1
= 7.89, Mdrive
2
= 8.80, Mdrive
3
= 6.24), how-
ever there is a potential trend emerging between the means for
the two warning interval conditions (M 7.5s warning = 6.79,
M 4.5s warning =8.51). This trend was observed with signifi-
cance testing, F(1,4) = 60.6, p = .001.
Brake and Throttle Input
Furthermore, we examined the brake input as well as
the throttle input during take-over. This measure was selected
to determine if there were any significant relations as to how a
participant decreased their speed. There appears to be a gen-
eral decrease in brake input across the drives (Mdrive
1
= -1.23,
Mdrive
2
= -1.13, Mdrive
3
= -.77), however there was no trend
observed in significance testing. Similarly, there was no ob-
served trend in the throttle input (Mdrive
1
= 1.82, Mdrive
2
= -
1.64, Mdrive
3
= 1.96), where there was no observed trend via
significance testing either.
DISCUSSION
This study aimed to examine driver behavior during a
take-over that could take place when driving a limited self-
driving vehicle (level 3 autonomous). We observed this take-
over by using two warning conditions, and over a period of
three drives. Despite the small sample size, we observed some
patterns and trends. With more data, we will be able to proper-
ly examine the significance of these patterns.
One interesting trend was the potential improvement
in driver performance during task-over in the three drives, as
the speed change became less dramatic over the drives. This
may imply the need for training when introducing partially
autonomous driving during which take-overs may occur. This
finding further supports suggestions made by NHTSA (2013)
that training procedures need to be considered. There does not
seem to be any significant difference between the two warning
conditions in regards to speed, however there is a significant
difference in warning condition for lane position. This would
imply that the longer warning (.15 km) lead to better driver
control and consistency of the vehicles lane position at take-
over.
Moreover, the general trend seen in speed overall by
participants cause some concern regarding traffic flow. This
decrease in speed that results after take-over could trigger the
shockwave effect. The shockwave effect is a traffic phenome-
non which occurs when the traffic flow is disrupted by a de-
crease in speed (e.g. accident, distraction) (May, 1990).
Limitations
The STISIM Drive 3 simulator did not support the
development of a truly limited self-driving automation with
level 3 control, therefore we attempted to mimic this level of
automation within the technical possibilities by limiting the
possible events during driving. During the autonomous drive,
there were no events which would trigger a response from the
vehicle, considering the only functions we had available were
speed control and lane position control. Participants were in-
structed that the simulated driving was operating as a level 3
vehicle. Therefore, results from this study may not generalize
to a real autonomous driving situation given the technical dif-
ferences and participants’ beliefs in whether they were inter-
acting with a highly autonomous driving situation (although
our anecdotal observation of the 5 participants so far seem to
believe that). However, the results are interesting and further
analyses of data from a sufficient sample size could be in-
formative. Of course, validation of the results in a more ad-
vanced simulator or even naturalistic driving could significant-
ly improve the generalizability of the results.
Another limitation is due to the simulator’s inability
to collect driver performance during the autonomous portions.
As a result, it was difficult to obtain measures of driver per-
formance right after a warning was given and before a take-
over happened. We have recorded videos of participants’ be-
havior during driving (e.g., facial expression, hands and feet
movement). These observational data may provide some in-
sights into drivers’ behavior during the take-over period.
Future Study
This study was primarily exploratory, and it’s main
goal was to provide a platform for further analysis. It’s results
introduce some possible implications of the take-over and also
lead to various considerations that could be made in many
aspects of autonomous driving (e.g. training, traffic flow).
Additionally, comparing results between young participants
(current population) and elderly participants would be useful.
This would provide insight as to how to design these vehicles
with all populations being considered.
Another main aspect of this study that is to be con-
tinued is the incorporation of the survey that is administered
both before and after the experimental drive. This survey
could provide insight into driver’s opinions regarding this
technology and their performance (i.e. if a driver does not trust
the technology, they may perform worse).
REFERENCES
Bergholz, R., Timm, K., & Weisser, H. (2000). U.S. Patent
No. 6,151,539. Washington, DC: U.S. Patent and
Trademark Office.
BMW Group. (2015). Connected Drive: The Future of Con-
nected Drive.
http://www.bmwgroup.com/e/0_0_www_bmwgroup
_com/forschung_entwicklung/technologie/connected
_drive/zukunft_connected_drive.html
Endsley, M. R., & Kiris, E. O. (1995). The out-of-the-loop
performance problem and level of control in automa-
tion. Human Factors: The Journal of the Human
Factors and Ergonomics Society, 37(2), 381-394.
Hollands, J. G., & Wickens, C. D. (1999). Engineering psy-
chology and human performance. Journal of surgical
oncology.
Kaber, D. B., Onal, E., & Endsley, M. R. (2000). Design of
automation for telerobots and the effect on perfor-
mance, operator situation awareness, and subjective
workload. Human Factors and Ergonomics in Manu-
facturing, 10(4), 409-430.
Los Angeles Times. (2015, Feb 19). Volvo to Launch Self-
Driving Pilot Program in 2017. Available at:
http://www.latimes.com/business/autos/la-fi-hy-
volvo-self-driving-cars-20150219-story.html
Litman, T. (2014). Autonomous Vehicle Implementation Pre-
dictions. Victoria Transport Policy Institute, 28.
May, A. D. (1990). Traffic flow fundamentals.
Merat, N., Jamson, H. A., Lai, F., & Carsten, O. (2014). Hu-
man Factors of Highly Automated Driving: Results
from the EASY and CityMobil Projects. InRoad Ve-
hicle Automation (pp. 113-125). Springer Interna-
tional Publishing.
Merat, N., Jamson, A. H., Lai, F. C., Daly, M., & Carsten, O.
M. (2014). Transition to manual: Driver behaviour
when resuming control from a highly automated ve-
hicle. Transportation research part F: traffic psy-
chology and behaviour, 27, 274-282.
NHTSA [National Highway Traffic Safety Administration].
(2013). Preliminary statement of policy concerning
automated vehicles. Available at:
http://www.nhtsa.gov/staticfiles/rulemaking/pdf/Auto
mated_Vehicles_Policy.pdf
NHTSA [National Highway Traffic Safety Administration].
(2014). U.S. Department of Transportation announc-
es decision to move forward with vehicle-to-vehicle
communication technology for light vehicles. Avail-
able at:
http://www.nhtsa.gov/About+NHTSA/Press+Release
s/2014/USDOT+to+Move+For ward+with+Vehicle-
toVecle+Communication+Technology+for+Light+V
ehicles
Nissan. (2015). Nissan and NASA partner to jointly develop
and deploy autonomous drive vehicles by end of
year. Available at: http://nissannews.com/en-
US/nissan/usa/releases/nissan-and-nasa-partner-to-
jointly-develop-and-deploy-autonomous-drive-
vehicles-by-end-of-year
Parasuraman, R., & Riley, V. (1997). Humans and automa-
tion: Use, misuse, disuse, abuse. Human Factors: The
Journal of the Human Factors and Ergonomics Soci-
ety, 39(2), 230-253.
Sarter, N. B., & Woods, D. D. (1997). Team play with a pow-
erful and independent agent: A corpus of operational
experiences and automation surprises on the Airbus
A-320. Human Factors, 39, 553–569.
Schoettle, B., & Sivak, M. (2014). A Survey of Public Opin-
ion about Autonomous and Self-Driving Vehicles in
the U.S., the U.K., and Australia. The University of
Michigan Transportation Research Insti-
tute, (UMTRI-2014-21). Retrieved September 9,
2014, from
http://deepblue.lib.umich.edu/bitstream/handle/2027.
42/108384/103024.pdf?sequence=1&isAllowed=y
U.S. Department of Transportation. (2013). Traffic Safety
Facts. Research Note, (DOT HS 811 856). Retrieved
September 25, 2014, from http://www-
nrd.nhtsa.dot.gov/Pubs/811856.pdf
