Conference PaperPDF Available

Towards A Framework of Detecting Mode Confusion in Automated Driving: Examples of Data from Older Drivers 2020. Towards A Framework of Detecting Mode Confusion in Auto- mated Driving: Examples of Data from Older


Abstract and Figures

A driver's confusion about the dynamic operating modes of an Automated Vehicle (AV), and thereby their confusion about their driving responsibilities can compromise safety. To be able to detect drivers' mode confusion in AVs, we expand on a previous theoretical model of mode confusion and operationalize it by first defining the possible operating modes within an AV. Consequently, using these AV modes as different classes, we then propose a classification framework that can potentially detect a driver's mode confusion by classifying the driver's perceived AV mode using measures of their gaze behavior. The potential applicability of this novel framework is demonstrated by a classification algorithm that can distinguish between drivers' gaze behavior measures during two AV modes of fully-automated and non-automated driving with 93% average accuracy. The dataset was collected from older drivers (65+), who, due to changes in sensory and/or cognitive abilities can be more susceptible to mode confusion.
Content may be subject to copyright.
Towards A Framework of Detecting Mode Confusion in
Automated Driving: Examples of Data from Older Drivers
Shabnam Haghzare
Institute of Biomaterials and
Biomedical Engineering, University of
Toronto, Toronto, ON, Canada
Jennifer Campos, Ph.D.
The KITE Research Institute –
University Health Network, Toronto,
ON, Canada
Alex Mihailidis, Ph.D., P.Eng.
Department of Occupational Science
and Occupational Therapy, University
of Toronto, Toronto, ON, Canada
A driver’s confusion about the dynamic operating modes of an
Automated Vehicle (AV), and thereby their confusion about their
driving responsibilities can compromise safety. To be able to detect
drivers’ mode confusion in AVs, we expand on a previous theoretical
model of mode confusion and operationalize it by rst dening
the possible operating modes within an AV. Consequently, using
these AV modes as dierent classes, we then propose a classication
framework that can potentially detect a driver’s mode confusion by
classifying the driver’s perceived AV mode using measures of their
gaze behavior. The potential applicability of this novel framework
is demonstrated by a classication algorithm that can distinguish
between drivers’ gaze behavior measures during two AV modes
of fully-automated and non-automated driving with 93% average
accuracy. The dataset was collected from older drivers (65
), who,
due to changes in sensory and/or cognitive abilities can be more
susceptible to mode confusion.
Human-centered computing
Collaborative and social com-
puting; Collaborative and social computing theory, concepts and
paradigms; Computer supported cooperative work; Human com-
puter interaction (HCI); Interaction paradigms; Collaborative inter-
action; Human computer interaction (HCI); HCI theory, concepts
and models.
Automated Vehicles, Mode Confusion, Gaze Behavior, Classication,
Driver Monitoring
ACM Reference Format:
Shabnam Haghzare, Jennifer Campos, Ph.D., and Alex Mihailidis, Ph.D.,
P.Eng.. 2020. Towards A Framework of Detecting Mode Confusion in Auto-
mated Driving: Examples of Data from Older Drivers. In 12th International
Conference on Automotive User Interfaces and Interactive Vehicular Applica-
tions (AutomotiveUI ’20 Adjunct), September 21, 22, 2020, Virtual Event, DC,
USA. ACM, New York, NY, USA, 4 pages.
Permission to make digital or hard copies of part or all of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for third-party components of this work must be honored.
For all other uses, contact the owner/author(s).
AutomotiveUI ’20 Adjunct, September 21, 22, 2020, Virtual Event, DC, USA
©2020 Copyright held by the owner/author(s).
ACM ISBN 978-1-4503-8066-9/20/09.
The safety of Automated Vehicles (AV) in which the driving respon-
sibilities are shared between the driver and the AV depends heavily
on the eective cooperation between the two [
]. In non-automated
driving, the driver is responsible for tasks that are temporally and
hierarchically dependent on each other; lower level operational
tasks (steering and speed control), mid-level tactical tasks (object
and event detection and vehicle maneuvering), and higher level
strategic tasks (navigation) [
]; all of which require drivers’
constant monitoring. Given the interdependencies of the driving
tasks, the eective cooperation between the driver and the AV is
contingent on the driver’s accurate understanding of their new
responsibilities in automated driving.
However, the distribution of responsibilities between the driver
and the AV is not necessarily a zero-sum allocation of the non-
automated driving tasks. This is because vehicle automation does
not necessarily lessen the responsibilities of the driver; rather, it
changes the nature of the driver’s responsibilities [
]. The na-
ture of such new responsibilities depends heavily on (a) the tasks
that the AV is able to execute (automation scope), (b) the degree
to which the AV is automating the driving task in its scope (au-
tomation degree) [
], and (c) the driving conditions during which
the AV is able to execute the tasks in its scope to its specied de-
gree (automation operational limit) [
].The Levels of Automation
(LoA) taxonomy by the Society of Automotive Engineers (SAE) or
US National Highway Trac Safety Administration (NHSTA) pro-
vides a general guideline around dierent AV functionalities [
However, this taxonomy does not capture the variable and more
nuanced driving responsibilities in AVs of the same LoA [
]. In
addition, the terms used for branding commercially-available AVs
and the public’s limited understanding of their specic function-
alities can contribute to an unsafe miscalibration between the dri-
vers’ perceived responsibilities versus their actual responsibilities
[1, 18, 21]
Furthermore, most AVs still have an operational limit. When this
limit is reached (e.g. in response to varying environmental condi-
tions), the vehicle automation may transition to a non-automated
mode. Alternatively, the automation system may “gracefully de-
grade”, i.e., gradually narrow its scope or lower its degree of control
]. Therefore, even an AV that is designed to operate with a max-
imum scope and degree can have multiple and varying modes of
operation in response to changing driving conditions. The AV’s
dynamic operating mode has, in practice, led to driver’s confusion
or lack of awareness about AV’s current operating mode [
]. The
AutomotiveUI ’20 Adjunct, September 21, 22, 2020, Virtual Event, DC, USA Shabnam Haghzare et al.
rst AV-related fatal crash reported by NHSTA is speculated to have
been caused by the driver’s confusion about the AV’s operating
mode [
]. Thus, to detect driver’s mode confusion in AVs, it
becomes necessary to view AV functionalities as dynamic modes
that are each characterized by a distinct set of scope, degree, and
operational limits. This is because, for a safe AV-driver coopera-
tion, the drivers should have an accurate understanding of their
responsibilities in each of the dierent AV modes. In this paper,
we propose a framework that views AV functionalities as dynamic
modes (Section 2) and propose a classication approach (Section 3)
that can potentially detect instances of mode confusion. In Section
3.1, we present preliminary results of applying this framework on
a dataset collected from older drivers, who, extrapolating from lit-
erature on non-automated driving [
], can be more susceptible
to a lack of situational awareness, and therefore mode confusion
during automated driving due to potential age-related declines in
A recent theoretical model of driver’s mode confusion [
] de-
nes it as discrepancies between the driver’s perception of the
current AV mode and the true AV mode. This framework presents
a Hidden Markov Model (HMM) where the observed states are the
true AV modes and the hidden states are the driver’s perceived AV
To operationalize this theoretical model [
] in a way that practi-
cally detects mode confusion in AVs, and to generalize the model to
AVs of all LoAs, we present a framework of AV operation as a Finite-
State Markov Chain (F-SMC). Each possible state
(Si,i∈ {0, . ., M})
corresponds to an AV operating mode with a distinct combination of
scopes, degrees, and operational limits (Equations 1-3), where
is the combination of the driving tasks that the AV can perform
species whether the AV is merely aiding the driver
with the tasks in its Scope or fully automating these tasks. Each
of the states can have an
Oper ational Limit
dened as the set of
environmental/road conditions in which the AV can safely perform
the tasks in its
. However, due to the uncertain-
ties around the conditions that give rise to AV failures, the set of
such driving conditions is often not well-dened. This uncertainty
around state
Oper ational Limits
lends itself to the probabilistic
transitions in the model, in that, if the
Oper ational Limits
well-dened, the transitions as a result of reaching them would
have been deterministic, and the model could have consequently
been reduced to a Finite-State Machine.
Si=Scopei×Deдreei×Operat ional Limiti(1)
ScopeiP{Lonдitudinal Control ,Lateral Control,Monitor inд}
Deдreei{N one,DecisionAid,ActionImplementation :
Assist ance,Action Implement ation :Ful l }(3)
Corresponding to the nite number of AV modes, the model
has a nite number of states, and
indicates the ideal operat-
ing state with the widest scope and highest degree. The num-
ber of states/modes of an AV will therefore depend on both
and on how gradual the transitions to the non-automated
state (
) are planned. For instance, an AV that, in response
to reaching the state operational limit, degrades its state grad-
ually by one will have
1number of states (Figure 1a).
Whereas an AV can also be designed to abruptly transition
from an ideal state with all possible tasks in the
are fully
automated to a state where none of the tasks are automated
(Figure 1b).
This framework captures the dynamic states of an AV in which,
due to underspecied operational limits of the states, the transi-
tions between states are probabilistic. However, once the AV has
transitioned to an arbitrary state, that state is deterministic and
known. Therefore, to detect driver’s mode confusion, only the dri-
ver’s perceived AV state needs to be inferred. As such, an instance
of mode confusion can be detected if the inferred state is incon-
gruent with AV’s true and deterministic state. With the hypothesis
that drivers’ perceived AV states are associated with their moni-
toring behavior, we propose using gaze behavior measures to infer
the driver’s perceived AV state. Thus, morphing the theoretical
HMM model [
] into a practical problem where the observations
are features of drivers’ monitoring behavior and the hidden states
are one out of all possible states of an AV that correspond to the
driver’s perceived AV state. Depending on the number of states
in an AV (e.g., Min Figure 1a), this problem can be framed as
-class classication problem. In this setting, the objective
is to classify gaze behavior measures into one of the possible
classes. In this paper, we consider a 2-class classication problem
with the two classes corresponding to the AV states described in
Figure 1b.
Gaze behavior measures such as blinking, xations, and saccades
have long been successfully applied to indirectly measure driver’s
mental workload and monitoring behavior [
]. In this study, we
investigated the use of xation and saccade measures to distinguish
between the drivers’ monitoring behavior in fully-automated versus
non-automated driving where the drivers were aware of the current
state of the AV and were explicitly ensured that the AV operated
with no risks of failure.
3.1 Data Description
Gaze behavior data was collected from 33 older adults (65
while driving in an immersive, full-eld-of-view driving simula-
tor (DriverLab) using SmartEye Pro, a remote eye-tracking system.
Each driver completed six
8-min driving scenarios – three fully-
(Sf ul lyaut o )
and three non-automated (
Snon aut o
]. Participants were aware of the AV mode in each scenario,
hence the assumption that their perceived AV state corresponds
to the AV’s true state. After excluding the data from 16 unreli-
able scenarios, the average duration and the number of saccades
and xations were calculated for the rest of the scenarios, re-
sulting in 182 samples,
with the scaled feature vec-
and the associated state,
{Snon aut o ,Sf ull yaut o }
as class labels for each sample/scenario
Towards A Framework of Detecting Mode Confusion in Automated Driving: Examples of Data
from Older Drivers AutomotiveUI ’20 Adjunct, September 21, 22, 2020, Virtual Event, DC, USA
Figure 1: Trellis diagram of the mode/state sequence in the F-SMC model of two AVs. (a): An AV designed to ideally operate in
SM. (b): An AV with two states of fully-automated and non-automated.
Table 1: The results of the Gaussian Process Classier on dierent set of features.
Number of Features Features
AUC*(Mean ±SD)
4 F1 x F2 x F3 x F4 0.92 ±0.05 0.95 ±0.5 0.92 ±0.04
3 F1 x F2 x F3 0.93 ±0.03 0.95 ±0.05 0.93 ±0.02
F1 x F2 x F4 0.93 ±0.03 0.95 ±0.05 0.93 ±0.02
F1 x F3 x F4 0.85 ±0.04 0.91 ±0.03 0.85 ±0.04
F2 x F3 x F4 0.69 ±0.06 0.78 ±0.07 0.69 ±0.07
2 F1 x F2 0.80 ±0.04 0.88 ±0.04 0.79 ±0.04
F1 x F3 0.86 ±0.03 0.91 ±0.04 0.86 ±0.03
F1 x F4 0.86 ±0.03 0.91 ±0.04 0.86 ±0.03
F2 x F3 0.68 ±0.05 0.77 ±0.01 0.69 ±0.06
F2 x F4 0.68 ±0.05 0.77 ±0.07 0.69 ±0.06
F3 x F4 0.53 ±0.05 0.60 ±0.10 0.34 ±0.24
1 F1 (Average Saccade
0.81 ±0.07 0.89 ±0.05 0.79 ±0.07
F2 (Average Fixation
0.58 ±0.03 0.58 ±0.06 0.59 ±0.05
F3 (Number of
0.53 ±0.04 0.57 ±0.08 0.35 ±0.20
F4 (Number of
0.53 ±0.04 0.57 ±0.08 0.35 ±0.20
*AUC =Area Under the receiver operating characteristics Curve; *F1-Score: 2 x (P r e ci s io n×Re c al l )
(P r ec is i on+R ec al l )
3.2 Preliminary Results
A Gaussian Process classier with a Radial Basis Function kernel
] led to the best accuracy for classifying the gaze behavior fea-
tures into two classes of non-automated and fully-automated. Table
1 summarizes the results of a 5-fold cross-validation on dierent
feature sets (
⊂ Fd
) with all samples of a single individual in one
of the folds. As per Table 1, the average duration of saccade (F1)
successfully distinguished the two classes with the addition of other
features increasing the classication performance.
In this paper, we have presented a framework that can potentially
detect drivers’ mode confusion in an AV with two operating modes.
In this framework, gaze behavior measures were used to success-
fully classify drivers’ perception of the current AV mode into one
of the two possible modes. Consequently, mode confusion can be
detected if the classied drivers’ perceived AV mode is incongruent
with the AV’s true and deterministic operating mode. The current
AutomotiveUI ’20 Adjunct, September 21, 22, 2020, Virtual Event, DC, USA Shabnam Haghzare et al.
work has two major limitations. First, is the scalability of it to mul-
tiple AV modes where drivers’ monitoring behavior may not be
as distinct as the two extreme modes of the used dataset. Second,
the reported classications are based on the entire
8-min driving
scenario, whereas, to avoid unsafe consequences of mode confusion,
drivers’ mode confusion should be detected within a shorter time-
frame. Future work will utilize the current dataset to investigate
the use of other gaze behavior features that can classify the shorter
instances of gaze behavior data into dierent AV states.
We thank Katherine Bak (Toronto Rehabilitation Institute, Univer-
sity of Toronto) for her contributions to collecting the data used in
this work. This work was supported by Canadian Institute of Health
Research (CIHR), AGE-WELL Graduate Award in technology and
aging, and Vector Institute Postgraduate Aliate Award.
Hillary Abraham, Bobbie Seppelt, Bruce Mehler, and Bryan Reimer. 2017. What’s
in a Name: Vehicle Technology Branding & Consumer Expectations for Au-
tomation. In Proceedings of the 9th International Conference on Automotive User
Interfaces and Interactive Vehicular Applications (AutomotiveUI ’17).ACM Press,
New York, NY, USA, 226–234. DOI:
National Highway Trac Safety Administration. 2013. Preliminary statement of
policy concerning automated vehicles. Washington, DC, 1-14.
Lisanne Bainbridge. 1983. Ironies of automation. In Analysis, design and evalua-
tion of man–machine systems, 129-135.
Cheryl Actor Bolstad. 2001. Situation awareness: Does it change with age?. In
Proceedings of the human factors and ergonomics society annual meeting, vol.
45, no. 4, pp. 272-276. Sage CA, Los Angeles, CA. DOI:
Ryan J. Caserta, and Lise Abrams. 2007. The relevance of situation awareness
in older adults’ cognitive functioning: A review. European Review of Aging and
Physical Activity 4, no. 1, 3-13. DOI: 0018-x
SAE On-Road Automated Vehicle Standards Committee. 2018. Taxonomy and
denitions for terms related to driving automation systems for on-road motor
vehicles. SAE International: Warrendale, PA, USA. DOI:
Sidney W. A. Dekker and David D. Woods. 2002. MABA-MABA or abracadabra?
Progress on human–automation co-ordination. Cognition, Technology & Work
4, no. 4, 240-244. DOI:
Mica R. Endsley. 2017. Autonomous driving systems: A preliminary naturalistic
study of the Tesla Model S. Journal of Cognitive Engineering and Decision Making
11, no. 3, 225-238. DOI:
Frank Ole Flemisch, Frank Ole, Klaus Bengler, Heiner Bubb, Hermann Winner,
and Ralph Bruder. 2014. Towards cooperative guidance and control of highly
automated vehicles: H-Mode and Conduct-by-Wire. Ergonomics 57, no. 3, 343-360.
Kareem Habib and S. Ridella. 2017. Automatic vehicle control systems. National
Highway Trac Safety Administration, 13.
Shabnam Haghzare, Katherine Bak, Jennifer Campos, and Alex Mihailidis. 2019.
Factors inuencing older adults’ acceptance of fully automated vehicles. In Pro-
ceedings of the 11th International Conference on Automotive User Interfaces and
Interactive Vehicular Applications: Adjunct Proceedings (AutomotiveUI ’19).ACM
Press, New York,N Y, USA, 135–139. DOI:
Tobias Hecht, Anna Feldhütter, Jonas Radlmayr, Yasuhiko Nakano, Yoshikuni
Miki, Corbinian Henle, and Klaus Bengler. 2018. A review of driver state monitor-
ing systems in the context of automated driving. In Congress of the International Er-
gonomics Association. 398-408. Springer, Cham. DOI:
3-319- 96074-6_43
Tasuku Ishigooka, Satoshi Otsuka, Kazuyoshi Serizawa, Ryo Tsuchiya, and Fumio
Narisawa. 2019. Graceful Degradation Design Process for Autonomous Driving
System. In International Conference on Computer Safety, Reliability, and Security,
19-34. Springer, Cham. DOI: 26601-1_2
Christian P. Janssen, Linda Ng Boyle, Andrew L. Kun, Wendy Ju, and Lewis L.
Chuang. 2019. A hidden markov framework to capture human–machine interac-
tion in automated vehicles. International Journal of Human–Computer Interac-
tion 35, no. 11, 947-955. DOI:
Gerhard Marquart, Christopher Cabrall, and Joost de Winter. 2015. Review of
eye-related measures of drivers’ mental workload. Procedia Manufacturing 3,
2854-2861. DOI:
Natasha Merat, Bobbie Seppelt, Tyron Louw, Johan Engström, John D. Lee, Emma
Johansson, Charles A. Green et al. 2019. The “out-of-the-loop” concept in auto-
mated driving: Proposed denition, measures and implications. Cognition, Tech-
nology & Work 21, no. 1, 87-98. DOI: 8
John A. Michon. 1985. A critical view of driver behavior models: what do we
know, what should we do?. In Human behavior and trac safety, pp. 485-524.
Springer, Boston, MA. DOI: 2173-6_19
Micheal A. Nees. 2018. Drivers’ perceptions of functionality implied by terms
used to describe automation in vehicles. In Proceedings of the Human Factors
and Ergonomics Society Annual Meeting, vol. 62, no. 1, 1893-1897. Sage CA, Los
Angeles, CA. DOI:
Raja Parasuraman, Thomas B. Sheridan, and Christopher D. Wickens. 2000. A
model for types and levels of human interaction with automation. IEEE Transac-
tions on systems, man, and cybernetics-Part A: Systems and Humans 30, no. 3,
286-297. DOI:
Bobbie Seppelt, Bryan Reimer, Luca Russo, Bruce Mehler, Jake Fisher, and David
Friedman. 2018. Towards a Human-Centric Taxonomy of Automation Types.
White Paper. DOI:
Eric R. Teoh. 2020. What’s in a name? Drivers’ perceptions of the use of ve SAE
Level 2 driving automation systems. Journal of safety research 72, 145-151. DOI:
Carl E. Rasmussen and Christopher K. I. Williams. 2006. Gaussian Processes for
Machine Learning. the MIT Press. DOI:
... Especially combining NDRTs with driving mode transitions can introduce mode confusion [138]. Haghzare et al. [139] proposed the use of eye-tracking to predict mode confusion. To prevent mode confusion a-priori, others argued for adapting driver training [140,141], or introducing tactile, auditory, and visual information displays [142]. ...
Full-text available
The widespread implementation of mobile personal computing devices like notebooks and smartphones has changed knowledge work towards more mobility beyond the traditional office desk. Rising levels of driving automation on the road may initiate a similar shift. By changing the driver's role to that of the \emph{driver-passenger}, the demand for so-called \glspl{NDRT} grows. For example, commuters could use their time on the road to prepare for the upcoming office day, or truck drivers could do logistics planning between on- and offloading. However, driver-passengers still have the responsibility to stay ready to respond to \glspl{TOR}. They occur when a not-yet fully automated vehicle experiences a system failure or functional limitation. Accordingly, in this thesis, we investigate the concept of a mobile office in a \gls{SAE L3} vehicle. Its goals are to enable productive \gls{NDRT} engagement during automated driving phases but also safe manual driving after \glspl{TOR}. Therefore, user interfaces that face these challenges for the typical office tasks of text entry and comprehension in \gls{SAE L3} vehicles are developed and evaluated. They account for both office work and \gls{TOR}/driving ergonomics issues based on the user-centered design process. The designs are informed by standards, applied \gls{HCI} research literature, and cognitive resource and multitasking theories. Mixed-methods user studies with medium- to high-fidelity prototypes allowed quantitatively and qualitatively assessing the interfaces and their features regarding users' objective and subjective performance with them and physiological responses to them. Thereby, we inferred generalizable results on the design features, underlying theories, and the methods used to design and evaluate them. We found that merging knowledge from various areas of \gls{HCI} can promote safety and productivity of office work in \gls{SAE L3} vehicles to some extent when iteratively improving interface designs. Furthermore, the mixed-methods evaluations revealed detailed aspects of applying prevalent \gls{HCI} theory and applied research findings in a novel and complex domain. Overall, we report findings on various mobile office interface modalities and combinations concerning their impact on ergonomics factors such as performance, workload, situational awareness, and well-being. Additionally, we detail the methodological approach taken, including the infrastructure required to implement it.
Conference Paper
Full-text available
Fully Automated Vehicles (FAVs) have the potential to improve older adults' quality-of-life by enhancing their mobility. Such benefits can only be realized if FAVs are acceptable and thus used by older adults. However, older adults' acceptance of FAVs is reported to be the lowest amongst all Levels of Automation (LoA). The current driving simulation-based study provides preliminary insights into the factors that may be associated with older adults' acceptance of FAVs. Such insights can, in turn, inform user-centered FAV designs that are acceptable for older adults and can thereby enable the benefits of using FAVs by this population. Specific associations that were considered were those between older adults' acceptance of FAVs and internal factors characterizing the individuals, external factors characterizing the driving environment, and FAV features.
Full-text available
Introduction: Automobile manufacturers are developing increasingly sophisticated driving automation systems. Currently, the highest level of automation available on the market is SAE Level 2, which provides sustained assistance for both lateral and longitudinal vehicle control. The purpose of this study was to evaluate how drivers' perceptions of what behaviors secondary to driving are safe while a Level 2 system is operating vary by system name. Methods: A nationally representative telephone survey of 2005 drivers was conducted in 2018 with questions about behaviors respondents perceived as safe while a Level 2 driving automation system is in operation. Each respondent was asked about two out of five system names at random for a balanced study design. Results: The name "Autopilot" was associated with the highest likelihood that drivers believed a behavior was safe while in operation, for every behavior measured. There was less variation observed among the other four SAE Level 2 system names when compared with each other. A limited proportion of drivers had experience with advanced driver assistance systems and fewer of these reported driving a vehicle in which Level 2 systems were available. Drivers reported that they would consult a variety of sources for information on how to use a Level 2 system. Conclusions: The names of SAE Level 2 driving automation systems influence drivers' perceptions of how to use them, and the name "Autopilot" was associated with the strongest effect. While a name alone cannot properly instruct drivers on how to use a system, it is a piece of information and must be considered so that drivers are not misled about the correct usage of these systems. Practical Applications: Manufacturers, suppliers, and organizations regulating or evaluating SAE Level 2 automated driving systems should ensure that systems are named so as not to mislead drivers about their safe use.
Technical Report
Full-text available
A human-centric, consumer-facing automation taxonomy is proposed to address emergent issues of consumer confusion related to automation types and associated role responsibility. A set of surveys were fielded to help understand the extent to which consumers were able to accurately interpret a proposed consumer-facing taxonomy relative to the 6-level SAE J3016 taxonomy. Results show a mixed benefit of the proposed set compared to the J3016 set. Overall, across both taxonomies, consumers were best able to differentiate the extremes of automation types, leading to the question of whether or not it may be beneficial to provide a simplified representation of automation types to communicate functionality. A binary framing (“driving” vs. “riding”) is proposed to ensure consumer understanding. This framework may best serve consumer understanding until such time as educational or other efforts can be developed and tested to ensure consumers have the needed understanding to make informed decisions around the safe and effective use of vehicle automation.
Full-text available
The expectations induced by the labels used to describe vehicle automation are important to understand, because research has shown that expectations can affect trust in automation even before a person uses the system for the first time. An online sample of drivers rated the perceived division of driving responsibilities implied by common terms used to describe automation. Ratings of 13 terms were made on a scale from 1 (“human driver is entirely responsible”) to 7 (“vehicle is entirely responsible”) for three driving tasks (steering, accelerating/braking, and monitoring). In several instances, the functionality implied by automation terms did not match the technical definitions of the terms and/or the actual capabilities of the automated vehicle functions currently described by the terms. These exploratory findings may spur and guide future research on this under-examined topic.
Full-text available
Despite an abundant use of the term “Out of the loop” (OOTL) in the context of automated driving and human factors research, there is currently a lack of consensus on its precise definition, how it can be measured, and the practical implications of being in or out of the loop during automated driving. The main objective of this paper is to consider the above issues, with the goal of achieving a shared understanding of the OOTL concept between academics and practitioners. To this end, the paper reviews existing definitions of OOTL and outlines a set of concepts, which, based on the human factors and driver behaviour literature, could serve as the basis for a commonly-agreed definition. Following a series of working group meetings between representatives from academia, research institutions and industrial partners across Europe, North America, and Japan, we suggest a precise definition of being in, out, and on the loop in the driving context. These definitions are linked directly to whether or not the driver is in physical control of the vehicle, and also the degree of situation monitoring required and afforded by the driver. A consideration of how this definition can be operationalized and measured in empirical studies is then provided, and the paper concludes with a short overview of the implications of this definition for the development of automated driving functions.
Conference Paper
Full-text available
Vehicle technology naming has the potential to influence drivers’ expectations (mental model) of the level of autonomous operation supported by semi-automated technologies that are rapidly becoming available in new vehicles. If divergence exists between expectations and actual design specifications, it may make it harder to develop trust or clear expectations of systems, thus mitigating potential benefits. Alternately, over-trust and misuse due to misunderstanding increase the potential for adverse events. An online survey investigated whether and how names of advanced driver assistance systems (ADAS) and automation features relate to expected automation levels. Systems with “Cruise” in their names were associated with lower levels of automation. “Assist” systems appeared to create confusion between whether the driver is assisting the system or vice versa. Survey findings indicate the importance of vehicle technology naming and its impact in influencing drivers’ expectations of responsibility between the driver and system in who performs individual driving functions.
An autonomous driving system requires the safety and availability of automated driving. For example, an autonomous driving system with automation level 3 requires the functions to request the driver to take over driving and to sustain safe automated driving until the driver accepts the request if a hardware failure occurs. However, there is a demand to continue automated driving if the system maintains sufficient performance for automated driving after the failure occurs. Therefore, we propose a graceful degradation design process to improve the automated driving continuation rate by defining degradation functions against performance limitation and hardware failure. The process integrates and extends ISO/PAS 21448 and ISO26262 and carries out these tasks in the order of system-level, ECU-level, and microcontroller-level degradation design. Furthermore, we propose a framework to calculate worst-case mode switch time (WCMST), which means the time duration from failure detection to degradation processing, by utilizing degradation design results. To evaluate the proposed process and framework, we applied them to the prototype system with automation level 3. The evaluation results showed that the designed system can sustain automated driving against 86.1% of performance degradation factors and that the framework can improve the calculation accuracy of WCMST by 35.3%.
We introduce a Hidden Markov Model framework to formalize the beliefs that humans may have about the mode in which a semi-automated vehicle is operating. Previous research has identified various "levels of automation," which serve to clarify the di↵erent degrees of a vehicle's automation capabilities and expected operator involvement. However, a vehicle that is designed to perform at a certain level of automation can actually operate across di↵erent modes of automation within its designated level, and its operational mode might also change over time. Confusion can arise when the user fails to understand the mode of automation that is in operation at any given time, and this potential for confusion is not captured in models that simply identify levels of automation. In contrast, the Hidden Markov Model framework provides a systematic and formal specification of mode confusion due to incorrect user beliefs. The framework aligns with theory and practice in various interdisciplinary approaches to the field of vehicle automation. Therefore, it contributes to the principled design and evaluation of automated systems and future transportation systems.
Autonomous and semiautonomous vehicles are currently being developed by over14 companies. These vehicles may improve driving safety and convenience, or they may create new challenges for drivers, particularly with regard to situation awareness (SA) and autonomy interaction. I conducted a naturalistic driving study on the autonomy features in the Tesla Model S, recording my experiences over a 6-month period, including assessments of SA and problems with the autonomy. This preliminary analysis provides insights into the challenges that drivers may face in dealing with new autonomous automobiles in realistic driving conditions, and it extends previous research on human-autonomy interaction to the driving domain. Issues were found with driver training, mental model development, mode confusion, unexpected mode interactions, SA, and susceptibility to distraction. New insights into challenges with semiautonomous driving systems include increased variability in SA, the replacement of continuous control with serial discrete control, and the need for more complex decisions. Issues that deserve consideration in future research and a set of guidelines for driver interfaces of autonomous systems are presented and used to create recommendations for improving driver SA when interacting with autonomous vehicles.