Conference PaperPDF Available

S(C)ENTINEL: monitoring automated vehicles with olfactory reliability displays

Authors:

Abstract and Figures

Overreliance in technology is safety-critical and it is assumed that this could have been a main cause of severe accidents with automated vehicles. To ease the complex task of permanently monitoring vehicle behavior in the driving environment, researchers have proposed to implement reliability/uncertainty displays. Such displays allow to estimate whether or not an upcoming intervention is likely. However, presenting uncertainty just adds more visual workload on drivers, who might also be engaged in secondary tasks. We suggest to use olfactory displays as a potential solution to communicate system uncertainty and conducted a user study (N=25) in a high-fidelity driving simulator. Results of the experiment (conditions: no reliability display, purely visual reliability display, and visual-olfactory reliability display) comping both objective (task performance) and subjective (technology acceptance model, trust scales, semi-structured interviews) measures suggest that olfactory notifications could become a valuable extension for calibrating trust in automated vehicles.
Content may be subject to copyright.
S(C)ENTINEL - Monitoring Automated Vehicles with
Olfactory Reliability Displays
Philipp Wintersberger
CARISSMA
Technische Hochschule Ingolstadt
Ingolstadt, Germany
Johannes Kepler University
Linz, Austria
philipp.wintersberger@carissma.eu
Dmitrijs Dmitrenko
SCHI Lab, Creative Technology
Research Group, School of
Engineering and Informatics,
University of Sussex
Brighton, United Kingdom
d.dmitrenko@sussex.ac.uk
Clemens Schartmüller
Technische Hochschule Ingolstadt
Ingolstadt, Germany
Johannes Kepler University
Linz, Austria
clemens.schartmueller@thi.de
Anna-Katharina Frison
Technische Hochschule Ingolstadt
Ingolstadt, Germany
Johannes Kepler University
Linz, Austria
anna-katharina.frison@thi.de
Emanuela Maggioni
SCHI Lab, Creative Technology
Research Group, School of
Engineering and Informatics,
University of Sussex
Brighton, United Kingdom
e.maggioni@sussex.ac.uk
Marianna Obrist
SCHI Lab, Creative Technology
Research Group, School of
Engineering and Informatics,
University of Sussex
Brighton, United Kingdom
m.obrist@sussex.ac.uk
Andreas Riener
Technische Hochschule Ingolstadt
Ingolstadt, Germany
Johannes Kepler University
Linz, Austria
andreas.riener@thi.de
ABSTRACT
Overreliance in technology is safety-critical and it is assumed
that this could have been a main cause of severe accidents
with automated vehicles. To ease the complex task of per-
manently monitoring vehicle behavior in the driving en-
vironment, researchers have proposed to implement relia-
bility/uncertainty displays. Such displays allow to estimate
whether or not an upcoming intervention is likely. However,
presenting uncertainty just adds more visual workload on
drivers, who might also be engaged in secondary tasks. We
suggest to use olfactory displays as a potential solution to
Both rst and second author contributed equally
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies
are not made or distributed for prot or commercial advantage and that
copies bear this notice and the full citation on the rst page. Copyrights
for components of this work owned by others than the author(s) must
be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specic
permission and/or a fee. Request permissions from permissions@acm.org.
IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA
©
2019 Copyright held by the owner/author(s). Publication rights licensed
to ACM.
ACM ISBN 978-1-4503-6272-6/19/03. . . $15.00
https://doi.org/10.1145/3301275.3302332
communicate system uncertainty and conducted a user study
(N=25) in a high-delity driving simulator. Results of the ex-
periment (conditions: no reliability display, purely visual
reliability display, and visual-olfactory reliability display)
comping both objective (task performance) and subjective
(technology acceptance model, trust scales, semi-structured
interviews) measures suggest that olfactory notications
could become a valuable extension for calibrating trust in
automated vehicles.
CCS CONCEPTS
Human-centered computing Empirical studies in
HCI;Interaction techniques; Interaction paradigms.
KEYWORDS
Automated Driving, Olfactory, Reliability, SAE J3016, Human
Factors, Trust
ACM Reference Format:
Philipp Wintersberger, Dmitrijs Dmitrenko, Clemens Schartmüller,
Anna-Katharina Frison, Emanuela Maggioni, Marianna Obrist, and An-
dreas Riener. 2019. S(C)ENTINEL - Monitoring Automated Vehi-
cles with Olfactory Reliability Displays. In 24th International Con-
ference on Intelligent User Interfaces (IUI ’19), March 17–20, 2019,
Marina del Ray, CA, USA. ACM, New York, NY, USA, 11 pages.
https://doi.org/10.1145/3301275.3302332
IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA Wintersberger and Dmitrenko et al.
1 INTRODUCTION
Trust in automation is an important topic for a safe use of
automated driving systems (ADSs) [
71
]. According to the
classication of autonomy levels as proposed by SAE [
12
],
ADSs currently available on the market mainly operate on
level 2. Here, the driver is fully responsible for monitoring
the vehicle’s actions and thus the overall safety. However,
recent events (such as the fatal crash of a Tesla driver in
May 2016, but also less critical situations) indicate that many
drivers utilizing such systems tend to overtrust them, and
do not properly monitor ADSs even in scenarios they were
not designed for [
67
]. This is especially dangerous when
systems seem to work awlessly for a long time and in vary-
ing situations [
21
]. Since monitoring is a challenging task,
even for “highly motivated human beings” (irony of automa-
tion [
4
]), researchers have proposed to use so-called “reli-
ability/uncertainty displays”, that have shown to provide
benets in both level 2 [
6
] and level 3/4 automated driving
(AD) [
25
]. Such displays are able to reduce the chance of
mode awareness failures while increasing situation aware-
ness as well as system transparency, and thereby ultimately
lead to better calibrated trust [
49
]. They present the actual
system reliability (or uncertainty, what is the inversion of
reliability, but still follows the same concept – which kind of
information works better is still an ongoing research [
49
])
to the user to adjust his/her monitoring behavior. However,
especially when drivers are visually engaged in secondary
tasks, such displays can merely act as “proxy” for the system
state – instead of monitoring the vehicle and the environ-
ment itself, the driver has to frequently inspect the display,
what still demands his/her visual attention.
Since future intelligent and multimodal user interfaces should
adapt to dierent types of users [
51
] while using the full
range of human interaction and communication capabilities
[
63
], we claim that there is a need to evaluate other modali-
ties for communicating reliability information. A potential
modality in this regard could be the sense of smell, that, in
contrast to other typical approaches (such as haptics [
39
]
or auditory cues [
38
]), is still widely unused, but provides
some unique advantages: The sense of smell is a very pow-
erful interaction medium [
36
] enabling humans to extract
meaningful information [
60
]. For example, it has been shown
that odors trigger automatic and implicit retrieval of mental
representations of information related to the object the scent
is coming from [
11
], and enable automatic access to terms se-
mantically related to odors [
30
]. Moreover, scents can be very
ecient in activating the central neural system [
3
,
37
,
65
],
which is essential to keep the driver alert and more atten-
tive on the road [
56
]. Scents can also act as an arousing (e.g.,
when the driver is tired or inattentive [
23
,
73
]) or as a calming
(e.g., when the driver is stressed [
31
,
46
]) stimulus. In future
automated vehicles (AVs), classical perception channels (i.e.,
visual and auditory) will often be occupied by secondary
tasks (such as watching a video, what demands both visual
and auditory attention), while olfactory notications have
proven to be a valid way to gain user’s attention [
7
]. Conse-
quently, to the best of our knowledge, our study (see Figure
1) is the rst experiment including olfactory notications for
trust calibration in AD.
2 RELATED WORK
Trust in automation can be dened as “the attitude that an
agent will achieve an individual’s goals in a situation character-
ized by uncertainty and vulnerability” [
41
], and is a complex
construct built by analytic, analogical, and aective processes
before (dispositional trust), during (situational trust), and af-
ter (learned trust) direct system interaction [
29
]. To foster
safe use of automated systems (and thereby prevent both
disuse and misuse [
52
]), users should adjust their subjective
trust levels to t “an objective measure of trustworthiness”
(“calibration of trust” [
47
]). Reliability/uncertainty displays
should assist in the process of trust calibration (especially to
account for overtrust) by providing decision aids that allow
users to estimate an automated system’s performance in a
given situation [22].
Figure 1: Study setup: Participants had to frequently inter-
vene by actuating the brake pedal in case the automated
longitudinal system fails (low reliability indicated on the
central in-vehicle display), while performing a detection-
response task on a smartphone (left). To hide the activation
sound of the olfactory device (located outside the vehicle,
right), we used noise-canceling headphones for the sound
output of the driving simulation.
Important groundwork in the domain of AD is the study
conducted by Beller, Heesen and Vollrath [
6
], who demon-
strated the potential of a binary reliability display for AVs in
a dual-task experiment. Since then, various papers have ad-
dressed reliability/uncertainty displays in the driving domain.
Helldin et al. could show that such displays can also improve
S(C)ENTINEL - Monitoring AVs with Olfactory Displays IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA
performance and comfort in Take-Over scenarios [
25
]. Re-
cent studies have addressed potential metrics and design
approaches for in-vehicle displays [
49
], but also augmented
reality [
40
], or less obtrusive modalities such as haptics [
39
].
The presentation of dierent levels of reliability/uncertainty
became more and more ne grained in these experiments,
aiming to provide drivers more detailed information about
the system state. However, a problem reliability displays
share with any warning information is that, if (due to an
oensive warning strategy) users face too many false alarms,
they might simply ignore them (“cry wolf eect” [
10
]). Con-
sidering vehicle safety, drivers already seem to often ignore
warning lights in vehicles [32].
As in the near future more and more potentially safety crit-
ical systems will be operated by everyday consumers [
71
],
overtrust/overreliance is widely debated in the eld of ro-
botics [
67
] and AD [
66
]. For example, in a recent series of
simulator studies conducted by Volvo, nearly 30% of drivers
crashed in a provoked accident scenario, despite hands on
the wheel and eyes on the road, and the authors conclude
that more research is necessary to nd out how system lim-
itations can be communicated to drivers more eectively
[
66
]. We claim that olfactory notications could benet the
driver in such situations, as smell is a sense with a strong
emotional component [2, 20, 27].
For example, Baron and Kalsher [
5
] proved that the scent
of lemon increases alertness and the mood of the driver.
The emotion-eliciting eect of scents is particularly useful
in inducing mood changes because they are almost always
experienced clearly as either pleasant or unpleasant [
20
].
For instance, Alaoui-Ismaïli et al. [
1
] used scents of vanillin
and menthol to trigger positive emotions in their subjects
(mainly happiness and surprise), as well as methyl methacry-
late and propionic acid to trigger negative emotions (mainly
disgust and anger). The scents of lemon, peppermint, rose,
and lavender have been shown as ecient in improving the
hedonic experiences of the user [
16
,
17
,
55
], whereas lemon
and lavender have also demonstrated to be a good medium
of conveying useful information in the context of driving
and beyond [
9
,
16
,
18
,
19
,
44
]. On this aspect, it is essential
to keep olfactory stimuli synchronized with other modali-
ties [
48
]. Further, scents have already been proven to have
a positive impact on driving performance/behavior. Martin
and Cooper [
45
] showed that the scent of lemon can im-
prove drivers’ braking performance, while Dmitrenko et al.
[
16
] demonstrated that the scents of lemon, peppermint, and
lavender could help to reduce the number of errors. Further,
scents of peppermint, rosemary, eucalyptus and lemon have
been proven to be useful for keeping drowsy drivers awake
[
23
,
28
,
50
,
73
]. Scents could also help to remind drivers on
certain driving-relevant activities, as the sense of smell is
known to have a strong link with memories [9, 26, 58].
3 DRIVING SIMULATOR STUDY
To nd out if olfactory displays can ease monitoring for
drivers and thus provide a valuable extension of visual relia-
bility displays, we conducted a dual-task study in a driving
simulator. Participants had to drive in a semi-automated
(level 2) vehicle while performing a detection-response task
(DRT) on a smartphone (see Figure 1). To counter potential
criticism of our experimental setting (smartphone usage or
engagement in secondary tasks is strictly forbidden at level
2 driving in most countries), we want to emphasize that (1)
many drivers engage in side activities (for example on mobile
devices) despite given legislation [
72
], and (2) if successful,
the underlying concept can be easily adapted to other levels
of automation (for example to improve Take-Over requests
[25]) or even dierent safety-critical systems.
Method
Although recent studies on reliability displays often pre-
sented multiple levels [
40
,
49
], we chose to utilize a binary
display because of two reasons. First, we believed that for an
initial evaluation, drivers should not need to distinguish be-
tween multiple levels of uncertainty, and second, we wanted
to shift the principle from reliability to “responsibility”. Cur-
rently, drivers utilizing ADSs remain the responsible control
authority any time. However, to make AD successful in the
future, vehicle manufacturers must start taking over respon-
sibility for their vehicles’ actions when driving in automated
mode (this is a precondition to achieve driving automation
above level 2 [
12
]). Thus, the binary display utilized in our
study indicates either, that the vehicle itself takes over full
responsibility for the dynamic driving task (green color, see
Figure 2), or that the driver him/herself is responsible in case
system reliability drops, indicating that a manual interven-
tion is likely (red color, see Figure 2).
While performing the DRT, drivers had to monitor and in-
tervene (if necessary) in the longitudinal control system of
the AV. Participants could thereby rely on the given reliabil-
ity information – as long as reliability was high (green), no
manual intervention is necessary, in case of low reliability
(red) a system malfunction is likely to occur. Drivers thus
had to succeed in two tasks: (1) performing the DRT while
(2) monitoring/intervening in longitudinal control of the ve-
hicle. In randomized order, participants thereby faced the
following conditions:
Baseline Condition
: No information about the lon-
gitudinal system’s performance is presented, partici-
pants had to manually adjust their monitoring behav-
ior.
Reliability Display
: The reliability for longitudinal
control is presented to the user in form of a classic
binary reliability display.
IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA Wintersberger and Dmitrenko et al.
Olfactory-supported Reliability Display
: In addi-
tion to the visual information, an olfactory notication
will be issued in case the reliability display changes
its status (high to low and vice versa). Drivers thus
can keep their visual attention on the DRT until they
perceive the olfactory stimulus.
Figure 2: Visual design of the utilized binary reliability dis-
play (left: high system reliability, right: low system reliabil-
ity).
Measurements and Research estions
For statistical investigation we conducted the following mea-
surements: performance in the DRT (true and false positive
rate), braking behavior/overrides of the longitudinal control
system (number of manual interventions, average brake du-
ration, and intensity), as well as subjective scales addressing
user acceptance and trust. Therefore, we utilized the trust
scale (TS) by Jian et al. [
35
] (which is widely used among
trust researchers and provides sub-scales for both trust and
distrust), and the Technology Acceptance Model (TAM) pro-
posed by [
13
] (that assesses a user’s intention to actually
use a given system, determined by his/her perceived ease
of use, perceived usefulness, and attitude towards using [it]
[
13
]). Since product usage leads to positive/negative emo-
tions [
14
], and odors have a strong emotional component [
2
],
we also wanted to nd out if the presented interface aects
participants’ emotional response. Therefore, we utilized the
Positive/Negative Aect Scale (PANAS, [
68
]). Additionally,
we conducted semi-structured interviews (assessing their
perception of the system, potentially changed behavior, etc.)
with all participants after the experiment. By statistical evalu-
ation, we wanted to answer the following research questions:
RQ1:
Can olfactory notications increase performance
in the driving task (quantied by objective measure-
ments assessing braking behavior)?
RQ2:
Can olfactory notications increase performance
in the side activity (detection-response task, quantied
by true/false positive rates)?
RQ3:
How are olfactory notications trusted and ac-
cepted by potential users in comparison with visual or
the total absence of reliability displays (quantied by
standardized scales such as TAM, TS, and PANAS)?
Driving Task
We implemented our driving scenario (using IPG CarMaker)
on the basis of Beller et a. [
6
], where drivers had to monitor
an adaptive cruise control system. In our setting, the vehicle
was driving on the left lane of a straight 2-lane highway seg-
ment with 120km/h. Every 30 seconds, the AV encountered a
lead vehicle with a lower speed of just 70km/h, thus the sys-
tem had to slow down, what was followed by the lead vehicle
changing to the right lane (as soon as the ego-vehicle reached
the same speed as the lead vehicle). Then, the ego-vehicle
could accelerate again and continue driving with 120km/h
for roughly 30 seconds, before the next lead vehicle appeared.
We alternated phases of ca. 2 minutes (i.e., 4 lead vehicles) in
either high (all vehicles detected and the AV slows down by
itself) or low reliability (2 out of 4 cars not detected, where
the driver had to intervene and brake manually). Each drive
included 24 lead vehicles (roughly 12 minutes duration de-
pending on participants’ braking behavior during the three
phases that require manual interventions) and thus 3 alter-
nating phases of high and low reliability (in randomized
order).
Reliability Display and Olfactory Device
To communicate the reliability levels (low and high) we dis-
played either a green or red status symbol (see Figure 2)
prominent on a tablet in the vehicle’s center console (Google
Pixel C, see Figure 1). The symbol changed after clearing the
4th vehicle of every section in the driving task to the new
reliability level (condition reliability). In condition olfactory,
we additionally communicated a change in reliability levels
using two odors (lemon for a change to low and lavender
for a change to high reliability). We used these two scents
as both of them have been used to convey driving-relevant
information in the past [
16
,
17
]. Lemon was chosen for the
change to low reliability, because it is known to keep the dri-
ver alert [
5
,
56
] and to have an arousing eect on users [
34
].
Lavender was chosen for the switch to the high reliability
level, because it is known to help drivers become aware of
information they could have missed when relying only on
visual stimuli [
16
], and as it is one of the most commonly
used relaxing stimuli in olfactory research [3, 43, 46].
We presented these scents in an automated way, adapting
a custom-made and fully controllable scent-delivery device
(see [
15
] for design details). The device delivered the scented
air from an air compressor (Revell Masterclass) attached to
an air lter (5 micron lter from Shako Co Ltd.). The clean
air was propelled through glass jars (using plastic tubes of
4mm in diameter) containing 6g of 100% pure essential oils
("miaroma" essential oils from Holland & Barrett Int. Ltd.)
of lavender (Lavandula ocinalis) and lemon (Citrus limon)
S(C)ENTINEL - Monitoring AVs with Olfactory Displays IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA
with an air pressure of 1 bar. The scent-delivery nozzle (out-
put) was located above the glove compartment, pointing
towards the participant’s face, approximately 1.5m away
from the driver’s nose (this distance could be shortened, if
required by the application scenario [
17
]). The ow of air
was controlled using electric valves (SMC Compact Direct
Operated 2 Port Solenoid Valves) and an Arduino board [
15
].
The scent delivery was working with the vehicle’s AC sys-
tem being constantly on. All involved technical components
(smartphone for the DRT, scent delivery device, driving sim-
ulation, etc.) were synchronized by in-house software to
enable time-critical measurements and repeatability (as sug-
gested by [59]).
Detection-Response Task
For the secondary task we implemented an HTML5/JavaScript
application running on a OnePlus One 5.5" smartphone. On
white background, each cell of a 3x3 grid was updated ev-
ery second randomly showing numbers between 0 and 9
(or no number respectively, see Figure 1). Every time the
number “6” appeared, participants had to press a large but-
ton at the bottom of the screen (once, a second button press
was dismissed in this case). To evaluate performance in the
detection-response task, we calculated the average response
time for true positives, the true positive rate, and the false
positive rate.
Procedure
Prior to taking a part in the study, all participants were
screened for potential olfactory dysfunctions or adverse re-
actions to string scents. Upon arrival, participants were given
a consent form and the experimenter explained the exper-
iment verbally to participants before starting the driving
phase(s). Participants were encouraged to ask questions, if
anything remained unclear. After a short test drive helping
participants to get used to the driving simulator, the experi-
ment started with one of the three conditions (baseline – no
visual and no olfactory stimuli, reliability – with visual noti-
cations involved, or olfactory – visual notications combined
with the olfactory stimuli).
The order of the conditions was quasi-randomized. Each
condition lasted about 12 minutes and the switch between
the reliability levels took place every two minutes. In the
olfactory condition, the scent was triggered simultaneously
with the switch between the visual stimuli and was deliv-
ered for ve seconds. Participants were asked to complete
a short questionnaire assessing demographics before the
driving phases and the set of standardized scales after each
condition. At the end of the experiment, we further con-
ducted a ve minutes long semi-structured interview with
each participant.
4 RESULTS
In total, 25 participants aged between 19 and 38 years (
M=
24
,SD =
3
.
98 years, 10 female, 15 male) voluntarily partici-
pated in the study. Participants have reported to have no ol-
factory dysfunctions, adverse reactions to strong scents, res-
piratory problems or u, and female participants conrmed
that they are not pregnant. Participants were recruited on an
opportunity-sampling basis and all expressed written con-
sent. In the following, we present the results of our statistical
evaluation. Eects are reported as statistically signicant if
p< .
05, we used IBM SPSS Version 24 and (one way) repeated
measures ANOVA (Greenhouse-Geisser in case Mauchly’s
test for sphericity failed) with Bonferroni correction and
respectively Friedman ANOVA, if the data did not follow a
normal distribution. A summary of descriptive statistics and
evaluation results is presented in Tables 1 and 2.
Objective Measures
Driving Performance. Considering driving performance, we
calculated the number of brake pedal actuations, the average
duration, as well as the average intensity of braking actions.
All parameters showed no signicant dierences between
the conditions (see Table 1 for descriptive statistics). Fried-
man ANOVA (test for normal distribution failed) resulted in
χ2(
2
)=.
427
,p=.
080 for the average number of brake actu-
ations, and in
χ2(
2
)=.
250
,p=.
882 for the average duration
of a braking action. A repeated measures ANOVA (assump-
tions for normal distribution and data sphericity met) for
the average intensity of each braking action did not show
signicant dierences (
F(
2
,
46
)=
2
.
844
,p=.
068) as well.
However, pairwise comparisons using Bonferroni correction
would have shown a dierence between the conditions base-
line and olfactory (
p=
0
.
02, we report this fact as ANOVA
just slightly missed the signicance level of .05).
Secondary Task Performance. To assess the performance in
the detection-response task, we evaluated the average re-
action time for true positives, as well as the true and false
positive rate (see Table 1 for descriptive statistics). Only
the reaction time was not normally distributed, and there
were no signicant dierences applying Friedman ANOVA
(
χ2(
2
)=.
75
,p=.
687). For the true positive rate (TP),
we found a signicant dierence using repeated measures
ANOVA (
F(
2
,
46
)=
3
.
496
,p=.
039), however, post-hoc tests
using Bonferroni correction showed no individual dier-
ences (if any, conditions baseline and olfactory were slightly
above the signicance level with
p=.
63, where olfactory
showed the highest true positive rate). The false positive
rate, on the other hand, resulted in a signicant dierence
(
F(
2
,
46
)=
4
.
823
,p=.
013), where post-hoc tests revealed the
origin between conditions olfactory and baseline (
p=.
024,
where olfactory resulted in the lowest false positive rate).
IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA Wintersberger and Dmitrenko et al.
Condition Mean (SD) Statistics
Baseline Reliability Olfactory F Sig (η2
p)
Driving Behavior
Nr. of brakes 15.08 (12.89) 13.67 (5.92) 20.33 (26.89) - .808 (-)
Avg. brake duration (s) 2.91 (2.48) 2.46 (.92) 2.33 (1.01) - .882 (-)
Avg. brake intensity .52 (.19) .49 (.18) .46 (.20) 2.844 .068 (.11)
Secondary Task Performance
Avg. response time TP (s) .81 (.10) .80 (.10) .80 (.12) - .68 (-)
True positive rate .62 (.09) .63 (.09) .65 (.09) 3.496 .039 (.13)
False positive rate .0143 (.004) .0125 (.005) .0113 (.003) 4.823 .013 (.17)
Table 1: Descriptive and test statistics of objective data (braking behavior, secondary task performance). In case of missing
F-values, Friedman ANOVA was utilized (test statistics reported throughout the results section). Signicant dierences are
printed in bold face.
Subjective Measures
Standardized Scales. Reliability analysis showed acceptable
values for Cronbach’s alpha (above
.
722 or higher) for all
sub-scales, thus we were able to calculate mean scale values.
Considering the trust scale from Jian et al. [
35
], we found sig-
nicant dierences for both sub-scales of trust and distrust
(average of the respective scale items). Distrust signicantly
diers with respect to the conditions (Greenhouse-Geisser
since failed precondition for sphericity,
F(
1
.
606
,
38
.
550
)=
18
.
508
,p< .
001). Post-hoc analysis using Bonferroni correc-
tion showed dierences between the conditions baseline and
olfactory (
p< .
001), as well as reliability and olfactory (
p=
.
001), but not between baseline and reliability. Contrarily, in
the sub-scale trust (
F(
1
.
454
,
34
.
891
)=
22
.
725
,p< .
001), both
conditions olfactory (
p< .
001) and reliability (
p=.
001) sig-
nicantly diered from the baseline. However, no dierence
between reliability and olfactory was present here.
In the Technology Acceptance Model (TAM), we were
able to nd signicant dierences in all sub-scales. Per-
ceived ease of use (PEOU) signicantly diered in the re-
sult of the repeated measures ANOVA (Greenhouse-Geisser:
F(
1
.
428
,
34
.
273
)=
37
.
061
,p< .
001). Pairwise comparisons
(Bonferroni) revealed that the baseline diers to both con-
ditions olfactory (
p< .
001) and reliability (
p< .
001), while
there were no dierences between the latter two. Exactly
the same result was obtained for perceived usefulness (PU,
F(
1
.
348
,
32
.
349
)=
17
.
272
,p< .
001). Also here, only the base-
line diered to olfactory (
p=.
001) and reliablity (
p=.
001).
Regarding the attitude towards using the system (ATT), all
conditions have demonstrated signicant dierences,
F(
1
.
484
,
35
.
615
)=
37
.
061
,p< .
001. Condition olfactory was
rated as highest and showed dierences to reliability (
p=.
01)
and baseline (
p< .
001), but also reliability was signicantly
higher than the baseline condition (
p=.
018). Since inten-
tion to use the system (INT) does not represent a scale vari-
able, we utilized a non-parametric test (Friedman ANOVA,
χ2(
2
)=
19
.
3
,p< .
001). Here, pairwise comparisons showed
that only conditions baseline and olfactory signicantly dif-
fered (
p=.
002) from each other. When looking at the results
of the Positive/Negative Aect Scale (PANAS), we could
not nd any dierences regarding positive aect (PA, pre-
conditions for repeated measures ANOVA, as well as data
sphericity met,
F(
2
,
48
)=.
280
,p=.
869). The negative aect
(NA) on the other hand resulted in signicant dierences
(
F(
2
,
48
)=.
6
.
729
,p=.
003), where post-hoc tests using
Bonferroni correction revealed that condition olfactory had
signicantly less negative aect than reliability (
p=.
036)
and baseline (
p=.
02), while there was no dierences be-
tween baseline and reliability.
Semi-Structured Interviews. In the interviews, all 25 partic-
ipants have conrmed that they have perceived the scents
used in the experiment. They mainly emphasized the fact
that the scents were intense enough to get perceived quickly
and that this was helpful. For example, P19 said: “The scents
were always quite intense in the beginning. It was good, be-
cause I could always understand when the switch between the
reliability levels took place.Also, all the 25 participant had
experienced neither scent lingering, nor cross-contamination
during the driving phase. They particularly liked that the
timing of the scent-delivery was spot-on, that the scent dis-
appeared quickly and matched the visual notications very
well. For example, P12 said: “The scents were so succinct that
they appeared at the right time and were then gone relatively
quickly.
Scents were also perceived as helpful in performing the task
of driving and in monitoring the autonomous system. 20/25
participants had mentioned the scents as helpful in perceiv-
ing the change between the reliability levels and as support-
ive in capturing the visual information displayed in the center
console. For example, P14 said: “The scents helped, especially
when there was no eye contact with the display. Moreover,
19/25 participants admitted that they had to monitor the
S(C)ENTINEL - Monitoring AVs with Olfactory Displays IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA
Condition Mean (SD) Statistics
Baseline Reliability Olfactory F Sig (η2
p)
Trust Scale
Trust 1.71 (1.08) 2.99 (1.06) 3.35 (1.26) 22.725 <.001 (.486)
Distrust 3.93 (1.40) 3.15 (1.08) 2.28 (0.91) 18.508 <.001 (.435)
Technology Acceptance Model
Perceived ease of use 2.38 (1.28) 4.06 (1.02) 4.21 (0.92) 37.061 <.001 (.607)
Perceived usefulness 1.32 (1.22) 2.60 (1.41) 2.91 (1.65) 17.272 <.001 (.418)
Attitude towards using the system 1.92 (1.64) 3.11 (1.42) 3.84 (1.62) 15.232 <.001 (.388)
Intention to use the system 1.44 (1.66) 2.52 (1.61) 3.20 (2.00) - <.001 (-)
Positive and Negative Aect Scale
Positive Aect 2.82 (1.10) 2.74 (1.08) 3.07 (1.20) 2.038 .141 (.078)
Negative Aect 2.49 (1.57) 2.24 (1.34) 1.67 (1.34) 6.729 .003 (.219)
Table 2: Descriptive and test statistics of subjective scales (Trust scales, Technology Acceptance Model, Positive and Negative
Aect Scale). In case of missing F-values, Friedman ANOVA was utilized (test statistics reported throughout the results section).
Signicant dierences are printed in bold face.
display less, thanks to the scents. They argued that they had
to look on the display less, could rely on scents, and that
their attention was grasped by the scents. For example, P13
said: “Thanks to the scents, when I was interacting with the
phone, I was sure that with this system I can do anything.
In terms of usefulness, 18/25 participants also mentioned
that they nd olfactory interaction generally useful in au-
tomotive context, considering that the choice of scents is
performed carefully, appropriate training is carried out, and
the scent-delivery is well controlled. For example, P11 said:
“It’s a good idea, but you need to be careful about the choice of
scents., whereas P13 said: “It’s something brand new! It was
very nice! You just need to be careful that not too many scents
are used. . . 2-3 very dierent scents would be good, I think.
At the end of the interview, we encouraged participants to
suggest further scenarios, in which they consider olfactory
feedback to be eective. 5/25 participants recommended us-
ing scents as warnings for such non-urgent notications as a
trac jam or a bad weather alert, and a low petrol level noti-
cation. For example, P21 said: “I would use scents when there is
enough time, when I can decide what I can take over. Another
5/25 participants suggested rather using scents for safety crit-
ical notications (also as a support to visual stimuli), such
as ACC, inter-vehicle distance, and vehicles passing by on
the left/right. For instance, P4 said: “Scents could come when
you drive too close to a car in front of you, when a trac light
goes red, or when a child crosses the road. Furthermore, 3/25
participants expressed a wish for scents to convey vehicle
diagnosis-related data. P11 mentioned “engine overheating”,
P15 an “oil leak”, and P19 generalized this to “problems with
the car”. Two participants decided that scents are good for
“take a break” notications, e.g. P7 said: “It makes sense to use
scents in the car, because they make the driver awake on short-
term.Finally, 4/25 participants referred to the well-being of
the driver, e.g. P21 said: “When I get into the car and it smells
nice, it contributes to comfort, of course.
Summary of Results
Only the combination of visual and olfactory reliability infor-
mation (condition olfactory) showed dierences in driving be-
havior (less intensive brake pedal actuations) and secondary
task performance (lower false positive rate), while the provi-
sion of the visual display only (condition reliability) did not
result in an improvement compared to the baseline drive. In
subjective scales (TS, TAM, PANAS) a signicant dierence
was visible for both test conditions (reliability and olfactory)
compared to the baseline in most sub-scales, while olfactory
notications showed signicantly less distrust (TS) and neg-
ative aect (PANAS), as well as a higher attitude towards
using the system (TAM) compared to visual reliability infor-
mation only. Participants’ positive attitude towards olfactory
notications was further conrmed in semi-structured inter-
views.
5 DISCUSSION
The results of our study provide multiple interesting insights
and conrm, in general, the potential of olfactory notica-
tions for trust calibration. Regarding driving behavior (
RQ1
),
a statistically signicant dierence was only present between
the condition olfactory and the baseline (participants showed
less average braking intensity and thus braked “smoother”).
This dierence is only visible in post-hoc tests, while the
overall ANOVA result slightly missed meeting the signi-
cance level. Although it is often emphasized that pairwise
IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA Wintersberger and Dmitrenko et al.
comparisons should only be conducted in case of signi-
cant ANOVA results, there is also the view that pairwise
dierences can be seen as valid even if the global eect is
not signicant [
33
]. However, we believe it is necessary to
include a larger sample size to either conrm or reject the
observed tendency. At the present stage, our study cannot
fully conrm the results of Beller et al. [
6
] regarding strong
signicant dierences in braking behavior.
Considering
RQ2
, the combination of visual and olfactory
reliability display signicantly decreased the false positive
rate in the secondary task (visual detection-response task)
compared to the baseline condition. There also exist some
tendencies that condition olfactory resulted in a higher true
positive rate as compared to the baseline condition (ANOVA
result signicant but post-hoc not, however Bonferroni is
known to be conservative in pairwise comparisons [
70
]). The
provision of a visual reliability display only did not signi-
cantly improve secondary task performance compared to the
baseline, what highlights the potential benet of olfactory
notications presented along the visual stimuli.
Regarding standardized subjective scales assessing trust and
user acceptance (
RQ3
), we can report increased trust and
acceptance towards olfactory notications. Condition olfac-
tory induced signicantly less distrust and received signi-
cantly higher attitude towards using the system, compared
to visual-only provision of reliability. Also, the provision
of olfactory cues resulted in a signicantly lower negative
aect in PANAS, thus this modality was not perceived nega-
tively among study participants. Subjects’ positive attitude
towards olfactory notications was further conrmed in
semi-structured interviews.
Our observations are in line with the previous ndings on
multimodal in-car interfaces, where drivers were shown to
perform better when assisted by notications consisting of
multiple modalities (e.g., as per [
16
,
53
]). Study results con-
rm this in the scope of AD and trust in automation. Our
ndings are also matching the evidence found in the elds
of psychology and neuroscience, where the sense of smell
has been demonstrated as an ecient medium of conveying
semantically congruent cues [
24
]. Some might question that
olfactory stimuli meet the requirements of time-sensitive
notications, but while vision, audition, and touch traverse
the perceived information en route from periphery to pri-
mary cortex, in olfaction information reaches the primary
cortex directly [
54
], what is a unique advantage compared
to other senses. Further, it is known that a high odorant con-
centration can spontaneously shift our attention to olfaction
[61].
Still, we do not suggest combining olfactory notications
with other modalities permanently, and we would neither
suggest the same for any additional form of reliability display.
We rather emphasize that olfactory notications (as other
modes of communication such as haptics [
39
]) are a valu-
able extension to be included in intelligent user interfaces:
Olfactory stimuli could be applied as feedback messages, in
cases when visual notications are likely to be missed [
16
].
As it cannot be guaranteed that drivers can be reached with
any given modality due to their engagement in arbitrary
secondary activities (that will vary with respect to the de-
mand of dierent perceptional channels), future interfaces
should become context-aware and thereby take both envi-
ronmental/operational properties of the situation, as well as
personal preferences [51] of dierent drivers into account.
The National Transportation Safety Board (NTSB) reported
that in the fatal Tesla accident in 2016, the driver did not
respond to multiple visual and auditory warnings issued by
the system prior to the accident. We do not claim that ol-
factory notications would have made the dierence in this
situation, but we want to raise the question: what could have
happened, if a strong scent, for example the smell of a broken
engine, would have been issued to gain the driver’s attention
[
57
]? An answer to this question, as well as when and how
the support of olfactory notications yields the best results,
should be addressed in future studies and detailed research.
However, our study provides initial insights that highlight
the potential of olfactory notications for trust calibration
in AVs.
6 LIMITATIONS AND FUTURE WORK
As our ndings demonstrate promising tendencies in terms
of olfactory enhanced reliability displays, it is worth explor-
ing multiple levels of reliability conveyed by scents in the
future. This could be achieved by either using two scents of
dierent intensity levels (e.g., as in [
15
]) or by extending the
range of scents (e.g., as in [
16
,
17
]) and assigning a certain
scent to every urgency level (e.g., as in [
44
]). When working
with scents, it is important to acknowledge the subjective
element of scent perception. For example, four of our partic-
ipants said in the post-experiment interview that they did
not like the scent of lavender. In the future, it would be nec-
essary to explore customizable olfactory interfaces, allowing
participants to select the scent of their preference. Also, as
the selection of the scents might not work for everyone and
in every situation (e.g., not in case of a u), it would be a
good idea to explore other modalities, such tactile [
8
,
62
]
and ambient light [
42
] interfaces for conveying automation
reliability-relevant information. We have tested our olfactory
interface in a high-delity driving simulator, with a real car
interior. Interviews conducted with the participants have
revealed no scent lingering or cross-contamination artifacts
experienced during the experiments. This suggests that the
interface is suitable for the use in a real car. However, this
would need to be supported by further studies in the real
S(C)ENTINEL - Monitoring AVs with Olfactory Displays IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA
road environment and for an extended time frame. The loca-
tion of the scent-delivery nozzle and the interference of the
scented airow with the vehicle’s AC (or air coming through
an open window) would need to be investigated further. This
might include positioning the nozzle closer to the driver’s
nose (as per [
15
]) or temporarily replacing olfactory stimu-
lation by other modalities (e.g., touch [
62
]). On-road studies
would also help understand how do drivers feel about using
scents over a longer period of time, how their sensitivity to
the olfactory stimuli changes over time, and if scents get
absorbed by the car’s interior on long term. Furthermore,
such explorations could reveal the eciency of the olfactory
stimuli in the presence of external scents (e.g., coee or a
dog on the rear seat). In terms of neutralizing the delivered
scents, it would be useful to investigate dierent ventilation
parameters (as per [
15
]) and to explore the application of the
“olfactory white” [69].
7 CONCLUSION
In this paper, we have evaluated the potential of an added
olfactory UI in supporting reliability displays for trust cal-
ibration in automated vehicles. Results of a driving simu-
lator study (N=25) comparing three conditions (visual reli-
ability display only, visual display supported by olfactory
notications, baseline condition without any reliability in-
formation) conrm our assumption that olfactory cues can
improve performance in a dual-task setting. We can report
tendencies that, with support of olfactory cues, participants
showed smoother braking behavior compared to the base-
line condition (quantied as brake pedal actuations during
manual interventions in case the longitudinal control sys-
tem failed). Also, adding olfactory notications to the visual
stimuli resulted in signicantly higher performance in the
visual detection-response task. In addition, this (at least in
the context of trust calibration) yet unused modality was
subjectively preferred by study participants based on sub-
jective evaluation. Participants rated the system with added
olfactory cues signicantly better in sub-scales of the Tech-
nology Acceptance Model (TAM) [
64
], the Trust Scale [
35
],
and the Positive/Negative Aect Scale (PANAS) [
68
]. Partici-
pants’ positive attitude towards olfactory notications was
further conrmed in semi-structured interviews, where 80%
of the participants stated that olfactory cues are helpful in
perceiving a change in vehicle reliability levels. Overtrust
is an issue that already led to (even fatal) accidents with
automated vehicles [
67
,
71
] and could hinder a success of
the automated driving technology. Identiying additional
methods to calibrate trust is, thus, timely and important [
66
].
Olfactory cues could become a valuable asset helping to re-
gain attention of drivers that are engaged in secondary tasks
(and thus out of the loop), allowing them to more reliably
assess and react to unknown circumstances.
ACKNOWLEDGMENTS
We applied the FLAE approach for the sequence of authors.
This work is supported under the FH-Impuls program of the
German Federal Ministry of Education and Research (BMBF)
under Grant Number 13FH7I01IA (SAFIR) and by the Euro-
pean Research Council (ERC) under the European Union’s
Horizon 2020 research and innovation program under the
grant agreements No 638605 and No 737576.
REFERENCES
[1]
O. Alaoui-Ismaïli, O. Robin, H. Rada, A. Dittmar, and E. Vernet-Maury.
1997. Basic emotions evoked by odorants: comparison between auto-
nomic responses and self-evaluation. Phys. & Beh. (1997).
[2]
J. Amores and P. Maes. 2017. Essence: Olfactory Interfaces for Un-
conscious Inuence of Mood and Cognitive Performance. In CHI ’17.
ACM, New York, NY, USA.
[3]
T. Atsumi and K. Tonosaki. 2007. Smelling lavender and rosemary
increases free radical scavenging activity and decreases cortisol level
in saliva. 150 (03 2007).
[4]
Lisanne Bainbridge. 1983. Ironies of automation. In Analysis, Design
and Evaluation of Man–Machine Systems 1982. Elsevier, 129–135.
[5]
R. A. Baron and M. J. Kalsher. 1998. Eects of a Pleasant Ambient
Fragrance on Simulated Driving Performance: The Sweet Smell of...
Safety? Env. and Beh. (1998).
[6]
Johannes Beller, Matthias Heesen, and Mark Vollrath. 2013. Improving
the driver–automation interaction: An approach using automation
uncertainty. Human factors 55, 6 (2013), 1130–1141.
[7]
Adam Bodnar, Richard Corbett, and Dmitry Nekrasovski. 2004.
AROMA: Ambient Awareness Through Olfaction in a Messaging Ap-
plication. In Proceedings of the 6th International Conference on Mul-
timodal Interfaces (ICMI ’04). ACM, New York, NY, USA, 183–190.
https://doi.org/10.1145/1027933.1027965
[8]
S.S. Borojeni, T. Wallbaum, W. Heuten, and S. Boll. 2017. Comparing
Shape-Changing and Vibro-Tactile Steering Wheels for Take-Over
Requests in Highly Automated Driving. In AutomotiveUI ’17. ACM.
[9]
S. Brewster, D. McGookin, and C. Miller. 2006. Olfoto: Designing a
Smell-based Interaction. In CHI ’06. ACM, New York, NY, USA.
[10]
Shlomo Breznitz. 2013. Cry wolf: The psychology of false alarms. Psy-
chology Press.
[11]
U. Castiello, G.M. Zucco, V. Parma, C. Ansuini, and R. Tirindelli. 2006.
Cross-Modal Interactions between Olfaction and Vision When Grasp-
ing. Chemical Senses 31, 7 (2006).
[12]
SAE On-Road Automated Vehicle Standards Committee. 2014. Tax-
onomy and denitions for terms related to on-road motor vehicle
automated driving systems.
[13]
Fred D Davis. 1993. User acceptance of information technology: system
characteristics, user perceptions and behavioral impacts. International
journal of man-machine studies 38, 3 (1993), 475–487.
[14]
Pieter MA Desmet and Paul Hekkert. 2007. Framework of product
experience. International journal of design 1, 1 (2007), 57–66.
[15]
D. Dmitrenko, E. Maggioni, and M. Obrist. 2017. OSpace: Towards
a Systematic Exploration of Olfactory Interaction Spaces. In ISS ’17.
ACM, New York, NY, USA.
[16]
D. Dmitrenko, E. Maggioni, and M. Obrist. 2018. I Smell Trouble: Using
Multiple Scents To Convey Driving-Relevant Information. In ICMI ’18.
ACM, New York, NY, USA.
[17]
D. Dmitrenko, E. Maggioni, C.T. Vi, and M. Obrist. 2017. What Did I
Sni?: Mapping Scents Onto Driving-Related Messages. In Automo-
tiveUI ’17. ACM, New York, NY, USA.
IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA Wintersberger and Dmitrenko et al.
[18]
D. Dmitrenko, C.T. Vi, and M. Obrist. 2016. A Comparison of Scent-
Delivery Devices and Their Meaningful Use for In-Car Olfactory In-
teraction. In AutomotiveUI ’16. ACM, New York, NY, USA.
[19]
D. Dobbelstein, S. Herrdum, and E. Rukzio. 2017. inScent: A Wearable
Olfactory Display As an Amplication for Mobile Notications. In
ISWC ’17. ACM, New York, NY, USA.
[20]
H. Ehrlichman and L. Bastone. 1992. Olfaction and emotion. In Science
of olfaction. Springer.
[21]
Mica R Endsley. 2017. Autonomous driving systems: a preliminary
naturalistic study of the Tesla Model S. Journal of Cognitive Engineering
and Decision Making 11, 3 (2017), 225–238.
[22] Richard Finger and Ann M Bisantz. 2002. Utilizing graphical formats
to convey uncertainty in a decision-making task. Theoretical Issues in
Ergonomics Science 3, 1 (2002), 1–25.
[23]
H. Funato, M. Yoshikawa, M. Kawasumi, S. Yamamoto, M. Yamada, and
Y. Yanagida. 2009. Stimulation eects provided to drivers by fragrance
presentation considering olfactory adaptation. In Intelligent Vehicles
Symposium 2009. IEEE.
[24]
J.A. Gottfried and R.J. Dolan. 2003. The Nose Smells What the Eye
Sees: Crossmodal Visual Facilitation of Human Olfactory Perception.
Neuron 39, 2 (2003).
[25]
Tove Helldin, Göran Falkman, Maria Riveiro, and Staan Davidsson.
2013. Presenting system uncertainty in automotive UIs for support-
ing trust calibration in autonomous driving. In Proceedings of the 5th
international conference on automotive user interfaces and interactive
vehicular applications. ACM, 210–217.
[26]
R.S. Herz and T. Engen. 1996. Odor memory: Review and analysis.
Psychonomic Bulletin & Review 3, 3 (1996).
[27]
R. S. Herz. 2002. Inuences of Odors on Mood and Aective Cognition.
In Olfaction, Taste, and Cognition. Cambridge University Press.
[28]
S. Hiroike, S. Doi, T. Wada, E. Kobayashi, M. Karaki, N. Mori, T. Kusaka,
and S. Ito. 2009. Study of olfactory eect on individual driver under
driving. In ICME ’09.
[29]
Kevin Anthony Ho and Masooda Bashir. 2015. Trust in automation:
Integrating empirical evidence on factors that inuence trust. Human
Factors 57, 3 (2015), 407–434.
[30]
R.W. Holland, M. Hendriks, and H. Aarts. 2005. Smells Like Clean
Spirit Nonconscious Eects of Scent on Cognition and Behavior. Psy-
chological Science 16, 9 (2005).
[31]
T. Hongratanaworakit. 2008. Relaxing Eect of Rose Oil on Humans.
Natural Product Communications (2008).
[32] HPI. 2018. How long do you leave your dashboard lights on? Online.
Retrieved October 01, 2018 from https://goo.gl/dMf99W.
[33]
Jason Hsu. 1996. Multiple comparisons: theory and methods. Chapman
and Hall/CRC.
[34]
J. Ilmberger, E. Heuberger, C. Mahrhofer, H. Dessovic, D. Kowarik, and
G. Buchbauer. 2001. The inuence of essential oils on human attention.
I: Alertness. Chemical Senses 26, 3 (2001).
[35]
Jiun-Yin Jian, Ann M Bisantz, and Colin G Drury. 2000. Foundations
for an empirically determined scale of trust in automated systems.
International Journal of Cognitive Ergonomics 4, 1 (2000), 53–71.
[36]
L.M. Kay. 2011. Olfactory Coding: Random Scents Make Sense. Current
Biology 21, 22 (2011).
[37]
J.K. Kiecolt-Glaser, J.E. Graham, W.B. Malarkey, K. Porter, S. Lemeshow,
and R. Glaser. 2008. Olfactory inuences on mood and autonomic,
endocrine, and immune function. Psychoneuroendocrinology 33, 3
(2008).
[38]
Jeamin Koo, Jungsuk Kwac, Wendy Ju, Martin Steinert, Larry Leifer,
and Cliord Nass. 2015. Why did my car just do that? Explaining
semi-autonomous driving actions to improve driver understanding,
trust, and performance. International Journal on Interactive Design and
Manufacturing (IJIDeM) 9, 4 (2015), 269–275.
[39]
Alexander Kunze, Steve Summerskill, Russell Marshall, and Ashleigh J
Filtness. 2018. Preliminary evaluation of variables for communicating
uncertainties using a Haptic seat. (2018).
[40]
Alexander Kunze, Stephen J Summerskill, Russell Marshall, and Ash-
leigh J Filtness. 2018. Augmented Reality Displays for Communicating
Uncertainty Information in Automated Driving. In Proceedings of the
10th International Conference on Automotive User Interfaces and Inter-
active Vehicular Applications. ACM, 164–175.
[41]
John D Lee and Katrina A See. 2004. Trust in automation: Designing
for appropriate reliance. Human factors 46, 1 (2004), 50–80.
[42]
A. Löcken, W. Heuten, and S. Boll. 2015. Supporting Lane Change
Decisions with Ambient Light. In AutomotiveUI ’15. ACM, NY, USA.
[43]
H.W. Ludvigson and T.R. Rottman. 1989. Eects of ambient odors of
lavender and cloves on cognition, memory, aect and mood. Chem.
Sen. (1989).
[44]
E. Maggioni, R. Cobden, D. Dmitrenko, and M. Obrist. 2018. Smell-
O-Message: integration of olfactory notications into a messaging
application to improve users’ performance. In ICMI ’18. ACM, New
York, NY, USA.
[45]
G. N. Martin and J. A. Cooper. 2007. Adding zest to dicult jour-
neys: Odour eects on simulated driving performance. In BPS Annual
Conference.
[46]
M. Moss, J. Cook, K. Wesnes, and P. Duckett. 2003. Aromas of rosemary
and lavender essential oils dierentially aect cognition and mood in
healthy adults. Int. J. of Neuroscience (2003).
[47]
Bonnie M Muir. 1987. Trust between humans and machines, and the
design of decision aids. International Journal of Man-Machine Studies
27, 5-6 (1987), 527–539.
[48]
N. Murray, G.M. Muntean, Y. Qiao, and B. Lee. 2018. Olfaction-Enhanced
Multimedia Synchronization. Springer International Publishing.
[49]
Brittany E. Noah, Thomas M. Gable, Shao-Yu Chen, Shruti Singh, and
Bruce N. Walker. 2017. Development and Preliminary Evaluation
of Reliability Displays for Automated Lane Keeping. In Proceedings
of the 9th International Conference on Automotive User Interfaces and
Interactive Vehicular Applications (AutomotiveUI ’17). ACM, New York,
NY, USA, 202–208. https://doi.org/10.1145/3122986.3123007
[50]
C. Oshima, A. Wada, H. Ando, N. Matsuo, S. Abe, and Y. Yanigada.
2007. Improved delivery of olfactory stimukus to keep drivers awake.
In Workshop on DSP for in-Vehicle and Mobile Systems.
[51]
Shimei Pan, Oliver Brdiczka, Giuseppe Carenini, Duen Horng Chau,
and Per Ola Kristensson. 2016. What’s Hot in Intelligent User Inter-
faces.. In AAAI. 4333–4334.
[52]
Raja Parasuraman and Victor Riley. 1997. Humans and automation:
Use, misuse, disuse, abuse. Human factors 39, 2 (1997), 230–253.
[53]
I. Politis, S. Brewster, and F. Pollick. 2015. To Beep or Not to Beep?:
Comparing Abstract Versus Language-Based Multimodal Driver Dis-
plays. In CHI ’15. ACM, New York, NY, USA.
[54]
J.L. Price. 1990. Olfactory system. The human nervous system (1990).
[55]
N. Ranasinghe, P. Jain, N. Thi Ngoc Tram, K.C.R. Koh, D. Tolley, S.
Karwita, L. Lien-Ya, Y. Liangkun, K. Shamaiah, C. Eason Wai Tung,
C.C. Yen, and E.Y. Do. 2018. Season Traveller: Multisensory Narration
for Enhancing the Virtual Reality Experience. In CHI ’18. ACM, USA.
[56]
B. Raudenbush, R. Grayhem, T. Sears, and I. Wilson. 2009. Eects of
Peppermint and Cinnamon Odor Administration on Simulated Driving
Alertness, Mood and Workload. N.A.J. of Psychology (2009).
[57]
Andreas Riener. 2010. Sensor-actuator supported implicit interaction in
driver assistance systems. Springer.
[58]
F.R. Schab and R.G. Crowder. 2014. Memory for odors. Psychology
Press.
[59]
Clemens Schartmüller, Philipp Wintersberger, and Andreas Riener.
2017. Interactive Demo: Rapid, Live Data Supported Prototyping with
U.S.E.. In Adjunct Proceedings of the 9th International Conference on
S(C)ENTINEL - Monitoring AVs with Olfactory Displays IUI ’19, March 17–20, 2019, Marina del Ray, CA, USA
Automotive User Interfaces and Interactive Vehicular Applications. ACM,
Oldenburg, 239–241. https://doi.org/10.1145/3131726.3132046
[60]
A. Seigneuric, K. Durand, T. Jiang, J.-Y. Baudouin, and B. Schaal. 2010.
The nose tells it to the eyes: Crossmodal associations between olfaction
and vision. Perception 39, 11 (2010).
[61]
L. Sela and N. Sobel. 2010. Human olfaction: a constant state of change-
blindness. Experimental Brain Research (2010).
[62]
G. Shakeri, A. Ng, J.H. Williamson, and S.A. Brewster. 2016. Evaluation
of Haptic Patterns on a Steering Wheel. In Automotive’UI 16. ACM,
New York, NY, USA.
[63]
Matthew Turk. 2014. Multimodal interaction: A review. Pattern Recog-
nition Letters 36 (2014), 189–195.
[64]
Viswanath Venkatesh, Michael G Morris, Gordon B Davis, and Fred D
Davis. 2003. User acceptance of information technology: Toward a
unied view. MIS quarterly (2003), 425–478.
[65]
E. Vernet-Maury, O. Alaoui-Ismaïli, A. Dittmar, G. Delhomme, and J.
Chanel. 1999. Basic emotions induced by odorants: A new approach
based on autonomic pattern results. 75 (03 1999).
[66]
Trent W Victor, Emma Tivesten, Pär Gustavsson, Joel Johansson,
Fredrik Sangberg, and Mikael Ljung Aust. 2018. Automation Expecta-
tion Mismatch: Incorrect Prediction Despite Eyes on Threat and Hands
on Wheel. Human factors (2018), 0018720818788164.
[67]
Alan R Wagner, Jason Borenstein, and Ayanna Howard. 2018. Overtrust
in the robotic age. Commun. ACM 61, 9 (2018), 22–24.
[68]
David Watson, Lee Anna Clark, and Auke Tellegen. 1988. Development
and validation of brief measures of positive and negative aect: the
PANAS scales. Journal of personality and social psychology 54, 6 (1988),
1063.
[69]
T. Weiss, K. Snitz, A. Yablonka, R.M. Khan, D. Gafsou, E. Schneid-
man, and N. Sobel. 2012. Perceptual convergence of multi-component
mixtures in olfaction implies an olfactory white. Proceedings of the
National Academy of Sciences (2012).
[70] Eric W Weisstein. 2004. Bonferroni correction. (2004).
[71]
Philipp Wintersberger and Andreas Riener. 2016. Trust in technology
as a safety aspect in highly automated driving. i-com 15, 3 (2016),
297–310. https://doi.org/doi:10.1515/icom-2016- 0034
[72]
Philipp Wintersberger, Andreas Riener, Clemens Schartmüller, Anna-
Katharina Frison, and Klemens Weigl. 2018. Let Me Finish before I
Take Over: Towards Attention Aware Device Integration in Highly
Automated Vehicles. In Proceedings of the 10th International Conference
on Automotive User Interfaces and Interactive Vehicular Applications.
ACM, 53–65.
[73]
M. Yoshida, C. Kato, Y. Kakamu, M. Kawasumi, H. Yamasaki, S. Ya-
mamoto, T. Nakano, and M. Yamada. 2011. Study on Stimulation
Eects for Driver Based on Fragrance Presentation. In MVA2011 IAPR
Conference on Machine Vision Applications.
... For example, in the context of wearable design, Essence [2] and Bioessence [1] are necklaces designed to release scents based on biometric or contextual data (e.g., heart rate and respiration). Similarly, in the context of virtual systems, the use of scents can be used to enhance the sense of immersion in gaming [10], to provide a more realistic and immersive virtual environment for entertainment [25,65], to enhance the driver's attention in in-car settings [9,24], and to reduce mental workload [84]. In the context of interaction design, Maggioni et al. designed a theoretical framework to introduce scents in the interaction space [25]. ...
Article
Previous research has shown the influence of smell on emotions, memories, and body image. However, most of this work has taken place in laboratory settings and little is known about the influence of smell in real-world environments. In this paper, we present novel insights gained from a field study investigating the emotional effect of smell on memories and body image. Taking inspiration from the cultural design probes approach, we designed QuintEssence, a probe package that includes three scents and materials to complete three tasks over a period of four weeks. Here, we describe the design of QuintEssence and the main findings based on the outcomes of the three tasks and a final individual interview. The findings show similar results between participants based on the scent. For example, with cinnamon, participants experienced feelings of warmth, coziness, happiness, and relaxation; they recalled blurred memories of past moments about themselves and reported a general feeling of being calm and peaceful towards their bodies. Our findings open up new design spaces for multisensory experiences and inspire future qualitative explorations beyond laboratory boundaries.
... Such manoeuvres could be more dangerous than HAD. The provision of specific feedback indicating that the HAD system is performing normally (Beller et al., 2013;Helldin et al., 2013;Wintersberger et al., 2019) could limit drivers' urgent need to take control. ...
Article
Trust in Automation is known to influence human-automation interaction and user behaviour. In the Automated Driving (AD) context, studies showed the impact of drivers’ Trust in Automated Driving (TiAD), and linked it with, e.g., difference in environment monitoring or driver’s behaviour. This study investigated the influence of driver’s initial level of TiAD on driver’s behaviour and early trust construction during Highly Automated Driving (HAD). Forty drivers participated in a driving simulator study. Based on a trust questionnaire, participants were divided in two groups according to their initial level of TiAD: high (Trustful) vs. low (Distrustful). Declared level of trust, gaze behaviour and Non-Driving-Related Activities (NDRA) engagement were compared between the two groups over time. Results showed that Trustful drivers engaged more in NDRA and spent less time monitoring the road compared to Distrustful drivers. However, an increase in trust was observed in both groups. These results suggest that initial level of TiAD impact drivers’ behaviour and further trust evolution.
... Under the assumption that fighting motion sickness is at least partly a cognitive task, Wicken's MRT would suggest improved performance and less overall workload with an olfactory stimulus (if it would include olfaction as a sense). Scents were previously shown to be able to increase alertness and reduce drowsiness [352], improve well-being [186], driving performance [353,354,355], and act as notification modality [356,357]. Concerning sickness, a review of essential oils and aromatherapy against nausea and vomiting by Lin Lua and Zakaria [358] found that the existing studies suffer from methodological flaws and incompleteness. However, there are hints that ...
Thesis
Full-text available
The widespread implementation of mobile personal computing devices like notebooks and smartphones has changed knowledge work towards more mobility beyond the traditional office desk. Rising levels of driving automation on the road may initiate a similar shift. By changing the driver's role to that of the \emph{driver-passenger}, the demand for so-called \glspl{NDRT} grows. For example, commuters could use their time on the road to prepare for the upcoming office day, or truck drivers could do logistics planning between on- and offloading. However, driver-passengers still have the responsibility to stay ready to respond to \glspl{TOR}. They occur when a not-yet fully automated vehicle experiences a system failure or functional limitation. Accordingly, in this thesis, we investigate the concept of a mobile office in a \gls{SAE L3} vehicle. Its goals are to enable productive \gls{NDRT} engagement during automated driving phases but also safe manual driving after \glspl{TOR}. Therefore, user interfaces that face these challenges for the typical office tasks of text entry and comprehension in \gls{SAE L3} vehicles are developed and evaluated. They account for both office work and \gls{TOR}/driving ergonomics issues based on the user-centered design process. The designs are informed by standards, applied \gls{HCI} research literature, and cognitive resource and multitasking theories. Mixed-methods user studies with medium- to high-fidelity prototypes allowed quantitatively and qualitatively assessing the interfaces and their features regarding users' objective and subjective performance with them and physiological responses to them. Thereby, we inferred generalizable results on the design features, underlying theories, and the methods used to design and evaluate them. We found that merging knowledge from various areas of \gls{HCI} can promote safety and productivity of office work in \gls{SAE L3} vehicles to some extent when iteratively improving interface designs. Furthermore, the mixed-methods evaluations revealed detailed aspects of applying prevalent \gls{HCI} theory and applied research findings in a novel and complex domain. Overall, we report findings on various mobile office interface modalities and combinations concerning their impact on ergonomics factors such as performance, workload, situational awareness, and well-being. Additionally, we detail the methodological approach taken, including the infrastructure required to implement it.
Article
Drivers occasionally need to resume vehicle control when an automated driving system (ADS) cannot handle a situation. However, a lack of driver readiness can prevent a smooth transition. For example, in an obstacle avoidance situation, a method to transfer the vehicle control based on the driver’s input of the steering angle can be adopted where rapid steering operation by the driver is required immediately after resuming control. It was observed from the previous studies based on a fixed-based driving simulator that the discontinuity in control due to a sudden disengagement of the control torque of the ADS resulted in steering instability. In addition, the previous studies had proposed a shared mode, in which haptic shared control (HSC) was placed between the automated and manual driving. It was demonstrated that steering stability could be improved through the shared mode. However, in the previous studies, the observation of steering instability and verification of HSC effectiveness of the shared mode were limited to fixed-based driving simulator experiments, in which there was no vehicle motion. In addition, for practical applications, a method using a torque sensor in the previous method is expected to be replaced by a more robust method, because it may introduce noise and the use of a lowpass filter leads to some time lag. Therefore, in this study, we developed a new control transition method that uses only the steering angle. We conducted experiments using a real car, in which the participants were instructed to resume steering control from the automated driving mode. The results demonstrate that the discontinuity in control during the control transition deteriorates the steering stability and vehicle motion, and the shared mode can improve them.
Conference Paper
Full-text available
Social interactions are multisensory experiences. However, it is not well understood how technology-mediated smell can support social interactions, especially in collaborative tasks. To explore its effect on collaboration, we asked eleven pairs of users to work together on a writing task while wearing an interactive jewellery designed to emit scent in a controlled fashion. In a within-subjects experiment, participants were asked to collaboratively write a story about a standardized visual stimulus while exposed to with scent and without scent conditions. We analyzed video recordings and written stories using a combination of methods from HCI, psychology, sociology, and human communication research. We observed differences in both participants' communication and creation of insightful stories in the with scent condition. Furthermore, scent helped participants recover from communication breakdown even though they were unaware of it. We discuss the possible implications of our findings and the potential of technology-mediated scent for collaborative activities.
Article
The human sense of smell is a primal ability that has the potential to reveal unexplored relationships between user behaviors and technology. Humans use millions of olfactory receptor cells to observe the environment around them. Olfaction studies are gaining popularity with the progression of scent delivering (commercial and prototype) devices. This influx of research features various software and hardware designs. Additionally, previous studies have explored numerous target audiences and evaluation methodologies. This article presents a systematic review of pertinent literature that investigates olfactory-based computing (OBC) systems in the field of Human-Computer Interaction. Last, this article highlights state-of-the-art study/system designs, evaluation methods, and offers insights on ways to address current challenges/contributions relevant to OBC technologies.
Chapter
In recent years, the goal of companies to retain customers through good usability has evolved into a more holistic view to enhance the user experience. The purely pragmatic view is to be extended by hedonic aspects in order to touch the users also on the emotional level. Although everyone talks about user experience (UX), it still seems to be just “old wine in new bottles”. Despite extensive UX theory research in recent years, UX is still often used as a synonym for usability. Due to increasing vehicle automation, the automotive industry now also has to rethink its (long) existing processes and develop new strategies in order to keep its customers loyal to the brand in the future. Traffic will change fundamentally—and drivers will often neither drive themselves nor own a vehicle. With this book chapter we want to create the basis for this transformation process. After an overview of the current state of UX practice in the development of user interfaces for vehicle automation, the topic is systematically unfolded from the perspective of academia (literature studies) and industry (expert interviews). Based on the findings, the “DAUX framework” is presented as part of a need-centered development approach. It serves as a structured guide on how to define and evaluate UX in consideration of the challenges of automated driving. For this purpose, it provides guidelines on how (a) relevant needs for hypotheses/UI concept development can be identified and (b) UX can be evaluated by triangulating behavioral-, product-, and experience-oriented methods. To demonstrate its potential, the framework is applied in three case studies, each addressing a different level of automation (SAE L2, SAE L3, and SAE L4). This demonstrates that the “DAUX framework” promotes a holistic view of UX to encourage the development of UIs for driving automation. In particular, it is intended to help resolve technical constraints faced by designers and developers in the different levels of automation with the aim to create a positive UX.
Chapter
Automated driving is transforming the driving experience in the 21st-century vehicle. As a result, interacting with in-vehicle information systems, infotainment, in-car productivity or social interactions and real-life experiences with other passengers in the car, are slowly emerging as primary activities. UX researchers focus more and more on the users not only by developing products and services for them and enhancing their experiences but also actively involving them in co-designing for their own experience. Our research with designers inside the automotive industry suggests that the industry is exceptionally traditional regarding the methods and tools used to design and evaluate interactive experiences in comparison to other domains. In this chapter, we will report on the limitations of the industry in comparison to academia. Besides, we will report on the needs of the automotive UX practitioners and discuss the state of the art methods and tools that are most valued in the automotive industry.
Conference Paper
Full-text available
Cars provide drivers with task-related information (e.g. "Fill gas") mainly using visual and auditory stimuli. However, those stimuli may distract or overwhelm the driver, causing unnecessary stress. Here, we propose olfactory stimulation as a novel feedback modality to support the perception of visual notifications, reducing the visual demand of the driver. Based on previous research, we explore the application of the scents of lavender, peppermint, and lemon to convey three driving-relevant messages (i.e. "Slow down", "Short inter-vehicle distance", "Lane departure"). Our paper is the first to demonstrate the application of olfactory conditioning in the context of driving and to explore how multiple olfactory notifications change the driving behaviour. Our findings demonstrate that olfactory notifications are perceived as less distracting, more comfortable, and more helpful than visual notifications. Drivers also make less driving mistakes when exposed to olfactory notifications. We discuss how these findings inform the design of future in-car user interfaces.
Conference Paper
Full-text available
A major promise of automated vehicles is to render it possible for drivers to engage in nondriving related tasks, a setting where the execution pattern will switch from concurrent to sequential multitasking. To allow drivers to safely and efficiently switch between multiple activities (including vehicle control in case of Take-Over situations), we postulate that future vehicles should incorporate capabilities of attentive user interfaces, that precisely plan the timing of interruptions based on driver availability. We propose an attention aware system that issues Take-Over Requests (1) at emerging task boundaries and (2) directly on consumer devices such as smartphones or tablets. Results of a driving simulator study (N=18), where we evaluated objective, physiological, and subjective measurements, confirm our assumption: attention aware Take-Over Requests have the potential to reduce stress, increase Take-Over performance, and can further raise user acceptance/trust. Consequently, we emphasize to implement attentive user interfaces in future vehicles.
Article
Full-text available
Objective: The aim of this study was to understand how to secure driver supervision engagement and conflict intervention performance while using highly reliable (but not perfect) automation. Background: Securing driver engagement-by mitigating irony of automation (i.e., the better the automation, the less attention drivers will pay to traffic and the system, and the less capable they will be to resume control) and by communicating system limitations to avoid mental model misconceptions-is a major challenge in the human factors literature. Method: One hundred six drivers participated in three test-track experiments in which we studied driver intervention response to conflicts after driving highly reliable but supervised automation. After 30 min, a conflict occurred wherein the lead vehicle cut out of lane to reveal a conflict object in the form of either a stationary car or a garbage bag. Results: Supervision reminders effectively maintained drivers' eyes on path and hands on wheel. However, neither these reminders nor explicit instructions on system limitations and supervision responsibilities prevented 28% (21/76) of drivers from crashing with their eyes on the conflict object (car or bag). Conclusion: The results uncover the important role of expectation mismatches, showing that a key component of driver engagement is cognitive (understanding the need for action), rather than purely visual (looking at the threat), or having hands on wheel. Application: Automation needs to be designed either so that it does not rely on the driver or so that the driver unmistakably understands that it is an assistance system that needs an active driver to lead and share control.
Article
The ACM Conference on Intelligent User Interfaces (IUI) is the annual meeting of the intelligent user interface community and serves as a premier international forum for reporting outstanding research and development on intelligent user interfaces. ACM IUI is where the Human-Computer Interaction (HCI) community meets the Artificial Intelligence (AI) community. Here we summarize the latest trends in IUI based on our experience organizing the 20th ACM IUI Conference in Atlanta in 2015.
Conference Paper
Smell is a powerful tool for conveying and recalling information without requiring visual attention. Previous work identified, however, some challenges caused by user's unfamiliarity with this modality and complexity in the scent delivery. We are now able to overcome these challenges, introducing a training approach to familiarise scent-meaning associations (urgency of a message, and sender identity) and using a controllable device for the scent-delivery. Here we re-validate the effectiveness of smell as notification modality and present findings on the performance of smell in conveying information. In a user study composed of two sessions, we compared the effectiveness of visual, olfactory, and combined visual-olfactory notifications in a messaging application. We demonstrated that olfactory notifications improve users' confidence and performance in identifying the urgency level of a message, with the same reaction time and disruption levels as for visual notifications. We discuss the design implications and opportunities for future work in the domain of multimodal interactions.
Conference Paper
Safe manual driving performance following takeovers in conditionally automated driving systems is impeded by a lack in situation awareness, partly due to an inappropriate trust in the system's capabilities. Previous work has indicated that the communication of system uncertainties can aid the trust calibration process. However, it has yet to be investigated how the information is best conveyed to the human operator. The study outlined in this publication presents an interface layout to visualise function-specific uncertainty information in an augmented reality display and explores the suitability of 11 visual variables. 46 participants completed a sorting task and indicated their preference for each of these variables. The results demonstrate that particularly colour-based and animation-based variables, above all hue, convey a clear order in terms of urgency and are well-received by participants. The presented findings have implications for all augmented reality displays that are intended to show content varying in urgency.
Conference Paper
Recent findings have indicated that the communication of uncertainties is a promising approach for overcoming human factors challenges associated with overtrust issues. The existing approaches, however, are limited in that they require operators to monitor the instrument cluster to perceive changes. As a consequence, significant changes may be missed and operators are regularly interrupted in the execution of non-driving related tasks even if the system is performing well. To overcome this, unobtrusive interfaces are required that are only interruptive if needed. This paper presents a lab-based study aiming at the preliminary evaluation of haptic variables for communicating automation uncertainties using a haptic vehicle seat. The initial results indicate that particularly increases in amplitude as well as a rhythm consisting of long vibrations separated by short breaks are well suited for communicating the exceedance of specified uncertainty thresholds. The communication of decreases in uncertainty using vibration cannot be recommended.