Conference PaperPDF Available

Scoping Out the Scalability Issues of Autonomous Vehicle-Pedestrian Interaction

Authors:

Abstract and Figures

Autonomous vehicles (AVs) may use external interfaces, such as LED light bands, to communicate with pedestrians safely and intuitively. While previous research has demonstrated the effectiveness of these interfaces in simple traffic scenarios involving one pedestrian and one vehicle, their performance in more complex scenarios with multiple road users remains unclear. The scalability of AV external communication has therefore attracted increasing attention, prompting the need for further investigation. This scoping review synthesises information from 54 papers to identify seven key scalability issues in multi-vehicle and multi-pedestrian environments, with Clarity of Recipients, Information Overload, and Multi-Lane Safety emerging as the most pressing concerns. To guide future research in scalable AV-pedestrian interactions, we propose high-level design directions focused on three communication loci: vehicle, infrastructure, and pedestrian. Our work contributes the groundwork and a roadmap for designing simplified, coordinated, and targeted external AV communication, ultimately improving safety and efficiency in complex traffic scenarios.
Content may be subject to copyright.
Scoping Out the Scalability Issues of Autonomous
Vehicle-Pedestrian Interaction
Tram Thi Minh Tran
tram.tran@sydney.edu.au
Design Lab, Sydney School of
Architecture, Design and Planning,
The University of Sydney
Sydney, NSW, Australia
Callum Parker
callum.parker@sydney.edu.au
Design Lab, Sydney School of
Architecture, Design and Planning,
The University of Sydney
Sydney, NSW, Australia
Martin Tomitsch
martin.tomitsch@sydney.edu.au
Design Lab, Sydney School of
Architecture, Design and Planning,
The University of Sydney
Sydney, NSW, Australia
ABSTRACT
Autonomous vehicles (AVs) may use external interfaces, such as
LED light bands, to communicate with pedestrians safely and intu-
itively. While previous research has demonstrated the eectiveness
of these interfaces in simple trac scenarios involving one pedes-
trian and one vehicle, their performance in more complex scenarios
with multiple road users remains unclear. The scalability of AV
external communication has therefore attracted increasing atten-
tion, prompting the need for further investigation. This scoping
review synthesises information from 54 papers to identify seven
key scalability issues in multi-vehicle and multi-pedestrian envi-
ronments, with Clarity of Recipients, Information Overload, and
Multi-Lane Safety emerging as the most pressing concerns. To guide
future research in scalable AV-pedestrian interactions, we propose
high-level design directions focused on three communication loci:
vehicle, infrastructure, and pedestrian. Our work contributes the
groundwork and a roadmap for designing simplied, coordinated,
and targeted external AV communication, ultimately improving
safety and eciency in complex trac scenarios.
CCS CONCEPTS
General and reference Surveys and overviews.
KEYWORDS
autonomous vehicles, external communication, eHMIs, vulnerable
road users, vehicle-pedestrian interaction, scalability
ACM Reference Format:
Tram Thi Minh Tran, Callum Parker, and Martin Tomitsch. 2023. Scoping
Out the Scalability Issues of Autonomous Vehicle-Pedestrian Interaction. In
15th International Conference on Automotive User Interfaces and Interactive
Vehicular Applications (AutomotiveUI ’23), September 18–22, 2023, Ingolstadt,
Germany. ACM, New York, NY, USA, 11 pages. https://doi.org/10.1145/
3580585.3607167
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for prot or commercial advantage and that copies bear this notice and the full citation
on the rst page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specic permission
and/or a fee. Request permissions from permissions@acm.org.
AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany
©2023 Copyright held by the owner/author(s). Publication rights licensed to ACM.
ACM ISBN 979-8-4007-0105-4/23/09. . . $15.00
https://doi.org/10.1145/3580585.3607167
1 INTRODUCTION
Autonomous vehicles (AVs) are often hailed as a safer alternative to
human drivers because they can react faster and more accurately
to hazards on the road [
3
,
33
]. However, before AVs can be safely
integrated into our transportation system, certain roadblocks must
be overcome, including the ability of AVs to communicate and
interact seamlessly with vulnerable road users (VRUs), such as
pedestrians [
36
,
70
]. The absence of interpersonal cues, such as
drivers’ hand signals and eye contact, is likely to cause uncertainty
and misunderstanding for pedestrians in ambiguous situations. To
address this challenge, researchers and the automotive industry
are investigating the use of external Human-Machine Interfaces
(eHMIs) as a way for AVs to communicate with pedestrians [
22
,
24
,
74
]. For example, the Mercedes-Benz F015 concept projects
a pedestrian crossing onto the road to provide pedestrians with
important information about the vehicle’s intended actions [63].
The study of AV external communication has seen signicant
growth in recent years, leading to a plethora of eHMI design con-
cepts that can be classied in various ways [
24
]. These interfaces are
often evaluated using scenario-based approaches, initially focusing
on basic encounters between a single AV and a pedestrian [
20
,
83
].
Preliminary evidence suggests that eHMIs may help to alleviate
uncertainty [
44
] and foster trust [
32
] in AVs. As such, the automo-
tive research community within the eld of HCI is increasingly
exploring the scalability of eHMIs in more complex trac scenarios
involving multiple road users.
Existing literature on AV external communication includes sev-
eral notable works that delve into the scalability of eHMIs. In their
literature analysis, Colley et al
. [20]
examined evaluation setups of
38 studies with regard to scalability factors (number of pedestri-
ans, vehicles, lanes, and noise). Colley and Rukzio
[18]
proposed
a design space that incorporated situational factors that aect the
scalability of eHMIs, such as communication relationships (one-
to-one, one-to-many, many-to-one, and many-to-many). Dey et al
.
[24]
developed a classication taxonomy for eHMIs and highlighted
scalability-related dimensions of a design concept, including com-
munication strategy (mass or targeted), communication resolution
(clarity of the intended recipients), and the number of users that
can be addressed at a time. However, to date, no comprehensive
review of the scalability issues that may arise has been conducted,
limiting a thorough understanding of the community’s collective
challenges to inform future research eorts.
In this study, we conducted a scoping review of AV external
communication from 2014 - 2022 to identify scalability issues that
have been discussed and/or empirically observed. These issues are
167
AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany Tran et al.
categorised into two groups: those caused by the increase in the
number of AVs (as well as mixed trac
1
) and those caused by the
increase in the number of pedestrians. Considering the most press-
ing issues, we pinpointed essential communication qualities for
eHMIs to excel in complex trac scenarios and propose high-level
research directions focused on three communication loci: vehicle,
infrastructure, and pedestrian.
Contribution Statement: This study contributes to the domain
of AV external communication within HCI by oering a compre-
hensive review of the scalability issues of eHMIs in complex trac
scenarios. Our analysis illustrates the attention these issues have
received within the automotive research community and introduces
high-level approaches to address them. The study aims to inform
the development of more eective and scalable eHMIs, ultimately
contributing to the safe integration of AVs into the transport sys-
tems of future cities.
2 METHODS
We conducted a scoping review, a preliminary assessment of the
available literature on a particular topic, following the methodolog-
ical framework proposed by Arksey and O’Malley [5].
2.1 Data Sources and Search Strategies
We retrieved relevant literature on AV external communication
from four major databases: ACM Digital Library, IEEE Xplore, Sci-
enceDirect, and Google Scholar. We performed a keyword search
(within the title, abstract, and author keywords) using a query
adapted from Dey et al
. [24]
:‘autonomous OR automated OR self-
driving OR driverless AND car OR vehicle AND pedestrian AND
interface OR interaction OR communication’ (see Appendix A for
the exact query syntax used for each database). We did not use
‘scalability’ as a keyword because not all publications explicitly
used this term. We limited the search results to publications dated
between 2014 and 2022, which corresponds to a period of acceler-
ated development in eHMI concepts [
24
]. The last search date was
October 22, 2022.
Our search query yielded 1305 entries (ACM = 233, IEEE Xplore
= 461, ScienceDirect = 111, Google Scholar
2
= 500). After importing
research results to a spreadsheet and removing 230 duplicates, 1075
publications remained to be screened.
2.2 Eligibility Criteria and Paper Selection
We included papers that met the following criteria:
Language: The paper must be written in English, the primary
language of the research team.
Publication type: We considered full conference papers, late-
breaking work (work in progress), journal articles, and tech-
nical reports, allowing for a comprehensive and timely un-
derstanding of the topic.
Relevance: The paper should focus on AV external commu-
nication. Furthermore, it must discuss and/or investigate the
1
Mixed trac refers to situations in which AVs must interact with human-driven
vehicles on the same roads.
2
The search on Google Scholar returned a large number of entries 19,100; therefore,
we decided to conclude the search at page 50 after three consecutive pages of not
nding any relevant entries.
scalability of eHMIs in complex trac scenarios involving
multiple vehicles or pedestrians.
Our goal was to identify not only distinct arguments but also
monitor the frequency of scalability discussions, leading us to in-
clude publications with similar viewpoints. We excluded papers
that, despite featuring a multi-user setup, did not contain any in-
formation about the scalability of eHMIs. For example, the virtual
environment in the study by Böckle et al
. [11]
included a vehicle in
the adjacent lane alongside the autonomous shuttle; however, the
presence of the additional vehicle was not the main focus of the
research, it had no textual description, and it did not demonstrate
any reported impact on the ndings.
The rst author conducted the screening of the 1075 papers in
two stages. In the initial stage, titles and abstracts were assessed for
their relevance to AV external communication, resulting in 230 pa-
pers. The subsequent stage involved a thorough review of full-text
papers to determine the presence of eHMI scalability information,
resulting in 46 papers. Additional eligible papers were incorporated
from the references of the aforementioned related works [
18
,
20
,
24
],
as well as from recently published papers not initially captured in
the search. This brought the total number of publications to be
analysed to 54. A PRISMA (Preferred Reporting Items for System-
atic Reviews and Meta-Analyses) ow diagram, which illustrates
the inclusion and exclusion of papers throughout each stage of the
review process, along with a list of included publications, can be
found in Appendix B.
2.3 Data Charting and Data Analysis
The rst author charted the following details from each included
paper: author, year, title and eHMI scalability information (quoted in
the authors’ original words). For papers featuring empirical studies
that examined eHMIs in complex trac situations, additional data
on eHMI design concept, eHMI placement (locus), positions of
additional pedestrians, trac type (mixed or all AVs), and evaluation
method were also recorded.
During the familiarisation phase with the data, it was observed
that the information related to eHMI scalability issues was largely
well-dened, suggesting a lower likelihood of varying interpreta-
tions. For instance, a statement like ‘the increase in the cognitive
load imposed on a pedestrian as the number of AVs with LED boards
increases’ [
72
] denotes an issue of information overload. As a re-
sult, a single coder with subject matter expertise was engaged to
ensure uniformity throughout the analysis process. The rst author
undertook a thematic analysis [
12
] of the collected scalability infor-
mation, applying the anity diagramming method. This analysis
process employed a bottom-up approach, consolidating discussions
around similar issues and resulting in the identication of seven
distinct themes corresponding to seven scalability issues: Informa-
tion Overload,Multi-Lane Safety,Audibility,Clarity of Recipients,
Group Inuence,Visibility, and Privacy. Additionally, a top-down
approach was utilised to categorise these issues into two groups:
those that arise in scenarios involving multiple vehicles, and those
surfacing in scenarios with multiple pedestrians.
To mitigate the risk of bias or oversight in this setup, weekly
discussions were scheduled with the second author during the cod-
ing process. Upon identifying the nal issues, they were presented,
168
eHMI Scalability Issues AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany
along with their associated scenarios and underlying causes, to the
remaining authors to determine whether they agreed with these
ndings or had additional insights. Although the identied scalabil-
ity issues remained unchanged after the validation process, several
critical discussions emerged. Primarily, the top-down division may
inadvertently simplify the intricate nature of the interactions at
play. Indeed, some issues may not be strictly conned to either a
multi-vehicle or a multi-pedestrian scenario; instances where they
cross these boundaries exist. Consider Information Overload, this
issue is classied under multi-vehicle scenarios, but it could also
occur in a multi-pedestrian situation. Imagine a single AV that must
communicate with multiple pedestrians dispersed in various loca-
tions. Simultaneously processing these multiple signals could lead
to information overload for the pedestrian. Despite this potential
overlap, we maintain the division to highlight the context where
the issue is most likely to surface, thus facilitating a more targeted
approach to addressing these issues.
3 RESULTS
3.1 Scalability Issues
In Table 1, we categorised eHMI-related scalability issues into those
relating to multiple vehicles and those concerning multiple pedestri-
ans. We also highlighted empirical studies that examined eHMIs
in multi-vehicle and/or multi-pedestrian scenarios. In Table 2, we
summarised the commonalities of these empirical studies.
3.1.1 Information Overload. This concern has been raised as early
as initial investigations into eHMIs. The term generally refers to
a situation where the amount of information provided exceeds a
person’s cognitive processing capacity, resulting in poor decision
quality [
30
,
37
]. Moore et al
. [65]
were among the scholars arguing
that the eHMIs might transform street-crossing into an ‘analytical
process’ and suggested the use of implicit motion cues for routine
interactions. In classifying 70 eHMI concepts proposed by industry
and academia, Dey et al
. [24]
also found that many designs utilised
multiple modalities and displays (i.e., communication devices) and
conveyed numerous messages. The authors thus emphasised the
crucial need for creating a balance between leveraging redundancy
(to foster interpretation and accuracy of use) and avoiding informa-
tion overload. Mahadevan et al
. [62]
conrmed this issue, where
the study participants responded unfavourably to a mixed interface
consisting of LED lights on the street, a printed hand mounted on
the vehicle, and an auditory alert from a smartphone. The interface
was deemed confusing and time-consuming as pedestrians had to
wait for all go-ahead signals to be activated. Nonetheless, reducing
the number of stimuli may not necessarily result in a reduced cog-
nitive load. For example, participants in a study by Dey et al
. [23]
preferred to have eHMIs communicate an AV’s intent at all times
rather than only during interactions (e.g., giving way).
A proliferation of AVs on urban streets is expected to exac-
erbate the issue of information overload in several ways. First,
without a standardised universal set of symbols, manufacturers
may adopt dierent interface designs to communicate a vehicle’s
awareness and intents [
87
]. Second, a dynamic trac scenario
may involve AVs with various levels of automation (e.g., partially
and fully automated) [
47
,
48
] and in dierent states of operation
(e.g., cruising and stopping) [
23
]. The increased number of inter-
faces and their myriad dierences are expected to cause a signi-
cant amount of visual clutter and demand greater attention from
pedestrians [
23
,
47
,
72
,
76
,
88
]. Information overload could easily
lead to the inability to identify the most critical pieces of informa-
tion [
23
,
73
] and, thus, pose a high level of safety risks [
47
,
48
].
Trac eciency may also be adversely aected in multi-lane cross-
ing scenarios where pedestrians must process multiple cues to cross
the road safely [
46
,
89
]. Holländer et al
. [46]
found that the indi-
vidual AV signals (e.g., projected crosswalks) caused participants
to stay longer in the rst lane while waiting for the second lane
to clear. Consequently, several studies have aimed to combine in-
formation from multiple vehicles and provide pedestrians with an
augmented reality (AR) safety corridor [41,80,84].
3.1.2 Multi-Lane Safety. In a mixed trac environment, pedestri-
ans are likely to encounter AVs with varying degrees of automation
alongside conventional vehicles. In light of this, Mok et al
. [64]
and Moore et al
. [65]
mentioned the risks of eHMIs diverting pedes-
trian attention away from other road users and their motion cues.
A VR-based study investigating pedestrian crossing behaviour in
mixed trac [
61
], however, revealed that participants acted more
cautiously when encountering vehicles that did not communicate
explicitly (i.e., those without an interface).
Multi-vehicle scenarios and mixed trac conditions also necessi-
tate caution in selecting the message perspective for eHMIs. Studies
have found that egocentric messages suggesting a pedestrian action,
such as ‘Walk’, have higher clarity ratings and faster response times
compared to allocentric messages communicating the AV’s intent
and states, such as ‘Braking’ [
29
]. As stated by Eisma et al
. [29]
,
the former perspective explains why AVs are stopping and does
not require pedestrians to shift their mental perspective. Nonethe-
less,eHMI-design best practices are moving away from the use of
advice and instructions due to concerns about safety and liability
issues (e.g., ISO/TR 23049:2018 [
35
]). Firstly, instructional eHMIs
may lessen pedestrians’ responsibility in making crossing deci-
sions, leading to an overreliance on the user interface [
62
]. More
importantly, such eHMIs risk creating a false expectation of the
state of the surrounding (mixed) trac, which is beyond the AV’s
control [
4
,
22
,
24
,
39
,
44
,
54
]. For instance, a design concept where
a zebra-crossing is projected onto the area in front of the vehicle
could lead pedestrians to cross the subsequent lane when it might
not be safe to do so [
46
,
58
]. Given this, Löcken et al
. [58]
and Colley
et al
. [17]
emphasised the importance of AVs considering the status
of the other lanes when providing a crossing signal to pedestri-
ans. One example involves the concept of the omniscient narrator,
where a vehicle changes the communication message based on its
knowledge of the trac situation [19].
3.1.3 Audibility. Mahadevan et al
. [62]
posited that multiple un-
synchronised AVs communicating with pedestrians using audio
messages may result in unpleasant noise rather than benecial
information. Investigating a trac scenario with two AVs oering
auditory cues, Colley et al
. [19]
found that some participants heard
only one message or an echo when the vehicles arrived simulta-
neously. Furthermore, in densely populated urban areas, auditory
cues may be drowned out by the noise arising from the high vehicle
volume on the streets [61].
169
AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany Tran et al.
Table 1: Scalability issues of eHMIs in multi-vehicle and multi-pedestrian scenarios.
Scalability issues Studies*
(15) Information Overload [41] [46] [47] [73] [84][23] [24] [48] [62] [65] [72] [76] [87] [88] [89]
(14) Multi-Lane Safety [19] [46] [61][4] [17] [22] [24] [29] [39] [44] [54] [58] [64] [65]
(3) Audibility [19] [61][62]
(30) Clarity of Recipients
[
16
] [
27
] [
28
] [
43
] [
46
] [
47
] [
84
] [
90
][
1
] [
2
] [
9
] [
22
] [
24
] [
25
] [
26
] [
29
]
[38] [40] [44] [51] [52] [54] [56] [69] [72] [76] [78] [79] [80] [88]
(5) Group Inuence [15] [50] [61] [14][55]
(5) Visibility [85][6] [31] [71] [88]
(1) Privacy [57]
*Bold citations refer to empirical studies that examined eHMIs in multi-vehicle and/or multi-pedestrian scenarios.
Table 2: Empirical studies examining eHMIs in multi-vehicle or multi-pedestrian scenarios.
References eHMI Locus Pedestrians*Vehicles Environment**
Troel-Madec et al. [85] Lateral LED display Vehicle - Multi AVs Virtual reality (VR)
Rossi-Alvarez et al. [73] LED light band Vehicle - Multi AVs Test track
Hesenius et al. [41] Wearable AR Pedestrian - Multi AVs Image
Tran et al. [84] Wearable AR Pedestrian - Multi AVs VR
Colley et al. [19] Auditory message Vehicle - Multi AVs VR
Holländer et al. [46] Projection, Smart curbs
Vehicle, Infrastructure
Opposite side Multi AVs VR
Dietrich et al. [28] Projection, Signaling light Vehicle Opposite side - VR
Colley et al. [16] Texts on windshield Vehicle Opposite side - VR
Wilbrink et al. [90] LED light band Vehicle Same, Opposite side - Video
Holländer et al. [47] Smartphone Pedestrian Same side Multi AVs Video
Hoggenmüller et al. [43] LED light band Vehicle Same side - VR
Dey et al. [27] Contextual interfaces Vehicle Same side - VR
Colley et al. [15] LED light band Vehicle Group - VR
Joisten et al. [50] Walking man, Smiling face Vehicle Group - VR
Chen et al. [14] LED light band Vehicle Group Mixed trac Public roads
Mahadevan et al. [61]
Mixed (LED light, physical hand,
haptic cue, auditory message)
Vehicle, Infrastructure,
Pedestrian
Group Mixed trac VR
*Additional pedestrian(s) in the scenario: forming a group with the participant or standing at a distance (same side or opposite side).
**Evaluation environment
3.1.4 Clarity of Recipients. Directing eHMI communication mes-
sages at specic recipients is deemed imperative to ensure that
pedestrians are condent they are being addressed by the AV, anal-
ogous to the acknowledgement conveyed by human eye contact. A
study investigating dierent light-based eHMIs in a shared space
setting revealed that most participants noticed the signal but did not
assume it was intended for them, even though the scene involved
only one pedestrian [40].
The directedness of eHMIs may be further compromised when
multiple pedestrians are present, potentially leading to misunder-
standings about which pedestrian the AV is attempting to com-
municate with [
1
,
2
,
9
,
16
,
22
,
24
29
,
38
,
43
,
44
,
46
,
47
,
51
,
52
,
54
,
56
,
69
,
72
,
76
,
78
80
,
84
,
88
,
90
]. Studies have highlighted that the
presence of other pedestrians introduces uncertainty to crossing
scenarios [
90
] and reduces the communication clarity of eHMIs [
16
].
Moreover, non-directed eHMIs signicantly increased a pedes-
trian’s willingness to cross compared to the baseline condition with
no eHMI [
27
,
90
], even in scenarios where the yielding message
was intended for another [
27
]. This nding suggests that without
clear and directed communication, pedestrians are more likely to
misinterpret AV communication and respond inappropriately, re-
sulting in severe consequences in real-world trac situations [
27
].
Notably, the likelihood of undesirable or adverse outcomes may
increase when eHMIs give commands or instruct pedestrians to
cross [
1
,
22
,
29
,
44
,
52
,
54
,
72
,
76
,
79
]; a display that signals ‘Go
ahead’, for instance, could prompt non-targeted pedestrians to en-
gage in risky behaviour.
3.1.5 Group Influence. Research has indicated that pedestrians
crossing in groups tend to be more careless and pay less attention
to crosswalks and approaching trac [
70
]. It is, therefore, of value
to determine whether such group inuence remains when pedes-
trians interact with AVs equipped with eHMIs [
55
]. Mahadevan
et al
. [61]
investigated the scenario of AI-based agents with dier-
ent crossing behaviours (none, early crossers, and timely crossers)
crossing the road alongside the participants in mixed trac. Al-
though the agents’ behaviours did not signicantly impact the
participants’ willingness to cross, half of the participants indicated
that the agents may have aected their crossing strategy. In their
observation study, Chen et al
. [14]
found stronger evidence of the
prevalence of herd behaviour among pedestrians, with most in-
dividuals in a group crossing situation not making any eort to
observe the trac if another person was already present in the
crosswalk. To investigate whether the eect of eHMIs is stronger
than the inuence of group pedestrian behaviour, Colley et al
. [15]
examined a situation where virtual pedestrians in close proximity to
the participants ignored the AV’s non-yielding intention. Although
the authors were unable to determine which factor had a greater
170
eHMI Scalability Issues AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany
impact, they found that eHMIs continue to positively aect trust,
cognitive load, and communication quality.
It should be noted that the actual behaviour of other pedestrians,
rather than their mere presence, impacts participants’ crossing
behaviour [
15
,
16
,
50
]. For instance, a standing pedestrian group
that did not engage in any specic action (e.g., step onto the street to
cross) had no eect on participants’ crossing duration [
16
] and did
not induce any imitable behaviour [
50
]. Nonetheless, participants
in the study by Joisten et al
. [50]
felt that crossing the street in a
group increased their safety and condence as they believed the
AV could more easily detect a group than a single pedestrian.
3.1.6 Visibility. An eHMI display located on the exterior of a ve-
hicle or projected onto the road might present visibility issues,
especially in shared spaces where pedestrians approach from vari-
ous directions [
31
,
71
]. A co-design study conducted by Asha et al
.
[6]
also revealed that wheelchair users may encounter various vi-
sual obstructions, such as roadside elements and the presence of
other pedestrians. A line of multiple vehicles one after the other
further complicates the visibility of eHMIs. According to 3D simula-
tion results [
85
], a frontal eHMI display is optimal only if a vehicle
is the rst one in a lane. This issue has led to the incorporation
of multiple displays [
60
], a 360-degree LED light band [
13
], or a
roof-mounted cylindrical interface [88] in several eHMI concepts.
3.1.7 Privacy. In ride-sharing scenarios, directed eHMIs can help
pedestrians identify the AV that is trying to pick them up [
38
].
However, eHMIs that disclose private information have received
negative evaluations [
57
]. Rather, when the goal is to easily locate
a car in a crowd, using a distinctive colour or graphical symbol or
comprising a combination of letters and numbers has been deemed
acceptable [
13
,
43
,
57
]. Anthropomorphic robotic eyes that can
convey ne-grained turning directions may also aid in identica-
tion [38].
3.2 General Remarks
3.2.1 Research Foci. The Clarity of Recipients (55.6%, 30) has formed
the primary concern of the research community, followed by Infor-
mation Overload (27.8%, 15) and Multi-Lane Safety (25.9%, 14). This
may be the case since the issue of recipient clarity extends beyond
scenarios involving multiple pedestrians. In addition to pedestri-
ans, other VRUs, such as cyclists [
45
], and drivers of conventional
vehicles may also benet from AV explicit communication. The
inclusion of these diverse trac participants may signicantly in-
crease the complexity of ensuring that AV communication reaches
the intended road users. For instance, in examining the implicit
and explicit communication of an AV interacting simultaneously
with a pedestrian and a driver of a conventional vehicle, Hübner
et al
. [49]
highlighted an issue with recipient clarity, as a majority
of the participants reported feeling wrongly addressed when the
AV signalled the right of way to the other human road user.
Out of 54 selected papers, 16 (29.6%) were empirical studies,
suggesting that researchers are actively investigating eHMI scala-
bility and seeking evidence to support their hypotheses and ideas.
However, it also indicates that there may still be room for more em-
pirical research to solidify the knowledge base in this area. Within
this context, researchers have taken dierent approaches. Some
have assessed established design concepts, such as the LED light
bands, in more complex trac scenarios [
15
,
27
,
73
,
90
]. These
studies contribute valuable insights regarding the underlying con-
ditions, the severity of scalability issues, and whether they are
signicant enough to warrant further investigation. In contrast,
other researchers have proposed new design concepts to address
specic scalability issues, such as Smart Curbs [
46
] and wearable AR
solutions [
41
,
84
]. It should be noted that at the time of the review,
several new concepts had not undergone formal evaluation [
80
,
88
]
and were not counted as empirical studies. Nonetheless, the emer-
gence of these ideas further emphasised the growing interest and
innovation in scaling up AV external communication.
3.2.2 Evaluation Method. The identied empirical studies have
been largely conducted in a controlled laboratory environment,
with VR simulations being the most frequently used prototype
representation (62.5%). This nding suggests that VR is a suitable
test bed for exploring the scalability of eHMIs as it allows for the
design and evaluation of novel design concepts under various traf-
c scenarios while ensuring the safety of participants. According
to a review of VR studies on AV–pedestrian interaction [
83
], the
eectiveness of VR simulations could be further enhanced through
the use of more realistic AV driving behaviour and the inclusion
of other road users in the same scenario using coupled/distributed
simulators [
8
]. The latter approach is particularly appealing as it is
currently unclear how the use of human-controlled avatars rather
than AI virtual agents might impact group inuence and the clarity
of recipients.
Notably, two of the studies evaluated eHMIs in a more realis-
tic environment (i.e., a closed test track facility [
73
] and public
roads [
14
]). Rossi-Alvarez et al
. [73]
conrmed some problems pre-
viously identied in VR simulations [
84
], such as participants miss-
ing eHMI light patterns due to looking back and forth between
the two AVs. Meanwhile, in their public experiment, Chen et al
.
[14]
observed a stronger eect of group inuence on pedestrians’
crossing behaviour. In both studies, the use of physical prototypes
helped elucidate how the visibility of the interface was aected by
factors such as the sun or the angle at which it was viewed [
14
,
73
].
These ndings demonstrate the value of evaluating AV external
communication under real-world trac conditions.
4 DISCUSSION
Despite the growing number of relevant empirical studies in recent
years, the most pressing scalability issues of eHMIs have yet to be
fully resolved. As a rst step to structure future research eorts, we
propose overarching research directions aimed at designing scalable
AV-pedestrian interactions and reect on the study limitations.
4.1 Research Directions based on
Communication Locus
According to the comprehensive review by Dey et al
. [24]
, around
two-thirds of existing eHMI design concepts incorporated visual
and auditory cues onto vehicles. This underlines the necessity
for scalable vehicle-based eHMIs. Nevertheless, researchers have
also proposed extravehicular solutions such as urban infrastruc-
tures and pedestrians’ personal devices (e.g., wearables and smart-
phones) [
24
,
27
,
79
]. As a result, three distinct communication loci,
171
AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany Tran et al.
vehicle-based, infrastructure-based, and pedestrian device-based,
are being explored simultaneously as potential solutions to address
eHMI scalability issues. This section revisits the denition and char-
acteristics of communication locus, aiming to establish a common
understanding within the research community before delving into
their potential in facilitating simplied communication (presenting
information in a clear, concise, and easily understandable manner),
coordinated communication (synchronising and harmonising the
exchange of information between dierent systems), and targeted
communication (providing the right information to the right recip-
ients at the right time). These three communication qualities are
identied as particularly relevant in overcoming the most signi-
cant challenges related to eHMI scalability.
The locus of AV communication, or interface location, was orig-
inally dened by Mahadevan et al
. [62]
in a participatory design
study. This design dimension includes four categories based on
where participants placed their interfaces: (1) vehicle-only, (2) vehi-
cle and infrastructure, (3) vehicle and pedestrian, and (4) mixed. In
this proposal, while the vehicle is always responsible for communi-
cating with pedestrians, this responsibility may be shared with road
infrastructures (e.g., trac lights) and pedestrians’ personal devices
(e.g., smartphones). In the design space for AV external communica-
tion proposed by Colley and Rukzio
[18]
, the communication locus
remains the same, though the categories are revised to represent
independent placements, including those of the vehicle, infrastruc-
ture, and pedestrian. In this paper, we classied projection-based
eHMIs (e.g., [
66
]) as part of the vehicle category, mainly because
the projections originate from and reect the vehicle’s intentions
even though they are not physically located on the vehicle.
The primary distinction between infrastructure and pedestrian-
based eHMIs versus vehicle-based eHMIs is that the communication
signals are detached from the vehicle. Being detached eHMIs, there
exists two dierent roles infrastructures and personal devices can
play:
As communication outlets: In this role, infrastructures and
personal devices transmit communication signals from AVs
to pedestrians. For instance, Mahadevan et al
. [62]
used a
smartphone to relay the message ‘I can see you’ to pedestri-
ans [
62
]. However, their ndings showed that pedestrians
considered this type of message more reliable when it origi-
nated from the vehicle rather than a third-party source, as it
is closely connected to the vehicle’s operation.
As communication sources: In this second role, infrastruc-
tures serve to facilitate trac, such as the Starling Cross-
ing, which adjusts its road markings in real time to guide
pedestrians, cyclists, and drivers [
86
]. On the other hand,
pedestrian devices oer personal guidance for navigating
the environment; for example, a smartphone application
might provide on-screen directions to enhance pedestrian
situational awareness while crossing roads [47].
Distinguishing between these two roles is crucial because they
may have diverse safety and liability implications, varied user ex-
periences, and dierent technical requirements in terms of hard-
ware, software, and communication protocols. Furthermore, the
second role goes beyond the classic understanding of eHMIs, which
mainly involve external interfaces that convey AV intentions and
operational states to pedestrians and other VRUs. In the context
of designing scalable AV-pedestrian interaction, our focus will be
solely on the second role.
4.1.1 Vehicle-based: In densely populated urban areas, the pres-
ence of numerous vehicles each equipped with a unique eHMI can
potentially increase the visual and auditory stimuli, contributing
to an already dynamic and possibly overwhelming environment.
Given the abundance of proposed eHMI design concepts, standard-
isation emerges as an essential and imminent direction to achieve
simplied communication. Standardised cues not only make the
learning process easier for road users but also reduce the variety of
cues they encounter. This uniformity lessens the cognitive eort
involved in interpreting and reacting to eHMI signals. The notion
of less cognitive demand is based on the principle of cognitive econ-
omy, [
21
] where our minds seek to minimise the amount of mental
eort spent on processing stimuli. International Organisation for
Standardisation (ISO) has established a working group
3
(ISO/TC
22/SC 39/WG 8) dedicated to advanced driver assistance systems,
automated driving, and eHMI standardisation for external com-
munication. The process of standardising eHMIs is intricate and
ongoing, requiring close collaboration among diverse stakeholders
in the automotive industry. These stakeholders must determine
which dimensions to standardise while considering individual, cul-
tural, and regulatory dierences to ensure the eectiveness and
widespread adoption of the resulting standards.
One important research direction in the development of eHMIs
is determining under which circumstances eHMIs are truly neces-
sary, as opposed to maintaining an always-on mode of operation.
This approach may help to prevent excessive stimuli and conserve
system resources such as energy and processing power. In line with
this, Dey et al
. [23]
examined the need for an eHMI to communi-
cate an AV’s non-yielding intent. However, the ndings indicated a
preference for eHMIs that communicate an AV’s intent explicitly
at all times, highlighting the need to determine a threshold beyond
which the information provided becomes detrimental [62].
Another solution involves using interconnected eHMIs, where
a single vehicle communicates the intentions of all other vehicles
in the vicinity [
19
,
46
]. Such eHMIs, enabled by connected vehi-
cle technology, can potentially lower pedestrians’ cognitive load
by minimising the number of communicating AVs and enhance
pedestrians’ safety in heavy trac by tailoring the communication
to the prevailing trac conditions. However, preliminary ndings
have been mixed due to pedestrians’ unfamiliarity with the con-
cept of connected vehicles [
19
]. From a technical perspective, the
interconnected eHMI approach adds complexity to communica-
tion protocols, such as deciding which AV takes on the primary
communication role. Addressing potential challenges related to
data security and issues like delays or loss of connectivity is also
crucial to maintain pedestrian safety. As a result, further research
is necessary to determine the viability of interconnected eHMIs
as a design concept, taking into account both human factors and
implementation feasibility.
With respect to targeted communication, vehicle-based eHMIs
can direct their messages in a specic direction. For example, Gui
et al
. [38]
investigated how robotic eye gazes can convey an AV’s
3https://www.iso.org/committee/5391225.html, last accessed April 2023
172
eHMI Scalability Issues AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany
ne-grained moving directions, enabling pedestrians to clear the
path in an open space or determine if the AV (taxi) intends to pick
them up. The Volvo 360c concept incorporated directional warn-
ing sounds aimed at pedestrians who unexpectedly step onto the
road [
13
]. Kaup et al
. [53]
proposed the concept of directed signal
lamps capable of addressing multiple road users separately with dis-
crete light channels. Another approach to achieving targeted com-
munication involves presenting a pedestrian’s position on an eHMI
display in a simplied schematic form (e.g., a dot [
88
] or a strip [
67
])
or adjusting communication based on the distance between the
vehicle and the pedestrian (distance-dependent eHMIs) [
24
,
25
].
According to the taxonomy established by Dey et al
. [24]
, all these
design concepts can be classied as partially scalable, meaning they
can address multiple road users concurrently but only to a certain
extent. We argue that this limited degree of scalability may not
necessarily be viewed a drawback, given that most trac situa-
tions and interactions usually involve a manageable number of
road users. Instead, the more critical aspect is the eHMI’s ability
to deliver unambiguous information about whether it is safe for
specic road users to proceed. For example, road projections visual-
ising the precise location at which the vehicle will stop can clearly
convey the intended recipient [
23
]. As a result, high communication
resolution should be made a key design goal for eHMIs and must
be thoroughly assessed in scenarios that are likely to occur, such
as urban intersections and pick-up/drop-o zones.
4.1.2 Infrastructure-based: The majority of infrastructure-based
design concepts leverage existing trac elements that are both
familiar to pedestrians and universally understood, such as ze-
bra crossings [
42
,
59
,
86
]. By integrating interactive features into
these well-known elements, infrastructure-based designs may en-
able pedestrians to quickly adapt to the new systems without fac-
ing a steep learning curve. In situations with multiple vehicles,
infrastructure-based systems provide a centralised source of infor-
mation for pedestrians, making it easier to process the data without
having to attend to multiple vehicle-based displays simultaneously.
One primary advantage of infrastructure-based concepts in co-
ordinated and targeted communication lies in their integration of
adaptive trac management systems. These systems can dynami-
cally adjust trac control measures by leveraging real-time trac
data gathered from various sources, such as sensors, cameras, and
connected networks. For instance, the Starling Crossing [
86
] adapts
its conguration to dierent road users, environmental conditions,
times of day, trac volumes, and pedestrian behaviours, includ-
ing distractions or haste. Similarly, Smart Curbs [
46
] extend the
traditional use of in-ground LED trac signal lights [
68
]. They
detect approaching cars and use LEDs embedded in the curbs to
communicate crossing instructions to pedestrians based on their
current position. These systems can prioritise pedestrian crossings,
promoting city walkability while optimising trac ow [
46
]. Addi-
tionally, infrastructure-based concepts are often perceived by road
users as having more authority in facilitating trac [
84
], which
may contribute to a higher level of trust and compliance.
Despite the potential of smart infrastructures to enhance urban
environments and transportation systems, the research commu-
nity also acknowledges the challenges it presents, such as high
initial development costs, ongoing maintenance, and regular up-
grades, which can be particularly burdensome for resource-limited
cities and developing countries [
79
]. Other concerns include the
impact of signal interference and network congestion on the re-
liability of wireless communication [
79
], as well as the potential
for vandalism or tampering to cause system malfunctions or in-
accurate data [
19
]. Consequently, future directions should focus
on developing cost-eective, resilient solutions and strategically
implementing them in areas where they can signicantly improve
safety and eciency, such as pedestrian-dense environments and
accident-prone zones. Alongside these eorts, it is crucial to em-
phasise user-centred design and human factors research to prevent
overreliance on technology and ensure individuals maintain a sense
of personal responsibility while navigating trac.
4.1.3 Pedestrian-based: Leveraging wearables [
41
,
84
] and smart-
phones [
34
,
47
], these design concepts oer the potential to consol-
idate communication cues from multiple AVs and enable commu-
nication with numerous relevant pedestrians [
24
]. This approach
enables tailored communication based on individual preferences
and needs and supports context-aware communication by account-
ing for factors such as the user’s location, speed, and direction of
movement. Furthermore, the advent of connected vehicle technol-
ogy may allow personal devices to function as virtual pedestrian
light systems, creating an innovative, infrastructure-free means of
requesting pedestrian right-of-way at intersections [82].
While Vehicle-to-Pedestrian (V2P) smartphone applications con-
tinue to be the primary focus in research for AV-pedestrian commu-
nication [
75
], there is a growing interest in wearable AR technology
as an emerging research avenue [
41
,
69
,
79
81
,
84
]. This interest
stems from the potential advantages of wearable AR devices over
traditional smartphone applications, such as oering a more im-
mersive and intuitive experience, enhancing situational awareness,
and enabling hands-free operation [
77
]. Furthermore, wearable AR
can bolster spatial awareness by delivering contextual information
about the user’s environment, including the location and trajectory
of nearby vehicles, potential hazards [
81
], or suggested routes [
41
].
However, wearable AR devices may face challenges in terms of
long-term use or user acceptance due to their form factor, weight,
and design [
7
,
10
]. Additionally, these devices, by design, can collect
more personal data like location and gaze direction compared to
smartphones, potentially raising privacy concerns for some users.
More research is needed to address these challenges and develop
user-friendly, privacy-preserving solutions that can facilitate wide-
spread adoption. However, in order to foster inclusivity and equity
in AV-pedestrian communication, it is crucial not to rely solely on
wearable AR technologies [79].
4.1.4 Summary. A comparative analysis of three communication
loci—vehicle, infrastructure, and pedestrian based on their poten-
tial for simplied, coordinated, and targeted communication is
summarised in Table 3. In the pursuit of designing scalable AV-
pedestrian interactions, each of these loci exhibits distinct benets
and drawbacks. However, most of them are derived from theoret-
ical reasoning and assumptions rather than empirical data. This
necessitates further research and validation through real-world
experiments, user studies, and data-driven approaches. Addition-
ally, exploring multiple communication loci rather than focusing
173
AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany Tran et al.
Table 3: Overview of potential research directions for enhancing scalability in AV external communication: A comparative
analysis based on Vehicle, Infrastructure, and Pedestrian loci.
VEHICLE
INFRASTRUCTURE
PEDESTRIAN
Simplied Standardised cues
On-demand operation
Interconnected eHMIs
Central source of information Cue consolidation
Coordinated Interconnected eHMIs Adaptive trac management V2P applications
Targeted High-resolution
communication
Instructions based on
pedestrian position
Tailored communication
via wearables/smartphones
on just one enables the development of more adaptable, reliable,
and inclusive eHMI solutions that cater to a wide range of situa-
tions, user needs, and abilities. By considering the strengths and
weaknesses of each communication locus, researchers can iden-
tify potential synergies and develop more eective, evidence-based
designs that enhance the safety and eciency of AV-pedestrian
interactions in diverse contexts. In fact, a hybrid loci approach,
which incorporates built-in redundancy and increased reliability,
has been previously explored and recommended in the works of
Mahadevan et al
. [62]
and Löcken et al
. [59]
, demonstrating the
potential value of combining multiple communication loci in eHMI
solutions.
4.2 Limitations
Our work has several limitations. Primarily, the comprehensive-
ness of our review could be impacted by our choice of keywords
and selection strategy, potentially resulting in the omission of rele-
vant studies. Moreover, the lack of multiple coders independently
analysing the data could raise questions about the consistency of
data interpretation and limit the generalisability of our ndings.
Within the scope of this paper, our primary emphasis is on AV-
pedestrian interactions. We made a conscious decision to not delve
into the complex dynamics involving other road users such as cy-
clists, motorbike riders, and conventional drivers. This focus, whilst
necessary for the initial exploration, might lead to an incomplete
understanding of the overall communication ecosystem and limit
our insights on the scalability of eHMIs across dierent types of
road users.
5 CONCLUSION
Our review revealed that since the 2020 study by Colley et al
. [20]
,
which reported eHMIs being predominantly evaluated in simple
trac scenarios, there has been a growing focus within the au-
tomotive research community on the ability of eHMIs to handle
larger numbers of pedestrians and vehicles in complex trac en-
vironments. Analysing 54 papers discussing scalability issues, in-
cluding 16 empirical studies examining eHMIs in multi-vehicle or
multi-pedestrian situations, allowed us to identify and organise the
challenges faced by AV external communication. Our review oers
researchers insight into the possible challenges of implementing
eHMIs in real-world situations. Additionally, we have highlighted
essential communication qualities required for eective eHMIs in
complex situations and proposed high-level research directions
focused on three communication loci: vehicle, infrastructure, and
pedestrian, to guide future eorts in this domain.
ACKNOWLEDGMENTS
This research is supported by an Australian Government Research
Training Program (RTP) Scholarship and through the ARC Discov-
ery Project DP200102604, Trust and Safety in Autonomous Mobility
Systems: A Human-centred Approach. We extend our gratitude to
the anonymous reviewers for their insightful comments and sugges-
tions. Additionally, we thank the coordinator for their invaluable
assistance throughout the shepherding process. Their eorts have
signicantly contributed to the nal version of this paper.
REFERENCES
[1]
Claudia Ackermann, Matthias Beggiato, Sarah Schubert, and Josef F Krems. 2019.
An experimental study to investigate design and assessment criteria: What is
important for communication between pedestrians and automated vehicles?
Applied Ergonomics 75 (2019), 272–282. https://doi.org/10.1016/j.apergo.2018.11.
002
[2]
Sander Ackermans, Debargha Dey, Peter Ruijten, Raymond H. Cuijpers, and Bas-
tian Peging. 2020. The Eects of Explicit Intention Communication, Conspicuous
Sensors, and Pedestrian Attitude in Interactions with Automated Vehicles. In
Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems
(Honolulu, HI, USA) (CHI ’20). Association for Computing Machinery, New York,
NY, USA, 1–14. https://doi.org/10.1145/3313831.3376197
[3]
James M Anderson, Kalra Nidhi, Karlyn D Stanley, Paul Sorensen, Constantine
Samaras, and Oluwatobi A Oluwatola. 2014. Autonomous vehicle technology: A
guide for policymakers. Rand Corporation, Santa Monica, CA, USA.
[4]
Jonas Andersson, Azra Habibovic, Maria Klingegård, Cristofer Englund, and
Victor Malmsten-Lundgren. 2017. Hello human, can you read my mind? ERCIM
News 109 (2017), 36–37.
[5]
Hilary Arksey and Lisa O’Malley. 2005. Scoping studies: towards a methodological
framework. International Journal of Social Research Methodology 8, 1 (2005), 19–32.
https://doi.org/10.1080/1364557032000119616
[6]
Ashratuz Zavin Asha, Christopher Smith, Georgina Freeman, Sean Crump,
Sowmya Somanath, Lora Oehlberg, and Ehud Sharlin. 2021. Co-Designing In-
teractions between Pedestrians in Wheelchairs and Autonomous Vehicles. In
Designing Interactive Systems Conference 2021 (Virtual Event, USA) (DIS ’21).
Association for Computing Machinery, New York, NY, USA, 339–351. https:
//doi.org/10.1145/3461778.3462068
[7]
Ronald T. Azuma. 2019. The road to ubiquitous consumer augmented reality
systems. Human Behavior and Emerging Technologies 1, 1 (2019), 26–32. https:
//doi.org/10.1002/hbe2.113
[8]
Pavlo Bazilinskyy, Lars Kooijman, Dimitra Dodou, and Joost CF de Winter. 2020.
Coupled simulator for research on the interaction between pedestrians and
(automated) vehicles. In Driving Simulation Conference Europe. Driving Simulation
Association, Antibes, France, 2 pages.
174
eHMI Scalability Issues AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany
[9]
Ola Benderius, Christian Berger, and Victor Malmsten Lundgren. 2017. The
Best Rated Human–Machine Interface Design for Autonomous Vehicles in the
2016 Grand Cooperative Driving Challenge. IEEE Transactions on Intelligent
Transportation Systems 19, 4 (2017), 1302–1307. https://doi.org/10.1109/TITS.
2017.2749970
[10]
Mark Billinghurst. 2021. Grand challenges for augmented reality. Frontiers in
Virtual Reality 2 (2021), 12. https://doi.org/10.3389/frvir.2021.578080
[11]
Marc-Philipp Böckle, Anna Pernestål Brenden, Maria Klingegård, Azra Habi-
bovic, and Martijn Bout. 2017. SAV2P: Exploring the Impact of an Interface
for Shared Automated Vehicles on Pedestrians’ Experience. In Proceedings of
the 9th International Conference on Automotive User Interfaces and Interactive
Vehicular Applications Adjunct (Oldenburg, Germany) (AutomotiveUI ’17). As-
sociation for Computing Machinery, New York, NY, USA, 136–140. https:
//doi.org/10.1145/3131726.3131765
[12]
Virginia Braun and Victoria Clarke. 2006. Using thematic analysis in psychology.
Qualitative Research in Psychology 3, 2 (2006), 77–101. https://doi.org/10.1191/
1478088706qp063oa
[13]
Volvo Cars. 2018. 360c a new way to travel. https://w ww.volvocars.com/intl/
v/cars/concept-models/360c
[14]
Henry Chen, Robin Cohen, Kerstin Dautenhahn, Edith Law, and Krzysztof Czar-
necki. 2020. Autonomous Vehicle Visual Signals for Pedestrians: Experiments and
Design Recommendations. In 2020 IEEE Intelligent Vehicles Symposium (IV). IEEE,
New Jersey, NJ, USA, 1819–1826. https://doi.org/10.1109/IV47402.2020.9304628
[15]
Mark Colley, Elvedin Bajrovic, and Enrico Rukzio. 2022. Eects of Pedestrian
Behavior, Time Pressure, and Repeated Exposure on Crossing Decisions in Front
of Automated Vehicles Equipped with External Communication. In Proceedings of
the 2022 CHI Conference on Human Factors in Computing Systems (New Orleans,
LA, USA) (CHI ’22). Association for Computing Machinery, New York, NY, USA,
Article 367, 11 pages. https://doi.org/10.1145/3491102.3517571
[16]
Mark Colley, Jan Henry Belz, and Enrico Rukzio. 2021. Investigating the Eects
of Feedback Communication of Autonomous Vehicles. In 13th International Con-
ference on Automotive User Interfaces and Interactive Vehicular Applications (Leeds,
United Kingdom) (AutomotiveUI ’21). Association for Computing Machinery, New
York, NY, USA, 263–273. https://doi.org/10.1145/3409118.3475133
[17]
Mark Colley, Stefanos Can Mytilineos, Marcel Walch, Jan Gugenheimer, and
Enrico Rukzio. 2020. Evaluating Highly Automated Trucks as Signaling Lights.
In 12th International Conference on Automotive User Interfaces and Interactive
Vehicular Applications (Virtual Event, DC, USA) (AutomotiveUI ’20). Association
for Computing Machinery, New York, NY, USA, 111–121. https://doi.org/10.
1145/3409120.3410647
[18]
Mark Colley and Enrico Rukzio. 2020. A Design Space for External Communi-
cation of Autonomous Vehicles. In 12th International Conference on Automotive
User Interfaces and Interactive Vehicular Applications (Virtual Event, DC, USA)
(AutomotiveUI ’20). Association for Computing Machinery, New York, NY, USA,
212–222. https://doi.org/10.1145/3409120.3410646
[19]
Mark Colley, Marcel Walch, Jan Gugenheimer, Ali Askari, and Enrico Rukzio.
2020. Towards Inclusive External Communication of Autonomous Vehicles for
Pedestrians with Vision Impairments. In Proceedings of the 2020 CHI Conference on
Human Factors in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association
for Computing Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/
3313831.3376472
[20]
Mark Colley, Marcel Walch, and Enrico Rukzio. 2020. Unveiling the Lack of
Scalability in Research on External Communication of Autonomous Vehicles. In
Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing
Systems (Honolulu, HI, USA) (CHI EA ’20). Association for Computing Machinery,
New York, NY, USA, 1–9. https://doi.org/10.1145/3334480.3382865
[21]
Andrew M Colman. 2015. A dictionary of psychology. Oxford University Press,
Oxford, UK.
[22]
Joost de Winter and Dimitra Dodou. 2022. External human–machine interfaces:
Gimmick or necessity? Transportation Research Interdisciplinary Perspectives 15
(2022), 100643. https://doi.org/10.1016/j.trip.2022.100643
[23]
Debargha Dey, Azra Habibovic, Melanie Berger, Devanshi Bansal, Raymond H.
Cuijpers, and Marieke Martens. 2022. Investigating the Need for Explicit Com-
munication of Non-Yielding Intent through a Slow-Pulsing Light Band (SPLB)
EHMI in AV-Pedestrian Interaction. In Proceedings of the 14th International Con-
ference on Automotive User Interfaces and Interactive Vehicular Applications (Seoul,
Republic of Korea) (AutomotiveUI ’22). Association for Computing Machinery,
New York, NY, USA, 307–318. https://doi.org/10.1145/3543174.3546086
[24]
Debargha Dey, Azra Habibovic, Andreas Löcken, Philipp Wintersberger, Bastian
Peging, Andreas Riener, Marieke Martens, and Jacques Terken. 2020. Taming
the eHMI jungle: A classication taxonomy to guide, compare, and assess the
design principles of automated vehicles’ external human-machine interfaces.
Transportation Research Interdisciplinary Perspectives 7 (2020), 100174. https:
//doi.org/10.1016/j.trip.2020.100174
[25]
Debargha Dey, Kai Holländer, Melanie Berger, Berry Eggen, Marieke Martens,
Bastian Peging, and Jacques Terken. 2020. Distance-Dependent EHMIs for the
Interaction Between Automated Vehicles and Pedestrians. In 12th International
Conference on Automotive User Interfaces and Interactive Vehicular Applications
(Virtual Event, DC, USA) (AutomotiveUI ’20). Association for Computing Machin-
ery, New York, NY, USA, 192–204. https://doi.org/10.1145/3409120.3410642
[26]
Debargha Dey, Marieke Martens, Chao Wang, Felix Ros, and Jacques Terken.
2018. Interface Concepts for Intent Communication from Autonomous Vehicles to
Vulnerable Road Users. In Adjunct Proceedings of the 10th International Conference
on Automotive User Interfaces and Interactive Vehicular Applications (Toronto, ON,
Canada) (AutomotiveUI ’18). Association for Computing Machinery, New York,
NY, USA, 82–86. https://doi.org/10.1145/3239092.3265946
[27]
Debargha Dey, Arjen van Vastenhoven, Raymond H. Cuijpers, Marieke Martens,
and Bastian Peging. 2021. Towards Scalable EHMIs: Designing for AV-VRU
Communication Beyond One Pedestrian. In 13th International Conference on
Automotive User Interfaces and Interactive Vehicular Applications (Leeds, United
Kingdom) (AutomotiveUI ’21). Association for Computing Machinery, New York,
NY, USA, 274–286. https://doi.org/10.1145/3409118.3475129
[28]
André Dietrich, J-H Willrodt, Karolin Wagner, and Klaus Bengler. 2018. Projection-
Based External Human Machine Interfaces Enabling Interaction between Auto-
mated Vehicles and Pedestrians. In Proceedings of the Driving Simulation Confer-
ence 2018 Europe VR. Driving Simulation Association, Antibes, France, 43–50.
[29]
YB Eisma, A Rei, L Kooijman, D Dodou, and JCF De Winter. 2021. Exter-
nal human-machine interfaces: Eects of message perspective. Transporta-
tion research part F: trac psychology and behaviour 78 (2021), 30–41. https:
//doi.org/10.1016/j.trf.2021.01.013
[30]
Martin J Eppler and Jeanne Mengis. 2004. The Concept of Information Overload:
A Review of Literature from Organization Science, Accounting, Marketing, MIS,
and Related Disciplines. The Information Society 20, 5 (2004), 325–344. https:
//doi.org/10.1080/01972240490507974
[31]
Roja Ezzati Amini, Christos Katrakazas, Andreas Riener, and Constantinos Anto-
niou. 2021. Interaction of automated driving systems with pedestrians: challenges,
current solutions, and recommendations for eHMIs. Transport Reviews 41, 6 (2021),
788–813. https://doi.org/10.1080/01441647.2021.1914771
[32]
Stefanie M. Faas, Andrea C. Kao, and Martin Baumann. 2020. A Longitudinal
Video Study on Communicating Status and Intent for Self-Driving Vehicle
Pedestrian Interaction. In Proceedings of the 2020 CHI Conference on Human Factors
in Computing Systems (Honolulu, HI, USA) (CHI ’20). Association for Computing
Machinery, New York, NY, USA, 1–14. https://doi.org/10.1145/3313831.3376484
[33]
Daniel J Fagnant and Kara Kockelman. 2015. Preparing a nation for autonomous
vehicles: opportunities, barriers and policy recommendations. Transportation
Research Part A: Policy and Practice 77 (2015), 167–181. https://doi.org/10.1016/j.
tra.2015.04.003
[34]
Scott Fitzgerald. 2020. Australia’s rst vehicle-to-pedestrian technology
trial. https://imoveaustralia.com/news-articles/personal- public-mobility/
australia-rst- vehicle-to- pedestrian-technology- trial/
[35]
International Organization for Standardization. 2018. ISO/TR 23049:2018 Road
Vehicles Ergonomic aspects of external visual communication from automated
vehicles to other road users. https://www.iso.org/obp/ui/#iso:std:iso:tr:23049
[36]
Rebecca Greeneld. 2016. Google’s self-driving cars lack a human’s intuition for
what other drivers will do. https://slate.com/technology/2016/03/google- self-
driving-cars- lack-a- humans-intuition- for-what- other-drivers- will-do.html
[37]
Bertram Myron Gross. 1964. The managing of organizations: The administrative
struggle. Vol. 2. Free Press of Glencoe, New York, NY, USA.
[38]
Xinyue Gui, Koki Toda,Stela Hanbyeol Seo, Chia-Ming Chang, and Takeo Igarashi.
2022. “I Am Going This Way”: Gazing Eyes on Self-Driving Car Show Multiple
Driving Directions. In Proceedings of the 14th International Conference on Auto-
motive User Interfaces and Interactive Vehicular Applications (Seoul, Republic of
Korea) (AutomotiveUI ’22). Association for Computing Machinery, New York, NY,
USA, 319–329. https://doi.org/10.1145/3543174.3545251
[39]
Azra Habibovic, Victor Malmsten Lundgren, Jonas Andersson, Maria Klingegård,
Tobias Lagström, Anna Sirkka, Johan Fagerlönn, Claes Edgren, Rikard Fredriks-
son, Stas Krupenia, et al
.
2018. Communicating intent of automated vehicles to
pedestrians. Frontiers in psychology 9 (2018), 1336. https://doi.org/10.3389/fpsyg.
2018.01336
[40]
Ann-Christin Hensch, Isabel Neumann, Matthias Beggiato, Josephine Halama,
and Josef F Krems. 2019. Eects of a light-based communication approach as an
external HMI for Automated Vehicles–a Wizard-of-Oz Study. Transactions on
Transport Sciences 10, 2 (2019), 18–32. https://doi.org/10.5507/tots.2019.012
[41]
Marc Hesenius, Ingo Börsting, Ole Meyer, and Volker Gruhn. 2018. Don’t
Panic! Guiding Pedestrians in Autonomous Trac with Augmented Reality.
In Proceedings of the 20th International Conference on Human-Computer Inter-
action with Mobile Devices and Services Adjunct (Barcelona, Spain) (Mobile-
HCI ’18). Association for Computing Machinery, New York, NY, USA, 261–268.
https://doi.org/10.1145/3236112.3236148
[42]
Marius Hoggenmueller and Martin Tomitsch. 2019. Enhancing Pedestrian Safety
through In-Situ Projections: A Hyperreal Design Approach. In Proceedings of the
8th ACM International Symposium on Pervasive Displays (Palermo, Italy) (PerDis
’19). Association for Computing Machinery, New York, NY, USA, Article 29,
2 pages. https://doi.org/10.1145/3321335.3329682
[43]
Marius Hoggenmüller, Martin Tomitsch, and Stewart Worrall. 2022. Designing
Interactions with Shared AVs in Complex Urban Mobility Scenarios. Frontiers in
175
AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany Tran et al.
Computer Science 4 (2022), 59. https://doi.org/10.3389/fcomp.2022.866258
[44]
Kai Holländer, Ashley Colley, Christian Mai, Jonna Häkkilä, Florian Alt, and
Bastian Peging. 2019. Investigating the Inuence of External Car Displays
on Pedestrians’ Crossing Behavior in Virtual Reality. In Proceedings of the 21st
International Conference on Human-Computer Interaction with Mobile Devices and
Services (Taipei, Taiwan) (MobileHCI ’19). Association for Computing Machinery,
New York,N Y,USA, Article 27, 11 pages. https://doi.org/10.1145/3338286.3340138
[45]
Kai Holländer, Mark Colley, Enrico Rukzio, and Andreas Butz. 2021. A Taxonomy
of Vulnerable Road Users for HCI Based On A Systematic Literature Review. In
Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems
(Yokohama, Japan) (CHI ’21). Association for Computing Machinery, New York,
NY, USA, Article 158, 13 pages. https://doi.org/10.1145/3411764.3445480
[46]
Kai Holländer, Marius Hoggenmüller, Romy Gruber, Sarah Theres Völkel, and
Andreas Butz. 2022. Take It to the Curb: Scalable Communication Between
Autonomous Cars and Vulnerable Road Users Through Curbstone Displays.
Frontiers Comput. Sci. 4 (2022), 844245. https://doi.org/10.3389/fcomp.2022.844245
[47]
Kai Holländer, Andy Krüger, and Andreas Butz. 2020. Save the Smombies: App-
Assisted Street Crossing. In 22nd International Conference on Human-Computer
Interaction with Mobile Devices and Services (Oldenburg, Germany) (MobileHCI
’20). Association for Computing Machinery, New York, NY, USA, Article 22,
11 pages. https://doi.org/10.1145/3379503.3403547
[48]
Kai Holländer, Philipp Wintersberger, and Andreas Butz. 2019. Overtrust in
External Cues of Automated Vehicles: An Experimental Investigation. In Pro-
ceedings of the 11th International Conference on Automotive User Interfaces and
Interactive Vehicular Applications (Utrecht, Netherlands) (AutomotiveUI ’19). As-
sociation for Computing Machinery, New York, NY, USA, 211–221. https:
//doi.org/10.1145/3342197.3344528
[49]
Maximilian Hübner, Alexander Feierle, Michael Rettenmaier, and Klaus Bengler.
2022. External communication of automated vehicles in mixed trac: Addressing
the right human interaction partner in multi-agent simulation. Transportation
research part F: trac psychology and behaviour 87 (2022), 365–378. https:
//doi.org/10.1016/j.trf.2022.04.017
[50]
Philip Joisten, Ziyu Liu, Nina Theobald, Andreas Webler, and Bettina Abendroth.
2021. Communication of Automated Vehicles and Pedestrian Groups: An In-
tercultural Study on Pedestrians’ Street Crossing Decisions. In Proceedings of
Mensch Und Computer 2021. Association for Computing Machinery, New York,
NY, USA, 49–53. https://doi.org/10.1145/3473856.3474004
[51]
Anees Ahamed Kaleefathullah, Natasha Merat, Yee Mun Lee, Yke Bauke Eisma,
Ruth Madigan, Jorge Garcia, and Joost de Winter. 2022. External human–machine
interfaces can be misleading: an examination of trust development and misuse
in a CAVE-based pedestrian simulation environment. Human factors 64, 6 (2022),
1070–1085. https://doi.org/10.1177/0018720820970751
[52]
Christina Kaß, Stefanie Schoch, Frederik Naujoks, Sebastian Hergeth, Andreas
Keinath, and Alexandra Neukum. 2020. Standardized test procedure for external
Human–Machine Interfaces of automated vehicles. Information 11, 3 (2020), 173.
https://doi.org/10.3390/info11030173
[53]
Marc Kaup, Jan-Henning Willrodt, A Schieben, and Marc Wilbrink. 2019. inter-
ACT D4. 3–Final design and HMI solutions for the interaction of AVs with user
on-board and other trac participants ready for nal implementation.
[54]
Tobias Lagström and Victor Malmsten Lundgren. 2016. AVIP-Autonomous vehicles’
interaction with pedestrians-An investigation of pedestrian-driver communication
and development of a vehicle external interface. Master’s thesis. Chalmers Univer-
sity of Technology.
[55]
Yee Mun Lee, Ruth Madigan, Chinebuli Uzondu, Jorge Garcia, Richard Romano,
Gustav Markkula, and Natasha Merat. 2022. Learning to interpret novel eHMI:
The eect of vehicle kinematics and eHMI familiarity on pedestrian crossing
behavior. Journal of safety research 80 (2022), 270–280. https://doi.org/10.1016/j.
jsr.2021.12.010
[56]
Yeti Li, Murat Dikmen, Thana G. Hussein, Yahui Wang, and Catherine Burns.
2018. To Cross or Not to Cross: Urgency-Based External Warning Displays on
Autonomous Vehicles to Improve Pedestrian Crossing Safety. In Proceedings of
the 10th International Conference on Automotive User Interfaces and Interactive
Vehicular Applications (Toronto, ON, Canada) (AutomotiveUI ’18). Association for
Computing Machinery, New York, NY, USA, 188–197. https://doi.org/10.1145/
3239060.3239082
[57]
Dokshin Lim and Byungwoo Kim. 2022. UI Design of eHMI of Autonomous
Vehicles. International Journal of Human–Computer Interaction 38, 18-20 (2022),
1944–1961. https://doi.org/10.1080/10447318.2022.2061123
[58]
Andreas Löcken, Carmen Golling, and Andreas Riener. 2019. How Should Auto-
mated Vehicles Interact with Pedestrians? A Comparative Analysis of Interaction
Concepts in Virtual Reality. In Proceedings of the 11th International Conference
on Automotive User Interfaces and Interactive Vehicular Applications (Utrecht,
Netherlands) (AutomotiveUI ’19). Association for Computing Machinery, New
York, NY, USA, 262–274. https://doi.org/10.1145/3342197.3344544
[59]
Andreas Löcken, Philipp Wintersberger, Anna-Katharina Frison, and Andreas
Riener. 2019. Investigating User Requirements for Communication Between
Automated Vehicles and Vulnerable Road Users. In 2019 IEEE Intelligent Vehicles
Symposium (IV). IEEE, New Jersey, NJ, USA, 879–884. https://doi.org/10.1109/
IVS.2019.8814027
[60]
Cheyenne Macdonald. 2018. The self-driving car with screens
to warn pedestrians: Drive.ai launches its standout cars in Texas.
https://www.dailymail.co.uk/sciencetech/article-6008793/The-self-driving-car-
screens-warn- pedestrians-Drive- ai-launches- standout-cars- Texas.html
[61]
Karthik Mahadevan, Elaheh Sanoubari, Sowmya Somanath, James E. Young,
and Ehud Sharlin. 2019. AV-Pedestrian Interaction Design Using a Pedestrian
Mixed Trac Simulator. In Proceedings of the 2019 on Designing Interactive Systems
Conference (San Diego, CA, USA) (DIS ’19). Association for Computing Machinery,
New York, NY, USA, 475–486. https://doi.org/10.1145/3322276.3322328
[62]
Karthik Mahadevan, Sowmya Somanath, and Ehud Sharlin. 2018. Communi-
cating Awareness and Intent in Autonomous Vehicle-Pedestrian Interaction. In
Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
(Montreal QC, Canada) (CHI ’18). Association for Computing Machinery, New
York, NY, USA, 1–12. https://doi.org/10.1145/3173574.3174003
[63]
Mercedes-Benz. 2015. The Mercedes-Benz F 015 Luxury in Mo-
tion. https://www.mercedes-benz.com.au/passengercars/mercedes-benz-
cars/campaigns/mercedes-benz- f-015.html
[64]
Chun Sang Mok, Pavlo Bazilinskyy, and Joost de Winter. 2022. Stopping by
looking: A driver-pedestrian interaction study in a coupled simulator using head-
mounted displays with eye-tracking. Applied Ergonomics 105 (2022), 103825.
https://doi.org/10.1016/j.apergo.2022.103825
[65]
Dylan Moore, Rebecca Currano, G. Ella Strack, and David Sirkin. 2019. The Case
for Implicit External Human-Machine Interfaces for Autonomous Vehicles. In
Proceedings of the 11th International Conference on Automotive User Interfaces
and Interactive Vehicular Applications (Utrecht, Netherlands) (AutomotiveUI ’19).
Association for Computing Machinery, New York, NY, USA, 295–307. https:
//doi.org/10.1145/3342197.3345320
[66]
Trung Thanh Nguyen, Kai Holländer, Marius Hoggenmueller, Callum Parker, and
Martin Tomitsch. 2019. Designing for Projection-Based Communication between
Autonomous Vehicles and Pedestrians. In Proceedings of the 11th International
Conference on Automotive User Interfaces and Interactive Vehicular Applications
(Utrecht, Netherlands) (AutomotiveUI ’19). Association for Computing Machinery,
New York, NY, USA, 284–294. https://doi.org/10.1145/3342197.3344543
[67]
Nissan. 2015. Nissan IDS Concept: Nissan’s vision for the future of EVs and
autonomous driving. https://europe.nissannews.com/en- GB/releases/release-
139047
[68]
Korea Now. 2021. This is how Korea innovates to prevent trac accidents.
https://www.youtube.com/watch?v=op6L4cW4oIQ
[69]
F Gabriele Pratticò, Fabrizio Lamberti, Alberto Cannavò, Lia Morra, and Paolo
Montuschi. 2021. Comparing State-of-the-Art and Emerging Augmented Reality
Interfaces for Autonomous Vehicle-to-Pedestrian Communication. IEEE Transac-
tions on Vehicular Technology 70, 2 (2021), 1157–1168. https://doi.org/10.1109/
TVT.2021.3054312
[70]
Amir Rasouli and John K. Tsotsos. 2020. Autonomous Vehicles That Interact With
Pedestrians: A Survey of Theory and Practice. IEEE Transactions on Intelligent
Transportation Systems 21, 3 (2020), 900–918. https://doi.org/10.1109/TITS.2019.
2901817
[71]
Malte Risto, Colleen Emmenegger, Erik Vinkhuyzen, Melissa Cefkin, and Jim Hol-
lan. 2017. Human-vehicle interfaces: The power of vehicle movement gestures in
human road user coordination. In Driving Assessment Conference, Vol. 9. Univer-
sity of Iowa, IA, Iowa, USA, 186–192. https://doi.org/10.17077/drivingassessment.
1633
[72]
Lionel P. Robert. 2019. The Future of Pedestrian-Automated Vehicle Interactions.
XRDS 25, 3 (apr 2019), 30–33. https://doi.org/10.1145/3313115
[73]
Alexandria I Rossi-Alvarez, Kevin Grove, Charlie Klauer, Melissa Miles, Andy
Schaudt, Zachary Doerzaph, Safety through Disruption, et al
.
2022. Impact
of Highly Automated Vehicle (L4/5 AV) External Communication on Other Road
User Behavior. Technical Report. Safety through Disruption (Safe-D) University
Transportation Center (UTC).
[74]
Alexandros Rouchitsas and Håkan Alm. 2019. External human–machine inter-
faces for autonomous vehicle-to-pedestrian communication: A review of empiri-
cal work. Frontiers in Psychology 10 (2019), 2757. https://doi.org/10.3389/fpsyg.
2019.02757
[75]
Parag Sewalkar and Jochen Seitz. 2019. Vehicle-to-pedestrian communication
for vulnerable road users: Survey, design considerations, and challenges. Sensors
19, 2 (2019), 358. https://doi.org/10.3390/s19020358
[76]
Timo Singer, Jonas Kobbert, Babak Zandi, and TranQuoc Khanh. 2022. Displaying
the Driving State of Automated Vehicles to Other Road Users: An International,
Virtual Reality-Based Study as a First Step for the Harmonized Regulations of
Novel Signaling Devices. IEEE Transactions on Intelligent Transportation Systems
23, 4 (2022), 2904–2918. https://doi.org/10.1109/TI TS.2020.3032777
[77]
Thad Starner, Steve Mann, Bradley Rhodes, Jerey Levine, Jennifer Healey, Dana
Kirsch, Rosalind W Picard, and Alex Pentland. 1997. Augmented reality through
wearable computing. Presence: Teleoperators & Virtual Environments 6, 4 (1997),
386–398. https://doi.org/10.1162/pres.1997.6.4.386
[78]
Helena Strömberg, Ingrid Pettersson, Jonas Andersson, Annie Rydström, De-
bargha Dey, Maria Klingegård, and Jodi Forlizzi. 2018. Designing for social
176
eHMI Scalability Issues AutomotiveUI ’23, September 18–22, 2023, Ingolstadt, Germany
experiences with and within autonomous vehicles exploring methodological
directions. Design Science 4 (2018), e13. https://doi.org/10.1017/dsj.2018.9
[79]
Wilbert Tabone, Joost De Winter, Claudia Ackermann, Jonas Bärgman, Martin
Baumann, Shuchisnigdha Deb, Colleen Emmenegger, Azra Habibovic, Marjan
Hagenzieker, Peter A Hancock, et al
.
2021. Vulnerable road users and the coming
wave of automated vehicles: Expert perspectives. Transportation Research Interdis-
ciplinary Perspectives 9 (2021), 100293. https://doi.org/10.1016/j.trip.2020.100293
[80]
Wilbert Tabone, Yee Mun Lee, Natasha Merat, Riender Happee, and Joost de
Winter. 2021. Towards Future Pedestrian-Vehicle Interactions: Introducing
Theoretically-Supported AR Prototypes. In 13th International Conference on Au-
tomotive User Interfaces and Interactive Vehicular Applications (Leeds, United
Kingdom) (AutomotiveUI ’21). Association for Computing Machinery, New York,
NY, USA, 209–218. https://doi.org/10.1145/3409118.3475149
[81]
Yourui Tong, Bochen Jia, and Shan Bao. 2021. An augmented warning system for
pedestrians: user interface design and algorithm development. Applied Sciences
11, 16 (2021), 7197. https://doi.org/10.3390/app11167197
[82]
Ozan Tonguz, Rusheng Zhang, Lin Song, and Adhishree Jaiprakash. 2021. System
and method implementing virtual pedestrian trac lights. US Patent App.
17/190,983.
[83]
Tram Thi Minh Tran, Callum Parker, and Martin Tomitsch. 2021. A Review of
Virtual Reality Studies on Autonomous Vehicle–Pedestrian Interaction. IEEE
Transactions on Human-Machine Systems 51, 6 (2021), 641–652. https://doi.org/
10.1109/THMS.2021.3107517
[84] Tram Thi Minh Tran, Callum Parker, Yiyuan Wang, and Martin Tomitsch. 2022.
Designing Wearable Augmented Reality Concepts to Support Scalability in Au-
tonomous Vehicle-Pedestrian Interaction. Frontiers in Computer Science 4 (2022),
39. https://doi.org/10.3389/fcomp.2022.866516
[85]
Maureen Troel-Madec, Laurence Boissieux, Stan Borkoswki, Dominique Vaufrey-
daz, Julien Alaimo, Sandrine Chatagnon, and Anne Spalanzani. 2019. EHMI
Positioning for Autonomous Vehicle/Pedestrians Interaction. In Adjunct Proceed-
ings of the 31st Conference on l’Interaction Homme-Machine (Grenoble, France)
(IHM ’19 Adjunct). Association for Computing Machinery, New York, NY, USA,
Article 1, 8 pages. https://doi.org/10.1145/3366551.3370340
[86]
Umbrellium. 2017. Case study - make roads safer, more responsive & dynamic.
https://umbrellium.co.uk/case-studies/south- london-starling- cv/
[87]
Himanshu Verma, Guillaume Pythoud, Grace Eden, Denis Lalanne, and Florian
Evéquoz. 2019. Pedestrians and Visual Signs of Intent: Towards Expressive
Autonomous Passenger Shuttles. Proc. ACM Interact. Mob. Wearable Ubiquitous
Technol. 3, 3, Article 107 (sep 2019), 31 pages. https://doi.org/10.1145/3351265
[88]
Rutger Verstegen, Debargha Dey, and Bastian Peging. 2021. CommDisk: A Holis-
tic 360°EHMI Concept to Facilitate Scalable, Unambiguous Interactions between
Automated Vehicles and Other Road Users. In 13th International Conference on
Automotive User Interfaces and Interactive Vehicular Applications (Leeds, United
Kingdom) (AutomotiveUI ’21 Adjunct). Association for Computing Machinery,
New York, NY, USA, 132–136. https://doi.org/10.1145/3473682.3480280
[89]
Ke Wang, Gang Li, Junlan Chen, Yan Long, Tao Chen, Long Chen, and Qin Xia.
2020. The adaptability and challenges of autonomous vehicles to pedestrians
in urban China. Accident Analysis & Prevention 145 (2020), 105692. https:
//doi.org/10.1016/j.aap.2020.105692
[90]
Marc Wilbrink, Manja Nuttelmann, and Michael Oehl. 2021. Scaling up Auto-
mated Vehicles’ EHMI Communication Designs to Interactions with Multiple
Pedestrians Putting EHMIs to the Test. In 13th International Conference on
Automotive User Interfaces and Interactive Vehicular Applications (Leeds, United
Kingdom) (AutomotiveUI ’21 Adjunct). Association for Computing Machinery,
New York, NY, USA, 119–122. https://doi.org/10.1145/3473682.3480277
177
... Previous research has shown that with displays on the outside of the AV, called external human-machine interfaces or eHMIs, pedestrians can make more effective road-crossing decisions Bindschädel et al., 2022;Dey et al., 2020). However, eHMIs have certain disadvantages in that they typically cannot address an individual pedestrian (Colley et al., 2020;Tran et al., 2023) and that they can be difficult to perceive in some cases, for example due to a temporary occlusion by another object (Dey et al., 2022;Troel-Madec et al., 2019). ...
Preprint
Full-text available
Communication from automated vehicles (AVs) to pedestrians using augmented reality (AR) could positively contribute to traffic safety. However, previous research was mainly conducted through questionnaires or experiments in virtual instead of real environments. In this study, 28 participants conducted trials outdoors with an approaching AV and were supported by four different AR interfaces. The AR experience was created by having participants wear a Varjo XR-3 headset with see-through functionality, with the AV and AR elements virtually overlaid onto the real environment. The AR interfaces were vehicle-locked (Planes on vehicle), world-locked (Fixed pedestrian lights; Virtual fence), or head-locked (Pedestrian lights HUD). Participants had to hold down a button as long as they felt safe to cross, and their opinions were obtained through rating scales, interviews, and a questionnaire. The results showed that AR interfaces were more preferred and effective than no AR interface, evidenced by higher perceived safety to cross for yielding AVs. The fixed pedestrian lights scored lower than the other interfaces, presumably due to low saliency and the fact that participants had to visually identify both the AR interface and the AV. In conclusion, in our outdoor assessment of AR interfaces for AV-pedestrian interactions, AR was preferred over no interface, but preference depended on design elements like placement, salience, and visual attention requirements. Looking ahead, as AR gains more prominence, the need for human-subject studies in real-world settings becomes crucial. This study offers insights into the potential benefits and challenges of implementing AR in real-world scenarios.
Article
Full-text available
The last few years have seen a wealth of research on external human-machine interfaces (eHMIs). It has been argued that eHMIs are vital because they fill the social interaction void that arises with the introduction of automated vehicles (AVs). However, there is still much discussion about whether eHMIs are needed. The present article surveys arguments for and against eHMIs. We list three arguments against eHMIs: (1) Implicit communication dominates pedestrian-AV interaction, and there is no social interaction void to be filled, (2) There is a large variety of eHMI concepts and a lack of standardization and consensus, and (3) eHMIs may elicit various negative effects such as distraction, confusion, and overreliance. Next, we present five reasons why eHMIs may be useful or required: (1) eHMIs can make planned actions of the AV visible, thereby increasing the efficiency of pedestrian-AV interaction, (2) Participants value an eHMI compared to no eHMI, (3) eHMIs do not have to be limited to showing instructions or the AV's planned actions; showing the AV mode or the AV's cooperative or detection capabilities are other uses of eHMIs, (4) Recent research shows that driver eye contact is important in traffic, and a social interaction void thus exists, and (5) A large portion of pedestrian-vehicle accidents in current traffic is caused by unclear implicit communication, suggesting that pedestrians may benefit from explicit eHMIs. It is hoped that this article contributes to the critical discussion of whether eHMIs are needed and how they should be designed.
Article
Full-text available
Automated vehicles (AVs) can perform low-level control tasks but are not always capable of proper decision-making. This paper presents a concept of eye-based maneuver control for AV-pedestrian interaction. Previously, it was unknown whether the AV should conduct a stopping maneuver when the driver looks at the pedestrian or looks away from the pedestrian. A two-agent experiment was conducted using two head-mounted displays with integrated eye-tracking. Seventeen pairs of participants (pedestrian and driver) each interacted in a road crossing scenario. The pedestrians' task was to hold a button when they felt safe to cross the road, and the drivers' task was to direct their gaze according to instructions. Participants completed three 16-trial blocks: (1) Baseline, in which the AV was pre-programmed to yield or not yield, (2) Look to Yield (LTY), in which the AV yielded when the driver looked at the pedestrian, and (3) Look Away to Yield (LATY), in which the AV yielded when the driver did not look at the pedestrian. The driver's eye movements in the LTY and LATY conditions were visualized using a virtual light beam. Crossing performance was assessed based on whether the pedestrian held the button when the AV yielded and released the button when the AV did not yield. Furthermore, the pedes-trians' and drivers' acceptance of the mappings was measured through a questionnaire. The results showed that the LTY and LATY mappings yielded better crossing performance than Baseline. Furthermore, the LTY condition was best accepted by drivers and pedestrians. Eye-tracking analyses indicated that the LTY and LATY mappings attracted the pedestrian's attention, while pedestrians still distributed their attention between the AV and a second vehicle approaching from the other direction. In conclusion, LTY control may be a promising means of AV control at intersections before full automation is technologically feasible.
Article
Full-text available
In this article, we report on the design and evaluation of an external human-machine interface (eHMI) for a real autonomous vehicle (AV), developed to operate as a shared transport pod in a pedestrianized urban space. We present insights about our human-centered design process, which included testing initial concepts through a tangible toolkit and evaluating 360-degree recordings of a staged pick-up scenario in virtual reality. Our results indicate that in complex mobility scenarios, participants filter for critical eHMI messages; further, we found that implicit cues (i.e., pick-up manoeuvre and proximity to the rider) influence participants' experience and trust, while at the same time more explicit interaction modes are desired. This highlights the importance of considering interactions with shared AVs as a service more holistically, in order to develop knowledge about AV-pedestrian interactions in complex mobility scenarios that complements more targeted eHMI evaluations.
Article
Full-text available
The urban traffic system is most likely to change in the next years to a mixed traffic with human drivers, vulnerable road users, and automated vehicles. In the past, the development of external communication approaches for automated vehicles focused on scenarios where an automated vehicle communicates with either a pedestrian or a human driver. However, interactions with more than one traffic partner are more realistic. Therefore, a study with 42 participants was conducted with a multi-agent simulation in which an automated vehicle interacted simultaneously with two participants, a pedestrian and a driver of a manual vehicle. In this study, two main scenarios were investigated in order to evaluate the safety and efficiency of the interactions and to determine whether the human road users feel correctly addressed. In one scenario, the pedestrian had to cross the road in front of the automated and the manual vehicle, which were approaching from different sides. In the other, the manual vehicle had to drive through a bottleneck in front of the oncoming automated vehicle, while the pedestrian had to cross the road after both vehicles passed. The communication approach of the automated vehicle consisted of implicit signals using a speed profile and lateral offset within its lane, and explicit signals using an external human–machine interface. The results of the study show that no collisions were observed in terms of safety and no significant negative effects on efficiency were measured. However, in contrast to single agent interactions, a majority of participants felt wrongly addressed in situations where the automated vehicle signals the right-of-way to the other human road user. It can be concluded that the communication approach of the automated vehicle needs to be modified in order to address certain road users more clearly.
Article
Full-text available
Automated driving will require new approaches to the communication between vehicles and vulnerable road users (VRUs) such as pedestrians, e.g., through external human–machine interfaces (eHMIs). However, the majority of eHMI concepts are neither scalable (i.e., take into account complex traffic scenarios with multiple vehicles and VRUs), nor do they optimize traffic flow. Speculating on the upgrade of traffic infrastructure in the automated city, we propose Smart Curbs, a scalable communication concept integrated into the curbstone. Using a combination of immersive and non-immersive prototypes, we evaluated the suitability of our concept for complex urban environments in a user study (N = 18). Comparing the approach to a projection-based eHMI, our findings reveal that Smart Curbs are safer to use, as our participants spent less time on the road when crossing. Based on our findings, we discuss the potential of Smart Curbs to mitigate the scalability problem in AV-pedestrian communication and simultaneously enhance traffic flow.
Article
Full-text available
Wearable augmented reality (AR) offers new ways for supporting the interaction between autonomous vehicles (AVs) and pedestrians due to its ability to integrate timely and contextually relevant data into the user's field of view. This article presents novel wearable AR concepts that assist crossing pedestrians in multi-vehicle scenarios where several AVs frequent the road from both directions. Three concepts with different communication approaches for signaling responses from multiple AVs to a crossing request, as well as a conventional pedestrian push button, were simulated and tested within a virtual reality environment. The results showed that wearable AR is a promising way to reduce crossing pedestrians' cognitive load when the design offers both individual AV responses and a clear signal to cross. The willingness of pedestrians to adopt a wearable AR solution, however, is subject to different factors, including costs, data privacy, technical defects, liability risks, maintenance duties, and form factors. We further found that all participants favored sending a crossing request to AVs rather than waiting for the vehicles to detect their intentions-pointing to an important gap and opportunity in the current AV-pedestrian interaction literature.
Article
In recent years there has been an increasing trend for autonomous vehicles (AV) to feature an external human-machine interface (eHMI). This study investigates the usability issues of an eHMI composed of light bands and displays that need to communicate with multiple types of road users and how designers intend to map all this and cover all these needs in one user interface design. In the first experiment, a physical prototype was built in a conventional car and shown to 18 participants. In the second experiment, the same eHMI concept, with improvements made from the first usability test, was shown to 500 participants in a digital prototype. In both tests, the usability of eHMIs, as well as the overall acceptance level of AVs before and after reviewing the eHMI concept, were evaluated. The results indicate that our eHMI designs positively influenced overall acceptance because they have clearer driving information and sense the visibility of the AV system. In close traffic negotiations when AVs communicate on the basis of what they sense and perceive, people felt uncomfortable with AVs perceiving them, which will be an increasingly important issue in the design conversation between fully autonomous AVs and general road users.