Content uploaded by David R. Large
Author content
All content in this area was uploaded by David R. Large on Apr 28, 2023
Content may be subject to copyright.
Ghost Busting: A Novel On-Road Exploration of
External HMIs for Autonomous Vehicles
David R. Large, Madeline Hallewell, Xuekun Li, Catherine Harvey, Gary Burnett
Human Factors Research Group, University of Nottingham
SUMMARY
The absence of a human driver in future autonomous vehicles means that explicit pedestrian-driver
communication is not possible. Building on the novel ‘Ghost Driver’ methodology to emulate an
autonomous vehicle, we developed prototype external human-machine interfaces to replace existing
cues, and report preliminary, qualitative findings captured from a sample of pedestrians (n=64) who
encountered the vehicle when crossing the road, as well as reflecting on the method.
KEYWORDS
Autonomous vehicles, external HMI, Ghost Driver
Introduction
There has been considerable interest amongst behavioural scientists in the potential impact of
highly and fully autonomous ‘self-driving’ vehicles (AVs) on the behaviour of pedestrians. These
vehicles, operating at SAE level 4 or 5, are unlikely to have a human driver present, and as such,
explicit visual cues (head, eye, hand/arm gestures etc.) that are traditionally exchanged between a
driver and a pedestrian, will be absent. Typically, these aim to establish a mutual understanding of
perception (Have you seen me?) and intent (Will you give way?) (Merat et al., 2018), and are
important to overall traffic safety especially in low-speed crossing scenarios in complex urban
settings (Lee et al., 2020). However, studying genuine, naturalistic behaviours of people responding
to AVs presents a number of challenges (limited public trials, requirement to have a ‘safety driver’
present etc.). A novel solution is to use a Wizard-of-Oz (WoZ) approach to give the appearance that
the car is driving on its own, even when it is not. This can be achieved by hiding the driver using a
bespoke seat cover (aka ‘Ghost Driver’ method) (Rothenbücher et al., 2016). To date, no such
studies have been reported in the UK. In addition, the Ghost Driver method has not been employed
specifically to evaluate external human-machine interfaces (eHMIs).
Method
A ‘Ghost Driver’ WoZ study was devised in which the driver was hidden in a bespoke seat-suit,
thereby giving the appearance that the vehicle (Nissan Leaf) was driving by itself (Figure 1). The
seat-suit was designed and fabricated to enable the driver to maintain safe control of the vehicle,
whilst ensuring that they could not be seen by a passing pedestrian glancing into the vehicle. Three
eHMIs were created. These were informed by the literature and prototyped using an individually
addressable RGB-LED matrix and strip attached to the outside of the vehicle (front of bonnet and at
top of windscreen, respectively). The eHMIs were programmed using an Arduino Mega board and
manipulated with push-button controls from inside the vehicle. The eHMI designs employed
varying degrees of anthropomorphism (implicit, explicit, low) to aid interpretation and build trust.
The first (implicit) utilised the LED strip only and mimicked the pupillary response of an eye:
lateral movement demonstrated scanning/awareness, and blinking provided an implicit cue of the
vehicle’s intention to give way. The second (explicit) presented a face and eyes on the LED matrix
to scan the road and used humanlike language to ‘talk’ to the pedestrians (Figure 1). The third (low)
used a vehicle icon and vehicle-centric language on the LED matrix. For each eHMI design, four
modes were created: scanning, giving way (pedestrian on right), giving way (pedestrian on left) and
giving way (pedestrians on both sides of road). A second researcher, seated in the back seat of the
ego-vehicle, controlled the current state of the eHMI in response to the observable pedestrians in the
vicinity of the vehicle. The study took place on the extensive University of Nottingham campus and
a circuitous route was selected that included several marked and unmarked crossings. Over 10 hours
of video data were captured using a dashcam and GoPro recorder to document pedestrians’
responses to the ‘driverless’ vehicle and eHMIs. In addition, researchers were located at specific
crossing points, and invited pedestrians who encountered the vehicle to complete a survey.
Results and Discussion
Video analysis is ongoing. Here, we report qualitative findings, including illustrative comments and
responses related to the vehicle and eHMI concepts. Results show that over eighty percent of
respondents believed that the car was driving on its own (“There was no driver, just a passenger in
the back passenger seat”), and this surprised many people (“I was mostly just shocked, so I stopped
and observed”). Nevertheless, many people still appeared to interact with the vehicle as if a driver
were present (e.g. waving to thank the vehicle for stopping), highlighting the value of an eHMI to
replace interactions with a driver, and supporting the inclusion of ‘human’ elements. Comments
suggest the eHMIs impacted the trust relationship (“I was a bit curious about why the car
stopped...when I saw the screen that explained a lot”), with most comments suggesting support for
the concepts (“I quickly became aware that it was helping me to cross”, “[the eHMI] matched
observed behaviour of vehicle”, “I understood that the eyes were looking out for people”), whereas
others were more cautious towards the technology (“Would need to encounter it more before I fully
trusted it”), and a few respondents admitted being confused by the messages (“I wasn't entirely sure
what the message was conveying”). This did not necessarily change pedestrians’ crossing
behaviour, with most respondents still stating that they crossed in front of the vehicle as they
normally would. It did, however, inspire some additional curiosity: “Had seen it…earlier and was
curious to see if it would stop or not.” The different eHMIs appeared to inspire different emotional
responses. For example, the explicit anthropomorphism encouraged positivity – smiling, laughing
etc., whereas responses to the low anthropomorphism were more perfunctory; survey ratings
indicated that the latter provided the highest clarity in conveying its intended messages. Overall,
initial findings support the use of a hidden ‘ghost driver’ to explore pedestrians’ interactions with an
AV, with observed behaviour suggesting high ecological validity. In addition, explicit
communication using eHMIs (employing elements of anthropomorphism) appears to encourage safe
crossing behaviours, help pedestrians interpret vehicle behaviour and intent, and increase their
confidence and build appropriate trust when interacting with a driverless vehicle.
Figure 1: Driver in seat-suit (left); hidden driver operating car (centre); example eHMIs (right)
Acknowledgements
The study was undertaken as part of the ServCity project (https://www.servcity.co.uk/), which is
funded by the UK Innovation Agency (Innovate UK) and Centre for Connected & Autonomous
Vehicles (CCAV) (Grant number: 105091). We would like to thank Nissan Technical Centre
Europe (NTCE) for use of the Nissan Leaf vehicle and their support and guidance throughout.
References
Lee, Y. M., Madigan, R., Giles, O., Garach‑Morcillo, L., Markkula, G., Fox, C., . . . Merat, N.
(2020). Road users rarely use explicit communication when interacting in today’s traffic:
implications for automated vehicles. Cognition, Technology and Work.
Merat, N., Lee, Y. M., Markkula, G., Uttley, J., Camara, F., Fox, C., . . . Schieben, A. (2018). How
do we study pedestrian interaction with automated vehicles? Preliminary findings from the
European interACT project. In G. Meyer & S. Beiker (Eds.), Lecture Notes in Mobility (pp.
21-33). Cham: Springer.
Rothenbücher, D., Li, J., Sirkin, D., Mok, B., & Ju, W. (2016). Ghost driver: A field study
investigating the interaction between pedestrians and driverless vehicles. Paper presented at
the 2016 25th IEEE international symposium on robot and human interactive communication
(RO-MAN).
