Content uploaded by Philipp Wintersberger
Author content
All content in this area was uploaded by Philipp Wintersberger on Sep 17, 2018
Content may be subject to copyright.
Man vs. Machine: Comparing a Fully
Automated Bus Shuttle with a Manu-
ally Driven Group Taxi in a Field Study
Philipp Wintersberger*
CARISSMA
Technische Hochschule
Ingolstadt (THI), Germany
Philipp.Wintersberger@thi.de
Anna-Katharina Frison*
Technische Hochschule
Ingolstadt (THI), Germany
Anna-Katharina.Frison@thi.de
Andreas Riener
Technische Hochschule
Ingolstadt (THI), Germany
Andreas.Riener@thi.de
*The first two authors contributed
equally
Permission to make digital or hard copies of all or part of this work for personal or
classroom use is granted without fee provided that copies are not made or distributed
for profit or commercial advantage and that copies bear this notice and the full citation
on the first page. Copyrights for components of this work owned by others than the
author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee. Request permissions from permissions@acm.org.
Copyright held by the owner/author(s). Publication rights licensed to ACM.
AutomotiveUI ’18 Adjunct,, September 23–25, 2018, Toronto, ON, Canada
ACM 978-1-4503-5947-4/18/09.
https://doi.org/10.1145/3239092.3265969
Abstract
Automated driving functions are traditionally tested in on-
road studies, however, mainly focusing on technological
aspects (sensor accuracy, etc.). Field studies addressing
users’ individual needs and expectations are still rare. As
a consequence, it is still unclear whether or not automated
driving systems will reach a comprehensive market pene-
tration. To address this issue, we set-up a user study and
compared users’ acceptance (utilizing TAM) as a passen-
ger (N=12) of a traditional group taxi vs. an automated bus
shuttle both driving in regular traffic. Results show that par-
ticipants questioned the usefulness of the automated bus
shuttle, mainly due to the reduced speed, but, on the other
hand, rated their perceived ease of use and their attitude
towards using the ADS more positive than expected. Thus,
we conclude that with further development of the technol-
ogy and by including a user-centered design approach, high
user acceptance of ADSs can finally be achieved.
Author Keywords
Automated Driving, SAE J3016, User Acceptance, Field
Studies
CCS Concepts
•Applied computing →Computers in other domains;
•Human-centered computing →Empirical studies in HCI;
Introduction
Automated driving has the potential to radically alter estab-
lished mobility patterns and thereby eliminate the boundary
between individual and public transportation, while provid-
ing higher safety and traffic efficiency [6]. However, these
systems can only become successful if they are accepted
and frequently used by potential customers. Currently it
is still not clear if users will adopt automated vehicles, but
latest research reveals that many potential users express
skepticism towards this novel technology [13]. By providing
last-mile and mobility-on-demand concepts, autonomous
shuttles could be a valuable asset for mobility providers by
satisfying individuals’ transportation demands [4]. Since
existing shuttles in real environments suffer various down-
sides (such as low driving speed or the need to have an op-
erator on board [7]), it has been questioned if public demon-
stration in early development phases could even have a
negative impact on the public opinion [4].
With this paper, we investigate user acceptance (utilizing
the technology acceptance model TAM [3]) of a shared, fully
automated shuttle before and after initial use in a field study
and directly compare the results to a manually driven group
taxi. We are especially interested if consumer expectations
in automated shuttles preserve after initial use, and if there
exist significant differences to well established modes of
transportation.
Figure 1: EasyMile EZ10, the
automated bus shuttle evaluated
in this study.
Figure 2: A VW Multivan was
evaluated as baseline condition to
allow a direct comparison.
Related Work
Vehicle automation will allow transport systems to become
more flexible, and expand their services directly to con-
sumers [11]. Whilst comprehensive automated vehicles will
pervade onto our streets over the next decades [1], com-
mon traffic behavior will change by millennials and subse-
quent generations (regarding vehicle ownership, locational
factors, etc. [9]). Thus, evaluating user acceptance of auto-
mated vehicles recently became an important topic. Stud-
ies thereby often utilize online surveys [10, 1, 8] or driving
simulator studies [13, 5]. However, little research has yet
been conducted in real field studies. Distler et al. [4] evalu-
ated user acceptance/experience criteria using immersive
experiences, concluding that many participants perceived
autonomous mobility-on-demand as ineffective. We use a
similar approach (also utilizing an extended TAM model with
trust as additional factor [12, 2]), including samples from
other age groups (i.e., younger/older generation) and fur-
ther compare user acceptance of the automated bus shuttle
directly with a manually driven group taxi.
Field Study
The automated bus shuttle under investigation (EasyMile
EZ10, see Figure 1) is regularly operating in a small Bavar-
ian city and connects two important areas with four inter-
mediate stops in an 8-minute drive. Due to the relatively
short travel distance (roughly one kilometer), that further
traverses multiple pedestrian areas, we choose a slight de-
tour for our manually driven alternative (VW Multivan, see
Figure 2). The route with the manual vehicle encompassed
a short rural section including higher speeds, leading to a
comparable travel time of about 5-6 minutes.
Procedure and Measurements
Study participants were recruited directly on site, with the
precondition, that they did not use the automated shuttle
before. We recruited singles, couples, and as well groups
up to 3 people, that took the trips together (other passen-
gers were not allowed). Thus, group dynamics may have
influenced their reactions during the drives (our experi-
menters were present in the vehicles to observe passen-
gers’ behavior). To minimize such effects during the semi-
structured interviews, groups were split after the rides and
we conducted the interview with each participant individ-
ually. Because of legal reasons, an operator was present
during all drives with the shuttle, who was instructed not
to engage in conversations with study participants. After
completing a consent form and demographics, participants
completed two rides, one with the automated shuttle (con-
dition shuttle), and another with the manually driven group
taxi (condition taxi) in alternating order. Before each drive,
participants were emphasized to rate their expectations us-
ing the TAM2-model, followed by a short group discussion.
We then issued participants to take the short trip in the re-
spective condition while behaving naturally as in any other
public transport ride. Afterwards, they again completed the
TAM sub-scales and revealed their opinion in short semi-
structured interviews. The experiment took about 45 min-
utes and each participant was compensated.
Automated Shuttle
M(before) M(after)
PEOU 5 6
ATT 6 6
Trust 5.5 6.5
PU 2 2.5
Int. 4 3
Group Taxi
M(before) M(after)
PEOU 5 4
ATT 5 5
Trust 6 6
PU 5 5
Int. 5 5
Table 1: Medians of TAM ratings
for the fully automated bus shuttle
(top) and the manually driven
group taxi (bottom) before and
after actual use.
Results
In total, 24 subjects participated in the experiment. We
sampled two age groups - 12 younger adults below 35 and
12 elderly aged above 60. Results and comparisons with
elderly people will be published elsewhere; in this work we
solely present the results of the younger age group (M = 23,
SD = 5.77 years, see descriptive statistics of TAM ratings in
Table 1). For statistical comparisons we utilized Wilcoxon
rank tests. Due to the small sample size we can only speak
of tendencies instead of effects.
Expectations vs. Reflections After Initial Use
Considering TAM ratings in the automated bus shuttle, we
could observe significant differences in 3 out of 5 TAM sub-
scales. Perceived ease of use (PEOU) was rated signifi-
cantly higher after the trip (Mdn = 6), although expectations
were already high (Mdn = 5, Z=116; p < .001). Also, atti-
tude towards using the system (ATT) showed higher scores
after (Mdn=6) than before (Mdn=6) exposure (Z=169, p =
.013). Furthermore, trust increased after taking the trip
with the automated bus shuttle (before: Mdn=5.5, after:
6.5, Z=269.5, p <.001). Regarding perceived usefulness
(PU) and intention to use the system (Int), no differences
between the expectations and the post-assessment was
present. However, it should be mentioned that those scales
received lower scores than the other dimensions (PU be-
fore: Mdn = 2, PU after: Mdn = 2.5, Int before: Mdn = 4, Int
after: Mdn = 3). Contrarily, regarding the manually driven
shuttle (condition taxi), no differences between the expecta-
tions and the retrospective assessment could be revealed in
any dimension.
Comparison of Automated Bus and Manually Driven Shuttle
When comparing TAM sub-scales of the automated bus
shuttle with the group taxi, expectations in the taxi (Mdn=5)
were much higher than in the shuttle (Mdn=2, Z=22; p =
.003) regarding perceived usefulness (PU). Although we did
not find any other differences in users’ expectations, ratings
fundamentally changed after the trips.
Here, three out of five TAM sub-scales are in favor of the
automated bus: Perceived ease of use (PEOU, shuttle:
Mdn = 6, taxi: Mdn = 4, Z=115; p = .009), Attitude towards
using the system (Att, shuttle: Mdn = 6, taxi : Mdn = 5,
Z=218.5; p = .013), and Trust (shuttle: Mdn = 7, Taxi: Mdn
= 6, Z=234; p = .002). However, there was no difference re-
garding intention to use the system (shuttle: Mdn = 3, taxi :
Mdn = 5), and the group taxi was still rated significantly
higher in terms of perceived usefulness (shuttle: Mdn =
2, Taxi : Mdn = 5, Z=24.50, p = .043).
Group Discussions and Interviews
In semi-structured interviews and group discussions, the
most critical negative aspect of the automated bus shuttle
was the low speed, where some even believed to be “faster
by foot”. Many participants emphasized that driving with the
shuttle was “interesting and pleasant”, and that there were
no trust issues causing fear or anxiety. However, some also
stated that “after a short phase of novelty, it quickly felt as
common as any other ride”, and that thus in the current
scenario a manually driven vehicle would be preferred. Par-
ticipants mentioned the higher speed and the familiarity with
common taxis as important advantages of manual vehicles.
Also, subjects tended to trust manual drivers to the same
extent as automated driving systems. It seemed, that in
the end, travel time plays the most important role (“I would
choose whatever is faster” ) when choosing between differ-
ent modes of transportation. What is also worth to mention
is that many participants quickly attributed some forms of
personality to the shuttle, some calling it “him”,“sweet”, but
also “clumsy”.
Discussion
Results of our study differ to those obtained by Distler et al.
[4], who could evaluate a drop in participants’ TAM ratings
after using a fully automated bus shuttle. Although their ex-
periment addressed mobility-on-demand (while ours did
not), we claim the results to be somehow comparable as
a similar type of vehicle was evaluated before and after a
fully automated ride in regular traffic. In our study, utilizing
a sample of young adults, TAM ratings of three dimensions
(PEOU, ATT, Trust) significantly increased after initial use
compared to participants’ prior expectations. The same di-
mensions were also rated higher compared to a manually
driven group taxi. However, participants still attributed the
manual vehicle higher perceived usefulness, mainly be-
cause of reasons such as higher speeds (and consequently
less travel time) or familiarity. Thus, also participants’ actual
intention to use the system was rated lower for the auto-
mated bus shuttle. Nevertheless, most subjects could imag-
ine to use automated buses in future for last mile scenarios
(as addition to other public transport modalities).
Although our study was conducted in a real environment,
there are some issues that prevent generalization of the
results. First, the sample of only 12 participants was rela-
tively small - statistical results thus represent just tenden-
cies and should be confirmed with larger samples in future
studies. Second, the shuttle drives on a fixed route with typ-
ical stops, and we could thus not evaluate door-to-door or
mobility-on-demand concepts. Finally, the low speed of the
vehicle played a major role in its assessment. Future exper-
iments should evaluate the concept with more distinct user
groups in different environments, such as more busy urban
areas.
Conclusion
The study presented in this paper evaluated user accept-
ability and acceptance using the TAM model before and
after initial use of a fully automated bus shuttle in a real en-
vironment, and further compared the results to rides with a
manually driven group taxi. Ratings of our sample consist-
ing 12 younger adults (presumably, the “users of tomorrow”)
showed increasing acceptance criteria after initial use, re-
garding trust, attractiveness, as well as perceived ease of
use. Those sub-scales also were rated higher for the auto-
mated shuttle than for the manually driven group taxi. Par-
ticipants, however, still valued the manually driven vehicle
to be more useful, since it is a fast and well known mode
of transportation. We conclude that future automated bus
shuttles must thus either become faster or provide means
that allow to utilize time for passengers in more efficient
way. Nevertheless, as participants initial skepticism could
be partially removed after their first drive, we also argue
that with further technical advantages and user-centered
optimization, there is nothing more to prevent the success-
ful implementation of ADS (in public traffic).
Acknowledgements
This work is supported under the FH-Impuls program of
the German Federal Ministry of Education and Research
(BMBF) under Grant No. 13FH7I01IA (SAFIR). We ac-
knowledge Isabella Thang for supporting us in the field
study as research assistant.
REFERENCES
1. Prateek Bansal and Kara M. Kockelman. 2017.
Forecasting Americans’ long-term adoption of
connected and autonomous vehicle technologies.
Transportation Research Part A: Policy and Practice 95
(2017), 49–63.
2. Daniel Belanche, Luis V Casaló, and Carlos Flavián.
2012. Integrating trust and personal values into the
Technology Acceptance Model: The case of
e-government services adoption. Cuadernos de
Economía y Dirección de la Empresa 15, 4 (2012),
192–204.
3. Fred D Davis and Viswanath Venkatesh. 1996. A
critical assessment of potential measurement biases in
the technology acceptance model: three experiments.
International Journal of Human-Computer Studies 45,
1 (1996), 19–45.
4. Verena Distler, Carine Lallemand, and Thierry Bellet.
2018. Acceptability and Acceptance of Autonomous
Mobility on Demand: The Impact of an Immersive
Experience. In Proceedings of the 2018 CHI
Conference on Human Factors in Computing Systems.
ACM, 612.
5. Anna-Katharina Frison, Philipp Wintersberger, Andreas
Riener, and Clemens Schartmüller. 2017. Driving
Hotzenplotz: A Hybrid Interface for Vehicle Control
Aiming to Maximize Pleasure in Highway Driving. In
Proceedings of the 9th International Conference on
Automotive User Interfaces and Interactive Vehicular
Applications (AutomotiveUI ’17). ACM, New York, NY,
USA, 236–244.
6. Satu Innamaa, Scott Smith, Isabel Wilmink, and Nick
Reed. 2018. Impact Assessment. In Road Vehicle
Automation 4. Springer, 45–55.
7. Vincenzo Luca. 2017. TÜV SÜD schafft Grundlagen für
ersten automatisiert fahrenden öffentlichen Bus. (Oct.
2017).
last
access July 17, 2018.
8. Sina Nordhoff, Joost De Winter, B Van Arem, and
Riender Happee. 2018. Acceptance of driverless
vehicles : Results from a large cross-national
questionnaire study. Journal of Advanced
Transportation March (2018).
9. Steven E Polzin, Xuehao Chu, and Jodi Godfrey. 2014.
The impact of millennials’ travel behavior on future
personal vehicle travel. Energy Strategy Reviews 5
(2014), 59–65.
10. Christina Rödel, Susanne Stadler, Alexander
Meschtscherjakov, and Manfred Tscheligi. 2014.
Towards autonomous cars: the effect of autonomy
levels on acceptance and user experience. In
Proceedings of the 6th International Conference on
Automotive User Interfaces and Interactive Vehicular
Applications. ACM, 1–8.
11. Stefan Trommer, Viktoriya Kolarova, Eva Frädrich, Lars
Kröger, Benjamin Kickhöfer, Tobias Kuhnimhof,
Barbara Lenz, and P Phleps. 2016. Autonomous
driving: The impact of vehicle automation on mobility
behaviour. (2016).
12. Viswanath Venkatesh, Michael G Morris, Gordon B
Davis, and Fred D Davis. 2003. User acceptance of
information technology: Toward a unified view. MIS
quarterly (2003), 425–478.
13. Philipp Wintersberger, Andreas Riener, and
Anna-Katharina Frison. 2016. Automated Driving
System, Male, or Female Driver: Who’d You Prefer?
Comparative Analysis of Passengers’ Mental
Conditions, Emotional States & Qualitative Feedback.
Proceedings of the 8th International Conference on
Automotive User Interfaces and Interactive Vehicular
Applications - Automotive’UI 16 October (2016), 51–58.
