DataPDF Available

Abstract

Protocol for Exploring Voice Assistant Risks and Potential With Technology-based Users Version 2.0 / 2020
Exploring Voice Assistant Risks and Potential
with Technology-based Users
Protocol for Exploring Voice Assistant Risks and Potential
With Technology-based Users
Version 2.0 / 2020
Andreas M. Klein
University of Seville
Seville, Spain
Andreas Hinderks
University of Seville
Seville, Spain
Maria Rauschenberger
Social Computing Systems, Max Planck Institute for
Software Systems, Saarbrücken, Germany
Jörg Thomaschewski
University of Applied Sciences Emden/Leer
Emden, Germany
29th October 2020
Department of Computer Languages and Systems
University of Seville
Contents
1!Versions ..................................................................................................................................... 3!
2!Research Goal .......................................................................................................................... 3!
3!Questionnaire ............................................................................................................................ 3!
4!Conclusion ................................................................................................................................. 3!
5!Future work ............................................................................................................................... 4!
Bibliography ....................................................................................................................................... 4!
Appendix – Questionnaires ............................................................................................................. 4!
1 Versions
Version 1.0 / 2020: Questionnaire in the original German version only.
Version 2.0 / 2020: Questionnaire in German language and English translation.
2 Research Goal
Klein, Hinderks, Rauschenberger and Thomaschewski explore the opinion of a German
technology-based user group to find opportunities for improvement of voice assistants
(Klein et al., 2020a). The focus is on the interaction behaviour, frequency of use, and
concerns of this target group because they express a wider variety of interaction as well as
deep privacy concerns while using voice assistants (BVDW e.V., 2017).
This research report gives detailed information about the designed questionnaire in the
original German language and English translation. The publication of Klein, Hinderks,
Rauschenberger and Thomaschewski provides detailed information about that topic (Klein
et al., 2020a).
This document is a supplement of the article:
Klein, A. M., Hinderks, A., Rauschenberger, M., & Thomaschewski, J. (2020). Exploring
Voice Assistant Risks and Potential with Technology-based Users. WEBIST’2020.
3 Questionnaire
Klein, Hinderks, Rauschenberger and Thomaschewski designed a questionnaire, which
contains qualitative and quantitative elements to explore voice user interfaces and their
parameters, usability and user experience. Questions about availability and usage,
frequency of use, the context of use and the potential to improve VAs are essential
aspects. The questionnaire was evaluated in two pre-tests and is available in the original
German language and English translation. Further information and a detailed description
are included in the publication.
4 Conclusion
Klein, Hinderks, Rauschenberger and Thomaschewski can identify as the result of this pilot
study a broad potential for improvement. Non-users could become users if privacy and
speech intelligibility are enhanced. On one side, we have much proportion of intensive use,
and on the other side, deep concerns about privacy in our target group.
5 Future work
Future work includes the collection of more data from different user groups to validate our
results and to understand the potential user groups, e.g., consumer vs professional use.
Therefore, we want to explore power or routine users with structured interviews and
observations. Finally, we plan to apply the UEQ+ framework (Schrepp, M., &
Thomaschewski, 2019) with the new scales for voice quality (Klein et al., 2020b).
Bibliography
Klein, A. M., Hinderks, A., Rauschenberger, M., & Thomaschewski, J. (2020a). Exploring
Voice Assistant Risks and Potential with Technology-based Users. WEBIST’2020.
BVDW e.V. (2017). Digital Trends Umfrage zu digitalen Sprachassistenten.
Bundesverband Digitale Wirtschaft (BVDW) e.V. [Digital Trends Survey on digital language
assistants. Federal Association of Digital Economy].
https://www.bvdw.org/themen/publikationen/detail/artikel/digital-trends-umfragezu-digitalen-
sprachassistenten/.
Schrepp, M., & Thomaschewski, J. (2019). Design and validation of a framework for the
creation of user experience questionnaires. International Journal of Interactive Multimedia
and Artificial Intelligence. DOI:10.9781/ijimai.2019.06.006.
Klein, A. M., Hinderks, A., Schrepp, M., & Thomaschewski, J. (2020b). Construction of
UEQ+ Scales for Voice Quality. In Proceedings of the Conference on Mensch und
Computer, MuC ’20, page 1–5, New York, NY, USA. Association for Computing Machinery.
DOI: 10.1145/3404983.3410003.
Appendix – Questionnaires
The following pages show the used paper-pencil questionnaires. Pages are shown in the
original German version used for the collection of the data and the English translation.
Fragebogen zum Thema „Nutzung von Sprachassistenten und Geräten mit
Sprachsteuerung“
Dem Einsatz von Sprachassistenten und Geräten mit Sprachsteuerung wird laut Medien,
wissenschaftlicher Studien und Geräteherstellern eine große Zukunft prognostiziert.
Demgegenüber stehen die tatsächliche Nutzung und die häufig medial thematisierte
Skepsis der Nutzer bei der Verwendung solcher Systeme.
Der Fragebogen wird im Rahmen eines wissenschaftlichen Projektes durchgeführt und soll
den Umfang der aktuellen Nutzung zeigen, die Vor- und Nachteile sowie
Verbesserungsvorschläge liefern.
Die Umfrage findet anonym statt und die Bearbeitungszeit beträgt ca. 5-10 Min.
Vielen Dank für Ihre Bereitschaft zur Teilnahme
Andreas Klein
Fragebogen zu Sprachassistenten
1) Wie alt sind Sie? _________
2) Geschlecht: ¡ männlich ¡ weiblich ¡ keine Angabe
3) Welche Sprachassistenten haben Sie? (Mehrfachnennung möglich)
o Siri
o Alexa
o Cortana
o Google Assistent
o Sonstige und zwar: ___________________________
o Keine
4) Welche Sprachassistenten nutzen Sie? (Mehrfachnennung möglich)
o Siri
o Alexa
o Cortana
o Google Assistent
o Sonstige und zwar: ___________________________
o Keine
5) Gibt es Gründe, warum Sie bestimmte Sprachassistenten besitzen, aber nicht
verwenden?
_______________________________________________________________
_______________________________________________________________
_______________________________________________________________
_______________________________________________________________
6) Wie oft nutzen Sie Sprachassistenten insgesamt?
¡ mehrmals täglich
¡ ungefähr täglich
¡ mehrmals wöchentlich
¡ ungefähr wöchentlich
¡ mehrmals monatlich
¡ ungefähr monatlich oder seltener
¡ nie (weiter mit Frage 9)
¡ kommt ganz darauf an. Mal mehr, mal weniger, je nach ______________________
___________________________________________________________________
7) Warum nutzen Sie Sprachassistenten?
1 (trifft zu)
2
3
4
5
6
Keine
Aussage
möglich
Für mehr Zeitersparnis im Alltag
O
O
O
O
O
O
O
Für mehr Bequemlichkeit im Alltag
O
O
O
O
O
O
O
Für mehr Sicherheit im Alltag (Beispiel im Auto)
O
O
O
O
O
O
O
Für mehr Spaß im Alltag
O
O
O
O
O
O
O
Weil ich meine Smart-Geräte steuern möchte
O
O
O
O
O
O
O
Weil ich mehrere Dinge gleichzeitig tun möchte
O
O
O
O
O
O
O
Weil ich gerne neue Techniken ausprobiere
O
O
O
O
O
O
O
Weil ich mit „jemandem“ reden möchte
O
O
O
O
O
O
O
Weitere Nutzungen:________________________________________________________
8) In welchem Umfeld nutzen Sie Sprachassistenten?
1 (oft)
2
3
4
5
6
7 (nie)
Keine
Aussage
möglich
Zu Hause
zur Haussteuerung (Licht-, Heizungssteuerung, Smart Home
etc.)
O
O
O
O
O
O
O
O
Zu Hause
zur Auswahl von Medien (Musik abspielen, TV etc.)
O
O
O
O
O
O
O
O
Zu Hause
zur Kommunikation (Anrufe tätigen etc.)
O
O
O
O
O
O
O
O
Zu Hause
zur Suche von Unterlagen, Materialien oder im Web
O
O
O
O
O
O
O
O
Unterwegs
zur Haussteuerung (Licht-, Heizungssteuerung, Smart Home
etc.)
O
O
O
O
O
O
O
O
Unterwegs
zur Auswahl von Medien (Musik abspielen, TV etc.)
O
O
O
O
O
O
O
O
Unterwegs
zur Kommunikation (Anrufe tätigen etc.)
O
O
O
O
O
O
O
O
Unterwegs
zur Suche von Unterlagen, Materialien oder im Web
O
O
O
O
O
O
O
O
Nutzen Sie Diktierfunktionen (z.B. für Texte, E-Mails oder
Textnachrichten)?
O
O
O
O
O
O
O
O
Nutzen Sie Sprachübermittlung (z.B. via WhatsApp?)
O
O
O
O
O
O
O
O
Weitere Nutzungen: _______________________________________________________
9) Welche Gründe gibt es Ihrer Meinung nach, keine Sprachassistenten zu nutzen?
1 (trifft zu)
2
3
4
5
6
Keine
Aussage
möglich
Weil der Assistent mich nicht versteht
O
O
O
O
O
O
O
Weil der Assistent anders als erwartet reagiert
O
O
O
O
O
O
O
Weil das Reden mit dem Assistenten ungewohnt ist
O
O
O
O
O
O
O
Weil meine Daten missbraucht werden könnten
O
O
O
O
O
O
O
Weil die Geräte mich überwachen können
O
O
O
O
O
O
O
Weil die Geräte zu teuer sind
O
O
O
O
O
O
O
Weil die Geräte noch nicht ausgereift sind
O
O
O
O
O
O
O
Weil ich klassische Eingabegeräte bevorzuge
O
O
O
O
O
O
O
Weitere Gründe: _________________________________________________________
10) Was sollte an Sprachassistenten verbessert werden?
1 (trifft zu)
2
3
4
5
6
Keine
Aussage
möglich
Das Erkennen von Sprache im allgemeinen
O
O
O
O
O
O
O
Das Erkennen von schnellem Sprechen
O
O
O
O
O
O
O
Das Erkennen von undeutlichem Sprechen
O
O
O
O
O
O
O
Der Assistent sollte schneller antworten
O
O
O
O
O
O
O
Der Assistent sollte natürlicher klingen
O
O
O
O
O
O
O
Der Assistent sollte Nutzer unterscheiden können
O
O
O
O
O
O
O
Der Assistent sollte Stimmungen und Gefühle erkennen und
darauf reagieren können
O
O
O
O
O
O
O
Die Lernfähigkeit sollte verbessert werden
O
O
O
O
O
O
O
Der Assistent sollte sich besser bedienen lassen
(Beispiel: Touchscreen als Korrekturhilfe)
O
O
O
O
O
O
O
Der Assistent sollte sich mit Geräten, die Sprach-steuerung
unterstützen, besser verbinden lassen
O
O
O
O
O
O
O
Die Privatsphäre sollte besser geschützt werden
O
O
O
O
O
O
O
Weitere Verbesserungsvorschläge: ___________________________________________
11) Generelle Empfindungen
1 (trifft zu)
2
3
4
5
6
7 (trifft
nicht zu)
Keine
Aussage
möglich
Es ist irgendwie unnatürlich mit Maschinen zu reden
O
O
O
O
O
O
O
O
Ich empfinde Unbehagen, wenn Maschinen „menschlich“ reden
O
O
O
O
O
O
O
O
Ich empfinde Unbehagen, wenn Maschinen „künstlich“ reden
O
O
O
O
O
O
O
O
Weitere Empfindungen: ____________________________________________________
Questionnaire on the topic "The Use of Voice Assistants and Devices with Voice
Control."
According to the media, to scientific studies and to device manufacturers, the use of digital
voice assistants (VAs) and devices with voice control has been predicted to have a great
future. In contrast to these predictions stands the actual use of these systems and the
scepticism of users – a topic often addressed in the media.
This questionnaire is being carried out as part of a research project and is intended to
demonstrate the extent of current use, the advantages and disadvantages of the systems
and to make suggestions for improvement.
This survey is anonymous and the time to take it is only 5-10 minutes.
Thank you very much for your willingness to participate!
Andreas Klein
Questionnaire for voice assistants
1) How old are you? _________
2) Gender: ¡ male ¡ female ¡ No information
3) Which voice assistants do you have? (multiple answers possible)
o Siri
o Alexa
o Cortana
o Google Assistant
o Others (please name): ___________________________
o None
4) Which voice assistants do you use? (multiple answers possible)
o Siri
o Alexa
o Cortana
o Google Assistant
o Others (please name): ___________________________
o None
5) Are there reasons why you have certain voice assistants, but do not use them?
_______________________________________________________________
_______________________________________________________________
_______________________________________________________________
_______________________________________________________________
6) How often do you use Voice Assistants in total?
¡ several times a day
¡ approximately once a day
¡ several times a week
¡ approximately once a week
¡ several times a month
¡ approximately once a month or less
¡ never (continue with question 9)
¡ It depends. Sometimes more, sometimes less, depending on ________________
___________________________________________________________________
7) Why do you use voice assistants?
1 (highly
relevant)
2
3
4
5
6
7
(completely
irrelevant)
No
statement
possible
To save time in everyday life
O
O
O
O
O
O
O
O
For more convenience in everyday life
O
O
O
O
O
O
O
O
For more security in everyday life (e.g., in a car)
O
O
O
O
O
O
O
O
For more fun in everyday life
O
O
O
O
O
O
O
O
Because I want to control my smart devices
O
O
O
O
O
O
O
O
Because I want to do several things at once
O
O
O
O
O
O
O
O
Because I like trying out new technologies
O
O
O
O
O
O
O
O
Because I want to talk to ‘someone’
O
O
O
O
O
O
O
O
Further uses: ________________________________________________________________
8) In what environment do you use voice assistants?
1 (often)
2
3
4
5
6
7 (never)
No
statement
possible
At home
Smart home
O
O
O
O
O
O
O
O
At home
Selection of media (music, TV etc.)
O
O
O
O
O
O
O
O
At home
Communication (make phone calls)
O
O
O
O
O
O
O
O
At home
Search for files, materials or web searches
O
O
O
O
O
O
O
O
On the road
Smart home
O
O
O
O
O
O
O
O
On the road
Selection of media (music, TV etc.)
O
O
O
O
O
O
O
O
On the road
Communication (make phone calls)
O
O
O
O
O
O
O
O
On the road
Search for files, materials or web searches
O
O
O
O
O
O
O
O
Do you use dictation functions? (e.g., for emails)
O
O
O
O
O
O
O
O
Do you use voice transmission? (e.g., WhatsApp)
O
O
O
O
O
O
O
O
Further uses: ________________________________________________________________
9) In your opinion, what are the reasons for not using VAs?
1 (highly
relevant)
2
3
4
5
6
7
(completely
irrelevant)
No
statement
possible
Because the assistant does not understand me
O
O
O
O
O
O
O
O
Because the assistant reacts differently than expected
O
O
O
O
O
O
O
O
Because talking to the assistant is unfamiliar
O
O
O
O
O
O
O
O
Because my data could be used improperly
O
O
O
O
O
O
O
O
Because the devices can monitor me
O
O
O
O
O
O
O
O
Because the devices are too expensive
O
O
O
O
O
O
O
O
Because the devices are still not fully developed
O
O
O
O
O
O
O
O
Because I prefer classic input devices
O
O
O
O
O
O
O
O
Further uses: ________________________________________________________________
10) What should be improved about voice assistants?
1 (highly
relevant)
2
3
4
5
6
7
(completely
irrelevant)
No
statement
possible
Speech recognition in general
O
O
O
O
O
O
O
O
Recognize fast speech
O
O
O
O
O
O
O
O
Recognize unclear speech
O
O
O
O
O
O
O
O
Ability to answer quicker
O
O
O
O
O
O
O
O
Sound more natural
O
O
O
O
O
O
O
O
Ability to distinguish between users
O
O
O
O
O
O
O
O
Ability to recognise feelings and take them into account
O
O
O
O
O
O
O
O
Increase learning ability
O
O
O
O
O
O
O
O
Additional interaction e.g. touchscreen for corrections
O
O
O
O
O
O
O
O
Better integration with other speech-controlled devices
O
O
O
O
O
O
O
O
Better protection of privacy
O
O
O
O
O
O
O
O
Further improvement suggestions: _______________________________________________
11) General feelings
1 (highly
relevant)
2
3
4
5
6
7
(completely
irrelevant)
No
statement
possible
It is somehow unnatural to talk to machines.
O
O
O
O
O
O
O
O
I feel uneasy when machines speak like humans.
O
O
O
O
O
O
O
O
I feel uneasy when machines speak ‘artificially’.
O
O
O
O
O
O
O
O
... Adaptions such as new VUI parameters beeing turned into, for example, the UEQ, are costly in terms of time and personnel. Hence, we designed a questionnaire (Klein et al., 2020a) for our research questions, which contains both qualitative and quantitative elements to explore VUIs and their parameters as well as usability and UX. Its essential aspects are questions about availability and usage, frequency of use, the context of use and the potential to improve VAs. ...
Conference Paper
Full-text available
Voice user interfaces (VUIs) or voice assistants (VAs) such as Google Home or Google Assistant (Google), Cortana (Mircosoft), Siri (Apple) or Alexa (Amazon) are highly available in the consumer sector and present a smart home trend. Still, the acceptance seems to be culture-dependent, while the syntax of communication poses a challenge. So, there are some basic questions: 'Why do people buy VAs?' 'What do they use them for?' 'What could be improved in the future?'. We explore the opinion of a German technology-based user group to identify the challenges and opportunities of VAs. We focus on the interaction behaviour, frequency of use, concerns, and opinions of this target group as they show a higher variety of interaction as well as privacy concerns in representative population studies. Our preliminary findings confirm previous results (missing accuracy of commands and serious concerns about privacy issues) and show that technology-based users from Germany are intensive users, although with particular concerns about data collection. Probably, there is a correlation between privacy concerns and speech intelligibility as queries relating to VAs are problematic due to repetitions and refinement.
ResearchGate has not been able to resolve any references for this publication.