PresentationPDF Available

Sprachassistenten: Die Navigation der Zukunft?

  • May 2022
  • Conference: Forschungsforum 2022 der Hochschule Emden/Leer
Authors:
Andreas M. Klein at Universidad de Sevilla
Andreas M. Klein
Maria Rauschenberger at Hochschule Emden/Leer
Maria Rauschenberger
Download file PDF
Read file
Download citation

Abstract

Virtuellen digitalen Assistenten (VAs) wird weltweit ein enormes Wachstum bei Unternehmensanwendungen vorausgesagt. Eine Form von VAs sind Sprachassistenten, welche in vielen Geräten und Systemen integriert sind (z.B. Smartphones). VA-Qualitätskriterien, wie korrekte Befehlsausführung, Sprachverständlichkeit und der Datenschutz, sind gleichzeitig Herausforderungen bei der Nutzung. Daher ist die Messung der Qualität des Benutzererlebnisses für die Bewertung und Verbesserung von VAs von großem Interesse. Dieser Beitrag zeigt, wie Sprachinteraktion in Kombination mit dem neuen Fragebogenkonzept UEQ+ (https://ueqplus.ueq-research.org) gemessen werden kann.
Content uploaded by Andreas M. Klein
Author content
All content in this area was uploaded by Andreas M. Klein on May 13, 2022
Content may be subject to copyright.
Sprachassistenten: Die Navigation der Zukunft?
Andreas M. Klein Prof. Dr. Maria Rauschenberger
Department of Computer Language and Systems Fachbereich Technik
University of Seville, Spain Hochschule Emden/Leer, Deutschland
andreas.klein@ux-researchgroup.com maria.rauschenberger@hs-emden-leer.de
Last updated: 22-05-12
FORSCHUNGSFORUM 2022 ・12. Mai 2022
Department of Computer
Languages and Systems
University of Seville
Content
1. Introduction
2. Challenges using Voice Assistants
3. Background
4. Research gap
5. UX measurement for Voice Assistants
6. Summary
7. References
2
Measuring UX quality of Voice User Interfaces ・Andreas M. Klein
Department of Computer
Languages and Systems
University of Seville
Introduction: Andreas M. Klein
•Lecturer & research assistant University of Applied Sciences Emden/Leer, Germany
oMedia-/Business Informatics (Mathematics, Computer Graphics)
oMedia Technology (Internships with focus on audio)
oSince Sept. 2020 PhD candidate
Department of Computer Languages and Systems at the University of Seville, Spain
•Research focus:
oInvestigation of voice assistant systems: UX quality measurement
oFive international peer-reviewed paper (2020 –2021)
https://scholar.google.com/citations?hl=en&user=I5IDWsAAAAAJ
https://www.researchgate.net/profile/Andreas-Klein-11/research
•Collaborations
oResearch Group for Agile Software Development and User Experience (UX)
Head: Prof. Dr. Jörg Thomaschewski, Prof. Dr. Eva-Maria Schön, Prof. Dr. Maria Rauschenberger
University of Applied Sciences Emden/Leer, Germany
oResearch Group Engineering and Science for Software (ES3), Spain
Director: Proff. Dr. Maria José Escalona
Recognized in the Andalusian Research Plan; (FIDETIA, FIUS University of Seville);
(https://grupo.us.es/es3uses)
3
Measuring UX quality of Voice User Interfaces ・Andreas M. Klein
Department of Computer
Languages and Systems
University of Seville
Introduction
•Voice User Interface (VUI)
o“VUI is what a person interacts with when communicating with
spoken language application.” (Cohen et al. 2004)
•Voice assistants (VAs)
oDevices or systems including VUIs (Hoy 2018)
•Google Assistant (Google), Alexa (Amazon), Siri (Apple),
Cortana (Microsoft), …
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al. 4
Department of Computer
Languages and Systems
University of Seville
Challenges
Exploring Voice Assistant Risks and
Potential with Technology-based Users
(Klein, A. M., Hinderks, A., Rauschenberger, M., & Thomaschewski, J., 2020a)
•Pilot-study:
oGerman technology-based (affine) users
oFocus: Interaction behavior, frequency of
use and concerns
•Goal:
oHow the target group uses VAs to explore
possibilities and improvements?
•Design of a questionnaire with
qualitative and quantitative elements
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al.
Auszug mit Beispielen:
Fragebogen zu Sprachassistenten
Wie alt sind Sie? _________
Geschlecht: ¡männlich
¡weiblich
¡keine Angabe
Welche Sprachassistenten nutzen Sie?
(Mehrfachnennung möglich)
oSiri
oAlexa
oCortana
oGoogle Assistent
oSonstige und zwar:
_________________________
oKeine
Warum nutzen Sie Sprachassistenten?
5
Department of Computer
Languages and Systems
University of Seville
Challenges
•Example: Privacy has to be enhanced.
(Klein, A. M., Hinderks, A., Rauschenberger, M., & Thomaschewski, J., 2020a)
•Example: Privacy comparing Germany and Spain
(Klein, A. M., Rauschenberger, M., Thomaschewski, J., and Escalona, M. J. 2021)
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al.
Example of pilot-study results compared to a population-representative online survey (BVDW e.V. 2017)
6
Department of Computer
Languages and Systems
University of Seville
Challenges
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al.
•Further challenges, e.g., correct command execution,
comprehensibility, context-sensitivity,…
•For example:
7
Department of Computer
Languages and Systems
University of Seville
Background
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al.
•Why focus on the context of use?
oUser, system, and context influence Human-Computer Interaction
(HCI) significantly (Hassenzahl & Tractinsky 2006).
•Why do we measure User Experience (UX)?
oUX is an essential aspect for evaluating interactive systems and
improving HCI to meet user’s needs.
SystemUser
Context of use
Use
User Experience
Pragmatic and
hedonic quality
8
Department of Computer
Languages and Systems
University of Seville
Research gap
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al.
•Measuring UX quality of VUIs (Klein, Andreas M. 2021)
oSeveral approaches are available, B U T:
§No consideration of the VUI context of use.
§No comprehensive UX measurement for VUIs is available, yet.
§Many UX questionnaires are available.
§Which are suitable methods?
Research gap
9
Department of Computer
Languages and Systems
University of Seville
Core of the research
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al.
•UX tool selector for VUIs (Klein, Andreas M. 2021)
Published at W4A2021 conference: Google doctoral consortium (awardee)
oContext-dependent measurement recommendations.
oSelection of suitable measurements from the UX toolbox.
Recommendations
for measurement
Test objectUser
Context of use
•Questionnaires
•Interviews
•…
UX measurement toolbox for VUIs
UX tool selector for VUIs
10
Department of Computer
Languages and Systems
University of Seville
UEQ+ Voice quality scales
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al.
•UEQ+ Framework (Modular questionnaire concept)
(Schrepp, M. and Thomaschweski, J. 2019)
oContains 16 scales to measure different UX aspects.
oScales can be combined to create product-related questionnaire.
oConstruction of three scales to measure UX aspects of VAs.
(Klein, A. M., Hinderks, A., Schrepp, M., and Thomaschewski, J. 2020b, 2020c)
§Response behaviour
§Response quality
§Comprehensibility
oFuture work: Scale validation
http://ueqplus.ueq-research.org
11
Response behavior
In my opinion the response behavior of
the voice assistant is
artificial
!
!
!
!
!
!
!
natural
unpleasant
!
!
!
!
!
!
!
pleasant
unlikable
!
!
!
!
!
!
!
likable
boring
!
!
!
!
!
!
!
entertaining
Department of Computer
Languages and Systems
University of Seville
UEQ+ Voice quality scales
•Product-related questionnaire
oSelect relevant scales
oProduct-specific UX aspects first (Winter et al. 2017)
oFurther criteria, e.g., for marketing (Winter et al. 2017)
oMore information: UEQ+ Handbook (http://ueqplus.ueq-research.org)
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al. 12
Questionnaire example 1:
Users applying smart home
Questionnaire example 2
Voice assistant customer service
§Perspicuity
§Trust
§Trustworthiness of content
§Quality of content
§Efficiency
§Perspicuity
§Dependability
§Trust
§Response behaviour
§Response quality
§Comprehensibility
Voice quality scales
Department of Computer
Languages and Systems
University of Seville
Summary
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al.
Sprachassistenten: Die Navigation der Zukunft?
•Herausforderungen vs. Qualitätskriterien
oZ.B. Privatsphäre und Datenschutz, Sprachverständlichkeit, Kontext-
sensitive Interaktion, …
oNutzungsbarrieren überwinden z.B. mittels „Acceptance by Design”
(Rauschenberger 2021)
•Messung der UX Qualität von Sprachassistenten
oNEU: Kontext-abhängige UX-Messung
oNEU: Flexible UX-Messung mit UEQ+ Sprachskalen
oZur Evaluierung und Verbesserung von Sprachassistenten
oZur Unterstützung des Human-Computer Design (HCD) Prozesses
•Fazit: Sprachassistenten können die Human-Computer
Interkation womöglich langfristig revolutionieren.
(Klein, A. M., Rauschenberger, M., Thomaschewski, J., and Escalona, M. J. 2021)
13
Department of Computer
Languages and Systems
University of Seville
References
•BVDW e.V. Digital Trends Umfrage zu digitalen Sprachassistenten.Technical report,Bundesverband Digitale Wirtschaft (BVDW)
e.V. Bundesverband Digitale, 2017.
•Cohen, M. H., Giangola, J. P., and Balogh, J. (2004). Voice User Interface Design.Addison Wesley Longman Publishing Co., Inc.,
USA.
•Hassenzahl,Marc and Tractinsky,Noam (2006). User experience - a research agenda.Behaviour &Information Technology,
25(2):91–97.
•Hoy, Matthew B. (2018)Alexa, Siri, Cortana, and More:An Introduction to Voice Assistants.Medical Reference Services
Quarterly, 37(1):81–88.
•Klein, A. M., Hinderks, A., Rauschenberger, M., & Thomaschewski, J., (2020a).Exploring Voice Assistant Risks and Potential with
Technology-based Users.In Proceedings of 16th International Conference on Web Information Systems and technology
(WEBIST), pages 1–8, SCITEPRESS.
•Klein, A. M., Hinderks, A., Schrepp, M., and Thomaschewski, J. (2020b).Measuring User Experience Quality of Voice Assistants.
In 2020 15th Iberian Conference on Information Systems and Technologies (CISTI).IEEE, Seville, Spain, 1–4.
https://doi.org/10.23919/CISTI49556.2020.9140966
•Klein, A. M., Hinderks, A., Schrepp, M., and Thomaschewski, J. (2020c).Construction of UEQ+ Scales for Voice Quality.In
Proceedings of the Conference on Mensch Und Computer, MuC ’20,page 1–5, New York, NY, USA.Association for Computing
Machinery.DOI:https://dl.acm.org/doi/10.1145/3404983.3410003
•Klein, Andreas M., 2021.Toward aUser Experience Tool Selector for Voice User Interfaces.In Proceedings of the 18th
International Web for All Conference (W4A ’21). Association for Computing Machinery,New York, NY, USA, 1–2.
https://doi.org/10.1145/3430263.3456728
•Klein, A. M., Rauschenberger, M., Thomaschweski, J., and Escalona, M. J. (2021). Comparing Voice Assistant Risks and Potential
with Technology-Based Users: A Study from Germany and Spain.Journal of Web Engineering, 7(16):1991–2016.
•Rauschenberger, M. (2021). Acceptance by Design:Voice Assistants.In 1st AI-Debate Workshop, Magdeburg,Germany.
OvGU.http://dx.doi.org/10.25673/38476
•Schrepp,M.and Thomaschewski, J. (2019). Design and Validation of aFramework for the Creation of User Experience
Questionnaires.International Journal of Interactive Multimedia and Artificial Intelligence, 5(7):88–95.
•Winter, D., Hinderks, A., Schrepp, M. & Thomaschewski, J., (2017). Welche UX Faktoren sind für mein Produkt wichtig? In: S.
Hess & H. Fischer (Eds.), Mensch und Computer 2017—Usability Professionals.Regensburg:Gesellschaft für Informatik e.V. (pp.
191–200).
FORSCHUNGSFORUM 2022 ・Sprachassistenten: Die Navigation der Zukunft? ・Klein et al. 14
Vielen Dank für ihre Aufmerksamkeit!
Sprachassistenten: Die Navigation der Zukunft?
Andreas M. Klein Prof. Dr. Maria Rauschenberger
Department of Computer Language and Systems Fachbereich Technik
University of Seville, Spain Hochschule Emden/Leer, Deutschland
andreas.klein@ux-researchgroup.com maria.rauschenberger@hs-emden-leer.de
Link: https://www.researchgate.net/publication/360499267_Sprachassistenten_Die_Navigation_der_Zukunft
FORSCHUNGSFORUM 2022 ・12. Mai 2022
ResearchGate has not been able to resolve any citations for this publication.
Acceptance by Design: Voice Assistants
Conference Paper
Full-text available
  • Sep 2021
Voice Assistants (VAs), Speech-based Technology (SBT), and Voice User Interfaces (VUI) have gained popularity in recent years; however, privacy concerns stop people from using them. Although the terms are used for slightly di?erent purposes, we mainly use "VAs" for simplicity. To increase the adoption rate and address users’ needs, we propose reducing barriers with Acceptance by Design. By that, we mean using, e.g., VA designs that implicitly show if they collect data or how much data is collected. We can do that by ?rst exploring the user requirements and concerns with methods from the Human-Centered Design Process [4] to provide guidelines for the future. This is a complex challenge that requires a deep understanding from di?erent disciplines to design VAs for users skeptical about them. The HCD o?ers a holistic approach for capturing user requirements or discovering new dimensions of the UX that in?uence the user’s perception or awareness of the interactive system. We must keep in mind that users have di?erent needs (e.g., children vs. parents vs. people with disabilities) that could result in di?erent design solutions. Research has already started to explore with questionnaires [9, 10] and interviews the risks and opportunities of VAs (e.g., cultures [6, 22], use cases [1, 3], existing users, or potential users [1, 3, 22]). Furthermore, these guidelines might help others to design future interactive systems for di?erent use cases, such as augmented reality glasses for university students in labs.
View
Exploring Voice Assistant Risks and Potential with Technology-based Users
Conference Paper
Full-text available
  • Nov 2020
Voice user interfaces (VUIs) or voice assistants (VAs) such as Google Home or Google Assistant (Google), Cortana (Mircosoft), Siri (Apple) or Alexa (Amazon) are highly available in the consumer sector and present a smart home trend. Still, the acceptance seems to be culture-dependent, while the syntax of communication poses a challenge. So, there are some basic questions: 'Why do people buy VAs?' 'What do they use them for?' 'What could be improved in the future?'. We explore the opinion of a German technology-based user group to identify the challenges and opportunities of VAs. We focus on the interaction behaviour, frequency of use, concerns, and opinions of this target group as they show a higher variety of interaction as well as privacy concerns in representative population studies. Our preliminary findings confirm previous results (missing accuracy of commands and serious concerns about privacy issues) and show that technology-based users from Germany are intensive users, although with particular concerns about data collection. Probably, there is a correlation between privacy concerns and speech intelligibility as queries relating to VAs are problematic due to repetitions and refinement.
View
View
Design and Validation of a Framework for the Creation of User Experience Questionnaires
Article
Full-text available
  • Dec 2019
Existing user experience questionnaires have a fixed number of scales. Each of these scales measures a distinct aspect of user experience. These questionnaires can be used with little effort and provide a number of useful support materials that make the application of such a questionnaire quite easy. However, in practical evaluation scenarios it can happen that none of the existing questionnaires contains all scales necessary to answer the research question. It is of course possible to combine several UX questionnaires in such cases, but due to the variations of item formats this is also not an optimal solution. In this paper, we describe the development and first validation studies of a modular framework that allows the creation of user experience questionnaires that fit perfectly to a given research question. The framework contains several scales that measure different UX aspects. These scales can be combined to cover the relevant research questions.
View
Comparing Voice Assistant Risks and Potential with Technology-Based Users: A Study from Germany and Spain
Article
Currently, voice assistants (VAs) are trendy and highly available. The VA adoption rate of internet users differs among European countries and also in the global view. Due to speech intelligibility and privacy concerns, using VAs is challenging. Additionally, user experience (UX) assessment methods and VA improvement possibilities are still missing, but are urgently needed to overcome users’ concerns and increase the adoption rate. Therefore, we conducted an intercultural study of technology-based users from Germany and Spain, expecting that higher improvement potential would outweigh concerns about VAs. We investigated VA use in terms of availability versus actual use, usage patterns, concerns, and improvement proposals. Comparing Germany and Spain, our findings show that nearly the same amount of intensive VA use is found in both technology-based user groups. Despite cultural differences, further results show very similar tendencies, e.g., frequency of use, privacy concerns, and demand for VA improvements.
View
View
Construction of UEQ+ Scales for Voice Quality
  • Jan 2020
  • 1-5
  • A M Klein
  • A Hinderks
  • M Schrepp
  • J Thomaschewski
• Klein, A. M., Hinderks, A., Schrepp, M., and Thomaschewski, J. (2020c). Construction of UEQ+ Scales for Voice Quality. In Proceedings of the Conference on Mensch Und Computer, MuC '20, page 1-5, New York, NY, USA. Association for Computing Machinery. DOI: https://dl.acm.org/doi/10.1145/3404983.3410003