To read the full-text of this research, you can request a copy directly from the authors.


Currently, voice assistants (VAs) are trendy and highly available. The VA adoption rate of internet users differs among European countries and also in the global view. Due to speech intelligibility and privacy concerns, using VAs is challenging. Additionally, user experience (UX) assessment methods and VA improvement possibilities are still missing, but are urgently needed to overcome users’ concerns and increase the adoption rate. Therefore, we conducted an intercultural study of technology-based users from Germany and Spain, expecting that higher improvement potential would outweigh concerns about VAs. We investigated VA use in terms of availability versus actual use, usage patterns, concerns, and improvement proposals. Comparing Germany and Spain, our findings show that nearly the same amount of intensive VA use is found in both technology-based user groups. Despite cultural differences, further results show very similar tendencies, e.g., frequency of use, privacy concerns, and demand for VA improvements.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Possible reasons for non-use are diverse, e.g, fear of data misuse and monitoring. Yet, on the other end of the spectrum is a group of intensive users (Klein et al., 2021). These intensive users show an appreciation for the use of VUIs that goes beyond the pure functionality, i.e., user experience aspects of VUIs. ...
... We should know which UX aspects users take into account when evaluating the quality of VUIs, since different UX aspects are important for different users or products (Meiners et al., 2021). For example, some users are concerned about which data is collected and how, while others mention the need for higher accuracy of commands (Rauschenberger, 2021;Klein et al., 2021). ...
... Current challenges when using VUIs are, e.g., speech intelligibility, correct command execution, data security, and privacy (Klein et al., 2021;Tas et al., 2019;Rauschenberger, 2021). UX assessment by considering specific UX aspects for VUIs is an essential evaluation method for overcoming barriers and skepticism as well as meeting users' needs. ...
Conference Paper
Full-text available
Voice User Interfaces (VUIs) are becoming increasingly available while users raise, e.g., concerns about privacy issues. User Experience (UX) helps in the design and evaluation of VUIs with focus on the user. Knowledge of the relevant UX aspects for VUIs is needed to understand the user’s point of view when developing such systems. Known UX aspects are derived, e.g., from graphical user interfaces or expert-driven research. The user’s opinion on UX aspects for VUIs, however, has thus far been missing. Hence, we conducted a qualitative and quantitative user study to determine which aspects users take into account when evaluating VUIs. We generated a list of 32 UX aspects that intensive users consider for VUIs. These overlap with, but are not limited to, aspects from established literature. For example, while Efficiency and Effectivity are already well known, Simplicity and Politeness are inherent to known VUI UX aspects but are not necessarily focused. Furthermore, Independency and Context-sensitivity are some new UX aspects for VUIs.
... o Devices or systems including VUIs (Hoy 2018) • Google Assistant (Google), Alexa (Amazon), Siri ( • Privacy has to be enhanced. (Klein, A. M., Hinderks, A., Rauschenberger, M., & Thomaschewski, J., 2020a) • Further improvement proposals (Klein, A. M., Hinderks, A., Rauschenberger, M., & Thomaschewski, J., 2020a) • Challenges when using VUIs: (Tas et al. 2019, Rauschenberger 2021 o Correct command execution o Speech intelligibility o Context-sensitivity ...
... • Challenges vs. quality features when using VUIs o e.g., Privacy, context-sensitivity, … o To overcome barriers, e.g., approach "Acceptance by design" (Rauschenberger 2021) o UX quality measurement for VUIs § NEW: Context-dependent UX measurement (UX tool selector) (Klein, Andreas M. 2021) § NEW: Flexible UX measurement with UEQ+ voice scales Thomaschewski, J. 2020b, 2020c) § To evaluate and improve VUIs § To support the HCD process for VUI design o VUIs may probably revolutionize HCI in the long run when researchers take users' needs into account by applying new UX assessment methods. (Klein, A. M., Rauschenberger, M., Thomaschewski, J., and Escalona, M. J. 2021) ...
Full-text available
Virtuellen digitalen Assistenten (VAs) wird weltweit ein enormes Wachstum bei Unternehmensanwendungen vorausgesagt. Eine Form von VAs sind Sprachassistenten, welche in vielen Geräten und Systemen integriert sind (z.B. Smartphones). VA-Qualitätskriterien, wie korrekte Befehlsausführung, Sprachverständlichkeit und der Datenschutz, sind gleichzeitig Herausforderungen bei der Nutzung. Daher ist die Messung der Qualität des Benutzererlebnisses für die Bewertung und Verbesserung von VAs von großem Interesse. Dieser Beitrag zeigt, wie Sprachinteraktion in Kombination mit dem neuen Fragebogenkonzept UEQ+ ( gemessen werden kann.
Full-text available
Protocol for Comparing Voice Assistant Risks and Potential with Technology-Based Users: A Study from Germany and Spain. Version 3.0 / 2021.
Conference Paper
Full-text available
Voice user interfaces (VUI) are currently a trending topic but the ability to measure and improve the user experience (UX) is still missing. We aim to develop a tool selector as a web application that can provide a suitable tool to measure UX quality of VUIs. The UX tool selector for VUIs will include a UX measurement toolbox containing several existing and new VUI assessment methods. The UX tool selector will provide context-dependent measurement recommendations without prior extensive research to evaluate and improve VUIs.
Conference Paper
Full-text available
The UEQ+ is a modular framework for the construction of UX questionnaires. The researcher can pick those scales that fit his or her research question from a list of 16 available UX scales. Currently, no UEQ+ scales are available to allow measuring the quality of voice interactions. Given that this type of interaction is increasingly essential for the usage of digital products, this is a severe limitation of the possible products and usage scenarios that can be evaluated using the UEQ+. We describe in this paper the construction of three specific scales to measure the UX of voice interactions. Besides, we discuss how these new scales can be combined with existing UEQ+ scales in evaluation projects. CCS CONCEPTS • Human-centred computing • Human computer interaction • HCI design and evaluation methods
Conference Paper
Full-text available
Voice User Interfaces have become part of our daily life by being integrated in smartphones, computers, smart home devices and many other consumer products. However, despite their potential, voice assistants like Alexa, Google Assistant or Siri are not that widely used. Why is that? What are their pain points? How can the interaction and dialog flow between the user and the voice user interface (UI) be improved? In a research and development project at designaffairs, insights from user research were used to develop an interaction and dialog concept for a voice UI. The concept draft was refined with user feedback to develop a software prototype. This early voice UI prototype was then evaluated in a user test which demonstrated its potential in better satisfing the user’s wishes for a natural dialog flow. Overall, the iterative user centered approach of the project revealed crucial pain points of the human-machine interaction and further opportunity areas to meet the user’s need.
Full-text available
Existing user experience questionnaires have a fixed number of scales. Each of these scales measures a distinct aspect of user experience. These questionnaires can be used with little effort and provide a number of useful support materials that make the application of such a questionnaire quite easy. However, in practical evaluation scenarios it can happen that none of the existing questionnaires contains all scales necessary to answer the research question. It is of course possible to combine several UX questionnaires in such cases, but due to the variations of item formats this is also not an optimal solution. In this paper, we describe the development and first validation studies of a modular framework that allows the creation of user experience questionnaires that fit perfectly to a given research question. The framework contains several scales that measure different UX aspects. These scales can be combined to cover the relevant research questions.
Conference Paper
Full-text available
Der Erfolg von Technik hängt vor allem davon ab, wie Nutzer den Umgang mit einem inter-aktiven Produkt wahrnehmen, erleben und bewerten. Verschiedene Aspekte sind hierbei von Bedeutung, unter anderem die Usability, die ästhetische Gestaltung, die soziale Kommunika-tion persönlicher Werte sowie die emotionale Einstellung und die motivationale Bereitschaft, das Produkt auch zukünftig zu verwenden. Zur adäquaten Erfassung dieser Aspekte wurde auf der Basis eines weithin etablierten und empirisch abgesicherten Modells zum Nutzungs-erleben, dem CUE-Modell von Thüring und Mahlke (2007), ein modular aufgebauter Frage-bogen entwickelt. Insgesamt besteht dieser aus drei separat anwendbaren Modulen, die sich auf die "Produktwahrnehmung" (Nützlichkeit, Benutzbarkeit, visuelle Ästhetik, Status und Bindung), auf "Nutzeremotionen" (positive und negative Emotionen) und auf "Konsequen-zen" der Produktinteraktion (Loyalität und Nutzungsintention) beziehen. Die Konstruktion des Fragebogens und die Auswahl von Items erfolgte auf Basis zweier online durchgeführter Datenerhebungen, an denen jeweils n = 238 Probanden teilgenommen haben. Eine erste Validierung fand im Rahmen einer laborexperimentellen Studie statt, bei der n = 67 Personen jeweils drei verschiedene interaktive Produkte bewerteten. Die Ergebnisse stützen sowohl die Reliabilitätsannahme der konstruierten Skalen, als auch deren diskriminative, kriteriums-bezogene und Konstruktvalidität bei der Bewertung interaktiver Technologie.
Full-text available
Developer, manager and user feedback is needed to optimize products. Besides the basic Software qualities textendash usability and user experience are important properties for improving your product. Usability is well known and can be tested with e.g. a usability test or an expert review. In contrast user experience describes the whole impact a product has on the end-user. The timeline goes from before, while and after the use of a product. We present a tool that allows you to evaluate the user experience of a product with little effort. Furthermore the tool is available in different languages and we are using the new Spanish Version. We show how this tool can be used for a continuous user experience assessment.
Full-text available
Over the last decade, 'user experience' (UX) became a buzzword in the field of human – computer interaction (HCI) and interaction design. As technology matured, interactive products became not only more useful and usable, but also fashionable, fascinating things to desire. Driven by the impression that a narrow focus on interactive products as tools does not capture the variety and emerging aspects of technology use, practitioners and researchers alike, seem to readily embrace the notion of UX as a viable alternative to traditional HCI. And, indeed, the term promises change and a fresh look, without being too specific about its definite meaning. The present introduction to the special issue on 'Empirical studies of the user experience' attempts to give a provisional answer to the question of what is meant by 'the user experience'. It provides a cursory sketch of UX and how we think UX research will look like in the future. It is not so much meant as a forecast of the future, but as a proposal – a stimulus for further UX research.
Conference Paper
Full-text available
An end-user questionnaire to measure user experience quickly in a simple and immediate way while covering a preferably comprehensive impression of the product user experience was the goal of the reported construction process. An empirical approach for the item selection was used to ensure practical relevance of items. Usability experts collected terms and statements on user experience and usability, including ‘hard’ as well as ‘soft’ aspects. These statements were consolidated and transformed into a first questionnaire version containing 80 bipolar items. It was used to measure the user experience of software products in several empirical studies. Data were subjected to a factor analysis which resulted in the construction of a 26 item questionnaire including the six factors Attractiveness, Perspicuity, Efficiency, Dependability, Stimulation, and Novelty. Studies conducted for the original German questionnaire and an English version indicate a satisfactory level of reliability and construct validity.
Over the past two years the Ubicomp vision of ambient voice assistants, in the form of smart speakers such as the Amazon Echo and Google Home, has been integrated into tens of millions of homes. However, the use of these systems over time in the home has not been studied in depth. We set out to understand exactly what users are doing with these devices over time through analyzing voice history logs of 65,499 interactions with existing Google Home devices from 88 diverse homes over an average of 110 days. We found that specific types of commands were made more often at particular times of day and that commands in some domains increased in length over time as participants tried out new ways to interact with their devices, yet exploration of new topics was low. Four distinct user groups also emerged based on using the device more or less during the day vs. in the evening or using particular categories. We conclude by comparing smart speaker use to a similar study of smartphone use and offer implications for the design of new smart speaker assistants and skills, highlighting specific areas where both manufacturers and skill providers can focus in this domain.
Conference Paper
Recently, commercial Voice User Interfaces (VUIs) have been introduced to the market (e.g. Amazon Echo and Google Home). Although they have drawn much attention from users, little is known about their usability, user experiences, and usefulness. In this study, we conducted a web-based survey to investigate usability, user experiences, and usefulness of the Google Home smart speaker. A total of 114 users, who are active in a social-media based interest group, participated in the study. The findings show that the Google Home is usable and user-friendly for users, and shows the potential for international users. Based on the users' feedback, we identified the challenges encountered by the participants. The findings from this study can be insightful for researchers and developers to take into account for future research in VUI.
Conference Paper
In-home, place-based, conversational agents have exploded in popularity over the past three years. In particular, Amazon's conversational agent, Alexa, now dominates the market and is in millions of homes. This paper presents two complementary studies investigating the experience of households living with a conversational agent over an extended period of time. First, we gathered the history logs of 75 Alexa participants and quantitatively analyzed over 278,000 commands. Second, we performed seven in-home, contextual interviews of Alexa owners focusing on how their household interacts with Alexa. Our findings give the first glimpse of how households integrate Alexa into their lives. We found interesting behaviors around purchasing and acclimating to Alexa, in the number and physical placement of devices, and in daily use patterns. Participants also uniformly described interactions between children and Alexa. We conclude with suggestions for future improvement for intelligent conversational agents.
Digitization, the rise of the Internet and mobile devices have changed the way people interact with each other and with companies. In recent years, the voice interface has become a growing feature in mobile devices. Industry reports indicate that in mid-2016, 20 percent of Android searches were voice-based and Siri received two bn. requests per week. ComScore predicts that by 2020, 50 percent of all searches will be voice searches. Furthermore, it is anticipated that voice will become the default method to control a variety of interfaces including mobile devices, Internet of Things (IoT) appliances, and automobiles.
Conference Paper
Voice interface is becoming a common feature in mobile devices such as tablets and smartphones. Moreover, voice recognition technology is touted to mature and become the default method to control of a variety of interfaces, including mobile devices. Thus, it is critical to understand the factors that influence the use of voice activated applications in the public domain. The present study examined how the perceived acceptability of using the Voice-Activated Personal Assistant (VAPA) in smartphones influences its reported use. Participants were U.S. smartphone users recruited from Amazon Mechanical Turk. Results showed that participants preferred using the VAPA in a private location, such as their home, but even in that environment, they were hesitant about using it to input private or personally identifying information in comparison to more general, non-private information. Participants’ perceived social acceptability of using the VAPA to transmit information in different contexts could explain these preferred usage patterns.
Distribution of university students enrolled at Spanish universities in 2018/2019, by field of study and gender
  • Fernando Forte
Fernando Forte. Distribution of university students enrolled at Spanish universities in 2018/2019, by field of study and gender, 2020.
STEM first-year students at German universities by
  • Statista Gmbh
Statista GmbH. STEM first-year students at German universities by 2019/2020, 2020.
Usability measurement for speech systems : SASSI revisited
  • Kate Hone
Kate Hone. Usability measurement for speech systems : SASSI revisited. In Proceedings of CHI, 2014.
An Introduction to Voice Assistants
  • B Matthew
  • Hoy
  • Alexa
  • Cortana Siri
Matthew B Hoy. Alexa, Siri, Cortana, and More: An Introduction to Voice Assistants. Medical Reference Services Quarterly, 37(1):81-88, 2018.
Voice Assistant Consumer Adoption Report
  • Bret Kinsella
  • Ava Mutchler
Bret Kinsella and Ava Mutchler. Voice Assistant Consumer Adoption Report. Technical report,, 2018.
Exploring Voice Assistant Risks and Potential with Technology-based Users
  • Andreas M Klein
  • Andreas Hinderks
  • Maria Rauschenberger
  • Jörg Thomaschewski
Andreas M. Klein, Andreas Hinderks, Maria Rauschenberger, and Jörg Thomaschewski. Exploring Voice Assistant Risks and Potential with Technology-based Users. In Proceedings of the 16th International Conference on Web Information Systems and Technologies -Volume 1: WEBIST,, pages 147-154. INSTICC, SciTePress, 2020.
With P from pandemic podcast II Voice and audio study in Spain
  • Prodigioso Volcán
Prodigioso Volcán. With P from pandemic podcast II Voice and audio study in Spain, 2020.
Source Code von mächtigem DDoS-Tool Mirai veröffentlicht
  • Dennis Schirrmacher
Dennis Schirrmacher. Source Code von mächtigem DDoS-Tool Mirai veröffentlicht, 2016.
Voice Assistants in Germany. WIK Wissenschaftliches Institut für Infrastruktur und Kommunikationsdienste GmbH
  • Serpil Tas
  • Christian Hildebrandt
  • René Arnold
Serpil Tas, Christian Hildebrandt, and René Arnold. Voice Assistants in Germany. WIK Wissenschaftliches Institut für Infrastruktur und Kommunikationsdienste GmbH, 2019. Nr.441.