Martin Schrepp

SAP Research | SAP · SAP Cloud Platform

Protocol for Validation of the UEQ+ Scales for Voice Quality Version 02 / 2025

Measuring User Experience (UX) with standardized questionnaires is a widely used method. A questionnaire is based on different scales that represent UX factors and items. However, the questionnaires have no common ground concerning naming different factors and the items used to measure them. This study aims to identify general UX factors based on t...

Questionnaires are a widely used tool for measuring the user experience (UX) of products. There exists a huge number of such questionnaires that contain different items (questions) and scales representing distinct aspects of UX, such as efficiency, learnability, fun of use, or aesthetics. These items and scales are not independent; they often have...

Measuring User Experience (UX) with questionnaires is essential for developing and improving products. However, no domain-specific standardized UX questionnaire exists for Augmented Reality (AR) in Corporate Training (CT). Thus, this study introduces the UXAR-CT questionnaire - an AR-specific UX questionnaire for CT environments. We describe the co...

User Experience (UX) Research covers various methods for gathering the users' subjective impressions of a product. For this, practitioners face different activities and tasks related to the research process. This includes processing a large amount of data based on qualitative and quantitative data. However, this can be very laborious in practice. T...

Standardized questionnaires are an efficient and reliable method to measure the user experience of a product, service or system. However, response rates of such surveys are often quite low. The length of a survey has an impact on the willingness to respond. We investigate in this paper if it is useful to split a questionnaire into single items to r...

Measuring User Experience (UX) with questionnaires is essential for developing and improving products. However, no domain-specific standardized UX questionnaire exists for Augmented Reality (AR) in Corporate Training (CT). Thus, this study introduces the UXAR-CT questionnaire-an AR-specific UX questionnaire for CT environments. We describe the cons...

User Experience (UX) Research covers various methods for gathering the users' subjective impressions of a product. For this, practitioners face different activities and tasks related to the research process. This includes processing a large amount of data based on qualitative and quantitative data. However, this can be very laborious in practice. T...

Converting customer survey feedback data into usable insights has always been a great challenge for large software enterprises. Despite the improvements on this field, a major obstacle often remains when drawing the right conclusions out of the data and channeling them into the software development process. In this paper we present a practical end-...

Questionnaires are a widely used tool for measuring the user experience (UX) of products. There exists a huge number of such questionnaires that contain different items (questions) and scales representing distinct aspects of UX, such as efficiency, learnability, fun of use, or aesthetics. These items and scales are not independent; they often have...

Applying Mobile Augmented Reality (MAR) in a Corporate Training (CT) environment can improve the learning effectiveness of trainees. Therefore, a positive User Experience (UX) concerning the MAR application is crucial. To ensure this, the UX of MAR must be measured. However, hardly any standardized UX questionnaire for MAR can be found in the liter...

Protocol of Prolific data collection of Alexa/Siri/Google Assistant evaluation in the UK/USA with UEQ+ Scales for Voice Quality

As digital technologies advance, user experience (UX) has become crucial for software and services success. The User Experience Questionnaire Plus (UEQ+) is a flexible tool used to evaluate UX through questionnaires tailored to specific problems, yet a critical factor often overlooked is Trust. Trust, understood as a user's belief in a software's a...

Measuring user experience is vital for long-term success of interactive products. Questionnaires like the modular extension of the User Experience Questionnaire (UEQ+) are an established instrument for this purpose. Different item formats are available for these questionnaires, such as the number of response options (most frequent 5-or 7-point Like...

Measuring User Experience (UX) with standardized questionnaires is a widely used method. A questionnaire is based on different scales that represent UX factors and items. However, the questionnaires have no common ground concerning naming different factors and the items used to measure them. This study aims to identify general UX factors based on t...

Users form an overall impression concerning the user experience (UX) based on their perception of special UX qualities. Therefore, measuring users’ perceptions of these particular UX aspects is essential for determining the UX of a product. The measured hedonic qualities, e.g. stimulation or aesthetics, and pragmatic qualities, e.g. efficiency or l...

User experience (UX) is a holistic concept. We conceptualize UX as a set of semantically distinct quality aspects. These quality aspects relate subjectively perceived properties of the user interaction with a product to the psychological needs of users. Not all possible UX quality aspects are equally important for all products. The main use case of...

This report contains the data of several studies that investigate the effect of a polarity change for the UEQ. Currently, items of the UEQ are semantic differentials with randoomized polarity (positive term is in half of the items placed in the left and in the other half of the items in the right position). This original version is compared in stud...

An analysis of the typical standard deviations for UEQ scales and a description how this can be applied for the estimation of required sample sizes for studies and for an interpretation how much participants agree or disagree in their evaluation of UX quality aspects.

Report explaining the enhanced heuristics for data cleansing of the UEQ and UEQ-S questionnaires that are implemented in the data analysis sheets.

A loyal customer base depends upon a good user experience over the product’s complete lifetime. Successful products are continuously developed over a long period. Their functionality and complexity typically grow over years, so it is important to measure their user experience continuously. A carefully selected, effective questionnaire can collect q...

This paper aims to adapt and validate the User Experience Questionnaire Plus (UEQ+) in the Indonesian context. The UEQ+ is a modular extension of the User Experience Questionnaire (UEQ), which has been adapted to the Indonesian context and used in many studies. The UEQ+ was originally developed in German and English. As a modular extension, the UEQ...

In order to be able to meaningfully classify the user experience and thus the popularity of products, UX questionnaires such as the UEQ, SUS or UMUX are frequently used in practice to measure the UX. This makes it possible to specifically evaluate the ratings of pragmatic and hedonic UX factors. However, it is conceivable that, in addition to users...

Protocol for A Comparison of three short User Experience Questionnaires. How different are the results of SUS, UMUX-LITE and UEQ-S for common interactive products?

Questionnaires are a popular method to measure User Experience (UX). These UX questionnaires cover different UX aspects with their scales. However, UX includes a huge number of semantically different aspects of a user's interaction with a product. It is therefore practically impossible to cover all these aspects in a single evaluation study. A rese...

Inductive item tree analysis is an established method of Boolean analysis of questionnaires. By exploratory data analysis, from a binary data matrix, the method extracts logical implications between dichotomous test items. For example, assume that we have the problems i and j of a test that can be solved or failed by subjects. With inductive item t...

Die User Experience ist ein relevanter Aspekt für eine erfolgreiche Produktentwicklung. Ein geeignetes Instrument für die quantitative Messung der User Experience sind Fragebögen wie der UEQ+. Dieser erlaubt es, mithilfe der zur Verfügung gestellten Skalen produktindividuelle Fragebögen zu erstellen. Es ist jedoch wahrnehmbar, dass manche UX-Faktor...

Version 1.0/2021 des Protokolls zur Ermittlung relevanter UX-Faktoren je Produktkategorie für den UEQ+

Protocol of determining important UX factors from the UEQ+ framework using importance evaluations of different established software products and product categories and rank correlation analysis

A good user experience is extremely important for the success of an interactive product. It is therefore important for the design and improvement of the product to be able to measure this aspect. Different users often have strongly diverging opinions about the user experience of a product. Because questionnaires allow data to be collected from larg...

This paper aims to evaluate the user experience of a mobile health application called Halodoc to keep the user using the application and keep from losing a potential source of revenue for Halodoc. Halodoc is one of the companies that use the internet to provide health services for its users. Halodoc has services such as features for consultation wi...

Inductive item tree analysis is an established method of Boolean analysis of questionnaires. By exploratory data analysis, from a binary data matrix, the method extracts logical implications between dichotomous test items based on their positive item scores. For example, assume that we have the problems i and j of a test that can be solved or faile...

The UEQ+ is a modular framework for the construction of UX questionnaires. The researcher can pick those scales that fit his or her research question from a list of 16 available UX scales. Currently, no UEQ+ scales are available to allow measuring the quality of voice interactions. Given that this type of interaction is increasingly essential for t...

If, as UX professionals, we measure the user experience of a product with a questionnaire, then scale reliability is important. If the scales of the questionnaire show low reliability, we cannot guarantee stable measurements and should not base important design decisions on such data. There are several methods available to estimate the reliability...

Klein, A. M., Hinderks, A. Schrepp, M. and Thomaschewski, J., (2020) describe the construction of three scales to measure UX aspects specific to voice systems and how these can be used in the UEQ+ framework (Schrepp & Thomaschewski, 2019) to measure UX of such systems. This research report gives detailed information about data analysis done for VUI...

To be successful, interactive products need to fulfil user expectations and create a positive user experience (UX). An established method to measure UX involves questionnaires. What we aim in this paper is to present a list of user experience and usability questionnaires and its applicability for different digital products. A total of 13 questionna...

Several studies reported a dependency between perceived beauty and perceived usability of a user interface. But it is still not fully clear which psychological mechanism is responsible for this dependency. We suggest a new explanation based on the concept of visual clarity. This concept describes the perception of order, alignment and visual comple...

We study three web sites to see whether there are systematic differences between women and men in their rating of the user experience of the sites. One of the sites addresses especially the target group of women, another the target group of men, whereas the third site is neutral in this respect. The selection of the sites was safeguarded with gende...

Existing user experience questionnaires have a fixed number of scales. Each of these scales measures a distinct aspect of user experience. These questionnaires can be used with little effort and provide a number of useful support materials that make the application of such a questionnaire quite easy. However, in practical evaluation scenarios it ca...

Slides and results of the workshop that took place on Sept 11, 2019 at UP2019 in Hamburg.

Wir stellen einen modularen Ansatz vor, mit dem man sich aus einem Katalog von 16 UX Aspekten einen perfekt passenden UX Fragebogen bauen kann. Grundlage sind die 6 Skalen des UEQ, die um 10 weitere Skalen erweitert wurden. Wir beschreiben die Konstruktion und erste Evaluationsergebnisse zu diesen neuen Skalen. Es wird weiterhin beschrieben, wie UX...

We investigate if the cultural background of a person influences the subjective importance of UX aspects for several common product categories. To clarify this, we replicated a published study with German students in Indonesia. Results show significant differences concerning the rated importance of UX aspects for many product categories. However, a...

(Experten-)Reviews sind eine wirksame, kostengünstige und bewährte Methode, Softwareprodukte in nahezu allen Design-oder Entwicklungsstadien zu evaluieren. Beim Einsatz in der betrieblichen Praxis stellen sich für UX Professionals jedoch eine Reihe von zentralen Fragen: Wie sieht es mit der Akzeptanz der Ergebnisse bei Management und Entwicklung au...

Neben Usability-Tests und dem Einsatz von Fragebögen, sind Experten-Reviews eine kostengünstige Methode, die User Experience (UX) eines Produkts zu beurteilen. Wir stellen ein einfaches und flexibel einsetzbares Verfahren zum Experten-Review vor, das sowohl pragmatische als auch hedonische UX Aspekte berücksichtigt und gleichermaßen geeignet ist, S...

Ziel des Workshops ist es, Personen zusammenzubringen, die an der Entwicklung von UX Fragebögen arbeiten oder Erfahrung im praktischen Einsatz solcher Fragebögen haben. Es sollen Erfahrungen ausgetauscht werden und offene Fragestellungen und Probleme angesprochen werden. Mögliche Themen betreffen methodische Fragen bei der Entwicklung und Validieru...

Wortwolken sind eine häufig verwendete Darstellungsform, um die Bedeutung von Begriffen in einem bestimmten Kontext zu visualisieren. Sie sind einfach zu interpretieren und bei geeignetem Layout ein ansprechendes und interessantes Gestaltungselement. Wir zeigen, wie man Wortwolken aus etablierten Fragebögen zur User Experience erzeugen und zur Komm...

Many software products are created by considering targeted users from different countries. For such products, it is significant to investigate users' expectations related to cultural aspects to achieve user acceptance as broad as possible in all relevant countries. The researchers of the current study examined whether the cultural background of an...

User Experience Review (short UER) is a procedure to conduct an expert review of a product concerning its user experience. This handbook describes the basic steps of the method. In addition, the materials that are available to support UX professionals to set up an UER with minimal effort are described.

Decisions in Companies are made typically by using a number of entirely different key figures. A user experience key figure is one of many important key figures that represents one aspect of the success of the company or its products. What we aim in this article is to present to those responsible for a product a method of how a user experience key...

The UEQ (Laugwitz, Schrepp & Held, 2008) is a frequently used questionnaire that measures user experience (short UX) on 6 distinct scales (Attractiveness, Efficiency, Perspicuity, Dependability, Stimulation, Novelty). Of course, these 6 scales do not cover the entire spectrum of UX. For some products special UX aspects not contained in the UEQ are...

This handbook describes how to use the UEQ+ (a modular Extension of the User Experience Questionnaire UEQ). Scales, items and basic information about best practices are provided.

Quasi-orders are reflexive and transitive binary relations and have many applications. Examples are the dependencies of mastery among the problems of a psychological test, or methods such as item tree or Boolean analysis that mine for quasi-orders in empirical data. Data mining techniques are typically tested based on simulation studies with unbias...

Quasi-orders are reflexive and transitive binary relations and have many applications. Examples are the dependencies of mastery among the problems of a psychological test, or methods such as item tree or Boolean analysis that mine for quasi-orders in empirical data. Data mining techniques are typically tested based on simulation studies with unbias...

The formula described here is to combine for every participant the value and assessed importance of each UEQ scale. Therefore, we calculated for each participant the relative importance of each scale and multiplied this with the value of the UEQ scale. This allows to generate a meaningful UX KPI based on the UEQ, that we have named UEQ KPI. The ste...

To enable an interactive product to provide adequate user experience (UX), it is important to ensure the quantitative measurability of this parameter. The User Experience Questionnaire (UEQ) is a well-known and popular method for such a UX measurement. One of the key features of this questionnaire is a benchmark that helps to interpret measurement...

Zusammenfassung UX-Fragebögen werden oft eingesetzt, um die User Experience eines Produktes zu ermitteln. Etabliert haben sich im deutschsprachigen Raum einige Fragebögen, die einen unterschiedlichen Bekanntheits-grad haben und in der Praxis oftmals aufgrund der Bekanntheit verwendet werden. In diesem Beitrag werden die Dimensionen der etablierten...

Wir untersuchen drei verbreitete Anwendungen (Whatsapp, Netflix, Moodle) mit drei im deutschsprachigen Raum häufig verwendeten UX Fragebögen (VISAWI, meCUE, UEQ). Ziel ist die Beziehung der Skalen dieser Fragebögen näher zu untersuchen. Überraschenderweise zeigte sich, dass die Korrelationen der Skalen innerhalb eines Fragebogens stets deutlich höh...

User Experience (UX) hat viele Facetten. Zahlreiche Standardfragebögen beanspruchen, solche Facetten in jeweils eigenen Skalen zu messen; diese sind jedoch regelmäßig hoch miteinander korreliert. Dem gegenüber steht die Beobachtung, dass die Skalenwerte einzelner Befragungsteilnehmer in der Regel erheblich streuen. Ein Konzept einer einheitlichen S...

We investigate if the cultural background of a person has an influence on the subjective importance of UX aspects for several common product categories. To clarify this, we replicate an already published study with German students (Winter et al., 2017) in Indonesia. Results show significant differences concerning the rated importance of UX aspects...

Fragebögen sind eine weit verbreitete Methode zur Messung von User Experience (UX). Die Methoden zur Konstruktion und Validierung von UX Fragebögen orientieren sich aktuell an den etablierten Methoden der psychologischen Testtheorie. Allerdings gibt es einige prinzipielle Unterschiede zwischen einem UX Fragebogen und einem klassischen psychologisch...

Fragebögen sind eine weit verbreitete Methode zur Messung der User Experience. Allerdings gibt es bzgl. Konstruktion und Einsatz von UX Fragebögen auch noch eine ganze Reihe schwieriger und noch nicht vollständig gelöster Probleme. Ziel des Workshops ist es, Personen zusammenzubringen, die an der Entwicklung von UX Fragebögen arbeiten oder viel Erf...

Experten-Reviews sind eine sehr schnell durchführbare und kostengünstige Methode, um die User Experience einer Anwendung zu beurteilen. Allerdings sind die bekannten Review-Verfahren stark auf das Auffinden von konkreten Nutzungsproblemen fokussiert. Heute hat sich aber eine deutlich weitere Sicht auf UX etabliert, d.h. es spielen auch nicht-aufgab...

Zweifellos ist die Umsetzung von Prinzipien des User Centered Design eine entscheidende Voraussetzung dafür, Endbenutzern Software zur Verfügung zu stellen, die möglicht effektiv, effizient und zufriedenstellend ist. Im Idealfall überzeugt die Gebrauchstauglichkeit eines Softwareprodukts auch den Pesrsonenkreis, der über den Kauf dieser Software en...

A good user experience is a must for modern software products. However, in software product design we have to often deal with conflicting requirements. A new software product feature may increase efficiency, but on the other hand decrease learnability of the product. Such types of conflicts often cause lengthy discussions, since it is often not cle...

In diesem Evaluationsbericht sind die detaillierten Ergebnisse einer Studie mit drei verschiedenen UX Fragebögen und drei Testobjekten dargestellt. In dieser Studie sollten UX Fragebögen miteinander verglichen werden, die sowohl in einer deutschen Version frei zur Verfügung stehen als auch über die notwendigen Auswertetools verfügen. Wir haben den...

Eine gute User Experience ist für den Erfolg eines interaktiven Produkts sehr wichtig. Für die Gestaltung und die kontinuierliche Verbesserung des Produktdesigns ist es daher hilfreich, diesen Qualitätsaspekt auch quantitativ messen zu können. Andererseits haben verschiedene Nutzer in Bezug auf die User Experience eines Produkts oft stark abweichen...

In many companies, decisions are mostly made based on the key figures, such as turnover, profit, employee satisfaction, or the Net Promoter Score (NPS). Managers are accustomed to information being summarized and available via key figures. This leads to the demand for the user experience to also be mapped in a single KPI (UX KPI). Our main goal i...

Langwierige Diskussionen, bei denen sich die beteiligten Personen über einzelne Aspekte eines Designs nicht einigen können, sind für erfahrene UX-Professional nichts Neues. Solche Konflikte sind in der Regel kraft- und zeitraubend. Diese Konflikte resultieren oft aus unterschiedlichen Einschätzungen bzgl. der Wichtigkeit bestimmter Qualitätseigensc...

Manager sind daran gewöhnt, Informationen zu wichtigen Aspekten der aktuellen Unternehmenssituation in einer einzigen Kennzahl (KPI) zusammenzufassen, z.B. Umsatz oder Gewinn. Das führt naturgemäß zum Wunsch, auch bezüglich User Experience (UX) eine solche Kennzahl zu nutzen. Andererseits ist UX ein multidimensionales Konstrukt. Damit eine gute UX...

Der User Experience Questionnaire (UEQ) ist ein weit verbreiteter Fragebogen zur Messung der subjektiv wahrgenommenen User Experience. Der Fragebogen ist mit 3-5 Minuten Zeitbedarf schon recht effizient, aber für einige spezielle Anwendungsszenarien noch zu aufwändig. Wir beschreiben in diesem Beitrag die Erstellung einer Kurzversion des UEQ mit nu...

Many products are developed for an international audience. But product design is typically done by a small group of designers, which is often homogeneous concerning their cultural background. In addition, the adaption of products to different cultural contexts is an expensive exercise and is thus in most cases not even considered in the product pla...

Products should guarantee a sufficiently high user experience for all intended user groups. A good user experience means not only that it is sufficient to allow users to work effectively and efficiently with a product, but also that non-task-related quality aspects need to be considered. To attain various levels of user experience thus requires an...

Questionnaires are a cheap and highly efficient tool for achieving a quantitative measure of a product’s user experience (UX). However, it is not always easy to decide, if a questionnaire result can really show whether a product satisfies this quality aspect. So a benchmark is useful. It allows comparing the results of one product to a large set of...

The user experience questionnaire (UEQ) is a widely used questionnaire to measure the subjective impression of users towards the user experience of products. The UEQ is a semantic differential with 26 items. Filling out the UEQ takes approximately 3-5 minutes, i.e. the UEQ is already reasonably efficient concerning the time required to answer all i...

Quasi-orders, that is, reflexive and transitive binary relations, have numerous applications. In educational theories, the dependencies of mastery among the problems of a test can be modeled by quasi-orders. Methods such as item tree or Boolean analysis that mine for quasi-orders in empirical data are sensitive to the underlying quasi-order structu...

Zusammenfassung Fast alle aktuell verwendeten Fragebögen zur Messung von Usability bzw. User Experience basieren auf faktorenanalytischen Verfahren. Bei der Durchführung der Faktorenanalyse ist unter anderem die Größe der Stichproben von Bedeutung. Zur besseren Interpretation der gängigen Empfehlungen über die Stichprobengröße wurde ein Algorithmus...