ArticlePDF Available

Efficient Measurement of the User Experience of Interactive Products. How to use the User Experience Questionnaire (UEQ). Example: Spanish Language Version

Authors:

Abstract and Figures

Developer, manager and user feedback is needed to optimize products. Besides the basic Software qualities textendash usability and user experience are important properties for improving your product. Usability is well known and can be tested with e.g. a usability test or an expert review. In contrast user experience describes the whole impact a product has on the end-user. The timeline goes from before, while and after the use of a product. We present a tool that allows you to evaluate the user experience of a product with little effort. Furthermore the tool is available in different languages and we are using the new Spanish Version. We show how this tool can be used for a continuous user experience assessment.
Content may be subject to copyright.
International Journal of Artificial Intelligence and Interactive Multimedia, Vol. 2, Nº 1.
-39-
Abstract Developer, manager and user feedback is needed to
optimize products. Besides the basic Software qualities usability
and user experience are important properties for improving your
product.
Usability is well known and can be tested with e.g. a usability
test or an expert review. In contrast user experience describes the
whole impact a product has on the end-user. The timeline goes
from before, while and after the use of a product. We present a
tool that allows you to evaluate the user experience of a product
with little effort. Furthermore the tool is available in different
languages and we are using the new Spanish Version. We show
how this tool can be used for a continuous user experience
assessment.
Keywords Software Quality, User Experience,
Questionnaire, Usability, Test, Development
I. INTRODUCTION
S your redesign of the website better than the old version?
Has the development effort spent to increase user experience
really paid off? If you want to answer such questions you need
a quantitative method to measure user experience [1]. An
efficient and inexpensive method to do such measurements is
the usage of rigorously constructed and validated
questionnaires.
The concept of user experience combines well-known
aspects like efficiency and effectiveness with additional
criteria like aesthetics, joy-of-use or attractiveness. The first
group of criteria is often referred as pragmatic quality aspects
[2], while the second group is called hedonic quality aspects.
Another often-used terminology to distinguish both classes of
quality criteria is usability goals versus user experience goals
[3]. The dependency of pragmatic and hedonic quality is
presented in Fig. 1.
One well investigated research question is the relationship
of pragmatic and hedonic quality. Empirical evidence proves
that products, which are perceived to show a high level of
hedonic quality, are also perceived as easy to use [4], [5], [6].
These and similar observations cause some authors [7] to state
that ‘What is beautiful is usable’. In contrast other studies
point out [8], [9] an opposite dependency. The perception of
the aesthetic value of a user interface increased when the
number of concrete usability problems decreased. Thus, in this
study a What is usable is beautiful effect was observed.
Fig. 1. Grouping of different quality attributes.
Why are perceived hedonic and pragmatic quality aspects
associated? As possible explanation for this connection halo-
effects [10], mediation by the mood of the user [11] or
mediation by other variables [6] have been suggested. Since it
is quite difficult to separate these effects experimentally [8] it
is currently unclear which of these hypotheses are able to
explain this effect.
These results indicate that it is necessary to consider both
pragmatic and hedonic aspects if we want to measure how
satisfied users are with a given product.
This is the underlined idea of constructing the User
Experience Questionnaire (UEQ) [12], [13] that is described in
this paper. In the context of the questionnaire user experience
is understood as the overall impression of a user when he or
she interacts with a product, i.e. covers both pragmatic and
hedonic quality aspects.
The UEQ allows a quick assessment of the user experience
for any interactive product. The scales of the questionnaire are
designed to cover a comprehensive impression of user
experience. The questionnaire format supports the user
response to immediately express feelings, impressions, and
attitudes that arise when they use a product.
If a new product is rolled out or if an existing product is
evaluated the first time typical questions are ‘Does the product
create a positive user experience?’ or How do users feel
about the product?’. To answer such questions it is sufficient
that a representative sample of users of the new product fill out
the UEQ. 30 answers are usually enough to get a valid
impression. For example, the answers can come from
participants of a usability test or pilot users.
Efficient Measurement of the User Experience
of
Interactive Products. How to use the User
Experience
Questionnaire (UEQ).Example: Spanish Language Version
Maria Rauschenberger, MSP Medien-Systempartner, Germany; Martin Schrepp, SAP AG, Germany
Manuel Pérez Cota, University of Vigo, Spain; Siegfried Olschner, DATEV eG, Germany
Jörg Thomaschewski, University of Applied Science Emden/Leer, Germany
I
DOI: 10.9781/ijimai.2013.215
Special Issue on Artificial Intelligence and Social Application
-40-
Another application is the continuous quality assessment of
a software product within a development process [14]. In this
approach a measurement with the UEQ is collected with each
new version of the software. Thus, we can directly see if new
versions bring an improvement in user experience if the scale
values for the six scales of the UEQ increase with the new
version (for an example on the concrete implementation of
such a process, see [14]). An application of the UEQ in the
process of idea and innovation management is described in
[15].
User experience is not only a snapshot of the present usage
a product has. It is an entire impression a product makes on the
user. Even more, the user’s judgement starts before touching
and using a new product. In addition the change of impression
carries on during and after the usage [1]. The UEQ is able to
present the distinct results over time for the result analysis.
The UEQ is a semantic differential. For such questionnaires
it is especially important that users see the items in their native
language. So far the UEQ was available in German, English,
French and Italian. We present in this paper the Spanish
language version of the questionnaire.
We describe in the following how the UEQ was constructed
and validated. In addition, the structure of the questionnaire
and the meaning of the subscales are explained. We then show,
how the UEQ should be applied in a company and how the
results can be analyzed. Besides, the DATEV eG a big
business software company is presenting their design process
with the UEQ. Finally, we describe the creation of the Spanish
language version of the UEQ.
II. CONSTRUCTION AND VALIDATION OF THE USER EXPERIENCE
QUESTIONNAIRE (UEQ)
The items and scales of the UEQ were created by a data
analytical approach. First, a set of 229 potential items was
built as a result of several brainstorming sessions with usability
experts. Second, this set was reduced to an 80 items raw
version by an expert evaluation. Third, the eighty items raw-
version of the questionnaire was used in several studies
focusing on the quality of interactive products, including e. g.
a statistics software package, cell phone address book, online-
collaboration software, or business software. In total the data
of 153 participants were collected for the initial data set.
Finally, the scales and the items representing each scale were
extracted from the data by factor analysis (principal
components, varimax rotation). Six factors resulted from this
analysis. Details concerning the process can be found in [12],
[13].
The reliability (i.e. the scales are consistent) and validity
(i.e. the scales do really measure what they intend to measure)
of the UEQ scales was investigated in several studies (in 11
usability tests with a total number of 144 participants and an
online survey with 722 participants). A review of all available
studies showed that reliability (Cronbach’s Alpha was used for
an estimation of internal consistency) of the scales was
sufficiently high. In addition, the validity of the scales was
investigated in a number of studies [12], [13], [14]. Results
indicate good construct validity.
III. STRUCTURE OF THE QUESTIONNAIRE
The user experience questionnaire contains 6 scales with 26
items in total:
1) Attractiveness: General impression towards the product.
Do users like or dislike the product? This scale is a pure
valence dimension. Items: annoying / enjoyable, good /
bad, unlikable / pleasing, unpleasant / pleasant, attractive
/ unattractive, friendly / unfriendly
2) Efficiency: Is it possible to use the product fast and
efficient? Does the user interface looks organized? Items:
fast / slow, inefficient / efficient, impractical / practical,
organized / cluttered
3) Perspicuity: Is it easy to understand how to use the
product? Is it easy to get familiar with the product? Items:
not understandable / understandable, easy to learn /
difficult to learn, complicated / easy, clear / confusing
4) Dependability: Does the user feel in control of the
interaction? Is the interaction with the product secure and
predicable? Items: unpredictable / predictable,
obstructive / supportive, secure / not secure, meets
expectations / does not meet expectations
5) Stimulation: Is it interesting and exciting to use the
product? Does the user feel motivated to further use the
product? Items: valuable / inferior, boring / exiting, not
interesting / interesting, motivating / demotivating
6) Novelty: Is the design of the product innovative and
creative? Does the product grab users attention? Items:
creative / dull, inventive / conventional, usual / leading
edge, conservative / innovative
The dependency of the UEQ scale is presented in Fig. 2.
Fig. 2. Scale structure of the UEQ questionnaire.
For the specific questionnaire the order of the items and
their orientation (starting with the positive or the antonym
statement) is randomized. The specific English questionnaire
is shown in Fig. 3 and the Spanish questionnaire is shown in
Fig. 7.
International Journal of Artificial Intelligence and Interactive Multimedia, Vol. 2, Nº 1.
-41-
Fig. 3. English version of the UEQ
IV. HOW TO APPLY THE QUESTIONNAIRE
For a successful application of the UEQ the acceptance of
following two groups are needed: Users and Managers. To
achieve a high user acceptance of the UEQ you should take
following points into account:
- Background and benefits of the method should be clear to
the user
- A personal contact should be available for the responders
- The time interval between repeated measurements should
be long enough
To achieve acceptance by product managers consider to:
- Provide help during the interpretation of the UEQ factor
values. Define your range of good, medium and bad
and explain the theoretical background
- Combine old and new UEQ values in one picture and
show the changes during the development in order to
increase the intelligibility of the measurements
- Search for other user feedback that supports the
interpretation of the UEQ outcome and integrate it into
your report
- Enhance the UEQ results with concrete enhancement
suggestions based on user experience expertise and use
this as a base for further discussions about the next
development goals
V. ANALYZING RESULT
After collecting the answers from the users a three step
analysis as presented can follow. To reduce the effort for data
analysis an MS Excel file is created, doing all the necessary
calculations. Only the raw data of the questionnaire results
have to be entered into the tool. The tool then calculates the
scale values, creates a bar chart to visualize the results and
calculates some basic statistical indicators necessary for an
interpretation of the data, for example confidence intervals for
the scales. Fig. 4 presents an example of a result and Fig. 45
shows an example of a comparison of two product versions.
A. Verifying the validation
The first step is to confirm the Cronbach’s Alpha data,
which describes the consistency of the items of the scales (i.e.
if all items in the scale measure the same quality). It is
calculated automatically for each study in the excel sheet
which can be downloaded from www.ueq-online.org.
If the Alpha value for a scale is small this is an indication
that some of the items in this scale are possibly misinterpreted
or interpreted in a direction that does not reflect their intention
in the context of the UEQ. In this case it is questionable if this
specific scale can be interpreted for the final result.
There are two well-known effects that can cause a small
value of the Alpha-Coefficient for a scale. First, it is possible
that the context in which the questionnaire is applied yields to
a misinterpretation of some items in the scale. For example, in
a study with informatics students the item ‘secure/not secure’
was referred from the users to the security (i.e. absence of
malware or spyware) of the web-service and not to the
dependability of the interaction.
Second, a scale may be simply irrelevant in the context in
which the questionnaire is applied. Thus, the participants may
have problems to interpret the items of the scale properly,
which lowers the correlations between the items of the scale
and thus decreases the Alpha-Coefficient.
If the alpha coefficient is higher or equal than 0,7 the scales
show high consistency, i.e. all items in a scale measure the
same aspect and it is unlikely that one of the items is
misinterpreted in the given context.
But it can also happen that all items in a scale are influenced
by a context specific effect, i.e. one of the scales differs highly
from the other scales due to a special target group.
In a study with 20 participants the scale novelty had low
results caused by a target group with different age. The VoIP-
Software Skype was evaluated. The younger group had no
enthusiasm about the technology, because they had known it
for a long time. It was not exciting anymore. Elsewise the
older group did not know Skype or any similar product. It was
their first contact with this technology and they found it very
fascinating. The consequence was that one group perceived
Skype very stimulating and the other not.
Special Issue on Artificial Intelligence and Social Application
-42-
After exanimating the Alpha value next step is the
interpreting of the overall result as descripted in Chapter B.
B. Intepretate the overall result
The items are scaled from -3 to +3. Thus, -3 represents the
most negative answer, 0 a neutral answer, and +3 the most
positive answer. When analyzed the following aspect should
be considered. Scale values above +1 indicate a positive
impression of the users concerning this scale, values below -1
a negative impression. Due to well-known answer effects, like
the avoidance of extremes, observed scales means are in
general in the range of -2 to +2. More extreme values are
rarely observed, so a value near +2 represents a very positive
near optimal impression of participants.
Fig. 4 shows an example for an overall result for a product.
The graphic is automatically generated by the data analysis
sheet (Excel) that can be downloaded together with the
questionnaire.
Fig. 4. Example of an overview result.
Thus, this particular product created a slightly positive
impression concerning Attractiveness and Stimulation, but is
judged neutral concerning the other 4 scales. The error bars
represent the 5% confidence intervals for the scale means, i.e.
the probability that the true value of the scale mean lies outside
this interval is less than 5%. The width of the error bars
depend on the number of respondents and on the level of
agreement between the respondents. Thus, the more the
participants that filled out the questionnaire agree concerning
their evaluation of the product the smaller are typically the
width of the error bars. Thus, if there are many respondents to
the questionnaire and the error bars are still wide, this can be
an indication that there are different sub-groups of participants
with quite opposite options about the product.
Two different products or product versions can thus easily
be compared concerning their user experience by comparing
the scale means. See Fig. 5 for a comparison of two product
versions concerning the observed scale means.
Fig. 5. Example of a comparison of two product versions concerning the UEQ
scales.
In this example version 2 is much better concerning
Attractiveness, Perspicuity Efficiency and Dependability.
Concerning the hedonic scales Stimulation and Novelty both
versions seems to be comparable.
To find out if the difference concerning the scale values is
significant on the 5% level (or any other level you choose) it is
necessary to apply a statistical test that compares the scale
means (for ex. a t-test). It is not sufficient to check if the error
bars do not overlap. If they do not overlap it can be concluded
that the difference is significant at 5% level. But the opposite
is not true. The error bars can overlap and the difference may
still be significant!
The scales can be grouped into three categories.
Atractiveness is a pure valence dimension. The scales
efficiency, perspicuity and dependability describe the
pragmatic quality of the product. The scales stimulation and
novelty describe the hedonic quality of the product.
C. Analyzing the results of the individual items
After the overview the details have to be examined. First if
you have two software versions with the UEQ results the items
results are placed opposite each other. Items with extreme
differences give a hint which areas have been improved or not.
These way product versions can be compared easily and exact
with one another. Also the detail analyzing shows, which areas
should be improved for the next release (See Fig. 6). If it is the
first product release see if some items show extreme results
compared to other in the same UEQ results.
While analyzing each item the target group could give hints
about what caused the significant distinction. Therefore the
basic demographic data has to be collected with the UEQ
results as well.
The UEQ exists in different languages which are tested
reliably. Nevertheless, because of the complexity of language,
it is also possible that translation deviance the results.
International Journal of Artificial Intelligence and Interactive Multimedia, Vol. 2, Nº 1.
-43-
Fig. 6. Example for the detail analyzing of the results from the UEQ-Excel-
Sheet (a specimen of the first three items)
VI. APPLICATION IN THE COMPANY DATEV EG
This part presents an example how the UEQ is applied for
benchmarking in a big business software company. A general
impression of a process is presented in [16].
A. About DATEV eG
The cooperative DATEV eG, Nuremberg (Germany), is a
software company and IT service provider for tax consultants,
auditors and lawyers as well as their clients. Roughly 5800
employees produce more than 220 applications and provide
service for about 39800 cooperative members.
B. Usage of UEQ within a defined Design Process
The concept of user centered design is meanwhile part of
the official DATEV eG software development model and the
UEQ is an integral component among other UCD methods like
classical usability testing, focus groups, persona development
and heuristic evaluation. The questionnaire is used to get user
feedback at different development stages and all UEQ data are
collected in one database.
C. Scenarios of use
One major goal is to perform a regular standardized survey
with our users in consultant companies and enterprises. The
challenge here is the integration into software release plans
and market research activities. The UEQ is currently used
successfully in three scenarios:
- Evaluation of new beta versions by selected beta testers
- Assessment of released software by randomly selected
users
- At the end of a classic usability test to evaluate a new
prototype
In the last scenario it is not the primary goal to get an
accurate assessment, but the outcome will give an orientation
whether the new software design will bring a significant
improvement compared to the DATEV eG benchmark and
previous measurements for the tested application. Of course
one must be cautious, the tasks in a laboratory test do not
represent the entire application and the demonstrated
improvements in some parts will perhaps have no effect on the
overall user experience of the complete application.
A current project is the test of the combination of online
questionnaire and focus group. The outcome of the online-
UEQ should be the base for questions in asynchronous online
focus groups. Another example how to use the UEQ is
described in an article concerning user experience for business
software [16].
Because of the special form of the UEQ it is important that
participants fill out the questionnaire in their natural language.
Thus, it is for companies that use the UEQ on multi-national
level important to have language versions of the questionnaire
available.
VII. CREATION OF A SPANISH LANGUAGE VERSION
First, the German version of the UEQ was translated into
Spanish by a native speaker and a bilingual person. After that
the Spanish version had been retranslated into German. If the
words turned out to match the original words the translation
was declared to be successful. Otherwise the process was
repeated until all words matched. To demand a one-to-one
translation from one language into another is not entirely
possible. The reason for that are the different meanings of one
word, which make it difficult to find synonym in any language.
The translator was open minded and didn’t know the
questionnaire before. For more information see [17].
Fig. 7. Spanish version of the UEQ
Special Issue on Artificial Intelligence and Social Application
-44-
VIII. VALIDATION STUDIES FOR THE SPANISH VERSION
The Spanish Language version of the UEQ is already
validated in two bigger studies.
In the first study 94 students evaluated the user experience
of the Amazon web-shop (www.amazon.es). The scale means
and confidence intervals are shown in Fig. 8.
Thus, overall the participants had a slightly positive or
neutral impression concerning the user experience of the
Amazon web-shop. The impression concerning the pragmatic
quality (Perspicuity, Efficiency and Dependability) is clearly
higher than the impression concerning the hedonic quality
(Stimulation, Novelty).
An analysis of the Cronbach Alpha coefficient showed that
the single scales showed high consistency values
(Attractiveness: 0.85, Perspicuity: 0.59, Efficiency: 0.74,
Dependability: 0.48, Stimulation: 0.75, Novelty: 0.64). This is
an indicator that the scales are sufficiently consistent.
In a second study 95 students evaluated the user experience
of Skype. Again scale means and confidence intervals are
shown in Fig. 9.
Fig. 9. Result for Skype.
The impression concerning the Skype user experience is
quite positive. Again pragmatic quality is judged better than
hedonic quality aspects. If we compare these evaluations to the
results for the Amazon web-shop we clearly see that Skype
creates a better user experience.
As in the first study alpha coefficient for the scales shows
high values (Attractiveness: 0.83, Perspicuity: 0.71,
Efficiency: 0.72, Dependability: 0.55, Stimulation: 0.78,
Novelty: 0.71) again indicating sufficient scale consistency.
Of course further studies are necessary to finally judge if the
psychometric properties of the Spanish version are identical to
the existing and well-evaluated German and English version.
But these first results are positive.
IX. AVAILABILITY
The UEQ questionnaire can be used free of charge. The
questionnaire itself, a data analysis tool and literature
describing the construction of the questionnaire can be
downloaded from www.ueq-online.org. The questionnaire and
the analysis tool are available in several languages. Currently
German, English, French, Italian and the Spanish version are
available. It is worked on a Portuguese Version as well.
X. SUMMARY
We described the construction, the result analyzing and the
validation studies of the Spanish language version of the User
Experience Questionnaire. This questionnaire allows a fast
evaluation of the user experience of interactive products. It
measures not only usability aspects like efficiency, perspicuity
and dependability, but also user experience aspects like
stimulation or originality.
Since the UEQ has the form of a semantic differential, it is
quite important that participants can rate a product in their
natural language. Thus, the new language version allows the
application of the UEQ in Spanish speaking target groups.
The first available validation studies suggest that the scale
quality of the Spanish version is sufficient to apply the
questionnaire in projects to collect feedback about user
impressions.
REFERENCES
[1] DIN EN 9241-210, 2011-01, Ergonomics of human-system interaction
Part 210: Human-centred design for interactive systems, Beuth,
Berlin, 2011.
[2] Hassenzahl, M., “The effect of perceived hedonic quality on product
appealingness”, Internatinal Journal of Human-Computer Interaction,
13, pp. 481-499, 2001.
[3] Preece, J., Rogers, Y., Sharp, H., “Interaction design: Beyond human-
computer interaction”. Wiley, New York , 2002.
[4] Kurosu, M., Kashimura, K., “Apparent usability vs. inherent usability:
experimental analysis of the determinants of the apparent usability”. In:
Conference Companion of human factors in computing systems, pp.
292--293. Denver, Colorado, 1995.
[5] Tractinsky, N., “Aesthetics and Apparent Usability: Empirical
Assessing Cultural and Methodological Issues”. CHI’97 Electronic
Publications http://www.acm.org/sigchi/chi97/proceedings/paper/nt.htm
1997.
[6] Hassenzahl, M., Monk, A., The influence of perceived usability from
beauty. Human-Computer Interaction, 25(3), pp. 235-260, 2010.
[7] Tractinsky, N., Katz, A.S., Ikar, D., “What is beautiful is usable”,
Interacting with Computers, 13, pp.127--145, 2000.
[8] Ilmberger, W.; Schrepp, M. , Held, T., “What kind of cognitive process
causes the relationship between aesthetics and usability“. In: Holzinger,
A. (Ed.): USAB 2008, LNCS 5298, pp. 43-54, 2008.
8. Result for Amazon web-shop.
International Journal of Artificial Intelligence and Interactive Multimedia, Vol. 2, Nº 1.
-45-
[9] Tuch, A.N., Roth, S.P., Hornbaek, K., Opwis, K. & Bargas-Avila, J., Is
beautiful really usable? Toward understanding the relation between
usability, aesthetics, and affect in HCI. Computers in Human Behavior
28 (5), pp. 5961607, 2012.
[10] Hartmann, J., Sutcliffe, A., DeAngeli, A., Towards a theory of user
judgement of aesthetics and user i nterface quality. ACM Transactions
on Computer-Human Interaction, 15, 2008.
[11] Norman, D., Emotional design. Why we love (or hate) everyday
things. New York: Basic books, 2004.
[12] Laugwitz, B.; Held, T., Schrepp, M. Construction and evaluation of a
user experience questionnaire. In: Holzinger, A. (Ed.): USAB 2008,
LNCS 5298, pp. 63-76, 2008.
[13] Laugwitz, B.; Schrepp, M., Held, T. Konstruktion eines Fragebogens
zur Messung der User Experience von Softwareprodukten
[Construction of a questionnaire to measure the user experience of
software products] . In: A.M. Heinecke, H. Paul (Eds.): Mensch &
Computer 2006 Mensch und Computer im Strukturwandel.
Oldenbourg Verlag, pp. 125 134, 2006.
[14] Laugwitz, B.; Schubert, U.; Ilmberger, W.; Tamm, N.; Held, T.,
Schrepp, M., Subjektive Benutzerzufriedenheit quantitativ erfassen:
Erfahrungen mit dem User Experience Questionnaire UEQ
[Quantitative measurement of subjective user satisfaction: Experiences
with the user experience questionnaire UEQ]. In: Brau, H.;
Diefenbach, S.; Hassenzahl, M.; Kohler, K.; Koller, F.; Peissner, M.;
Petrovic, K.; Thielsch, M.;Ulrich, D., Zimmermann, D. (Eds.); Usability
Professionals 2009, pp. 220 225, 2009.
[15] Winter, D., Pietschmann, J., UX in den frühen Phasen des
Innovationsprozesses. User Experience von Anfang an bedenken[UX in
the early phases of the innovation process. Consider User Experience
from scratch.]. In: Brau, H.; Lehmann, A.; Petrovic, K.; Schroeder, M.
(Eds.); Usability Professionals 2012 , pp. 274-279, 2012.
[16] Rauschenberger, M., Hinderks, A.; Thomaschewski, J.,
“Benutzererlebnis bei Unternehmenssoftware: Ein Praxisbericht über die
Umsetzung attraktiver Unternehmenssoftware [User experencing of
Busnisssoftware: A practical report how attractive Busnisssoftware
could be developt.]. In: Brau, H; Lehmann, A.; Petrovic, K.; Schroeder,
M.C. (Eds.); Usability Professionals 2011, pp. 158 - 163, 2011.
[17] Rauschenberger, M., Schrepp, M.; Olschner, S.; Thomaschewski, J.; M.
P. Cota, Measurement of user experience: A Spanish Language Version
of the User Experience Questionaire (UEQ) . In: Information Systems
and Technologies (CISTI), Vol. 1, pp. 471-476, 2012
Maria Rauschenberger has a Bachelor of Engineering in
Media technology and is studying the Master of Science in
Media Informatics from the University of Applied Science
Emden/Leer. Furthermore is she working as a Project
manager for MSP Medien-Systempartner, Germany.
Currently, she pursues the finals of Master degree and
taking part of the regional user experience group Bremen
(UPA). She is involved in research activities dealing with
interaction design, process optimizing, usability and user experience since
2011.
Dr. Martin Schrepp works since 1994 as user interface
designer for SAP AG. He finished in 1990 his studies in
Mathematics with a Diploma from the University of
Heidelberg (Germany). In addition, he received in 1993
a PhD in Psychology (also from the University of
Heidelberg). His research interests are the application of
psychological theories to improve the design of software
interfaces, the application of Design for All principles to
increase accessibility of business software, measurement of usability and user
experience and the development of general methods for data analysis. He
published several papers in these research fields.
Prof. Dr. Manuel Pérez Cota is Professor and
Researcher at the state-owned university Universidade de
Vigo (UVIGO) in Vigo, Spain. He is graduated with
honors in Electrical Engineering (Universidad La Salle)
and Electronics and Communications Engineering
(Universidad Nacional Autónoma de México UNAM,
1980), a PhD with honors in Industrial Engineering
(Universidad de Santiago de Compostela, 1990). He is
the director of the international research group SI1-GEAC
(http://cuautla.uvigo.es/si1-geac/). He was the first director and developer of
the Informatics (Computer Science School of the University of Vigo) and he
was director, also, of the Informatics Department. He widely collaborates in
different Master and PhD programs in Spain, Portugal, Germany, Argentina
and Bolivia, and he has been supervisor in several PhDs, with some others in
progress. He is part in different European and International projects. He has
published quite extensively and has a lot of publications (including books,
book chapters, Scientific Citation Index journal articles, and international
journal articles, as well as publications in refereed conference proceedings).
He is member of different international committees and associations (ACM,
IEEE, AISTI, AIPO, ANALCT).
Prof. Dr. Jörg Thomaschewski. He became full
professor at the University of Applied Sciences
Emden/Leer, Germany from September 2000. He
received the Doctor degree in Physics from the
University of Oldenburg, Germany, in 1996. His
research interests are in the fields "Internet
Applications" focusing on human computer interaction,
e-learning and software engineering. He is author of
various online modules, e.g. "Human Computer Communication", which is
used in the Virtual University (Online) at six university sites. He has wide
experience in usability training, analysis and consulting.
Dr. Siegfried Olschner is a senior member of the User
Experience and Design team of DATEV eG, Nuremberg,
Germany. His principal activities are dialogue design,
workflow design and general user experience consulting
for colleagues from the development departments. He
also deals with the research and development of
standardized test methods that are used as parts within
the DATEV User Centered Design Process. One
example is the improvement of the UEQ.
... and often stay in touch with local language version scientists beyond that. The Spanish version of the UEQ has been carefully created and evaluated (see [10], [11]). Particularly in Latin America, regional variations of the language have developed in each country. ...
... In a next step, the translation was checked with two different studies [11]. The web shop amazon.de ...
... and the communication software Skype were used, each with 94 participants. The two studies were conducted in Spain (Vigo) and found to have good internal consistency, determined with the Cronbach's Alpha [11]. ...
Article
Full-text available
This paper analyses changes in some items of the User Experience Questionnaire (UEQ) for use in the context of Costa Rican culture. Although a Spanish version of the UEQ was created in 2012, we use a double-translation and reconciliation model for detecting the more appropriate words for Costa Rican culture. These resulted in 7 new items that were added to the original Spanish version. In total, the resulting UEQ had 33 items. 161 participants took part in a study that examined both the original items and the new ones. Static analyses (Cronbach's Alpha, mean, variance, and confidence interval) were performed to measure the differences of the scales of the original items and the new UEQ variant with the Costa Rican words. Finally, confidence intervals of the individual items and Cronbach’s Alpha coefficient average of the affected scales were analysed. The results show, contrary to initial expectations, that the Costa Rican word version is neither better nor worse than the original Spanish version. However, this shows that the UEQ is very robust to some changes in the items.
... The ISO 9241:11-2018 standard considers not only the functional attributes of the system, but it encompasses how the user feels towards the system. These metrics are commonly referred to as the Hedonic Characteristics in User Experience 77 (RAUSCHENBERGER et al., 2013). In this new framework, how the user experiences the system is a relevant metric -as relevant as how efficient the system performs the desired operations. ...
... The Attractiveness scale, due to its wide interpretation is regarded as a higher level characteristic, with six dimensions; the other scales each have four dimensions. (SCHREPP, 2019;RAUSCHENBERGER et al., 2013) A schematic representation of these elements is shown in Figure 13. ...
Thesis
Full-text available
Professional training is a vital process for companies, as it allows for the the specialization of workforce, enhancing the company’s competitive stance and allowing to provide better service to the clients. In the context of high risk activities, such as live line maintenance, training assumes a pivotal stance as failures may result in the occurrence of infrastructure problems, harm to workers and in extreme cases, even loss of lives. To contribute to the training process, the RV2 research group was created aiming to develop a virtual reality system for substation live line training. Besides the pedagogical training, one of the project requirements was to create an environment in which the trainee would voluntarily engage in a recurrent manner. To allow for such interactions, the serious game theory was incorporated in the development of the virtual system, to enhance the motivation and satisfaction of the learners while interacting with the system. The project also defined that the pedagogical context should be observed during development, which prompted the research group to, out of numerous possibilities, choose Gagné’s Instructional Theory as the pedagogical framework for the system. However, these two theoretical frameworks still needed to be connected in order to create a unified base for the deployment and continued development of the virtual learning system. Such technique would have to allow to incorporate the needed pedagogical premises in the game design theory and allow for any future additions to fall in line with the preexisting framework. To satisfy this need, research was conducted to verify how the elements could be connected in order to describe the whole system. Initially, Model, Dynamics, Aesthetics game design framework was chosen to guide the analysis of the serious game aspect of the system. This framework was subsequently aligned in a conceptual manner to the Instructional Design process and the Instructional Events described by Gagné. The process initially had to be modelled, which evidenced the need to create an interview capable of harvesting specialist knowledge. The trainees’ experience also had to be asserted to be possible to analyze how the training environment fared in the opinion of the intended audience. To allow this data gathering, the discourse analysis theory was employed as a means to show how the trainees felt during the training exercises. The developed work allows for the conclusion that Gagné’s Instructinal Method can be aligned with the Model Dynamics Aesthetics framework to produce a comprehensive serious game framework, in which pedagogical and game design theories are both observed. Meanwhile, the player experience can be asserted by semi-structured interviews and the data gathered can be used in the continued development of the virtual learning system.
... Artinya UEQ relevan untuk dibuat dalam Bahasa selain Bahasa inggris. (Rauschenberger et al., 2013). Menurut Firstama dkk, UEQ sering juga sering digunakan sebagai alas an atau landasan bagi peneliti untuk melakukan perbaikan pada aplikasi. ...
... User experience diukur dengan menggunakan User Experience Questionnaire (UEQ) yang dikembangkan oleh Laugwitz et al. (2008). UEQ tidak hanya merupakan potret dari penggunaan produk tetapi juga impresi pengguna terhadap produk sehingga dihasilkan pengukuran pragmatis dan impresi (Rauschenberger et al., 2013). Tujuan dari user experience adalah meningkatkan tingkat kepuasan pengguna dengan mengembangkan usability, accessibility, dan kesenangan dari interaksi antara pengguna dan produk digital (Stewart, 2015). ...
... This study uses a User Experience Questionnaire (UEQ) (Schrepp, Hinderks, et al., 2017b), where the questionnaire can provide a simpler and faster way to express user feelings, impressions, and attitudes when using a product (Laugwitz et al., 2008). UEQ has been applied in studies such as the evaluation of enterprise software, websites and web services, and social networks (Rauschenberger et al., 2013). The main purpose of using UEQ is to allow a rapid assessment by the user which includes a comprehensive impression of the preferred user experience . ...
Article
Full-text available
The Semarang City Manpower Service or also known as (DISNAKER) is a government agency that can foster and control the manpower sector. The Manpower Office has its website application, namely the work system (SIKER) which is an integrated employee recruitment information system from the Semarang city government, companies, and also the community or job seekers. This study aims to understand the level of comfort in terms of User Experience in using the SIKER application. Analysis of the SIKER website application by applying the User UEQ method approach given to 32 respondents who have used this website. The UEQ assessment focuses on six aspects, namely: efficiency, attractiveness, accuracy, clarity, novelty, and stimulation. The results obtained after conducting the analysis are that the average respondent gives below average results, at the User Experience level the efficiency item gets a sufficient value, on the clarity, accuracy, stimulation, attractiveness, and novelty scales it gets a value below the average. This tends to be the lack of seeker users with this SIKER application. In other words, the reason is that there are still many users who don't understand this application because this application is still relatively new.
... Researchers have utilized customer's experience to find the information system's success. Rauschenberger et al. (2018) formed a customer's experience questionnaire to evaluate the customer's experience for any service quickly. The questionnaire scales are made to include a detailed impression of the customer's experience. ...
Article
This systematic research study examined the connection between customer experience dimensions and e- government success in Abu Dhabi in United Arab Emirates (UAE). The organized study utilized a descriptive and cross-sectional survey research design with a positivism philosophy. A sample of 382 participants out of 108,825 target population was chosen employing a table developed by Morgan & Krejcie (1970). Facts were gathered utilizing a questionnaire, simple random sampling and examined applying Descriptive Statistics and Structural Equation Modeling using the Smart Partial Least Squares (SPLS) Version 3.3.3 software to test the hypotheses. The inquiry study findings came up with a significant correlation between customer experience dimensions (comfort, security and usability) and e- government success at significant level of 0.05. It was thus concluded that the customer experience dimensions significantly affect e- government success. The logical investigation study recommended that the UAE government should augment the budget for e-government successful implementation programs.
Article
Full-text available
Agile methods are used more and more frequently to develop products by reducing development time. Requirements are typically written in user stories or epics. In this paper, a new method called UX Poker is presented. This is a method to estimate the impact of a user story on user experience before development. Thus, there is the opportunity that the product backlog can also be sorted according to the expected UX. To evaluate UX Poker, a case study was conducted with four agile teams. Besides, a workshop followed by a questionnaire was conducted with all four agile teams. The goal of being able to estimate the UX even before development was achieved. Using UX Poker to create another way to sort the product backlog can be considered achieved in this first evaluation. The results show that UX Poker can be implemented in a real- life application. Additionally, during the use of UX Poker, it was found that a shared understanding of UX began. The participants clarified in the team discussion about UX Poker what related to influence the user stories had on UX and what UX meant for their product.
Article
The User Experience Questionnaire (UEQ) is a standard, widely used questionnaire for evaluating user experience with many (software) products and services. This article describes UEQ translation into Croatian, which involved English and German language experts, and its verification based on an online survey conducted among 231 university students about perceived experience with different video conferencing tools in online learning during the COVID-19 lockdown. The sample was large enough to confirm the construct validity of the questionnaire. Psychometric evaluation and validation relied on factor and reliability analyses of all six scales of the full, 26-item UEQ and of the pragmatic and hedonic scales in the short, 8-item UEQ version. All scales show a rather high internal reliability, which confirms the validity of the Croatian translation. Croatian-speaking user experience researchers and practitioners can use it with confidence.
Article
Full-text available
This paper aims to adapt and validate the User Experience Questionnaire Plus (UEQ+) in the Indonesian context. The UEQ+ is a modular extension of the User Experience Questionnaire (UEQ), which has been adapted to the Indonesian context and used in many studies. The UEQ+ was originally developed in German and English. As a modular extension, the UEQ+ has more user experience (UX) scales compared to the UEQ and can be used to evaluate products in special scenarios. Several steps were carried out to adapt and validate the UEQ+: translating the questionnaire into Bahasa by involving UX practitioners, evaluating the translation results by involving a UX expert and practitioners, and conducting face validity and reliability testing through two case studies (Zoom and Learn Quran Tajwid as online learning tools). The results showed that the findings from the open-ended questionnaire were consistent with the results of the six scales. Future work is needed to investigate whether the UEQ+ can capture some of the UX-related themes identified from the two case studies.
Article
Full-text available
For the purpose of comparing the concept of the apparent usability with the concept of the intended inherent usability and the experienced inherent usability, two psychological experiments were conducted to see how much the apparent usability is related to the inherent usability. In both experiments, two kinds of usability concepts showed low correlations with each other. The result indicated that the apparent usability is a different concept from the inherent usability and is rather related to the visual impression of the interface. This result also means that the inherent usability is difficult to be understood by just looking at the interface and suggests that the interface designers have to make efforts to make the interface 'look' usable as well as to make it 'actually' usable.
Conference Paper
Full-text available
Mit dem User Experience Questionnaire wurde ein Fragebogen entwickelt, der eine schnelle Messung verschiedener Kriterien der Softwarequalität erlaubt. Die Relevanz der Kriterien für die Beurteilung wurde durch eine empirische Selektion sichergestellt. Experten sammelten und reduzierten eine große Menge potenziell relevanter Begriffe und Aussagen, die sowohl „harte“ Usability-Kriterien als auch „weichere“ User Experience-Kriterien einschlossen. Der daraus entstandene ursprüngliche Fragebogen mit bipolaren 80 Items wurde in mehreren Untersuchungen eingesetzt und durch eine Faktorenanalyse auf 26 Items reduziert, die sich den sechs Faktoren Attaktivität, Durchschaubarkeit, Effizienz, Vorhersagbarkeit, Stimulation und Originalität zuordnen lassen. Erste Validierungsuntersuchungen deuten auf eine zufriedenstellende Konstruktvalidität hin.
Conference Paper
Full-text available
Das Gebrauchstauglichkeitskriterium der Benutzerzufriedenstellung kann auf verschiedene Weise quantifiziert werden. Eine Möglichkeit sind Fragebögen, die auf eine spontane und möglichst unreflektierte Beurteilung eines Produkts abzielen können. Ein Beispiel für einen solchen Fragebogen ist der User Experience Questionnaire UEQ (Laugwitz et al. 2006). Berichtet wird über die Verwendung des Fragebogens in der Qualitätssicherung bei der Entwicklung von Business-Software bei DATEV. Zum anderen werden zwei wissenschaftliche Studien vorgestellt, in denen der UEQ ebenfalls eingesetzt wurde. Aus den dargestellten Erfahrungen werden zusammenfassend Vorzüge, Einschränkungen und Empfehlungen für den erfolgreichen Einsatz von Fragebögen in Usability-Praxis und -Forschung abgeleitet.
Conference Paper
Full-text available
To improve a product you will need most likely developers, managers and user feedback. Besides the basic software qualities other important properties are usability and user experience for developing a good product. Usability is well known and can be tested with e.g. a usability test or an expert review. In contrast user experience describes the whole impact a product has on the end-user. The timeline goes from before, while and after the use of a product. We present a tool that allows to evaluate the user experience of a product with little effort. We show in addition how this tool can be used for a continuous user experience assessment.
Article
Full-text available
A review of 15 papers reporting 25 independent correlations of perceived beauty with perceived usability showed a remarkably high variability in the reported coefficients. This may be due to methodological inconsistencies. For example, products are often not selected systematically, and statistical tests are rarely performed to test the generality of findings across products. In addition, studies often restrict themselves to simply reporting correlations without further specification of underlying judgmental processes. The present study’s main objective is to re-examine the relation between beauty and usability, that is, the implication that “what is beautiful is usable.” To rectify previous methodological shortcomings, both products and participants were sampled in the same way and the data aggregated both by averaging over participants to assess the covariance across ratings of products and by averaging over products to assess the covariance across participants. In addition, we adopted an inference perspective to qualify underlying processes to examine the possibility that, under the circumstances pertaining in most studies of this kind where participants have limited experience of using a website or product, the relationship between beauty and usability is mediated by goodness. A mediator analysis of the relationship between beauty, the overall evaluation (i.e., “goodness”) and pragmatic quality (as operationalization of usability) suggests that the relationship between beauty and usability has been overplayed as the correlation between pragmatic quality and beauty is wholly mediated by goodness. This pattern of relationships was consistent across four different data sets and different ways of data aggregation. Finally, suggestions are made regarding methodologies that could be used in future studies that build on these results.
Article
Full-text available
Usability can be broadly defined as quality of use. However, even this broad definition neglects the contribution of perceived fun and enjoyment to user satisfaction and preferences. Therefore, we recently suggested a model taking "hedonic quality" (HQ; i.e., non-task-oriented quality aspects such as innovativeness, originality, etc.) and the sub-jective nature of "appealingness" into account (Hassenzahl, Platz, Burmester, & Leh-ner, 2000). In this study, I aimed to further elaborate and test this model. I assessed the user perceptions and evaluations of 3 different visual display units (screen types). The results replicate and qualify the key findings of Hassenzahl, Platz, et al. (2000) and lend further support to the model's notion of hedonic quality and its importance for subjective judgments of product appealingness.
Conference Paper
Full-text available
We conducted an experiment to find out which cognitive processes underlie the high correlations between aesthetic impression and perceived usability of a user interface reported in several studies. Four online shops differing in colour scheme and usability level were created and tested before and after an interaction period to gain support for two differing models of explanation. One stems from Norman who hypothesizes that mood is a mediator between design and perceived usability whereas the other is based on the well known “What is beautiful is good” phenomenon from social psychology and market research. The data provide support for the approach of Norman whereas the high correlations couldn’t be replicated. Another interesting finding was that our results militate rather for a “What is usable is beautiful”- relation than the other way round as hypothesized.
Article
This paper analyzes the relation between usability and aesthetics. In a laboratory study, 80 participants used one of four different versions of the same online shop, differing in interface-aesthetics (low vs. high) and interface-usability (low vs. high). Participants had to find specific items and rate the shop before and after usage on perceived aesthetics and perceived usability, which were assessed using four validated instruments. Results show that aesthetics does not affect perceived usability. In contrast, usability has an effect on post-use perceived aesthetics. Our findings show that the “what is beautiful is usable” notion, which assumes that aesthetics enhances the perception of usability can be reversed under certain conditions (here: strong usability manipulation combined with a medium to large aesthetics manipulation). Furthermore, our results indicate that the user’s affective experience with the usability of the shop might serve as a mediator variable within the aesthetics–usability relation: The frustration of poor usability lowers ratings on perceived aesthetics. The significance of the results is discussed in context of the existing research on the relation between aesthetics and usability.