Conference PaperPDF Available

Mixed Feelings? The Relationship between Perceived Usability and User Experience in the Wild

Authors:

Abstract and Figures

Although both user experience and perceived usability have been extensively studied, the relationship between the two is less well understood. Prior empirical research suggests that perceived usability influences especially negative user experiences, but the effect depend on goals, contexts, and expectations. The paper contributes on this theme with description of a field study covering self-reporting of 12 subjects using a new smartphone. The findings confirm some earlier views on the relationship but also permit a richer understanding. Unlike prior work, the results show that perceived usability can play an important role in ambivalent experiential episodes. These episodes emerge from a clash between desired uses and either poor perceived usability or lack of appropriateness in the broader social context. We discuss our findings in relation to prior studies.
Content may be subject to copyright.
Mixed Feelings? The Relationship between Perceived
Usability and User Experience in the Wild
Eeva Raita
University of Helsinki and
Helsinki Institute for Information Technology HIIT
Antti Oulasvirta
Aalto University
ABSTRACT
Although both user experience and perceived usability have
been extensively studied, the relationship between the two
is less well understood. Prior empirical research suggests
that perceived usability influences especially negative user
experiences, but the effect depend on goals, contexts, and
expectations. The paper contributes on this theme with de-
scription of a field study covering self-reporting of 12 sub-
jects using a new smartphone. The findings confirm some
earlier views on the relationship but also permit a richer un-
derstanding. Unlike prior work, the results show that per-
ceived usability can play an important role in ambivalent
experiential episodes. These episodes emerge from a clash
between desired uses and either poor perceived usability or
lack of appropriateness in the broader social context. We dis-
cuss our findings in relation to prior studies.
Author Keywords
Usability; User experience; Field studies; Qualitative stud-
ies; Ambivalent experiences; Diary method
ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g., HCI):
Miscellaneous.
INTRODUCTION
Perceived usability is studied to understand users’ percep-
tions of task-related efficiency, effectiveness, and satisfac-
tion. However, as computers see increasing use for leisure,
not only work, metrics focusing on task-achievement have
been criticized as insufficient for capturing what users en-
joy. User experience has been posited as an alternative and
a more holistic perspective that goes “beyond the instru-
mental” (e.g., [11, 17, 26, 28]). Yet it remains ambiguous
how instrumental and non-instrumental qualities are related
to each other, or how task-related elements influence the
formation of pleasurable experiences.
Currently, the relationship between user experience and
perceived usability is relatively unexplored. For example,
only 23% of empirical user-experience studies examined in
a recent review included some analysis of the relation be-
tween user experiences and perceived usability [1]. The
ambiguity over the relationship between the two is evident
even in the research methodology itself: some maintain that
usability criteria can be utilized to measure user experience
[35], while others have stressed the need for new research
methodologies [8] or developed measurements for captur-
ing perceived usability as a part of user experience [7].
Perceived usability and user experience have been studied
in relation to the initial finding that non-instrumental quali-
ties such as aesthetics can influence perceptions of instru-
mental qualities [34]. Some related papers have assessed
the effect of both kinds of qualities on user experience, as
measured as an emotional experience (e.g., [11, 25, 33]) or,
in other cases, an overall evaluation of goodness or prefer-
ence (e.g., [4, 10, 17, 36]). These studies have found that
perceived usability influences emotional reactions: good
perceived usability results in positive reactions, and poor
perceived usability in negative ones [24, 25, 33]. However,
it has also been found that it is not a source of outstanding
positive emotions [12]. According to prior studies, perceived
usability influences preference and overall appraisals but
only in combination with other factors [10, 17] or depend-
ing on user expectations and the context of use [4, 9, 13].
We contribute to this discussion by pointing out how in-
strumental qualities are present in experiential episodes and
influence the emergence of pleasurable user experiences.
Our data come from a mixed-methods field study covering
12 participants’ detailed self-reporting of the first weeks of
using a new smartphone. Prior studies have focused on user
experiences as emotional reactions and overall evaluations
and measured them mostly with predetermined rating scales.
This paper contributes by studying how user experiences
are conveyed in spontaneous reports in the wild.
The novel finding is that perceived usability is related to
ambivalent experiential episodes. These are episodes
wherein both positive and negative user experiences are
present. For example, in the study certain apps were found
desirable but not as usable as they could have been. In addi-
tion, certain use behaviors (mostly “killing time”) were
found to be enjoyable but also pointless or inappropriate.
We conclude the paper by discussing the significance of
ambivalent experiential episodes and their contribution to
the study of perceived usability and user experience.
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. Copyrights for com-
ponents of this work owned by others than the author(s) must be honored.
Abstracting with credit is permitted. To copy otherwise, or republish, to
post on servers or to redistribute to lists, requires prior specific permission
and/or a fee. Request permissions from Permissions@acm.org.
NordiCHI '14, October 26 - 30 2014, Helsinki, Finland
Copyright is held by the owner/author(s). Publication rights licensed to
ACM. ACM 978-1-4503-2542-4/14/10…$15.00.
http://dx.doi.org/10.1145/2639189.2639207
Definitions Used In This Paper
Before reviewing related work, we discuss the two con-
structs used in this study: user experience and perceived us-
ability. In general, user experience is a broad and ambigu-
ous concept [22]. Individual researchers have focused on
different aspects and conceptualizations of it. Despite per-
ceived usability being considered a more straightforward
concept; it too has been approached in multiple ways (for a
review of current usability practices, see [15]). For the pur-
poses of this paper, we differentiate between perceived usa-
bility and “objective” usability, where the latter involves
those performance-related aspects of the interaction that are
independent of perception [15]. In practice, usability is of-
ten understood as ease of use and learning (e.g., [27]), but
other researchers’ definitions include also the usefulness of
the system and the satisfaction of the user [2].
In this paper, perceived usability is understood in reference to
the ISO-9241-210 definition: perceptions of the extent to
which a product can be used by specified users to achieve
specified goals with effectiveness, efficiency, and satisfac-
tion in a specified context of use [16]. This broad under-
standing of usability includes the usability of both the prod-
uct and functions in enhancing of task-achievement.
There are multiple conceptualizations of user experience,
and some researchers emphasize that it should be viewed
pluralistically [8]. User experience has been defined: as a
constant stream of self-talk (e.g., [8]), as an emotional re-
sponse or experience (e.g., [23, 24]), as an experiential epi-
sode with a beginning and end (e.g., [8, 13, 18]), as co-
experience (jointly created) (e.g., [8]), and as the overall
evaluation based on momentary experiences (e.g., [13]). In
addition, empirical user-experience studies have assessed
user experience in terms of generic user experience, emo-
tion, enjoyment, aesthetics, appeal, hedonic quality, en-
gagement, flow, and a few other constructs (reviewed in
[1]). Some studies have focused on only one aspect of user
experience, while others utilize multiple perspectives simul-
taneouslyfor example by studying both emotional experi-
ences and the overall user experience (e.g., [21, 25]).
In this paper, user experience too is defined in reference to
the ISO-9241-210 standard: as a person's perceptions and
responses that result from the use or anticipated use of a
product, system, or service [16]. We focus on self-reports of
user experience, which capture reflexive and articulated ex-
periences as a part of use episodes (see also [18]).
RELATED WORK
In the discussion that follows, we review related works in
terms of their focus on user experience, either 1) as emo-
tions or 2) as overall preference or evaluation of goodness.
In the last subsection we summarize prior findings particu-
larly from longitudinal field studies.
Perceived Usability and Emotional Experiences
Studies assessing the effect of perceived usability on emo-
tional experiences often draw their inspiration from the ini-
tial finding that a product’s aesthetic appeal influences its
perceived usability [34]. Consequently, they often compare
the effect of instrumental (i.e., usability and usefulness) and
non-instrumental (i.e., aesthetics, or hedonic) qualities.
These studies suggest that perceived usability influences
emotions: good perceived usability results in positive, and
poor perceived usability in negative, reactions [24, 25, 33].
However, it has been found that instrumental qualities are
not a source of outstanding positive emotional experiences
[12].
In experimental studies, it has been shown that both instru-
mental and non-instrumental quality perceptions (influ-
enced by manipulated system properties) influence emo-
tional experiences as measured via subjective feelings, faci-
al expressions, and physiological responses [24, 25, 33].
These studies have not found an interactive effect between
usability and aesthetics; the two factors influence emotions
additively. In addition, perceived usability has been found
to have a somewhat greater effect than perceived aesthetics
[25, 33]—especially during goal-driven use [24].
In contrast, in a questionnaire study by Hassenzahl et al.
[12], perceived usability was not found to be a cause of pos-
itive emotions. In the study, participants were asked to re-
port a recent outstanding positive experience with technolo-
gy in terms of need-fulfillment, experienced affect, attribu-
tion, and product perception and evaluation. Hedonic quality
(i.e., stimulation and identification) was found to capture the
product’s perceived ability to create positive affect through
need-fulfillment. Pragmatic quality (good perceived usabil-
ity) could eliminate barriers to need-fulfillment but was not
a source of positive affect per se.
Perceived Usability and Overall Appraisals
Studies approaching user experience in terms of overall
evaluation or preference have found that perceived usability
influences overall evaluation but only together with other
factors [10, 17] or depending on the context of use and us-
ers’ expectations [4, 9].
One of the first studies to explore this relationship was that
of Jordan on pleasure in technology use [17]. According to
that interview-based study, perceived usabilitybut also
aesthetics, performance, and reliabilityinfluence how
pleasurable a product is. Later, Hassenzahl [10] studied the
interplay between perceived usability and hedonic quality
in forming overall judgments pertaining to beauty and
goodness. He found that judgments of beauty are more in-
fluenced by the user’s perception of the hedonic qualities,
while judgments of overall goodness are affected by both
hedonic quality and perceived usability.
In a study of preferences between two Web sites with the
same content but different interface style, it was found that
the importance of perceived usability depends on preferred
interface style and the context of use [4]. While both per-
ceived usability and expressive aesthetics were good pre-
dictors of overall preference, this happened mainly because
participants appeared to discount negative attributes of their
favorite interface style. In addition, the importance of per-
ceived usability, or expressive aesthetics, changed with the
context of use. In a parallel to this, a paper [9] covering
three experimental studies found that the importance of us-
ability and aesthetics to the overall preference depends on
the context of use, the tasks, and users’ expectations. Good
perceived usability is favored especially in a “serious” use
context, while perceived aesthetics have a greater impact in
more playful contexts. Likewise, a study comparing the ef-
fect of the presence or absence of instrumental goals found
that users’ evaluations focused on usability issues especial-
ly in the presence of instrumental, task-related goals [13].
Longitudinal Studies
The longitudinal field studies on the theme have studied the
formation of user experiences [19] and the effect of emotions
and memories on overall evaluations [21]. According to
Karapanos et al. [19], the overall “goodness” of a product is
associated mostly with perceived usability but also with use-
fulness, stimulation, and identification. While early experi-
ences are related to hedonic aspects of product use, pro-
longed experience is tied more to how the product becomes
meaningful in one’s life. Kujala and Miron-Shatz [21] found
that negative emotions are related to perceived usability, and
positive emotions to good overall user experience. In addi-
tion, they found that in the early stages users focused more
on their user experiences, and the importance of usability in-
creased over time.
THE FIELD STUDY
Prior empirical research suggests that perceived usability
influences especially negative user experiences [10, 21], but
the effect depends on goals [13], contexts, and expectations
[4, 9]. Most of these findings have been obtained in the la-
boratory or with rating scales. In contrast, we studied spon-
taneous user reports capturing reflexive and articulated user
experiences in the wild. Some longitudinal studies [19, 21]
have utilized a setting similar to ours, but their focus has
been on the longitudinal development of user experiences. In
contrast, we are interested in understanding how perceived
usability and user experiences are connected to each other
in self-reported experiential episodes.
The study covered the first two weeks of participants’ owner-
ship of a new smartphone. During the study period, user ex-
periences were measured with diaries utilizing principles
from the Day Reconstruction Method (DRM) [18], referred
to as self-reports in the rest of the paper. Perceived usability
was studied through identification of perceptions of usability
from the self-reports and also via measurement of a global,
summary usability score with the System Usability Scale
(SUS) [3]. The study began with a kick-off meeting at
which participants received the new device, answered de-
mographic questions, and completed a questionnaire ad-
dressing their prior uses of mobile devices and expectations
of the device’s usability with a future-form System Usabil-
ity Scale [4]. During the study period, participants reported
experiences arising in the use of the device with DRM nar-
ratives and filled in the SUS questionnaire for perceived us-
ability daily. After the study period, the users filled in the
questionnaire once more. The study was conducted in the
native language of the participants (Finnish); the data cited
in the paper are reported in translation.
The Smartphone
The data for this study were gathered in 2010. We utilized a
device that presented what was new technology for our par-
ticipants. The device was the Nokia N97, a smartphone
launched in 2009 especially for browsing the Web. The op-
erating system of the N97 is Symbian S60, and the phone
has both a resistive touchscreen and a QWERTY keyboard.
The phone features a five-megapixel camera; a music-player;
3G, WiFi, and HSDPA connections; and an FM receiver.
Participants
The participants were 12 university students (three of them
female), of ages 2127. The subjects’ previous phones were
typically not smartphones, with the exception of partici-
pants 3, 5, 7, and 12, of whom subjects 3 and 12 had owned
a smartphone for approximately two years and the others
for less than a year. Participants received the N97 phones in
connection with a wider study. As compensation, they did
not have to pay for voice, text, or data use during the study.
Data Collection
SUS questionnaires
The System Usability Scale is a widely used and validated
quantitative metric for determining a global, summary usa-
bility score [4]. With this scale, participants rated 10 state-
ments about the usability of the device (e.g., “I thought the
system was easy to use”) on a five-point Likert scale from
“strongly disagree” to “strongly agree.” The SUS yields a
single number representing a composite measure of the
overall usability, within the range 0100.
Self-reporting
The Day Reconstruction Method, developed by Kahneman
et al. [16], is used for studying how people spend their time
and how they experience the activities and settings of their
lives. With this method, participants reconstruct their activi-
ties and experiences of the preceding day via procedures
designed to reduce recall biases (see also [19, 21]). For this
study, we formulated self-reporting forms, on which partic-
ipants filled in all daily activities for five given time bands
(morning, lunchtime, afternoon, evening, and night). After
having formed an overview of the day, the subject de-
scribed all episodes of anticipated use, use, or deliberate
non-use, along with the related user experiences. Partici-
pants filled in the reports every evening for the previous
day and sent them to the researcher.
ANALYSIS
We analyzed the self-reports with open-coding content anal-
ysis [21]. Open coding is a technique wherein the coding cat-
egories are not determined in advance; instead, they are cre-
ated and modified during the coding process. Our coding was
actually semi-open, since user experience was utilized as a
sensitizing concept. In particular, the coding categories were
formulated through exploration of the data in relation to the
definition of user experience [16]: How do participants de-
scribe their perceptions and responses that result from or are
related to the use, anticipated use, or deliberate non-use of the
device?
Classification of Experiential Episodes
We broke the self-reports down into experiential episodes
(N=739), which could include multiple user experiences
simultaneously but were schematized with an identifiable
beginning and end, such as a bus trip, a party, or an evening
at home (see also [8]). Neutral episodes (N=288) were de-
scriptions of the use, anticipated use, or deliberate non-use
of the device that did not show any positive or negative va-
lence (e.g., I call my friend”), and we detach them from
further analysis. Positive (N=169), and negative (N=158)
episodes involved only either positive or negative valence,
while ambivalent episodes (N=124) featured both.
Classification of User Experiences in the Episodes
We analyzed user experiences in the negative, positive, and
ambivalent experiential episodes (N=451) by distinguishing
among the ways in which user experiences were present in
them. This analysis yielded three coding categories for user
experiences:
1. Perception of a feature or functioning
2. Perception of an interaction with the device
3. Description of an emotion
User experiences in the first two categories were percep-
tions about either a feature or functioning of the device
(e.g., The newsfeed is useful”), or interaction with it (e.g.,
Browsing the Web feels easy”). In the last category, the
user experiences were descriptions of emotions (e.g., I feel
happy about owning this device”).
OVERVIEW OF THE DATA
Before reporting on the findings, we provide an overview of
the main tendencies in the dataset. In total, we identified
619 user experiences, across the selected experiential epi-
sodes. The participants differed in the number of user expe-
riences they reported. The participant-specific variations
were: 37 negative and 318 positive perceptions of feature
or functioning, 112 negative and 09 positive perceptions
of interaction with the device, and 222 negative and 324
positive descriptions of emotions.
Perception of a feature or functioning (Category 1): Posi-
tive perception (N=97) had to do most often either with us-
ability in terms of something being easy to use, or working
well / better than expected (38%), or with the broader
goodness of a certain feature (30%). Usefulness (23%) and
hedonic qualities such as aesthetics, fun, or “niceness” (9%)
were evaluated too. Negative perceptions (N=96) were re-
lated mostly to dimensions of usability such as slowness, or
clumsiness or to the device not working properly (73%). A
few negative evaluations pertained to uselessness (11%) or
to something being poor (8%) or ugly/awful (7%).
Perceptions of interaction with the device (Category 2):
Positive perceptions (N=53) were related to aspects of usa-
bility such as interaction succeeding or feeling easy (39%),
usefulness of interaction (36%), or the interaction feeling
intuitive/pleasant (25%). Negative perceptions (N=80)
mostly had to do with usability in terms of interaction fail-
ing or being difficult/slow (90%). A few pertained to use-
lessness (6%) or unpleasant interaction (4%).
Descriptions of emotions (Category 3): Positive emotions
(N=126) included descriptions of feeling content (N=54),
happy (N=34), pleased (N=18), delighted (N=8), excited
(N=10), and proud (N=2). Positive emotions were related
mostly to usefulness (46%), especially in terms of being
able to access information efficiently (N=18) and relieve
boredom by killing time (N=22). In addition, consequences
of use such as cheering up after calling a friend or enjoying
the relaxed feeling of not using the device (24%) and he-
donic elements related to feeling proud for owning the de-
vice or to enjoying a “cool ringtone” (19%) were related to
positive emotions. A few positive emotions had to do with
usability or with functionality surpassing expectations (11%).
Negative emotions (N=167) included descriptions of felt
annoyance and frustration (N=100), confusion (N=29), dis-
appointment (N=10), negative feelings (N=10), shame
(N=11), and nervousness (N=7). Negative emotions were
related mostly to usability problems such as the phone be-
coming unresponsive or being hard to use (73%) and to use-
fulness in terms of the device or a certain app not having
specific expected features (17%). A few were linked to con-
sequences of use such as having an unpleasant phone con-
versation (8%) and also to poor aesthetics in design (2%).
Correlations between User Experiences and SUS
For a rough grasp of the relationship between user experi-
ences and perceived usability we calculated correlations of
the quantity of user experiences and SUS scores. The aver-
age of daily SUS scores correlated with the number of
negative user experiences per day (r =-.64, p<.05): the more
negative user experiences, the lower the SUS score. The
expected SUS value correlated negatively with the number
of positive user experiences during the study (r=-.58,
p<.05): the higher the expected SUS score, the fewer posi-
tive user experiences there were during the study. No other
statistically significant correlations were found.
Evolving of User Experiences and SUS
Figure 1 provides an overview of how user experiences and
perceived usability evolved during the study. It presents the
count of positive and negative user experiences calculated
by summing participants’ daily numbers of user experienc-
es. The timeline shows a peak in negative and positive user
experiences. Participants reported many user experiences
during the first days of use, after which the number of user
experiences started to decrease and stabilize.
Figure 2 plots mean SUS scores against time in relation to
expected and end SUS score. Mean SUS scores have been
calculated as averages of participants’ daily scores. Over
the first few days, participants’ SUS scores crashed to about
SUS 55, after which they started to recover to nearly the
expected level. The difference in SUS ratings between ex-
pected SUS (mean=69) and SUS score after the first day
(mean=56) is statistically significant: t(11)=3.610, p<.005.
Figure 1. Count of positive and negative user experience from
one day to the next during the study period.
Figure 2. The mean day-to-day SUS scores during the study,
with expected (EXP) and end (END) SUS scores.
FINDINGS
We present qualitative findings in relation to four themes:
1) comparisons, 2) justifications, 3) triggers, and 4) ambiva-
lence. Firstly, we discuss how user experiences emerged
from comparisons to expectations and in relation to other
devices. Secondly, we analyze how expectations and per-
ceptions of instrumental qualities (i.e., perceived usability
and usefulness) were utilized to justify the use, anticipated
use, or deliberate non-use of the device. Thirdly, we discuss
the difference in the triggers of positive vs. negative emo-
tions. Fourthly, we analyze ambivalent experiential epi-
sodes in greater depth.
Comparisons with Existing Devices and Expectations
Many user experiences arose from comparison with other
devices, such as prior phone models or devices with similar
functions.
[P3] I miss my old phone that had a virtual keyboard that I
had gotten used tousing the hardware keyboard feels re-
ally burdensome, and I haven’t yet found a virtual
QWERTY keyboard for the N97.
[P12] I browse the Web a lot, because the new display feels
luxurious when compared to my old one. Unfortunately the
browser is as undeveloped as the one on my old phone, and
actually even my prior phone from 2003 had basically the
same browser. This makes me really disappointed.
Contrasts were often made against expectations. These ei-
ther were specific to the device, a certain feature or app, or
the functioning or involved broader expectations as to the
user-friendliness of all similar devices. Participants were
disappointed when their expectations were not met and pos-
itively surprised when certain aspects exceeded their expec-
tations or eclipsed prior experiences.
[P10] When I write an SMS, I use an a” instead of the
“ä” character, because I once again can’t discover how to
get the ä” letter conveniently. Feels stupid, but I don’t
have time to get to know the keyboard better; one would
expect that in the design of a cell phone, users are taken
better into account.
[P1] During breakfast, I download a few songs to my phone
just for testing. The downloading goes quickly and without
the hardships I somewhat expected.
Instrumental Qualities As Justifications for Use Behaviors
In the self-reports, participants often explained why they
had been doing certain things in a certain way. Expected
and perceived instrumental qualities (i.e., usability and use-
fulness) were utilized often as justification for how partici-
pants proceeded toward their goals. Efficiency and ease of
use were employed as justifications for use, and uselessness
and difficulty of use as justifications for non-use.
[P12] Maybe the biggest change in my life is that I’ve start-
ed to use only the phone’s calendar whereas before I used
both a phone’s and laptop’s calendar. Using the N97’s cal-
endar is just so convenient, because you can just pinpoint a
day and mark a meeting.
[P4] I did not use the phone when I was doing my workout
at the gym. A workout, an aerobic one especially, could in-
clude music, but it feels too difficult to adjust the phone to
play the workout music.
[P5] I still haven’t had the time to update the phone; I ha-
ven’t had the motivation, because at worst it can make the
whole thing unresponsive.
Reported Triggers for Positive vs. Negative Emotions
In addition to perceptions, user experiences included de-
scriptions of positive and negative emotions. Positive emo-
tions (N=126) were related especially to usefulness con-
nected to quick access to information and an opportunity to
relieve boredom with the smartphone.
[P6] I utilized my phone to fill in an Internet form. I was re-
ally pleased at being able to do that immediately with the
phone, since I was only on my way home, where I would
normally have to have filled in the form on my laptop.
[P10] It feels relieving that I can take the Internet with me
on the trip, just in case we get lost.
[P12] During a lecture, I learned that one flying game is a
really pleasant way to spend time, if the professor starts to
talk about something meaningless.
In addition, there were positive emotions related to conse-
quences of use wherein the focus was on not the act of us-
ing (e.g., calling) but on what using “resulted” in (e.g., talk-
ing to a friend).
[P10] I study the whole evening in the university’s computer
class, and about once an hour or two I call my friends for
some gossip so that I can take a break from work. My mind
always cheers up after a chat.
Some more hedonic aspects, such as enjoying a cool ring-
tone or feeling proud for owning abusiness” phone, were
connected to positive emotions.
[P7] We admire my new phone together, and I told my
friend how a large proportion of the functions are totally
useless for me but it just feels so good to own a phone with
which to amaze another goofy fool.
[P5] I forgot to put the phone on silent, but I don’t have to
be ashamed at all to show the phone, because it is the new
business phone.
Negative emotions (N=167) were associated especially with
usability problems with the device, a certain feature of it, or
an app being hard to use or not working properly / at all.
[P3] I receive a phone call, but I don’t know how to answer
itI try to press the answer button many times, but I don’t
realize that I need to open the keypad lock. My ringtone
rings loudly in the quiet corridor, and I decide to change it
to something less obtrusive once I get home. The phone call
ends up in voice mail, and I have to call the caller back. I
feel annoyed.
[P7] It made me mad that the phone became unresponsive
after I had quit one phone call; that is, immediately after
hanging up, the display presented those things that are usu-
ally present when someone is calling. When I tapped the
phone, it vibrated, but the screen stayed frozen with the
same view. When I restarted the phone, the problem went
away, but the whole thing made me quite upseton the ba-
sis of the first two days of use, I would not buy an N97.
Participants struggled especially with the initialization of
the device. Problems in this phase were decisive, because
they caused negative experiences also later, when the de-
vice could not be used for expected ends.
[P3] We listen to music from the speakers for about 15
minutes, and it irritates me that I was not able to transfer
the music in the morning. Now we have to listen to the
background music from a LocoRoco game I transferred
earlier to be my ringtone, and this really amuses my team.
[P1] I wasn’t able to transfer my calendar, so I try to re-
member events. I have to take along my old phone. It frus-
trates me that the synchronization doesn’t work.
Ambivalent Experiential Episodes
While prior studies have differentiated between positive
and negative user experiences, we identified also ambiva-
lent experiential episodes (N=124), which included positive
and negative user experiences simultaneously. Many of the-
se contradictions emerged when participants were pleased
with the possibility of using the device for a certain purpose
but at the same time felt that this functionality could have
been more usable and/or useful in a certain respect.
[P5] I ate breakfast and browsed through the news in the
mobile HS newspaper; the app was quite convenient,
though loading takes a really long time in comparison to
the iPhone. Everything occurs after a few seconds’ delay.
[P4] I started to look for people’s phone numbers on the
Net, so that I could call them and let them know I was late
for the seminar preparations (I should have been there al-
ready at 10am). This per se was nice; however, the phone
does not seem to have a copy-paste function, so I did find
the numbers but I could not call them. It annoyed me a bit.
[P11] Later in the evening, I also checked the HSL pages
for which bus my friend should take to get home. The pages
do not work as nicely as with the computer, and definitely
not as I would wish them to work. Finally I get the right di-
rections open and we check the route. Nifty even though it
took quite a lot of time.
Ambivalence was evident when participants commented on
something not being as usable as it could have been but still
enjoying its use because it was “better than nothing.Espe-
cially when participants felt bored, they claimed that utiliz-
ing hard-to-use apps for killing time was still able to relief
boredom.
[P8] I watched some videos on YouTube with my phone, be-
cause I couldn’t get to sleep. Watching the videos with the
3G connection is not nearly as quick as with broadband,
but the YouTube app is still a nice feature. At least it is re-
ally apt for boring moments and killing time.
[P2] (Day 4) On my way, I log in to Facebook, but the
phone gets somehow stuck, and I don’t really even like my
phone’s Facebook app. On the basis of my experience, I de-
cide to take the Facebook app off my home screen, because
at least now I don’t think I will use it. […] (Day 10) On the
train, I browse through Facebook, I guess I have gotten
used to it or maybe travelling on the train is just really bor-
ing and I go through my friends’ status updates to kill time.
Ambivalent experiential episodes arose also when partici-
pants enjoyed something that was perceived as pointless or
somewhat inappropriate in the broader social context. This
source of enjoyment was often time-killing activities. On
one hand, all participants commented that killing time made
them feel they were making use of their time at least some-
how, and it made the time go more quickly. On the other
hand, it was also perceived as unproductive or as something
that one should not really be doing. This resulted in am-
bivalent experiential episodes: enjoying passing the time
but simultaneously feeling guilt or annoyance.
[P4] I browsed the Web, feeling ashamed because I was one
of the organizers of the seminar. I wanted some amusement
to relieve boredom this way.
[P9] I played some games with my phone when using the
bathroom. The good side is that I enjoy the bathroom visits
much more, but the bad side is that my visits have started to
take a really long time because of my phone.
[P7] I feel ashamed to admit it, but today I got into playing
that guitarhero game found on my Nokia again. It is sur-
prisingly hooking, and surprisingly stupid. At the same time
addictive and annoying. One definitely does not become re-
laxed when playing those games. Yet they do offer a great
extra for moments when it is boring and one does not want
to get into unpleasant social situations.
DISCUSSION
In a parallel with some prior studies, we found perceived
usability connected especially to negative user experiences.
In a contrast to what prior perspectives suggest, positive us-
er experiences were often connected to instrumental quali-
ties, mostly usefulness. The self-reports showed a large
number of ambivalent experiential episodes, including both
positive and negative user experiences. These episodes in-
dicate that, even though good perceived usability was rarely
the cause of positive emotions, poor usability did make de-
sired uses less enjoyable. Also, a desired use found to be
inappropriate in the broader context evoked mixed feelings.
Perceived Usability Connected Especially to Negative
User Experiences
We found that perceived usability is connected especially to
negative user experiences. In the self-reports, 80% of nega-
tive perceptions had to do with poor usability, such as
hard-to-use features, or the device becoming unresponsive.
Usability problems caused 73% of the negative emotions
such as frustration arising from the device not working
properly, or annoyance with not understanding how the de-
vice functions. The number of negative user experiences
per day correlated with the average of daily SUS scores
(r=-.64, p<.05): the more negative user experiences, the
lower the SUS score. Positive user experiences were not
connected to perceived usability to the same extent. There
was no correlation between perceived usability and positive
user experiences. In the self-reports, 39% of positive per-
ceptions and 11% of positive emotions were related to good
usability.
The finding that perceived usability is connected more to
negative than to positive user experiences is in line with
prior empirical studies addressing the theme (e.g., [10, 19,
21]). For example, in a questionnaire study by Hassenzahl
et al. [12], good usability could eliminate barriers to
need-fulfillment but was not a source of positive affect per
se. These findings suggest that perceived usability acts as a
necessary precondition to positive user experiences, the
main causes of which lie elsewhere (see also [10, 11, 22]).
In the difference between negative and positive user experi-
ences, we see an analogy to Herzberg’s two-way model of
user satisfaction (see [14] and also [37]). Originally formu-
lated to explain work satisfaction, Herzberg’s model pro-
poses that dissatisfaction and satisfaction with work are not
polar opposites but related to different elements. Dissatis-
faction arises from a lack of “hygiene” factors (e.g., good
company policies, supervision, and salary) and satisfaction
from the presence of motivation factors (e.g., meaningful
work content, recognition, and responsibility). Studies on
this theme with respect to user satisfaction have found that
hygiene factors have more to do with the functional ele-
ments and motivation factors with the user-system interac-
tion and user involvement [37]. In consistency with recent
propositions (e.g., [6, 30]), usability can be understood as a
hygiene factor for minimizing users’ negative user experi-
ences. However, if it is to produce pleasurable user experi-
ences, good usability needs to be paired with motivating
factors that bring added value to users’ lives.
If good usability acts mostly as a hygiene factor, what are
the motivating factors that cause positive user experiences
in the HCI context? Some prior studies have focused on the
difference between instrumental “do” goals such as making
a phone call and more non-instrumental “be” goals such as
feeling related. This research suggests that the latter, “be”
goals produce pleasurable experiences through the fulfill-
ment of human basic needs (e.g., [11, 12, 13]). Before one
equates “be” goals with motivating factors, however, it is
worth discussing why in this study positive user experienc-
es were connected in large numbers to instrumental quali-
ties.
The Dominance of Instrumental Quality Perceptions
The concept of user experience has been offered as an al-
ternative to usability that focuses on positive user experi-
ences and goes “beyond the instrumental” (e.g., [11, 17, 26,
28]). While some research has emphasized the importance
of non-instrumental qualities in the formation of positive
user experiences (e.g., [11, 28]), positive user experiences
were often related to instrumental qualities in this study. In
the self-reports, 66% of positive perceptions and 57% of
positive emotions were related to instrumental qualities
(i.e., usefulness and usability); while good usability rarely
evoked positive emotions (11%), usefulness did so quite of-
ten (46%).
In light of existing views, there emerge at least three possi-
ble explanations for this finding: 1) instrumental qualities
are especially important for this device, or this class of de-
vices; 2) non-instrumental qualities influence user experi-
ences unconsciously and are not present in reflective self-
reports and 3) instrumental qualities have become a legiti-
mate discourse for arguing about a digital devices and may
dominate in self-reports.
Firstly, also prior longitudinal studies of smartphones have
found that instrumental qualities act as an important dimen-
sion along which users describe their user experiences [19].
What might explain this phenomenon is that smartphones
are often used to reach instrumental goals. According to
prior studies, the presence of instrumental goals makes us-
ers focus on them in their evaluations (e.g., [9, 12]).
Secondly, it is possible that non-instrumental qualities in-
fluence user experiences more unconsciously than instru-
mental qualities do and are not present in spontaneous, re-
flective self-reports. This explanation calls for more re-
search on the theme, because prior studies indicate that per-
ceived aesthetics influence perceived usability [34]. If this
holds, non-instrumental qualities, especially aesthetics,
could influence those reflective self-reports that explicitly
concern instrumental qualities. In contrast, more recent
studies on the theme have not found an interactive effect
between the two (e.g., [24, 25, 33]).
Thirdly, it is possible that instrumental qualities have be-
come a concurrent way of arguing for/against and perceiv-
ing new devices. Prior research has discussed the dilemma
of the hedonic, which arises when people overemphasize
the instrumental aspects over the more non-instrumental
qualities in a choice situation [6]. That research suggests
that non-instrumental qualities are what people would actu-
ally enjoy, because they pertain to reaching of
non-instrumental, “be” goals related to basic human needs
(such as feeling related) (e.g., [11, 12, 13]). However, in
choice situations, users seem to find it easier to utilize justi-
fications related to instrumental task-achievement [6]. Cor-
respondingly, in this study, instrumental qualities were uti-
lized to justify use behaviors: use was justified with effort-
lessness and usefulness, non-use with expecting certain use
to be burdensome and/or useless.
Broader Contexts of Experiencing
The self-reports bring out the importance of the broader
context in the formation of perceived usability and user ex-
perience. When forming reflexive user experiences, partici-
pants often compared it to what they expected and to expe-
riences with prior devices. Comparisons influenced emo-
tional experiences: participants described feeling disap-
pointed when their expectations were not met and feeling
content when certain functionality surpassed what had been
possible with prior devices. Prior studies too have acknowl-
edged the importance of expectations in the forming of
emotional reactions (e.g., [4, 9] and overall evaluations
(e.g., [11]). However, while short-term laboratory studies
have identified a boosting effect of expectations [29], in this
study the effect was contrariwise: failing to meet expecta-
tions led to disappointments.
Expectations influenced also SUS scores. Here we observed
a crash during the first days of use in relation to expected
SUS score. This crash was consistent with the high number
of negative user experiences. However, as users kept using
the system, the quantity of negative user experiences started
to decrease and SUS scores rebounded to close to the ex-
pected level. One explanation for this is that in the longer
term the effect of expectations starts to vanish [19]. How-
ever, while this might hold for expectations formulated be-
fore usage, we speculate on the extent to which new com-
parisons arise with use; that is, since comparisons seem to
be a way of articulating experiences, different comparison
points might emerge in tandem with use. This is what hap-
pens, for example, when a new appealing device comes to
market and leads users to disregard older ones. In addition,
in the self-reports we identified expectations that were not
specific to the device and, rather, pertained to broader ex-
pectations of all similar devices, such as expectations relat-
ed to user-friendliness. We speculate that expectations de-
rived in relation to experience and word-of-mouth on simi-
lar devices have a more lasting effect than more specific
expectations. That is, if the device fails to deliver what is
already known to take place with similar devices, there
probably does not occur a transition from fantasy to reality
[19]; in contrast, one sees what P7 argued: “on the basis of
the first two days of use, I would not buy an N97.”
The broader context was present also in those ambivalent
episodes wherein users felt ashamed for enjoying a desired
use when it was found somehow inappropriate. Prior stud-
ies have discussed the difference between instrumental do
goals and more non-instrumental begoals (e.g., [11, 12,
13]). We too believe that it is analytically important to dif-
ferentiate between these “do” goals and be” goals if we are
to understand both the what and the why of goal-pursuit
(see also [5]). However, while “do” and be” goals are dif-
ferent in analytical terms, we find that the two often inter-
twine. Doing something means also being something, and
users seem to be quite aware of this. Why else would they
feel ashamed for doing things that they enjoy but are not
socially appropriate? Supporting this idea, in a review on
empirical user experience studies instrumental and non-
instrumental goals were often found interwoven and insepa-
rable [1]. We suggest, that in day-to-day life, different goals
are related to each other in multiple ways, and pursuing one
aim can either promote or inhibit the reaching of other goals
(see also [31]).
Mixed Feeling?
Our novel finding is a sizable proportion of what we call
ambivalent experiential episodes (N=124). They are ambiv-
alent because they include both positive and negative user
experiences. The existence of this class of experiences
beckons further research. We observed ambivalence mostly
when certain use was perceived as desirable but lacking in
usability. For example, participants were pleased with the
new opportunities to use the Web for fast information ac-
cess and killing time, but they simultaneously perceived the
multiple usability problems (such as hard-to-use apps) neg-
atively. If we interpret this in relation to Herzberg’s two-
way model [14], these episodes comprise motivating factors
(desired, potentially pleasure-evoking uses) paired with
lack of hygiene factors (poor perceived usability).
The results pertaining to ambivalent episodes complement
findings from other longitudinal studies on the theme. Simi-
larly to what was seen in the study by Karapanos et al. [19],
usefulness emerged through the appropriation of the device
in specific use contexts (such as when travelling) and in rela-
tion to the changes this brought to participants’ lives (such as
having fast access to information). Karapanos et al. suggest
that, if meaningful mediation emerging from usefulness is
to be supported, there is a need for designs that are specific
enough to address a specific need but flexible enough to en-
able artful appropriation. Along with this conclusion, we
stress the importance of perceived usability: it is hard to en-
joy meaningful mediation if the device is not working
properly. Meaningful mediations may even go unexplored
on account of poor expected usabilityas seen in our data
when participants justified non-use with expecting some-
thing to be too difficult/burdensome relative to its benefits.
Our study dealt with only one device, and further studies on
the theme are needed. Supporting our findings, psychologi-
cal research has identified the importance of ambivalent at-
titudes and emotions (e.g., [31]). Consequently, we believe
that ambivalent experiential episodes offer an interesting
perspective and one complementary to that of user experi-
ence research. From the design point of view, ambivalent
episodes can help designers to identify users’ desires with
respect to a certain device in connection with the obstacles
(poor perceived usability or inappropriateness in the social
context) hindering complete fulfillment of these desires. For
embracing the users’ perspective and the holistic study of
experiences, it is important to utilize approaches that leave
room for ambivalent experiential episodes. Instead of iden-
tifying what people love or hate, we should understand
what makes them feel both things at the same time.
Conclusion
We studied the relationship between perceived usability and
user experiences, measured as articulated and reflexive ex-
periences. In a parallel with a few prior studies, we found
perceived usability connected more to negative than to posi-
tive user experiences. Unlike some prior studies we found
positive user experiences often connected to instrumental
qualities. A novel finding in our work is that perceived usa-
bility is related also to ambivalent experiential episodes.
These episodes emerged from a clash between desired uses
and either poor perceived usability or lack of appropriate-
ness. Desired uses can become less enjoyable via hard-to-
use features or if the use is experienced as inappropriate in
the broader social context. These findings complement ear-
lier findings and call for more research on the intersection
of user experience and perceived usability.
Acknowledgements
Raita acknowledges funding from UCIT Graduate School
and from the Doctoral School of University of Helsinki.
This work has been supported by the project Theseus fund-
ed by Tekes. We thank our colleagues and independent re-
viewers for their valuable feedback.
REFERENCES
1. Bargas-Avila, J. A. & Hornbæk, K. (2011). Old wine in
new bottles or novel challenges? A critical analysis of
empirical studies of user experience. In Proc. CHI 2011
(pp. 26892698). New York: ACM Press.
2. Bevan, N. (1995). Measuring usability as quality of use.
Journal of Software Quality, 4, 115130.
3. Brooke, J. (1996). SUSa quick and dirty usability
scale. In: P. W. Jordan, B. Thomas, B. A. Weerd-
meester, & I. L. McClelland (eds.), Usability Evaluation
in Industry (pp. 189194). London: Taylor and Francis.
4. De Angeli, A., Sutcliffe, A., & Hartmann, J. (2006). In-
teraction, usability and aesthetics: What influences users'
preferences? In Proc. DIS 2006 (pp. 271280). New
York: ACM Press.
5. Deci, E. L. & Ryan, R. M. (2000). The “what” and
“why” of goal pursuits: Human needs and the self-
determination of behavior. Psychological Inquiry, 11,
227268.
6. Diefenbach, S. & Hassenzahl, M. (2011). The dilemma
of the hedonicappreciated, but hard to justify. Inter-
acting with Computers, 23, 461472.
7. Finstad, K. (2010). The usability metric for user experi-
ence. Interacting with Computers, 22, 323327.
8. Forlizzi, J. and Battarbee, K. (2004) Understanding ex-
perience in interactive systems. In Proc. DIS 2004 (pp.
261268). ACM Press.
9. Hartmann, J., Sutcliffe, A., & De Angeli. A. (2008).
Towards a theory of user judgment of aesthetics and us-
er interface quality. ACM Transactions on Computer
Human Interaction, 15.
10. Hassenzahl, M. (2004). The interplay of beauty, good-
ness, and usability in interactive products. Human
Computer Interaction, 19, 319349.
11. Hassenzahl, M. (2008). User experience (UX): Towards
an experiential perspective on product quality. In Proc.
IHM 2008 (pp. 1115). New York: ACM Press.
12. Hassenzahl, M., Diefenbach, S., & Göritz, A. (2010).
Needs, affect, and interactive productsfacets of user
experience. Interacting with Computers, 22, 353362.
13. Hassenzahl, M. & Ullrich, D. (2007). To do or not to do:
Differences in user experience and retrospective judg-
ments depending on the presence or absence of instru-
mental goals. Interacting with Computers, 19, 429437.
14. Herzberg, F. (1987). One more time: How do you moti-
vate employees? Harvard Business Review, 65, 5362.
15. Hornbæk, K. (2006). Current practice in measuring usa-
bility: Challenges to usability studies and research. Inter-
national Journal of HumanComputer Studies, 64, 79
102.
16. ISO 9241-210. Ergonomics of humansystem interac-
tionPart 210: Human-centred design for interactive sys-
tems.
17. Jordan, P. (1998). Human factors for pleasure in product
use. Applied Ergonomics, 29, 2533.
18. Kahneman, D., Krueger, A. B., Schkade, D. A.,
Schwarz, N., & Stone, A. A. (2004). A survey method
for characterizing daily life experience: The day recon-
struction method. Science, 306, 17761780.
19. Karapanos, E., Zimmerman, J., Forlizzi, J., & Martens, J.
B. (2009). User experience over time: An initial frame-
work. In Proc. CHI 2009 (pp. 729738). New York:
ACM Press.
20. Krippendorff, K. (1980). Content Analysis: An Introduc-
tion to Its Methodology. Beverly Hills, California: Sage
Publications.
21. Kujala, S. & Miron-Shatz, T. (2013). Emotions, experi-
ences and usability in real-life mobile phone use. In
Proc. CHI 2013 (pp. 10611070). New York: ACM
Press.
22. Lai-Chong Law, E., Roto, V., Hassenzahl, M., Ver-
meeren, A., & Kort, J. (2009). Understanding, scoping
and defining user experience: A survey approach. In
Proc. CHI 2009 (pp. 719728). New York: ACM Press.
23. Lindgaard, G. & Dudek, C. (2003). What is this evasive
beast we call user satisfaction? Interacting with Com-
puters, 15, 429452.
24. Mahlke, S. & Lindgaard, G. (2007). Emotional experi-
ences and quality perceptions of interactive products. In:
J. Jacko (ed.), HumanComputer Interaction, Part I,
HCII 2007, LNCS 4550 (pp. 164173). Berlin: Springer.
25. Mahlke, S. & Thüring, M. (2007). Studying antecedents
of emotional experiences in interactive contexts. In
Proc. CHI 2007 (pp. 915918). ACM.
26. McCarthy, J. & Wright, P. (2004). Technology As Ex-
pe-rience. Massachusetts: MIT Press.
27. Nielsen, J. (1994). Usability inspection methods. In
Proc. CHI 1994 (pp. 413414). ACM Press.
28. Norman, D. A. (2002). Emotion and design: Attractive
things work better. Interactions Magazine, IX, 3642.
29. Raita, E. & Oulasvirta, A. (2011). Too good to be bad:
Favorable product expectations boost subjective usabil-
ity ratings. Interacting with Computers, 23, 363371.
30. Robert, J.-M. & Lesage, A. (2010). From usability to user
experience with interactive systems. In: G. A. Boy (ed.),
Handbook of HumanMachine Interaction. UK: Ashgate.
31. Tamminen, S., Oulasvirta, A., Toiskallio, K., &
Kankainen, A. (2004). Understanding mobile contexts.
Personal and Ubiquitous Computing, 8, 135143.
32. Thompson, M. M., Zanna, M. P., & Griffin, D. W.
(1995). Let’s not be indifferent about (attitudinal) am-
bivalence. In: R.E. Petty (ed.) an J. A. Krosnick, (ed.)
Attitude Strength: Antecedents and Consequences, 4,
361386.
33. Thüring, M. & Mahlke, S. (2007). Usability, aesthetics
and emotions in humantechnology interaction. Interna-
tional Journal of Psychology, 42, 253264.
34. Tractinsky, N., Katz, A. S., & Ikar, D. (2000). What is beau-
tiful is usable. Interacting with Computers, 13, 127145.
35. Tullis, T. & Albert, W. 2008. Measuring the User Expe-
rience: Collecting, Analyzing and Presenting Usability
Metrics. Massachusetts: Morgan Kaufmann.
36. van Schaik, P. & Ling, J. (2008). Modelling user experi-
ence with web sites: Usability, hedonic value, beauty
and goodness. Interacting with Computers, 20, 419443.
37. Zhang, P. & von Dran, G. (2000). Satisfiers and dissat-
isfiers: A two-factor model for Website design and
evaluation. JASIST, 51, 1253-1268.
... User experience (UX) according to ISO-9241-210 is "a person's perceptions and responses that result from the use or anticipated use of a product, system or service". One of its factors is usability [9] which is "extent to which a product can be used by specific users to achieve specific goals with effectiveness, efficiency and satisfaction in a specified context of use" (ISO/TS 20282-2:2013). Usability of a website can be measured by how easily and effectively the user can browse and perform specific tasks on the website. ...
... One of the components of UX are emotions, which can be measured by questionnaires or by various sensors [7,9]. They are commonly represented by Ekman's model of six basic emotions (joy, fear, sadness, disgust, anger, and surprise) [3] or a dimensional approach [11], which distinguishes two main dimensions, namely valence (how positive the emotion is) and arousal (the strength of an emotion). ...
... The researchers have not found a connection between significantly positive emotions and a level of usability yet, but it seems that a bad usability level can result into negative ones [9], especially when the user performs a task with a specific goal. This field of study is still to be explored, because the majority of papers on detection of emotions are concerned with affective corpora, such as watching videos or other multimedia content [7], instead of the impact of a user interface. ...
Chapter
User Experience is one of the most important criteria when designing and testing user interfaces with emotions as its essential element. To assess, how emotions could be used for automatic detection of usability issues, we carried out a user study with a website which included intentionally inserted usability issues. We classified valence of emotions, i.e., negative vs. positive ones based on data from electroencephalography (EEG) and facial expressions recognition. The study results confirmed that usability issues cause negative emotional response of the user and that presence of a negative emotion is a good predictor of a usability issue presence. When detecting negative and positive emotional states from the acquired dataset, we achieved the accuracy of 94% for samples with seconds granularity and 70% for the task granularity.
... First, we choose to add the Perceived Usability in our model, because is important for the e-learning usage (Ardito et al., 2006;Pangestu & Karsen, 2016;Kipkurui & Ikoha, 2014;Ramadiani et al., 2016;Alshehri et al., 2019;Wang et al., 2019); Eltahir et al., 2019;Harefa, 2020;Sholiq et al., 2021). It has been successfully proven that Perceived Usability has a positive influence on Perceived Usefulness (Lacka & Chong, 2016), Satisfaction (Belanche et al., 2012 andConfirmation (Raita &Oulasvirta, 2014). However, that influences are studied in Furthermore. ...
... This is because when users feel that they can achieve their goals through a system, their expectations tend to increase, meet their needs and their desires (Confirmation). In reports from Raita and Oulasvirta (2014), 80% of negative perceptions were related to poor usability, such as features that are difficult to use, or the device being unresponsive. Based on this explanation, the following hypothesis can be formulated as follow: H7: Perceived Usability has a positive effect on Confirmation. ...
... The seventh hypothesis is accepted with the test result t (150) = 11,840 and p < 0.05. These results are consistent with reports from Raita and Oulasvirta (2014) that the perception of users feeling unable to achieve their goals is related to poor usability. Poor usability is a confirmation of the user that the perceived experience is less than expected. ...
Article
Full-text available
E-Learning has been massively used in higher education, one of which is in Universitas Indonesia. The e-learning system in Universitas Indonesia is called Student Centered E-Learning Environment Universitas Indonesia (SCELE UI). This research determines the effect of usability on the intention to use SCELE UI in a sustainable way. The Expectation Confirmation Model (ECM) is used as a basis of the research model. Perceived Usability, Learnability, Presentation, Navigation and Error variables are added to the model. The Partial Least Square is used to analyze and test the model. Of the 12 proposed hypotheses, 10 hypotheses were accepted and 2 hypotheses were rejected. This research gives theoretical and practical contributions. On the theoretical side, this research successfully proposes and examines the integration of usability and ECM in the context of sustain e-learning usage intention. This research reveals the indirect effect of Perceived Usability on continued usage intention of the e-learning system. On the practical side, this research also provides insights for developers and universities that can help them to effectively foster the improvement of their system that can ensure sustained usage by their students.
... We see several potential benefits of adapting Herzberg's work to HCI. First, research in UX has hinted that hygienes and motivators are of relevance to UX [Calvillo-Gámez et al. 2010;Hassenzahl et al. 2010;Hertzum 2010;Raita and Oulasvirta 2014]. For instance, Hassenzahl et al. [2010] used the hygiene-motivator distinction to discuss the relation between need fulfillment and pragmatic and hedonic quality; others have suggested that usability may be a hygiene factor (e.g., Cockton [2006a]), meaning that low usability can create dissatisfaction but high usability in itself cannot create a satisfactory or even a great product experience. ...
... For instance, Karapanos et al. [2009] found that during the phase of incorporation (i.e., when users incorporate the product in their lives and it becomes meaningful in diverse use contexts), the overall "goodness" of a product is strongly associated with usefulness. In a recent field study that involved collecting daily self-reports over a period of 2 weeks of users using a new smartphone, Raita and Oulasvirta [2014] showed that instrumental qualities such as usefulness are often connected to positive experiences. As this is not in line with some earlier research that has emphasized the importance of non-instrumental qualities in regard to positive user experiences, Raita and Oulasvirta [2014] discussed three possible explanations for their findings: "(1) instrumental qualities are especially important for smartphones; (2) non-instrumental qualities influence user experiences unconsciously and are not present in reflective self-reports; and (3) instrumental qualities have become a legitimate discourse for arguing about a digital devices and may dominate in self-reports" (p. 7). ...
... In a recent field study that involved collecting daily self-reports over a period of 2 weeks of users using a new smartphone, Raita and Oulasvirta [2014] showed that instrumental qualities such as usefulness are often connected to positive experiences. As this is not in line with some earlier research that has emphasized the importance of non-instrumental qualities in regard to positive user experiences, Raita and Oulasvirta [2014] discussed three possible explanations for their findings: "(1) instrumental qualities are especially important for smartphones; (2) non-instrumental qualities influence user experiences unconsciously and are not present in reflective self-reports; and (3) instrumental qualities have become a legitimate discourse for arguing about a digital devices and may dominate in self-reports" (p. 7). These points also apply to our study as we have collected self-reported events. ...
Article
Full-text available
This article investigates Herzberg's [1959] notion of hygienes, factors contributing to dissatisfaction but not to satisfaction, and motivators, factors contributing to satisfaction but not to dissatisfaction, in the context of user experience (UX). Earlier work has theorized that the notion of hygienes and motivators applies to UX but has neither shown empirical evidence for this theory nor exemplified what such factors would look like in UX. We adapt Herzberg's methodology to analyze 303 events where users felt good or bad about their smartphone and derive factors that may work as hygienes or motivators. We identified technical quality and price as hygienes, and utility and convenience as motivators. These factors do not correspond to those mentioned as typical examples of hygienes and motivators in the UX literature (i.e., instrumental qualities such as usability for hygienes and non-instrumental qualities such as beauty for motivators). We discuss this discrepancy in the context of pragmatic and hedonic quality and psychological need fulfillment.
... Podle dosavadních výsledků grafické zpracování a vizuální přitažlivost webových stránek přímo ovlivňují jejich vnímanou použitelnost (Tractinsky et al., 2000;Mahlke & Lindgaard, 2007;Mahlke & Thüring, 2007;Raita & Oulasvirta, 2014;Thüring & Mahlke, 2007;Kurtzu, 2012). Při procházení webových stránek jsou uživatelé vystaveni značnému množství informací, přičemž neustále musí tyto informace třídit, rozlišovat mezi spolehlivými a nespolehlivými a vyřazovat tzv. ...
Book
Full-text available
Tato publikace se zaměřuje na analýzu parametrů webového designu, u kterých je předpokládán potenciální dopad na uživatelskou zkušenost a chování návštěvníka webu. Hlavním cílem publikace je především formulace výchozí definice konceptu kvality webových stránek na základě identifikace vzorců uživatelského chování v online prostředí v závislosti na funkčních a estetických parametrech designu. Stěžejním výstupem publikace je návrh metodického postupu měření kvality webových stránek a návrh vhodné metodiky pro vývoj těchto optimalizovaných aplikací.
... Studies examining the effect of perceived usability on experiences start with the initial finding that a product's visual appeal influences its perceived usability, as discussed by Tractinsky et al. [115], and they suggest that good perceived usability results in positive reactions and experiences, while poor perception leads to negative ones [71,72,89,114]. While perceived usability is an outcome which describes the match between user, task, and system in a context of use, in the following text we focus primarily on system related factors. ...
Article
Full-text available
A significant amount of research has been published to date studying various measures and influence factors related to the user experience when browsing Web content on different devices. For the most part these studies come from two different communities: the Quality of Experience (QoE) community and the User Experience (UX) community, and span different disciplines. While the QoE community has primarily focused on technical aspects and subjective perception of waiting times, the UX community has been working on issues of acceptance, experience, and crucial design factors extensively for a long time. This paper aims to provide a survey of literature related to QoE modelling for Web browsing by addressing studies that deal with the impact of a wide set of system, context, and human influence factors. The survey shows that the QoE community has for the most part neglected relevant aspects studied by the UX community, which are needed for a more holistic understanding of Web QoE. On the other hand, UX studies may benefit from insights into research conducted in the QoE domain in terms of the impact of more technical factors on UX. Thus, by bridging these findings we argue the need for future multidisciplinary and multidimensional studies on Web QoE modelling, whose product, that is, multidimensional Web QoE models, are of interest to multiple stakeholders involved in the service delivery chain. Readers of this paper will benefit from a systematic analysis of surveyed papers, summary of key findings, and a discussion of open research topics that contribute to setting a research agenda in this domain.
Poster
This poster presents the main aspects of our short paper with the title "Constant Companion: How Frequent Phone Use and Interpersonal Communication Are Related to Users' Emotional Appraisal". It is available under https://dl.acm.org/doi/10.1145/3473856.3474013 as supplemental material.
Conference Paper
Emotional appraisal is a core aspect of user experience. This study examines whether frequency of mobile phone use can have an impact on emotional aspects of mobile phone user experience. It also comments on the role of interpersonal communication for emotional evaluation. The results of an online survey in Germany (N = 836) reveal that frequencies are related to positive and negative emotions differently. Frequent phone users experience more positive emotions than others, but the frequent use of a mobile phone does not seem to reduce negative emotions. Surprisingly, this effect does not depend on the extent of interpersonal communication. Positive emotions seem to be stronger connected to a positive user experience, whereas negative emotions probably occur due to a bad usability. This can be interpreted as an evidence for Herzberg’s two-factor theory applied to user experience.
Chapter
Online business, commonly referred to as e-commerce, describes the process of buying, selling, and delivering services using electronic media. Because e-commerce relies most of its activities on websites that serve as the main channel of communication between the company and the customer, they must be designed to meet all customer needs. In general, the following six elements are mentioned in the context of e-commerce web design: (1) context: aesthetic and functional layout; (2) content: presented on the website in the form of texts, images, or multimedia; (3) community: ways in which pages facilitate interaction between customers with the same interests, such as sharing or commenting on social networks; (4) personalization: the ability of websites to tailor the display of content to customer preferences; (5) communication: the ways in which the website allows communication between the seller and the buyer; (6) trade: the ability of websites to mediate commercial transactions. This chapter uses previous research related to online user experience to list and classify the factors influencing the processing of online information and the overall perception of a website in relation to the website’s purpose.
Article
Full-text available
In this paper we describe an evaluation of two websites with the same content but different interface styles (traditional menu-based and interactive metaphors). A formative usability evaluation was carried out with heuristic assessment of aesthetics, and questionnaire assessment of aesthetics, content, information quality, usability and post-test memory. The study revealed that perception of information quality is affected by the interaction style implemented in the interface, in a manner resembling the halo effect in person perception. Implications for website design and evaluation are discussed.
Article
Full-text available
In the past, research on human-technology interaction has almost exclusively concentrated on aspects of usefulness and usability. Despite the success of this line of research, its narrow perspective has recently become a target for criticism. To explain why people prefer some systems over others, factors such as aesthetic qualities and emotional experiences play an important role in addition to instrumental aspects. In the following, we report three experiments that illustrate the importance of such factors. In the first experiment, we study the role of emotions in human-technology interaction by using Scherer's (1984) component theory of emotions as a theoretical foundation. A combination of methods is derived from that theory and employed to measure subjective feelings, motor expressions, physiological reactions, cognitive appraisals, and behaviour. The results demonstrate that the manipulation of selected system properties may lead to differences in usability that affect emotional user reactions. The second experiment investigates the interplay of instrumental and non-instrumental system qualities. The results show that users' overall appraisal of a technical device is influenced by both groups of qualities. In the third experiment, we join the approaches of the first two studies to analyse the influence of usability and aesthetics within a common design. The results indicate that systems differing in these aspects affect the perception of instrumental and non-instrumental qualities as well as the users' emotional experience and their overall appraisal of the system. Summarizing our results, we present a model specifying three central components of user experience and their interrelations (CUE-Model). The model integrates the most important aspects of human-technology interaction and hints at a number of interesting issues for future research.
Conference Paper
Full-text available
User Experience (UX) is not just "old wine in new bottles". It is a truly extended and distinct perspective on the quality of interactive technology: away from products and problems to humans and the drivers of positive experience. This paper will present my particular perspective on UX and will discuss its implications for the field of Human-Computer Interaction.
Article
Proliferating Web-user interface studies prompt a need for theoretical approaches. This study presents a two-factor model that can guide Website design and evaluation. According to the model, there are two types of Website design factors: hygiene and motivator. Hygiene factors are those whose presence make a Website functional and serviceable, and whose absence causes user dissatisfaction (thus dissatisfiers). Motivator factors, on the other hand, are those that add value to the Website by contributing to user satisfaction (thus satisfiers). An empirical study is conducted in two phases. In Phase I, 44 core features and 12 categories of features were identified by a total of 76 subjects as Web design factors. In Phase II, 79 different subjects distinguished hygiene and motivator factors in the context of a particular Website (CNN.com). The results showed that the two-factor model provides a means for Web-user interface studies. In addition, Subjects in Phase II commented that, as time passes or familiarity increases with certain design factors, their identification of what are hygiene and motivator factors might change, promoting further investigation and possible expansion of the model. Suggestions for Website designs and evaluation, and further research directions are provided.
Article
Proliferating Web-user interface studies prompt a need for theoretical approaches. This study presents a two-factor model that can guide Website design and evaluation. According to the model, there are two types of Website design factors: hygiene and motivator. Hygiene factors are those whose presence make a Website functional and serviceable, and whose absence causes user dissatisfaction (thus dissatisfiers). Motivator factors, on the other hand, are those that add value to the Website by contributing to user satisfaction (thus satisfiers). An empirical study is conducted in two phases. In Phase I, 44 core features and 12 categories of features were identified by a total of 16 subjects as Web design factors. In Phase II, 79 different subjects distinguished hygiene and motivator factors in the context of a particular Website (CNN.com). The results showed that the two-factor model provides a means for Web-user interface studies. In addition, Subjects in Phase II commented that, as time passes or familiarity increases with certain design factors, their identification of what are hygiene and motivator factors might change, promoting further investigation and possible expansion of the model. Suggestions for Website designs and evaluation, and further research directions are provided.
Article
How to measure usability is an important question in HCI research and user interface evaluation. We review current practice in measuring usability by categorizing and discussing usability measures from 180 studies published in core HCI journals and proceedings. The discussion distinguish several problems with the measures, including whether they actually measure usability, if they cover usability broadly, how they are reasoned about, and if they meet recommendations on how to measure usability. In many studies, the choice of and reasoning about usability measures fall short of a valid and reliable account of usability as quality-in-use of the user interface being studied. Based on the review, we discuss challenges for studies of usability and for research into how to measure usability. The challenges are to distinguish and empirically compare subjective and objective measures of usability; to focus on developing and employing measures of learning and retention; to study long-term use and usability; to extend measures of satisfaction beyond post-use questionnaires; to validate and standardize the host of subjective satisfaction questionnaires used; to study correlations between usability measures as a means for validation; and to use both micro and macro tasks and corresponding measures of usability. In conclusion, we argue that increased attention to the problems identified and challenges discussed may strengthen studies of usability and usability research.
Article
It's a manager's perennial question: "How do I get an employee to do what I want?" The psychology of motivation is very complex, and what has been unraveled with any degree of assurance is small indeed. But the dismal ratio of knowledge to speculation has not dampened managers' enthusiasm for snake oil, new forms of which are constantly coming on the market, many of them with academic testimonials. The surest way of getting someone to do something is to deliver a kick in the pants-put bluntly, the KITA. Because of the inelegance of a physical KITA and the danger that a manager might get kicked in return, companies usually resort to positive KITAs, ranging from fringe benefits to employee counseling. But while a KITA might produce some change in behavior, it doesn't motivate. Frederick Herzberg, whose work influenced a generation of scholars and managers, likens motivation to an internal generator. An employee with an internal generator, he argues, needs no KITA. Achievement, recognition for achievement, the work itself, responsibility, and growth or advancement motivate people. The author cites research showing that those intrinsic factors are distinct from extrinsic, or KITA, elements that lead to job dissatisfaction, such as company administration) supervision, interpersonal relationships, working conditions, salary, status, and job security. Managers tend to believe that job content is sacrosanct. But jobs can be changed and enriched. Managers should focus on positions where people's attitudes are poor, the investment needed in industrial engineering is cost-effective, and motivation will make a difference in performance.