Content uploaded by Pariya Kashfi
Author content
All content in this area was uploaded by Pariya Kashfi on Jul 02, 2020
Content may be subject to copyright.
Integrating UX Principles and Practices into Software
Development Organizations:
A Case Study of Influencing Events
Pariya Kashfi∗
, Robert Feldt, Agneta Nilsson∗
Division of Software Engineering, Department of Computer Science and Engineering,
Chalmers University of Technology, Gothenburg, Sweden
Abstract
Current studies on User eXperience (UX) integration often do not investigate
or reflect on the transition companies go through from only developing Graph-
ical User Interfaces (GUI) to also considering usability and more recently UX.
Understanding this transition provides a more holistic and realistic picture of
integration and can be a rich source of knowledge for improving UX integration
in the software industry. Applying case study and grounded theory research
we show that UX integration, like other organizational changes, can include a
mixture of planned and emergent initiatives, and is influenced by various inter-
twined events; not only those that reside inside an organization but also those
external to it. We also show that different decisions that are made outside the
authority of UX practitioners have an inevitable impact on enabling or pro-
hibiting UX integration. In addition, we found that for a successful integration,
practitioners need to explicitly consider and address the characteristics of UX,
otherwise, the integration efforts may have a lopsided focus on the pragmatic
aspect of UX, consequently, leave the hedonic aspect unaddressed. Based on
our findings, we present four lessons learned and five pitfalls companies should
consider to go beyond GUI design and usability to also address UX.
Keywords: User eXperience (UX), usability, software quality, case study,
grounded theory, organizational change
1. Introduction
Delivering a large set of functions is often no longer enough for the business
success of software, rather various software quality characteristics also need to
be considered in design and development [12]. One such characteristic is User
eXperience (UX) that relates to the actual experience of the end users with the
∗Corresponding author
Email address: pariya.kashfi@chalmers.se (Pariya Kashfi)
Preprint submitted to Elsevier September 28, 2019
software. ISO/IEC 9241 [31] defines UX as “a consequence of the presentation,
functionality, system performance, interactive behavior, and assistive capabili-
ties of an interactive system, both hardware and software. It is also a conse-
quence of the user’s prior experiences, attitudes, skills, habits, and personality.”
Good UX not only contributes to higher work motivation and performance, but
can also affect the well-being of users, and is crucial to maintain or gain market
shares [27, 47]. Although practitioners cannot guarantee a certain experience
(e.g., excitement or curiosity), they are recommended to consider principles and
practices that can make it more likely to deliver an overall appealing UX [27].
We refer to these principles and practices as UX principles (e.g. UX is dy-
namic and changes over time) and UX practices (e.g. identify users’ emotional
requirements).
Applying UX principles and practices in isolation is not enough and, as
empirical research findings show, early and continuous attention to them is re-
quired to ensure delivering a good UX through the developed software [1, 17, 48].
Hence, UX principles and practices need to be integrated into the development
processes and considered early on and throughout projects in order to have an
impact [16, 32]. We refer to the timely process of integrating UX principles
and practices into development processes and organizations as UX integration.
Here, by integration, we emphasize making these principles and practices an in-
tegral part of the development processes and not merely add-ons. UX principles
and practices should be adjusted to and aligned with already existing software
development principles and practices. Most importantly, it is not enough to in-
troduce them only in later stages of software development, rather, organizations
need an early and continuous commitment to these principles and practices for
them to have an impact [16, 32].
Similar to the case of integrating usability [22, 52] and quality characteristics
in general [5, 49], practitioners face various challenges in UX integration [36,
41, 42, 43]. These challenges vary in nature and range from more practical
challenges (e.g. lack of tools and methods to support UX practices in software
development processes [32, 44, 64]) to more fundamental challenges (e.g. lack
of a unified understanding of the concept of UX in the software industry [37,
39, 62]).
To support practitioners in their UX integration efforts, various empirical
studies report on challenges and success factors for usability and UX integra-
tion [22, 33, 36, 41, 42, 43, 52]. These studies, however, often do not inves-
tigate or reflect on the transition companies go through from only developing
Graphical User Interfaces (GUI) to also considering usability and more recently
UX Similarly, these studies do not investigate how these challenges and suc-
cess factors or their influence on integration change over time. Hence, little
is known about how these challenges and success factors emerge and influence
UX integration over time and as an organization moves beyond usability to also
address UX. Understanding the transition provides a more holistic and realis-
tic picture of UX integration and can be a valuable source of knowledge for
both researchers and practitioners who aim to improve UX integration in cer-
tain organizations or in the software industry in general. This knowledge can
2
help the community to learn from, apply, or customize and extend the existing
ways to improve integration in other contexts (e.g. usability integration) also
in the case of UX. Such knowledge can help practitioners better predict and
plan to overcome such challenges through employing success factors that suit
their organizations. Researchers can also better support practitioners through
developing more industry relevant findings.
We performed a case study [55] to address this research question: How does
UX integration unfold over time within the context of an organization? And,
what are the main intertwining events that impact UX integration as it unfolds?
We gathered longitudinal (retrospective) data and performed a Grounded The-
ory (GT)-based [59] analysis of our data to investigate the main events that in
an interplay with each other impacted the phenomenon under study [55], in our
case, UX integration. Here, by event we mean any decision, activity, action, or
circumstance that contributed to changes in the organization. For this purpose,
we investigated over two decades of events in a Swedish software development
company following the principles of GT research.
This paper is organized as follows: Section 2 describes the background and
related work. Section 3 presents the research methods applied in this study.
Section 4 describes the findings. Section 5 includes our discussion and implica-
tions of the findings. Finally, Section 6 presents some concluding remarks and
future directions of the research.
2. Background
Usability is often seen as a necessary precondition for good UX yet differ-
ent from it [26, 38]. One of the widely used definitions of usability is given by
ISO/IEC 9241 [31]: “the extent to which a system, product or service can be used
by specified users to achieve specified goals with effectiveness, efficiency, and sat-
isfaction in a specified context of use.” UX has five unique characteristics that
differentiate it from usability (and all quality characteristics for that matter):
UX is subjective (heavily relies on human perception), holistic (includes both
hedonic and pragmatic aspects of use), dynamic (changes over time), context-
dependent (is situated in context), and worthwhile (encompasses positive and
meaningful consequences of use) [27].
Although practitioners cannot guarantee a specific experience, applying cer-
tain principles and practices can increase the likelihood of delivering a good
UX [27]. We refer to such principles and practices as UX principles and prac-
tices. Here, by principle we mean “a comprehensive and fundamental law,
doctrine, or assumption” [11]. Principles provide the basis for many different
software practices [7] and are important factors and fundamental concepts that
practitioners need to take into account in their work. UX principles, in fact,
reflect the understanding of UX as a phenomenon. We separate principles from
practices, activities that practitioners need to perform in order to satisfy the
principles [7]. Practices are performed throughout the life-cycle of a software
system and in different steps of the process (analysis, design, development, eval-
3
Term Description Examples
UX principles important factors and fundamen-
tal concepts that reflect the under-
standing of UX as a phenomenon.
Practitioners need to take these
principles into account in their
work
both hedonic and pragmatic aspect
of software use play an important
role in forming UX, UX is temporal
UX practices activities that practitioners need
to perform in order to satisfy the
principles
identify users’ personal goals and
preferences, create prototypes, in-
volve users in the design process,
evaluate the software from b oth
pragmatic and hedonic perspec-
tives
UX methods impose structure on the practices
with the goal of making them sys-
tematic and ultimately more likely
to be successful
survey, questionnaire, mind map-
ping, field study, cognitive map-
ping, design studio
UX tools computer-based programs or ana-
log means that assist practitioners
in performing various practices and
are often designed to support par-
ticular methods and like them in-
tend to make the work of practi-
tioners more systematic
persona, eye-tracking programs,
visual design and prototyping
tools, Attrakdiff (a specific type of
satisfaction questionnaire)
Table 1: The definitions of UX principles, practices, tools, and methods
uation). Tools and methods specify in more details ‘how’ these practices shall
be performed to satisfy the principles [7].
Practitioners still do not have access to standards or agreed upon lists of
principles and practices that can support delivering a good UX through the
developed software. Scattered examples of UX practices and principles can be
found in UX literature. Table 1 tabulates the description of the above terms
and provides examples for each.
Admittedly, practitioners can also apply User-Centered Design (UCD) prin-
ciples and practices to address UX [30]. Yet, empirical data shows that although
an ideal UCD process should focus on the overall UX, this aspect of UCD is
often ignored in practice [45]. Furthermore, whilst research on UX emphasizes
the hedonic aspect of software use, practitioners who apply UCD still mainly
focus on functional and usability issues [37, 62]. Therefore, in this paper, we
differentiate between UX integration and usability integration (these two are
also referred to as UCD integration in current UX and usability integration
literature). While the latter includes efforts to assure usability, the former con-
centrates on going beyond usability and also focusing on the hedonic aspect of
UX.
As we mentioned, applying UX principles and practices in isolation is not
enough; they rather need to be integrated into the development processes and
organizations to be effective [1, 17, 48]. However, many organizations still face
various challenges that prevent them from achieving a sustainable UX integra-
tion [2, 62, 37], an integration successful not only in a short period of time but
also maintained over time.
Empirical studies show that a successful integration requires a long-term
4
commitment and can be achieved over a long period of time through a combi-
nation of changes to the processes or organizations [23]. Hence, when analyzing
UX integration challenges and success factors, we need to investigate how they,
or their influence on integration, may change over time rather than only inves-
tigate a snapshot of these challenges and their influence on integration. More
importantly, we also need to investigate how organizations move from only devel-
oping user interfaces to also paying attention to usability and then UX [25, 27].
The insights that can be gained from such investigations can help the commu-
nity to learn from, apply, or customize and extend the existing ways to improve
integration in other contexts (e.g. usability integration) also in the case of UX.
Nevertheless, such analyses of UX integration challenges and success factors are
rare in existing research.
In our study of the related work, we found at least four shortcomings in the
current literature on UX integration:
•these studies often do not differentiate UX and usability or address how
these differences can impact day-to-day work of practitioners or integra-
tion efforts (compare for instance Schaffer’s guideline on usability integra-
tion [56] and Schaffer and Lahiri’s UX integration guideline [57]). These
studies sometimes even use the terms UX and usability interchangeably
(e.g [3, 15, 40]).
•those studies that explicitly take differences between UX and usability
into account, mainly focus on how the concept of UX is perceived in the
industry (e.g. [37, 39, 62]), or on evaluation activities and the role of
UX measures in challenges practitioners face (e.g. [32, 44, 64]). They,
therefore, do not often discuss other topics related to UX integration (e.g.
communication and collaboration between UX and non-UX practitioners).
•those studies that report on challenges and success factors often give a
short snapshot of the current state of challenges and success factors in
software companies and do not investigate the transition from GUI design
to also addressing usability and UX. Examples are survey studies that
gather practitioners’ views on UX or usability challenges and success fac-
tors [22, 52, 63]. These studies provide a valuable collection of challenges
and success factors but, because of their non-longitudinal nature, cannot
necessarily reflect on the transition, or describe how these challenges and
success factors influence integration over time.
•the existing limited number of longitudinal studies also often only focus on
usability or UCD practices in general or do not clearly differentiate them
from UX and UX integration (e.g. [9, 10, 15, 23]). Furthermore, when
investigating events that can influence integration, these studies tend to
focus more on events that happen inside the organizations. In addition,
these studies often focus on direct manipulation of processes and orga-
nizations by researchers to transfer knowledge and expertise, in form of
action research, and then investigate the impact of these manipulations.
5
They, therefore, do not often explore other types of events that in a real
industrial setting may influence integration over time.
Below we elaborate on the handful of longitudinal studies we found as they
are the closest to our study in their approach to investigating integration. Gul-
liksen et al. [23] performed a longitudinal case study to investigate challenges in
usability work in a large Swedish organization. They applied an action research
approach and over four years took an active role in making changes to usability
work and introducing principles and practices of UCD. Gulliksen et al. describe
various activities that in an interrelation with each other impacted usability in-
tegration in the organization. These activities are divided into three categories
based on their nature: strategic (what the organization needs to do), process
(usability practices), and individual (who performs what and how their atti-
tudes impact usability integration). Gulliksen et al. conclude that integrating
UCD requires a long-term commitment and can be achieved over a long pe-
riod of time through a combination of changes to strategy, process, and internal
stakeholders’ attitude as well as day-to-day work [23].
Through two longitudinal action research pro jects, Cajander et al. [9, 10]
studied how UCD principles can be integrated into software companies. They
first identified the problem areas, i.e. challenges to usability integration, in
the case companies then proposed and implemented solutions for them. One
such solution is usability coaching that supports practitioners in reflecting on
their views and actions, as well as their role in promoting usability in their
organizations. Similarly, Eriksson et al. [14] studied how usability roles can be
introduced into software companies. They interviewed the practitioners hold-
ing such roles in five case companies to better understand problems they face
in their day-to-day work. They then proposed improvements to enhance the
effectiveness of these roles.
Federoff et al. [15] studied how a transition from waterfall to agile in a
software company negatively impacted UX practices and what strategies could
decrease this impact and contribute to a better UX integration. They, however,
seem to use the terms UX and usability interchangeably. As another example,
Winter et al. [65] investigated eight years of UCD process in a product devel-
opment company. They discuss how in the context of this company, principles
and practices of UCD are operationalized. In addition, Winter et al. applied
an action research approach to improve the state of usability integration in the
company. They, for instance, introduced new UCD practices in the company
that could potentially address some of the identified challenges. One example is
a new way of presenting the results of usability testing to internal stakeholders.
The above studies show the difficult and time-consuming nature of efforts
to integrate UX and usability principles and practices into development pro-
cesses and organizations. They also highlight the importance of the long-term
commitment of internal stakeholders and applying a combination of activities
in different levels of the organization, from for instance higher level strategies
to day-to-day work of stakeholders [23]. However, as we mentioned before they
either merely focus on usability or do not clearly differentiate it from UX.
6
This study extends the work previously done by the above studies. Here,
rather than focusing on generating a comprehensive list of challenges and success
factors to UX integration, we aim to reflect on and learn from a chronological
study of events that over two decades impacted GUI development, usability
and UX integration in a case company. We reflect on the company’s transi-
tion towards UX integration, the interrelation between various events, and the
facilitating and prohibiting roles they played in this transition.
3. Research Method
The core of this paper is a chronological analysis of the transition the case
company has gone through over the last two decades to improve UX integration.
Our analysis accentuates the way in which UX integration emerges and develops
through time and uses ‘events’ as our unit of analysis. Here, by event we mean
any decision, activity, action, or circumstance that contributed to changes in
the organization.
We performed a case study [55] using longitudinal (retrospective) data and
performed a Grounded Theory (GT)-based [59] analysis of our data. As Figure 1
depicts, we followed Strauss and Corbin’s guidelines [59] to guide our data gath-
ering and analysis. GT is a popular method in various fields including software
engineering and development as it is suitable for investigating ‘what is going on’
and generating new theories rather than verifying an existing theory [58].
Performing a case study using longitudinal data provided the following ben-
efits [55] which suited our research aim:
•it is the recommended methodology for investigating the evolution of com-
plex phenomena over time
•it facilitates addressing research questions over time
•it reduces the risk that the findings only reflect a transient phenomenon
3.1. The Case Company
The case company is a medium-sized international software development
company in Gothenburg, Sweden. At the time of performing this research, the
company had around 2500 employees, and developed software using an agile de-
velopment process. This company was a suitable case for our research purpose
for four main reasons. First, the company is a medium-sized software develop-
ment company with various organizational units and different roles and respon-
sibilities across them which could give us the possibility to study the influence
of these units on integration over time. Second, the company’s main product,
which we studied, is a business-to-business (b2b) software system customized
for and sold to large international corporations. Studying UX integration for a
b2b software was in particular interesting for us as current literature on UX has
a lopsided focus on business-to-consumer products while research shows that
integration for b2b software is more challenging [34]. Third, the company has
7
Coding
Data collection
Literature review Formulating the
research questions
Memo sorting Writing up &
refining the theory
no
Goal
Goal
open
Interviews
generated
UX
artifacts
project
project
project
documents
Goal
Goal
semi-
structured
interviews
project
project
organisational
documents
Goal
Goal
observations
Goal
Goal
observations
UX integration
retrospective
workshops
Theoretical
saturation?
Figure 1: In this case study, we gathered longitudinal (retrospective) data and performed a
Grounded Theory (GT)-based [59] analysis of our data. We followed Strauss and Corbin’s
guidelines [59] to guide our data gathering and analysis
8
gone through a transition from waterfall to agile software development which
in our view could provide valuable insight on the influence of these approaches
on integration efforts. Fourth, the company is located in Gothenburg Sweden
which could give us easier and more frequent access to practitioners and other
data sources.
The company is structured in three main units. The first unit is the develop-
ment unit which is responsible for developing the core features of the products.
UX integration initiatives are mainly performed in this unit. At the time of
performing our research (2015), this unit hired a UX expert to take the re-
sponsibility of UX integration in the organization and coordinate the related
activities. This unit also includes a UX guild, a volunteer group that holds
weekly meetings to discuss UX-related topics such as (i) what activities are the
members performing currently in their teams, (ii) what problems are they fac-
ing in relation to these activities, (iii) what specific design problems are they
facing in their projects. In addition, the guild meetings involve more generic
discussions about how UX integration can be improved in the organization or
how UX advocates can get buy-in from other internal stakeholders in particular
management. The second unit is the customization unit which is responsible
for customization of the features according to the needs of each customer. The
third unit is the business unit which mainly consists of product owners and
product managers. This unit is in large responsible for aligning the products
with business model and strategy of the company.
Data Gathering
In our study, data was gathered through a collection of empirical methods
including observations, interviews, document analysis and workshops, as sum-
marized in Table 2. In our data gathering, we included a variety of roles with
different responsibilities and authorities. Examples are business analysts who
were responsible for requirements gathering and analysis or agile roles such as
product owners or scrum masters.
The first author was located at the case company for a period of four months
(Feb 2015 to June 2015) to facilitate easier access to the practitioners and data.
Besides the methods described below, the first author had regular email and
telephone contact with the practitioners who participated in the study.
We performed 19 interviews across the three units of the organization in two
steps. At the first step, 13 open and semi-structured interviews were performed
by the first author. The initial interviews were open interviews since we aimed
to discover the main events that influenced UX integration over the years. This
helped us explore the events related to various initiatives, activities, UX roles
and responsibilities, practices, tools and methods, and artifacts. As we learned
more about these events, we performed interviews to gather detail information
about the main identified events.
In order to better understand and gather more information about current
UX activities in the company, a number of main meetings performed by the
UX expert (e.g. design studio, effect mapping workshop), and weekly meetings
of the UX guild were observed by the first author. The above meetings were
9
Method Quantity &
Date
Description/Examples Purpose
Interview 19 interviews:
- Dec. 2014 (8)
- Feb. 2015 (7)
- Mar. (3)
- Apr. (1)
the roles we interviewed:
- management (1)
- product manager (1)
- product owner (5)
- scrum master (2)
- UX advocate/developer (3)
- developer (3)
- tester (1)
- business analyst (2)
- UX expert (1)
- identify the main
events
- investigate their influ-
ence on integration
Archival
data anal-
ysis
organizational
documents &
project arti-
facts generated
1992-2015
- organizational charts & job de-
scriptions
- mockups
- UX & usability guidelines
- personas
- usability reports
- UX waves
- triangulate the findings
of the interviews
- identify the influence
of various events on in-
tegration
Workshop two workshops:
- Apr. 2015
- May 2015
- the first workshop included the
representatives of the three orga-
nizational units,
- the second one included the UX
expert
- validate & reflect on
the identified events
- investigate their influ-
ence on integration
- identify future ap-
proaches to address the
identified challenges
Observation seven in:
- Dec. 2014 (2)
- Feb. 2015 (1)
- Mar. 2015 (2)
- Apr 2015 (1)
- May 2015 (1)
- guild meetings (5)
- design studio (1)
- effect mapping workshop held
by the UX expert (1)
- to gather data on cur-
rent integration efforts
- investigate the attitude
of various stakeholders
towards these efforts
Table 2: Data gathering methods used in this study
10
both audio recorded and transcribed for coding and analysis. Because we aimed
to investigate the attitudes of various internal stakeholders towards the role of
the UX expert and current integration efforts, in these meetings, we mainly
focused on the collaboration and communication between the UX expert and
the meeting participants, their reflections, reactions and perceptions rather than
the outcome of the meetings, i.e. the UX artifacts.
After both meetings the first author gathered feedback of the participants
via short interviews to understand how they perceived work of the UX expert
and in particular the tools and methods she suggested. In addition, the first
author was located in the same room as the UX expert to be able to closely
observe various activities performed by this role. Regular discussions with this
role also naturally happened due to this co-location.
Beside the UX expert, the UX guild was another rich source of information
about current UX practices in the company. Hence, five guild meetings were
also observed by the first author. The meeting discussions were recored in form
of extensive notes and the summary of the main points were distributed among
the participants for validation. These observations provided rich data on how
UX guild members perceived the UX challenges and success factors.
In addition, we investigated the UX artifacts generated from 1992 to 2015.
The aim of this document analysis was mainly triangulation of the interview
data, i.e. verify the existence of the artifacts mentioned by the interviewees
or discover the ones that were missing from our data. We, therefore, did not
aim to analyze the quality of these artifacts. Admittedly, such analysis could
provide valuable information regarding UX integration, for instance additional
explanation for why some artifacts were generated but not used. Nevertheless,
we believed such a judgment required a more controlled setting and longer
investigation of generation and use of these artifacts. This was therefore not
feasible and out of the scope of our study. In particular, since a number of these
artifacts were generated before the start of our research study.
Other sources of data were two workshops held at the company. A retrospec-
tive workshop was held in the company with participants representing the three
units. The aim of the workshop was to reflect on the events identified through
interviews and their influence on UX integration in the company. The UX ex-
pert was not invited to the meeting mainly to give the participants a chance to
freely reflect on and share their views about her role and responsibilities, and
motivations for hiring her.
A second retrospective workshop was held at the end of the data gathering.
This workshop was mediated with an impartial researcher with knowledge in
organizational change. In this workshop, the UX expert and the first author
reflected on the factors that, in their view, had facilitated or prohibited UX
integration in the company. The UX expert also reflected on her work in the
company, what practices, tools and methods she had proposed, and how they
were received by other stakeholders. Through these workshops we not only
validated our findings but also gathered more data on the identified events and
their influence on integration.
11
Data Analysis
In this study, we performed a Grounded Theory (GT)-based [59] analysis
of our data. Following the principles of Grounded Theory (GT), we simulta-
neously performed data gathering and analysis until we achieved theoretical
saturation [59]. We also performed constant memoing while coding the data.
GT has two main different variations, Glaserian (aka. classic) [19] and Straus-
sian [59]. These approaches differ in their steps and principles (for a comparison
please see [58]). In this study, we picked Straussian version and followed Strauss
and Corbin’s guidelines for performing GT [59] including the following steps
(Figure 1):
•open coding is the step in which the concepts are identified in the data.
For this purpose, the raw data (e.g., interview transcripts, or observation
notes) was broken down into manageable analytical pieces. These pieces
then were openly tagged with codes (i.e., concepts and categories) by iden-
tifying key points represented in the segment which included events and
their enabling or prohibiting influence on integration. Our date resulted
in 143 codes in this step.
•axial coding is the process of relating categories to their subcategories. In
this step, the generated codes were related to each other via a combina-
tion of inductive and deductive thinking to form the main categories (i.e.
themes and sub-themes) of events and their enabling or prohibiting role
in the integration process. The main categories, however, merely describe
the data and need to be further developed into a theory. Our data resulted
in 27 categories in total which resulted in forming three main themes. The
themes that emerged from our data concern UX practices and responsibili-
ties, internal and external stakeholders’ beliefs and attitude about software
quality and UX, and the company’s business model and strategy and how
UX integration was perceived in relation to that. The sub-themes concern,
for instance, managing UX practices, the role of politics in integration, the
role of power-relations in integration, the relation of UX to GUI design
and development, the importance of software quality in general and UX in
particular, the importance of the end user needs’ compared to the needs of
customers, the relation of integration to value delivery, and UX being part
of the business model and strategy of the organization. Further analysis
of the identified categories and their impact on integration over the years
showed that they spread over at least four noticeable time periods that re-
flect the paradigm shifts in the organization in particular in relation to the
main three identified themes. These periods, in fact, are our interpretation
of the company’s transition from GUI development to integrating usabil-
ity and UX. A transition which was a response to various events and their
enabling or prohibiting roles.
•selective coding is when the core category is selected from the main cat-
egories identified in axial coding. The theory is refined and developed
12
through linking the identified categories around the core category. We
realized that we have rich data on organizational change aspect of in-
tegration hence selected this aspect as our main focus to further analyze
UX integration in the case company which resulted in generating 4 lessons
learned and 5 pitfalls.
In presenting our results, we structure them based on the four identified
time periods instead of the identified categories. In other words, we use a
narrative of events to show how UX integration emerged over time, situated in
its context, i.e. the organization. We present these events and their influence
from the perspective of practitioners. Our main motivation was to present a
coherent story of the changes in the organization, present various event, and
describe their enabling or prohibiting roles through this story. In our view, such
a presentation, can better show the company’s journey over time which was
one of the main aspects of our analysis. Therefore, we discuss the interrelation
among the identified events as well as between events and the organization and
show how they may have influenced UX integration positively or negatively.
However, we do not argue the relative importance of these events or the extent
of their impact on the integration, rather focus on whether, in general, they
served as an enabler or prohibiter to integration efforts. Also, the aim of these
periods is not to present an exact time box of the identified events, rather to
show the paradigm shift in the company over the years.
In this study, we did not draw cause and effect relationships from the data.
We rather identified indications of improvement or worsening of UX integration
mainly based on the participants’ opinions and our document analysis of the UX
artifacts, i.e. tangible outputs of UX practices in projects, e.g. ‘UX guideline’.
In addition, we do not provide a full coverage of all the events across the
organization in the identified periods. In our analysis, we coded and catego-
rized the events that based on our data have had an interesting impact on UX
integration within the case company. Clearly, in earlier years, the term UX
or even usability were not used and these concepts to a large extent were not
yet adopted in the industry. But since many of the events in those early years
became the basis of UX integration in later years, we collectively refer to them
as influencing events. Still, to better reflect the company’s history, we have
explicitly distinguished between various terminologies used in different periods.
4. Results
In this section, we present the four identified time periods or paradigm shifts
(summarized in Table 3) and their including events. We found that some of the
identified events happened outside the organization while some were internal
to it. In the following sub-sections (i.e. time periods), we have presented the
internal events according to their chronological orders. However, as our data
gathering was done inside the organization, we did not have enough data to
extract a reliable chronological order for the external events. These events are,
13
Period Main paradigm shift
Period
one
investing in creating Graphical User Interfaces (GUIs) as opposed to command-
line interaction
Period
two
initiating usability integration that focused on improving the developed GUI
through enhancing its usability. For this purpose, principles and practices of
usability were introduced to the organization and started to be integrated to the
development processes ( mainly through applying user-centered design (UCD))
Period
three
initiating UX integration that focused on enhancing the experience delivered
through the developed products. For this purpose, in addition to principles and
practices of UCD (with focus on usability), UX principles and practices were
introduced to the organization and started to be integrated to the development
processes. UX integration covers usability integration as well
Period
four
improving UX integration that focused on enabling a better UX integration in
the organization through new initiatives and applying various practices
Table 3: We identified at least four periods or paradigm shifts in the organization that concern
GUI design and development, usability, and UX. Some of the events we identified span across
these periods and their presence or influence on integration was observed in more than one
period. These periods, therefore, shall be seen as paradigm shifts in the organization rather
than exclusive time boxes of events
therefore, presented and discussed in relation to the influence they have had on
the internal events.
The identified events have a complex and multifaceted relation with the
main themes and sub-themes that emerged from our data. For instance, hiring
a manager with HCI background in the company could be associated to changes
in business model and strategy and a positive attitude towards software quality
and usability in that time. It also directly related to usability-related prac-
tices and responsibilities and also contributed to even more positive attitudes
towards usability in the organization. Although we acknowledge this complex
relation between the events and the themes, for each internal event (Table 4,
Table 6, Table 8, and Table 10), we have listed the theme that in our view had
the strongest connection to the event. We believe such a categorization when
summarizing the events can help the reader better understand these events and
their relation to integration from various aspects.
However, for the external events, such a categorization was not possible as
each external event could have influenced various internal events, hence, influ-
enced the integration concerning more than one theme. Therefore, we used,
other categories that emerged from our data with regards to the external events
which concerned the nature of these events and not their influence on the or-
ganization: We identified at least three main such categories: (i) technological
advances (e.g. introduction of mouse as an interaction medium), (ii) increas-
ing knowledge and awareness (the concept of UX becoming widespread in the
field of software development), and (iii) educational advances (university pro-
grams that covered UX). Please note that these external events, their order,
and influence on the organization is presented as perceived by our participants.
4.1. Period One: Focusing on Graphical User Interfaces
In this period which mainly happened between the years of 1993 to 2002,
the company started investing in creating Graphical User Interfaces (GUIs) in
response to the following changes and events. The main events that happened
14
inside the organization are summarized in Table 4 while Table 5 summarizes the
main events that happened outside the organization yet influenced the internal
events and integration efforts.
The interviewees highlighted that various events external to the company
impacted how internal and external stakeholders expected a software system
to look like and behave. The interviews for instance mentioned introduction
and plurality of mouse as an interaction medium, graphical user interfaces, and
desktop and home computers. Regarding the advances in the field, a developer
stated:
“Anybody now knows what a desktop is, but when we started in 1990, nobody knew.
It didn’t exist as a common metaphor. Our first manual had a picture saying, ‘Here’s
what a mouse looks like. Here is what you use the buttons for”’ (p8, developer).
This interviewee further emphasized:
“if you design a user interface today, you have to imagine that users are not going
to click on a mouse, they’re going to point to the screen and drag their fingers, and
they expect things to work this way. Which they didn’t do 20 to 30 years ago. So,
UX design also means that you have to conform to what industry standard is today”
(p8, developer).
During these years, the company targeted new market areas, i.e. applica-
tion domains, that needed more interactive products. Second, more and more
developers were hired and the company grew in size. In addition, the company
was mainly hiring practitioners with competences in developing core functions,
and algorithms (e.g. people with backgrounds in computer science and math-
ematics). This hiring policy was mainly motivated by the business model and
strategy of the company which focused on algorithms and offering unique core
functions to the customers. Consequently, an even stronger engineering culture
in the case company formed over these years. In later years, this historical fo-
cus on core functions contributed to more resistance to the concept of UX and
related practices.
Still, in response to the new expectations rising in the market, and to bene-
fit from the new technologies, the company consciously planned to improve the
user interfaces of the product to enhance users’ access to the core functional ca-
pabilities. Therefore, besides core developers, developers with GUI design and
development skills were also hired during these years. However, the number
of such developers was much less than those with core development compe-
tencies. Even practitioners hired for GUI development had technical education
and backgrounds (e.g. computer science) and not Human-Computer Interaction
(HCI). Similarly, the company hired much less number of developers who had a
business perspective and understood the importance of business goals and their
relations to functions and GUI. Regarding this a product manager emphasized:
“people we hired as developers hadn’t had the opportunity to be on the customer
side, to understand the business value of the system and how they work as a whole
and also meet with the users” (p1, product manager).
15
Category Internal Events
Business model & strategy the company entered new market areas
where customers required more interac-
tive software
Business model & strategy the company grew in size fast
Business model & strategy more practitioners with technical skills
were hired
Business mo del & strategy the company’s business strategy focused
on functions
Beliefs & attitudes an engineering culture formed in the
company
Practices & responsibilities GUI designers & developers with graph-
ical design skills were hired
Beliefs & attitudes developers emphasized separating the
GUI layer from business logic
Beliefs & attitudes developers emphasized making the GUI
configurable
Beliefs & attitudes developers emphasized enhancing fonts
& colors on the GUI
Table 4: The main internal events in period one: Focusing on Graphical User Interfaces. To
help the reader, these events are listed in their chronological order and in the same order as
narrated in the summary of the corresponding period.
The main focus of the hired GUI developers was to separate the GUI layer
from the business logic, and to make it as configurable as possible, which was
expected considering their competences and background. These developers,
having limited design mindsets, believed that usability or UX is something that
can be achieved through configuring fonts and colors by users after deploying
the software. For instance, one of the developers explained how she approached
improving the design as:
“the user interface is programmable. You can choose colors, you can choose sizes
and, you know, fonts and things. And for me that is good enough for the user, they
can choose themselves what they want [later on]” (p8, developer).
The company took pride in the fact that their products supported mouse
interaction, or that it even offered any GUI while similar products in the mar-
ket offered only command-line interaction. As the interviewees stated, GUI and
interaction through mouse were still uncommon and ‘unexpected’ for the cus-
tomers and users; therefore easily created positive experiences for them. This
can explain why still many practitioners in the company emphasize they have
always been working with UX, for instance:
“I wonder if anyone talked about UX design. But we were ahead of times. We were
using pointing devices before they even called it mouse...We realized early that we
need to work a little bit with GUIs” (p7, product owner).
4.2. Period Two: Initiating Usability Integration
This period mainly happened during the years of 2003 to 2006 when usability
integration was initiated in the company as the following events intertwined with
the events in period one. Here, by usability integration we mean introducing
16
Category External event Example internal event
Technological
advances
new interaction medium,
mouse, became widespread
the software was designed for mouse in-
teractions which created more possibili-
ties for better UX design
Technological
advances
GUI mostly replaced
command-line interactions
stakeholders became motivated to de-
velop a software system with GUI
Technological
advances
desktop & home computers
became widespread
customers & users experienced vari-
ous software systems & expected better
UX hence internal stakeholders became
more motivated to enhance UX
Increasing Knowl-
edge & awareness
customers & users became fa-
miliar with Windows operat-
ing system
customers & users had higher expecta-
tions about GUIs hence internal stake-
holders became more motivated to en-
hance UX
Table 5: The main external events influences of which were first noticed in the organization in
period one and contributed to a focus on Graphical User Interfaces (GUIs). External events
indirectly influence integration as they often lead to other events inside the organization;
examples of such internal events are listed in the last column. These external events are listed
in the same order as the internal events they influence.
usability principles and practices to the organization and integrating them to
the development processes. This differs from period one in which the focus was
on developing Graphical User Interfaces (GUI) without necessarily taking their
usability into account.
The main events that happened inside the organization are summarized in
Table 6 while Table 7 summarizes the main events that happened outside the
organization yet influenced the internal events and integration efforts.
In this period, management and other internal stakeholders (e.g. developers,
sales and marketing) realized that customers paid more attention than before
to end users’ opinions and views. The customers’ increasing knowledge and
awareness about the importance of users’ opinions and views motivated the
internal stakeholders to take usability into consideration. Regarding this an
interviewee stated:
“I was working with sales a lot. [for instance] I demonstrated the system for 30
end-users. There was one manager and the manager was the one who made the end
decision at the end of the day but it was pretty obvious that if 30 end-users didn’t
like the system, she was going to have a very tough time selling that system to them.
That was one of the drivers for understanding that end-users’ opinion matters” (p6,
manager).
With the advances in web and mobile technologies, the concept of UX was
introduced in the field of software development. Accordingly, a marketing cam-
paign was started in the company that advertised the products as ‘a product
that provides one coherent experience for the users’. The marketing and sale
units had realized that a unified experience is appealing to the customers and
users and can contribute to more sells and profit. However, in practice, this was
not still the case and even the GUIs were not yet unified.
During these years, management had realized that considering the advances
in the field, only providing core functions was not sufficient for the success of
the product. In their view, at least a minimum level of usability in the GUIs
17
was required in order to better succeed in the market as the customers and users
paid more attention also to look and feel of software systems. Regarding this a
developer stated:
“So from a selling perspective, I think, our manager was right saying that we have
to have a good look and feel, that everybody wants a good look and feel, and our
system looks a bit old and clunky” (p8, developer).
Therefore, a GUI design forum was established where various stakeholders
in both development and business units would discuss the design of the GUI.
Consequently, a set of guidelines were generated for developers in the GUI de-
velopment teams.
In these years, the company had more access to practitioners with usability
and interaction design skills in the job market. This was expected as university
programs that focused on usability and interaction design became widespread.
Accordingly, a director of Research and Development (R&D), with a background
in Human-Computer Interaction (HCI), was hired specifically to initiate and co-
ordinate the activities required to improve the usability of the products. For
instance, through facilitating end users involvement in the process and improv-
ing interaction design of the product. It was also agreed to follow Microsoft
design guidelines in developing the GUIs. Many of the changes made to the
GUI in this period became the base of the UX design of the products in later
years.
The size of the company was now growing even more and more develop-
ers with different expertise were hired. Hence, specialized teams, units, and
departments were formed to better structure and organize the responsibilities
of these developers. For instance, a unit specialized in GUI development was
established under R&D department in 2005. In later years, this unit became
the source of many UX integration initiatives. Increasing size of the company
also led to an increasing gap between developers and customers and end users.
Since the number of developers was increasing, it was no longer possible for all
developers to meet the customers’ sites and have direct contact with the end
users. Regarding this a requirement analyst/developer said:
“when customers were here you actual ly got started to talk and therefore we ex-
changed phone numbers and mail addresses and when you started developing you
actually got feedback this way” (p13, requirements/developer).
Hence, practitioners felt a need for more formal ways of gathering informa-
tion about the business scenarios and the end users’ preferences. In this period,
therefore, practitioners initiated applying a number of UCD practices, includ-
ing user studies and user evaluations. This initiative was supported by various
educational material on the topic of usability that was now available in the field
of software development. Regarding this, a product manager stated:
“Before 2006 or earlier than that, everyone was involved in implementing our sys-
tems on the customer side, everyone... During your career, you basically moved
from the implementation side [where you had a chance to] meet the customer to be
a more back-end developer, just developing core functions... so there was a need to
have more formal practices” (p1, product manager).
18
Therefore, following UCD practices, user studies were performed in form of
interviews and observations at the users’ workplace (for different customers the
organization had at the time). The result of the user studies was presented in
form of unstructured text documents and included various observations, design
flaws, and design improvement proposals.
The increasing size of the company also negatively impacted the quality of
software in general. This could be explained by the hiring policies that, as we
described before, focused on programming skills. Regarding this, a developer
stated:
“things were easier when we were fewer people. As more and more people came in,
the [quality] bar was lowered because the new people didn’t bring a UX mentality into
the team. The general mentality is to just make it work and not to make it better
UX-wise. I don’t mean that they don’t want to, it’s just that people don’t know how
to, or don’t think about it automatically. That’s not only a problem with UX; we
have that with software quality in general” (p2, developer).
Since the business units believed developers needed to better understand the
business rules and how the system is being used in real situations, a previous
end user was hired as a business analyst. The role of this person was to be in
close contact with the developers to discuss design ideas. This role, however,
did not have a large impact on design, until the process changed to agile as
described in the section on period four.
Another major event in this period was the acquisition of the organization by
a large international company. Although in the beginning, day-to-day work of
developers did not change much because of this event, in the later years, this led
to other events; for instance, following new development processes that the ac-
quiring company demanded (e.g. Rational Unified Process (RUP)1that mainly
happened in period three and transformation to agile that mainly happened in
period four as we have described in the following sections).
4.3. Period Three: Initiating UX Integration
This period mainly happened during the years of 2007 to 2012 when UX
integration was initiated in the company as the following events intertwined
with the events in the two previous periods. The events that happened inside
the organization are summarized in Table 8 while Table 9 summarizes the events
that happened outside the organization yet influenced the internal events or
integration efforts.
During this period, the term UX and its associated principles and practices
became popular in the software industry. As university programs with a focus
on UX became widespread, there were more practitioners in the job market who
either already were educated on UX practices and principles or were interested
in acquiring UX-related competencies. Younger generations of GUI designers
1Rational Unified Process (RUP) is an iterative software development process framework
which insists that architecture sit at the heart of the project team’s efforts to shape the
system [35]
19
Category Internal Events
Beliefs & attitudes awareness about the importance of the end users’ view in-
creased in the company
Business model & strategy the marketing unit used UX as one of the selling points of
the company’s products in their advertisements
Beliefs & attitudes management’s awareness about importance of interaction
design & usability increased
Practices & responsibilities a GUI design forum was established in the company
Practices & responsibilities design guidelines were created in the company
Business model & strategy a manager with HCI background was hired
Practices & responsibilities specialized units in the organization were formed (e.g. core
development & GUI development)
Practices & responsibilities the gap between developers & end users increased
Practices & responsibilities various user-centered design practices were applied in the
company
Business model & strategy a former end-user was hired as business analyst
Business model & strategy a GUI and visualization specialist was hired
Business model & strategy the company was acquired by a large international company
Table 6: The main internal influencing events in period two: Initiating Usability Integration.
To help the reader, these events are listed in their chronological order and in the same order
as narrated in the summary of the corresponding period.
Category External Event Example Influence
Increasing knowl-
edge & awareness
customers paid attention to
end users’ opinions
internal stakeholders became motivated
about usability integration
Technological
advances
mobile & web platforms be-
came widespread
practitioners had the opportunity to de-
liver different experiences through dif-
ferent platforms
Theoretical ad-
vances
the concept of UX influenced
software industry
internal stakeholders realized the im-
portance of experience, as well as func-
tions, in attracting customers
Increasing knowl-
edge & awareness
customers & users paid atten-
tion to look & feel
internal stakeholders became motivated
to invest on look & feel became
Increasing knowl-
edge & awareness
practitioners with usability
skills became available in the
job market
practitioners who joined the company
were more likely to advocate or engage
in emergent usability initiatives
Educational ad-
vances
university programs with in-
teraction design & usability
focus became widespread
the company had more access to prac-
titioners with usability knowledge who
engaged in emergent usability initiatives
Educational ad-
vances
educational resources on
usability (e.g.books) became
widespread
internal stakeholders had more possibil-
ity to improve their usability skills
Table 7: The main external events influences of which were first noticed in the organization
in period two and contributed to initiating usability integration. External events indirectly
influence integration as they often lead to other events inside the organization; examples of
such internal events are listed in the last column. These external events are listed in the same
order as the internal events they influence.
20
and developers included graduates from HCI programs with knowledge on UX.
During this period, the company also started following Rational Unified Process
(RUP) as their development process. In addition, customers became informed
about the concept of UX and its importance.
In this period, internal knowledge and awareness about usability and prag-
matic aspects of UX increased to a great extent. The term UX was used for
the first time in the company although in practice the day-to-day work was
still mainly limited to practices that addressed pragmatic aspects of UX, i.e.
usability practices and principles.
One of the developers that was hired in this period to join the GUI teams
possessed an interaction design degree and knowledge about UX. Regarding the
role of this developer in raising awareness in the company an interviewee stated:
“She was interested in UX. She didn’t work with me but I noticed that she sort of
tried to pursue UX. Like ‘This is something that we need to focus on.’ At least it
was about then that we started at least to think that there might be some other ways
to solve the problems. For instance, if there’s a common thing that the user does
ten times more often than other things, maybe it should be easier to access” (p2,
developer).
The hired developer together with a number of other developers who got
interested in UX became UX advocates in the company. These advocates played
a major role in starting the bottom-up initiatives that contributed to increasing
internal knowledge and awareness about UX and impacted UX integration in the
upcoming years. However, this was mainly motivated by personal interest than
responsibilities assigned by management. These UX advocates established a UX
study group to learn more about UX and usability practices and principles (e.g.,
heuristic evaluation, personas, user studies, storyboards, and usability testing).
To exercise these practices, UX advocates ran a pilot project inside the com-
pany. They realized that these practices seemed simple but it took time to
master them. Moreover, they concluded that heuristic evaluations and story-
boards were easier to perform than user research and usability testing. They,
therefore, started applying the former practices in their day-to-day work, only
in a limited number of teams and not widespread across the organization. Fol-
lowing the study group, a UX interest group was established where interested
developers across the organization could participate to exchange knowledge and
ideas about UX, usability and related principles and practices, or UX related
issues in the ongoing projects.
UX advocates also strove to increase other practitioners’ knowledge and
awareness about UX. They had already realized that limited knowledge and
awareness about UX is a big challenge to a better UX integration in the company.
Hence, in order to improve UX integration, they decided to raise the awareness
especially among project managers and even customers. For instance, wiki
pages were set up that included descriptions of various UX tools and methods.
UX advocates also continuously strove to get a mandate from management to
perform more UX practices in projects.
The reports from user studies performed in this period varied in structure
and quality. While some were free text including random observations and
21
thoughts, others were more structured including specific sections about types
of users, context, work scenarios, example scenarios, etc. One explanation can
be that practitioners did not yet have an agreed-upon process, tools, methods,
or template to use when performing or reporting the result of the user studies.
Hence, these results were dependent on the experience and knowledge of the
practitioners who performed them. Another explanation may be that these
practices were performed by developers who were self-taught and not formally
educated in HCI, usability or UX.
In addition, UX advocates performed a heuristic evaluation of the product
following Nilsen’s ten heuristics. Storyboarding was another practice that be-
came popular in the organization in this period. Not only the GUI development
practitioners but also the business unit started using mock-ups and storyboards
as a means of communicating requirements and design ideas. Balsamic mock-ups
and storyboards were used during this period and even in later years by product
management to communicate their design ideas to customers and developers,
also by requirements analysts to communicate with users and customers.
In this period, the company’s software development process changed from
waterfall to a customized version of Rational Unified Process (RUP). This
change was initiated by higher management and led to getting more mandate for
UX practices since a number of these practices were explicitly considered as part
of this process. These practices mainly addressed the pragmatic aspect of UX,
and included: Stakeholder Analysis, User & Domain Analysis, UX Design Vi-
sion, Form & Behavior Specification, and Usability Evaluation Plan & Report.
They also applied practices such as User & Domain Analysis, Storyboarding,
Heuristic Evaluation, and Usability Evaluation. Despite these changes, practi-
tioners from the business unit (including product management and requirement
analysts) were still the gateway to customers and the end users. This was prob-
lematic for UX advocates since this unit still resisted UX practices emphasizing
that UX is less important than functions. Another problem was that various
internal stakeholders still did not differentiate customers and end users.
Following the increased internal knowledge and awareness, also the process
change to RUP, the UX advocates received new job titles: UX designer. An
interviewee emphasized these practitioners received the title mainly because
they were more interested in UX and were involved in developing GUIs:
“Because they were working with the user interface. There was a lot of people
working with that, but they maybe were the most interested people, and actually cared
more about it. So they were the ones who were mentioned as the UX designers” (p5,
product owner).
Despite the title changes, the everyday work of these practitioners did not
change much and was still mainly developing GUI:
“we were sitting in some kind of meeting and some people started saying that we have
some UX designers, and first I was asking, ‘What is UX design?’ Then when people
explained it to me I said, ‘Wow! Do we have that?’ Until I realized it was some of
our developers we gave another title. At that point, it wasn’t so much different in
the way we worked, but the way people started talking about it was different because
someday we’d cal l some people UX designers” (p5, product owner).
22
In addition, the responsibilities of UX designers were not well defined and
still largely overlapped the work of requirements analysts. Moreover, it was not
clear how these roles should collaborate with each other. Another problem was
the limited UX knowledge and competences in the company:
“cause you have creative very good front-end developers so they want to get involved
in UX work, but if they do not have the needed UX knowledge then you will not
have a better UX either” (p11, UX expert).
In this period, customers and users expected good UX in the products.
Customers even asked for specific roles in the projects, e.g. UX lead, to ensure
supporting experience of the users. Consequently, in addition to the new role of
UX designer, two other roles were defined in this period: general UX lead and
project UX lead. General UX lead was responsible to present and negotiate the
scenarios and initial storyboards with customers to get feedback. Project UX
lead had a similar responsibility in sub-projects:
“The focus of this role was primarily to coordinate the UX aspect of various sub-
projects running towards customer X, focusing a lot on look-and-feel, being the con-
tact person, aligning the design in sub-projects. She was also the speaking partner
towards the product development for discussing UX guidelines” (p19, manager).
The UX lead roles, however, were not well received by the UX advocates
mainly because of disagreements on the way of working. Regarding this, one of
these advocates stated:
“the UX lead was more or less a person that follows the rules, I mean if there are 10
steps that you should make she will make all the 10 steps. So in RUP, for instance,
she wanted to produce all these artifacts even if they were not needed, she did not
have a very ‘lean’ mindset. So it was a little bit of clash in our ways of working”
(p9, UX guild, scrum master).
Another main reason was that these UX advocates felt ‘left out’ after the
role of UX lead was introduced:
“we had worked with UX for many years. But the first thing she wanted to do was
performing a user study, and that’s Ok but she actually book it and went there alone.
You should be 2-3 people to go there and observe the users. But she thought since
she was the lead UX person, she should do it alone. And then she has the knowledge
and she should tell us how to design. But we have been part of UX work for many
years, we know how it works and it’s not new to us. I mean, you don’t have to
reinvent the wheel” (p10, UX guild, developer).
4.4. Period Four: Improving UX Integration
This period mainly happened during the years of 2012 to 2015. Efforts to
improve UX integration continued in the company during this period as the
following events intertwined with the events in the three previous periods. The
events that happened inside the organization are summarized in Table 10 while
Table 11 summarizes the events that happened outside the organization yet
influenced the internal events or integration efforts.
According to our data, the hiring policy in the case company still emphasizes
technical and programming skills. Regarding this, a developer told us:
23
Category Internal Events
Beliefs & attitudes the concept of UX become more known in the company
Business model & strategy a GUI designer with UX knowledge was hired
Beliefs & attitudes number of UX advocates in the company increased
Beliefs & attitudes a UX study group was established in the company
Practices & responsibilities UX advocated run a pilot project to learn more ab out UCD
Beliefs & attitudes a UX interest group was established in the company
Practices & responsibilities Wiki pages for sharing UX-related information were created
Practices & responsibilities user studies & evaluations were performed as part of the
development processes
Practices & responsibilities Rational Unified Process (RUP) was introduced in the com-
pany
Practices & responsibilities UX advocates received more mandate for UX practices &
responsibilities
Practices & responsibilities UX advocates received the job title “UX designer”
Practices & responsibilities the role of “UX lead” was introduced in the company
Table 8: The main internal events in period three: Initiating UX Integration. To help the
reader, these events are listed in their chronological order and in the same order as narrated
in the summary of the corresponding period.
Category External event Example internal event
Increasing knowl-
edge & awareness
UX became widespread in
software industry & in the
market
internal & external stakeholders became
familiar with the concept & benefits of
UX
Educational ad-
vances
university programs with a fo-
cus on UX became widespread
the company hired practitioners with
UX knowledge who advocated or en-
gaged in emergent UX integration ini-
tiatives
Increasing knowl-
edge & awareness
there were more practitioners
in the job market who were
educated/interested in UX
company hired practitioners with UX
competencies
Increasing knowl-
edge & awareness
users & customers expected
good UX
internal stakeholders in particular man-
agement became more motivated about
UX integration
Increasing knowl-
edge & awareness
customers asked for UX-
related roles in projects
internal stakeholders in particular man-
agement became motivated to enhance
UX integration
Table 9: The main external events influences of which were first noticed in the organization in
period three and contributed to initiating UX integration. External events indirectly influence
integration as they often lead to other events inside the organization; examples of such internal
events are listed in the last column. These external events are listed in the same order as the
internal events they influence.
24
“When people are hired, they’re not hired because they know how to design good user
interfaces. They’re rather hired because they know data structures and how to write
code” (p2, developer).
However, according to the UX expert, the company could benefit from more
front-end developers since its product offered a variety of graphical user inter-
faces (GUI):
“it is a front-end product so it needs more front-end people. I think we would benefit
from more front-end developers because it is a lot of back-end people in there” (p11,
UX expert).
In this period, practitioners had access to variety of UX educational material,
to learn how to improve UX integration in the company. For instance, in the
UX study group though a book on Lean UX [21] the advocates learned more
about how UX practices can create revenue with fewer costs. Nevertheless, these
practitioners did not still get the mandate to operationalize the process of Lean
UX in their work.
In this period, in response to the popularity of agile processes in the soft-
ware industry, the organization became agile. Consequently, feature teams,
guilds, and chapters were created in this period. Each feature team was a
cross-functional team, consisting of practitioners with different competencies
(e.g. testing, architecture, UX). Each team was responsible for designing and
developing a collection of features from core functions to GUI. Since competen-
cies were spread across feature teams, there was a need for alignment. Guilds
and chapters are organizational entities that supported alignment of various ar-
eas of competency such as testing, architecture, and UX. The main differences
between a guild and chapter were that: (i) having a representative in a guild
was not mandatory for all teams and projects while joining a chapter was, (ii)
while chapters had decision power and authority, a guild did not have deci-
sion making power and decisions made in the guild, e.g. UX guidelines, were
‘recommendations’ that other practitioners could choose to follow or not.
During this transformation, UX advocates struggled to form a UX chapter
instead of a UX guild. However, higher management, product management, and
product owners did not support this idea and the UX study group was turned
into a guild and not a chapter. One main reason why a UX chapter was not
formed was a lack of competences to lead such a unit. Regarding this a manager
stated:
“In order to have a UX chapter, there needs to be a person to lead that. When I look
internally at some skills required for that, I don’t think that we have the right person
that I would feel comfortable leading the chapter, to be honest. I think we have some
talented UI developers that also care a lot about UX but they don’t necessarily have
the leadership skills or negotiation skills required” (p6, manager).
In this period, the UX advocates in the company continued performing
heuristic evaluations and creating storyboards for different features. They also
performed more user observations and kept the UX guidelines up to date. These
guidelines, however, were not followed in all the teams, but only those who had
25
a representative in the guild. UX practices were also mainly performed in those
teams where a UX advocate was present.
Scrum methodology was selected as the agile methodology to follow. Con-
sequently, a UX backlog was created in JIRA2, the agile project management
tool the company used. The aim was to explicitly include UX design issues in
agile iterations. This was however not well received by product owners since
in their view discussions about UX issues often were not constructive and no
decision could be made because of the subjectivity of the matter. Regarding
this a product manager said:
“Some of the processes which we decided upon to use actually made the develop-
ment process so very very long because no one could take a decision” (p1, product
manager).
This led to a frustration also among the UX guild members because as they
stressed, even though they performed evaluations, UX issues did not get priority
in the backlog and were left mostly unattended. Regarding this one of the UX
advocates said:
“we have just given up because it is no use in doing user testing if you don’t care
about the results. Because once we finish the user testing we already have burned
all the money for the feature, so you don’t get the time and money to go back and
fix it. we don’t do this iterative development, we just do small waterfalls” (p9, UX
guild member/scrum master).
Another consequence of transforming to agile processes was the change of
power relations between developers and product owners. Product owners now
had more power to make decisions about what features to develop and in what
way. However, this change was not in favor of UX since these product owners
did not have the necessary UX mindset or did not even value UX. Regarding
this, a manager stated:
“at the end of the day we’re trying to be more and more agile, and we work a lot with
product owners. Product owners are the ones who would accept or decline a certain
function and you need to get UX awareness as well to the level where product owner
will be ready to decline a function because it doesn’t fulfil l the UX aspects in a good
way; we’re far away from there...[today] that judgment or that assessment will be
based on their own personal experience not being trained or educated in UX. A lot
of that will be shaped by what they’re used to seeing and what they’re used to seeing
is our old products. So, we need to raise awareness not only within development
teams but also on the business side” (p6, manager).
As this interviewee emphasized that to improve UX practices in agile, there
is a need to increase knowledge and awareness also among product owners and
product managers because these roles are calling the shots regarding design
decisions.
Comparing the current situation to earlier years when developers had more
power in making such decisions, a product owner told us:
2https://atlassian.com
26
“Before 2010, our business expert, a former end user, had so many discussions about
UX with developers, and their response was like, ‘Okay, now we heard what you
said,’ Then they still implemented what they had in their own head anyway, because
it was real ly the power of the developers at that point. And that was something which
changed as the product owner role came in because now we own the requirement and
now product owners are the ones signing off that” (p4, product owner).
However, product owner authority was not necessarily in favor of UX:
“the product owner shouldn’t decide exactly how it should be done. She should decide
what the thing should do when it is done, not exactly how the buttons should be laid
out or how it’s supposed to be implemented” (p2, developer).
During this period, practitioners with UX-related skills became available and
accessible in the job market. In addition, differences between UX and usability
gained the attention of various internal and external stakeholders. Consequently,
as an effort to improve UX practices, an interaction designer was hired by man-
agement to join the internal UX advocates to improve UX delivered through the
products. Management was looking for an expert who not only had knowledge
and experience on UX practices and principles but also could balance them with
technical feasibility. Management was also looking for someone who could pro-
vide leadership and direction for all aspects of the UX vision and strategy. Most
importantly, the UX expert was expected to collaborate with a wide variety of
different roles including customers, product managers, and developers. The UX
expert was hired as part of the architecture chapter to contribute to one of the
missions of the chapter: improving quality of the products.
Hiring a UX expert, however, was not fully successful for at least two main
reasons. First, the expectations from this role were unrealistic. A product owner
for instance expressed:
“I thought more that there are some basic things that you know if you are an inter-
action designer, and it felt like there was nothing like that. It was really depending
on understanding the end users, and so on. I was a little bit disappointed... I
know when we talk about UX, we’re talking about seeing how end users work and
everything ...[so] just forget about bringing someone in, because how would they then
know if they didn’t work in this domain before, because then it would take such a
long time to actually get them up and running, and really provide some value” (p4,
product owner).
Second, the context in which the UX expert could effectively apply her com-
petences did not still exist in the organization. For instance, she did not have the
opportunity to collaborate enough with the relevant product owners to discuss
design ideas mainly because no specific time was assigned to this collaboration.
In this period, variety of tools & methods to address UX were available
to the software development practitioners. For instance, the UX expert intro-
duced new practices, tools, and methods including business impact map, UX
wave lines, proto-personas. These tools and methods were clearly closer to UX
than usability, however still mostly addressing the pragmatic aspect of UX. The
personas, for instance, were different from the existing ones in that they included
not only the work goal but also more personal goals of users: e.g. I want to
27
make my staff happy. Still, the majority of the needs or goals included in these
personas were still pragmatic: e.g. manual override, visible calculations. The
UX expert also aimed to educate not only the UX advocates but also product
owners and product management.
One major problem that the UX expert felt during her work at the company
was lack of collaboration with product owners and product management and
that her role was not agreed upon and communicated to other stakeholders
in the company. For instance, the role was going to be hired as part of the
development unit while the business unit believed they do not have a need for
such a role yet, and if they did, it should have been the business unit hiring a UX
expert and owning UX practices in the company. Similarly, the UX advocates
(i.e. UX guild members) felt uninformed about how such a role was going to
collaborate with them. Mobile platforms were more popular and widespread in
the industry during this period. The members of the business unit believed UX
was more the focus of mobile and web products and in fact part of the business
strategy to attract more end users (and customers). They, therefore, did not
see a need for investing on UX for the desktop products. On the contrary, the
development unit believed UX was also important for the desktop as a part of
quality improvement efforts.
The newly hired UX expert felt similar problems in her work as the previous
interaction design consultants that were hired in previous periods of time. She
found herself in a context that was not yet ready for the practices she could of-
fer. She introduced more UX practices and principles but they did not become
widespread and were resisted mainly by product management and product own-
ers. In general, product managers and product owners believed UX practices
and processes often were time-consuming mainly because it was hard to agree
on them or their outcomes. Regarding this a product manager said:
”We have tried different processes I think since 2009 but one thing which always
failed is that we didn’t define who has the final say, who can actually decide what to
do and how. I think we still have different views here in the organization, unfortu-
nately” (p1, product manager).
Similarly, a requirement analyst expressed that negative previous experiences
had caused frustration in the company hence more resistance to UX integration.
He emphasized:
“many people here like UX design but are tired of all these discussions” (p13, re-
quirements/developer).
Another major event during this period was that the company started devel-
oping an off-the-shelf version of its product. Product management and product
owners, therefore, became more keen on features and design ideas that would
mainly benefit this product than specific customers. As one product owner
emphasized:
“the core is going to be given to a lot of other customers” (p4, product owner)
28
Category Internal Events
Practices & responsibilities the company started using agile processes
Beliefs & attitudes UX advocates learned about lean UX in the UX study group
Practices & responsibilities guild & chapters (including the UX guild) were created in
the company as part of agile transformation
Practices & responsibilities feature teams were created as part of agile transformation
Practices & responsibilities UX backlog was created as part of agile transformation
Business model & strategy a UX expert was hired
Practices & responsibilities UX-specific tools & methods were applied
Practices & responsibilities UX guidelines were created
Business mode & strategy an off-the-shelf product was created
Table 10: The main internal events in period four: Improving UX Integration. To help the
reader, these events are listed in their chronological order and in the same order as narrated
in the summary of the corresponding period.
Category External Events Example Internal Event
Educational ad-
vances
educational resources on
UX (e.g. books) became
widespread
internal stakeholders could enhance
their UX knowledge
Theoretical ad-
vances
agile processes became
widespread in the industry
the company applied an agile develop-
ment process
Increasing knowl-
edge & awareness
practitioners with UX-related
skills became more available
in the job market
the company could hire practitioners for
UX-specific roles
Increasing knowl-
edge & awareness
differences between UX & us-
ability gained the attention of
various stakeholders
practitioners advocated addressing UX
as well as usability
Theoretical ad-
vances
industry relevant UX tools &
methods were introduced
practitioners had access to variety of
tools & methods to address UX in their
daily work
Technological
advances
mobile platforms (e.g. iPhone
and iPad) became widespread
experiences with mobile applications
became part of daily life of end users &
provided more marketing opportunities
for the company
Table 11: The main external events influences of which were first noticed in period four and
contributed to improving UX Integration. External events indirectly influence integration as
they often lead to other events inside the organization; examples of such internal events are
listed in the last column. These external events are listed in the same order as the internal
events they influence.
Therefore the product owners would not necessarily listen to users’ wishes if
that could impact the core features or design of the off-the-shelf product.
We finalized our research collaboration with the company in 2015 when the
efforts to improve UX integration were still ongoing, and a satisfactory level of
UX integration was not yet achieved.
5. Discussion
In this paper, we presented a case study of how a software development
company, over the years, may move from merely developing GUIs to also con-
sidering usability and more recently UX. Our aim was to answer the following
research question: How does UX integration unfold over time within the context
of an organization? And, what are the main intertwining events that impact UX
integration as it unfolds?
29
Type of event Examples
Internal events reside within the borders of
the company
changes in the organizational structure or devel-
opment processes
External events reside outside the borders of
the company
educational (e.g. courses), theoretical (e.g. tools
& methods), & technological advances (touch-
screen interactions); increasing knowledge &
awareness in the community
Direct events explicitly & directly change
UX integration in the company
introducing new UX-related roles & responsibili-
ties
Indirect events influence UX integration al-
though this is not their main purpose
changes in the development process from waterfall
to agile
Planned events are all internal events &
planned (aka. top-down) change initiatives
designed to influence UX integration
assigning a new head of R&D to improve UX in-
tegration in the company
Emergent events are all internal events &
grass-root (aka. bottom-up) change initia-
tives performed to influence UX integration
establishing UX meetings by a number of UX ad-
vocates in the company
Table 12: Different types of events that may influence UX integration
Our findings show that UX integration unfolds over time through an inter-
play among various events that differ in their nature and origins: (i) internal and
external, (ii) direct and indirect, (iii) planned and emergent (see Table 12). As
we mentioned before, these events (Table 4 to Table 11) relate to the three main
identified themes and clearly have a multifaceted and complex set of relations
with these themes.
Through better understanding these events and their enabling or prohibiting
role with regards to integration and more specifically the identified themes, we
identified 4 lessons learned that concern the organizational change aspect of UX
integration. Furthermore, we explicitly investigated the transition from usability
integration to UX integration and identified 5 pitfalls companies should avoid if
they want to go beyond usability integration and also address UX. As Table 13
summarizes, the identified lessons learned and pitfalls bridge existing gaps in
current literature on UX integration.
5.1. Lessons Learned Concerning UX Integration
The following lessons learned merged from our data analysis and should be
taken into account by practitioners to better adjust their integration efforts
and by researchers to define more industry-relevant research. These lessons
learned underline the importance of the organizational aspect of UX integration
and in particular its organizational change nature. These lessons learned are
summarized in Figure 2 and elaborated below.
Lessons Learned 1: Considering the influencing events
Achieving a sustainable UX integration does not merely depend on adopting UX
principles and practices through direct and planned initiatives. It additionally de-
pends on emergent change initiatives inside the organization and also external events
that may indirectly influence integration.
Current studies on UX integration often mainly focus on internal, planned
and direct events. Examples are action research studies [9, 10, 14, 15, 23, 65].
Admittedly, such studies provide valuable insight about how practitioners can
30
Literature Gap Lessons learned and Pitfalls
Current studies often do not differen-
tiate UX and usability or address how
these differences can impact day-to-day
work of practitioners or integration ef-
forts (e.g [3, 15, 40]). In addition, the
existing limited number of longitudi-
nal studies often only focus on usability
or UCD practices in general or do not
clearly differentiate them from UX and
UX integration (e.g. [9, 10, 15, 23])
To ensure moving b eyond usability to also ad-
dressing UX, organizations shall prevent various
pitfalls that concern the differences between UX
and usability: Blind transition, Saying UX but
doing usability, Associating UX only to GUI
Current studies rarely focus on com-
munication and collaboration between
UX and non-UX practitioners or among
UX practitioners. In addition, current
studies often associate the resistance
to integration with non-UX practition-
ers, their mindsets, and work priorities
(e.g. [4, 52])
The resistance to UX integration initiatives is
not shown merely by non-UX practitioners but
also UX practitioners. In addition, organizations
shall prevent Striving for the sole ownership of
UX and advocate shared ownership to improve
the communication and collaboration between UX
and non-UX practitioners
Current studies often mainly report on
a snapshot of integration and do not in-
vestigate it from a longitudinal perspec-
tive (e.g [22, 52, 63])
Presence and severity of success factors and chal-
lenges to UX integration may change over time
and are in fact influenced by the strategies to en-
able or prohibit them
Current studies often mainly focus on
internal, planned and direct events
(e.g [9, 10, 14, 15, 23, 65])
Achieving a sustainable UX integration does not
merely depend on adopting UX principles and
practices through direct and planned initiatives.
It additionally depends on both emergent change
initiatives inside the organization and external
events that may indirectly influence integration
Only a limited number of current stud-
ies investigate integration from an orga-
nizational change perspective (e.g [23,
66])
UX integration is a type of organizational change
hence to increase the effectiveness and success
of their initiatives, researchers and practitioners
should apply the already existing guidelines on
how to better implement changes in organizations.
However, for moving beyond usability to UX in
their change initiatives, organizations shall pre-
vent Taking pride in the past GUI or usability
achievements
Table 13: Gaps in current literature and their relation to the identified lessons learned and
pitfalls.
31
Lessons learned about UX integration
adopt UX principles & practices through direct & planned change initiatives
investigate & reflect on emergent UX integration initiatives inside the
organization
investigate & reflect on events inside the organisation that may indirectly
influence UX integration
remember that presence & severity of challenges & success factors to UX
integration may change over time
remember that presence & severity of challenges & success factors to UX
integration are influenced by strategies to prohibit or enable them
remember that resistance to UX integration initiatives may be shown by
both UX & non-UX practitioners
remember that UX integration is a type of organizational change
Considering
influencing
events
investigate & reflect on events outside the organisation that may indirectly
influence UX integration
Considering
success factors
& challenges
Considering
resistance to
UX integration
Considering UX
integration
guidelines apply existing guidelines on organizational change to plan, execute &
support UX integration in the organization
Figure 2: We identified four lessons learned on UX integration (listed to the left of the picture).
On the right, we present our recommendations for practitioners based on these lessons learned.
32
plan and execute their efforts for improving UX integration in their companies.
Nevertheless, such limited focus can give a distorted image of UX integration,
how it may have evolved in the complex context of organizations, and its as-
sociated challenges and success factors. Our study contributes to the current
body of knowledge on UX integration through providing a more realistic and
comprehensive image of these intertwining factors of various types and natures.
The fact that literature mainly focuses on planned UX integration initiatives
is expected as traditionally the planned approach to change has been perceived
to be more common even in the literature on organizational change [8]. More
recent studies, however, show that planned and emergent approaches to change
are complementary rather than competing [8]. These studies, therefore, recom-
mend that instead of focusing on one best approach, organizations should seek
to identify the approach which is best suited to certain changes they wish to
undertake in their specific organizational context [8]. In case of UX integration,
for instance, for establishing new organizational units to support UX integra-
tion a planned approach may be more suitable while for introducing new UX
practices an emergent approach.
To better support practitioners, researchers can develop empirically sup-
ported guidelines to evaluate the effectiveness of, and therefore choose among,
planned or emergent approaches. The guidelines can, for instance, include a
map between different organizational characteristics, different changes required
to support UX integration, and effectiveness of emergent or planned approaches
for these changes. Most importantly, this map should include common changes
that are required to support a transition from usability integration to UX inte-
gration in organizations. The work of Iivari et al. [29] is one example of such
guidelines. Their guideline maps different types of cultures and tools and meth-
ods for usability work. For instance, they suggest that a hierarchical culture
requires tools and methods that emphasize rules, standard procedures, docu-
mentation, and control. On the contrary, a rational culture requires tools and
methods that emphasize measurement and cost-benefit analyses that show the
business benefits of the change efforts. To create such guidelines, our work can
be extended by future empirical studies on identifying common emergent and
planned events that influence UX integration in various organizational contexts
with different characteristics, e.g., size, structure, culture, leadership style, etc.
Lessons Learned 2: Considering success factors & challenges
Presence and severity of success factors and challenges to UX integration may
change over time and are in fact influenced by the strategies to enable or prohibit
them.
As an example, the extent of resistance to UX changed in the case company
over time and was influenced by a verity of internal and external events. Cur-
rent literature, however, often portrays resistance to UX integration as a static
phenomenon that is mainly rooted in the differences between the two fields of
HCI and SE (e.g. [52]). By understanding how these success factors and chal-
lenges are formed and changed over time in response to different internal and
external factors, practitioners have more possibility to adjust their integration
initiatives and strategies in different periods of time depending on the severity
33
and the extent of success factors or challenges in these periods. We, therefore,
suggest future research to analyze UX challenges and success factors from a
more dynamic and longitudinal perspective.
Lessons Learned 3: Considering the resistance to UX integration
The resistance to UX integration initiatives is not showed merely by non-UX prac-
titioners but also UX practitioners.
We found that the showed resistance to UX integration in the organization
was not limited to non-UX practitioners, or as previous studies call it only a
result of developers’ mindsets and work priorities [4, 52]. A resistance was also
shown by UX advocates who were involved in bottom-up integration initiatives.
There was also an obvious power-struggle between this group of practitioners
and practitioners with new roles or competences assigned by management to im-
prove UX integration. To the best of our knowledge, this type of resistance and
power-struggle is merely explored in current UX integration literature. One
explanation for such resistance and power-struggle can be the clash between
emergent and planned initiatives. For instance, when management assigns roles
to handle UX integration, those practitioners who have been involved in emer-
gent events (i.e., bottom-up initiatives) may feel to some extent ‘left-out’ in the
new planned programs to change. To overcome such a challenge, organizations
should, for instance, better investigate previous emergent changes and better
align them with planned ones. This further underlines the importance of taking
various types of factors into account when investigating and planning UX inte-
gration in organizations, as elaborated above. Similar to the above insights we
gained on resistance, other challenges to UX integration can also be analyzed
better in their context, and from a more dynamic and longitudinal perspective.
Lessons Learned 4: Considering UX integration guidelines
UX integration is a type of organizational change hence to increase the effectiveness
and success of their initiatives, researchers and practitioners should apply the already
existing guidelines on how to better implement changes in organizations.
Through identifying and analyzing various types of factors that influence
UX integration, we highlight the fact that UX integration is, in fact, a type of
organizational change. Nevertheless, so far, only a limited number of studies
have investigated UX integration in its socio-technical context and through the
lens of organizational change. Usability researchers [29, 51, 66] emphasize that
by understanding the change nature of usability integration, practitioners can
better address related organizational issues, e.g., organizational culture, man-
agement commitment to changes, or communication and collaboration between
designers and developers; we expect similar benefits in case of UX integration.
5.2. Pitfalls in Transition from Usability to UX Integration
Current studies often do not reflect on the transition from usability to UX
and how this transition happens over time in an interaction with various fac-
tors inside and outside the organizations. Here, we explicitly differentiated UX
and usability concepts, investigated the company’s transition in the last years,
34
and accordingly identified 5 pitfalls regarding the transition. These pitfalls can
negatively impact integration efforts in organizations and prevent achieving a
sustainable UX integration. The pitfalls and proposals on how to overcome
them are summarized in Figure 3 and elaborated below.
Pitfall 1: Blind transition from usability to UX
Transition between GUI development to usability and then from usability to UX may
not be explicit and clear enough, meaning that differences and implications of new
concepts are not discussed during the transition nor disseminated among internal
stakeholders to raise awareness and get support. The transition also may suffer
from lack of reflection on and learning from past events influencing the transition.
As our case study shows, being influenced by the changes in the field of SE
and HCI, companies go through different stages before achieving a suitable level
of UX integration. However, in the rush to adopt emerging concepts such as
UX, practitioners may fail to sufficiently become aware of the implications these
concepts have for their organizations and processes. To avoid a blind transition,
and instead undergo a more informed transition, we suggest organizations to
not only pay attention to the differences and similarities as well as implications
of these concepts but also to reflect on and learn from past integration efforts,
whether successful or not. Such reflection is known to facilitate better planning
for and acting on future efforts and increase the likelihood of their success [9,
10, 13].
Pitfall 2: Saying UX but doing usability
Organizations may have the intention and ambition to address UX in their practices
however in reality their practices may still only remain usability-focused.
This pitfall is a more specific instance of blind transition and may hap-
pen as a result of overlooking the differences between UX and usability, and
consequently their integration into organizations. This pitfall can result in a
lopsided focus on the pragmatic aspect of UX which has been highlighted by
previous research [62]. Simply applying generic UCD practices without care-
ful attention to, and being committed to addressing differences between UX
and usability does not necessarily result in a better UX integration. UCD, for
instance, recommends identifying the end users’ personal needs, but without
knowledge on various human emotional needs (e.g curiosity, connectedness), a
practitioner may not be able to discover such needs and is likely to only focus
on usability needs of users (e.g. learnability, error prevention). Therefore, UCD
practices should be performed by practitioners who have sufficient knowledge
about these differences and the required competencies to support the whole
scope of UX rather than only its pragmatic aspect, i.e. usability. To go beyond
usability, UX advocates should get enough mandate to extend their activities
otherwise, as we saw, they may only be able to continue usability work in spite
of their ambition for UX. Most importantly, the organization should see a need
to support not only the pragmatic but also the hedonic aspect of UX. Orga-
nizations that want to incorporate UX should pay enough attention to how it
differs from usability or other similar concepts. This helps to more effectively
benefit from addressing this concept in their products and services and to go
beyond usability.
35
Pitfalls in transitioning from usability to UX integration
failing to raise awareness about
relations between UX & other relevant
concepts
failing to reflect on & learn from
previous integration efforts
failing to address the implications of
differences between UX & other
relevant concepts
Blind transition
from usability
to UX
discuss & disseminate the differences
& similarities between UX, usability &
GUI design
How to overcome?
take into account the implications of
these similarities & differences
reflect on & learn from past integration
efforts, whether successful or not
lopsided focus on the pragmatic
aspect of UX
lacking practitioners with knowledge
on the hedonic aspect of UX
applying generic UCD practices
without attention to characteristics of
UX
Saying UX but
doing usability
give mandate to UX advocates to
extend their activities to address the
hedonic aspect as well
extend UCD practices to go beyond
usability & coordinate among various
units to ensure a coherent UX
hire practitioners with knowledge on
both pragmatic & hedonic aspects of
UX
pay more attention to who the
advocates are than what UX really is
Associating UX
only to GUI
raise awareness about the relation of
UX to service design, business
strategy & innovation
various stakeholders from different
disciplines struggling to gain the sole
ownership of UX
Striving for the
sole ownership
of UX
advocate & facilitate a shared
ownership of UX
create a respectful attitude among
disciplines
adovate & facilitate a close
collaboration between UX & non-UX
practitioners
taking pride in previous achievements
in GUI design or usability
down-prioritizing UX integration
because of relying on past
achievements
considering current UX good-enough
through relying on past user feedback
Taking pride in
the past GUI or
usability
achievements
remember good UX means delivering
both satisfaction & pleasure
(delivering expected & unexpected)
consider current & future user needs &
competitions in the market to deliver
both expected & unexpected
iteratively evaluate & enhance UX to
satisfy the ever-changing user needs
& to deliver 'unexpected'
Figure 3: The identified pitfalls in UX integration and proposed strategies to overcome them
36
Pitfall 3: Associating UX only to GUI
UX may be perceived by stakeholders as a concept only concerned with GUI because
the concept of UX has its roots in the field of HCI which has traditionally been
associated with GUI design and look and feel.
Various stakeholders may limit UX as a concept merely concerned with GUI
design. As our findings show, the fact that UX advocates in the company were
mainly GUI developers further contributed to this association. As we saw, a
main consequence of limiting UX only to GUI design is to down-prioritize it in
the organization. However, the field of HCI is much broader in scope than only
look and feel of the GUI and usability of the products and services [25]. UX is
also known to be associated with service design [20, 50], business strategy [46, 60]
and innovation in organizations [28, 53, 61]. Therefore, to avoid this pitfall,
organizations need to increase knowledge and awareness about the role of UX
in the above topics and emphasize that UX is not only about GUI design and
aesthetics but also directly related to value delivery to customers and end users.
Pitfall 4: Striving for the sole ownership of UX
Different groups of internal stakeholders with different areas of competencies may
strive for the sole instead of a shared ownership.
It is expected to observe a power-struggle among various groups of practi-
tioners concerning the ownership of UX. UX practices are multidisciplinary in
nature and overlap with the work of practitioners in various disciplines such as
business, sales and marketing, and requirements, among others [24]. Therefore,
we agree with those researchers and practitioners that advocate a shared own-
ership of UX [6, 18]. Successful UX integration necessitates close cooperation
between UX and non-UX practitioners to ensure common goals. For instance,
in agile settings, this requires taking the UX practices schedule into account
when ordering the task list [37]. Moreover, enabling such approach necessitates
respectful attitude between disciplines [17].
Pitfall 5: Taking pride in the past GUI or usability achievements
Internal stakeholders may take pride in their previous achievements in GUI design
and development, or usability. These practitioners, therefore, may down-prioritize
UX integration, or efforts to improve UX design of the products.
Delivering a good UX often requires not only taking the expectations of
users into account and satisfying them but also delivering pleasure to the users
through providing unexpected features and qualities [25]. Expectations of users
are not static rather dynamic and influenced by various other external events, for
instance introducing new interaction medium or being exposed to other products
with better UX [25, 50]. Hence, although an organization may successfully
deliver good UX to its end users in certain time periods, its success may fade
away over time. Organizations, therefore, cannot rely on their previous success,
and instead, require to constantly investigate and enhance the UX delivered
through their products to address changes in users’ needs and expectations in
response to various external events. This is important for the business success
of companies as nowadays, many companies compete on the basis of providing
a better UX and not merely usability or functions [27, 50].
37
5.3. Limitations and Threats to Validity
Threats to validity are outlined and discussed based on the classification by
Runeson and H¨ost [54] that includes four threats to case study research, namely,
construct validity, internal validity, external validity, and reliability. Runeson
and H¨ost acknowledge that the above terms are usually used in controlled ex-
periments. Nevertheless, they operationalize these categories of threats for case
studies instead of applying other terms (e.g. credibility, transferability, depend-
ability, confirmability).
Construct validity concerns whether the studied phenomenon is relevant to
validly address the research questions and whether the operational measures
that are studied really represent what the researcher have in mind. In the case
of our study, the aim was to gather data on the company’s integration jour-
ney over the last years and study how various events positively or negatively
influenced integration. This was achieved through interviewing various prac-
titioners, observations, workshops, and document analysis. Applying various
data gathering methods helped improving construct validity of our study.
However, to prevent this threat, it was also important to select practitioners
that have been present during various studied events. We selected the intervie-
wees together with our main collaborators in the company and based on these
criteria: work experience in the company, current and previous roles, and at-
titude towards UX integration (both positive and negative). We also provided
a written description of the interviewees we were looking for to avoid any mis-
understandings regarding the criteria. As we were interested in the events that
happened in the past, we ensured to include also those interviewees who were
present in the company since early 90s. Five of our interviewees have been em-
ployed at the company at least since the beginning of the first period of events.
In addition, four of the interviewees have been employed at the company at
least since the middle of the first period of events (late 90s or early 2000s).
The interviews were face-to-face, audio recorded, and lasted between 30 to 60
minutes. The presence of a researcher may influence the behavior and response
of the subjects. This threat is an inherent limitation of the research method
used, but was alleviated somewhat by the guarantee of confidentiality of the
data and cross checking the results with participants in the study.
The construct validity could have also been improved by investigating the
quality of the UX and usability artifacts or even developed products over the
years. We investigated the UX artifacts generated from 1992 to 2015 however
our aim was mainly triangulation of the interview data rather than analyzing
the quality of these artifacts. We believed judging the quality of the artifacts
or the UX delivered through the products required a more controlled setting
and longer investigation of these artifacts or products. This was therefore not
feasible, in particular, since a number of these artifacts were generated before
the start of our research study or that we did not have access to the previous
versions of the products. More importantly, it is not possible to evaluate UX
delivered through these products out of their context and time e.g. a product
that is perceived as ‘novel’ in the 90s, may be perceived as ‘old’ in 2015.
38
In any empirical study, incorrect data is a threat to internal validity. Inter-
views were audio recorded to mitigate this threat. The authors also analyzed
the material in several rounds of independent as well as joint sessions to gradu-
ally reach consensus on the intended meaning of the responses. We also cross-
checked the results of our analysis with the interviewees to validate and confirm
the findings. Using variety of methods (observations, document analysis and
interviews) also helped alleviating this threat.
Another limitation is that in the period under study, the case company
itself has evolved and naturally gone through changes outside the scope of our
study. There are, therefore, other sources that may have also influenced UX
integration but are not reflected in our data. However, we draw the reader’s
attention to this limitation and highlight this study does not provide cause and
effect relationships of events and their enabling or prohibiting role in integration.
Rather, the study aims to provide a holistic picture of UX integration and the
company’s transition to the extent supported by our data.
External validity concerns the ability to generalize the results beyond the
actual study. In case studies, the intention is to enable analytical generalization
where the results are extended to cases which have common characteristics and
hence for which the findings are relevant. The case company is a medium-
sized Swedish software development company, developing a business-to-business
product. We therefore expect our findings to be valid for companies with similar
characteristics. In addition, we have compared and contrasted our findings
to the existing theories on organizational change which further contributes to
external validity of our research. Another concern is that our data gathering
was performed in 2014-2015. We finalized our research collaboration with the
company in 2015 and at that time the efforts to improve UX integration were still
ongoing, and a satisfactory level of UX integration was not yet achieved as our
data shows. However, as we have not gathered more data on the organization
since 2015, our data may not reflect today’s UX state-of-practice in the studied
organization. However, the data is valid when interpreted in its own time frame.
In particular, since the aim of this study was to understand the transition the
company has gone through over the years. In addition, to minimize the effect of
the time frame on our analysis, we have included recent studies published since
2014 when discussing the results.
Another threat concerns reliability, the extent to which the data and analysis
are dependent on the specific researchers. Although the coding process is per-
formed by the first author, to improve reliability of the generated themes, the
three authors individually and independently conducted a pilot coding of these
segments. The outcomes of the pilot coding were discussed in several sessions
with all three authors, and the differences in coding were analyzed and resolved.
Also, we had carefully designed the workshops and interviews before running
them. We also defined the coding process after the interviews and before an-
alyzing the data. The initial codes were therefore identified mainly based on
observed interview responses. We also ensured the themes are not imposed on
the data rather emerged from it.
39
6. Conclusion
Through an investigation of the organization’s transition from GUI design
and development to addressing usability and more recently UX, we provided
a more holistic and realistic picture of UX integration. We identified various
events with different natures (internal or external, direct or indirect, emergent
or planned) that impacted the transition to UX integration. Outside the bor-
ders of organizations, technological, theoretical and educational advances in the
fields of HCI and SE contribute to an increase in knowledge and awareness
of both internal and external stakeholders, changes in their expectations, atti-
tudes, and priorities. These advances also may facilitate UX integration as they
lead to organizations’ easier access to required UX competencies and tools and
methods. Internal events span across organizational units, are in a constant
interplay with each other and the context of the organizations, and inevitably
influence UX integration over time. Our findings extend current literature on
UX (and usability) integration that often focuses on planned and internal events.
We highlight the organizational change nature of UX integration and empha-
size that software community can benefit from already existing organizational
change guidelines and best practices to facilitate a sustainable UX integration in
software companies. Most importantly, we show practitioners need to explicitly
address differences between usability and UX in their change initiatives in or-
der to ensure moving beyond usability integration to UX integration. Although
our findings, including the lessons learned and pitfalls, are specific to UX, they
can also shed light on integrating less mature and new knowledge areas such as
software quality characteristics into software companies.
Acknowledgments
The authors would like to thank the case company and in particular practi-
tioners who participated in this study.
References
[1] Abrah˜ao, S., Juristo, N., Law, E. L.-C., Stage, J., nov 2010. Interplay be-
tween usability and software development. Journal of Systems and Software
83 (11), 2015–2018.
[2] Alves, R., Valente, P., Nunes, N. J., 2014. The state of user experience eval-
uation practice. In: Proceedings of the 8th Nordic Conference on Human-
Computer Interaction: Fun, Fast, Foundational (NordiCHI ’14). ACM,
New York, NY, pp. 93–102.
[3] Ardito, C., Buono, P., Caivano, D., Costabile, M. F., Lanzilotti, R., Dit-
trich, Y., 2014. Human-centered design in industry: Lessons from the
trenches. Computer 47 (12), 86–89.
40
[4] Bak, J. O., Nguyen, K., Risgaard, P., Stage, J., 2008. Obstacles to Usability
Evaluation in Practice. In: 5th Nordic conference on Human-computer
interaction building bridges - NordiCHI ’08. ACM Press, New York, New
York, USA, p. 23.
[5] Berntsson Svensson, R., Gorschek, T., Regnell, B., Torkar, R., Shahrokni,
A., Feldt, R., 2012. Quality Requirements in Industrial Practice- An Ex-
tended Interview Study at Eleven Companies. IEEE Transactions on Soft-
ware Engineering 38 (4), 923–935.
[6] Bogaards, P., Priester, R., 2005. User experience: back to business. Inter-
actions 12, 23–25.
[7] Bourque, P., Fairley, eds, R. (Eds.), 2014. Guide to the Software Engineer-
ing Body of Knowledge. IEEE Computer Society.
[8] Burnes, B., 2004. Emergent change and planned change – competitors or
allies?: The case of XYZ construction. International Journal of Operations
and Production Management 24 (9), 886–902.
[9] Cajander, ˚
A., Eriksson, E., Gulliksen, J., 2010. Towards a Usability Coach-
ing Method for Institutionalizing Usability in Organisations. HCIS 2010:
Human-Computer Interaction, 86–97.
[10] Cajander, ˚
A., Janols, R., Eriksson, E., 2014. On the establishment of
user-centred perspectives. In: Proceedings of the 8th Nordic Conference
on Human-Computer Interaction Fun, Fast, Foundational - NordiCHI ’14.
ACM Press, New York, New York, USA, pp. 103–112.
[11] Chapra, S. C., Canale, R. P., 1998. Numerical methods for engineers. Vol. 2.
Mcgraw-hill, New York.
[12] Chung, L., Nixon, B. A., Yu, E., Mylopoulos, J., 2000. Non-functional re-
quirements in software engineering. Kluwer Academic Publishing, Boston,
MA.
[13] Dingsøyr, T., Moe, N., Schalken, J., St˚alhane, T., 2007. Organizational
Learning Through Project Postmortem Reviews - An Explorative Case
Study. In: Proceedings of the 14th European software process improvement
conference (EuroSPI 2007). pp. 136–147.
[14] Eriksson, E., Gulliksen, J., Cajander, ˚
A., 2008. Introducing usability roles
in public authorities. Proceedings of the 5th Nordic conference on Human-
computer interaction building bridges - NordiCHI ’08, 113.
[15] Federoff, M., Courage, C., 2009. Successful User Experience in an Agile
Enterprise Environment. In: Smith, M. J., Salvendy, G. (Eds.), Human In-
terface and the Management of Information. Designing Information Envi-
ronments. Vol. 5617 of Lecture Notes in Computer Science. Springer Berlin
Heidelberg, Berlin, Heidelberg, pp. 233–242.
41
[16] Ferreira, J., Sharp, H., Robinson, H., 2011. User experience design and agile
development: managing cooperation through articulation work. Software:
Practice and Experience 41 (9), 963–974.
[17] Ferreira, J., Sharp, H., Robinson, H., 2012. Agile Development and User
Experience Design Integration as an Ongoing Achievement in Practice. In:
Proceedings of the 2012 Agile Conference (AGILE 2012). IEEE, pp. 11–20.
[18] Gabriel-Petit, P., may 2005. Introduction: sharing ownership of UX. inter-
actions 12 (3), 16.
[19] Glaser, B. G., Strauss, A. L., Strutzel, E., 1968. The Discovery of Grounded
Theory; Strategies for Qualitative Research. Nursing Research 17 (4), 364.
[20] Goldstein, S. M., Johnston, R., Duffy, J., Rao, J., 2002. The service con-
cept: The missing link in service design research? Journal of Operations
Management 20 (2), 121–134.
[21] Gothelf, J., Seiden, J., 2013. Lean UX: Applying Lean Principles to Improve
User Experience. O’Reilly Media.
[22] Gulliksen, J., Boivie, I., Persson, J., Hektor, A., Herulf, L., 2004. Making a
difference: a survey of the usability profession in Sweden. In: Proceedings
of the 3rd Nordic conference on Human-computer interaction (NordiCHI
’04). ACM, New York, NY, pp. 207–215.
[23] Gulliksen, J., Cajander, ˚
A., Sandblad, B., Eriksson, E., Kavathatzopoulos,
I., 2009. User-centred systems design as organizational change: A longitudi-
nal action research project to improve usability and the computerized work
environment in a public authority. International Journal of Technology and
Human Interaction (IJTHI) 5 (3), 13–53.
[24] Hartson, R., Pyla, P., 2012. The UX Book: Process and Guidelines for
Ensuring a Quality User Experience. Morgan Kaufmann.
[25] Hassenzahl, M., 2003. The Thing and I: Understanding The Relationship
Between User and Product. In: Blythe, M. A., Monk, A. F., Overbeeke,
K., Wright, P. C. (Eds.), Funology: from usability to enjoyment. Springer,
Dordrecht, Ch. 3, pp. 31–42.
[26] Hassenzahl, M., 2008. User experience (UX): towards an experiential per-
spective on product quality. In: IHM ’08: Proceedings of the 20th Inter-
national Conference of the Association Francophone d’Interaction Homme-
Machine. ACM, New York, NY, pp. 11–15.
[27] Hassenzahl, M., 2010. Experience Design: Technology for All the Right
Reasons. Morgan & Claypool, San Francisco, CA.
[28] Hoonhout, J., Meerbeek, B., Buie, E., 2012. I just love this product!: look-
ing into wow products, from analysis to heuristics. CHI ’12 Extended Ab-
stracts on Human Factors in Computing Systems, 2731–2734.
42
[29] Iivari, N., 2006. ’Representing the User’ in software development-a cultural
analysis of usability work in the product development context. Interacting
with Computers 18 (4), 635–664.
[30] ISO, 2000. ISO/TR 18529 Ergonomics- Ergonomics of human- system
interaction- Human-centred lifecycle process descriptions. International Or-
ganisation for Standardisation, Geneva, Switzerland.
[31] ISO, 2010. ISO 9241: Ergonomics of human-system interaction – Part 210:
Human-centred design for interactive systems. International Organisation
for Standardisation, Geneva, Switzerland.
[32] Isomursu, M., Sirotkin, A., Voltti, P., Halonen, M., 2012. User Experience
Design Goes Agile in Lean Transformation – A Case Study. In: Proceedings
of the 2012 Agile Conference (AGILE 2012). IEEE, pp. 1–10.
[33] Kashfi, P., Nilsson, A., Feldt, R., 2016. Integrating User eXperience Prac-
tices into Software Development Processes: The implication of Subjectivity
and Emergent nature of UX. PeerJ Computer Science (in submission).
[34] Kashfi, P., Nilsson, A., Feldt, R., 2017. Integrating User eXperience prac-
tices into software development processes: implications of the UX charac-
teristics. PeerJ Computer Science 3, e130.
[35] Kroll, P., Kruchten, P., 2003. The Rational Unified Process Made Easy:
A Practitioner’s Guide to the RUP. Addison-Wesley Longman Publishing
Co., Inc., Boston, MA, USA.
[36] Kuusinen, K., 2015. Task Allocation Between UX Specialists and Devel-
opers in Agile Sfotware Development Projects. In: Proceedings of the 15h
IFIP TC 13 International Conference Human-Computer Interaction (IN-
TERACT 2015). Springer, pp. 27–44.
[37] Kuusinen, K., V¨a¨an¨anen-Vainio-Mattila, K., 2012. How to make agile UX
work more efficient: management and sales perspectives. In: Proceedings
of the 7th Nordic Conference on Human-Computer Interaction: Making
Sense Through Design (NordiCHI ’12). ACM, New York, NY, pp. 139–148.
[38] Lallemand, C., Gronier, G., Koenig, V., feb 2015. User experience: A con-
cept without consensus? Exploring practitioners’ perspectives through an
international survey. Computers in Human Behavior 43, 35–48.
[39] Lallemand, C., Koenig, V., Gronier, G., 2014. How Relevant is an Ex-
pert Evaluation of User Experience based on a Psychological Needs-Driven
Approach? In: Proceedings of the 8th Nordic Conference on Human-
Computer Interaction: Fun, Fast, Foundational (NordiCHI ’14). ACM,
New York, NY, pp. 11–20.
43
[40] Lanzilotti, R., Costabile, M. F., Ardito, C., Informatica, D., Aldo, B.,
2015. Addressing Usability and UX in Call for Tender for IT Products.
In: Proceedings of the 15h IFIP TC 13 International Conference Human-
Computer Interaction (INTERACT 2015). pp. 1–8.
[41] L´arusd´ottir, M., Gulliksen, J., Cajander, ˚
A., 2016. A license to kill – Im-
proving UCSD in Agile development. Journal of Systems and Software 000,
1–9.
[42] L´arusd´ottir, M. K., Cajander, ˚
A., Gulliksen, J., 2012. The Big Picture
of UX is Missing in Scrum Projects. In: Proceedings of the 2nd Interna-
tional Workshop on the Interplay between User Experience and Software
Development (I-UxSED 2012). RWTH Aachen University, pp. 43–48.
[43] Law, E. L.-C., Roto, V., Hassenzahl, M., Vermeeren, A. P. O. S., Kort,
J., 2009. Understanding, Scoping and Defining User Experience: A Survey
Approach. In: Proceedings of the SIGCHI conference on Human factors in
Computing Systems (CHI’09). CHI ’09. ACM, New York, NY, USA, pp.
719–728.
[44] Law, E. L.-C., van Schaik, P., Roto, V., 2014. Attitudes towards user ex-
perience (UX) measurement. International Journal of Human-Computer
Studies 72 (6), 526–541.
[45] Mao, J.-Y., Vredenburg, K., Smith, P. W., Carey, T., 2005. The state of
user-centered design practice. A Survey of User-Experience Development
at Enterprise Software Companies 48 (3), 105–109.
[46] Marcus, A., Ashley, J., Knapheide, C., Lund, A., Rosenberg, D., Vre-
denburg, K., 2009. A survey of user-experience development at enterprise
software companies. HCII 2009 5619 LNCS, 601–610.
[47] Nass, C., Adam, S., Doerr, J., Trapp, M., 2012. Balancing User and Busi-
ness Goals in Software Development to Generate Positive User Experience.
In: Zacarias, M., de Oliveira, J. (Eds.), Human-Computer Interaction: The
Agency Perspective SE - 2. Vol. 396 of Studies in Computational Intelli-
gence. Springer Berlin Heidelberg, pp. 29–53.
[48] Ovad, T., Larsen, L. B., 2015. The Prevalence of UX Design in Agile Devel-
opment Processes in Industry. In: Proceedings of the 2015 Agile Conference
(AGILE 2015). IEEE, pp. 40–49.
[49] Paech, B., Kerlow, D., 2004. Non-Functional Requirements Engineering
- Quality is essential. In: Proceedings of the 10th International Working
Conference on Requirements Engineering: Foundation for Software Quality
(REFSQ’04). pp. 237–250.
[50] Pine, B. J., Gilmore, J. H., 1998. Welcome to the experience economy.
Harvard business review 76.
44
[51] R¨onkk¨o, K., Hellman, M., Dittrich, Y., 2008. PD method and socio-political
context of the development organization. Proceedings of the Tenth Anniver-
sary Conference on Participatory Design 2008, 71–80.
[52] Rosenbaum, S., Rohn, J. A., Humburg, J., 2000. A toolkit for strategic
usability. In: Proceedings of the SIGCHI conference on Human factors in
Computing Systems (CHI’00). Vol. 2. ACM, New York, NY, pp. 337–344.
[53] Roto, V., Kaasinen, E., Nuutinen, M., Sepp¨anen, M., 2016. UX Expedi-
tions in Business-to-Business Heavy Industry. Proceedings of the 2016 CHI
Conference Extended Abstracts on Human Factors in Computing Systems
- CHI EA ’16, 833–839.
[54] Runeson, P., H¨ost, M., 2008. Guidelines for conducting and reporting case
study research in software engineering. Empirical Software Engineering
14 (2), 131–164.
[55] Runeson, P., H¨ost, M., Rainer, A., Regnell, B., 2012. Case Study Research
in Software Engineering: Guidelines and Examples, 1st Edition. Wiley Pub-
lishing.
[56] Schaffer, E., 2004. Institutionalization of usability: a step-by-step guide.
Addison-Wesley Professional.
[57] Schaffer, E., Lahiri, A., 2013. Institutionalization of UX: A Step-by-step
Guide to A User Experience Practice [ux], 2nd Edition. Addison-Wesley.
[58] Stol, K.-j., Ralph, P., Fitzgerald, B., 2016. Grounded theory in software
engineering research. In: Proceedings of the 38th International Conference
on Software Engineering - ICSE ’16. ACM Press, New York, New York,
USA, pp. 120–131.
[59] Strauss, A. L., Corbin, J., 1998. Basics of Qualitative Research: Techniques
and Procedures for Developing Grounded Theory, 2nd Edition. Sage Pub-
lications.
[60] Sward, D., Macarthur, G., 2007. Making user experience a business strat-
egy. In: E. Law et al.(eds.), Proceedings of the Workshop on Towards a
UX Manifesto. Vol. 3. pp. 35–40.
[61] Treviranus, J., 2009. You say tomato, I say tomato, let’s not call the whole
thing off: the challenge of user experience design in distributed learning
environments. On the Horizon 17 (3), 208–217.
[62] V¨a¨an¨anen-Vainio-Mattila, K., Roto, V., Hassenzahl, M., 2008. Now Let’s
Do It in Practice: User Experience Evaluation Methods in Product De-
velopment. In: Proceedings of Extended Abstracts on Human Factors in
Computing Systems (CHI’08). ACM, New York, NY, pp. 3961–3964.
45
[63] Venturi, G., Troost, J., Jokela, T., 2006. People, Organizations, and Pro-
cesses: An Inquiry into the Adoption of User-Centered Design in Industry.
International Journal of Human-Computer Interaction 21 (2), 219–238.
[64] Vermeeren, A. P. O. S., Law, E. L.-C., Roto, V., Obrist, M., Hoonhout,
J., V¨a¨an¨anen-Vainio-Mattila, K., 2010. User experience evaluation meth-
ods: Current State and Development needs. In: Proceedings of the 6th
Nordic Conference on Human-Computer Interaction Extending Boundaries
(NordiCHI ’10). ACM, New York, NY, pp. 521–530.
[65] Winter, J., R¨onkk¨o, K., 2010. SPI success factors within product usability
evaluation. Journal of Systems and Software 83 (11), 2059–2072.
[66] Winter, J., R¨onkk¨o, K., Rissanen, M., 2014. Identifying organizational bar-
riers - A case study of usability work when developing software in the au-
tomation industry. Journal of Systems and Software 88 (1), 54–73.
46
