Conference PaperPDF Available

The Usage of Usability Techniques in Scrum Projects

Authors:

Abstract and Figures

Over the past decades, usability techniques have been introduced into software development practices. At the same time many software development teams have started to use the agile development process --- Scrum --- to plan and organize their software projects. The focus of this study is to explore how usability techniques are integrated during software development in Scrum projects. The most commonly used usability technique in Scrum projects is workshops, followed by lo-fi prototyping, interviews and meetings with users, all used by more than half of the participants. The technique that is most frequently used is lo-fi prototyping used by more than half of the participants two to four times a month. All these usability techniques are informal, meaning that these techniques can be used quickly without much preparation. Formal usability evaluation with users is a highly ranked technique by the participants but not commonly used by them.
Content may be subject to copyright.
The Usage of Usability Techniques in Scrum Projects
Yuan Jia1, Marta Kristin Larusdottir2 and Åsa Cajander3
1Indiana University, Indiana Avenue 719 Indianapolis, United States
jiayuan@umail.iu.edu
2Reykjavik University, Menntavegur 1, 101 Reykjavik, Iceland
marta@ru.is
3Uppsala University, Lägerhyddsvägen 2 Uppsala, Sweden
asa.cajander@it.uu.se
Abstract. Over the past decades, usability techniques have been introduced into
software development practices. At the same time many software development
teams have started to use the agile development process Scrum to plan and
organize their software projects. The focus of this study is to explore how
usability techniques are integrated during software development in Scrum
projects. The most commonly used usability technique in Scrum projects is
workshops, followed by lo-fi prototyping, interviews and meetings with users,
all used by more than half of the participants. The technique that is most
frequently used is lo-fi prototyping used by more than half of the participants
two to four times a month. All these usability techniques are informal, meaning
that these techniques can be used quickly without much preparation. Formal
usability evaluation with users is a highly ranked technique by the participants
but not commonly used by them.
Keywords: Usability techniques, User centred design, user involvement,
usability, agile software development, Scrum.
1 Introduction
Scrum, as one of the agile software development processes, has been gaining
popularity in software development over the last few years to plan and organize
software development projects [5]. In Scrum the projects are split up in two to four
weeks long periods called sprints, each ending up with a potential shippable product
that the end users should be able to use right after delivery. In Scrum self organizing
and well compounded software development teams are heavily emphasized, typically
with six to eight interdisciplinary team members [7]. The main characteristics of the
process are simplicity and speed [1] which possibly is one of the reasons for its
popularity in industry. One of the benefits of using agile development processes was
claimed to be that customers needs are taken more into account than when
developing software using more traditional processes [7]. Traditional Scrum has been
criticized for not involving real users in their software process and for not adequately
addressing their usability needs [8]. One of the main conclusions in an extensive
literature study on the integration of the usability needs into agile processes is that the
2 Yuan Jia1, Marta Kristin Larusdottir2 and Åsa Cajander3
end user needs have not yet been sufficiently included in the agile development
processes [9].
At the same time as Scrum became popular in industry, the term usability emerged
during the mid 1980s, and was accepted in the 1990s by the software industry. This
was partly as a response to the new challenges that web-based software - to be used
by a large number of diverse users - put on IT (Information Technology)
professionals. However, the body of knowledge of usability is large and includes
various perspectives from usability engineering to more context-oriented approaches
and these have not yet been accepted fully by industry [3].
For the past decade, usability techniques used in various areas in the industry have
been studied by researchers. For example, Venturi & Troost [10] studied how User-
Centered Design (UCD), one of the main approaches in the usability field, was
integrated in software development. Larusdottir et al. [5] studied the effect of using a
particular development process in industry on the use of user involvement techniques.
In that study, about half of the participants were using their own process to plan their
work and about one third were using Scrum as their development process. When
asked about, if usability is important the participants using the Scrum process were
the most negative ones. The most popular user involvement method was meetings
with users used by almost all the participants. The use of user involvement methods
varied quite extensively according to which process is used for software development.
The results from these studies motivated us to examine the use of usability techniques
in projects using one particular process, namely the Scrum process.
This paper describes and discusses the results of a survey study on how usability
techniques are being used in software projects using the Scrum process to plan and
organize the work. The focus of the study is to explore what usability techniques are
used, if the usage of one technique is correlated with the usage of the other
techniques, how often the techniques are used and how useful they are for IT
professionals. The motivation for the study is to gain understanding of what IT
professionals need to be able to integrate usability activities more extensively while
using the Scrum process in software development.
2 Background
Several studies have been conducted on how usability techniques are integrated into
software development. It has been explored in some of the studies what software
development processes are used, but often it is not analyzed how or if the software
process affects how usability techniques were used. In this chapter an overview is
given of some of the current literature on how usability techniques have been
integrated in software development in industry.
Bygstad, Ghinea, and Brevik [3] surveyed professionals working at Norwegian IT
companies to investigate the relationship between software development
methodologies and usability activities. In their findings, there was a gap between
intention and reality. The IT professionals expressed interests and concerns about the
usability of their products, but they were less willing to spend resources on it in
industrial projects with time and cost constraints. The results of their survey also
Error! No text of specified style in document. 3
revealed that the IT professionals perceived usability activities and software
development methods to be integrated, which the authors believed is a positive sign.
Bark et al. [2] conducted a survey on the usage and usefulness of HCI methods
during different development phases. They examined whether the type of the software
projects had any effects on HCI practitioners’ perception of the usefulness of the
methods. The results show that there was fairly little correlation between the
frequency of using a particular technique and how useful it was perceived by the HCI
practitioners. One conclusion in the study is that HCI practitioners tend to have a
personal and overall evaluation of the different techniques rather than evaluating the
actual usefulness of the methods in their daily work when developing particular
software.
An international web-based survey by Monahan et al. [6] reported the state of
using several field study techniques and how effective they were considered to be by
usability practitioners in education and industry. The results show that more than half
of the respondents rated observations as an extremely effective method and about
40% of the respondents rated user testing as extremely effective. The most influential
factor for choosing a method for participants working in the software industry was
time constrains.
Venturi, Troost and Jokela [11] investigated the adoption of user centred design
(UCD) in software industry. The results of the study show that the most frequently
used method was user interviews. Additionally, hi-fi and low-fi prototyping methods
were frequently used. Overall, the most frequently used evaluation methods are
qualitative, allowing rapid feedback to the design activities using expert and heuristic
evaluation or “quick and dirty” usability test methods. The results also show that
UCD methods are typically used during the early phases of the product life cycle.
A survey study on the usage of 25 usability techniques was conducted in Sweden
by Gulliksen et al. [4]. The results show that the usability techniques that received the
highest rating by the usability professionals were those that were informal, involved
users and were concerned with design issues. Techniques such as expert-based
evaluations and benchmarking that do not involve users, received the lowest ratings
by the usability professionals. There was a general agreement among the participants
that it is important to integrate usability techniques into the software development
process they were using. Some participants mentioned difficulties during the
integration, especially those that were using RUP (Rational Unified Process) as their
development process.
3 Research Method
The research method in this study was a questionnaire-based survey. The
questionnaire was distributed to IT professionals who were experienced in using
Scrum as their software development process and were using usability techniques in
their software development.
In the survey we asked IT professionals about their usage of different usability
techniques and how useful they rated the techniques. We define usability techniques
4 Yuan Jia1, Marta Kristin Larusdottir2 and Åsa Cajander3
as the various techniques and methods used in software development to enhance the
usability of an IT system.
3.1 Survey Conduction
The survey was constructed in QuestionPro, which is an online survey tool
(http://www.questionpro.com/). The survey included 41 multiple-choice questions
and 5 open questions. The questions were developed according to a literature review
and interviews with two IT professionals experienced in using Scrum.
The list of 13 usability techniques was a result from a literature review based on
Gulliksen et al. [4], Venturi et al. [11] and Larusdottir et al. [5]. However, not all
techniques from these studies were chosen since we did not want the list to be too
long and risk a low response rate. We chose 5 data gathering techniques: interviews,
workshops, questionnaires, meetings with users and field studies, three techniques
often used for analysis: usability goals, scenarios and personas, two types of design
techniques: lo-fi prototyping and digital prototyping and three techniques for
evaluation: formal usability evaluations with users, informal evaluations with users
and one expert evaluation technique, the heuristic evaluation. With this selection we
wanted to cover a wide range of usability techniques used in software development
practice today. The respondents were asked if they had used any other techniques.
Two participants responded to that question naming one technique each.
The questions in the questionnaire were grouped into four sections: (1) Information
on the companies/organizations and experience of the respondents, (2) the Scrum
process in one particular project, (3) the usability techniques that have been used in
the particular Scrum project and (4) open questions on usability activities.
Two pilot tests were conducted in order to enhance the quality of the survey. The
participants were experts in HCI working in the software industry and using Scrum as
their development process. To estimate the approximate time for taking the survey,
the first pilot test was timed without any interruption. An interview was conducted
after the test. Think-aloud method was used in the second pilot test to detect problems
while answering the survey.
3.2 Survey Distribution
The survey link was distributed in three different ways. First, the survey link was sent
to an email list of software development companies in the Stockholm and Uppsala
area. The email list was provided by the Uppsala Tax Office and Lokaldelen, which is
a website offering information on companies in Sweden (http://www.lokaldelen.se/).
The second way of sending the survey was through directly contacting the target
respondents. In order to get more responses, the survey link was also posted on an
online discussion group called Scrum Alliance, (http://groups.google.com/group/
scrumalliance) which is an international forum for IT professionals using Scrum.
During a period of 40 days, from 25th April 2011 to 4th June 2011, totally 49
respondents responded to the survey and 35 of the participants completed all the
questions.
Error! No text of specified style in document. 5
3.3 Respondents
Respondents came from 7 countries around the world. The vast majority, 78% of the
respondents came from Sweden, the rest of them came from China (8%), USA (6%),
France (2%), Greece (2%), Lithuania (2%) and South Africa (2%).
About 70% of the respondents had a university degree either a Master or a
Bachelor degree. Fourteen out of 49 respondents were certified Scrum Masters, which
is a particular role while using the Scrum process. The result shows that about 30% of
the respondents had programming as their main job role and 20% responded that
usability engineering was their main job role. Around 20% of respondents had a
management role in their projects. Others worked for example on code design (10%),
on requirement gathering (10%) and on UI design (8%). None of the participants
indicated that software testing or evaluation was their main job task.
About one third of the respondents were employed at companies or organizations
having up to 50 employees, one third of the respondents at companies/organizations
having 50-249 employees and one third of respondents in companies/organizations
that had more than 250 employees. Over 40% of the IT professionals worked on
projects for the Internet or the e-commerce area, around 20% worked in the IT
industry in general, about 20% were working in particular domains like in the health
and medical sector, telecommunication sector or in the financial sector. Respondents
also reported some other business types that were not listed in the question.
In the second part of the survey the respondents were asked to select one particular
project to give information on. Seventy present of respondents had been working on
multiple projects simultaneously for the last 3 months. When asked about the type of
the project that they had selected, about 40% responded that the particular project was
web related. About 25% of the professionals were developing software products for
sale, around 15% were developing software systems for clients and about 15% were
developing internal software systems. The remaining respondents mentioned
developing other types of systems, including embedded software systems and
hardware systems.
4 Results
This chapter presents the results on what usability techniques are used by IT
professionals, how often they use the techniques, how the IT professionals rate the
usefulness of the usability techniques and how the usage of one usability technique is
correlated with the usage of other usability techniques.
4.1 Usability Techniques Used in Scrum Projects
Before asking about the usability techniques, all techniques were listed and explained
to make a common understanding of the steps taken while using the techniques and
what the names of the techniques actually meant. In Table 1 the usability techniques
are listed according to how many respondents had used them. The results in Table 1
show that workshops are the most commonly used usability technique in Scrum
6 Yuan Jia1, Marta Kristin Larusdottir2 and Åsa Cajander3
projects followed by lo-fi prototyping, interviews and meetings with users. Heuristic
evaluation was used by only 11% of the participants and questionnaires were used by
about 20% of the participants. Evaluations with users are not that common, about one
fourth of the participants had conducted informal evaluation with users and about one
third of the participants had conducted formal usability evaluation with users.
Table 1. The Usage of Usability Techniques
4.2 The Frequency of Using Usability Techniques
The respondents were asked about the frequency of using the usability techniques
during one Scrum project. The results from that question are shown in Table 2.
Table 2. The Frequency of Using the Usability Techniques
* N represents the number of respondents who had used the technique in their projects.
** With users participating.
Usability techniques
Used
Total
Percentage
Workshops
30
41
73%
Lo-fi prototyping
20
36
56%
Interviews
25
46
54%
Meetings with users
21
40
53%
Scenarios
17
36
47%
Digital prototyping
17
36
47%
Personas
15
35
43%
Field studies
17
40
46%
Usability goals
15
38
40%
Formal usability evaluation with users
11
36
31%
Informal usability evaluation with users
8
35
23%
Questionnaires
9
42
21%
Heuristic evaluation
4
35
11%
Usability techniques
Once a
week or
more
2 -3
times a
month
7 12
times
a year
2 6
times
a year
Once
a year
or less
N*
9%
13%
22%
44%
13%
25
0%
0%
0%
25%
75%
9
7%
7%
25%
50%
11%
30
15%
10%
30%
35%
15%
21
0%
0%
7%
53%
40%
17
21%
7%
29%
29%
14%
15
24%
24%
18%
24%
12%
17
6%
19%
13%
25%
38%
15
24%
12%
6%
35%
24%
17
40%
20%
15%
20%
5%
20
0%
0%
18%
82%
0%
11
25%
25%
13%
50%
13%
8
0%
25%
0%
50%
25%
4
Error! No text of specified style in document. 7
The technique that is most frequently used is lo-fi prototyping, used once a week by
about 40% of the respondents. About half of the participants use scenarios two to four
times a month. About 75% of respondents who had used questionnaires said that they
used that technique only once a year or less frequently. About 40% of the respondents
used personas once a year or less than that. The remaining techniques were used two
to six times a year.
4.3 The Ratings of the Usability Techniques
The ratings of how useful the respondents find the usability techniques are presented
in Table 3. The participants were asked to rate the techniques on a five-point scale
from very good to very bad.
Table 3. The Rating of the Usability Techniques
* N represents the number of respondents who had used the technique in their projects.
** With users participating.
The result reveals that about 75% of the respondents thought that formal usability
evaluation with users was very good and about 60% rated field studies and digital
prototyping as very good. Around half of the respondents said usability goals, lo-fi
prototyping worked very well. No participant rated questionnaires as a very good
technique.
If the ratings “Very good and Fairly good are combined, the top five rated
usability techniques used by IT practitioners are: 1) workshops 2) informal usability
evaluation with users 3) meetings with users 4) scenarios and 5) formal usability
evaluation with users.
Usability techniques
Very
good
Fairly
good
Neither
good or
bad
Fairly
bad
Very
bad
N*
28%
60%
8%
4%
0%
25
0%
33%
56%
11%
0%
9
38%
62%
0%
0%
0%
30
38%
57%
5%
0%
0%
21
59%
29%
12%
0%
0%
17
53%
20%
27%
0%
0%
15
35%
59%
0%
6%
0%
17
40%
40%
13%
7%
0%
15
59%
30%
12%
0%
0%
17
50%
25%
20%
5%
0%
20
73%
18%
9%
0%
0%
11
25%
75%
0%
0%
0%
8
25%
50%
0%
25%
0%
4
8 Yuan Jia1, Marta Kristin Larusdottir2 and Åsa Cajander3
4.4 Correlation of the Usage of Usability Techniques
Some significant and interesting correlations were found among the usage of usability
techniques in Scrum projects. Results in Table 4 shows that there were totally six
usability techniques, which had significant correlations with field studies. These
techniques are interviews, workshops, meetings with users, personas, lo-fi prototyping
and heuristic evaluation. Interviews are significantly correlated with meetings with
users. Questionnaires are significantly correlated with digital prototyping, formal
usability evaluation with users and informal usability evaluation with users.
Workshops are significantly correlated with personas. Usability goals are significantly
correlated with scenarios. Digital prototyping is significantly correlated with heuristic
evaluation. Lo-fi prototyping is significantly correlated with personas and formal
usability evaluation with users.
Table 4 Correlation of the Usage of Usability Techniques
Interviews
Questionnaires
Workshops
Meetings with user
Field study
Usability goals
Scenarios
Personas
Digital prototyping
Lo-fi prototyping
Formal usability evaluation
Informal usability evaluation
Heuristic evaluation
Interviews
+
*
Questionnaires
+
*
+
Workshops
+
*
Meetings with users
+
+
Field studies
*
+
+
+
+
*
Usability goals
+
Scenarios
+
Personas
*
+
+
Digital prototyping
+
+
Lo-fi prototyping
+
+
+
Formal usability
evaluation
*
+
Informal usability
evaluation
+
Heuristic
evaluation
*
+
+ Represents that the correlation is significant at the 0.05 level
* Represents that the correlation is significant at the 0.01 level
Error! No text of specified style in document. 9
Some of these techniques are related in character like meetings with users and
interviews could be conducted in a similar way. Furthermore, interviews are often
conducted in the field, so the correlation between field studies and interviews could
be expected. On the contrary, the correlation between field studies and heuristic
evaluation is rather surprising. The fact that there is a correlation between using the
techniques does not mean that these techniques are necessarily used at the same time,
but that an IT professional is more likely to conduct heuristic evaluation if that person
conducts field studies. Furthermore, a correlation between formal usability evaluation
with users and digital prototypes could be expected, but formal evaluations are
correlated only with lo-fi prototypes. Questionnaires seem to be used in correlation
with evaluation with users, both during formal and informal evaluation. Usability
goals seem to be stated in correlation with writing scenarios.
5 Discussion
The survey asked the respondents to rate both the usefulness and how often they use
the 13 usability techniques. The results show that the top five rated usability
techniques used by practitioners are: 1) workshops 2) informal usability evaluation
with users 3) meetings with users 4) scenarios and 5) formal usability evaluation with
users. This result is partly consistent with the practitioners’ perception of the
effectiveness of usability techniques in software development projects presented by
Gulliksen et al. [4]. The top five rated techniques in that study were: 1) the think-
aloud method 2) lo-fi prototyping 3) interviews 4) field studies and 5) scenarios.
Our results show that the top five frequently used usability techniques by IT
professionals are: 1) workshops 2) lo-fi prototyping 3) interviews 4) meeting with
users and 5) scenarios. This result is also partly consistent with the study from Venturi
et al. [11]. Their top five frequently used usability techniques are: 1) user interviews
2) heuristic evaluation 3) qualitative usability evaluations 4) hi-fi prototyping and
5) lo-fi prototyping.
When comparing our results on the top five rated and frequently used usability
techniques with those of other studies, it is difficult to draw conclusions regarding the
differences. First, the study presented in this paper provided an exact description for
each usability technique in the survey, which was not done in the other studies. It may
not be viable to compare two ranks of usability techniques when the researchers and
the respondents may have had different understanding of what it includes to use a
particular usability technique. Secondly, the usability techniques used in the different
studies were not the same. For example, workshops are rated as the top usability
technique in this study, but workshops are not included in the study by Gulliksen et al.
[4]. Moreover, all the usability techniques mentioned in this study were used in
software project using the software development process Scrum, but the other two
papers are about software development in general.
However, despite these differences some interesting things are worth noticing
when comparing the results from the studies. The results from this study show that
heuristic evaluation is not often used but the techniques get reasonable ranking in this
study and the results from Venturi et al. [11] show that heuristic evaluation was one
10 Yuan Jia1, Marta Kristin Larusdottir2 and Åsa Cajander3
of the top five frequently used techniques. One possible explanation to this difference
is that heuristic evaluation may not fit well in the Scrum process. Another possible
reason is that the usage of heuristic evaluation as an evaluation method has been
decreasing in software development in general, and that the survey results from this
study, which was conducted in 2011, indicate a general trend.
The usefulness rated by the participants and the frequency of using the techniques
are not correlated. For example, about 75% of the respondents consider the technique
formal usability evaluation with users very good, but only about 30% used it in their
projects. One possible explanation for this inconsistency is that the most important
characteristic of Scrum is speed. The duration of one Scrum sprint usually last two to
four weeks, but sometimes practitioners need longer time to use a particular usability
technique. It usually takes practitioners long time to prepare formal usability testing,
recruit participants and conduct the tests, for example.
The techniques digital prototyping and field studies were also rated highly and
used two to six times a month. Still, these techniques were only used by one third of
the respondents. The technique lo-fi prototyping is more frequently used than the
technique digital prototyping. Still the usefulness of digital prototypes was higher
ranked than the usefulness of lo-fi prototyping. One of the main advantages of lo-fi
prototyping is its quickness and accessibility to any team member in the development
process. One explanation could be the focus on speed during Scrum projects.
As shown in the results of the correlation among usability techniques, field studies
are significantly correlated with interviews, workshops, meetings with users,
personas, lo-fi prototyping and heuristic evaluation. Field studies can help collecting
and preparing interview questions and the technique interviews also can provide some
valuable information for the process of the field studies. The correlation between field
studies and heuristic evaluation is more surprising, because these types of evaluations
are rarely conducted in the field. Furthermore, the knowledge that is needed to use the
two techniques is not that related. Another technique formal usability evaluation
with users has strong correlation with the technique questionnaires. One reasonable
explanation is that questionnaires can be used to prepare the test tasks or as a post-test
survey for the formal usability evaluation with users. Formal evaluation has also a
correlation with lo-fi prototypes, which is positive, because it has been suggested in
the HCI literature for many years that evaluations should start as soon as possible
during the software development.
6 Conclusions
This study investigates the integration of usability techniques in software projects
using the Scrum process in industry. The findings from the study indicate that the
technique workshops are the most commonly used usability technique in Scrum
projects, followed by: lo-fi prototyping, interviews, meetings with users and
scenarios. Besides, the top five highest rated usability techniques according to
usefulness for the IT professionals are: 1) workshops 2) informal usability evaluation
with users 3) meetings with users 4) scenarios and 5) formal usability evaluation with
users. A novel contribution of this study is that there were significant correlations of
Error! No text of specified style in document. 11
the usage among different usability techniques in Scrum projects. If IT professionals
use workshops they are more likely to use personas is an example of these
correlations.
7 Acknowledgements
We would like to thank all the participants in the study that took their time in
answering the questionnaire. Additionally, we would like to thank the PhD student
Simon Tschirner and the former PhD students Niklas Hardenborg and Stefan
Blomkvist in Human Computer Interaction in Uppsala University for taking part in
piloting the questionnaire. Furthermore, we would like to thank the research and
teaching assistant Tao Yang at School of Informatics in Indiana University in U.S. for
his valuable comments when reviewing the paper.
References
1. Abrahamsson, P., Warsta, J., Siponen, M.T., Ronkainen, J.: New Directions on Agile
Methods: a Comparative Analysis. In: Proceedings of the 25th International Conference on
Software Engineering. IEEE Computer Society, Los Alamitos (2003)
2. Bark, I., Følstad, A., Gulliksen, J., McEwan, T. and Benyon, D.: Use and usefulness of HCI
methods: Results from an exploratory study among nordic HCI practitioners, In: People
and Computers XIX - The bigger picture. London: Springer, 201-217 (2006)
3. Bygstad, B., Ghinea, G. & Brevik, E.: Software development methods and usability:
Perspectives from a survey in the software industry in Norway. Interacting with Computers,
20 (3), 375-385 (2008)
4. Gulliksen, J., Boivie, I., Persson, J., Hektor, A. & Herulf, L.: Making a Difference: A
Survey of The Usability Profession in Sweden. In: Proceedings of NordiCHI 2004,
Tampere, Finland 207-215 (2004)
5. Larusdottir, M.K., Haraldsdottir, O. and Mikkelsen, B.: User involvement in Icelandic
software industry. In: Proceedings of the I-Used 2009 workshop at INTERACT 2009,
Uppsala, Sweden, 51 52 (2009)
6. Monahan, K., Lahteenmaki, M., Mcdonald, S. & Cockton, G.: An Investigation into the use
of Field Methods in the Design and Evaluation of Interactive Systems. In: Proceedings of
the 22nd British HCI Group Annual Conference, People and Computers, 99-108 (2008)
7. Schwaber, K.: Scrum development process. OOPSLA’95 Workshop on Business Object
Design and Implementation (1995)
8. Singh, M.: U-SCRUM: An Agile Methodology for Promoting Usability. In: Proceedings of
the AGILE 2008 conference, Toronto, Canada, 555-560 (2008)
9. Sohaib, O. and Khan, K.: Integrating usability engineering and agile software development:
A literature review. In: Proceedings of the ICCDA 2010 - International conference on
computer design and applications, Qinhuangdao, Heibei, China (2010)
10. Venturi, G. & Troost, J.: Survey on the UCD integration in the industry. In: Proceedings of
the NordiCHI 2004 conference, Tampere, Finland, 449-452 (2004)
11. Venturi, T., Troost, J., & Jokela, T.: People, organizations, and processes: An Inquiry into
The Adoption of User-Centered Design in Industry. International Journal of Human-
Computer Interaction, 21(2), 219-238. (2006)
... This paper contributes knowledge about User eXperience Design (UXD) approaches within Agile processes, abbreviated as Agile UXD. Agile UXD has been studied over the last twenty years [1122,38,46,48]. A literature study from 2015 found 83 published papers on usability and Agile [8]. ...
... The most frequently used methods for gathering usability requirements were practice from previous projects and interviews with users. Survey results from 2012 show that the most frequently used UX methods in Scrum projects were: workshops, low fidelity prototyping, interviews and meetings with users [48]. The highest-rated method in this study was formal usability testing, even though the professionals in the study did not frequently use that method because of time constraints. ...
... Agile UXD has been studied over the last twenty years [1122,38,46,48]. A literature study from 2015 found 83 published papers on usability and Agile [8]. ...
Article
Context : The usage of User Experience (UX) methods has been studied through the years. However, little is known about UX professionals’ lifelong learning processes related to UX methods in Agile, choosing what UX methods to use, and the enablers and hindrances for using the UX methods. Objective : The study aims to broaden current knowledge about UX professionals’ lifelong learning practices to understand their work situations better. The paper describes how UX professionals learn about and choose UX methods, their frequency of use, and the enablers and barriers when using the UX methods in Agile. Method : An interview study was conducted with 13 UX professionals from various industries and two countries working with Agile and UX. We used a qualitative approach, and a thematic analysis was carried out to answer the research questions. Results : The results show that support from colleagues is an essential component for learning about the methods and how to use UX methods. Time pressure makes UX professionals choose methods they know will deliver their desired results. Prototyping, user testing, user journeys, and workshops are the most frequently used UX methods. Additionally, the results show that UX professionals think that the UX methods are often too complicated and take too long to learn. Additionally, they find it challenging to integrate UX methods into Agile. Conclusion : These findings indicate that UX methods might work better if designed to be less complicated and deliver results more efficiently. Moreover, collegial and peer learning is central to UX professionals. The HCI community could be more active in supporting this culture by sharing information and learning. Finally, the usability and UX of the tools affect which UX methods are used.
... Others have criticized that personas are not used that much during the software development and therefore there is even less reason to take on this time-consuming task. A study on the usage of UCD methods shows that personas are not frequently used by IT professionals even though the methods is quite highly ranked as being useful [16]. ...
Chapter
Full-text available
Personas is a suggested method to extend IT professionals' understanding of users and users’ needs. A common advantage expressed is that personas extend the IT professionals’ empathy for the users, but a disadvantage is that personas are typically defined at the start of a software project and gradually are forgotten, since there is little reference to the personas through the software development project. In this paper we report experiences of coaching IT professionals in defining agile user stories based on personas, called: Persona User Stories (PUS). The aim of these workshops, was to extend the usage of personas and thereby extend the IT professionals’ understanding of their users. In a research project with three companies, we coached teams of IT professionals in three-hour workshops with 76 participants in total. The workshops were conducted at each company using personas already defined by the IT professionals. The persona descriptions were based on three types of information: (a) assumptions, (b) secondary research, and (c) data specific to a project. Our findings show that personas based on assumptions result in the participants questioning the description of the personas and having difficulties in understanding the personas. For making the persona user stories (PUS), the participants used themselves more often as a reference when working with the assumption based personas, than the participants using the other two types of personas.
... Encouraging a more agile approach could support increased communication by shortening the time between development and test. However, at the same time caution is needed, as others have indicated that in agile projects more informal usability methods like workshops tend to be used [19]. In addition, an increased awareness is needed about the fact that different stakeholders have different needs and interests, so that involving just one group is not sufficient. ...
Conference Paper
Full-text available
This paper summarizes the results of an online study on stakeholder involvement, user-centered design methods, and communication practices. Overall, 27 Austrian organizations actively involved in innovation projects focusing on the development of solutions in the field of Active and Assisted Living (AAL) took part in the study. The results reveal a wide involvement of various stakeholders in most projects, however insurance companies and public authorities are rarely actively involved. With regard to methodological approaches, our results show a high literacy on advanced user research methods, but only a limited variety of methods actually applied. Finding a common language and choosing the right ways to communicate user requirements have been identified as a necessity for successful collaboration within AAL project teams.
... Some of the methods for focusing on either the expected UX or the UX after users have used a particular system, including interviews with users, surveys, observations and user testing [19]. IT professionals rated formal user testing as the most useful method for active participation of users in their software development for understanding the UX of the developed system [11]. ...
... At the same time, it is generally known that practitioners combine different HCD methods in real-world development projects. This is a common point of departure in practitioner guidebooks (e.g., Hall, 2013;Hyysalo, 2010;Kuniavsky, 2003;Sharon, 2012), but there is surprisingly little academic research seeking to establish what kinds of method mixes are used and which combinations may be sufficient or even optimal in different projects (Jia, Larusdottir, & Cajander, 2012;Johnson et al., 2014a;Johnson et al., 2014b;Mäkinen, Hyysalo, & Johnson, 2018;van Turnhout et al., 2014). ...
Article
Full-text available
Human-centered design (HCD) has developed an impressive number of methods for gaining a better understanding of the users throughout the design process. The dominant orientation in HCD research has been to develop and validate individual methods. However, there has been growing amount of criticism towards this dominant orientation, as companies and designers seldom design services or products as entirely separate projects, let alone use single methods for doing so. Our longitudinal case study that is based on interviews, meeting observations, and company documentation was conducted at a high HCD-mature company. The study shows that instead of conducting new user research or testing for each project, designers draw information from previous studies and other user insight sources in the company. HCD work is mostly accomplished through a combination of methods and other information sources on the users. The cumulation of user knowledge gained during the past projects and employment years is notably high among designers, product managers, projects, and in the company as a whole. In addition, knowledge based on user research and HCD methods does not replace other sources such as customer insight from marketing but, rather, complements these. The chosen approach of studying method mixes in an organization provided useful insights into understanding the user information sources in an organization.
Article
Full-text available
Context Software development companies use Agile methods to develop their products or services efficiently and in a goal-oriented way. But this alone is not enough to satisfy user demands today. It is much more important nowadays that a product or service should offer a great user experience—the user wants to have some positive user experience while interacting with the product or service. Objective An essential requirement is the integration of user experience methods in Agile software development. Based on this, the development of positive user experience must be managed. We understand management in general as a combination of a goal, a strategy, and resources. When applied to UX, user experience management consists of a UX goal, a UX strategy, and UX resources. Method We have conducted a systematic literature review (SLR) to analyse suitable approaches for managing user experience in the context of Agile software development. Results We have identified 49 relevant studies in this regard. After analysing the studies in detail, we have identified different primary approaches that can be deemed suitable for UX management. Additionally, we have identified several UX methods that are used in combination with the primary approaches. Conclusions However, we could not identify any approaches that directly address UX management. There is also no general definition or common understanding of UX management. To successfully implement UX management, it is important to know what UX management actually is and how to measure or determine successful UX management.
Article
Full-text available
Previous research has established that leaders in information and communication technology (ICT) are crucial for establishing a user-centred systems design perspective in ICT for work-related tasks. This paper therefore describes the perspectives of 18 ICT leaders in three kinds of leadership roles (managers, project leaders and specialists) in order to understand their views of user-centred systems design concerning ICT. It uses the concept of technological frames of reference to analyse three domains: technology-in-use, technology strategy and nature of technology. The results show that many specialists see user involvement as a critical factor in successfully establishing new information and communication technologies, but that these systems are currently built around the needs of management rather than end users. Looking forward, all three relevant social groups are optimistic about how ICT will become more user-centred and more strategically aligned in the future. However, changes in ICT are described as extremely energy-consuming and difficult – akin to ‘walking in the jungle with a machete’. Finally, we discuss the relevance of technological frames and present some implications for the successful establishment of user-centred system design as a perspective in organisations.
Chapter
Personas are a popular method of user centered design (UCD) in technical product development and have become indispensable in software development. In agile software development, which is nowadays predominantly used in modern projects, personas do not have a role of their own. User needs are usually introduced into development via the Product Owner role by means of user stories. Especially when software is developed for user groups with special needs, it is often difficult for developers to understand the needs of the users based on common user stories.
Book
Full-text available
The INTERACT Conferences are an important platform for researchers and practitioners in the field of human-computer interaction (HCI) to showcase their work. They are organised biennially by the International Federation for Information Processing Technical Committee on Human–Computer Interaction (IFIP TC13), a committee of 30 member national societies and 9 Working Groups. The 17th IFIP TC13 International Conference on Human-Computer Interaction (INTERACT 2019) took place during 2-6 September 2019 in Paphos, Cyprus. The conference was held at the Coral Beach Hotel Resort, and was co-sponsored by the Cyprus University of Technology and Tallinn University, in cooperation with ACM and ACM SIGCHI. With an emphasis on inclusiveness, these conferences work to lower the barriers that prevent people in developing countries from participating in conferences. As a multidisciplinary field, HCI requires interaction and discussion among diverse people with different interests and backgrounds. This volume contains the Adjunct Proceedings to the 17th INTERACT Conference, and comprises a series of papers from the workshops. It follows the INTERACT Conference tradition of the publication of adjunct proceedings by a University Press which has a connection to the conference itself. This tradition has been established to enhance the outreach and reputation of the University Press chosen. For INTERACT 2019, both the Conference Program Chair, Dr Fernando Loizides, and the Adjunct Proceedings Chair of the conference, Dr Usashi Chatterjee, work at Cardiff University which is the home of Cardiff University Press.
Article
Full-text available
This paper reports the first results from a recent study done on user involvement in Icelandic software industry. A questionnaire survey was made to gather information on the software processes used and to what extent user involvement methods are used by software developers in the different processes. The results show that the majority of the respondents use their own process where they have adjusted their development process to their needs. More than one third of the respondents use the agile process Scrum. That group is the most skeptical one when rating the importance of usability in software development. Meetings are the most popular method for involving users.
Article
Full-text available
The aim of this article is to improve our understanding of user-centered design (UCD) adoption and provide accordingly useful advice to the UCD community. UCD adoption was investigated through a Web survey. The results show that the early involvement of UCD practitioners in the product life cycle is more frequent compared to 10 years ago. It is also true that the methods and the techniques employed have shifted their focus from summative evaluation to rapid development cycles and from quantitative to qualitative evaluation methods. Based on the survey, there are several organizational factors UCD practitioners and their management should consider. UCD should be part of the business strategy and supported by higher management. Usability goals must be set through competitive analysis and practitioners should be rewarded if goals are reached or exceeded. For bespoke systems, usability goals should be explicitly discussed with the customer. Special attention should be paid to communication inside and outside the company so as to clarify the outcomes and benefits of the UCD approach.
Conference Paper
Full-text available
As an HCI practitioner, it would be of great value to know which methods other HCI practitioners find most useful in different project phases. Also it would be interesting to know whether the type of ICT projects has any effects on HCI practitioners’ perception of the usefulness of the methods. This paper presents results from an exploratory survey of HCI practitioners in the Nordic countries conducted in the fall of 2004. 179 of the respondents were usability professionals or UI designers with two or more years of experience. The survey results give insights with regard to whether or not HCI practitioners are included in those project phases regarded as most important. Also it describes which HCI methods that are used in different project phases, and how useful different HCI methods are perceived to be. The study complements existing HCI practitioner survey investigations by an explicit allocation of the HCI methods under consideration to concrete project phases, and by including analyses of group differences between practitioners working with different kinds of development projects.
Conference Paper
Full-text available
This paper reports the results of an international web-based survey on the use of field studies in the design and evaluation of interactive systems, which was conducted between December 2006 and February 2007. The results suggest that the advantages and disadvantages of field methods are generally well understood, but guidance is needed in their application and use. Field studies were most frequently used for understanding context, and respondents preferred a more varied approach to method use rather than following a defined methodology such as Contextual Design. Observations were rated as the most effective technique overall, although interviews appeared to be more frequently used. Significant areas of further improvement for field methods were identified as improvements in data collection/analysis tools and improvements in adaptability of methods.
Conference Paper
Full-text available
Poor usability in interactive systems/products is still a major problem for users and buyers, despite efforts made by an increasing number of usability professionals. How come this is so and what are the main obstacles to usability work?
Conference Paper
The various agile software development methodologies have promoted since their inception and even demanded high degree to improve the quality of the software product. Usability engineering has made its way into the software mainstream and has caught the attention of software engineers and researchers worldwide due to rapidly growing and volatile internet software industry, despite their different perspectives on creating software both have a major role in making good software. Usability focuses on how the end users will work with the software and agile development focuses on how the software should be developed. The way these two perspectives are being combined in practice is still not well understood. This study is a preliminary literature review that describes the key question that how usability-engineering practices should be integrated with agile software development in order to make stronger and effective usable software system. This paper focuses on identifying the tensions between usability and agile methods. The research aim is to identify the common approach of agile methods and usability engineering by surveying extensive related work on integration of usability and agile methods.
Article
This paper investigates the relationship between software development methodologies and usability. The point of departure is the assumption that two important disciplines in software development, one of software development methods (SDMs) and one of usability work, are not integrated in industrial software projects. Building on previous research we investigate two questions; (1) Will software companies generally acknowledge the importance of usability, but not prioritise it in industrial projects? and (2) To what degree are software development methods and usability perceived by practitioners as being integrated? To this end a survey in the Norwegian IT industry was conducted. From a sample of 259 companies we received responses from 78 companies. In response to our first research question, our findings show that although there is a positive bias towards usability, the importance of usability testing is perceived to be much less than that of usability requirements. Given the strong time and cost pressures associated with the software industry, we believe that these results highlight that there is a gap between intention and reality. Regarding our second research question our survey revealed that companies perceive usability and software development methods to be integrated. This is in contrast to earlier research, which, somewhat pessimistically, has argued for the existence of two different cultures, one of software development and one of usability. The findings give hope for the future, in particular because the general use of system development methods are pragmatic and adaptable.
Conference Paper
The primary contribution of this paper is investigating how the User Centered Design approach is integrated into the industry. Employing a structured web-survey, targeted to the usability practitioners, we find out that UCD is particularly employed in big companies, but with a relatively low ratio: practitioners represent less than one percent of the company employees. User interviews and both low and high fidelity prototyping are the most frequently used techniques. We eventually validate our hypothesis that UCD integration is facilitated by factors related to management support, infrastructure and communication; companies interested in producing better usable and fit-for-use products should take all of these issues into serious consideration.