'Include', a Toolbox of User Research for Inclusive Design

Abstract

In order to empower more people to become more self-reliant in society, interactive products and services should better match the skills and values of diverse user groups. In inclusive design, relevant end-user groups are involved early on and throughout the design and development process, leading to a better user experience. However, for IT businesses not operating in the academic domain, getting access to appropriate user research methods is difficult. This paper describes the design and prototype development of the Include Toolbox, in close cooperation with practitioners of small to medium sized enterprises (SMEs) in IT. It consists of an interactive app paired with a book. The app helps to find suitable research methods for diverse user groups such as older people, people with low literacy, and children. The book offers background information on the advantages of inclusive design, information on different user groups, and best practices shared by other companies.

Full text

‘Include’, a Toolbox of User Research for Inclusive Design
Hanna Zoon
Eindhoven Technical University
Department of Industrial Design
P.O. Box 513, 5600 MB
Eindhoven, The Netherlands
info@hannazoon.com
Anita Cremers
TNO
P.O. Box 23, 3769 ZG
Soesterberg, The Netherlands
a.cremers@tno.nl
Berry Eggen
Eindhoven Technical University
Department of Industrial Design
P.O. Box 513, 5600 MB
Eindhoven, The Netherlands
j.h.eggen@tue.nl
+31 40 247 5227
ABSTRACT
In order to empower more people to become more self-
reliant in society, interactive products and services should
better match the skills and values of diverse user groups. In
inclusive design, relevant end-user groups are involved
early on and throughout the design and development
process, leading to a better user experience. However, for
IT businesses not operating in the academic domain, getting
access to appropriate user research methods is difficult.
This paper describes the design and prototype development
of the Include Toolbox, in close cooperation with
practitioners of small to medium sized enterprises (SMEs)
in IT. It consists of an interactive app paired with a book.
The app helps to find suitable research methods for diverse
user groups such as older people, people with low literacy,
and children. The book offers background information on
the advantages of inclusive design, information on different
user groups, and best practices shared by other companies.
Author Keywords
Inclusive design; user research methods; low literacy;
elderly; children; SMEs; IT; societal participation
ACM Classification Keywords
H.5.m. Information interfaces and presentation (e.g., HCI):
Miscellaneous.
INTRODUCTION
The world changes at an amazing pace, and technological
innovations happen faster and faster [1]. Behaviour, needs,
wants and likes of people also change, and many user
groups are less homogenous than before; fewer and fewer
people can be characterised by a traditional stereotype [31].
Older people: a 70-year old internet pioneer taking
programming classes, and a 90-year old going shopping by
bicycle. Disabled people: a 16 year old teen who battles
BMX riders in the skatepark and calls himself an extreme
sportsman, but also happens to be in a wheelchair [33].
Businesses cannot rely on existing knowledge alone; they
need increasingly up-to-date insights into the perception
and behaviour of their diverse customers, and need
inspiration on how to shape the future of their enterprise.
As public services and products become more interactive
and are increasingly offered solely online, it is important to
not exclude anyone from using these services. In many
countries, there is already legislation to ensure access to
essential online services and products for everyone, for
example ensuring a blind person can buy a train ticket, or a
low literate person can apply for benefits online. But also
for less essential interactive services and products, it can be
an advantage to design them in an inclusive way, taking
into account the diversity of the user group.
Inclusive design can be defined as the design of mainstream
products or services that are accessible to, and usable by as
many people as reasonably possible [20]. In inclusive
design, relevant end-user groups are involved early on [29]
and throughout the design and development process [3],
leading to more varied inspiration and more user-friendly
solutions. Making a product more inclusive, usually means
better usability for everyone [7]. It is not only a way to
solve problems, but also a strategy to identify problems to
solve. With the growing importance of brand image and
sustainable business practice, inclusive design can be a
selling point and may broaden the potential customer base.
It can seem easier and cheaper to self-reference than to
involve users. But to cater an application to a 28-year-old
male programmer, is actually designing for a tiny minority.
It can be very costly to launch an application and only then
discover it does not suit the people who were intended to
use it. From our previous research, it can be concluded that
awareness of inclusive design is still lacking. Unfamiliarity
with its advantages and methods seems to be a barrier to
practicing it.
User involvement does not need to be difficult, time-
consuming or expensive. For a business seeking inspiration
from customers, or wanting to get a feel of the perception of
their product by users, smaller informal sessions suit an
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise,
or republish, to post on servers or to redistribute to lists, requires prior
specific permission and/or a fee.
Copyright is held by the author(s).
Published in: van Leeuwen, JP, Stappers, PJ, Lamers, MH, Thissen,
MJMR (Eds.) Creating the Difference: Proceedings of the Chi Sparks
2014 Conference, April 3, 2014, The Hague, The Netherlands.

iterative design process much better and are more practical.
Sometimes, even one participant can be enough to get
inspiration for new business ideas [26], and a session with
five participants can uncover most of the general attitudes
about a subject [6]. User research methods are generally
very robust and even shorter, simplified execution will give
usable results.
It can be challenging to find a user involvement method
suitable for the goal of a project, time frame, resources and
the type of target users. Academic as well as popular
literature describes hundreds of research, ethnography and
usability methods [Sanders, E.B.N., personal communica-
tion], [4]. Especially for businesses not operating in the
academic domain, getting access to and finding the right
research methods can be a challenging task, that is often not
even started. As a result, users are often not included in the
development process.
There are many websites and toolkits available that offer
techniques to help getting new inspiration by thinking
differently, and tools for visual design. There are only a few
that offer user involvement methods, or are designed for use
by practitioners in the information technology domain (IT).
In this paper, we will describe the design and prototype
development of a toolbox for inclusive design and end-user
involvement, during which we worked closely together with
practitioners of small to medium sized enterprises (SMEs)
in the field of IT, that had little or no prior experience with
user research.
The Include Toolbox prototype that resulted from this
cooperation consists of an interactive app paired with a
book. The Include App (Figure 1) helps to find suitable
research methods for diverse user groups such as older
people, people with low literacy, and children. The Include
Book offers background information on the advantages of
Inclusive Design, information on different user groups, and
describes best practices shared by other companies.
The Include Toolbox can help bridge the knowledge gap
between academia and practice, and make Inclusive Design
possible for more businesses. Ultimately, the toolbox could
lead to interactive products and services that match the
skills and values of diverse target groups better, in order to
empower these people to become more self-reliant in
society.
Figure 1. Home page of the Include Toolbox App (below)

RELATED WORK
Amongst the many toolkits available for designers, most
offer inspiration techniques and tools for visual or product
design. There are only a few that offer user involvement
methods, of which a selection is described here.
IDEO cards [17]
The classic IDEO method cards are widely used and the
methods on them became accepted practice in a lot of
businesses. The deck of 51 sturdy cards, developed by the
IDEO design firm, describes mostly inspirational methods
and also some user involvement methods, and are
specifically geared towards product designers. Already over
10 years old, it is one of the most well-known collections of
methods and is often used as basis for other card sets and
toolboxes, such as HCD Connect and Designing with
People (see below). On the cards is a colour-coded system
that helps designers to pick a suitable method. Of the
toolkits described here, it is the only one that is not free.
HCD Connect - Bill & Melinda Gates Foundation / IDEO
[14]
The Human-Centered Design Toolkit was designed
specifically for people, nonprofits, and social enterprises
that work with low-income communities throughout the
world. The HCD Toolkit walks users through a human-
centred design process and offers design- as well as user
involvement methods, bases on the IDEO cards. It is
divided in three parts: Hear, Create, Deliver; with the
appropriate methods listed under pictograms.
Designing with People - Helen Hamlyn Centre for
Design, Royal College of Art [11]
Designing with people is a website with very complete
information on research methods, design methods, user
groups background and stories, personas, and tips on doing
user research. It is set up in a clear way, so that it is easy to
decide which part of the information to view. The research
methods that are available in this toolbox, are based on the
IDEO cards and described in a compact way but further
references are also given. The Designing with People
website is one of the outputs from i~design 3, the final
phase of a collaborative research programme on inclusive
design funded by the Engineering and Physical Sciences
Research Council. It has been designed to work in
conjunction with the Inclusive Design Toolkit developed by
project partner, the Engineering Design Centre at
Cambridge University.
Inclusive Design Toolkit - Engineering Design Centre,
Cambridge University [15]
The Inclusive Design Toolkit, next to explaining inclusive
design and offering business rationale and design patterns,
offers physical tools to assist inclusive design, including a
Vision and Hearing Impairment Simulator, Cambridge
Simulation Gloves and Glasses, and an Exclusion
calculator. The design patterns mention user involvement
methods but for descriptions refers to the closely linked
Designing with People website. The Engineering Design
Centre also offers courses on inclusive design.
UCD Toolbox - Tristan Weevers UCD / Delft University of
Technology [12]
The UCD Toolbox is a very complete resource of methods
for user centred design. It contains an overview of 35
design methods, which can be filtered by criteria: type of
product, design goal, resources, participants and method
characteristics. Also, a pre-selection of methods can be
made for various target groups: elderly, children, physically
challenged, visual/hearing impaired or cognitively
challenged. However, no background information on
specific target groups is offered and it does not become
clear why the methods are suitable for the target groups.
Method descriptions contain a lot of information and it is
not always clear how much time, skill and effort a method
will take. Because of the extensiveness of this toolbox, it
can be hard for IT practitioners to find the specific
information they need.
UX Toolbox: Better Web for Citizens - British Columbia
Government [16]
A resource for government interaction and web designers,
the UX Toolbox is a complete manual containing
information on user experience, design research, web
strategy, information architecture, content design, and web
standards. The design research section describes research
methods in detail, some of which involve end-users. It
includes, research plans, reporting and managing tips. The
methods are sorted into categories based on the type of
research method, of which the terminology can be hard to
understand for non-experienced researchers. The UX
Toolbox also has a team that practitioners can contact for
information and help.
55plusToolbox – Saxion [13]
The 55plusToolbox (in Dutch) focuses on topics that
change the innovation process as a result of choosing the
user group of people over the age of 55. Target users of the
toolbox are entrepreneurs, focusing on both product
development and marketing and sales. It contains
information on the user group as well as case studies.
Suitable tools for the particular phase and user group are
suggested and illustrated in factsheets containing step by
step guidance, visualisations, relevant links and references.
Universal Methods of Design - Bruce Hanington, Bella
Martin [10]
The fact that Universal Methods of Design is a physical
book, allows for another kind of interaction. When flipping
through the pages, one gets a clear overview but not too
much information at once, even though the book describes a
total of 100 methods. The book contains design-, research-
and user involvement methods, ordered alphabetically and
coded for each design phase. The target users of the book

are designers, but it is written in a very accessible way.
There is a short description given for each method,
completed with references and examples.
All toolkits and related work mentioned above are in theory
also usable for IT practitioners. However, because of
factors such as terminology and emphasis on creative
techniques, some are clearly meant to be used by designers,
not programmers. The British Columbia UX Toolbox is
made for the IT-practitioner target group, but requires
previous knowledge of and experience with user research.
In most existing toolboxes, it can be difficult to find a
suitable method, sometimes because there are so many,
sometimes because it is difficult to get an overview [28].
Often exact descriptions of a method are not given,
sometimes there are references to this information. But
even then it is often still hard to determine the level of
difficulty of the method, and the amount of time it will take.
Few toolboxes offer sufficient background information on
the needs and abilities of different target groups. Many of
the resources mentioned above, cover some aspect of what
we think an inclusive design toolbox needs to offer, but
there is not yet a resource available that caters specifically
to the needs and wishes of practitioners of SMEs in IT.
REQUIREMENTS TOOLBOX
In earlier research [3], a workshop setting was used to
gather preliminary requirements for an inclusive design
toolbox, which formed the basis for this project.
To further define and refine the requirements, we worked
together with a total of 11 practitioners from seven SMEs in
IT. The project was set up in an Agile way [18] with many
short iterations. In the earlier stages of the project, focused
interviews were conducted about existing business
processes and views on inclusive design. In later stages,
two practitioners worked with the toolbox in their projects
[19] (Figure 2), providing feedback on the practical applica-
Figure 2. Practitioner performing user research using the
Business Origami method from the Include Toolbox
bility, while others provided feedback on design and
business aspects. Throughout the project, new companies
were introduced to get fresh insights from people who had
never seen the toolbox before. Five companies were smaller
(5-20 employees) privately owned businesses, one
practitioner was a PhD. student in the field of computer
science and one company was medium-sized (~200
employees), all based in the Netherlands. The business
activities included designing, developing and building
websites, intranets, e-learning applications and apps, and
consultancy.
In the beginning of the project, a very persistent attitude we
saw in the companies was that user research would be
difficult, expensive and not useful. Statements such as
‘users do not know what they want’ and ‘they never say
anything useful’ were heard. With further questioning, it
became clear that the people we interviewed were more
frustrated with the lack of success, than convinced of the
uselessness of user research. For example, they hated it
when their app for helping elderly using public transit came
on the market but was rejected by the target group, who
claimed they did not need help. The company was now
wondering if working with older people in a more
structured way from the beginning, could have prevented
the need for an expensive redesign. Another business owner
saw the business opportunities of making a clients website
more inclusive for elderly, attracting more customers that
way.
By working closely together with practitioners and owners
from these and other companies, and using the knowledge
gathered in our earlier research, requirements were refined
and tested through prototyping throughout the project. This
process allowed for a thorough understanding of the needs
and business practice of our target user group, SMEs in IT.
It also made it possible to adapt and change the design and
setup of the toolbox continuously, to suit the requirements
more optimally.
The requirements we found for an Inclusive Design
Toolbox for SMEs in IT:
1. Easy to find the right method
Especially because it is so difficult for most people, and
therefore also end-users, to verbalise needs and imagine
products or services that do not yet exist, it helps to employ
a structured process for this task [25]. For academics,
searching a scientific literature library is easy. Finding a
specific, suitable method without knowing its name is
already more difficult. For non-academics there are books
and websites on user research, but because this is such a big
field, the amount of information is typically very large, and
the information is often geared towards experienced
researchers, not IT practitioners. The practitioners from our
interviews were often not aware of the existence of
scientific libraries or that there are many different user
research methods.

2. Decision aid versus freedom of choice
When looking for suitable user research methods, practi-
tioners wanted the autonomy to explore the methods
themselves first, to see what there was on offer. Only after
that, they would request toolbox recommendations. It was
mentioned many times that they would like to keep their
options open and choose themselves, and have the freedom
to deviate from recommended methods.
3. Methods: fast, robust and cheap
For a scientific research project, it can make perfect sense
to do a three-week long experiment with 120 participants.
But in the IT world, with Agile development cycles, the
time or budgets for this are often simply not there. And as
Nielsen [23] and Dix [6] argued, with about 3 to 5 people,
you will see most of the common behaviours from an entire
group, and each extra participant gives only limited extra
insights. Limiting the amount of participants limits the
expense of time and money, making user research more
likely to take place.
The methods should be robust so that even inexperienced
researchers making mistakes, can get useful information
from them. Therefore methods should be less dependent on
facilitator skills, and require only limited previous
knowledge of the target groups. Since sometimes the people
in IT businesses who want to do user research, have to
convince others, or need some convincing themselves, the
methods that are offered must be well-known or validated
methods, to inspire credibility.
4. Background information
Once the practitioners became more convinced of the
importance of taking the needs of end-users into account in
the development process, they often recommended that the
toolbox would have ready-made personas and background
information on user groups.
5. Examples (best practices)
Another requirement that came out of the interviews, was to
supply examples or best practices, so that the practitioners
could see how other businesses had approached user
research. This way, it would also be possible to give
examples of the type of knowledge one could expect to gain
from user research, without presenting it as if this particular
knowledge is generally applicable.
6. Business processes
Amongst the companies we worked with, there was a range
of business processes practiced. Practitioners who were
used to Agile or Lean [27] ways of working, were often
looking for ways to verify early concepts with users within
a short timeframe, because these methodologies require
new versions or changes to a product to be validated of
before the next iteration can begin. In more traditional
linear (waterfall) processes, there appeared to be a bigger
concern with the cost and validity of user research in
general. If a project went over budget, evaluations were one
of the first things to be skipped to save costs. User research
was often only done when they needed specific proof to
back up decisions. But even here, there was a general
acceptance of the idea that it is cheaper to identify issues
sooner rather than later.
7. Brand image / Commercial value
Next to the user research activities, it seemed important to
companies to be able to show these activities off, and be
able to use inclusive design as a sales tool. While an app or
online toolbox would be practical in use, a physical
component that can be shown in the office or given to
clients is also important.
Being able to talk about inclusive design as a complete
strategy, rather than separate user research methods, can
show a brands involvement in sustainable business
practices. It is important that the toolbox allows IT
practitioners to learn about inclusive design as a holistic
concept, to be able to sell it convincingly.
8. Terminology and tone
Some of the practitioners we worked with, joked about the
words ‘design’ and ‘research’ as concepts they were not
interested in. They saw themselves first as developers, as
clients often provided the visual design and not much
research was done. To avoid estranging practitioners with
design jargon or academic terms, it is important to use more
generic language or use terms from the IT world. On the
other hand, the people that we worked with were highly
educated entrepreneurs and independent thinkers, who did
not appreciate to be talked down to.
DESCRIBING THE INCLUDE TOOLBOX
For a small to medium-sized company that works in the
field of IT, ‘Include’ is a toolbox of user research methods
providing an easy and efficient starting point for inclusive
design. The toolbox is an interactive application
accompanied by a hardcover book. The interactive part
gives an overview of easy user research methods, complete
with descriptions and workbooks, and helps filter them
according to the company’s needs and their user groups.
The book serves as a visual reminder, a reference for
background information and a way to show off the use of
inclusive design practice in the office.
The toolbox is currently available as a free application in
Beta, that has the purpose of creating awareness about
inclusive design in the IT sector. For future versions of the
toolbox, different business models could be considered. The
main functionalities and design elements of the toolbox are
described below and structured according to the
requirements listed above.
1. Directly from the home screen of the app, it is easy to
find the right method
Incorporated in the interactive toolbox are currently 10 user
research methods. This number of methods was arrived at

by weighing different factors: there must be enough
methods to suit four development phases and three specific
target groups, but not so many that it would become
overwhelming, especially for first-time users. Some
methods are suitable for more than one phase or user group.
The methods are all existing, validated methods, with
references. The methods can be browsed from the main
screen of the app, where after clicking a short description is
offered next to a recognisable picture for each method
(Figure 3). There is a filter bar, that can be hidden when
not in use, that contains two drop-down menus: “My end-
users include: [Older People, Children, People with Low
Literacy]” and “My goal is to: [Get Inspiration, Evaluate an
Idea, Review a Scenario, Test a Prototype]”. Using the filter
creates a recommendation of three selected methods, that
can be browsed through before making a choice.
Figure 3. Example of a method description card
2. The toolbox can act as a decision aid but also allows
for making alternative choices
Offered are two ways of looking for a method (browsing
and filtering). This has the benefit of being able to give
people who know what they want direct access to the
methods, while users looking for guidance can get it while
keeping the option to choose open.
3. The methods are fast, robust and cheap
From the requirements description, we distilled the
following criteria for user involvement methods that could
be useful to SMEs in IT:
• it must be possible to do the preparation, session and
analysis in a total of 8 hours, one half day at the
beginning of the week and another at the end
• no special skills or prior experience are necessary
• it must be a well-known or validated, existing method
• it looks like a fun activity for both the practitioner and the
participants
• all the necessary materials can be provided in a PDF
workbook in the form of tips or worksheets
• get useful results with 3 to 5 participants
From a literature study, the following methods were
selected based on the aforementioned criteria:
A day in the life [9]
Cultural Probe [8]
Business Origami [10]
Mission from Mars [5]
Card Sorting [30]
Peer Tutoring [21]
Co-Constructing Stories [25]
Co-research [32]
Technology Tea Party [2]
Think Aloud [22]
The methods are described in the following way: first a
short introduction, on the basis of which the method is
chosen. Then there are seven cards used within the app:
1.overview of the method
2.description of the preparation, step-by-step, but short
(Figure 4)
3.guide for the session itself, idem
4.tips for a quick and effective analysis, idem
5.filling in the name of the project and a short description
6.choosing how to use the PDF workbook: print or digital,
and after the research, archiving the work
7.rating the method on suitability for the user group and
ease of use of the method itself, providing tips and
comments, choosing to share or not
4. and 5. Background information and best practices are
available in-app and in a hardcover book
User group characteristics change, and due to market
fragmentation, user groups are not as homogenous as before
[31]. Providing pre-made personas (in contrast to carefully
constructed personas based on relevant user research data)
would mean to overly simplify and stereotype the people of
a user group [24], ignoring the needs and wants of sub-
groups or failing to spot underlying patterns in a user group
specific to a certain project. Therefore, even though
practitioners asked for ready-made personas, we decided
against this.

Figure 4. Example of a method workbook card
The toolbox does provide general background information
about the larger target groups, to get a first idea of the
characteristics of the group, and prepare the user
involvement sessions more efficiently. In the app, a menu
can be shown for access to more in-depth background
information, such as about inclusive design, the methods,
user group information and best practices.
From the method descriptions, there are also links to this
information. Because it is unlikely that users are going to
read many pages from within an app, the complete
information is also available in the form of a hardcover and
e-book. The contents of the book and the app overlap, to
accommodate both easy reading and flipping through paper
pages, as well as the ease-of-use, portability and the help of
links within in the app.
6. Implementation in iterative business processes
Because of the limited preparation and analysis time needed
for the methods covered by the toolbox, they are suitable to
be implemented in business processes with short iterations
such as Agile or Lean. User centred design based on design
cycles is also a good fit. For other project structures, early
user research can still be beneficial to starting the project on
a relevant course, and also wherever there is a decision
moment. In some cases, using the toolbox may even help
practitioners to see their development process clearer,
because the preparations for user research often include
describing and focusing on the end-goals of the project.
7. Brand image / Commercial value
Research results can be used as proof to clients, validating
development decisions. Being able to show that their
company is practicing inclusive design was also perceived
to be a selling point towards new clients. The book
accompanying the interactive app allows for that, and is
also a visual reminder in the office to think about special
user groups more often.
Another aspect is that being able to show customers what
user research is, can convince them a project needs it. It
also makes it easier to talk about inclusive design, if clients
can learn about it from a third party. More efficient
development processes combined with a better brand
image, can increase the overall competitiveness of a
business.
8. Terminology and tone
The descriptions of the methods are void of design jargon
and academic terminology as much as possible. The
background information in both the book and the app, is
written from a perspective of an SME entrepreneur, or IT
practitioner, linking back to what they can get out of it in
their work, or what it will do for their businesses.
Furthermore, the steps of each method are balanced so that
they give just the necessary amount of information without
overwhelming or condescending.
EVALUATION OF TOOLBOX
In addition to working with different SMEs in IT
throughout the project, the final prototype of the interactive
toolbox app was evaluated qualitatively on visual and
interaction design and user perception with five
participants, in three sessions. The participants of these
sessions were practitioners in IT, with limited previous
experience with user research. The evaluation method used
was based on the Co-Constructing Stories [25] interview
technique.
All participants said they thought they could easily execute
the methods offered by the Include Toolbox, and
appreciated the detailed workbooks. In general they were
surprised each method could be executed in 8 hours and
realising this was often a pivot point in the interview; from
talking about difficulty and hurdles to talking about the
benefits of involving users. They agreed that the toolbox
language should be available in Dutch for ease of use with
end-users from the Netherlands, especially for the target
groups older people, people with low literacy, and children,
who may not speak English. Another overall feedback was
that the participants could appreciate the visual language of
the toolbox, which was seen as colourful and inviting, and
the information architecture, which was perceived as easy
to use.

Participants commented on how they would use the toolbox
app, by saying they would browse the overview page for a
while first, before choosing a method to work with.
Methods with ‚research’ in the name came across as boring
and formal, and some inspirational methods were seen as
too creative. Getting a recommendation of three selected
methods when using the filters was a good number for
participants. In practice, participants would like to spend a
minimum amount of time in the app, just quickly choose a
method, print everything out and get to work.
Some participants would have liked to see more best
practices, some said they would need much more different
user groups and more specific user group information
because for each project the target user is different. At the
time this evaluation took place, the content of the toolbox
prototype was not yet complete, and some participants were
eager to get the remaining method descriptions, perhaps
showing their motivation to get started with the methods.
Participants also noted that it seemed to them that
answering the questions in the filter bar of the app, would
already bring some awareness to practitioners about
inclusive design. Using the toolbox to educate their clients
about user research, could give the clients more confidence
to discuss the details of user research, making them more
satisfied with the company in the long run. There were
some concerns if the option to share the research result as a
best practice, would safeguard the anonymity of the end-
users enough.
The hardcover book that accompanies the Include App
(Figure 5), was not yet finished by the time the final
evaluations took place. During the sessions, an interesting
contradiction arose when discussing background
information about topics such as inclusive design, user
involvement methods, user group descriptions and best
practices. For many practitioners, it was important to know
it was offered and how they could reach it easily. However,
when it was visually represented in the toolbox, links to the
information were appreciated, but not opened. In an
informal evaluation in an exhibition setting that did show
the book, people were mainly attracted to the big size, and
colourful layout.
Figure 5. The Include Book
CONCLUSIONS & DISCUSSION
Even though there were some criticisms from the
participants in the final evaluations, the overall feedback
was positive, with participants envisioning their use of the
Include Toolbox and expressing motivation to use it in their
work. The toolbox app could perhaps reach more
practitioners if it allows for more company profiles, next to
the currently very specific: practitioners from SMEs in IT
with little or no experience in user research.
Some participants would have liked to see more best
practices in the toolbox, and this is indeed planned by
allowing users of the toolbox to share their own
experiences. This way, over time the toolbox content
becomes an increasingly complete reference guide. Further
progress towards a more complete and continuously
updated toolbox could be made through making it possible
for practitioners to add or edit information, for example add
new methods, update target group information, provide tips
and rate methods.
Other practitioners wanted more specific user group
information, but by their own admission the specifics are
unique to each project. It is our intention to add support for
more end-user groups such as teenagers, non-native people,
or people with physical or mental disabilities. In our
opinion, next to using the general information in the user
group descriptions, it would also be necessary to work
closely with users in each project, thereby making sure of
having the most specific, unique and up-to-date insights as
possible.
In our evaluations where practitioners used a prototype of
the toolbox in real projects, we and the practitioners found
that inclusive design was within practical reach for them.
The final evaluations showed acceptance of and enthusiasm
for the visual design and the research methods offered.
FUTURE WORK
The next phase of this project is the departure from the
prototype stage, starting the development of the application,
incorporating everything learned from the prototype phase.
Compared with existing toolboxes and resources, we
believe the Include Toolbox is specifically suitable for
SMEs in IT, due to its visual design, setup, and the
difficulty level of the methods offered, more so than other
toolboxes. However, this is something that needs to be
further validated with a comparative study of toolkits
amongst IT practitioners.
Although the first informal evaluations have been positive,
more formal empirical study is needed to verify our claim
that with the Include Toolbox, IT practitioners are able to
perform user research with a quality adequate for their
commercial purposes. Also we need to further substantiate
the business rationale of inclusive design and user research.
To find out more about the need and use for background
information and a physical book, further evaluations need

to be done to find out if a book is the best form to present
the information, and how much background information is
in fact required by the SMEs.
ACKNOWLEDGMENTS
The current work was carried out as part of COMMIT, a
Dutch national public-private research program on
information and communication technology, project P02:
Interaction for universal access (http://www.commit-nl.nl).
We thank Bas Holleman for his help with the interaction
design.
REFERENCES
1. Brand, R., & Rocchi, S. Rethinking value in a changing
landscape. A model for strategic reflection and business
transformation. Philips design (2011).
http://www.design.philips.com/philips/shared/assets/desi
gn_assets/pdf/nvbD/april2011/paradigms.pdf
2. Coventry, L., Jones, E. The role of tea parties to elicit
technology requirements to support the mobility of older
adults. Proc. PETRA (2012).
3. Cremers, A. H. M., Neerincx, M. A., & de Jong, J. G.
M. Inclusive design: bridging theory and practice.
Engineering Psychology and Cognitive Ergonomics,
Applications and Services. Springer Berlin Heidelberg
(2013). 323-332.
4. Curedale, R. Various titles, Design Community College
Incorporated (2012-2013).
5. Dindler, C., Eriksson, E., Iversen, O. S., Lykke-Olesen,
A., & Ludvigsen, M. Mission from Mars. Proc. IDC
(2005).
6. Dix, A. Human–computer interaction: A stable
discipline, a nascent science, and the growth of the long
tail. Interacting with Computers, 22(1), (2010), 13-27.
7. Eikhaug, O., Gheerawo, R., Plumbe, C., Berg, M. S., &
Kunur, M. Innovating with People- The Business of
Inclusive Design. Norsk Designråd (2010).
8. Gaver, B., Dunne, T., & Pacenti, E. Design: cultural
probes. Interactions, 6(1), (1999), 21-29.
9. Gillen, J., Cameron, C. A., Tapanya, S., Pinto, G.,
Hancock, R., Young, S., & Gamannossi, B. A. A day in
the life, Early Child Development and Care Vol. 177,
Iss. 2, (2007).
10.Hanington, B. & Martin, B. Universal Methods of
Design: 100 Ways to Research Complex Problems,
Develop Innovative Ideas, and Design Effective
Solutions. Rockport Publishers (2012).
11.http://designingwithpeople.org
12.http://filter.ucdtoolbox.com
13.http://www.55plustoolbox.nl/Toolbox.php
14.http://www.hcdconnect.org
15.http://www.inclusivedesigntoolkit.com
16.http://www2.gov.bc.ca/gov/topic.page?id=157D6555F8
744850A439544F41727402
17.IDEO, Method Cards: 51 Ways to Inspire Design. Palo
Alto (2003).
18.Jongerius, P., Offermans, A., Vanhoucke, A.,
Sanwikarja, P., & van Geel, J. Get Agile!: Scrum for
UX, Design & Development. BIS Publishers (2013).
19.Kayal, A., Brinkman, W.P., Zoon, H., Neerincx, M.,
Van Riemsdijk, M. B. Using norms for social
applications to provide value-sensitive support to
families. Submitted for publication (2014).
20.Langdon, P., & Thimbleby, H. Inclusion and interaction:
Designing interaction for inclusive populations.
Interacting with Computers, 22(6), (2010). 439-448.
21.Markopoulos, P., Read, J. C., MacFarlane, S., &
Hoysniemi, J. Evaluating children's interactive products.
Morgan Kaufmann (2008).
22.Nielsen, J. Thinking Aloud: The #1 Usability Tool.
NNgroup (2012). Via http://www.nngroup.com/articles/
thinking-aloud-the-1-usability-tool/, retrieved 2013.
23.Nielsen, J., Landauer, T.K. A mathematical model of the
finding of usability problems. Proc. CHI, INTERACT
(1993), 206-213
24.Nielsen, L. Constructing the user. Human-computer
interaction: theory and practice. Part II 2 (2003), 430
25.Ozcelik-Buskermolen, D., & Terken, J. Co-constructing
stories. Proc. PDC (2012).
26.Razak, F. H. A. Single person study: Methodological
issues. Doctoral dissertation Lancaster University
(2008).
27.Ries, E. The Lean Startup: How today's entrepreneurs
use continuous innovation to create radically successful
businesses. Random House Digital, Inc (2011).
28.Seffah, A. Bridging the educational gap between SE and
HCI: what software engineers should know? Proc. IET
(2004), 88-95.
29.Sluis-Theischeffer, W., Bekker, T., & Eggen, B. Adding
user creativity to the UX toolbox: Exploring the use of
Creative UX methods. Proc. CHI Netherlands (2009).
30.Spencer, D. Card Sorting, O'Reilly Media, Inc. (2009).
31.Tidd, J., & Bessant, J. Managing innovation: integrating
technological, market and organizational change. Wiley
(2011), Chapter 1.3.
32.Van Doorn, F., Stappers, P. J., & Gielen, M. Design
research by proxy. Proc. SIGCHI (2013).
33.Wheelchair teen in record-breaking backflip, Metro UK
(2008). http://metro.co.uk/2008/11/24/wheelchair-teen-
in-record-breaking-backflip-175437/