Content uploaded by John V. H. Bonner
Author content
All content in this area was uploaded by John V. H. Bonner on Jul 28, 2015
Content may be subject to copyright.
University of Huddersfield Repository
Bonner, John V.H.
Lessons Learnt in Providing Product Designers with User-Participatory Interaction Design Tools
Original Citation
Bonner, John V.H. (2009) Lessons Learnt in Providing Product Designers with User-Participatory
Interaction Design Tools. In: Configuring User-Designer Relations. Computer Supported
Cooperative Work . Springer. ISBN 9781846289248
This version is available at http://eprints.hud.ac.uk/4319/
The University Repository is a digital collection of the research output of the
University, available on Open Access. Copyright and Moral Rights for the items
on this site are retained by the individual author and/or other copyright owners.
Users may access full items free of charge; copies of full text items generally
can be reproduced, displayed or performed and given to third parties in any
format or medium for personal research or study, educational or not-for-profit
purposes without prior permission or charge, provided:
•The authors, title and full bibliographic details is credited in any copy;
•A hyperlink and/or URL is included for the original metadata page; and
•The content is not changed in any way.
For more information, including our policy and submission procedure, please
contact the Repository Team at: E.mailbox@hud.ac.uk.
http://eprints.hud.ac.uk/
Chapter 5
Lessons Learnt in Providing Product Designers
with User-Participatory Interaction Design Tools
John V H Bonner
Department of Informatics, School of Computing and Engineering, Huddersfield University,
UK, j.v.bonner@hud.ac.uk
5.1 Introduction
This study investigates how participatory design (PD) tools (adapted from human–
computer interaction methods) could be provided specifically for industrial designers
to conceive, develop and evaluate novel interfaces for new and innovative appliance
technologies. The research was carried out over 3 years and involved a number of
studies at a major European consumer product manufacturer’s design group based in
the UK. The challenge was to investigate if these adapted tools could be used effec-
tively by designers (not trained in HCI or human factors-based methods) and poten-
tial users to develop novel interactive consumer product interfaces, particularly for
microwave and cooker user interfaces.
The chapter begins by discussing the evaluation criteria that were adopted to
measure the effectiveness of the design tools to form useful user and design require-
ments for a live commercial design project. The first study explored the use of a
simple card-sorting tool and its impact on introducing product designers to PD meth-
ods. Lessons learnt from this study helped provide a direction for the second study
that included a scenario design tool. The second study describes how the designer’s
confidence of using the design tools and engaging with user participants within the
design process improves positively. More significantly, perhaps, is how the design-
ers’ perceptions of participants-as-designers also changes. This is followed by a
discussion about the key findings from the two studies and the importance of organ-
isational survival as a critical factor to the successful implementation of PD tools.
The fundamental philosophy of PD is to involve the users of future systems or ar-
tefacts in design activity (Greenbaum and Kyng 1991). PD empowers users by re-
garding the users as the domain experts and assuming that any changes to a system
should improve their role within it (Schuler and Namioka 1993). Emphasis was
placed on the design tools being able to consider the ‘context’ of user–product inter-
action as context influences purposeful activity (Brown and Duguid 1994; Bødker
1991; Nardi 1996).
A. Voss et al. (eds.), Configuring User-Designer Relations,
DOI: 10.1007/978-1-84628-925-5_5, © Springer-Verlag London Limited 2008
88 Bonner
Structured methodologies analysing context and design have been developed
(Beyer and Holtzblatt 1998). However, appropriate abstraction of contextual user–
product interaction is notoriously difficult to capture, even for experts. Lewis et al.
found that data capture and analysis methods had to be pragmatic to fit commercial
demands (Lewis et al. 1996). Time constraints and lack of expertise required a more
streamlined approach to using interaction analysis. They devised a coding scheme
that was efficient with simple nomenclature. Both Lewis et al. (1996) and Beyer and
Holtzblatt (1998) used a form of representational model first to evolve an appropriate
and robust abstraction of the situation under study and then to test design proposals
against it. Ehn (1992) suggested the use of common ‘language games’ between users
and designers. The games need not have the same sense for users as for designers,
but the rules of participation have to make sense to both. Design artefacts should not
attempt to create ‘pictures of reality’ but should help users and designers articulate
current situations and envision future ones.
Before describing the studies, a brief summary of the rationale underpinning the
development of these PD tools is provided. The tools were to:
• be participatory – involving designers and users in the design process,
• be context sensitive – capturing as much of the rich context of use as possi-
ble,
• require no prior knowledge of user-centred design methods and be prag-
matic in approach, and
• provide applicability and specificity to a given interaction design problem.
Making these design tools in this study suitable for non-experts made the devel-
opment even more challenging. Good representational models seemed to offer a
good starting point by which designers and users could articulate their needs and
intentions. Therefore, the design tools were developed so that context-sensitive de-
sign data (i.e., user–product interaction evidence and information generated from the
workshops, which may influence design decisions) could be generated in parallel to
conventional design activity through the development of a series of interaction de-
sign models. Interaction design models would facilitate the transition from gaining
design data from the real world to producing problem-specific design guidelines as a
basis for usable interface design solutions.
The proposed tools would allow design solutions to emerge from users and de-
signers working together, with users and designers, in effect, controlling the design
problem and creating agreed solutions. In making judgements about the use and
benefits of the tools, their effectiveness was evaluated using a range of validity and
reliability criteria. Validity was interpreted as the degree to which designers could
derive accurate and consistent interpretations of any design claims made through the
use of the tools without misinterpretation. The reliability criteria were defined as the
level of consistency in resolving the same type and number of interaction design
issues across different design problems or workshops. As design data were now
generated within the design process, reliability and validity needed to be examined
closely, because there was now a stronger interdependency between users and de-
signers to identify and resolve interaction design issues together. It was important
Lessons Learnt in Providing Product Designers with User-Participatory 89
that the design tools could provide a stable and robust environment in which this
joint decision making could consistently occur.
Therefore, validity of these data was a concern and dependent on factors such as:
• Design and management of design tool exercises – this included tasks such as:
the design of cards; selection and representation of objects and tasks; and the se-
lection of workshop goals and motives.
• Procedural control of the design tools – the ability to: refine or adapt design
tools to alter design data outputs; manage collaborative discussion between de-
signers and user participants.
• Quality of experiential knowledge – how this is recognised and used.
• Interpretation of design data – effectiveness in interpreting data at different
levels of abstraction with adequate depth and breadth.
Reliability could be affected by the following factors:
• Design and management of design tool exercises – this included tasks such as:
accuracy of verbal and written instructions given to the user participants; re-
cording of design tool procedures and exercises during and between their use;
and choice and selection of users and designers between design tools and/or
workshops.
• Changes in procedural understanding of the design tools – changes over time in
skills to implement and control design tools by users and designers between de-
sign tools and/or workshops.
• Consistency of experiential knowledge – depth and breadth of designers’ and
users’ knowledge used between design tools and/or workshops.
Little practical evidence had been found about how to assess the overall effec-
tiveness of these design tools and therefore a wide range of evaluation criteria were
derived. These included:
• Reliability and validity of the tools – as discussed above.
• Interpretation of the interaction design models – how designers and users con-
vert outcomes into design requirements.
• Scope of the usability issues identified – what the tools uncover in terms of
usability.
• Ability to support novel interaction styles – how effective the tools are in facili-
tating new interaction styles with products.
• Usability of the design tools – how quickly and comfortably did the designers
and users work with the tools.
• Relevance of tools to designers – how did the design tools impact on current
design practice.
• Likelihood of organisational acceptance – factors that would affect the adoption
of the tools more widely within the organisation.
90 Bonner
5.2 Initial Evaluation of a Card-Sorting Tool
In this study, card-sorting games were developed to explore how designers and users
could collaboratively communicate their outline design and user requirements for
new novel product interfaces for a future range of smart cooker interfaces. Evidence
from the literature suggested that ‘card-sorting’ games might be a useful technique.
Card-sorting games have been used by Muller et al. (1995), the first stage known as
CARD (Collaborative Analysis of Requirements and Design) uses cards to facilitate
the articulation of task and communication within working groups. The CARD ap-
proach has been modified by Lafrenière (1996) in the design of computer-based user
interfaces (CUTA) to enable a simple, user-derived, task analysis to assist in inter-
face design. Both the CARD and CUTA methods are based on cards depicting ele-
ments of tasks activity such as task objects, for instance, telephones and notepads,
process-based activities like methods of working and situations; participants within
the task activity are also depicted. Both methods require the participants to select
task elements and place the cards in an agreed plan or sequence. Once complete, an
agreed summary of their ‘representation’ is given by participants.
The attraction in adopting this form of technique was the ease with which such a
method could be learned and implemented. A series of small, incremental card-
sorting exercises were proposed for the first tool and were carried out in a workshop
setting with designers and potential user participants.
Four different types of card-sorting tasks were designed each having a different
contextual focus. The first exercise required the user participants to plan out the
preparation and cooking of a meal using a series of cards describing cooking activi-
ties such as Check carrots to see if they are ready. Other supporting activities were
described on cards, such as Check to see if oven temperature is correct using mini-
mal references to technological support as possible. The intention was to allow the
participants to discuss the whole cooking process and allow their personal habits and
attitudes towards cooking to emerge. Participants laid cards out on the table in the
form of a task plan describing how they would cook a particular meal.
The second exercise was designed to build on the first by inserting function cards
containing descriptions of cooker features or technological support. Participants were
instructed to add features to their task plan only if they felt it would be used. Partici-
pants would be asked to openly discuss the advantages and disadvantages of each
feature before inserting a function or deciding to leave it out.
In the third exercise, participants were asked to think about a ‘week-in-the-life’
of a cooker and place cards depicting other cooking activities, ‘clean the cooker’ or
‘cook a quick snack’ under cards labelled with the days of the week. The purpose of
this exercise was to explore if participants would be able to make design decisions or
make inferences from their cooking habits that might affect broader or non-task-
specific interface design issues. The intention of this exercise was to build up a fre-
quency profile of usage and the type of tasks undertaken during a typical week.
In the fourth exercise, participants were provided with a series of cards contain-
ing character profiles describing fictional individuals with different levels of interest
in cooking and technology. Participants ‘matched’ these characters against some of
Lessons Learnt in Providing Product Designers with User-Participatory 91
the function cards used in exercise 2. The intention of this task was to examine if the
users could make ‘third party’ design decisions on behalf of fictional characters.
The card-sorting tool provided four main advantages. First, cards would provide
a quick and cheap discussion mechanism or act as ‘transitional objects’ allowing
more critical contextual thinking to occur about product interaction. Second, by
providing a broad range of card descriptors it would allow novel concepts to be in-
troduced without having to design the interface, allowing participants to interpret or
define the cards on their own terms. Third, card descriptions could be divorced from
defined or existing technology so future functionality could be discussed. Finally, the
placed cards could act as a conceptual ‘interaction model’ for analysis. There was no
need at this stage to have a single or coherent solution. The intention of the design
tool was to arrive at a series of creative design proposals for subsequent refinement.
The researcher selected a scenario, for example, preparation and cooking of a
Sunday meal. A set of speculative cards was prepared for the design team to reduce
their time commitment to the study. The designers ‘walked through’ the prepared
exercises, sorted the cards while acting as both designers and participants. This was
to help familiarise themselves with the exercises and to look for anticipated problems
or potential misunderstandings. For the main study, four groups of between 5–6 user
participants were recruited for workshops sessions, which lasted about 2 hours. The
participants were recruited by the design team and consisted of factory and office
volunteer workers from one of the manufacturing plants. Each group was given some
introductory explanation by the researcher who remained present throughout all
workshops. They were asked to discuss the process as a group and arrive at consen-
sual agreement if any differences in opinion were found.
After the exercises, participants and designers discussed their thoughts on per-
sonal cooking habits, perceptions of technology and the effectiveness of the card-
sorting exercises. These were recorded on video, and notes were made during the
workshops and the videotapes were analysed using the evaluation criteria.
5.2.1 Observations
All four workshops followed a similar procedural flow. The designers remained
generally passive throughout the card-sorting exercises and only intervened towards
the end of each workshop when more informal discussions began. In all groups, a
leader or chair from the participants emerged acting as the ‘controller’ of the cards
and also of group decision-making strategies. The assumed leader often re-evaluated
their task plan to ensure coherence to procedural instructions.
Reliability and Validity of Design Data
The design data generated from the workshops were rich and variable, but the man-
agement and control of its production were negligible, thus resulting in poor validity.
The exercises provoked wide and interesting anecdotal discussions about cooking
methods, preferences and strategies for using cooking technology, but this was not
controlled or steered by the designers; they did not take ownership of the problem,
which could effect the production of consistent discussions about common interac-
92 Bonner
tion problems. This was understandable at this stage. Few facilitating procedures had
been given to the designers prior to the workshops due to uncertainty about their
involvement. The participants gathered contextual data, but not in a controlled or
predictable manner. For example, there was a tendency to add ‘peripheral’ or unim-
portant cards to the task plan, like adding more utensils, rather than adding any real
new tasks that provided deeper design insights.
The reliability of the design tool management was high. Participant behaviour
across all four workshops was generally consistent. In all workshops there was con-
siderable anecdotal discussion about cooking habits prompted by the card-sorting
activities. Different sorting strategies were adopted within each group, but comments
made by the participants were similar across all groups. However, decisions were
less dependent upon the exercise task and based more on broad collective experi-
ences. Comments were frequently based on family habits rather than being driven by
the card-sorting exercises, which could affect the validity of the design data.
During all workshops, some discussion was given over to reviewing the task plan
after a natural phase of cooking activity had been discussed or a reasonable ‘compo-
nent’ of activity had been described through the cards. Omissions in task elements
were identified through this process and improved the quality of capturing design
data. This checking procedure revealed how cognitively different card-sorting activ-
ity was to real cooking tasks. Participants needed to remind themselves of procedural
steps and ensure that these were accurately reflected in the task plan, as one partici-
pant reflected, ‘this is harder than doing the real thing’. The cards forced participants
to deconstruct activity but not necessarily in a natural manner. In the example below,
Participant 1 (P1) was confused whether a completed task (Potatoes cooked) had
been represented.
P1: ‘you’ve turned them on…to put the potatoes on…didn’t you?’ [pointing to
‘Potatoes Cooked’ card]
P2: ‘Ah but the…’
P3: ‘you don’t need to put them on the hob yet’
P1: ‘should I put them in?’ [the oven]
P2: ‘go then yes…yes’
P3: ‘you’ve got to prepare your veg now’
The task plan did not naturally suggest where subtasks start or finish. Participants
found it difficult to model time-related conditions that would be obvious during real
activity.
During the second exercise, that is, inserting function cards into the task plan,
participants took a less purposeful approach. Function cards were read out and par-
ticipants arbitrarily inserted them into the task map without considering their impor-
tance or frequency of use.
P1: ‘Do you want an electric helper?’ [reading from card]
P2: ‘that would be at the beginning [of the task map] you’d want to know how to
boil an egg’
P1: ‘Would you want that to be electric?’
Lessons Learnt in Providing Product Designers with User-Participatory 93
P2: ‘Yer – she wants er…’
P3: ‘I need help when I cook’
P1: ‘Does it have to be electric or a book?’
P2: ‘A book’
P3: ‘A book might be easier to use’
P1: ‘Why – haven’t you read it? [Laughs]’
P2: ‘Do you mean an electric helper?' [Suggestion]
P1: ‘Would you use it?’
P3: ‘Yes I would’
D1: ‘What about an electronic book?’ [Suggestion – but related to one of the
function cards provided]
P3: ‘That would be even better’
D2: ‘This might be an electronic note book’
This exercise, however, did prompt discussions around the activity of cooking,
leading to more receptive discussions on the use of technology to support the cook-
ing of new or unusual meals. In the dialogue above, a subgroup (P1–3) discussed
how and when they might use some form of computer-based cooking assistant. They
were unclear how it might function or how it might be used. The designer (D1) of-
fered a more focussed solution but did not probe further. During one discussion of
this nature, one designer was surprised to observe participants making contradictory
demands for technology.
The third and fourth exercises were less successful in producing design data. In
the ‘week-in-the-life’ exercise, participants generally added typical meal types under
each weekday heading without discussion. The final exercise, ‘character profiles’
generated stereotypical comments about the usage of technology and again did little
to reveal any insightful comments that could be effectively used as design data.
Effectiveness of Interaction Models
The task plan was intended to form the backbone of the interaction model with each
exercise building up a contextually oriented representation of cooking activity. The
plans, card settings and comments produced by the participants were intended as a
record of design data. However, the designers paid little attention either to their con-
struction or the completed plans. This appeared, at the time, to be disappointing.
Design data, in the form of the task plan, were not formally documented but re-
mained verbal, undocumented reactions. Little was shared between the designers
about the knowledge gained.
Another purpose of the task plan was to create a common dialogue between de-
signers and participants; clearly this did not always occur. In the example below, the
designer (D) tried to identify why P had such little faith in the safety of her electrical
products. Terms like ‘hot product’ are commonly used in this organisation but unfa-
miliar to the participant and discussion is not pursued because of this. Many of these
deficiencies can be attributed to a lack of shared understanding between the design-
ers and participants. The design tools need to ensure that this form of breakdown
does not occur, this could be achieved perhaps by encouraging the designers to take
more active part in the card-sorting activity.
94 Bonner
D: ‘You mean you reset the clock every day?’
P: ‘Me electric goes off at the wall [at night time]. I could not afford for me
house to be burnt down, for safety everything goes off at the wall’
D: ‘Is this because this is a “hot product?”’
P: ‘No – it’s all off at the wall’
D: ‘Everything?’ [meaning other electrical products]
P: ‘Every electric product’
Supporting Novel Interaction Styles
In designing the card-sorting activities, it was difficult to understand how novel
concepts could be introduced or how they might be generated during the exercises.
One approach emerged entirely by accident. Writing vague or ambiguous statements
on the cards often prompted questions about their meaning or significance within the
card-sorting exercises. While clarifying their meaning, suggestions were often put
forward, which occasionally resulted in creative proposals.
Usability of the Design Tool
There was no doubt that the cards proved an effective vehicle for promoting discus-
sion. The exercises themselves were not difficult to accomplish, although procedural
problems were identified. For example, participants often needed reassurance on
‘rules of the game’. Task-planning exercises were very time consuming and often
had to be reviewed for consistency and errors by the participants to ensure that the
task map told a story.
For the designers, the level of engagement with the design tool was very low.
They did not intervene in the card-sorting activities and only occasionally offered
advice. When asked why they had not taken notes for use later on, they stated that
they did not feel it was necessary as the process had already provided them with
many new ideas. They felt they had a clear understanding of the direction they would
take with future cooker interface proposals.
Relevance to Proposed Target Audience
The designers were very encouraged by the workshops and found the exercises ex-
tremely illuminating and worthwhile. One designer said ‘in the five years I’ve been
here I have never been able to gather as much useful information from users as I’ve
been able to do here’. The design tool provided an opportunity to involve users in a
collaborative rather than consultative role. Persuading the design team to embark on
such a process was, at times, difficult and required a great deal of ‘hand holding’
from the researcher. Many of the procedural elements of the design tool were un-
tested, which contributed to their sense of unease about using them. Due to this un-
certainty, it was difficult to assign the designers clear roles, therefore resulting in
them becoming passive observers. Shifting ownership of the design tools to the de-
signers became the next important iterative step.
After the workshops, designers were asked to comment on the effectiveness of
the design tool. Comments were very favourable but there was little evidence, apart
Lessons Learnt in Providing Product Designers with User-Participatory 95
from the task plan that other forms of design data have been captured. A summary
statement was drawn up with the research investigator.
It is important that the cooker interface instils trust to the user by providing
ample feedback and information on the consequences of using any new or
novel technology. New features will not be used unless the user fully under-
stands the implications of such an action and can be confident that the action
has been accepted by the cooker.
The interface should avoid providing functions that are ‘owned’ by the cooker
rather than the user, for example, timing devices where the cooker is allowed
to own some time keeping tasks. Controls should always suggest that the user
is in charge by permitting clear and positive feedback of their purpose but
should also allow more adventurous users to feel ‘master’ of the cooker by al-
lowing some controls to be configured to their own needs.
The interface should provide a ‘supporting’ rather than ‘expert’ role either in
terms of food safety and hygiene or in introducing the user to new methods of
cooking or new types of cuisine.
The key criterion, however, by which any novel features for a proposed inter-
face must be assessed, is on TRUST.
The statement above reflected a user-centred tone with a strong emphasis on per-
ception towards technology rather than on specific functional requirements. This was
very encouraging as it was hoped that this insight would be achieved.
Likelihood of Organisational Survival
Prior to the workshops, all designers expressed concern about the card-sorting design
tool and were hesitant about a process they were not directly in control of. They were
unsure how participants would react to vague or unclear proposals and did not relish
the prospect of deliberately placing themselves in a situation where they had uncer-
tain or no design proposals to offer the participants. As one of the designers said, ‘we
don’t want them going away thinking we can’t design a cooker’. Certainly the de-
signers felt no ownership of the design tool before the workshops. However, later the
reaction was very different. They were encouraged with how participants dealt with
the situation and surprised at the participants’ level of creativity.
5.2.2 Reflections on Study 1 and Changes for the Second Study
There was sufficient evidence to suggest that contextually based user–product design
data could be gathered. The use of the design tools was clearly enjoyed by both par-
ticipants and designers and acceptance for their use was achieved.
However, the interaction model (a task plan based on the layout and placement of
cards on the table) was not used effectively as it could have been. The designers did
not support or assist in the insertion of function cards at recognised stages or urge
Au: Need page number
for this summary state-
ment for reference in the
conclusions.
96 Bonner
participants to consider the implications of adding functions into the task map. If this
had been done, it may have helped participants to consider the implications of their
actions more critically.
The relationship between objectives and outcomes of the card-sorting exercises
needed to be more clearly defined and to be made explicit for both the designers and
participants. Although a rich source of design data was gathered, the designers were
not equipped to capture or control the type and quality of design data generated.
More guidance by the researchers was required to help the product designers design
the cards, for example, coding methods, illustrations, colour, and shape of cards.
Card composition required more consideration to accurately trigger discussion about
potential user behaviour and needs. The designers felt little compulsion to support
their final design proposals using data gained from the card-sorting tool. To solve
this problem, the designers decided to design the cards and the card-sorting exercises
themselves. They streamlined the number of card-sorting exercises to one single
activity, which they thought yielded the best results. The exercise in effect was a
merger of the first two exercises in the first study – placement of task and function
cards. To ensure more discrimination and active selection of function cards, a ‘func-
tion filter’ was introduced where each card had to be assessed on two criteria: fre-
quency and importance of use. Functions that rated highly on both criteria were then
introduced first into the task plan.
The quality and detail of the design data using this type of design tool was ‘attenu-
ated’. That is to say, the detail and scope of design issues discussed will inevitably be
less detailed than using ethnographic or participative studies using trained designers or
researchers. In the following study, attention needed to be placed on providing the right
balance to gain the correct level of attenuation from the data-gathering process and the
impact of using outcomes from these data to inform design decision making.
Therefore, to improve the accurate selection of important and relevant design
data, a second, scenario-based, design tool was introduced. Scenario-based design
methods have been used in HCI to help design complex system requirements. The
main advantage of such an approach is allowing open-ended and ill-defined prob-
lems to be explored in structured and tangible ways. The complexity and subtlety of
interaction makes comprehensive descriptions of activity difficult. The use of scenar-
ios as a design tool has evolved in HCI as a mechanism to describe complex activity
and allow designers to engage in and articulate design intentions (Carroll 2000; Ja-
cobson et al. 1999) and have been successfully applied through the ‘use case’ ap-
proach in object-oriented software engineering (Constantine and Lockwood 1999).
A scenario-based design method appeared the most attractive option due to its
flexibility and openness to interpretation. The scenario design tool permitted design-
ers and participants to explore proposed concepts while acting or role playing within
a selected scenario. The intention was that participants would be able to make more
informed and context-sensitive judgements about the range of design proposals that
had been suggested in card-sorting exercises. In order to make the design tools suffi-
ciently malleable, participants used paper-based prototypes to enact their activity and
walk through the interaction procedure. Amendments would be made in discussion
with designers where possible new prototype variants could be rapidly introduced
into the scenario.
Lessons Learnt in Providing Product Designers with User-Participatory 97
5.3 Card Sorting and Scenario Design Tools Managed by
Designers
In contrast to the first study, the designers now facilitated the workshops and made
improvements, which they felt would increase their control over the process. To
obtain a realistic understanding of the applicability of the design tools, experimental
intervention was now kept to the absolute minimum. This was essential to ensure
that natural organisational factors influence the effectiveness of the design tools, and
not experimental procedure.
The design group had been commissioned to review their microwave product
range, including the design of new or improved interface design functions. Two
designers, who had been involved in the previous study, were provided with an hour-
long tutorial explaining how to use the scenario-based design tool. A cooking sce-
nario was selected, which the designers felt would require useful challenges in using
a microwave, such as planning when and how to use the microwave.
Two card-sorting workshops were first carried out, each with five non-design
employees. Figure 5.1 illustrates a card-sorting workshop in action. During the card-
sorting workshops, the designers recorded any thoughts or comments on a large flip
chart and photographed key events, such as the completed task map. The key out-
come from the card-sorting workshops was a user-requirements brief in the form of a
large (1.5 m wide and 1.0 m) board. This was used to form a tab board for the next
design tool, scenario design, which was based on preferred function cards clustered
into cells. Tabs were small, annotated sketches of preferred or suggested function
Fig. 5.1. Participants involved in card-sorting exercise
98 Bonner
Fig. 5.2. Tabs created by designers
variants produced from the card-sorting exercise for participants to select in the
scenario design workshops. Examples of tabs are provided in Fig. 5.2, illustrating
variant ‘start’ controls.
The scenario design workshops were planned in a similar way to card sorting
with two volunteer participants from the previous card-sorting workshops. Partici-
pants repeated the same task, but performed it as a real task using a working kitchen.
Although they had to use a microwave oven, they could only operate the microwave
‘through’ the tab board and prototype interface (see Fig. 5.3, the tab board is at the
rear of the picture). A conventional oven could not be used to force the participants
to select and consider tabs (control and display components) and to discuss the us-
ability of each component device.
Participants carried out the scenario by following a recipe and were encouraged
to discuss their thoughts and ideas on the design of the proposed microwave inter-
face. Alterations to any design proposal or tabs could be made at any time. The se-
lected tabs were used to build up a paper prototype (see Fig. 5.4) based purely on the
specific needs of the participants within a given scenario.
Lessons Learnt in Providing Product Designers with User-Participatory 99
Fig. 5.3. Tab board in use
Fig. 5.4. Tabs placed on prototype interface
100 Bonner
5.3.1 Observations
Reliability and Validity of Design Data
The design tools remained a very flexible, adaptable and potentially powerful de-
vices for collecting design data. However, it is precisely these important characteris-
tics that may affect the validity of the design data as this could increase the number
of possible interpretations. Providing consistent instruction to the designers was at
times difficult when so many of the exercises and procedures were untested. A hand-
book was provided with procedural instructions, but was not used in preference to
being given personal instruction by the researcher. Although the designers were now
facilitating the workshops, there was little evidence of managing collaborative design
dialogue. They perceived their role as providers of design concepts for participants to
test and to offer their interpretations on card depictions. Incomplete or ambiguous
assumptions were not collaboratively discussed or jointly resolved, thus decreasing
the validity of the design data. However, controlling validity was not an issue with
the designers, they were more concerned with ensuring that the workshops appeared
to be smooth running to the participants and that sufficient evidence was gathered to
produce an internal report.
The design tools forced interaction design issues to be principally focussed at the
physical device level, through a strong emphasis on depicting physical control and
display elements on the cards and tabs. Other capturing methods were provided, for
example, the layout of the task plan provided very useful information on where par-
ticipants felt that functions could support the cooking activity. This type of design
data could have helped in developing navigational support. However, the designers
did not use these data even after the researcher had explicitly pointed out task plan
patterns to them. A good example of this was how function cards clustered around
activities at the beginning and end of the cooking task, thus giving strong sugges-
tions to a possible navigational structure for controls and displays.
Designers created design data (cards and tabs) in quite a mechanical way. They
produced simple comparative tabs, for example, different types of time controllers,
for participants to comment on largely based on the outputs from the function filter
used in the card-sorting tool. Tab options were refined through approving, rejecting
or adapting them. Producing conventional prototype solutions reduced the likelihood
of more novel and user-driven solutions to emerge. The designers were very effec-
tive at controlling tab selection. While participants were constructing their prototype
interface, the designers often questioned their rationale for selecting or placing a tab.
Often this was to highlight a syntax problem, but, in some cases, this was to allow
the participants to consider alternative function variants. The designers did not re-
cord this form of design rationale. When questioned about this, they felt the proto-
type, as an outcome, offered sufficient constructive and concrete evidence. The de-
signers also had little expectations about the design tools and were not seeking
specific outcomes. For example, before the workshop, it was suggested to the de-
signers by the researcher that cards should appear more ‘rough and ready’ to infer
that changes and amendments could be made, thus allowing the participants to con-
trol the design of the cards. Professional pride prevented them from doing this.
Lessons Learnt in Providing Product Designers with User-Participatory 101
The participants used the tab board (interaction design model) to great effect but,
nevertheless, speculative decision making was observed.
P1 ‘If you cook something for 10 minutes and you take it out after 9 then as long
as you don’t start it again there will be a minute left in your memory’
P2 ‘Then you won’t get your time and date back – you could leave it another day
and all you would get, just sat there is 1 minute on the screen’
P1 ‘So when you put your next item in to cook it will automatically update itself,
the clock would run in the background’
D ‘I don’t know, that’s a feature you might decide to design, perhaps it clears it-
self after a period of time’
P2 ‘Yer, it gets bored’
P1 ‘auto clear’
P2 ‘but that might cause problems if you go away and answer the phone’
In this dialogue, a new ‘automatic resetting’ function was considered but deci-
sions become based more on anticipated future behaviour rather than based on the
context of the scenario. However, in another example, grounded experiential knowl-
edge was used effectively. One participant struggled with the concept of representing
weight on a display and suggested that it be in the form of ‘bags of sugar’; this was
more meaningful to her as she did not instinctively know how much a kilogram
would weigh.
The interpretation of design data was sometimes arbitrary. Participants made too
many speculative decisions that were not contributing towards effective design data.
The selecting, placement and planning of cards inhibited useful consideration of real
scenario-based activity. Decision making often related to stereotypical assumptions,
for example:
P ‘Well I could go for something as complicated as that, but I don’t do a lot of
cooking, but most women don’t want a lot of buttons on a cooker, they want to
turn it on and use it, you might be able to have a multifunction control but with a
turn knob’
Or, they were based on stereotypical preferences rather than on issues generated
through the design tool. This was particularly true while using the function filter.
P1 ‘We’ll need a START or STOP won’t we or STOP AUTOMATIC?’
P2 ‘No I have to…it’s on, you normally click it up on mine’
P3 ‘When it gets back to zero it switches off’
D ‘The bell rings’
P2 ‘Yes, you can stop it half way through’
P1 ‘So we are saying we don’t want a manual – we’ll do it through the timer’
P2 ‘So it’s “quite useful” that we don’t want to use it’
Here, automatic features were considered but references were made to other per-
sonal products rather than the design problem at hand thus adding to more variable
102 Bonner
design data. The graphical representation of tabs (control knobs and dials in this
example) did cause problems with interpretation of particularly temporal aspects.
Often designers had to explain how a function would operate.
P ‘Is that … does that operate the dial?’
D1 ‘Set it and then activate it’
P ‘Right, so you set the dial and then activate the dial’
D1 ‘No that’s actually on the screen, it’s above the dial in the screen… so you
turn that round’
P ‘So that’s the knob on there, is that what you are saying?’
D1 ‘No’
D2 ‘Imagine that’s your display it would be there’
P ‘Oh Sorry’
In this example the participant has confused a display having an illustrative icon
on it with a real control object that could be physically manipulated.
Adaptability of the design tools has been identified as an important aspect to im-
prove ownership of the tools and was encouraged. Providing this form of control did,
however, have an effect on the procedural understanding of methods adopted thus
affecting the reliability of the outcomes. Between the first and second card-sorting
exercise, the designers recognised by themselves that the first task-planning activity
was too detailed and procedural. The designers had produced cards with prescriptive
instructions resembling instructions from a cooking recipe. They also recognised that
card depiction was ‘text’ heavy and more graphical images would increase interpre-
tation and improve card recognition.
Interpretation of Interaction Design Models
When interviewing the designers after the card-sorting workshops, they appeared to
have very clear views about what the participants wanted from a microwave inter-
face. They asserted that the participants were reluctant to use non-tactile control
devices, as one designer stated, ‘that means no more touch screen interfaces’. The
designers concluded that any proposed interface should have no more than three
control devices, although this had not been discussed explicitly during the workshops
with the participants.
The function filtering activity (part of the card=sorting process to ensure that
only functions deemed important by the participants are considered first) was
thought to be the most productive interaction model in terms of establishing design
guidance in the form of user requirements. Although the task plan and function filter
were photographed, they were not analysed to produce further types of design data,
such as gaining a navigation model from the task map (bottom left hand corner of
Fig. 5.5). Only the preferred cards from the function filter (top right hand corner of
Fig. 5.5) were retained to form tabs for the scenario design workshops.
With the tab board, both designers and participants were more critical in their
evaluation, selection and use of the tabs. New tabs were devised if the participants
identified an alternative or improved way of achieving a goal. In some situations
Lessons Learnt in Providing Product Designers with User-Participatory 103
Fig. 5.5. Example of task map and function filter
participants made requests for functions that did not exist on the board. In this situa-
tion, the designer and participant would simply draw a new tab that suited their
needs. The process of design, build and test could be achieved in a matter of minutes.
Scope of Usability Issues Identified
Usability issues were more comprehensively addressed with the scenario design tool,
particularly issues such as consistency and compatibility. Participants did question
the usability of control labels and provided more meaningful labelling suggestions;
for example, replacing power levels, described in Watts, with a more contextual
value such as full and half power.
Supporting Novel Interaction Styles
A range of novel concepts was introduced by the designers with the card-sorting
tool, such as twin turntables, ready meal scanner, oven management system, univer-
sal input controls and menu cards. Most novel concepts were quickly rejected at the
card-sorting stage. Some further degree of novelty in interaction styles was intro-
duced in the scenario design, either by participants making requests for functionality
that was not provided on the tab board, or by collaborative discussion with designers.
Novel suggestions at this stage, however, were through the adaptation of control and
display features already provided as tabs rather than the consideration of radically
new interaction styles.
104 Bonner
Usability of Design Tools
Early guidance and learning of the design tools came through instruction rather than
reading the handbook that was provided. Progressively though, the designers used
their own initiative and spontaneously resolved many procedural problems as they
occurred. The designers gained confidence in using the tools and adapted the tools to
improve their performance with them. Once the first workshop was complete, both
designers expressed a clearer understanding of managing and conducting workshops
and identified further improvements for the second workshop, for example, they
revised the function filter.
Relevance of Design Tools to Designers
Initially, both designers expressed concern about the involvement of participants in
design decision making. They were wary of suggesting vague design ideas to partici-
pants in fear of appearing unprofessional or inexperienced. This concern quickly
evaporated once participants and designers became familiar with their roles. The
relationship between the designers and participants was more consultative than par-
ticipative in the card-sorting tool, but this quickly changed to positive active in-
volvement between both groups in scenario design. External support and advice from
the researcher was very important to the success and understanding of the design
tools, but it was also very evident that the designers progressively gained in confi-
dence and enjoyed using the tools. The designers were positive about the outcomes
and thought they had gained useful insights that would not have been gathered oth-
erwise.
Likelihood of Organisational Survival
The design manager and senior designer in the group were interviewed to discover
how they perceived the efficacy of the design tools, the quality of the design data and
interaction models and the quality of the final design solutions through reading the
management report.
The design manager was extremely encouraged by the adoption of the design
tools and felt that they reflected a recently implemented product development phi-
losophy. This process was driven by the organisation’s ‘core values’, including a
user-centred approach to product development rather than by historical organisa-
tional production methods. The introduction of the design tools was also regarded as
timely as the role and skills of the design team was beginning to change by develop-
ing more innovative and user-led product proposals. In order that the design tools
could gain greater acceptance, approval needed to be sought at a senior management
level. The design manager suggested that the design tools would only survive if the
final design solutions were sufficiently creative and in line with current product
development requirements. If this could be proven, the design tools could then be
‘sold’ outside of the immediate design group. It was important the design tools could
be explained and implemented succinctly to other disparate and culturally different
design groups within the organisation. This, he thought, could only be done through
effective training and not through a handbook, which should only be provided as
Lessons Learnt in Providing Product Designers with User-Participatory 105
reference material. He added that the design tools would have to compete against a
number of existing ‘tools’ used within the organisation.
The senior designer also expressed the importance of adaptability and again re-
ferred to the organisational changes that were currently underway. His view was that
design skills were going to be dissipated through the organisation and designers
would become part of ‘development groups’. In this sense, the design tools would
have to be acceptable to a much broader skills base. He also agreed that the design
tools could only survive if they could be integrated within their internal product
development process. There was little incentive for any member to use methods not
recognised as part of this process.
5.4 Reflections and Lessons Learnt from the Studies
5.4.1 Changing Designers’ Attitudes towards PD Takes Time
Observations from the studies clearly indicate the designers’ initial unease in involv-
ing users in the development process and allude to hesitancy and insecurity about
their own role in the participatory process. The designers were familiar with the use
of focus groups where consumers are consulted on their preferences to proposed
design variants. However, involving consumers in the design process and designing
through negotiation was anathema to their role as a designer. Trust needed to be
created between the researcher and designers before these more ‘radical’ studies
could take place. Many of the earlier iterative studies (not reported here) contributed
indirectly to the building of this trust between the researcher and designers. Without
these earlier iterative development studies, it would have been less likely that per-
mission would have been gained to carry out these studies on a live project. As one
of the designers reported, ‘if all this goes well – it will be our idea – if it all goes
wrong – it will be yours’.
Nevertheless, despite gaining this trust the designers still needed a high level of
support in implementing the design tools. At the introduction of any new design tool,
they were anxious to be provided with a ‘walk through’ to ensure they had grasped
the key concepts and flow of the workshops. The designers also spent time on careful
preparation of cards and tabs. This was despite the researcher’s strong encourage-
ment to use rough cards that offered more ambiguous, less detailed information to
prompt more open and potentially more fruitful collaborative discussion. This advice
was ignored in favour of cleanly composed cards and tabs for reasons that were due
mainly to professional pride. These resulted in some of the less-defined and poten-
tially more interesting concepts, such as cooking management systems, being
graphically represented as a complete solution, which gave the impression that they
were not open for discussion. As the designers’ confidence grew, and they became
more comfortable with their relationship with participants, the designers were pre-
pared to use more ‘rough and ready’ materials to encourage more critical debate
and allowed designers and participants to changes and alter cards and tabs where
necessary.
106 Bonner
5.4.2 Interaction Design Models are Critical to Participation
Many of the reported transcripts are centred on the interaction design models: the
task plan, tab board or the paper prototype interface. Much of the observational
analysis highlights limitations in the dialogue between designers and participants
where the interaction model has not been used as was anticipated, at least by the
researcher. Despite this, each model offered an environment where candidate sugges-
tions, with varying degrees of abstraction, could be proposed, designed and evalu-
ated. Interaction design models are essential to effective user–designer dialogue.
5.4.3 Organisational Credibility More Important than Experimental
Rigour
One of the key objectives of the study was to offer designers and participants a set of
design tools that permitted a systematic way of analysing an interaction design prob-
lem and that could, through reliable and valid means, provide guidance for design
solutions. One of the most interesting outcomes of these studies was that the re-
searcher’s misplaced emphasis on experimental rigour. For the researcher, the criti-
cal factor for success was achieving reliability and validity in the use of the design
tools. This was important because the aim was the eventual use of the design tools by
designers who would not have personal direction and support. It was assumed that
the design tools would only be accepted by the organisation if they could be proved
to provide consistent results. However, as the observations demonstrated, the design-
ers and design manager were not concerned about this at all. Effectiveness of the
tools was viewed entirely on the quality of the creative ideas generated from the
participants and how these could be translated into commercially oriented solutions.
Little regard was given to how this was achieved. The success of the tools was more
closely aligned to satisfying complex organisational demands and product develop-
ment procedures; they also needed to be marketable across different organisation
groups, be quick to learn while also producing commercially appropriate design
solutions.
The issue of organisational survival was investigated further. An opportunity
arose where the design tools could be examined within another product development
organisation. A discussion group session was set up with product designers within a
user-centred design group at a telecommunications company where the feasibility of
introducing new design tools into the product design group was explored. During this
discussion, hurdles were identified that might impede survival of the design tools.
Most of the product designers worked closely with human factors specialists, and
traditionally, capture of user requirements was regarded as a human factors role. The
designers needed to feel confident about using alternative design methods, which
they perceived as having a human factors philosophy, before exposing their design
methods to external criticism. Some concern was expressed about ‘treading on hu-
man factors territory’ and some of the group felt unsure about being able to provide a
robust design rationale for any design proposals using this approach. Concern was
also expressed about being able to gain access to users for workshop sessions. De-
signers were usually co-opted onto observation sessions and focus group meetings
Lessons Learnt in Providing Product Designers with User-Participatory 107
organised by other sections of the organisation. The group rarely organised participa-
tory sessions themselves.
It was not possible at the outset of this study to know how the design tools would
fit into the organisational culture but, certainly, a strong emphasis was given to pro-
viding a defensible design rationale. While this study has highlighted differing ex-
pectations between academic rigour and practical commercial needs, it has also re-
vealed the more precarious nature of implementing any form of innovative or radical
approach to product design. The successful implementation and use of PD tools must
address these organisational factors before success can be assured.
5.4.4 Rules of Engagement Between Designers and Participants (and
Researchers) Evolve Over Time
The reported observations illustrate how the roles of the three participant groups:
researcher, design team and participants changed over time. The extracted dialogues
demonstrate how participants move from ‘game players’ to active and engaging
members of the design team. Early dialogues deal with interpreting and understand-
ing the rules of engagement. This is replaced with more assertive discourse on how
to create and use more innovative interaction styles, particularly in the scenario de-
sign workshops. The users felt empowered and gained confidence in making useful
insightful comments, although this was often mixed with fanciful and unviable pro-
posals. Nevertheless, as far as the designers were concerned, this was not a negative
factor and simply added to their portfolio of possible design options. The designers
were very encouraged by outcomes from the workshops and many of the partici-
pants’ proposals were translated into design recommendations for a design manage-
ment report.
Similarly, the designers moved from passive observers to active facilitators of a
situated design process. The transcripts provide evidence of the designers’ moving
from supporting participants to active engagement. The designers often commented
on how much insight was gained about user behaviour from the studies. Observa-
tions from the two studies highlight how designers were often surprised at the level
of creativity and tolerance to ambiguity and contradictions in design decisions of-
fered. Very often, this helped broaden the scope of possible solutions while also
vindicating or rejecting preconceived design proposals. The designers, therefore,
increasingly used the design tools as a mechanism for testing preconceived design
proposals in preference to using them to create new interaction styles. They enjoyed
the design process but, as often noted, felt little compulsion to systematically probe
or document design decision-making activity.
The role of the researcher also changed. Initially, the researcher provided much
of the impetus to design and implement the tools, but as the confidence was gained
by the designers and participants this evolved to a passive and observation-based
role. The ‘summary statement’ (p*) was an important indicator that the tools were
beginning to work and the designers were beginning to view their products in terms
of an interactive dialogue with consumers. Once the designers took ownership of the
workshops, they could determine more precisely their own anticipated objectives.
Au: The summary
statement (p*) was
an…consumers’.
Please update page
number.
108 Bonner
These observations suggest the importance of recognising how rules of engage-
ment between participants and designers will alter over time as understanding and
confidence grows. In the formulation of PD methods, account should be made of
evolving engagement.
5.5 Conclusions
Organisational factors eventually overrode the potential implementation of these PD
methods. The organisational structure and the remoteness of the design group to key
design management decision making prevented their adoption, despite local success.
The studies, did, however, have a dramatic affect on the designers themselves. Dur-
ing an interview after these studies were carried out, two designers had reported how
the studies had radically changed their perception of what a designer’s role should
be. They had recognised the limitations of their user knowledge and as one of them
said, ‘you’d think I’d know how to use a cooker’. Furthermore, participation in the
studies added to the growing disillusionment about their role and impact in product
design decision making within the organisation. Two of the designers left shortly
after this research (this study was not a contributing factor!) and one of them used
this research as a case study in a subsequent job interview.
This study illustrates how design groups are often competing for funding, recog-
nition and credibility amongst a number of similar or related departments. For user-
participatory methods to gain broad acceptance within a manufacturing organisation,
they have to be first accepted by the design group and, more importantly, they need
to be confident of their acceptance within the organisation. User–designer involve-
ment in design methods may only survive if they are organisationally marketable as
well as liberating.
References
Beyer, H. and Holtzblatt, K. (1998). Contextual Design: Defining Customer-Centred Systems.
Morgan Kaufmann, San Francisco, California.
Bødker, S. (1991). Through the Interface: A Human Approach to User Interface Design. New
Jersey, Lawrence Erlbaum.
Brown, J.S. and Duguid, P. (1994). Borderline issues: Social and material aspects of design.
Human-Computer Interaction, 9(1), 3–36.
Carroll, J.M. (2000). Five reasons for scenario-based design. Interacting with Computers,
13(1), 43–60.
Constantine, L.L. and Lockwood, L.A.D. (1999). Software for Use: A Practical Guide to the
Models and Methods of Usage-Centred Design. Addison-Wesley, Reading, MA.
Ehn, P. (1992). Scandinavian design: On participation and skill. In: Adler, P.S. and Winograd,
T.A. (Eds.), Usability: Turning Technologies into Tools. Oxford University Press, New
York.
Greenbaum, J. and Kyng, M. (1991). Design at Work. Lawrence Erlbaum Associates, Hills-
dale, NJ.
Jacobson, I., Booch, G. and Rumbaugh, J. (1999). The Unified Software Development Process.
Addidson-Wesley, Reading, MA.
Lessons Learnt in Providing Product Designers with User-Participatory 109
Lafrenière, D. (1996). CUTA: A simple, practical and low-cost approach to task analysis.
Interactions, 3(5), 35–39.
Lewis, S., Mateas, M., Palmiter, S. and Lynch, G. (1996). Ethnographic data for product
development: A collaborative process. Interactions, 3(6), 52–69, November/December.
Muller, M.J., Tudor, L.G., Wildman, D.M., White, E.A., Root, R.W., Dayton, T., Carr, R.,
Diekmann, B. and Dykstra-Erickson, E.A. (1995). Bifocal tools for scenarios and repre-
sentations in participatory activities with users. In: Carroll, J.M. (Ed.), Scenario-Based
Design: Envisioning Work and Technology in System Development. John Wiley and Son,
New York.
Nardi, B.A. (1996). Studying context. In: Nardi, B.A. (Ed.), Context and Consciousness. MIT
Press, Cambridge, Massachusetts.
Schuler, D. and Namioka, A. (Eds.). (1993). Participatory Design: Principles and Practices.
Hillsdale, NJ, Lawrence Erlbaum Associates.
