ArticlePDF Available

DIMENSIONS OF KNOWLEDGE IN PROTOTYPING: A REVIEW AND CHARACTERISATION OF PROTOTYPING METHODS AND THEIR CONTRIBUTIONS TO DESIGN KNOWLEDGE

Authors:

Abstract and Figures

Whilst prior works have characterised the affordances of prototyping methods in terms of generating knowledge about a product or process, the types, or ‘dimensions’ of knowledge towards which they contribute are not fully understood. In this paper we adapt the concept of ‘design domains’ as a method to interpret, and better understand the contributions of different prototyping methods to design knowledge in new product development. We first synthesise a set of ten dimensions for design knowledge from a review of literature in design-related fields. A study was then conducted in which participants from engineering backgrounds completed a Likert-type questionnaire to quantify the perceived contributions to design knowledge of 90 common prototyping methods against each dimension. We statistically analyse results to identify patterns in the knowledge contribution of different methods. Results reveal that methods exhibit significantly different contribution profiles, suggesting different methods to be suited to different knowledge. Thus, this paper indicates potential for new methods, methodology and processes to leverage such characterisations for better selection and sequencing of methods in the prototyping process.
Content may be subject to copyright.
Cite this article: Real, R., Snider, C., Goudswaard, M., Hicks, B. (2021) ‘Dimensions of Knowledge in Prototyping: A
Review and Characterisation of Prototyping Methods and Their Contributions to Design Knowledge’, in Proceedings of
the International Conference on Engineering Design (ICED21), Gothenburg, Sweden, 16-20 August 2021. DOI:10.1017/
pds.2021.130
ICED21 1303
INTERNATIONAL CONFERENCE ON ENGINEERING DESIGN, ICED21
16-20 AUGUST 2021, GOTHENBURG, SWEDEN
ICED21 1
DIMENSIONS OF KNOWLEDGE IN PROTOTYPING: A
REVIEW AND CHARACTERISATION OF PROTOTYPING
METHODS AND THEIR CONTRIBUTIONS TO DESIGN
KNOWLEDGE
Real, Ricardo;
Snider, Chris;
Goudswaard, Mark;
Hicks, Ben
University of Bristol
ABSTRACT
Whilst prior works have characterised the affordances of prototyping methods in terms of generating
knowledge about a product or process, the types, or ‘dimensions’ of knowledge towards which they
contribute are not fully understood. In this paper we adapt the concept of ‘design domains’ as a
method to interpret, and better understand the contributions of different prototyping methods to design
knowledge in new product development. We first synthesise a set of ten dimensions for design
knowledge from a review of literature in design-related fields. A study was then conducted in which
participants from engineering backgrounds completed a Likert-type questionnaire to quantify the
perceived contributions to design knowledge of 90 common prototyping methods against each
dimension. We statistically analyse results to identify patterns in the knowledge contribution of
different methods. Results reveal that methods exhibit significantly different contribution profiles,
suggesting different methods to be suited to different knowledge. Thus, this paper indicates potential
for new methods, methodology and processes to leverage such characterisations for better selection
and sequencing of methods in the prototyping process.
Keywords: Design cognition, Design practice, New product development, Design learning
Contact:
Real, Ricardo
University of Bristol
Faculty of Engineering
United Kingdom
ric.real@bristol.ac.uk
1 INTRODUCTION
The creation of a prototype is a primary mediator between the designers cognitive model of a concept
and its materialisation in the physical world, Camere and Bordegoni (2016) describe this as an engage-
ment with the ’product-to-be’, querying the many elements inherent to an artefacts realisation. As such,
prototyping is regarded a critical activity in New Product Development (NPD) (Wall, Karl T. Ulrich,
and Flowers,1992) with a multitude of methods, and methodology encompassed within the creation
of prototypes; each influencing the nature of information and learning generated in the process (Gero,
1990). Prototypes, ranging from low to high fidelity, are frequently iterated throughout the NPD process
(Buchenau and Suri,2000) adopting various roles to probe a particular design challenge, or opportu-
nity (Camburn et al.,2017). Ullman (1992) classifies such prototypes into proofs of Concept, Product,
Process, and Production.
However, Bogers and Horst (2014) identify an understanding of prototyping to exist simultaneously
at different levels, an organisational level, and informal designer level where the tangible practices of
prototyping take place. Whilst the affordances of prototyping are generally understood organisationally,
the informal fuzzy front end of design lacks as clear a definition with designers often finding it difficult
to know which methods to use, and when to employ them in the design process (Lim, Stolterman, and
Tenenberg,2008). This suggests a differentiation between the types of knowledge implicit in prototyp-
ing, the designer requiring an imperative, procedural knowledge of processes and methods, as opposed
to a descriptive knowledge for more formal project management and operations (Engstrom,2009). Prior
works allude to a particular set of dimensions in which knowledge is required to realise a concept. Karl
T Ulrich (2003) defines the prototype as an approximation of the product along one or more dimensions
of interest, whilst Schon and Wiggins (1992) suggest a causality amongst dimensions, with actions in
one dimension having consequences in all others. It’s therefore hypothesised that by identifying these
dimensions in prototyping, and further characterising the contributions to design knowledge afforded by
various prototyping methods that designers could better select appropriate methods, and query specific
dimensions of interest, thus, supporting design processes and accelerating product development. To this
end, this paper aims to investigate and characterise differences in the knowledge contribution of differ-
ent prototyping methods, following which it will discuss the opportunities for application and how this
information may be used to better structure the prototyping process.
This paper begins by defining a set of knowledge dimensions for design prototyping, extracted from
related works. Following, it presents a study in which physical and digital prototyping methods at
varying fidelity were rated for their contribution to each knowledge dimension. Results were anal-
ysed statistically using Friedman’s ANOVA and post-hoc Dunn-Bonferroni pairwise comparisons to
detect differences in knowledge contributions of methods, and emergent patterns between method types.
Finally, the paper reflects on the impact of varying knowledge on prototype selection and sequencing,
and identifies opportunities for future work.
2 RELATED WORK
Prior works, particularly that of Schon and Wiggins (ibid.) in the study of architecture describe the
process of design as a reflective conversation with the materials of a design solution. The example
of a designer working in the medium of drawing is given, “the designer sees what is ‘there’ in some
representation of a site, draws in relation to it, and sees what has been drawn, thereby informing fur-
ther designing” the designer not only registers visual information, but identifies and gives meaning to
patterns beyond the medium itself.
Schon further outlines a set of ‘design domains’ in which these acts of making and ‘seeing’ 1register to
the different necessary elements in architectural design. These include Programme use, Siting, Build-
ing elements, Organisation of space, Form, Structure/Technology, Scale, Cost, Building Character,
Precedent, Representation, and Explanation. To develop the design, the designer must develop their
understanding in each of these domains and, as learning is tied to the activity, there is suggestion that
the media and activity of its use may substantially impact design development.
Prototypes in NPD take many forms with broad use, ranging from highly conceptual to near-final proof-
of-production models (Buchenau and Suri,2000). Such prototypes may range from analytic (virtual)
1Wittgenstein (2009) ’Seeing as’ judgement of a pattern as ’seen’ in the first sense.
1304 ICED21
to physical models, and may test individual components through to comprehensive systems (Karl T
Ulrich,2003). In so doing, prototypes will develop and test different aspects of the design (i.e. the
role, implementation, and look/feel prototypes of Houde and Hill (1997)) for a range of process goals
(Camburn et al.,2017), at a level appropriate to the process stage and required fidelity (Lim, Stolterman,
and Tenenberg,2008).
Given such breadth of method and purpose, that a set of design domains must be informed against to
produce a design, that learning is to some degree influenced by media, and that designers often do not
know which prototyping techniques to choose (ibid.), a question arises as to the inter-relation between
method, type, and knowledge produced. This paper aims to investigate this relationship through char-
acterisation of the ’design domains’ to which different prototyping methods contribute, and exploration
of how knowledge of such contributions may support better structuring of the prototyping process.
3 METHOD
To characterise this relationship, a study was performed in which participants were asked to rate a
variety of prototyping methods for their contribution towards the ’types’ of knowledge required to fully
develop a product. This section details the method followed in the paper, the study materials provided
to each participant and outlines the process participants followed in the study.
3.1 Extraction of Knowledge Dimensions from literature
As the ’design domains’ of Schon and Wiggins (1992) are situated within the architecture domain, it
was first necessary to recontextualise to NPD. Table 1 provides definitions of each as mapped to the
NPD domain. To provide differentiation against Schon and recognising differences in terminology of
the fields, these design domains are hereon termed Knowledge Dimensions.
Table 1. Knowledge dimensions, adapted fromSchon and Wiggins (1992)
ID Knowledge Dimension Description
KD1 Programme Use What the design is intended to do (i.e. its function)
KD2 Environment How the design performs within its intended conditions of use
KD3 Resources What is needed to make the design (e.g. materials, components, tools, time, cost)
KD4 Design Elements Identification of the features or components that will comprise the design
KD5 Form The shape and size of the design including how it looks and feels
KD6 Manufacturing Processes How the design will be made, the steps and tools required to make it
KD7 Configuration The arrangement of features and components, how the design fits together
KD8 Character How the design is supposed to look (including any context of brand and/or product family)
KD9 Explanation How the prototype, or elements of, communicate what it does
KD10 Lifecycle The envisaged life of the design, including its creation, use and disposal
3.2 Study Materials
Each participant was provided with a rating spreadsheet comprised of a structured table of prototyping
methods against which they were asked to rate each knowledge dimension. Methods were to be rated
on a 5-point Likert scale by the degree to which participants thought the method would inform each
respective knowledge dimension. Participants were additionally asked to consider prototyping in the
context of a single product and briefed using an example cordless drill; a labelled drawing of the drill
was included with study materials and available as reference throughout. Generation of study materials
required extraction of knowledge dimensions from literature, and generation of a list of prototyping
methods which will be detailed in the following sections. A snapshot of the rating spreadsheet is shown
in Figure 1.
3.3 Formulation and categorisation of prototyping methods
Prototyping incorporates a wide array of methods with varying properties and purposes (Karl T Ulrich,
2003). While it is unfeasible to generate an exhaustive list of potential methods, it is important that those
categorised within the study represent a breadth of the characteristics of interest within the study.
ICED21 1305
Figure 1. Extract from rating sheet
A list of 90 common prototyping methods was generated by the researchers based on literature and
experience. This list was reviewed by study participants prior to the study itself, who were asked to
suggest additional methods according to their own knowledge. The summary categories of extracted
methods are shown in Table 3. Following generation, methods were classified by the same researchers
according to the categories presented in Table 2. These categories were selected to support analysis
and align with wider research aims, see Section 1, namely spanning the prominent digital / physical
boundary in prototyping methods (Karl T Ulrich,2003), recognising that fidelity of prototyping may
vary depending on process stage and activity, and that both method domain (physical / digital) and level
of fidelity required may impact method choice in NPD. Although they were categorised, participants
rated each method individually. For brevity, Table 3 presents only categories and examples.
Table 2. Prototyping method classifications
Category Description
Physical or Digital Whether the method primarily comprises physical or digital matter
Level of Fidelity (Low or High) The degree to which dimensional precision is controlled for the method
Table 3. Prototyping method categories. All methods may be completed at a low or
high fidelity.
Name Physical/Virtual Description
Hand Processes Physical Any manual process implemented by the designer (e.g. foam modelling, card modelling)
Machine Fabrication Physical Any machine-based process manually controlled by the designer (e.g. turning, milling, cutting, joining)
CNC Machining Physical Any CNC controlled machining process including additive, subtractive and pattern cutting processes
Finishing Physical Any surface finishing process carried out by the designer (manual, or with machine tool assistance)
2D Drawing Physical Any drawing process, including thematic (e.g. sketch, rendering), schematic (e.g. technical drawing, layout),
and analytic (e.g. free body diagram)
2D Drawing Virtual Any computer based 2D drawing process (including categories as listed above)
3D Model Generation Virtual Generation of any 3D geometry
Visualisation Virtual Generation of renders, visualisations, and representations of as-final geometry
3.4 Study Process
Six participants partook in the study. Participants were all design engineers and active researchers
in the department of Mechanical Engineering, with expertise in prototyping processes. Participant
design/engineering industry experience was captured prior to commencing the study, with three partici-
pants indicating 5+ years experience, two participants between 1 and 5 years, and one participant noting
’other’ industry experience. The study comprised of two parts, first a participant briefing to familiarise
participants with study materials, and the prototyping method rating.
Participant Briefing Participants were provided with a study briefing document defining all elements
included within the rating spreadsheet, and providing examples of knowledge dimensions and prototyp-
ing methods. An initial meeting was held prior to rating with all participants, where the purpose of the
study was explained, all rated elements were defined, and participants were able to request clarification.
This process supported development of shared rater understanding.
1306 ICED21
Prototyping Method Rating Participants were requested to rate each prototyping method against each
knowledge dimensions on a 5-point Likert scale (1 - little or no information, 2, 3, 4, 5 - lots of informa-
tion). Of note is that participants rated each individual method against each knowledge dimensions
rather than the method category as a whole, where for brevity Table 3 shows categories only (i.e.
thematic, schematic, and analytic drawings were all separately rated). Participants were not given a
time limit for their rating. All participants completed all sections of the rating sheet. Once complete,
participants returned their sheets to the researcher for collation, processing, and analysis.
4 RESULTS
Once collected, survey results from each participant were collated into a single working sheet, and
median values for each category were extracted. Results were analysed to explore differences between
prototyping methods with respect to their contribution to knowledge dimensions, first through quan-
titative detection of difference, and second through characterisation of these differences. The results
comprise of two sections, the first compares knowledge dimensions across prototyping methods (Section
4.1) and the second identifies and explores differences between prototyping methods (Section 4.2).
All statistical tests were carried out with IBM SPSS 27. Note that throughout this section, knowledge
dimensions are referred both by name and ID as stated in Table 1.
4.1 Knowledge Dimensions across prototyping methods
Rated values for each knowledge dimension were analysed using SPSS through a non-parametric
Friedman’s repeated measures test, showing a statistically significant difference (χ2(9) = 288.390, p <
0.001) in the distribution of knowledge gained from different prototyping methods against the different
dimensions.
Following the Friedman test a post hoc analysis was carried out via means of a pairwise Dunn-
Bonferroni comparison to detect further quantitative difference in knowledge contributions to each of
the dimensions. The results for this are shown in Table 4. The results again show a statistical difference
in the contribution of methods to different knowledge dimensions, with a typcially higher contribution
to form (KD5) and programme use (KD2), and a lower contribution to lifecycle (KD10), explanation
(KD9), and manufacturing processes (KD6). Where knowledge against all dimensions is necessary for
completion of a design, this suggests that not all methods are equally appropriate.
Table 4. Knowledge
dimension ranking.
Knowledge Dimension Mean Rank
1 Form (KD5) 6.49
2 Programme Use (KD1) 6.18
3 Configuration (KD7) 5.91
4 Design Elements (KD4) 5.87
5 Character (KD8) 5.34
6 Resources (KD3) 5.29
7 Environment (KD2) 5.26
8 Manufacturing Processes (KD6) 5.11
9 Explanation (KD9) 5.06
10 Lifecycle (KD10) 4.49 Figure 2. Pairwise comparison of knowledge
dimensions
Additionally, results from the pairwise analysis indicate a measure of non-separability between dimen-
sions, as observed in the graph of pairwise comparisons (fig.2). The results suggest interdependence
between dimensions, showing a greater interdependence between dimensions illustrated in red, such as
Design elements (KD4)/Character (KD8), and Explanation (KD9)/Lifecycle (KD10) highlighted by the
distribution of prototyping method contributions in the analysis.
Table 5 lists the three prototyping methods that were rated as contributing to the highest degree to each
knowledge dimension. Difference between knowledge dimensions is evident, with (e.g.) programme
ICED21 1307
Table 5. Top-3 methods for each knowledge dimension. P/V indicate physical or virtual,
LF/HF indicate low or high fidelity (no index indicates both)
Dimension Top 3 Prototyping Methods
Programme Use P: CNC Additive(AM) HF P: Fabrication LF Analysis Drawings HF
Form P: CNC Subtractive(S) HF P: Moulding HF P: CNC AM HF
Resources V: CAM HF P: Moulding HF Schematic Drawing HF
Environment V: 3D Visualisation HF P: Fabrication HF P: CNC Subtractive HF
Design Elements P: Drawing HF V: CAD HF P: Fabrication LF
Configuration Schematic Drawing HF V: CAD HF V: 3D Visualisation HF
Manufacturing Process V: CAM P: CNC AM/S HF P: Moulding HF
Character V: 3D Visualisation HF P: Drawing HF P: Finishing HF
Explanation Drawing HF V: Visualisation P: Form Modelling LF
Lifecycle V: CAM HF P: Fabrication P: Moulding HF
use (KD1) and form (KD5) best informed by physical methods at high fidelity, while all other knowl-
edge dimensions show a mixture of physical and virtual. The following section investigates per-method
difference in detail.
4.2 Differences between prototyping methods
To identify high level differences between prototyping methods, each was grouped according to the
categories given in Table 2 (Physical or Digital, and Level of Fidelity). This was done to test the hypoth-
esis that each may have a significant impact on the efficacy of prototyping media against the knowledge
dimensions to which they contribute. Figure 3 shows the contribution of grouped methods to knowledge
dimensions.
Figure 3. Prototyping method categories and their contribution to knowledge dimensions
(5 showing a high contribution)
Several results are evident. First, low fidelity processes, whether physical or digital, contribute to a lower
degree against all knowledge dimensions when compared to their high fidelity counterparts. However
an exception to this is observed in lifecycle (KD10), where both high and low fidelity virtual processes
perform poorly. Secondly, physical processes consistently score higher to virtual across fidelities, with
the exceptions of programme use (KD1), form (KD5), design elements (KD4), configuration (KD7),
and character (KD8) where physical and virtual scores align. It is of note that those where alignment
1308 ICED21
occurs include dimensions more concerned with the geometry and layout of the prototype, than its wider
resource use, environment, and fabrication, where physical processes are considered more informative.
Of interest is that virtual processes are not considered as informative in explanation (KD9), which
concerns communication of the prototype and its use. As they are built towards specific purposes, this
finding may align with a) their lack of breadth in communication, where the specificity of the virtual
prototype restricts communication of its use as a whole product, and b) that building a virtual prototype
to fully mimic prototype use is a substantial task and less commonly performed.
Method-specific comparison
Tables 6 and 7 categories of prototyping methods against knowledge dimensions, showing low fidelity
methods and high fidelity methods respectively. Ratings shown are medians of participant answers with
variance calculated through median absolute deviation. Due to the lower number of participants in this
study, this variance lacks granularity and can be used for indication only.
Table 6. Median low fidelity prototyping method ratings against knowledge dimensions
Method Category Knowledge Dimensions
Prog. Use Form Res. Env. Des.Elem. Conf. Man. Char. Exp. Lifecycle
P: Hand Processes 3 3 2 2 2 2.5 2 3 2.5 1.5
P: Machine Fabrication 2.5 3 3 2 2 2 3 2.5 2 2
P: Machine CNC 2 3 2 2 2 2 2 2 2 1.5
P: Finishing 2 2.5 2 2 2 1 2 3.5 2 1
P: 2D Drawing 2.5 2 2 2 3 2.5 2 2 2 2
V: 2D Drawing 2.5 2 2 2 3 2.5 2 2 2 2
V: 3D Model Generation 2 3 2.5 2 2.5 3 2 3 2 2
Median 2.5 3 2 2 2 2.5 2 2.5 2 2
Variance 0.5 0 0 0 0 0.5 0 0.5 0 0
Table 7. Median high fidelity prototyping method ratings against knowledge dimensions
Method Category Knowledge Dimensions
Prog. Use Form Res. Env. Des.Elem. Conf. Man. Char. Exp. Lifecycle
P: Hand Processes 2 4 2 2 2.5 2 2 3.5 3 1
P: Machine Fabrication 3 4 3 2 2.5 3 3 3 2 3
P: Machine CNC 2.5 3.5 3 2 2.5 2 3 3 2 2
P: Finishing 2.5 3.5 3 2.5 2.5 2 3 3.5 3 2.5
P: 2D Drawing 3 3 3 2 3.5 3 2 2 2.5 2
V: 2D Drawing 3 2 2 2 3 3 2 2 3 2
V: 3D Model Generation 3 4 3 2 3 3 3 3 2 3
V: Visualisation 3 4 2 3 3 3.5 2 5 3.5 1
Median 3 3.75 3 2 2.75 3 2.5 3 2.75 2
Variance 0 0.25 0 0 0.25 0.25 0.5 0.5 0.5 0.75
Findings align with data shown in Fig. 3 above, with high fidelity methods rated as more contribu-
tory to each knowledge dimension. Exceptions exist only for lifecycle (KD10) and environment (KD2)
where both high and low fidelity were rated equally with a low score of 2. Most knowledge dimensions
show low variance across methods, particularly those of low fidelity, with only programme use (KD1),
configuration (KD7), and character (KD8) showing a non-zero median variance. This suggests either
that, in high level terms, many of the low fidelity prototyping methods are similarly capable across
knowledge dimensions, or that the Likert scale and number of participants within the study did not
provide sufficient data to discern a significant difference. This does not hold for high fidelity methods,
where a majority of knowledge dimensions show variance, with only programme use (KD1), resources
(KD3), and environment (KD2) clustered closely to central values. That some knowledge dimensions
show higher variation, particularly manufacture (KD6), character (KD8), explanation (KD9), and life-
cycle (KD10) for high fidelity (Table 7), and form (KD5), character (KD8), configuration (KD7) for
low fidelity (Table 6) suggests a need for some care during method selection. There may prove a higher
ICED21 1309
tendency for some methods to bias towards certain dimensions, and it may be likely that a requirement
for multiple prototypes exist when generating high fidelity prototypes that contribute to all.
Looking more specifically at the scores of grouped methods, several findings may be suggested.
For low fidelity methods, (Table 6):
Little appreciable difference is evident between methods with many showing very similar scores,
as evidenced by a low variance. This could suggest the viability of many methods to contribute
across knowledge dimensions during the low fidelity prototyping process.
Both virtual and physical drawing methods are better aligned with design elements (KD4) and
configuration (KD7), but rated less well against other dimensions. Both of these dimensions con-
cern more informative aspects with less reliance on geometry, function, and motion, and perhaps
are better suited to the informative representation style that drawings create. Should knowledge
against other dimensions be desirable, drawings may prove a less advisable choice.
Virtual model generation (i.e. CAD) is rated as capable or better than physical modelling methods
in the majority of categories. This suggests that low fidelity virtual modelling methods, perhaps
due to their flexibility in generation, viewing, and exploration, are well suited to prototype learning.
The exception here is in programme use (KD1), perhaps highlighting that functional assessment
of a low-fidelity prototype is difficult in the virtual domain.
Hand-based processes (i.e. manually constructed by the designer) in general score more highly
than machine-based process (either designer or computer controlled) in programme use (KD1),
configuration (KD7), and character (KD8). This may be due to the tangible exploratory nature of
hand fabrication and related cognitive processes, which require closer engagement with the feel of
the prototype, its arrangement, and in-process evaluation of use during fabrication.
Machine-based process are rated more highly in manufacturing (KD6) and resource (KD3), per-
haps due to the natural closeness of machine manufacturing to the final manufacturing processes
to be used during production.
Looking across high fidelity methods (Table 7), some similarities may be observed:
As with low fidelity, high fidelity virtual model generation methods show fairly similar values
to physical methods. It is apparent that the drop in virtual method capability when compared
to physical models shown in Figure 3 are due to lower scores for virtual drawing and visuali-
sation in specific dimensions, rather than general poor performance across all. For example, in
manufacturing (KD6), resources (KD3), and lifecycle (KD10).
The rating of hand-based process is relatively lower against machine-based processes, with lower
scores in programme use (KD1), resources (KD3), configuration (KD7), and manufacture (KD6).
This may be due to the importance placed on the physical form for utility in high fidelity prototyp-
ing, wherein machine fabrication methods are more likely to be of sufficient quality to be useful
(programme use (KD1)), and are representative of as-final manufacture (resources (KD3) and man-
ufacture (KD6)). Physical finishing and virtual visualisation - both methods of ensuring as-final
representation of the design, rate slightly higher against character (KD8), explanation (KD9), and
environment (KD2) than other methods. As methods that are closer aligned with communicating
as-final representation and use this is logical.
As with low fidelity, the least generally useful methods include drawing in both physical and
virtual. While some parity to others remains in design elements (KD4) and configuration (KD7),
drawings are considered substantially worse against form (KD5), perhaps due to their 2D nature
and need for cognitive interpretation.
Machine fabrication methods and virtual 3D modelling methods are rated as apparently strongest
across the knowledge dimensions, perhaps due to their precision against the design and viability for
use in evaluation, communication, and learning. For cross-dimension completeness, these methods
may prove a preferred choice.
5 DISCUSSION
These findings, while exploratory, present several potential implications for design prototyping and
selection of methods.
1310 ICED21
Contribution of methods to knowledge dimensions: It is evident from the findings that different
prototyping methods contribute to each knowledge dimension to differing degrees. Given the range of
prototyping methods that may be employed (see Karl T Ulrich,2003) this is not surprising - that a range
exists implies predisposition of methods to specific circumstances and learning. Across methods there
is a stronger contribution to the form (KD5) and programme use (KD1) dimensions, suggesting a pri-
mary mode of learning from prototypes aligns with geometry and utility against the design function,
as is likely to be desired in an engineering prototyping process. Some dimensions, namely lifecycle
(KD10), character (KD8), environment (KD2), and explanation (KD9) received either lower ratings or
higher variability in ratings, suggesting a lower capability of prototyping methods to contribute towards
them. Should knowledge in these dimensions be required, care in selection of prototyping methods may
be preferable. Of particular note is the low score received by lifecycle (KD10), indicating that product
considerations such as sustainability may not be well covered by the prototyping methods and therefore
need to be considered through other means within the product development process.
Selection and sequencing of prototyping methods: Prototyping typically requires a sequence of
physical and virtual iterations at varying fidelities (Goudswaard et al.,2021 - under review). Where
prototyping for a specific design requires contribution across dimensions or a specific focus may be
required, there may be a set of prototyping methods more suited to generation of this knowledge at
the level of fidelity the designer is currently working. For example, at low fidelities hand-processes are
rated higher than machine, while virtual 3D modelling processes are as high or higher in all dimensions
except programme use (KD1). Drawing methods are rated higher in informative rather than representa-
tive dimensions, highlighting a challenge in representation of geometry and function. At high fidelities,
there is a switch to higher ratings for machine-based processes over hand-based in physical, while vir-
tual 3D modelling maintains a high rating. Given such potential per-method differences, a given scenario
may be best suited to a specific or subset of methods depending on the knowledge dimensions required.
Further, prototyping methods are not equal in cost, time commitment, or skill requirement (Mathias
et al.,2019). The balancing of knowledge produced by methods against such variables may also allow
improved sequencing and selection of methods, whereby a range of variables may be balanced against
the required dimensions to meet process goals (i.e. reduced cost and time).
Study Limitations: While this study has produced statistically significant results, it used a compara-
tively small sample size and should be extended with further participants. In particular, extending the
range of industries and backgrounds of raters would provide interesting views on the variance associated
with each knowledge dimension and any relationships between variation and coder expertise. Findings
as presented hence provide direction, but require further exploration. Further, the findings of the study
should be validated against real prototyping processes as followed by designers in order to validate the
patterns that the study implies. These limitations form the basis for an ongoing study.
There is substantial possibility to extend results through further characterisation of the methods against
further variables such as process stage, and process performance metrics including cost, quality, and
time commitment. Findings here presented are context-neutral, and it is likely that method selection and
knowledge dimension rating may vary depending on the cost and time implications of method use.
Future work: Several areas of future work are implied by the findings, in addition to extensions to the
study discussed above. First, as methods vary in cost, skill, complexity, and time, it would be highly
beneficial to perform a deeper characterisation of the cost/benefit of each across various design prototyp-
ing scenarios, such that a guide to better method selection may be produced. In so doing, designers may
be guided towards methods and sequences of methods that are i) lowest cost, ii) quickest, iii) provide
the most complete, or iv) highest quality or contextually-appropriate contribution. Second, it is the act
of prototyping that in many cases supports generation of knowledge, and hence the transition between
iterations and methods that may prove of particular value. This implies the importance of sequencing
of prototype methods, whereby the transition between different forms and types may create emergent
learning. Specific study and characterisation of method sequencing and combination in context of the
cumulative addition to knowledge dimensions may support development of better prototyping processes
and tools. Finally, examination of the variation in methods across design scenarios would better inform
the extent to which knowledge dimensions may be used to classify method output.
ICED21 1311
6 CONCLUSION
With myriad prototyping techniques available, each with different media, method, and utility, that which
a designer selects to perform their prototyping activity potentially holds a large impact over the learnings
that are generated. Concurrently, designers often find it challenging to know which techniques to use
and when they should be employed. Using the concept of Knowledge Dimensions, this paper presents
a study relating prototyping methods to the form of information that they generate, thereby identify-
ing the relationship between methods and knowledge. A rating study was performed, evidencing first
that prototyping methods are not equal, with many showing strength in their contribution to geomet-
ric and function-based knowledge, but weakness in wider knowledge such as lifecycle. Further results
investigated methods across a high/low fidelity and physical/virtual boundary, and suggested equiva-
lence in some physical and virtual methods, and a switch in strength from hand-based to machine-based
methods as fidelity increases. Through such findings, this paper begins to form a body of knowledge
by which designers may better select methods with respect to the knowledge they require, and by
which researchers may better understand the relationships between prototyping activities, methods, and
knowledge, and support better structuring of the prototyping process.
ACKNOWLEDGMENTS
The work reported in this paper has been undertaken as part of the Twinning of digital-physical models
during prototyping project. The work was conducted at the University of Bristol, Design and Manufac-
turing Futures Laboratory (http://www.dmf-lab.co.uk) which is funded by the Engineering and Physical
Sciences Research Council (EPSRC), Grant reference (EP/R032696/1).
REFERENCES
Bogers, Marcel and Willem Horst (July 2014). “Collaborative Prototyping: Cross-Fertilization of Knowledge in
Prototype-Driven Problem Solving”. In: Journal of Product Innovation Management 31.4, pp. 744–764.
DOI: 10.1111/jpim.12121.
Buchenau, Marion and Jane Fulton Suri (2000). “Experience prototyping”. In: Proceedings of the 3rd conference
on Designing interactive systems: processes, practices, methods, and techniques, pp. 424– 433.
Camburn, Bradley et al. (2017). “Design prototyping methods: State of the art in strategies, techniques, and
guidelines”. In: Design Science 3.Schrage 1993, pp. 1–33. DOI: 10.1017/dsj.2017.10.
Camere, S. and M. Bordegoni (2016). “A lens on future products: An expanded notion of prototyping practice”.
In: Proceedings of International Design Conference, DESIGN DS 84.21100509737, pp. 155–164.
Engstrom, Stephen P (2009). The form of practical knowledge: A study of the categorical imperative. Harvard
University Press.
Gero, John S. (Dec. 1990). “Design Prototypes: A Knowledge Representation Schema for Design”. In: AI
Magazine 11.4, pp. 26–26. DOI: 10.1609/AIMAG.V11I4.854.
Goudswaard, Mark et al. (2021 - under review). “Characterising the prototyping practices of Design companies
in the South-West of the UK”. In: ICED 2021 conference proceedings.
Houde, Stephanie and Charles Hill (1997). “Chapter 16 - What do Prototypes Prototype?” In: Handbook of
Human-Computer Interaction (Second Edition). Ed. by Marting G. Helander, Thomas K. Landauer, and
Prasad V. Prabhu. Second Edition. Amsterdam: North-Holland, pp. 367–381. DOI: https://doi.
org/10.1016/B978-044481862-1.50082-0.
Lim, Youn Kyung, Erik Stolterman, and Josh Tenenberg (2008). “The anatomy of prototypes: Prototypes as
filters, prototypes as manifestations of design ideas”. In: ACM Transactions on Computer-Human
Interaction 15.2. DOI: 10.1145/1375761.1375762.
Mathias, David et al. (2019). “Accelerating product prototyping through hybrid methods: Coupling 3D printing
and LEGO”. In: Design Studies 62, pp. 68–99. DOI: 10.1016/j.destud.2019.04.003.
Schon, Donald A and Glenn Wiggins (1992). “Kinds of seeing and their functions in designing”. In: pp. 135–156.
Ullman, David G (1992). The mechanical design process. Vol. 2. McGraw-Hill New York.
Ulrich, Karl T (2003). Product design and development. Tata McGraw-Hill Education.
Wall, Matthew B., Karl T. Ulrich, and Woodie C. Flowers (1992). “Evaluating prototyping technologies for
product design”. In: Research in Engineering Design 3.3, pp. 163–177. DOI: 10.1007/BF01580518.
Wittgenstein, Ludwig (2009). Philosophical investigations. John Wiley & Sons.
1312 ICED21
... With prototyping comprising a range of media, activities, purposes, and outputs, it stands that a relationship (mapping) may exist between the media and activities that are chosen, and the learning that is generated. Prior work [8] has investigated such a mapping, highlighting different dimensions for knowledge that are required within a typical product design process to develop a solution, and proposing suitability and alignment of prototyping media with a set of 'knowledge dimensions' for product development. However, prior work focused on the generation of knowledge through the creation of a prototype, as opposed to by evaluating that which has been created. ...
... This paper first discusses related works and the need for the reported study, the methodology, and finally the findings from an analysis of evaluation methods against dimensions for knowledge in prototyping. The paper concludes by examining the coupling of evaluation methods with methods for prototype creation presented in prior works [8], exploring the selection and sequencing of prototyping tools (creation/evaluation) and the potential contributions to design knowledge across dimensions afforded by their coupling. ...
... Engineers often report that prototyping methods are chosen on an ad-hoc basis [11], and identify a lack of formal guidance on prototyping method selection [12], [13]. This uncertainty raises a risk and opportunity -through better understanding of prototyping methods and guidance on method selection, the prototyping process may be streamlined towards more efficient methods, while maintaining (or increasing) the relevance, detail, and utility of learning that is generated [8]. ...
Preprint
Full-text available
Presented at Design Computing and Cognition DCC'22. J.S. Gero (ed). Preprint to be published in Springer Nature 2022. Awarded runner up for best computing paper.
... Nelson et al. (2020) found that the use of digital prototyping tools was associated with fewer changes in requirements between prototyping stages, while traditional prototyping tools such as lathes showed no correlation with changes in requirements. In their work characterizing the types of knowledge provided by different prototyping methods, Real et al. (2021) found that physical prototyping methods were rated by designers as being consistently higher or equal with respect to providing design knowledge. Resource-intensive prototyping may lead to a perceived sunk cost (time and money) and induce design fixation, where designers may become attached to one design concept and neglect other worthy alternatives (Viswanathan & Linsey, 2011). ...
Article
Communication with external audiences is a critical task within the design process. Yet, we lack fundamental knowledge about how designers communicate design solutions and decisions to such audiences. This is particularly problematic for novice designers, as without such knowledge, we cannot develop pedagogical interventions to train novices as effective communicators. In this work, we study two strategies used by novices to communicate design knowledge – argumentation and prototypes. Through a move analysis and Markov modelling, we identified six unique rhetorical moves and how novices transitioned between them. We also identified several justifications and rhetorical devices used by novices that were driven by prototyping efforts. Educators can utilize these results to support students in scaffolding communication skills to develop design communication expertise.
... Further analysis of the results pointed to a differentiation in how teams interpreted the term low fidelity. Lim et al. [12], Liker & Pereira [11], Real et al. [19] and others have all contributed to the understanding and characterization of Fidelity levels and the variability in different prototyping methods. Some teams in this research picked an abstract starting point for their fidelity level of bananas, coins, olives, sugar packets, etc. ...
... In recent years, studies have investigated how one can capture early-stage prototyping activities and how the results of one prototyping activity have informed subsequent activities (Erichsen, et al., 2021;Giunta, et al., 2022), resulting in networks of connected activities. Other works have begun to identify the key knowledge elements that one should capture during a prototyping activity and their relationship with the prototyping media used (Real, et al., 2021). The work to date has highlighted the necessity of capturing the relationships between prototyping activities and their interconnections and fed back into the design process. ...
Article
Full-text available
Prototyping is a well-established and valued design process activity. However, capturing prototypes and the tacit knowledge that led to and was gained from their creation is a challenge. Beyond that, questions remain on how best to utilise that captured data. This paper looks at how one can exploit and generate insights from data that has been captured, specifically looking at graph databases, the network analysis techniques they permit and the differing fidelities of visualisation and interactivity that they enable.
Chapter
Prototyping in New Product Development (NPD) encompasses a broad selection of methods used to generate knowledge about a product or process. Whilst some methods focus on the creation of a prototype in its intended domain, others centre on its testing and evaluation, contributing to an understanding of the prototype’s performance against a set of design requirements or objective. Where prior works have explored the contributions to design knowledge afforded by methods of creation, methods used to evaluate prototypes lack a similar characterisation.
Chapter
This paper investigates team psychological safety (N = 34 teams) in a synchronous online engineering design class spanning 4 weeks. While work in this field has suggested that psychological safety in virtual teams can facilitate knowledge-sharing, trust among teams, and overall performance, there have been limited investigations of the longitudinal trajectory of psychological safety, when the construct stabilizes in a virtual environment, and what factors impact the building of psychological safety in virtual teams.
Article
Full-text available
This paper introduces Hybrid Prototyping as a way to couple different prototyping methods; combining their complementary affordances and mitigating their limitations. To characterise and investigate this approach, a simulation-based study was conducted into the coupling of low-cost 3D printing and LEGO®. Key benefits hypothesised are reduced fabrication time and increased reconfigurability. Six primitive 3D shapes are simulated using a continuum of hypothetical brick sizes. Results show a reduction in fabrication time of 45% and a reconfigurability of 57% at the optimum. A case study highlights the compounded improvements over 3D printing for an iterative prototyping process. These findings mean that increases in prototyping iterations can be made due to reduced time and material costs, accelerating the product development process.
Article
Full-text available
Prototyping is interwoven with nearly all product, service, and systems development efforts. A prototype is a pre-production representation of some aspect of a concept or final design. Prototyping often predetermines a large portion of resource deployment in development and influences design project success. This review surveys literature sources in engineering, management, design science, and architecture. The study is focused around design prototyping for early stage design. Insights are synthesized from critical review of the literature: key objectives of prototyping, critical review of major techniques, relationships between techniques, and a strategy matrix to connect objectives to techniques. The review is supported with exemplar prototypes provided from industrial design efforts. Techniques are roughly categorized into those that improve the outcomes of prototyping directly, and those that enable prototyping through lowering of cost and time. Compact descriptions of each technique provide a foundation to compare the potential benefits and drawbacks of each. The review concludes with a summary of key observations, highlighted opportunities in the research, and a vision of the future of prototyping. This review aims to provide a resource for designers as well as set a trajectory for continuing innovation in the scientific research of design prototyping.
Conference Paper
Full-text available
In this paper, we describe "Experience Prototyping" as a form of prototyping that enables design team members, users and clients to gain first-hand appreciation of existing or future conditions through active engagement with prototypes. We use examples from commercial design projects to illustrate the value of such prototypes in three critical design activities: understanding existing experiences, exploring design ideas and in communicating design concepts.
Article
This paper presents an inductive study that shows how collaborative prototyping across functional, hierarchical, and organizational boundaries can improve the overall prototyping process. Our combined action research and case study approach provides new insights into how collaborative prototyping can provide a platform for prototype-driven problem solving in early new product development (NPD). Our findings have important implications for how to facilitate multistakeholder collaboration in prototyping and problem solving, and more generally for how to organize collaborative and open innovation processes.
Article
Architectural designing is described as a kind of experimentation that consists in reflective 'conversation' with the materials of a design situation. A designer sees, moves and sees again. Working in some visual medium -- drawing, in the article examples -- the designer sees what is 'there' in some representation of a site, draws in relation to it, and sees what has been drawn, thereby informing further designing. In all this 'seeing' the designer not only visually registers information but also constructs its meaning -- identifies patterns and gives them meanings?? ???h) ?????????0*0*0*???? ???? beyond themselves. Words like 'recognize,' 'detect,' 'discover' and 'appreciate' denote variants of seeing, as do such terms as 'seeing that,' 'seeing as' and 'seeing in.' The purpose here is to explore the kinds of seeing involved in designing and to describe their various functions. At local and global levels, and in many different ways, designing is an interaction of making and seeing, doing and discovering. On the basis of a few minuscule examples, the authors suggest some of the ways in which this sort of interaction works. Some conditions that enable it to work are described. And some of its consequences for design education and for the development of computer environments useful to designers are drawn
Book
This book is about best practices for the design of mechanical products. It is available from Amazon and other sources at a reasonable price.
Article
Firms that design mechanical and electromechanical products confront a variety of difficult issues in their prototyping activities. For a given part, how can a choice among fabrication technologies be made? Where should investments in new prototyping technology be focused? How can new and existing prototyping technologies be evaluated? Our primary goal has been to develop a systematic method of evaluating prototyping processes in order to determine the best process for a given situation. A secondary goal has been to map the space of prototyping processes in order to determine future process development needs. Using data from a field study at the Kodak Apparatus Division, we have developed a systematic method for evaluating and selecting prototyping processes. Our data are drawn from (1) a user survey of prototyping perceptions and needs, (2) a survey to determine the importance of various prototype part performance attributes, and (3) estimates of the fabrication time, cost, and part performance for 104 parts and four prototyping processes.