Research methods in engineering design: a synthesis of recent studies using a systematic literature review

Abstract

The relation between scientific research and engineering design is fraught with controversy. While the number of academic PhD programs on design grows, because the discipline is in its infancy, there is no consolidated method for systematically approaching the generation of knowledge in this domain. This paper reviews recently published papers from four top-ranked journals in engineering design to analyse the research methods that are frequently used. The research questions consider the aim and contributions of the papers, as well as which experimental design and which sources of data are being used. Frequency tables show the high variety of approaches and aims of the papers, combining both qualitative and quantitative empirical approaches and analytical methods. Most of the papers focus on methodological concerns or on delving into a particular aspect of the design process. Data collection methods are also diverse without a clear relation between the type of method and the objective or strategy of the research. This paper aims to act as a valuable resource for academics, providing definitions related to research methods and referencing examples, and for researchers, shedding light on some of the trends and challenges for current research in the domain of engineering design.

Full text

Vol.:(0123456789)
1 3
Research in Engineering Design (2023) 34:221–256
https://doi.org/10.1007/s00163-022-00406-y
ORIGINAL PAPER
Research methods inengineering design: asynthesis ofrecent studies
using asystematic literature review
DavidEscudero‑Mancebo1,2 · NievesFernández‑Villalobos1,3 · ÓscarMartín‑Llorente1 ·
AlejandraMartínez‑Monés2
Received: 8 November 2021 / Revised: 26 October 2022 / Accepted: 23 December 2022 / Published online: 16 January 2023
© The Author(s) 2023
Abstract
The relation between scientific research and engineering design is fraught with controversy. While the number of academic
PhD programs on design grows, because the discipline is in its infancy, there is no consolidated method for systematically
approaching the generation of knowledge in this domain. This paper reviews recently published papers from four top-ranked
journals in engineering design to analyse the research methods that are frequently used. The research questions consider
the aim and contributions of the papers, as well as which experimental design and which sources of data are being used.
Frequency tables show the high variety of approaches and aims of the papers, combining both qualitative and quantitative
empirical approaches and analytical methods. Most of the papers focus on methodological concerns or on delving into a
particular aspect of the design process. Data collection methods are also diverse without a clear relation between the type of
method and the objective or strategy of the research. This paper aims to act as a valuable resource for academics, providing
definitions related to research methods and referencing examples, and for researchers, shedding light on some of the trends
and challenges for current research in the domain of engineering design.
Keywords Research methodologies in engineering design· Engineering design and evaluation
1 Introduction
Doctoral studies have a long tradition in higher education
systems (Bogle 2018). Doctoral studies are highly relevant
because they are considered as a key for technical devel-
opment and industrial excellence in developed countries.
Normally, a PhD diploma is compulsory for pursuing and
it is highly valued for getting involved in research projects
in companies. The goal of doctoral programs is to provide
postgraduates with competences for the generation of knowl-
edge in a given domain. The means to generate knowledge
depends on the area, being research methods and techniques
potentially different, and evolving in parallel with the devel-
opment of the domain. In young domains such as Engineer-
ing Design, the discussion about which research procedures
and paradigms should be employed is still open.
Simon (1996), in his book The Science of Design, defined
design as a search for an optimum in a space of alternatives
that take into account the specifications and restrictions of
a given problem. Hatchuel (2001) highlighted limitations
of Simon’s position discussing that designing cannot be
reduced to taking decisions among a bounded set because
the number of concepts related to the problem and the possi-
ble number of decisions to be taken could be expandable and
uncountable, not only due to human creativity but also to
social interaction. (Subrahmanian etal. 2020) place Simon
and Hatchuel’s approaches in a historical timeline that
describes different models of how designing is understood,
evidencing the challenges for research design as a discipline
that defines a common language that includes the impact of
context and users indesigning, in addition to the problems..
Probably due to the youth of design as a research discipline,
or due to its socio-technical nature, it does not yet have a
consolidated research methods and techniques. Blessing and
* David Escudero-Mancebo
descuder@infor.uva.es
1 Escuela de Ingenierías Industriales, Universidad de
Valladolid, Paseo del Cauce Paseo del Cauce 59,
47011Valladolid, Spain
2 Escuela de Ingeniería Informática, Universidad de Valladolid,
Paseo de Belén 15, 47011Valladolid, Spain
3 Escuela Técnica Superior de Arquitectura, Universidad de
Valladolid, Avenida Salamanca 18, 47014Valladolid, Spain
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

222 Research in Engineering Design (2023) 34:221–256
1 3
Chakrabarti (2009) proposed the DRM (Design Research
Methodology) motivated by “frustration about the lack of
a common terminology, benchmarked research method and
a common research methodology in design”. Through the
analysis of recent research papers, this work has the aim
to confirm how these visions about research in engineering
design are projected in current state-of-the-art publications.
Since the work of Blessing and Chakrabarti, there have
been some relevant proposals that have shed light on differ-
ent aspects of the global design research landscape. Koski-
nen etal. (2011) proposed the term ‘constructive design
research’ and presented alternatives to integrate research
within the practice of design. Joost etal. (2016) used the
term ‘design as research’ in a volume that compiled dis-
courses of experts about questions on design research and
its relationship with other disciplines. Vaughan (2017) pre-
sented a survey that collected different points of view related
to doctoral education in the opinions of design graduates
about practice-based research design. Redström (2017) pre-
sented an essay about how to develop theory -knowledge- by
practice, experimentation and making -design. These works
are a multi-faceted compendium of practical experiences and
visions of experts on how to perform activities related to
research in the domain of design. Although many examples
and discussions presented in the cited books focus on the
topic of research through/by design, rather than on research
in engineering design, all of them agree on the relevance
of research into the design due to the increasing number of
PhD programs that could benefit from background knowl-
edge about this topic. In this paper, we present an alternative
approach to shed light on the relations between research and
design: instead of collecting the personal visions of experts,
we summarise and classify research papers on research in
Engineering Design in terms of aims and contributions,
methods and approaches, data collection techniques, and
research instruments used for the collection of data. To this
end, we have carried out a systematic review of the literature
on research in engineering design. The overarching research
question (RQ) that drives the review is: What is the current
landscape of research methods in engineering design?
Access to doctoral studies normally requires candi-
dates to have a Master’s degree in which they have taken
courses about research methodologies. Doctoral stud-
ies normally culminate with the defense of a PhD thesis
in which postgraduates have to show their capabilities to
generate knowledge in a specific field. Submitting a PhD
thesis that includes activities previously reviewed in sci-
entific journals is generally considered as a quality war-
ranty of the research performed by the student. Although
publishing journal papers is not the only way to assess the
excellence of the research work performed in a PhD thesis,
the quasi-exponential increase of scientific publications we
are witnessing (Tenopir and King 2014) indicates that it is
probably becoming a universal standard for rating the qual-
ity of research. Therefore, being aware of the kind of works
published in scientific journals related to engineering design
could be of outstanding importance for scholars who have
to configure the contents of the courses related to research
methodologies in this field, as well as for PhD supervisors
and students to focalize efforts for being more productive in
terms of publications. The analysis of scientific papers about
research in engineering design performed presented in this
paper aims to contribute to this aim.
There are many possible ways to analyse, categorise or
classify research works because there are many dimensions
of analysis. Creswell (2009) presents a classical distinction
between (1) quantitative, (2) qualitative and (3) mixed-
methods (combining qualitative and quantitative research
methods). For quantitative methods experimental designs,
non-experimental design are distinguished. For qualitative,
narrative research, ethnographic research, phenomenologi-
cal research, grounded theory and case study research are
distinguished. For mixed-methods, sequential, concur-
rent and transformative methods are distinguished. Bless-
ing and Chakrabarti (2009) identified the following ones:
(1) paradigm, that includes empiricism (Randolph 2003;
Solomon 2007) and ethno-methodology (Atkinson 1988),
methodologies, theories, views and assumptions (Kothari
2004); (2) aim, research questions and hypotheses; (3)
nature of the study, including observational vs interventional
(Thiese 2014), comparative vs non-comparative; (4) units
of analysis; (5) data collection methods including record-
ings, interview, questionnaires (De Leeuw 2008); (6) role
of the researcher (Fink 2000); (7) time constraints, duration
and continuation of the research process; (8) observed pro-
cesses including layout drawing, prototype or product; (9)
setting referring to laboratory or field research (Paluck and
Cialdini 2014); (10) tasks including type and complexity
and nature; (11) number of cases, case size and participants
(Diggle etal. 2011); (12) object of analysis distinguishing
objects, companies, projects, documents… (13) coding and
analysis, analysis and (14) verification methods (Brewer and
Crano 2014); or (15) findings, that is, statement models or
conclusions resulting from the study. Reich and Subrahma-
nian (2021) use the PSI framework (Problem, Social and
Institutional space) to analyse and categorise research design
works focussing on dimensions related to the problem being
addressed concerning (1) disciplinary, (2) structural com-
plexity and (3) knowledge availability; dimensions related
with who is included in designing concerning (4) the per-
spective required to formulate the problem, (5) the inclu-
sion of participants in the design process and the (6) capa-
bilities of the design team; and finally dimensions related
with how designing is executed taking into account (7) the
ties or connections between actors, (8) the accessibility to
knowledge and (9) the institutional complexity (Reich and
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

223Research in Engineering Design (2023) 34:221–256
1 3
Subrahmanian 2020). The dimensions presented by Bless-
ing and Chakrabari have the ambition to classify different
aspects to be taken into account when research in engineer-
ing design works are tackled. The dimensions proposed by
Reich and Subrahmian are complementary and arise when
they analyse the factors influencing success in engineering
design projects. When analysing papers, some of the details
related to some of the listed dimensions could be missing in
the descriptions (timing, success validation etc.) so that we
had to devise alternative proposals.
Our analysis pivots around the division between empiri-
cal qualitative, quantitative research and mixed-methods
proposed by (Creswell 2009). This classification was com-
plemented with analytical research methods, as specified by
(Adrion 1993), cited by (Glass 1995) (defined in Sect.2.2).
From this germinal division, data-collection methods, strate-
gies, and contributions of the studies are reported in cross-
analysis tables. We aim to identify the main goals and results
pursued or obtained by researchers (dimensions 2 and 15 of
Blessing and Chakrabarti 2009), the strategies of enquiry
and methodologies they follow (dimensions 1, 3, 9, 10 of
Blessing and Chakrabarti 2009), and which data sources and
instruments are most (and least) commonly used (rest of
dimensions of Blessing and Chakrabarti 2009) in the domain
of engineering design.
The structure of the document is the following: First,
we present the review method and the categories used to
classify the papers. We then present the quantitative results
of the number of papers in each of the categories and the
cross relations of the different classes, shedding light on the
relative weight of each of the qualitative and quantitative
approaches and the most frequent data-collection methods
used. Next, we discuss the usefulness of the obtained results
for academics and professionals interested in research design
and the paper ends with the conclusions. Complementary
material is provided with a brief description of each of the
analysed papers.
2 Method
We follow Kitchenham etal. (2009) as a guideline for per-
forming the systematic review. The nature of the research
question did not suit a usual search in the databases, as we
were interested in analysing the approaches to research pub-
lished in the field of engineering design. For this reason, we
focused on identifying papers published in relevant journals
in the field. The data sources are journal papers in the field
of engineering design.
A simple search in the Journal of Citation Reports using
the term “Design” as a key search title criterion, generates
a list of 99 journals indexed in different categories. Only
80 are indexed in 2020, the rest of them in previous years.
As we aimed to high-impact journals reporting research in
engineering design, we focused on the journals indexed in
SCIE (Science Citation Index Expanded) related to Science
and Technology, discarding the 22 journals indexed in ESCI
(Emerging Sources Citation Index), the 10 indexed in AHCI
(Arts and Humanities Citation Index) and the 5 indexed
in SSCI (Social Sciences Citation Index). Among the 43
remaining journals indexed in SCIE, 13 of them correspond
to categories related to Chemistry and Biology (for example
Anti-Cancer Drug Design or Molecular System Design &
Engineering) 11 of them to Computer Science or Electrics
(for example Design Codes and Cryptography or Computer
Aid Design); 3 with Mathematics (for example Journal of
Combinatorial Design) and 2 with Building (Architectural
Engineering and Design Management or Structural Design
of Tall and Special Building). Closer to engineering design
are the 14 remaining journals: 4 indexed in Mechanics
Journal of Mechanical Design, Mechanics Based Design of
Structures and Machines, Journal of Advanced Mechanical
Design Systems for Manufacturing and Journal of Strain
Analysis for Engineering Design), 4 related to Materi-
als (Materials & Design, Proceedings of the Institution of
Mechanical Engineers, International Journal of Mechan-
ics and Materials in Design and Road Materials and Pave-
ment Design); and 2 related with vehicle design (Journal
of Ship Production and Design, and International Journal
of Vehicle Design). In spite of being closer to the topic of
research in engineering design, we discarded these journals
for being too specific. The remaining 4 journals were: (i)
Design Studies (DS), (ii) the International Journal of Design
(IJD), (iii) the Journal of Engineering Design (JED) and (iv)
Research on Engineering Design (RED). Table1 shows that
these journals share the category denominated “Engineering
Multidisciplinary”. In this category, there are 6 journals that
have the term “Design” in the title, the four selected plus
International Journal of Technology and Design Education
(also indexed in SSCI), Artificial Intelligence for Engineer-
ing Design Analysis and Manufacturing (also indexed in
Computer Science) that were discarded for being specialized
in education and in artificial intelligence with applications
in engineering design, respectively, and therefore, out of the
focus of our research.
Each of the selected journals declare in their presenta-
tion their aims and audience: RED focuses on design theory
and methodology, DS focuses on design processes, JED
focuses on different aspects of the design of engineered
products and systems, and IJD publishes research papers
in all fields of design. The audience of DS, JEC and IJD is
broader than the one of RED, which focuses on mechanical,
civil, architectural, and manufacturing engineering. Overall,
the four journals constitute a rich and representative sample
that includes works of diverse nature, applying a variety of
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

224 Research in Engineering Design (2023) 34:221–256
1 3
Table 1 Journals that are the focus of interest in the study. Eng Mult is the category named “Engineering, Multidisciplinary”, Eng Manu the one named “Engineering, Manufacturing” and Eng
Ind the one named “Engineering, Industrial” of the SCIE JCR index.SS Inter is the category “Social Sciences, Interdisciplinary” of the SSCI JCR index. In the cells, A/B figures mean number
of reviewed (A) papers versus total number of papers (B). Special issues are underlined.
Eng
Mult
Eng
Manu
Eng Ind SS Inter Impact
factor
(2020)
#TOTAL N/2018 D/2018 J/2019 F/2019 M/2019 A/2019 M/2019 J/2019 J/2019 A/2019 S/2019 O/2019 N/2019
Design
Studies
X X 2.780 17/35 0/0 0/0 1/6 0/0 1/1 0/0 2/4 0/0 8/8 0/0 2/7 0/0 3/9
Interna-
tional
Journal
of
Design
X X X 1.923 17/17 0/0 8/8 0/0 0/0 0/0 5/5 0/0 0/0 0/0 4/4 0/0 0/0 0/0
Journal of
Engi-
neering
Design
X 2.588 17/24 1/2 0/0 0/0 1/1 0/0 3/3 0/0 2/2 1/1 5/5 0/0 4/10 0/0
Research
in Engi-
neering
Design
X X X 2.655 17/28 0/0 0/0 7/8 0/0 0/0 4/8 0/0 0/0 4/7 0/0 0/0 2/5 0/0
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

225Research in Engineering Design (2023) 34:221–256
1 3
research methods and approaches to different problems in
the context of research in engineering design.
Sample selection in systematic literature reviews must
be structured, comprehensive, and transparent (Hiebl
2021). To comply with these three requirements, we estab-
lished a recent and limited temporal window and applied
random selection to select the sample. We collected 17
papers from each journal, as 17 is the number of papers
available in one of the journals under analysis (IJD) and
we chose to use the same number of papers per journal
to avoid bias (i.e., giving more importance to one journal
than another) in the study. For the journals with more than
17 papers in the period of analysis, random selection was
applied. We focused on papers published between Novem-
ber 2018 and November 2019, which was the most recent
available time window when this work was started.
This methodology led to a final total of 68 papers. We
followed a collaborative team-coding approach (Saldaña
2021). Papers were selected and assigned randomly to a
pair of reviewers. Each reviewer coded two papers every
two weeks. Disagreements and new code proposals were
resolved in periodic meetings involving the four research-
ers/authors. The first author of this paper played the role of
“codebook editor” (MacQueen and Guest 2008), updating
the code list after the meetings and he used the data from
the analysis to build the final tables and present the result-
ing themes derived from the study.
With the aim of answering the general question of this
review, RQ:, “What is the current landscape of research
methods in engineering design?”, we focused on the fol-
lowing more specific sub-questions:
RQ1: What are the research goals pursued by the analysed
works?
RQ2: What are the main experimental approaches found
in the reviewed papers?
RQ3: What data collection methods are employed in the
reviewed works?
RQ4: Which instruments are normally used to collect
these data?
To answer these questions, we followed an anticipated
data condensation approach (Miles etal. 2020). We defined
four overarching topics corresponding to the research sub-
questions: aims and contributions of the research; research
approach; data collection techniques; and instruments for
the collection of data. For each topic, we defined a set of
categories, based on our revision of engineering design
methods (see Sect.2). During the iterative coding work,
emerging categories were included when required. The
new categories were used to re-codify all the works. This
combination of deductive and inductive coding enabled us
to derive new meanings from the data.
In the rest of this section, we present the categories that
were identified in the analysis under each topic. Appen-
dix shows complementary information with representative
examples of the categories.
2.1 Aims andcontributions
Concerning the aims/contributions of the research (RQ1),
we started from an empty list of research targets which
was enriched as the number of reviewed papers increased.
Finally, the following research goals were identified through
the coding process:
To study or propose a methodology, that focuses on
papers whose main objective is to study an existing design
methodology by analysing its validity in works that propose
a new design methodology or that develop a part of it more
deeply.
To delve into a given aspect of design, which includes
papers that focus on exploring an aspect of a design (team
communication, sketching, generation of ideas, materi-
als...) or that explore one area of design that is recognised
as challenging (social design, inclusive design, ecological
design...).
To design, develop, or test a specific product, which
includes those papers that set out the process of creation or
development of a specific product or a group of them. Some
of these works describe the overall process of creating a
product, and others focus on a specific phase of its develop-
ment (research, ideation, testing, and validation).
To make recommendations or propose guidelines,
which include articles whose main aim is to systematize the
results of their research to provide advice, either at a meth-
odological level or in the design of new products.
Proposing a theory includes those articles that use logi-
cal reasoning or mental operations, such as imagination,
intuition, abstraction, and deduction, with the aim of enunci-
ating concepts or creating models, explanations, or theories
about the phenomena under study.
Proposing a framework of analysis or a taxonomy that
enables concepts or objects to be classified into categories.
More than one code could be assigned to each of the
papers. This could be the case of a paper that aims to develop
a specific product and ends by proposing guidelines.
2.2 Research approach
Concerning experimental approaches found in the reviewed
papers (RQ2), as explained in the introduction, we propose
the use of the distinction between quantitative, qualitative,
mixed, and analytical research methods, defined as:
Quantitative empirical studies are those that aim at test-
ing theories by examining relationships between variables,
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

226 Research in Engineering Design (2023) 34:221–256
1 3
based on the collection of numerical data which is analysed
using statistical procedures.
Qualitative empirical studies are those that aim at
exploring and understanding in depth the meaning that indi-
viduals or groups give to a problem. They usually involve
the collection of non-numerical data obtained in the partici-
pants’ settings and follow inductive analysis approaches in
which the researchers interpret the meanings of the collected
data.
Mixed-methods studies are those that combine both
quantitative and qualitative approaches at diverse levels
(data sources, analytical methods, etc.), so that the overall
study is stronger than using each of the two approaches (i.e.,
quantitative, or qualitative) separately.
Analytical studies are those that focus on the formali-
zation of a model and its demonstration. They start out by
proposing a formal model with a mathematical formulation,
derive results using deductive approaches, and, if possible,
compare these results with empirical observations.
With respect to quantitative empirical studies, we subcat-
egorize them into experiments, quasi-experiments and non-
experiments, depending on the way the subjects of interest
are assigned to an experimental group or to a control group:
Experiments: the assignment of subjects to the experi-
mental or to the control group is random.
Quasi-experiments: there is not a random assignment of
a subject to the groups.
Non-experiments: there is not control on the grouping
of subjects.
When a known qualitative strategy of inquiry is used,
it is also tagged. According to the definition proposed by
Creswell (2009), strategies of inquiry are types of methods,
designs or models that provide specific direction for proce-
dures in a research design.
Ethnographic research documents the beliefs and prac-
tices of a particular cultural group or phenomenon in its
natural environment from the perspective of insiders (Lapan
etal. 2012). The researcher stays on site for a considera-
ble amount of time to analyse practices and behaviours of
groups, by observing, interviewing and (sometimes) partici-
pating in the process under analysis. Very popular in social
sciences, it is also used in architecture (Cranz 2016).
In phenomenological research, the researcher identifies
the essence of human experiences about a phenomenon as
described by participants, while the researcher sets aside his
or her own perspective (Wilson 2015).
Grounded theory is a strategy of inquiry in which the
researcher derives a general theory grounded in the views
of participants, involving the use of multiple stages of data
collection (Jørgensen 2001).
Hermeneutics inquiry focuses on disclosing how partici-
pants’ interpretations of a phenomenon determine the way
they live in the world (Stigliano 1989). This technique is
popular in architecture (Pérez-Gómez 1999).
Case study research is an empirical strategy of inquiry
that investigates a contemporary phenomenon within its
real-life context (Yin 2009). It uses descriptions of pro-
grams, events, or other phenomena to construct a complete
portrayal of a case for interpretation and possible action
(Lapan etal. 2012).
Eikeland (2006) describes different approaches to action
research that involve applied research, moving experimen-
tation from laboratories to field, inviting the subjects of
research to join the community of researchers and involv-
ing practitioners in research with the insistence of think-
ing through personal practices. Action research is a very
popular approach in social sciences (Stringer 2008; Clark
etal. 2020) and it is also proposed for architecture (Herr
2015) and for the practice of product design (Swann 2002).
This method is related to the terms research-through-design,
practice-based-design research or research-by-design (Red-
ström 2017; Vaughan 2017), that has been discussed to be a
kind of action research in works like (Kennedy-Clark 2013;
Motta-Filho 2021).
Case study is generally used for exploratory research
or for pre-testing some research hypotheses (Blessing and
Chakrabarti 2009). Action research requires a high degree of
flexibility and is usually qualitative, data-driven, participa-
tory, and makes use of multiple data sources. Case study and
action research also appear in the following criteria of classi-
fication, following the proposal of Blessing and Chakrabarti
(2009) referring to data-collection techniques.
2.3 Data‑collection techniques
In this subsection, we present the list of data-collection tech-
niques we have tagged, to analyse what is proposed in RQ3.
Following the list of data-collection methods presented in
section A.4 of Blessing and Chakrabarti (2009), excluding
experiments, case studies and action research we prefer to
include in the list of inquiry research strategies presented in
the previous subsection.
Observation is a technique in which the researcher
records, in real time, what is happening, either by hand,
recording it or using measuring equipment. As Blessing and
Chakrabarti (2009) explain: ‘The quality of observational
data is highly dependent on the skill, training and compe-
tency of the observer’ (Blessing and Chakrabarti 2009).
Observations are the main source of data in ethnographic
studies (see Sect.2.1), but this strategy is also commonly
used in social sciences (Creswell 2009) and in visual design
(Goodwin 2000), architecture (Cuff 1992) and product
design practice (Wasson 2000).
Simultaneous verbalization refers to the situation in
which the participants speak aloud while using a system,
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

227Research in Engineering Design (2023) 34:221–256
1 3
with the aim of providing information about the cognitive
behaviour of the participants, which may not be obtained
through normal observation (Ohnemus and Biers 1993).
Often used to analyse problem-solving behaviour, its most
important feature is the real-time aspect. Simultaneous ver-
balization sessions usually last a few hours and never more
than a day, due to the effort required by both the partici-
pants and the researchers in their corresponding analysis.
Although audio recordings are sometimes used to record
simultaneous verbalization, they are understood as inappro-
priate for a process such as design, which usually involves
drawings and gestures, so video recordings are considered
more appropriate.
Collecting technical documents consists of obtaining
technical documents related to a particular project, topic
or product, from various sources (Rapley and Rees 2018).
Analysis of these documents is often used early in a research
project to understand the organisation, the background of the
project and the experience of the designers. It is commonly
employed in most observational studies. However, if it is
used as a single source of information, it can result in such
limitations as the usual lack of data on the context in which
the documents were created and the reason for their content.
It is, therefore, convenient to complement them with other
methods such as interviews.
Collecting physical objects involves mock-ups, proto-
types and other physical models that may be relevant for
developing a product or testing it. The model or prototype
could refer to a part of the product or the whole product. For
traditional engineering research, which focuses, for example,
on the analysis of product behaviour, the products are the
main source of data (Blessing and Chakrabarti 2009). In our
review, we consider those works that start collecting differ-
ent objects to carry out a study on their usefulness, or on
the behaviour of users, for example. The object is a general
term that can refer both to drawings and physical objects.
Among the former, we find all those sketches, drawings and
diagrams that have emerged throughout the conception of a
product or its development, or throughout a research process,
which could yield important information to organise ideas
and draw conclusions.
Questionnaires are used to collect people´s thoughts or
opinions about a certain product, process or method (Rad-
hakrishna 2007). A priori, they seem easier to use than real-
time methods, such as observation or simultaneous verbali-
zation, and they are useful to obtain data from a greater
number of cases. However, some of its disadvantages, such
as the time required by the participants and the potential bias
of the results, must also be taken into account.
Interviews have the same purpose as the questionnaires
but are carried out face-to-face (King etal. 2019). Some-
times they are not carried out individually but using a group
dynamic known as focus group: a group interview that
mixes aspects of interviews and observations, as it provides
information from the study of the interactions between par-
ticipants. Focus groups can provide richer information than
interviews, but they can have a negative effect on the con-
tribution of specific participants.
2.4 Instruments forthecollection ofdata
Data collection methods are supported by instrumentation.
This section describes the categories we found to respond
to RQ4, exposing the instruments that are normally used to
collect these data. Independently of the strategy of inquiry
applied, there are several instruments that are used to keep
records of the observations. These recordings are impor-
tant to keep evidence and to enable the reproducibility of
the analysis. We tagged the papers depending on the use of
classical audio, video and image recordings and the more
recent technique of eye tracking (Bergstrom and Schall
2014).
In experiments and case studies, we are also inter-
ested in physical measurements that are used to objectify
observations.
When questionnaires and/or interviews are the data-col-
lection techniques, we tagged who is the attendee, distin-
guishing between stakeholders, users of products or par-
ticipants (observed people) in the research and experts or
designers. We also found it relevant to tag when the study
uses workshops as a means to obtain information.
The last topic of interest that has been tagged is the fact
that the research work uses simulation algorithms or tools
as a source of information. We use this tag when the simula-
tion tools are a fundamental part of the research, as it pro-
vides the information analysed in the paper (Behera etal.
2019), or because the tool or the algorithm itself is the main
contribution (Mathias etal. 2019).
3 Results
3.1 Aims andcontributions ofthereviewed papers
Table2 shows the codes assigned to each of the papers ana-
lysed. This section summarises the results related to RQ1
(research goals). As shown in Table2, most of the works
focus on methodologies or on the analysis of a specific
aspect of the design processes. The presentation of a product
and the building up of knowledge with taxonomies, guide-
lines, theories, or reviews, are exceptions.
Five papers propose a theory: (Comi etal. 2019) pre-
sent the concept of shared professional vision; (Benavides
and Lara-Rapp 2019) present the principle of weaker
dependencies in axiomatic design; (Martinec etal. 2019)
introduce the state-transition model (synthesis, analysis,
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

228 Research in Engineering Design (2023) 34:221–256
1 3
Table 2 Coding of the papers analysed, according to the categories identified under each topic of the study
Ref Aim/results Approach
To study or
propose a
methodology
To study a
given aspect
of designing
To design,
develop or
evaluate
a specific
product
Recom-
mendations/
Proposing a
guideline
Proposing a
theory
Proposing a
framework of
analysis or a
taxonomy
Analytical Quantitative Mixed Qualitative
Design stud-
ies (Bresciani
2019)
X X X X
(Comi etal.
2019)
X X X
(Cooper
2019)
X X
(Goucher-
Lambert and
Cagan 2019)
X X
(Hanrahan
etal. 2019)
X X X
(Hyysalo
etal. 2019b)
X X X
(Khalaj and
Pedgley
2019)
X X
(Lloyd 2019) X X
(Luck 2019) X X
(Mathias etal.
2019)
X X
(McDonald
and Michela
2019)
X X
(McKinnon and
Sade 2019)
X X X
(Reimlinger
etal. 2019)
X X X
(Roy and War-
ren 2019)
X X X
(Self 2019) X X
(Van der
Linden etal.
2019b)
X X
(Van Kuijk
etal. 2019)
XXX
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

229Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Aim/results Approach
To study or
propose a
methodology
To study a
given aspect
of designing
To design,
develop or
evaluate
a specific
product
Recom-
mendations/
Proposing a
guideline
Proposing a
theory
Proposing a
framework of
analysis or a
taxonomy
Analytical Quantitative Mixed Qualitative
International
Journal of
Design
(Aktas and
Mäkelä
2019)
X X X
(Barati etal.
2019)
X X
(Daalhuizen
etal. 2019)
X X
(Feijs and
Toeters
2018)
X X
(Genç etal.
2018)
X X X
(Hobye and
Ranten
2019)
XXX
(Hyysalo
etal. 2019a)
X X
(Li and Luxi-
mon 2018)
X X X
(Park-Lee
and Person
2018)
XXX
(Pedgley etal.
2018)
X X X
(Petreca etal.
2019)
X X X X
(Roesler etal.
2019)
X X
(Selvefors
etal. 2018)
X X
(Takahashi
etal. 2018)
X X
(Tsai and Van
Den Hoven
2018)
X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

230 Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Aim/results Approach
To study or
propose a
methodology
To study a
given aspect
of designing
To design,
develop or
evaluate
a specific
product
Recom-
mendations/
Proposing a
guideline
Proposing a
theory
Proposing a
framework of
analysis or a
taxonomy
Analytical Quantitative Mixed Qualitative
(Van der
Linden etal.
2019a)
X X
(Vegt etal.
2019)
XXXX
Journal of
Engineering
Design
(Abi Akle
etal. 2019)
X X
(Chen etal.
2019a)
X X
(Belkadi etal.
2019)
X X
(Benavides
and Lara-
Rapp 2019)
X X X
(Boussuge
etal. 2019)
X X X
(Chen etal.
2019b)
X X
(Cheong and
Butscher
2019)
X X X
(Graeff etal.
2019)
X X
(Hagedorn
etal. 2019)
X X X
(Morkos etal.
2019)
X X X
(Ozer and
Cebeci
2019)
X X
(Pakkanen
etal. 2019)
X X
(Saravanan
and Jerald
2019)
X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

231Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Aim/results Approach
To study or
propose a
methodology
To study a
given aspect
of designing
To design,
develop or
evaluate
a specific
product
Recom-
mendations/
Proposing a
guideline
Proposing a
theory
Proposing a
framework of
analysis or a
taxonomy
Analytical Quantitative Mixed Qualitative
(Sung etal.
2019)
X X
(Valverde
etal. 2019)
X X X
(Wand etal.
2019)
X X
(Wlazlak
etal. 2019)
X X
Research in
Engineering
Design
(Behera etal.
2019)
X X
(De Lessio
etal. 2019)
X X X
(Franceschini
and Mai-
sano 2019)
X X X
(Garcia etal.
2019)
X X
(Gyory etal.
2019)
X X
(Jagtap 2019) X X
(Martinec
etal. 2019)
X X
(Menold etal.
2019)
X X
(Piccolo etal.
2019)
X X
(Saliminamin
etal. 2019)
X X
(Santolaya
etal. 2019)
X X
(Tahera etal.
2019)
X X
(Wood and
Mattson
2019)
X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

232 Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Aim/results Approach
To study or
propose a
methodology
To study a
given aspect
of designing
To design,
develop or
evaluate
a specific
product
Recom-
mendations/
Proposing a
guideline
Proposing a
theory
Proposing a
framework of
analysis or a
taxonomy
Analytical Quantitative Mixed Qualitative
(Yang etal.
2019)
X X X
(Li etal.
2019a)
X X
(Li etal.
2019b)
X X X
(Zhang and
Thomson
2019)
X X X
25 32 6 12 5 15 10 17 12 32
Ref Research method Data-collect method
Ethnogra-
phy
Phenom-
enological
study
Herme-
neutics
Grounded
theory
Action
research
Case study Experi-
ment
Observa-
tions
Simul-
taneous
verbalisa-
tion
Collecting
technical
documents
Collecting
objects
Question-
naires
Interview-
ing
Design
studies
(Bresciani
2019)
XXX XX
(Comi
etal.
2019)
X X X X
(Cooper
2019)
X
(Goucher-
Lambert
and
Cagan
2019)
X X X
(Hanrahan
etal.
2019)
X X X X
(Hyysalo
etal.
2019b)
X X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

233Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Research method Data-collect method
Ethnogra-
phy
Phenom-
enological
study
Herme-
neutics
Grounded
theory
Action
research
Case study Experi-
ment
Observa-
tions
Simul-
taneous
verbalisa-
tion
Collecting
technical
documents
Collecting
objects
Question-
naires
Interview-
ing
(Khalaj
and
Pedgley
2019)
X X X X
(Lloyd
2019)
X X
(Luck
2019)
X X
(Mathias
etal.
2019)
X X X
(McDon-
ald and
Michela
2019)
XXX
(McKin-
non and
Sade
2019)
X X
(Reim-
linger
etal.
2019)
X X
(Roy and
Warren
2019)
X X
(Self
2019)
X X
(Van der
Linden
etal.
2019b)
X X X X
(Van Kuijk
etal.
2019)
X X X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

234 Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Research method Data-collect method
Ethnogra-
phy
Phenom-
enological
study
Herme-
neutics
Grounded
theory
Action
research
Case study Experi-
ment
Observa-
tions
Simul-
taneous
verbalisa-
tion
Collecting
technical
documents
Collecting
objects
Question-
naires
Interview-
ing
Interna-
tional
Jour-
nal of
Design
(Aktas and
Mäkelä
2019)
X X
(Barati
etal.
2019)
X X X
(Daalhui-
zen etal.
2019)
X
(Feijs and
Toeters
2018)
X X
(Genç
etal.
2018)
X X X
(Hobye
and
Ranten
2019)
X X
(Hyysalo
etal.
2019a)
X X X X X
(Li and
Luximon
2018)
X X X
(Park-Lee
and
Person
2018)
X X
(Pedgley
etal.
2018)
X X X
(Petreca
etal.
2019)
X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

235Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Research method Data-collect method
Ethnogra-
phy
Phenom-
enological
study
Herme-
neutics
Grounded
theory
Action
research
Case study Experi-
ment
Observa-
tions
Simul-
taneous
verbalisa-
tion
Collecting
technical
documents
Collecting
objects
Question-
naires
Interview-
ing
(Roesler
etal.
2019)
X X X X X
(Selvefors
etal.
2018)
X X X X
(Takahashi
etal.
2018)
X X X X
(Tsai and
Van Den
Hoven
2018)
X X X X
(Van der
Linden
etal.
2019a)
X X
(Vegt etal.
2019)
X X X X
Journal of
Engi-
neering
Design
(Abi Akle
etal.
2019)
X X
(Chen
etal.
2019a)
X
(Belkadi
etal.
2019)
X
(Benavides
and
Lara-
Rapp
2019)
X
(Boussuge
etal.
2019)
X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

236 Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Research method Data-collect method
Ethnogra-
phy
Phenom-
enological
study
Herme-
neutics
Grounded
theory
Action
research
Case study Experi-
ment
Observa-
tions
Simul-
taneous
verbalisa-
tion
Collecting
technical
documents
Collecting
objects
Question-
naires
Interview-
ing
(Chen
etal.
2019b)
X X
(Cheong
and
Butscher
2019)
X X
(Graeff
etal.
2019)
X X X X
(Hagedorn
etal.
2019)
X
(Morkos
etal.
2019)
X X
(Ozer and
Cebeci
2019)
X X
(Pakkanen
etal.
2019)
X X X X X X
(Saravanan
and
Jerald
2019)
X
(Sung
etal.
2019)
X X X
(Valverde
etal.
2019)
X X X
(Wand
etal.
2019)
X
(Wlazlak
etal.
2019)
X X X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

237Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Research method Data-collect method
Ethnogra-
phy
Phenom-
enological
study
Herme-
neutics
Grounded
theory
Action
research
Case study Experi-
ment
Observa-
tions
Simul-
taneous
verbalisa-
tion
Collecting
technical
documents
Collecting
objects
Question-
naires
Interview-
ing
Research
in Engi-
neering
Design
(Behera
etal.
2019)
X X
(De Lessio
etal.
2019)
XXXXX
(Frances-
chini and
Maisano
2019)
X X X
(Garcia
etal.
2019)
X X
(Gyory
etal.
2019)
XXX
(Jagtap
2019)
X
(Martinec
etal.
2019)
X X X X
(Menold
etal.
2019)
X X
(Piccolo
etal.
2019)
X X
(Salimi-
namin
etal.
2019)
X X
(Santolaya
etal.
2019)
X X X
(Tahera
etal.
2019)
X X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

238 Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Research method Data-collect method
Ethnogra-
phy
Phenom-
enological
study
Herme-
neutics
Grounded
theory
Action
research
Case study Experi-
ment
Observa-
tions
Simul-
taneous
verbalisa-
tion
Collecting
technical
documents
Collecting
objects
Question-
naires
Interview-
ing
(Wood and
Mattson
2019)
X X X X
(Yang
etal.
2019)
X X
(Li etal.
2019a)
X X
(Li etal.
2019b)
X X
(Zhang
and
Thomson
2019)
X X
4 3 4 1 7 37 24 19 3 26 17 15 22
Ref Instrument Human input
Measure-
ments
Audio record-
ings
Video record-
ings
Photographs Eye tracking Simulation/
software
Stakeholder
opinions
User/partici-
pant opinions
Expert/
designer
opinions
Workshops
Design stud-
ies
(Bresciani
2019)
X
(Comi etal.
2019)
X X X X
(Cooper
2019)
(Goucher-
Lambert
and Cagan
2019)
X
(Hanrahan
etal. 2019)
X
(Hyysalo
etal. 2019b)
X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

239Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Instrument Human input
Measure-
ments
Audio record-
ings
Video record-
ings
Photographs Eye tracking Simulation/
software
Stakeholder
opinions
User/partici-
pant opinions
Expert/
designer
opinions
Workshops
(Khalaj and
Pedgley
2019)
X X X
(Lloyd 2019)
(Luck 2019)
(Mathias etal.
2019)
X X
(McDonald
and Michela
2019)
X X X
(McKinnon
and Sade
2019)
X X X
(Reimlinger
etal. 2019)
X X X X
(Roy and
Warren
2019)
(Self 2019) X X X X
(Van der
Linden etal.
2019b)
X X X
(Van Kuijk
etal. 2019)
X X
International
Journal of
Design
(Aktas and
Mäkelä
2019)
XXX
(Barati etal.
2019)
X X X
(Daalhuizen
etal. 2019)
(Feijs and
Toeters
2018)
X
(Genç etal.
2018)
X X X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

240 Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Instrument Human input
Measure-
ments
Audio record-
ings
Video record-
ings
Photographs Eye tracking Simulation/
software
Stakeholder
opinions
User/partici-
pant opinions
Expert/
designer
opinions
Workshops
(Hobye and
Ranten
2019)
X
(Hyysalo
etal. 2019a)
X X X
(Li and Luxi-
mon 2018)
X
(Park-Lee
and Person
2018)
X X
(Pedgley etal.
2018)
(Petreca etal.
2019)
(Roesler etal.
2019)
X
(Selvefors
etal. 2018)
X X
(Takahashi
etal. 2018)
X X X X X
(Tsai and Van
Den Hoven
2018)
X X X
(Van der
Linden etal.
2019a)
X X X
(Vegt etal.
2019)
X X
Journal of
Engineering
Design
(Abi Akle
etal. 2019)
X X
(Chen etal.
2019a)
X
(Belkadi etal.
2019)
X
(Benavides
and Lara-
Rapp 2019)
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

241Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Instrument Human input
Measure-
ments
Audio record-
ings
Video record-
ings
Photographs Eye tracking Simulation/
software
Stakeholder
opinions
User/partici-
pant opinions
Expert/
designer
opinions
Workshops
(Boussuge
etal. 2019)
X
(Chen etal.
2019b)
(Cheong and
Butscher
2019)
X
(Graeff etal.
2019)
X
(Hagedorn
etal. 2019)
X X X
(Morkos etal.
2019)
X
(Ozer and
Cebeci
2019)
X X X
(Pakkanen
etal. 2019)
X X
(Saravanan
and Jerald
2019)
X
(Sung etal.
2019)
X X X
(Valverde
etal. 2019)
X
(Wand etal.
2019)
X
(Wlazlak
etal. 2019)
X X
Research in
Engineering
Design
(Behera etal.
2019)
X
(De Lessio
etal. 2019)
X X
(Franceschini
and Mai-
sano 2019)
X
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

242 Research in Engineering Design (2023) 34:221–256
1 3
Table 2 (continued)
Ref Instrument Human input
Measure-
ments
Audio record-
ings
Video record-
ings
Photographs Eye tracking Simulation/
software
Stakeholder
opinions
User/partici-
pant opinions
Expert/
designer
opinions
Workshops
(Garcia etal.
2019)
X X
(Gyory etal.
2019)
X X
(Jagtap 2019)
(Martinec
etal. 2019)
X X
(Menold etal.
2019)
X X
(Piccolo etal.
2019)
X
(Saliminamin
etal. 2019)
X X X
(Santolaya
etal. 2019)
X
(Tahera etal.
2019)
X X
(Wood and
Mattson
2019)
X X X X
(Yang etal.
2019)
X X
(Li etal.
2019a)
(Li etal.
2019b)
X X
(Zhang and
Thomson
2019)
X X
10 14 11 6 3 20 2 22 20 11
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

243Research in Engineering Design (2023) 34:221–256
1 3
evaluation) in conceptual design and Lloyd (2019) defends
the theory of the social turn in design, Aktas and Mäkelä
(2019) focus on the relation between craft, materials,
makers.
Six works focus on the evaluation of a specific product:
a software product in Takahashi etal. (2018) and Belkadi
etal. (2019); or physical objects in the case of Roesler
etal. (2019), Hyysalo etal. (2019b) and McKinnon and
Sade (2019).
Concerning the works related to methodologies, we find
papers that propose a method based on analytical methods
or algorithmic solutions such as those related to axiomatic
design (Chen etal. 2019a) and those related to such meth-
ods as research-through-design, where the importance of
the method followed is prominent in the study (Tsai and Van
Den Hoven 2018; Hyysalo etal. 2019b; McKinnon and Sade
2019; Hanrahan etal. 2019); or methodologies for product
development such as Daalhuizen etal. (2019), with emphasis
on different aspects such as work in groups (Gyory etal.
2019), sustainability (Santolaya etal. 2019) or democratised
design (Hyysalo etal. 2019a).
A good number of papers present frameworks of analysis
or classifications with different purposes. Bresciani (2019)
for classifying visualization dimensions, McDonald and
Michela (2019) to classify moral goods, Roy and Warren
(2019) for card sets, Park-Lee and Person (2018) identify
three practices on briefing, Vegt etal. (2019) deduce 3 types
of invasiveness evoked by the rules in gamified brainstorm-
ing, Valverde etal. (2019) classify the type of feedback in
automotive push buttons, Cooper (2019) presents the five
waves in design research, Luck (2019) describes the frame-
work to distinguish between design, design research, archi-
tectural design research and practice, Hobye and Ranten
(2019) present five behavioural strategies for interactive
products and Van Kuijk etal. (2019) presents a framework to
analyse usability concepts of electronic products and Petreca
etal. (2019) for analysing the relation between sensors and
textile. We also include in this category the papers related to
ontologies, that are used to represent knowledge.
Proposing recommendations is a common result in the
analysed research papers, including a variety of themes such
as recommendations on the use of guidelines by new design-
ers (Reimlinger etal. 2019); the use of specific materials
(Genç etal. 2018; Pedgley etal. 2018; Aktas and Mäkelä
2019; Petreca etal. 2019); how to orient future studies on the
use of mobile technology by elderly people (Li and Luximon
2018), or about design and poverty (Jagtap 2019) or ethno-
graphic studies in developing countries (Wood and Matt-
son 2019); appliance design (Selvefors etal. 2018); use of
games in brainstorming (Vegt etal. 2019); or specifying
requirements (Morkos etal. 2019). Cooper (2019) proposes
interprets the history of design research through five waves.
The most frequent type of works delve into a particu-
lar aspect of product design such as sketching (Sung etal.
2019; Self 2019), prototyping (Menold etal. 2019; Mathias
etal. 2019), material (Pedgley etal. 2018; Aktas and Mäkelä
2019; Barati etal. 2019; Petreca etal. 2019), interaction
(Hobye and Ranten 2019; Valverde etal. 2019), briefing
(Park-Lee and Person 2018), working in groups (Graeff etal.
2019), iterations and testing (Tahera etal. 2019; Piccolo
etal. 2019); behavioural complexity (Hobye and Ranten
2019), manufacturing (Yang etal. 2019), or usability (Van
Kuijk etal. 2019).
3.2 Strategies ofinquiry andmethodologies
This section summarises the results related to RQ2 (main
experimental approaches founded): qualitative approaches
are a majority, but the number of quantitative or mixed-
methods studies is also relevant. Other approaches, such
as the use of analytical methods, are less frequent. Table3
shows that, when the goal of the paper is related to propos-
ing or studying a methodology (first column in Table3),
the percentage of pure quantitative papers is lower than in
the rest of the cases. Regarding whether there is a tendency
towards any methodology depending on the journal; Table2
shows that the Journal on Engineering Design seems to
focus more than the other journals on non-qualitative strat-
egies of inquiry.
When quantitative methods are used, experiments are
more frequent than quasi-experiments and non-experiments
(14 out of the 17 quantitative studies present an experiment).
We found 26 experimental studies, with 5 quasi-experiments
(Saliminamin etal. 2019; Vegt etal. 2019; Sung etal. 2019;
Self 2019; Santolaya etal. 2019) and 4 non-experiments
(Selvefors etal. 2018; Morkos etal. 2019; Roesler etal.
2019; Piccolo etal. 2019).
The use of case studies is pervasive in qualitative research
(more than half the studies that classified as qualitative base
the research on a case study). Furthermore, many quantita-
tive studies support results from case studies; for example,
some analytical studies in which case studies are used as
proof of concept of the proposed models (Chen etal. 2019b;
Zhang and Thomson 2019; Li etal. 2019a).
Nevertheless, other qualitative methods, such as ethnog-
raphy, hermeneutics, action research and phenomenological
studies, are also used. The use of specific methods related to
design is scarce (the discussion about this concern is dealt
with in detail below). Ethnography is used in three cases
(Roesler etal. 2019; Van der Linden etal. 2019a; Comi etal.
2019)—also the annotation as observation in the tables—and
one more paper uses ethnography as the study focus (Wood
and Mattson 2019). Hermeneutics is used by (McDonald
and Michela 2019; Cooper 2019; Lloyd 2019; Luck 2019).
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

244 Research in Engineering Design (2023) 34:221–256
1 3
Table 3 Number of works in each (sub)category per research approach
Aim/results Research method Data-collect method
To study
or pro-
pose a
method-
ology
To study
a given
aspect of
design-
ing
To
design,
develop
or evalu-
ate a
specific
product
Recom-
menda-
tions/
Proposing
a guide-
line
Propos-
ing a
theory
Propos-
ing a
frame-
work of
analysis
or a tax-
onomy
Ethnog-
raphy
Phe-
nom-
eno-
logical
study
Herme-
neutics
Grounded
theory
Action
research
Case
study
Experi-
ment
Obser-
vations
Simul-
taneous
verbali-
sation
Collect-
ing tech-
nical
docu-
ments
Col-
lecting
objects
Ques-
tion-
naires
Inter-
viewing
Analyti-
cal
7 1 0 1 1 4 0 0 0 0 0 9 2 0 0 5 0 1 0
Quanti-
tative
6 10 0 2 1 2 0 0 0 0 0 7 14 2 2 9 7 5 1
Mixed 4 5 3 4 0 0 1 2 0 0 1 5 10 2 1 1 1 6 5
Qualita-
tive
11 16 3 5 3 9 3 1 4 1 6 18 0 15 0 13 9 4 16
Instrument Human input
Measurements Audio record-
ings
Video record-
ings
Photographs Eye tracking Simulation/
software
Stakeholder
opinions
User/participant
opinions
Expert/designer
opinions
Workshops
Analytical 2 0 0 0 0 7 0 2 1 0
Quantitative 7 3 3 1 0 5 0 5 5 1
Mixed 3 1 3 0 3 4 2 7 4 3
Qualitative 0 10 5 5 0 6 0 9 10 7
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

245Research in Engineering Design (2023) 34:221–256
1 3
Action research is used by Pakkanen etal. (2019) to
investigate, in combination with case studies, modular
systems in industrial environments. The work of Bresciani
(2019) could be considered an action research study with
the goal of building a grounded theory evaluation technique
for visual thinking. McKinnon and Sade (2019) align their
work in the field of research through design using a set of
gadgets to obtain information about environmental home
good practices. Research through design is also used by
Genç etal. (2018) to explore new materials and Tsai and
Van Den Hoven (2018) to explore user experience. Hyysalo
etal. (2019b) and present the evaluation of a panel follow-
ing the principles of research through design. Close to this
method is that presented by Barati etal.(2019), who comple-
ment their study with workshops where a group of students
explores their proposals.
3.3 Data‑collection methods
Results regarding RQ3 (data collection methods) are sum-
marised in this section. Table3 shows which main methods
and techniques for collecting data are used in the different
studies. The analysis of the sources of information is com-
pleted with a revision of the instruments used to collect data
and with a discussion about the role of human input pre-
sented in the following sections. None of the data-collec-
tion methods identified seem to be dominant in the papers
studied.
Technical documents of diverse nature are the main
source of information used (Table2 reports 23 out of the
68 papers analysed using technical documents). Interview-
ing is also frequent (22 times reported in Table2). Expert
and user opinions are both used as sources of information,
but neither is a majority (22 and 20 papers, respectively,
reported in Table2. Observation is mostly used in qualita-
tive studies, where almost half use this technique. Concern-
ing quantitative studies, apart from measurements, expert
opinions appear as a frequent resource. This is because it
is common to collect the opinions of experts in question-
naires or in evaluation templates that convert opinions into
numeric values.
Verbalization is used in Martinec etal. (2019) and Gyory
etal. (2019) for team work analysis and in (Khalaj and
Pedgley 2019), where designers and users had to verbalize
impressions.
Objects are collected as a data source in a relevant num-
ber of studies. Some are the results of students’ work as in
Gralla etal. (2019); brainstorming outputs (Vegt etal. 2019);
prototypes (Feijs and Toeters 2018; Barati etal. 2019), or
commercial products (Roy and Warren 2019). Sketches
are the type of object analysed in (Genç etal. 2018; Mar-
tinec etal. 2019; Gyory etal. 2019; Goucher-Lambert and
Cagan 2019; Comi etal. 2019); while for (Li and Luximon
2018; Sung etal. 2019) sketches are the main concern of
the research.
Questionnaires are less frequently used, and when this
happens, they are designed ad-hoc for each study. Given
the wide variety of topics and aims of the reviewed works,
no standardised questionnaires have been found. Question-
naires, therefore, take different formats: Amazon Mechanical
Turk is used once (Goucher-Lambert and Cagan 2019); a
Likert scale tool evaluation (Graeff etal. 2019); binary and
open questions (Pakkanen etal. 2019); ranking of prefer-
ences (Franceschini and Maisano 2019); or ad-hoc software
tools (Li etal. 2019a).
Interviews are frequently used as a source of information
in qualitative and mixed strategies of inquiry. Interviews are
associated with phenomenological studies (Li and Luximon
2018; Park-Lee and Person 2018; Selvefors etal. 2018) and
also in ethnographic studies (Roesler etal. 2019; Van der
Linden etal. 2019a; Wood and Mattson 2019; Comi etal.
2019). The interviewed population can be a group of users
of a given technology (Li and Luximon 2018) or a group of
experts (Bresciani 2019).
Concerning the sample size used in the 24 papers whose
research method has been classified as experimental, and
taking into account that the sample may refer to studied
objects or to participants/users, which, in turn, may be indi-
viduals or teams, the number of participants/users varies
between 4, in Martinec etal. (2019), and 169, in Ozer and
Cebeci (2019). The number of studied objects also varies
from 6, in Mathias etal. (2019) to 256, in Li etal. (2019b).
In Santolaya etal. (2019) a methodology is experimentally
tested in 2 case studies.
3.4 Instruments
Results regarding RQ4 (instruments used to collect data) are
summarised in this section. Measurements refer both to met-
rics obtained with a physical device and to qualitative ratings
obtained from human-based scores. In the first group, we
can mention the metrics of energetic consumption (Selvefors
etal. 2018; Santolaya etal. 2019), mass material (Santolaya
etal. 2019), volumes of objects (Mathias etal. 2019), dis-
placement of buttons (Valverde etal. 2019), online shopping
user interaction data (Ozer and Cebeci 2019), or the timing
of tasks in (Mathias etal. 2019). In the second group, we
can cite (Saliminamin etal. 2019; Gyory etal. 2019), which
score the quality of design proposals, and (Franceschini and
Maisano 2019), who use design preferences as the input for
an analytical model.
Simulations and/or software developments of algorithms
take on an important role in several papers. Belkadi etal.
(2019) present a software tool; Chen etal. (2019a), Feijs and
Toeters (2018), Mathias etal. (2019) and Takahashi etal.
(2018) present or test software tools for different goals, such
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

246 Research in Engineering Design (2023) 34:221–256
1 3
as analysing Lego buildings, and generating fashion patterns
for projecting requirements into design parameters. Li etal.
(2019a) focus on modelling knowledge; Piccolo etal. (2019)
use analysis and visualization tools to present results; while
Ozer and Cebeci (2019) and Saravanan and Jerald (2019) use
machine learning techniques such as neural networks and
clustering. De Lessio etal. (2019) present a software tool to
support planning and Yang etal. (2019) to support manu-
facturing. Boussuge etal. (2019) propose using ontologies
to capture high-level modelling and idealisation decisions,
characterising the simulations of CAE models from CAD
assemblies. Other papers related to ontologies use software
to model them (Cheong and Butscher 2019; Hagedorn etal.
2019; Wang etal. 2019).
Workshops are frequently used for evaluating results and
sharing experiences by a group of experts with discussions
(Van der Linden etal. 2019a, b; McKinnon and Sade 2019;
Self 2019; Wlazlak etal. 2019). In (Genç etal. 2018; Marti-
nec etal. 2019), the workshops become designing activities
in the research-through-design methodology. In Takahashi
etal. (2018), workshops are used to observe users while
they interact with a system and, in Pakkanen etal. (2019),
to collect information from experts. In Garcia etal. (2019),
workshops are meetings with stakeholders.
The opinions of stakeholders can be the core of the
research study (Self 2019) or they can be used as part of
usability tests (Takahashi etal. 2018). Most often, question-
naires and interviews are performed with users of a product
(Selvefors etal. 2018; Roesler etal. 2019; Hanrahan etal.
2019; Ozer and Cebeci 2019); by active participants of the
process under analysis, such as professionals in companies
(Reimlinger etal. 2019; Wlazlak etal. 2019); or by students
that are required to do a project (Vegt etal. 2019; Li etal.
2019a; Abi Akle etal. 2019; Graeff etal. 2019). The experts
that participate in questionnaires or interviews are designers,
architects, engineers (Li and Luximon 2018; Park-Lee and
Person 2018; Pakkanen etal. 2019), or academic staff evalu-
ating results (Morkos etal. 2019; Sung etal. 2019; McKin-
non and Sade 2019). In interviews occurring in ethnographic
studies, the subjects providing information could be con-
sidered the topic of analysis (Wood and Mattson 2019), but
at the same time, they could be experts (Comi etal. 2019).
4 Discussion
4.1 Variety ofaims andapproaches
The principal finding of our research is that there is a very
high diversity in the works we have analysed in the journals
related to engineering design. This variety affects the aims
and scopes of the research works, the methods, and the data
sources. Table4 shows that variety affects the papers in the
four journals analysed with only minor differences among
them. Thus, DS (Design Studies) and RED (Research in
Engineering Design) seem to focus more on methodologi-
cal aspects, while IJD (International Journal of Design) and
JED (Journal of Engineering Design) focus more on delving
into particular aspects of the design process or on products,
but at most 7 papers out of the 17 falls into one of the cat-
egories. According to the results, DS and IJD journals attract
more papers with a qualitative approach (only 2 papers in
each journal are purely quantitative), while most of the
papers from JED and RED follow a quantitative or analyti-
cal approach (only 3 and 7 papers, respectively, are purely
qualitative). However, we have found papers with both
approaches in all the journals. RED uses less self-reported
data (interviews, questionnaires or workshops), while DS
uses this source of data the most, but in both journals there
are exceptions, such as the works of Mathias etal. (2019) in
DS or Garcia etal. (2019) in RED.
Despite this broad spectrum of papers, we found a clear
interest in methodologies and the in-depth analysis of a
given aspect of the whole process of designing generally
applied to a particular case study. The interest in both topics
is justified by the nature of the design and the youth of the
discipline. As a process of searching for optimum solutions,
design is clearly related to methodological concerns. As a
young discipline, the space for contributing to the different
tasks of the whole design process is huge. The analysis of
the process of engineering design has evolved from being
considered from a purely technical perspective to being stud-
ied as a socio-technical process. From a technical point of
view, (Beitz etal. 1996) distinguished between conceptual
design and embodied design for identifying a list of tasks
that contribute to facing problems of engineering design in
an effective and systematic way. From a socio-technical per-
spective, different authors have pointed out that the design
process is influenced by aspects related to teamwork capa-
bilities (Dorst 2004), the inclusion of participants (Van der
Bijl-Brouwer and Dorst 2017) or by the institutional com-
plexity (Reich and Subrahmanian 2020). Our study shows
that there is space for research works that focus on both
perspectives of analysis, being found works that are closely
related to tasks that affect conceptual design (Martinec etal.
2019; Benavides and Lara-Rapp 2019; Self 2019), embodied
design (Petreca etal. 2019) and also to social aspects of the
design process (Piccolo etal. 2019).
It has been observed that there are a relatively low num-
ber of papers proposing recommendations, guidelines,
frameworks, and taxonomies. We understand how difficult
it is generalizing and classifying a discipline with multiple
tasks, agents, approaches and sub-domains. Nevertheless,
generating these types of representations of knowledge could
be a substrate for the growth of the discipline. Design is a
context-specific endeavour, but trying to generalize results
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

247Research in Engineering Design (2023) 34:221–256
1 3
Table 4 Number of works in the different journals
Journal Aim/results Approach Research method
To study
or pro-
pose a
method-
ology
To study
a given
aspect of
design-
ing
To
design,
develop
or evalu-
ate a
specific
product
Recom-
menda-
tions/
Propos-
ing a
guide-
line
Propos-
ing a
theory
Propos-
ing a
frame-
work of
analysis
or a tax-
onomy
Analyti-
cal
Quanti-
tative
Mixed Qualitative Eth-
nogra-
phy
Phenom-
enologi-
cal study
Herme-
neutics
Grounded
theory
Action
research
Case
study
Experi-
ment
Design
Studies
7 8 2 1 2 6 0 2 4 11 2 0 4 1 3 5 5
Interna-
tional
Journal
of Design
4 10 2 6 1 4 0 2 4 11 1 3 0 0 3 9 5
Journal of
Engi-
neering
Design
6 8 1 2 1 5 7 5 2 3 0 0 0 0 1 13 6
Research
in Engi-
neering
Design
8 6 1 3 1 0 3 8 2 7 1 0 0 0 0 10 8
Total 25 32 6 12 5 15 10 17 12 32 4 3 4 1 7 37 24
Journal Data-collect method Instrument Human input
Observa-
tions
Simul-
taneous
verbali-
sation
Collecting
technical
docu-
ments
Col-
lecting
objects
Ques-
tion-
naires
Inter-
viewing
Measure-
ments
Audio
record-
ings
Video
record-
ings
Photo-
graphs
Eye
track-
ing
Simula-
tion/
software
Stakeholder
opinions
User/
par-
ticipant
opinions
Expert/
designer
opinions
Work-
shops
Design
Studies
4 1 5 7 5 8 1 5 4 2 2 1 1 3 10 4
Interna-
tional
Journal
of Design
10 0 1 6 3 9 1 5 4 3 0 4 1 7 5 3
Journal of
Engi-
neering
Design
2 0 9 1 4 2 2 1 1 0 1 8 0 5 4 2
Research in
Engi-
neering
Design
3 2 11 3 3 3 6 3 2 1 0 7 0 7 1 2
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

248 Research in Engineering Design (2023) 34:221–256
1 3
so that other authors could reuse the generated knowledge
in other domains would be positive for the growth of the
discipline. The selected papers include product development
and engineering design, which are two different areas, albeit
overlapping. Recommendations and guidelines are always
useful for the practice of engineering design, but more
importantly, classifying concepts and types of activities
with frameworks and taxonomies is an essential process in
the building of knowledge in any research area. The variety
of aims and approaches is probably the reason for this defi-
cit, but research in engineering design would benefit from
works analysing the many methodologies proposed from a
meta level that permits obtaining general concepts that are
domain-independent and universally applicable.
Results presented in Table2 and summarised in Table3
could be used to derive patterns or preferred styles in
research design.Papers using analytical approaches mainly
use case studies to validate the proposed models and they
use simulations to compare results with expectations. Here,
the case studies are used as proof of concept of the proposed
models. They do not consider human input as a main feature
of analysis. The ones related to methodological concerns are
the papers focusing on axiomatic design and the ones relat-
ing to specific aspects or to frameworks are the ones related
to ontologies. Most papers with quantitative approaches
use experimental setups in which they compare different
configurations of a given problem. The means to collect
numerical data highly depend on the type of work, with no
outstanding method or instrument. This approach is mainly
used when the goal is to study a given aspect of design,
which is coherent with the fact that experiments are meant
to measure variables that can be isolated, and therefore these
studies need to focus on specific features of the design pro-
cess. Like analytical papers, qualitative approaches are
mainly based on case studies. The main difference is related
to the nature of these case studies. In qualitative approaches,
the case studies aim at gaining insight into the complexity
of the studied design processes from the point of view of
the participants. In consequence, the preferred data collec-
tion methods are observations and interviews and/or work-
shops, to collect data from users and experts. They use rich
data sources (audio, photography, video or software tools)
to make observations rigorously. Qualitative approaches
are the most used methods, independently of the aim of the
paper, but they are dominant for proposing frameworks of
analysis or deriving guidelines and recommendations, prob-
ably because the active interpretation of experts is a must
for these concerns. Papers using mixed methods triangulate
the information obtained in quantitative experiments with
information obtained with qualitative methods. Therefore,
their pattern is closer to one of the papers using quantitative
methods than to the ones using qualitative methods.
Table 4 (continued)
Journal Data-collect method Instrument Human input
Observa-
tions
Simul-
taneous
verbali-
sation
Collecting
technical
docu-
ments
Col-
lecting
objects
Ques-
tion-
naires
Inter-
viewing
Measure-
ments
Audio
record-
ings
Video
record-
ings
Photo-
graphs
Eye
track-
ing
Simula-
tion/
software
Stakeholder
opinions
User/
par-
ticipant
opinions
Expert/
designer
opinions
Work-
shops
Total 19 3 26 17 15 22 10 14 11 6 3 20 2 22 20 11
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

249Research in Engineering Design (2023) 34:221–256
1 3
The application of one approach or another should
respond to what Subrahmanian etal. (2020) call the dif-
ferent models of designing. When the artifact or the pro-
cess is clear, analytical, and quantitative methods, closer to
approaches followed in natural science can be applied. When
people, culture, society, and politics must be taken into con-
sideration, the use of analytical and quantitative methods is
not appropriate. When individual designers play a role, and,
especially, when social aspects and context must be taken
into consideration, design processes become more complex
and dynamic, involving aspects that are better studied by
qualitative approaches that are able to capture the complex-
ity of the object of study and the participants' perspectives.
4.2 Implications fortheresearch intheengineering
design community
As mentioned in the introduction, one of the objectives
of this paper was to provide suggestions about the course
contents that doctoral studies in the domain of engineering
design must carry out. The first implication of our analysis
relates to the type of research methodologies that students
must be introduced to. According to the analysed papers,
it seems essential that future researchers receive training
in both qualitative and quantitative methods. The analysis
shows that qualitative research is very common and that rich
sources of data, such as observations or users and experts
opinions collected through interviews are frequent. Further-
more, pure qualitative research approaches, like ethnogra-
phy and phenomenology are commonly found. Neverthe-
less, experimental approaches should also have a relevant
role in the student curricula because it is frequently used as
well. We understand that this qualitative-quantitative duality
responds to the nature of engineering design, a complex field
that requires both technical background and the considera-
tion of behavioural and social aspects related to design.
A second implication has to do with the instruments
and data collection methods that researchers on engineer-
ing design must get familiar with. Research studies in this
domain could require accessing real design scenarios that
are authentic field studies rather than controlled lab studies.
This is a relevant divergence with respect to other research
domains that permit isolating variables and participants.
There are implications for the instruments used for collect-
ing data, with the need of considering techniques that per-
mit collecting information in real settings and during longer
periods of time. but also, that human fact is a relevant vari-
able that affects both design teams managements, commu-
nication with users and social aspects. This fact justifies the
use of technical reports, questionnaires, and observation as
the main sources of information in these studies.
It must be noted that publishing in a journal should not be
an end in itself, and the real value of a paper does not rely
on the journal in which it is published but on its contribu-
tion to the growth of the discipline (Bladek 2014). However,
there is a universal tendency to identify research quality and
impact with these publications, and students that pursue a
research career usually need to accomplish certain goals
related to publishing. For this reason, we think that doctoral
students in engineering design can find this work useful, as
it provides an overview and pointers to different types of
research work published in four top-quality journals in the
field, and this may give them tips on the kind of knowledge
they need to acquire to have their work published in these
journals or similar ones.
4.3 Relation toother surveys
Probably due to the youth of engineering design as a research
discipline, the number of papers devoted to literature reviews
in these fields is still sparse. From the few reviews found,
most refer to particular aspects of engineering design: such
as inspiration and fixation (Crilly 2019); sustainability
(Coskun etal. 2015); user value (Boztepe 2007); Alzheimer
and play experience (Anderiesen etal. 2015); performance
in industrial design(Candi and Gemser 2010); relation
between creativity, functionality, and aesthetics (Han etal.
2021); fuzzy front-ends for product development (Park etal.
2021); surrogate models and computational complexity (Ali-
zadeh etal. 2020); smart design (Pessôa and Becker 2020);
design and poverty (Jagtap 2019); mass customization (Fer-
guson etal. 2014); product stigma (Schröppel etal. 2021);
uncertainty (Han etal. 2020); decision-making methods
(Renzi etal. 2017); modular product design (Bonvoisin etal.
2016); or product-service systems (Vasantha etal. 2012).
More interesting, for their similarity with respect to the
present study, are the works presented by Tempczyk (1986)
and Cantamessa (2003), both presenting reviews or surveys
about research and studies on engineering design. These
two works and the one presented in this paper differ in their
sources of information. Tempczyk (1986) made a survey
by sending questionnaires to academic staff concerning
research subjects and methods; Cantamessa (2003) made
a review of the proceedings of two editions of the confer-
ence on engineering design. There is a temporal distance
of 17years between the work of Tempczyk (1986) and the
one of Cantamessa (2003) and 18years between the work
of Cantamessa (2003) and the present study, but we must
highlight the fact that the three studies report methodologies
as one of the main topics of research. Computer-aided prod-
ucts are reported by Tempczyk (1986) as a relevant topic,
and Cantamessa (2003) also refers to software tools as a
recurrent topic, while we also identified a category named
simulation which included software tools and algorithms.
The three works also report a high variety of approaches and
themes. The main difference between these studies and the
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

250 Research in Engineering Design (2023) 34:221–256
1 3
present one is that Tempczyk (1986) reports on training as
an important concern for researchers and Cantamessa (2003)
observes different streams of research, loosely coupled with
an excess of referencing to previous works. As regards refer-
ences to training concerns, we did not find any paper related
to training, probably because, nowadays, there are journals
specifically devoted to learning in the domain of engineering
and design. As regards the criticism of Cantamessa (2003)
concerning the notable amount of self-references in the ana-
lysed papers, we did not observe such a circumstance in
the journal papers we have reviewed. On the contrary, our
review has found that the papers reviewed contain complete
state-of-the-art sections in which other research groups are
referenced and other studies are discussed. This finding
partially contradicts what Cantamessa (2003) found in his
review. We think that the nature of the sources of data in his
review, based on proceedings which are shorter could have
influenced these divergent results. Our study may point to a
more mature stage of research that builds on the knowledge
already offered in the community. This finding may be based
on the fact we are working on journal papers that offer more
mature results.
4.4 Limitations
The systematic literature review presented in this paper cov-
ers a recent period of time spanning one year of publications.
The sample is representative of recent research in engineer-
ing design, but it does not provide information about tenden-
cies in the field. For example, we have observed a relevant
number of quantitative studies in comparison to qualitative
ones, but we cannot say if this is a tendency. Future work
would be required to compare our results with those of a
longitudinal study covering a larger period of years. We
expect that our work can be considered as the first step in
this longer-term study that could provide useful information
about the evolution of research into the young discipline of
engineering design.
By selecting Blessing and Chakrabarti (2009) as a frame-
work to categorize research papers, we did not pay attention
to the important concern of the success of the research which
could be a critical point for connecting the study aim, with
the approach, research method, etc. Reich and Subrahmanian
(2021) show that it is possible to use the PSI framework
(Problem, Social and Institutional space) to describe what
researchers and designers did in case studies to analyse the
matching of methods, aims and approaches with the success
of the projects. In spite of our work being merely descriptive
of the aims, methods and techniques used by authors, we
offer a corpus of categorised research papers for analysing
in future works on whether the research design is appropri-
ate for its goals.
The analysis of the sample of journal papers selected has
permitted us to build a consistent set of categories for clas-
sifying research works in engineering design. We consider
this sample comprehensive, based on a saturation analysis
carried out on the sample, that showed that all the catego-
ries used in the analysis could be identified with 69% of the
papers that were actually used in the analysis. Nevertheless,
while selecting 68 papers from only four journals, we could
have discarded other works that could include other alterna-
tive approaches also valid for research in engineering design.
Moreover, the choice of a single year-window is another
limitation of this study, as it does not enable us to provide
a full vision of the field and its evolution. Nevertheless, we
think that the classification presented in this paper could be
the basis for subsequent studies, which should consider a
broader timeframe, and therefore, a larger selection of papers
across several years. Other approaches for selecting the ana-
lysed papers like sampling at the same rate in all the journals
could also have led to representative results.
5 Conclusions
In this paper, we have presented a systematic review of
recent literature on research methods and instruments used
in a one-year period of research papers in the field of engi-
neering design. By taking this approach, we offer a "fixed
image" of recent research in the area and point to some gaps
and challenges in the field.
The review shows that there is no single methodological
approach accepted as the standard in the field; and that there
is a large variety of goals, approaches, data collection meth-
ods and instruments to collect them. In spite of this variety,
we have observed a certain preference towards qualitative
methods, which can be justified by the increasing considera-
tion of engineering design as a complex process affecting
humans and their contexts.
We think that this paper contributes to research in engi-
neering design by providing initial evidence for researchers
about the kind of work that are expected by high-impact sci-
entific journals in this domain. Additionally, academics can
find in this paper a list of topics (methodologies, data-col-
lection procedures, instruments, etc.…) that must be part of
the programme of courses on research in engineering design.
6 Appendix: Coding scheme: categories
andexamples
The tables included in this Appendix have aim to present the
knowledge generated in this paper in the form of a coding
scheme, that can be used as an instrument to describe the
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

251Research in Engineering Design (2023) 34:221–256
1 3
Table 5 Categories and examples of works corresponding to the topic of aim of the research
Topic: Aim of the research
Category Example
Proposing a methodology and analysing its validity Khalaj and Pedgley (2019) propose a methodology for identifying dis-
continuities between intended and realized semantics when comparing
users’ product impressions vs. designers’ product expressions
To study a given aspect of designing or a challenging area of design Sung etal. (2019) study a given aspect such as the influence of sketch-
ing instruction on students’ design cognition within elementary sci-
ence classrooms
To design or develop a specific product by describing the process of
creation or development of a specific product
Roesler etal. (2019) present the design process of the Anesthesia Medi-
cation Template which aims at improving medication handling safety
Proposing a guideline by providing advice, either at a methodological
level or in the design of new products
Selvefors etal. (2018) formulate guidelines that can aid appliance
designers in designing for less energy-intensive use
Proposing a theory by using logical reasoning with the aim of enunci-
ating concepts
Comi etal. (2019) propose a theory about how architects and engineers
mobilize visual objects to coordinate their professional visions around
a design issue
Proposing a framework of analysis that enables concepts or objects to
be classified into categories
Bresciani etal. (2019) establish a framework aimed at helping designers
make more informed decisions regarding the visualizations they work
with
Table 6 Categories and examples of works corresponding to the topic of research approaches
Topic: Research approaches
Categories Example
Analytical formalization of a model and its demonstration Franceschini and Maisano (2019) formalize a model to support the deci-
sions of teams of designers in early design stages
Quantitative experiments by examining relationships between vari-
ables with random selection of subjects to experimental and control
groups
Goucher-Lambert and Cagan (2019) use quantitative experiments to
explore the potential of using an untrained crowd workforce to gener-
ate stimuli for trained designers
Quantitative quasi-experiments where there is no random assignment
of a subject to the experimental and control groups
Santolaya etal. (2019) evaluate a methodology to project the design of
more sustainable products by comparing results before and after its
implementation
Quantitative non-experiments with no control on the grouping of
subjects in experimental and control groups
Piccolo etal. (2019) study the role of iterations in design by developing
a statistical model to test multiple hypotheses related to technical and
social factors
Qualitative ethnographic research to document the beliefs and prac-
tices of a particular group in its natural environment
Van der Linden etal. (2019b) use a mix of ethnographic techniques to
analyse the knowledge about the user experience the architects man-
age during their projects
Qualitative phenomenological studies to identify the essence of experi-
ences about a phenomenon as described by participants
Li and Luximon (2018) develop a phenomenological study about the
mobile technology usability by elder people for a designer to build
specialized interfaces
Qualitative hermeneutic studies to disclose how participants’ interpre-
tations determine the way they live in the world
McDonald and Michela (2019) perform a hermeneutic study into the
moral goods that are significant for design
studio instructors
Qualitative grounded theory to derive a general theory grounded in the
views of participants
Bresciani etal. (2019) establish a theoretically grounded framework
aimed at helping designers make more informed decisions regarding
visualizations they work with
Qualitative action research, that involve applied research, moving
experimentation from laboratories to field
Tsai and Van Den Hoven (2018) perform an action research study to
investigate how the accumulation of human traces on objects influ-
ences people’s remembering and usage
Qualitative case study methods, that are in-depth, detailed examination
of particular cases within a real-world context
Tahera etal. (2019) analyze the relationship between testing and design
process, by combining literature study with cases studies about design
and testing practice
Mixed to combine both quantitative and qualitative approaches at
diverse levels
Ozer and Cebeci (2019) use both qualitative and quantitative criteria to
analyse big data to offer customised and personalised online products
with appealing features
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

252 Research in Engineering Design (2023) 34:221–256
1 3
taxonomy of research aims (Table5), approaches (Table6),
data collection techniques (Table7), and instruments
(Table8) in engineering design.
Funding Open Access funding provided thanks to the CRUE-CSIC
agreement with Springer Nature. The authors acknowledge financial
support from Universidad de Valladolid (PID 2020 21 038), Agencia
Estatal de Investigación (PID2020-118216RB-I00) and Agencia Estatal
de Investigación (PID2020-112584RB-C32).
Data Availability All data generated or analysed during this study are
included in this published article [and its supplementaryinformation
files].
Open Access This article is licensed under a Creative Commons Attri-
bution 4.0 International License, which permits use, sharing, adapta-
tion, distribution and reproduction in any medium or format, as long
as you give appropriate credit to the original author(s) and the source,
provide a link to the Creative Commons licence, and indicate if changes
were made. The images or other third party material in this article are
included in the article's Creative Commons licence, unless indicated
otherwise in a credit line to the material. If material is not included in
the article's Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will
need to obtain permission directly from the copyright holder. To view a
copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.
Table 7 Categories and examples of works corresponding to the topic of data collection techniques
Topic: Data collection techniques
Category Example
Observations where researchers records, what is happening, either by
hand, recording it or using measuring equipment
Hyysalo etal. (2019a) detail the process of democratized design of
spaces and services of a public library where authors take field notes
from the meetings and workshops
Simultaneous verbalisation where participants verbalize their thoughts
when performing tasks
Martinec etal. (2019) model design activities of ideation and concept
review by collecting verbalisations in teamworks
Collecting technical documents generated during the engineering
design processes
Morkos etal. (2019) study the impact of requirement elicitation on
the final project. The requirement documents are quantified and cor-
related with the final results
Collecting objects like mock-ups, prototypes and other physical models
that may be relevant for the designing process
Roy and Warren (2019) study card-based design tools making a collec-
tion, review and analysis of 155 card decks for designers
Questionnaires and surveys that permit collecting people's thoughts or
opinions about a certain product, process or method
Vegt etal. (2019) investigates the effects of adding game rules to
brainstorms. Participants filled in a questionnaire about their behav-
ior and engagement
Interviews that are carried out face to face with people who provide
relevant information for the research
Genc etal. (2019) provides recommendations for incorporating tech-
nological components in fashion designs collecting information from
interviews with experts
Table 8 Categories and examples of works corresponding to the topic of instruments
Topic: Instruments
Category Example
Measurements, referring both to metrics obtained with a physical device
and to ratings obtained from human scores
Valverde etal. (2019) explore the quality of push-buttons’ haptic
feedback with kinaesthetic parameters measured from force-dis-
placement curves
Audio, video and image recordings that permit saving user and or prod-
uct interaction for offline analysis
Gyory etal. (2019) compare individual versus group problem-solving
using audio recordings to measure the similarity of the teams’
discourses.
Eye tracking systems for measuring the point of gaze where an inform-
ant is looking
Reimlinger etal. (2019) evaluate how engineers benefit from design
guidelines by capturing gaze sequences with eye-tracking glasses
Simulation and software tools with performing analytical studies or
modelling
Zhang and Thomson (2019) model the development of complex prod-
ucts with an agent-based simulation model
Opinions of stakeholders or groups of people supporting or involved in
the research project
Self (2019) studies communication through design sketches analysing
stakeholders’ interpretations
Opinions of participants in the research projects or product users Menold etal. (2019) explore how prototyping affects user satisfaction
Opinions of expert/designer with a recognized knowledge of the domain Barati etal. (2019) study the understanding of smart materials collect-
ing expert opinions of designers and scientists
Workshops with participation of different agents that jointly discuss and
cooperate
Wlazlak etal. (2019) study visual representations for the communica-
tion of new products in joint analysis workshops with researchers
and the project managers
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

253Research in Engineering Design (2023) 34:221–256
1 3
References
Abi Akle A, Yannou B, Minel S (2019) Information visualisation for
efficient knowledge discovery and informed decision in design by
shopping. J Eng Des 30:227–253. https:// doi. org/ 10. 1080/ 09544
828. 2019. 16233 83
Adrion WR (1993) Research methodology in software engineering. In:
Summary of the Dagstuhl workshop on future directions in soft-
ware engineering” Ed. Tichy, Habermann, and Prechelt, ACM
software engineering notes, SIGSoft, pp 36–37
Aktas BM, Mäkelä M (2019) Negotiation between the maker and mate-
rial: observations on material interactions in felting studio. Int
J Des 13:55–67
Alizadeh R, Allen JK, Mistree F (2020) Managing computational com-
plexity using surrogate models: a critical review. Res Eng Des
31:275–298. https:// doi. org/ 10. 1007/ s00163- 020- 00336-7
Anderiesen H, Scherder E, Goossens R etal (2015) Play experiences
for people with Alzheimer’s disease. Int J Des 9:155–165
Atkinson P (1988) Ethnomethodology: A Critical Review. Annu Rev
Sociol 14:441–465. https:// doi. org/ 10. 1146/ annur ev. so. 14.
080188. 002301
Barati B, Karana E, Hekkert P (2019) Prototyping materials experi-
ence: towards a shared understanding of underdeveloped smart
material composites. Int J Des 13:21–38
Behera AK, McKay A, Earl CF etal (2019) Sharing design definitions
across product life cycles. Res Eng Des 30:339–361. https:// doi.
org/ 10. 1007/ s00163- 018- 00306-0
Beitz W, Pahl G, Grote K (1996) MRS Bull 21:71.https:// doi. org/ 10.
1557/ S0883 76940 00357 76
Belkadi F, Le DuigouDall’Olio JL etal (2019) Knowledge-based plat-
form for traceability and simulation monitoring applied to design
of experiments process: an open source architecture. J Eng Des
30:311–335. https:// doi. org/ 10. 1080/ 09544 828. 2019. 16424 63
Benavides EM, Lara-Rapp O (2019) Ideal output for a robust concep-
tual design process. J Eng Des 30:103–154. https:// doi. org/ 10.
1080/ 09544 828. 2019. 15985 52
Bergstrom JR, Schall A (2014) Eye tracking in user experience design.
Elsevier, Amsterdam
Bladek M (2014) DORA: San Francisco declaration on research assess-
ment (May 2013). Coll Res Libr News 75:191–196
Blessing LTM, Chakrabarti A (2009) DRM: a design reseach method-
ology. Springer, Berlin
Bogle D (2018) 100 years of the PhD in the UK. In: Proceedings of
vitae researcher development international conference 2018, p 12
Bonvoisin J, Halstenberg F, Buchert T, Stark R (2016) A systematic
literature review on modular product design. J Eng Des 27:488–
514. https:// doi. org/ 10. 1080/ 09544 828. 2016. 11664 82
Boussuge F, Tierney CM, Vilmart H etal (2019) Capturing simulation
intent in an ontology: CAD and CAE integration application. J
Eng Des 30:688–725. https:// doi. org/ 10. 1080/ 09544 828. 2019.
16308 06
Boztepe S (2007) User value: competing theories and models. Int J
Des 1:55–63
Bresciani S (2019) Visual design thinking: a collaborative dimensions
framework to profile visualisations. Des Stud 63:92–124. https://
doi. org/ 10. 1016/j. destud. 2019. 04. 001
Brewer MB, Crano WD (2014) Research design and issues of validity.
In: Reis HT, Judd CM (eds) Handbook of research methods in
social and personality psychology, 2nd edn. Cambridge Univer-
sity Press, New York, NY, USA, pp 11–26
Candi M, Gemser G (2010) An agenda for research on the relationships
between industrial design and performance. Int J Des 4:67–77
Cantamessa M (2003) An empirical perspective upon design research.
J Eng Des 14:1–15. https:// doi. org/ 10. 1080/ 09544 82031 00007
8126
Chen B, Hu J, Chen W (2019a) DRE-based semi-automation of the axi-
omatic design transformation: from the functional requirement
to the design parameter. J Eng Des 30:255–287. https:// doi. org/
10. 1080/ 09544 828. 2019. 16272 96
Chen R, Liu Y, Fan H etal (2019b) An integrated approach for auto-
mated physical architecture generation and multi-criteria evalu-
ation for complex product design. J Eng Des 30:63–101. https://
doi. org/ 10. 1080/ 09544 828. 2018. 15632 87
Cheong H, Butscher A (2019) Physics-based simulation ontology: an
ontology to support modelling and reuse of data for physics-
based simulation. J Eng Des 30:655–687. https:// doi. org/ 10.
1080/ 09544 828. 2019. 16443 01
Clark JS, Porath S, Thiele J, Jobe M (2020) Action research. New
Prairie Press, Paris
Comi A, Jaradat S, Whyte J (2019) Constructing shared professional
vision in design work: the role of visual objects and their mate-
rial mediation. Des Stud 64:90–123. https:// doi. org/ 10. 1016/j.
destud. 2019. 06. 003
Cooper R (2019) Design research—its 50-year transformation. Des
Stud 65:6–17. https:// doi. org/ 10. 1016/j. destud. 2019. 10. 002
Coskun A, Zimmerman J, Erbug C (2015) Promoting sustainability
through behavior change: a review. Des Stud 41:183–204. https://
doi. org/ 10. 1016/j. destud. 2015. 08. 008
Cranz G (2016) Ethnography for designers. Routledge, London
Creswell JW (2009) Research design: qualitative, quantitative and
mixed approaches, 3rd edn
Crilly N (2019) Creativity and fixation in the real world: a literature
review of case study research. Des Stud 64:154–168. https:// doi.
org/ 10. 1016/j. destud. 2019. 07. 002
Cuff D (1992) Architecture: the story of practice. MIT Press, London
Daalhuizen J, Timmer R, Van Der Welie M, Gardien P (2019) An
architecture of design doing: a framework for capturing the ever-
evolving practice of design to drive organizational learning. Int
J Des 13:37–52
De Leeuw ED (2008) Choosing the method of data collection
De Lessio MP, Wynn DC, Clarkson PJ (2019) Modelling the planning
system in design and development. Res Eng Des 30:227–249.
https:// doi. org/ 10. 1007/ s00163- 017- 0272-5
Diggle PJ, Chetwynd AG, Chetwynd A (2011) Statistics and scientific
method: an introduction for students and researchers. Oxford
University Press, Oxford
Dorst K (2004) On the problem of design problems—problem solving
and design expertise. J Des Res 4:185–196. https:// doi. org/ 10.
1504/ JDR. 2004. 009841
Eikeland O (2006) The validity of action research—validity in action
research. In: Aagaard Nielsen K, Svensson L (eds) Action
research and interactive research. Shaker Publishing, Maastricht,
pp 193–240
Feijs L, Toeters M (2018) Cellular automata-based generative design
of Pied-de-poule patterns using emergent behavior: case study of
how fashion pieces can help to understand modern complexity.
Int J Des 12:127–144
Ferguson SM, Olewnik AT, Cormier P (2014) A review of mass
customization across marketing, engineering and distribution
domains toward development of a process framework. Res Eng
Des 25:11–30. https:// doi. org/ 10. 1007/ s00163- 013- 0162-4
Fink AS (2000) The role of the researcher in the qualitative research
process. A potential barrier to archiving qualitative data. In:
Forum Qualitative Sozialforschung/Forum: Qualitative Social
Research
Franceschini F, Maisano D (2019) Design decisions: concordance of
designers and effects of the Arrow’s theorem on the collective
preference ranking. Res Eng Des 30:425–434. https:// doi. org/ 10.
1007/ s00163- 019- 00313-9
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

254 Research in Engineering Design (2023) 34:221–256
1 3
Garcia JJ, Pettersen SS, Rehn CF etal (2019) Overspecified vessel
design solutions in multi-stakeholder design problems. Res Eng
Des 30:473–487. https:// doi. org/ 10. 1007/ s00163- 019- 00319-3
Genç Ç, Buruk OT, Yılmaz Sİ etal (2018) Exploring computational
materials for fashion: recommendations for designing fashion-
able wearables. Int J Des 12:1–19
Glass RL (1995) A structure-based critique of contemporary comput-
ing research. J Syst Softw 28:3–7. https:// doi. org/ 10. 1016/ 0164-
1212(94) 00077-Z
Goodwin C (2000) Practices of seeing visual analysis: an ethnometh-
odological approach. SAGE Publications Ltd, London
Goucher-Lambert K, Cagan J (2019) Crowdsourcing inspiration: using
crowd generated inspirational stimuli to support designer idea-
tion. Des Stud 61:1–29. https:// doi. org/ 10. 1016/j. destud. 2019.
01. 001
Graeff E, Maranzana N, Aoussat A (2019) Biomimetics, where are the
biologists? J Eng Des 30:289–310. https:// doi. org/ 10. 1080/ 09544
828. 2019. 16424 62
Gralla EL, Herrmann JW, Morency M (2019) Design problem
decomposition: an empirical study of small teams of facility
designers. Res Eng Des 30:161–185. https:// doi. org/ 10. 1007/
s00163- 018- 0300-0
Gyory JT, Cagan J, Kotovsky K (2019) Are you better off alone? Miti-
gating the underperformance of engineering teams during con-
ceptual design through adaptive process management. Res Eng
Des 30:85–102. https:// doi. org/ 10. 1007/ s00163- 018- 00303-3
Hagedorn TJ, Smith B, Krishnamurty S, Grosse I (2019) Interoper-
ability of disparate engineering domain ontologies using basic
formal ontology. J Eng Des 30:625–654. https:// doi. org/ 10. 1080/
09544 828. 2019. 16308 05
Han J, Forbes H, Schaefer D (2021) An exploration of how creativity,
functionality, and aesthetics are related in design. Res Eng Des
32:289–307. https:// doi. org/ 10. 1007/ s00163- 021- 00366-9
Han X, Li R, Wang J etal (2020) A systematic literature review of
product platform design under uncertainty. J Eng Des 31:266–
296. https:// doi. org/ 10. 1080/ 09544 828. 2019. 16990 36
Hanrahan BV, Yuan CW, Rosson MB etal (2019) Materializing inter-
actions with paper prototyping: a case study of designing social,
collaborative systems with older adults. Des Stud 64:1–26.
https:// doi. org/ 10. 1016/j. destud. 2019. 06. 002
Hatchuel A (2001) Towards design theory and expandable rationality:
the unfinished program of Herbert Simon. J Manag Gov 5:260–
273. https:// doi. org/ 10. 1023/A: 10140 44305 704
Herr CM (2015) Action research as a research method in architecture
and design. In: Proceedings of the 59th annual meeting of the
ISSS-2015 Berlin, Germany
Hiebl MRW (2021) Sample selection in systematic literature reviews
of management research. Organ Res Methods. https:// doi. org/ 10.
1177/ 10944 28120 986851
Hobye M, Ranten MF (2019) Behavioral complexity as a computa-
tional material strategy. Int J Des 13:39–53
Hyysalo S, Hyysalo V, Hakkarainen L (2019a) The work of democra-
tized design in setting-up a hosted citizen-designer community.
Int J Des 13:69–82
Hyysalo S, Marttila T, Perikangas S, Auvinen K (2019b) Codesign
for transitions governance: a mid-range pathway creation toolset
for accelerating sociotechnical change. Des Stud 63:181–203.
https:// doi. org/ 10. 1016/j. destud. 2019. 05. 002
Jagtap S (2019) Design and poverty: a review of contexts, roles of poor
people, and methods. Res Eng Des 30:41–62. https:// doi. org/ 10.
1007/ s00163- 018- 0294-7
Joost G, Bredies K, Christensen M etal (2016) Design as research:
Positions, arguments, perspectives. Birkhäuser, Basel
Jørgensen U (2001) Grounded theory: methodology and theory con-
struction. Int Encycl Soc Behav Sci 1:6396–6399
Kennedy-Clark S (2013) Research by design: design-based research
and the higher degree research student. J Learn Des 6:26–32
Khalaj J, Pedgley O (2019) A semantic discontinuity detection (SDD)
method for comparing designers’ product expressions with users’
product impressions. Des Stud 62:36–67. https:// doi. org/ 10.
1016/j. destud. 2019. 02. 002
King N, Horrocks C, Brooks J (2019) Interviews in qualitative research,
2nd edn. Sage, London
Kitchenham B, Pearl Brereton O, Budgen D etal (2009) Systematic
literature reviews in software engineering—a systematic litera-
ture review. Inf Softw Technol 51:7–15. https:// doi. org/ 10. 1016/j.
infsof. 2008. 09. 009
Koskinen I, Zimmerman J, Binder T etal (2011) Design research
through practice: from the lab, field, and showroom. Elsevier,
Amsterdam
Kothari CR (2004) Research methodology: methods and techniques.
New Age International, New Delhi
Lapan SD, Quartaroli MT, Riemer FJ (eds) (2012) Qualitative research:
an introduction to methods and designs. Jossey-Bass/Wiley, New
York
Li Q, Luximon Y (2018) Understanding older adults’ post-adoption
usage behavior and perceptions of mobile technology. Int J Des
12:93–110
Li Y, Roy U, Saltz JS (2019a) Towards an integrated process model
for new product development with data-driven features
(NPD3). Res Eng Des 30:271–289. https:// doi. org/ 10. 1007/
s00163- 019- 00308-6
Li Y, Shieh M-D, Yang C-C (2019b) A posterior preference articu-
lation approach to Kansei engineering system for product
form design. Res Eng Des 30:3–19. https:// doi. org/ 10. 1007/
s00163- 018- 0297-4
Lloyd P (2019) You make it and you try it out: Seeds of design disci-
pline futures. Des Stud 65:167–181. https:// doi. org/ 10. 1016/j.
destud. 2019. 10. 008
Luck R (2019) Design research, architectural research, architectural
design research: an argument on disciplinarity and identity. Des
Stud 65:152–166. https:// doi. org/ 10. 1016/j. destud. 2019. 11. 001
MacQueen KM, Guest G (2008) An introduction to team-based quali-
tative research. In: Guest G, MacQueen KM (eds) Handbook
for team-based qualitative research. Altamira Press, Lanham, pp
3–19
Martinec T, Škec S, Horvat N, Štorga M (2019) A state-transition
model of team conceptual design activity. Res Eng Des 30:103–
132. https:// doi. org/ 10. 1007/ s00163- 018- 00305-1
Mathias D, Snider C, Hicks B, Ranscombe C (2019) Accelerating prod-
uct prototyping through hybrid methods: coupling 3D printing
and LEGO. Des Stud 62:68–99. https:// doi. org/ 10. 1016/j. destud.
2019. 04. 003
McDonald JK, Michela E (2019) The design critique and the moral
goods of studio pedagogy. Des Stud 62:1–35. https:// doi. org/ 10.
1016/j. destud. 2019. 02. 001
McKinnon H, Sade G (2019) Exploring the home environment: fusing
rubbish and design to encourage participant agency and self-
reflection. Des Stud 63:155–180. https:// doi. org/ 10. 1016/j. des-
tud. 2019. 05. 001
Menold J, Simpson TW, Jablokow K (2019) The prototype for X frame-
work: exploring the effects of a structured prototyping framework
on functional prototypes. Res Eng Des 30:187–201. https:// doi.
org/ 10. 1007/ s00163- 018- 0289-4
Miles MB, Huberman AM, Saldaña J (2020) Qualitative data analysis:
a methods sourcebook, 4th edn. SAGE Publications Inc., London
Morkos B, Joshi S, Summers JD (2019) Investigating the impact of
requirements elicitation and evolution on course performance in
a pre-capstone design course. J Eng Des 30:155–179
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

255Research in Engineering Design (2023) 34:221–256
1 3
Motta-Filho MA (2021) Brand experience manual: bridging the gap
between brand strategy and customer experience. Rev Manag
Sci 15:1173–1204
Ohnemus KR, Biers DW (1993) Retrospective versus concurrent think-
ing-out-loud in usability testing. In: Proceedings of the human
factors and ergonomics society annual meeting. SAGE Publica-
tions Sage CA, Los Angeles, pp 1127–1131
Ozer M, Cebeci U (2019) Affective design using big data within the
context of online shopping. J Eng Des 30:368–384. https:// doi.
org/ 10. 1080/ 09544 828. 2019. 16568 03
Pakkanen J, Juuti T, Lehtonen T (2019) Identifying and addressing
challenges in the engineering design of modular systems—case
studies in the manufacturing industry. J Eng Des 30:32–61.
https:// doi. org/ 10. 1080/ 09544 828. 2018. 15527 79
Paluck EL, Cialdini RB (2014) Field research methods. In: Reis HT,
Judd CM (eds) Handbook of research methods in social and per-
sonality psychology, 2nd edn. Cambridge University Press, New
York, NY, USA, pp 81–97
Park-Lee S, Person O (2018) Briefng beyond documentation: an inter-
view study on industrial design consulting practices in Finland.
Int J Des 12:73–91
Park D, Han J, Childs PRN (2021) 266 Fuzzy front-end studies: current
state and future directions for new product development. Res Eng
Des 32:377–409. https:// doi. org/ 10. 1007/ s00163- 021- 00365-w
Pedgley O, Şener B, Lilley D, Bridgens B (2018) Embracing material
surface imperfections in product design. Int J Des 12:21–33
Pérez-Gómez A (1999) Hermeneutics as discourse in design. Des
Issues 15:71–79
Pessôa MVP, Becker JMJ (2020) Smart design engineering: a literature
review of the impact of the 4th industrial revolution on product
design and development. Res Eng Des 31:175–195
Petreca B, Saito C, Baurley S etal (2019) Radically relational tools: a
design framework to explore materials through embodied pro-
cesses. Int J Des 13:7–20
Piccolo SA, Maier AM, Lehmann S, McMahon CA (2019) Iterations as
the result of social and technical factors: empirical evidence from
a large-scale design project. Res Eng Des 30:251–270. https://
doi. org/ 10. 1007/ s00163- 018- 0301-z
Radhakrishna RB (2007) Tips for developing and testing question-
naires/instruments. J Ext 45:1TOT2
Randolph MF (2003) Science and empiricism in pile foundation
design. Géotechnique 53:847–875
Rapley T, Rees G (2018) Collecting documents as data. In: Flick U
(ed) The SAGE handbook of qualitative data collection. Sage,
London, pp 378–391
Redström J (2017) Making design theory. MIT Press, London
Reich Y, Subrahmanian E (2021) Mapping and enhancing design
studies with psI meta-theoretic design framework. Proc Des
Soc 1:2007–2016. https:// doi. org/ 10. 1017/ pds. 2021. 462
Reich Y, Subrahmanian E (2020) The PSI framework and theory of
design. IEEE Trans Eng Manag. https:// doi. org/ 10. 1109/ TEM.
2020. 29732 38
Reimlinger B, Lohmeyer Q, Moryson R, Meboldt M (2019) A com-
parison of how novice and experienced design engineers ben-
efit from design guidelines. Des Stud 63:204–223. https:// doi.
org/ 10. 1016/j. destud. 2019. 04. 004
Renzi C, Leali F, Di Angelo L (2017) A review on decision-making
methods in engineering design for the automotive industry. J
Eng Des 28:118–143. https:// doi. org/ 10. 1080/ 09544 828. 2016.
12747 20
Roesler A, Grigg EB, Martin LD etal (2019) Practice-centered
design of an anesthesia medication template to reduce medica-
tion handling errors in the operating room. Int J Des 13:53–68
Roy R, Warren JP (2019) Card-based design tools: a review and
analysis of 155 card decks for designers and designing. Des
Stud 63:125–154. https:// doi. org/ 10. 1016/j. destud. 2019. 04. 002
Saldaña J (2021) The coding manual for qualitative researchers, 4th
edn. SAGE Publications Ltd, Thousand Oaks
Saliminamin S, Becattini N, Cascini G (2019) Sources of creativity
stimulation for designing the next generation of technical sys-
tems: correlations with R&D designers’ performance. Res Eng
Des 30:133–153. https:// doi. org/ 10. 1007/ s00163- 018- 0299-2
Santolaya JL, Lacasa E, Biedermann A, Muñoz N (2019) A practical
methodology to project the design of more sustainable prod-
ucts in the production stage. Res Eng Des 30:539–558. https://
doi. org/ 10. 1007/ s00163- 019- 00320-w
Saravanan A, Jerald J (2019) Ontological model-based optimal
determination of geometric tolerances in an assembly using
the hybridised neural network and genetic algorithm. J Eng Des
30:180–198. https:// doi. org/ 10. 1080/ 09544 828. 2019. 16055 85
Schröppel T, Miehling J, Wartzack S (2021) The role of product
development in the battle against product-related stigma—a
literature review. J Eng Des 32:247–270. https:// doi. org/ 10.
1080/ 09544 828. 2021. 18790 31
Self JA (2019) Communication through design sketches: implica-
tions for stakeholder interpretation during concept design. Des
Stud 63:1–36. https:// doi. org/ 10. 1016/j. destud. 2019. 02. 003
Selvefors A, Marx C, Karlsson MAIC, Rahe U (2018) (How) can
appliances be designed to support less energy-intensive use?
Insights from a field study on kitchen appliances. Int J Des
12:35–55
Simon HA (1996) The science of design: creating the artificial
Solomon M (2007) Social empiricism. MIT Press, Cambridge
Stigliano A (1989) Hermeneutical practice. Saybrook Rev 7:47–67
Stringer ET (2008) Action research in education. Pearson Prentice
Hall, Upper Saddle River
Subrahmanian E, Reich Y, Krishnan S (2020) We are not users: dia-
logues, diversity, and design. MIT Press, Cambridge
Sung E, Kelley TR, Han J (2019) Influence of sketching instruc-
tion on elementary students’ design cognition: a study of three
sketching approaches. J Eng Des 30:199–226. https:// doi. org/
10. 1080/ 09544 828. 2019. 16174 13
Swann C (2002) Action research and the practice of design. Des
Issues 18:49–61
Tahera K, Wynn DC, Earl C, Eckert CM (2019) Testing in the incre-
mental design and development of complex products. Res Eng
Des 30:291–316. https:// doi. org/ 10. 1007/ s00163- 018- 0295-6
Takahashi I, Oki M, Bourreau B etal (2018) An empathic design
approach to an augmented gymnasium in a special needs
school setting. Int J Des 12:111–125
Tempczyk H (1986) A survey of research and studies on design.
Des Stud 7:199–215. https:// doi. org/ 10. 1016/ 0142- 694X(86)
90037-2
Tenopir C, King DW (2014) The growth of journals publishing. In:
The future of the academic journal. Elsevier, Amsterdam, pp
159–178
Thiese MS (2014) Observational and interventional study design
types; an overview. Biochem Medica 24:199–210
Tsai WC, Van Den Hoven E (2018) Memory probes: exploring ret-
rospective user experience through traces of use on cherished
objects. Int J Des 12:57–72
Valverde N, Ribeiro AMR, Henriques E, Fontul M (2019) An engi-
neering perspective on the quality of the automotive push-but-
tons’ haptic feedback in optimal and suboptimal interactions. J
Eng Des 30:336–367. https:// doi. org/ 10. 1080/ 09544 828. 2019.
16568 02
Van der Bijl-Brouwer M, Dorst K (2017) Advancing the strategic
impact of human-centred design. Des Stud 53:1–23. https://
doi. org/ 10. 1016/j. destud. 2017. 06. 003
Van der Linden V, Dong H, Heylighen A (2019a) Populating archi-
tectural design: introducing scenario-based design in residen-
tial care projects. Int J Des 13:21–36
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

256 Research in Engineering Design (2023) 34:221–256
1 3
Van der Linden V, Dong H, Heylighen A (2019b) Tracing architects’
fragile knowing about users in the socio-material environment
of design practice. Des Stud 63:65–91. https:// doi. org/ 10.
1016/j. destud. 2019. 02. 004
Van Kuijk J, Daalhuizen J, Christiaans H (2019) Drivers of usability
in product design practice: induction of a framework through
a case study of three product development projects. Des Stud
60:139–179. https:// doi. org/ 10. 1016/j. destud. 2018. 06. 002
Vasantha GVA, Roy R, Lelah A, Brissaud D (2012) A review of
product–service systems design methodologies. J Eng Des
23:635–659. https:// doi. org/ 10. 1080/ 09544 828. 2011. 639712
Vaughan L (2017) Practice-based design research. Bloomsbury Pub-
lishing, London
Vegt N, Visch V, Vermeeren A etal (2019) Balancing game rules
for improving creative output of group brainstorms. Int J Des
13:1–19
Wang R, Nellippallil AB, Wang G etal (2019) Ontology-based
uncertainty management approach in designing of robust
decision workflows. J Eng Des 30:726–757. https:// doi. org/
10. 1080/ 09544 828. 2019. 16689 18
Wasson C (2000) Ethnography in the field of design. Hum Organ
377–388
Wilson A (2015) A guide to phenomenological research. Nurs Stand
29:38
Wlazlak P, Eriksson Y, Johansson G, Ahlin P (2019) Visual rep-
resentations for communication in geographically distributed
new product development projects. J Eng Des 30:385–403.
https:// doi. org/ 10. 1080/ 09544 828. 2019. 16613 62
Wood AE, Mattson CA (2019) Quantifying the effects of various
factors on the utility of design ethnography in the develop-
ing world. Res Eng Des 30:317–338. https:// doi. org/ 10. 1007/
s00163- 018- 00304-2
Yang S, Santoro F, Sulthan MA, Zhao YF (2019) A numerical-based
part consolidation candidate detection approach with modulari-
zation considerations. Res Eng Des 30:63–83. https:// doi. org/
10. 1007/ s00163- 018- 0298-3
Yin RK (2009) Case study research: design and methods, 4th edn.
Sage Publications, Thousand Oaks
Zhang X, Thomson V (2019) Modelling the development of com-
plex products using a knowledge perspective. Res Eng Des
30:203–226. https:// doi. org/ 10. 1007/ s00163- 017- 0274-3
Content courtesy of Springer Nature, terms of use apply. Rights reserved.

1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com