Approaching Knowledge Dynamics Across the Product Development Process with Methods of Social Research

Abstract

Knowledge is a crucial factor in state-of-the-art product development. It is often provided by stakeholders from divers disciplinary and individual backgrounds and has to be integrated to create competitive products. Still, it is not fully understood, how knowledge is generated, transformed, transferred and integrated in complex product development processes. To investigate the dynamic interrelations between involved stakeholders, applied knowledge types and related artefacts, researchers at the TU Berlin conducted and evaluated a student experiment to study basic phenomena of development projects. In relation to research methods and instruments applied in this experiment, various improvement opportunities were identified. In this paper, the experimental setting and its results are critically analysed from a social science perspective in order to generate improved research design. Based on the results of this analysis, a first set of methods and instruments from social sciences are identified that can be applied in further experiments. The goal is to develop a methodological toolbox that can be used to approach research on knowledge dynamics in product development.

Full text

Cite this article: Wang, W.M., Mörike, F., Hergesell, J., Baur, N., Feufel, M., Stark, R. (2019) ‘Approaching Knowledge
Dynamics Across the Product Development Process with Methods of Social Research’, in Proceedings of the 22nd
International Conference on Engineering Design (ICED19), Delft, The Netherlands, 5-8 August 2019. DOI:10.1017/
dsi.2019.256
ICED19
INTERNATIONAL CONFERENCE ON ENGINEERING DESIGN, ICED19
5-8 AUGUST 2019, DELFT, THE NETHERLANDS
ICED19
APPROACHING KNOWLEDGE DYNAMICS ACROSS THE
PRODUCT DEVELOPMENT PROCESS WITH METHODS OF
SOCIAL RESEARCH
Wang, Wei Min; Mörike, Frauke; Hergesell, Jannis; Baur, Nina; Feufel, Markus; Stark, Rainer
Technische Universität Berlin
ABSTRACT
Knowledge is a crucial factor in state-of-the-art product development. It is often provided by
stakeholders from divers disciplinary and individual backgrounds and has to be integrated to create
competitive products. Still, it is not fully understood, how knowledge is generated, transformed,
transferred and integrated in complex product development processes. To investigate the dynamic
interrelations between involved stakeholders, applied knowledge types and related artefacts, researchers
at the TU Berlin conducted and evaluated a student experiment to study basic phenomena of
development projects. In relation to research methods and instruments applied in this experiment,
various improvement opportunities were identified. In this paper, the experimental setting and its results
are critically analysed from a social science perspective in order to generate improved research design.
Based on the results of this analysis, a first set of methods and instruments from social sciences are
identified that can be applied in further experiments. The goal is to develop a methodological toolbox
that can be used to approach research on knowledge dynamics in product development.
Keywords: Design theory, Organisation of product development, Knowledge management, Knowledge
dynamics in product development, Evaluation with students
Contact:
Wang, Wei Min
Technische Universität Berlin
Industrial Information Technology
Germany
w.wang@tu-berlin.de
2497

ICED19
1 INTRODUCTION
The development of modern products increasingly demands the networking of different disciplines
due to the convergence of mechanics, electrics/electronics, software and services. To create innovative
and successful products, knowledge from stakeholders with heterogeneous backgrounds must be
integrated. In industrial practice, the control and management of such product development processes
(PDPs) requires the continuous exchange of domain-specific and technical as well as organizational
and process-related knowledge (Berggreen and Kampf 2016). As knowledge per se arises in
individuals or groups of persons and is applied by those, the daily development activities in teams
directed to the achievement of objectives leads to a constant and situational increase and negotiation of
knowledge. This knowledge can be based on data and information and represents a variable mixture of
experiences, individual beliefs and contextual information. (Davenport et al., 1999) While engineering
sciences have developed a rich body of insights on how designers process knowledge individually in
the course of the PDP (e.g. Ahmed and Wallace, 2004; Mohedas et al. 2015), the understanding of
knowledge dynamics within PDP teams across the entire PDP delineates a gap, especially with regard
to which knowledge is required at which point in the development process and by whom it must be
provided.
In the DFG-funded (German Research Foundation, DFG) project “Knowledge in stage-gate driven
product development processes” researchers at the Technical University Berlin (TUB) sought to gain a
deeper understanding about knowledge-related activities in PDPs and how they vary along the process.
The aim is to develop a model to identify the knowledge related to specific development phases and
their dynamic interactions with stakeholders, activities and artefacts in the course of a stage-gate
driven PDP (SGPDP). A first part of the project was a student experiment conducted as a project on
product development to investigate basic phenomena of knowledge dynamics in SGPDP. As the
insights into knowledge dynamics from this initial student experiment turned out to be unsatisfactory,
this article presents a critical evaluation of the experiment in relation to the research methods applied
as well as lessons learned for an improved research design. A new dimension on approaching the issue
will be introduced by including the analysis of the experiment from the methodological perspective of
human factors research and sociology to ideate how the complex topic of knowledge dynamics in
SGPDPs can be approached with interdisciplinary knowledge. In particular, we seek to answer the
following research questions (RQ):
RQ1: Which methods of social research are suitable for assessing knowledge dynamics in
SGPDPs?
RQ2: How can these methods be adopted and mixed to improve the quality of research findings
on knowledge dynamics?
RQ3: What are the opportunities and potential limitations of methods of social research in this
context?
We like to emphasize, that the goal of this paper is not evaluation and discussion of the results of the
initial experiment, but to contribute to an improved research of knowledge dynamics by
conceptualizing an interdisciplinary research design. To enable full transparency, however, it is
necessary to present the original research design of the initial experiment as a case study in the
following section in order to consequently discuss possible error sources from the perspective of the
social research and answer the research questions.
2 CASE STUDY: INITIAL STUDENT EXPERIMENT - EVALUATION OF A
STUDENT PROJECT ON PRODUCT DEVELOPMENT
2.1 Original goal of the experiment
As mentioned above, the initial experiment, conducted as a student project, was part of the DFG-
funded research project. The overall goal was to identify an initial set of knowledge related activities
in PDPs by evaluating an exemplary development project. The results were intended as a basis to
create a first draft of a generic model to describe relationships between development activities that
demand the processing of knowledge, the knowledge bearers involved in those activities (people and
IT-systems), the related types of knowledge as well as the artefacts representing the knowledge.
2498

ICED19
2.2 Research design of the initial experiment
The experiment was integrated into a master’s course at the Department for Industrial Information
Technology (IIT) of the TUB. Two student teams consisting of seven persons each were tasked with
the development of a Product-Service System (PSS) concept for a single-axle tractor (see Figure 1).
The student groups role-played to be start-ups trying to convince an investor of their product. To
simulate organizational structures in actual industrial companies, each student was instructed to take
on one of the following project-specific roles with respective responsibilities: Project Manager (PM):
represents the team to external parties and is responsible for internal project planning and controlling;
Product Owner (PO): is responsible for the overall product view; the product should represent a good
overall package while adhering to the specifications; Sustainability Expert (SE): responsible for the
consideration (not implementation) of sustainability aspects; advises PO on sustainability aspects;
Product Developers (PD): responsible for technical feasibility.
The progress of the project was examined via three design reviews (DR) according to predefined
deliverables. During the DRs, the students had to present their results to a steering board consisting of
researchers from the IIT department. For DR1, students hat to develop and present three concepts for a
PSS. For DR2, they had to select and elaborate one of the three concepts. Among other things, this
included the development of an add-on module, a requirements list and a service blue-print for the
PSS concept. For DR3, they had to present finalized models (CAD models, system models, business
canvas etc.) of the PSS as well as marketing materials. At the end of the course, they had to deliver a
closing report summarizing the entire project.
Figure 1. Task for the student project - Development of a PSS concept including an add-on
module
2.3 Evaluation of the student project using semi-structured interviews und
documentary analysis
At the beginning of the course, the students were introduced to the research project and were informed
that they would participate in an experiment. The main evaluation method were semi-structured
interviews conducted as group interviews. This method was chosen in order to enable the students to
report on their activities in a narrative way, but also to steer the answers thematically through an
interview guide. (Flick, 2009, 2011) The method of individual interviews was discarded due to time
limitations of the course. In order to keep the interviews manageable, each of the student teams was
further split into two sub groups, one comprising roles with managerial tasks (i.e. PM, PO, SE) and the
other the PDs. It was assumed that the answers within these role-groups would be more homogeneous
due to similarities in the tasks and therefore facilitate the evaluation of the results. For the interviews, an
interview guide comprising 29 questions was developed. This interview guide was based on a conceptual
framework for analyzing knowledge dynamics in engineering science developed at the TUB. The
questions within the interview guide were organized along the key concepts of this framework:
knowledge acquisition, knowledge transformation, knowledge generation, knowledge transfer,
knowledge coordination and knowledge integration (cf. Märten et al., 2018). The interview guide was
pre-tested and refined with researchers of the IIT department. During the experiment, a total of three
interviews were conducted with each sub group, arranged one week before the respective DR. The
timing of the interviews was chosen as it was assumed that students would have to make all product
decisions in advance in order to present the final results in the DRs. The researchers who conducted the
interviews took part in the DRs, but were not involved in the rating process of the project results.
In addition to the interviews, both student teams were advised to save all project-related data (i.a.
CAD files, sketches, concept documentations, meeting minutes, presentations, reports, calculation
results) into a university-internal cloud-based shared folder (cf. Robinson, 2010). The students were
also instructed to follow a folder structure and a naming pattern, which allowed for the tracing of
version history, the respective editor and the date of a change. This facilitated the later analysis of
2499

ICED19
change logs and work patterns within the teams on the documents. The parallel observation of data
development in the shared folders was intended to provide information on which artefacts were
created in which project phase and to which tasks and roles they can be assigned. By triangulating the
interview results with the changes in the data folders in the corresponding period, the initial aim was to
establish a connection between the activities carried out and the artefacts involved.
3 CRITICAL ANALYSIS OF THE EXPERIMENT FROM SOCIAL SCIENCE
PERSPECTIVE
The results from the interviews and documentary analysis were quantified, organized in a database and
statistically analyzed using “R”. Comparative analyses of response frequencies were performed with
regard to the following aspects:
Timing: Differences in knowledge activities in the course of the project, e.g. from where and how
is knowledge acquired and transformed in specific phases of the project?
Role: Role-specific differences in knowledge processing between the managerial roles and PDs,
e.g. are managerial roles looking for different kind of knowledge than PDs?
Group: Differences in knowledge processing between the two development teams, e.g. was one
team using different knowledge sources than the other?
Bivariate analyses were also carried out, e.g. to examine differences between roles at a particular
stage. The evaluation of the results allowed for first insights about how development activities are
organized and carried out on an operative team level. Based on these insights, a first theoretical
framework could be derived that can be used to describe knowledge processing activities in PDPs
grounding on stakeholders, activities and artefacts involved in the PDP (Wang et al., 2017).
However, the evaluation of the experiment results also revealed shortcomings regarding the overall
research design, sampling, data collection instruments and data quality that limited the validity of the
results in several respects. Therefore, the present student experiment is critically analyzed from a social
science perspective to identify improvement approaches. It is necessary to reflect these methodological
questions to improve the data quality and to achieve a higher level of detail of questioning in future
evaluations. At the beginning of such an interdisciplinary cooperation, methodological problems should
first be identified based on the social research`s experience. Subsequently, some suggestions for
improvement can be formulated and alternative methodical approaches can be named.
3.1 Critical analysis of the research design
A well-known problem concerning the overall research design is artificiality (Thierbach and Lorenz,
2014) because the circumstances of product development in a student course deviates from a real
industrial environment. Among other things, there was no organizational or personal knowledge e.g.
on product history, the product and the target market. In addition, there was neither an organizational
structure with defined decision-making authorities nor a specified development process. The student
project aimed at simulating a rudimental organizational structure with rules for the allocation of tasks
and competencies by assigning roles with defined tasks and responsibilities. The “artificial
assignment” of roles turned out to be a problem, as the students were not socialized into these roles
and therefore could not simulate them according to the project’s goal. However, evaluation results
indicated that the students did not stick to their chosen roles during the course of the project (a, Figure
2). Thus, it could not be clearly determined whether there is a difference in knowledge processing
between roles.
The scenario of the experiment, in which the student teams represent start-ups that are seeking investor
money, plausibilises the framework conditions of the development project, but also results in decision
situations no longer coinciding with the DRs (b, Figure 2). In real life, decisions are actually made
during the review meetings (e.g. selecting a concept) and reviewers should play an active role in the
decision-making process, e.g. as a management board that determines the direction of development
(e.g. decides on alternative solutions based on the presentation) and approval for the next development
phase. Whereas in the evaluated student project, the students made product-related decisions before
the meeting (e.g. performing utility analyses) and merely presented and explained their decisions. The
reviewers took on the role of external evaluators instead of being part of the development project and
were not involved in the actual process of making product-related decision. Consequently, the
2500

ICED19
observation of the DR meetings could not give any hints about the actual decision making mechanisms
within the team.
3.2 Critical analysis of sampling
Due to the selected course, participants were rather homogenous: All of them were male students,
most of them being white middle-class of a similar age, mostly studying mechanical engineering in a
similar semester in a master’s degree. All of them had prior experience in product development and
with their role-specific tasks, gained either at university or in internships and student part-time jobs.
Only for the role of the sustainability expert did the participants hardly have any previous knowledge.
In order to generalize results, sampling need to be either random samples (Baur, 2014) or purposeful
samples (Creswell and Poth 2017, pp. 156-159). However, in our case study, sampling was arbitrary,
resulting in a biased sample (c, Figure 2) (; Byrne 2009; Yin 2014). Therefore, generalization beyond
the single case is not possible.
3.3 Critical analysis of data collection instruments
The data collection instruments used in the evaluation need to be improved in order to be able to
record dynamic aspects of the knowledge processing activities as extensively as possible (d, Figure 2).
In the course of the evaluation, it was revealed that semi-structured group interviews did not allow for
the necessary level of detail of questioning. The acquisition of information and knowledge is also
intuitively based on one’s own experience. To allocate applied or activated knowledge for certain
activities retrospectively, in-depth interviews or narrative techniques are more suitable. For a
replication of the evaluation in an industrial context, an interview method must be chosen that allows
in-depth questions to be put to a participant. It should also be asked critically whether interviews are
the most useful method for an exploratory analysis of knowledge dynamics. These issues could be
processed in an interdisciplinary cooperation with social research: The retrieval of experiences, e.g.,
often occurs unconsciously in the problem-solving process. A test interview with a member of the
scientific staff revealed that developers restrict the solution space according to their own experience or
make a preselection of solution principles without explicitly going through a formal selection process.
Hence, it could be useful to observe the knowledge processing situations, e.g. by participating in such
situations and using ethnographic methods.
3.4 Critical analysis of data quality
Concerning data quality of the semi-structured interviews, a first criticism is the interview situation (e,
Figure 2). In general, response behavior should be influenced as little as possible. Otherwise, a
distortion of the response behavior will lead to data bias. An unbiased interview setting could be
accomplished by interviewing respondents in a situation where they do not need to fear negative
consequences of their responses (e.g. bad marks). Although the interviewer in the present evaluation
explicitly announced that he would not be involved in the rating process of the course, his affiliation
IIT could have influenced the response behavior. The respondents were also interviewed in groups, so
that their responses could be biased by the social desirability of the group. For example, students may
seek to conceal within-group conflicts in order to protect individual students or aim at presenting their
group as successful. In addition, the interviews need to be anonymized. In the best case, no
conclusions can be drawn to the identity of the respondent, which makes it possible to give
“unpleasant” answers.
Furthermore, semi-structured interviews seek a compromise between open-ended narrative interviews
and surveys. However, in the case of this evaluation, this kind of compromise turned out to be more
detrimental: Surveys (Moser/Kalton 2017) usually have the advantage of being to process a higher
number of cases, fast data preparation and easy comparability. Due to the open-ended answers in the
semi-structured interviews, these advantages did not apply in the evaluation - instead, researchers had
to strongly interpret data, answers were not always comparable, and data preparation was rather time-
consuming. In contrast, both surveys and the semi-structured interviews used in the evaluation can
only achieve reliable results, if the relevant issues are already known in advance and can be
operationalized into precise, one-dimensional closed questions with a given set of equally precise and
exhaustive possible answers. However, the student evaluation pursued here had an exploratory
research interest. This means that in such an early phase of the research process, too little is known
about the relevant aspects of the student evaluation to formulate precisely closed questions.
2501

ICED19
Even if more about the topic was known, knowledge dynamics represent a complex social process.
This implies selecting a data collection method that allows access to semantic content and social
dynamics. In general, the best methodical approach for such objects are qualitative research designs
(Creswell and Poth, 2017), as standardized instruments limit respondents’ ability to answers in detail
and in their own words. As a result, respondents are unable to fully introduce their experiences into
their responses. As well, it is hard for the researchers to figure out new, unexpected aspects of their
research topic from respondents’ answers to closed questions. Therefore, the research process of the
student evaluation does not fully exploit the students’ potential to provide their knowledge. All in all,
the problems of the evaluation could also be a result of a lack of precision during the
operationalization of the questionnaire and the student’s very limited answer options (f, Figure 2).
Figure 2. Allocation of identified improvement potentials in the evaluation of the student experiment
4 APPROACHING COMPLEX RESEARCH QUESTIONS WITH METHODS OF
SOCIAL RESEARCH
The following discussion of error sources should not be understood as deterministic or strict
instruction for future experiments. Rather, it is a matter of showing suggestions for improvement from
the diverse range of social research approaches for the development of research designs, which of
course have to be adapted in individual cases for future student experiments.
4.1 Exploratory sequential mixed methods
For future evaluations, an exploratory sequential mixed methods design (Schoonenboom and Johnson,
2017) would be more suitable: (a) One would start with qualitative interviews. This procedure includes
inductive strategies in the research process and has the advantage that the respondents are very free in
their response behavior and so also previously unrecognized aspects of knowledge dynamics can be
included in the interview (Miller/Glassner 2016; Holstein/Gubrium 2016). Disadvantages of qualitative
interviews are a difficult comparability of the different interviews as well as an increased data analysis
effort. A less time-consuming qualitative methods are focus groups (Wilkinson 2016). In doing so, the
entire group of students would be interviewed together by asking them to discuss particular aspects of
their work. This procedure has the advantage that in addition to the concrete information that enables the
creation of a questionnaire, also negotiation processes and controversially discussed, possibly
problematic conflictive phases of the development process become accessible for the researchers. A
disadvantage of this approach is a greater data analysis effort and a possibly distorted response behavior,
as the group members did not want to speak openly before the other participants.
(b) After one has analyzed the qualitative data and developed first hypotheses, these hypotheses can be
tested in a survey - and even surveys would allow for integrating open-ended and standardized elements
can be combined. For instance, so-called open questions can be asked in a first short questionnaire
(Brymann 2016, 248). This means that respondents are stimulated by short but broad question and then
they can answer in their own words without being influenced by a suggestive posing of questions. These
answers can then be evaluated by the researchers. This approach has the advantage that the individual
answers of the respondents are well comparable with each other and the data analysis effort remains
rather low.
2502

ICED19
4.2 Ethnography in PDPs and “knowledge” as field of study
The previous section illustrated how the approach to gain insights into the role(s) knowledge plays in
PDPs through interviews could be improved within the logic of collecting interview data. As already
mentioned above, it would be also sensible to mix interviews with observation-based data collection
(Thierbach and Lorenz, 2014) in order to improve data quality. In this section, the methodological
toolbox is extended to include modes of in-situ data collection, more specifically ethnography.
Ethnography (Silverman, 2016) is a method originating from the realms of social and cultural sciences
with the approach to undertake research by systematically observing and participating (to a greater or
lesser degree) in the lives of the studied groups. The idea is to gain insights into practices through
presence in the same social setting, by “walking a mile in the shoes of others”(Madden, 2006).
Literally translated from Greek, ethnography means “writing about alien groups of people”,
highlighting its (stereo-)typical association with anthropological research in remote places (Gellner
and Hirsch, 2001). The application of ethnographic methods in design contexts garners a decade-long
tradition since its origins in the 1980s with the observations of ethnographer Lucy Suchman on how
people interacted with copy machines (Suchman, 1985). To date, design ethnography is an established
field in human-computer-interaction research (Nunez and Fitzpatrick, 2018), and increasingly applied
in PDPs (Butlewski et al., 2016). In engineering sciences, the use of ethnography is primarily focusing
on providing insights on users’ practices (Vinck, 2003), requirements and/or product evaluation
(Mohedas et al., 2015). In the suggested revised research design to approach knowledge dynamics,
ethnography is applied to gain insights into the PDP itself, which delineates a clear gap in current
research. This additional perspective should account for the situation that evaluations of knowledge
dynamics in student experiments seem to be coined by a number of perceived challenges, such as the
uncontrollable (and non-reproducible) dynamics of semester-length student work groups, the
inevitably hierarchical frame of interacting students and faculty staff, students’ heterogeneous
backgrounds on organizational knowledge about PDPs acquired in courses at university or at various
industry organizations, and, apparently numerous communication iterations within and outside course
room contexts. At the end of the first student experiment, hence, the decision-making mechanisms
within the teams and their knowledge base remained opaque to the research team. Another aspect is
that “knowledge” as a research topic is a concept which students, and most likely practitioners,
struggle to denominate and label when directly asked about, especially in the context of their
unreflexively everyday study or work life (Klein et al., 1989). The aim is to approach all those
seemingly challenging factors from a different angle and to turn them into a strength of the research
project. While such a setting is for the above reasons difficult to put into an evaluational research
context, the complexities are akin to those encountered in industry-based PDPs and therefore
worthwhile to include into the research design rather than to attempt to rule complexity out.
4.2.1 Participant observation: in-situ data collection
Participant observation stands at the center of ethnographic research and describes the seemingly
oxymoronic approach where a researcher takes part in the daily activities, interactions, and events of a
group of people as one of the means of learning the explicit and tacit aspects of their routines and
practices (Robson and McCartan 2016). For research on knowledge acquisition and negotiation
processes in student PDPs, the in-situ observation mode of ethnography depicts a promising method, as
it allows for a maximum level of flexibility to adapt data collection to the dynamics of project team
interactions. The ethnographer can be amongst the project teams to observe how tasks are distributed
amongst the team members and to witness role negotiations or decision-making processes in preparation
for milestone submissions. As long as the student team members accept the presence of the ethnographer
and are willing to provide insights into their ways of working as a team along the project cycle, the time
and context of data collection is independent from the actual course room setting. The communication
dynamics within the PDP teams might be even more active and less controlled by impression
management the farther away from pre-defined interaction formats such as milestone presentations and
course room locations. In short, participant observation “puts you were the action is and lets you collect
data” (Bernard, 2017), and precisely the documentation of this wide array of (inter-)action depicts the
backbone of ethnographic research. To achieve this, ethnographers take notes on relevant interactions
throughout the observation phase and consolidate these jottings shortly after or at the end of the day into
a set of detailed field notes (Emerson et al., 2011). Over the course of different observation phases, the
field notes allow for several layers of insights into the knowledge dynamics at play in stage-gate-driven
2503

ICED19
PDPs. To ensure rich sets of data for analysis and triangulation, ethnography can encompass the
collection of multiple additional types of data. Interviews, as detailed in the previous section three, are a
central data resource in the research process, as the students’ statements can ideally confirm data from
observation, add additional perspectives and/or suggest other foci for the ethnographer.
4.2.2 Tracing knowledge: cognitive methods of ethnography
While being engaged in participant observation ethnography can integrate a wide range of additional
methods to gain insights into their interlocutors’ practices. In the presented desideratum to find out
more about knowledge dynamics within PDPs, the research topic demands for methodological means
to elicit students’ ideas on their knowledge development and decision-making processes. An approach
to such complex and abstract research topics are cognitive methods of data collection as employed in
qualitative social research such as free lists, pile sorts or mental maps (Antweiler, 2008; Ball and
Ormerod, 2000). By asking the students for example to list and group different activities associated
with information handling and inquiring on their underlying strategies associated with that task, the
construction of knowledge as an abstract concept becomes more tangible. This input delineates a
relevant factor for data triangulation to verify in how far they coincide with statements from interviews
and effectively observed behaviors and where deviations can be seen. Due to the exploratory nature of
ethnography, findings from such a methodological approach depict a solidly informed basis to build a
framework of hypotheses for subsequent verification by wider-populated survey-based research.
5 CONCLUSION AND RESEARCH DESIGN SUGGESTION
As the case study on the initial experiment has shown, the applied approach to investigate knowledge
dynamics in SGPDPs encountered a number of issues, both in relation to the abstract nature of the
research topic as well as in relation to the methodological set up as student experiment. Based on the
critical reflection of the student experiment’s research design, these issues can be addressed with
social research methods and the research questions formulated at the beginning can be answered as
follows:
RQ1: The critical discussion of the employed interview method (semi-structured interviews)
revealed, that in order to investigate knowledge dynamics as complex social processes,
methodical approaches should be as open as possible in the early phases of the case study.
Qualitative interview strategies (such as in-depth interviews, narrative techniques or focus
groups) or ethnographic observations are necessary, which allow exploring work processes first
open, without limiting the student’s responses or the researcher’s interpretations by standardized
instruments.
RQ2: By using qualitative interviews approaches or ethnographic observations, as many as
possible aspects of the complex construction of knowledge in development processes can first be
identified and then, as a second step in the research process, hypotheses can be formulated and
verified by precisely posed closed questions. Therefore, it is a future promising approach to use
exploratory sequential mixed methods designs, which mix e.g. qualitative interviews or
ethnographies with structured interviews.
RQ3: The potential opportunities of methods of social research for analyzing knowledge dynamics
in student experiments are to generate access to previously ignored semantic context and covert
social dynamics, which remained hidden by standardized research designs. Especially ethnographic
observations make it possible to identify non-articulated or reflected aspects of knowledge
dynamics. But also qualitative interview strategies, which avoid distortion or restriction of the
student`s possibility to provide information’s, can lead to new findings. Further on, more flexible
research designs can be used to respond more effectively to problems arising during the research
process, such as insufficient sampling or data collection, and thus increase the validity of results.
However, the discussed qualitative or mixed methods approaches require high personnel and time
effort, which limits their use and has to be considered in future experiments. Another general
problem of student experiments arise from the artificiality of the experimental setting, as it became
apparent in the discussed case study in the assignment of roles: While the project teams’ structure
included the assignment of formal roles to individual student members, the findings suggested that
these roles were not played as consistently as one would expect in industry settings. On the other
hand, are several strong similarities between student PDPs and their counterparts in the industry, as
2504

ICED19
in both scenarios the pressure to deliver a defined output at the end of the assigned timeframe is
fundamentally existent. The complexities of interactions required to reach this goal are also highly
comparable and hence student projects in principle delineate an emulation of the “real world”
sufficiently close to research knowledge dynamics in such a setting.
The results of the critical analysis of the case study indicate how the research design, the setting and
data collection instruments can be improved by integrating of methods of social research and
ethnography (see Figure 3). By applying and testing the proposed methods in further evaluations of
student experiments, an interdisciplinary method toolbox can be established, which can be used to
approach research on knowledge dynamics in product development. Furthermore, such a toolbox can
be adopted for experiments and studies in the broader field of engineering research.
Figure 3. Research design suggestion to improve the validity of results from student
evaluations by integrating methods of social research
AKNOWLEDGEMENTS
This research was funded by the German Research Foundation (DFG, STA 1112/15-1). We thank all
the student participants for their cooperation and patience.
REFERENCES
Ahmed, S. and Wallace, K. M. (2004), “Identifying and supporting the knowledge needs of novice designers
within the aerospace industry’, Journal of Engineering Design, vol. 15, no. 5, pp. 475–492.
Antweiler, C. (2008), “Kognitive Methoden”. In: Beer, B. (Ed.), Methoden ethnologischer Feldforschung,
Reimer, Berlin, pp. 233–254.
Ball, L., and Ormerod, T. (2000), “Putting Ethnography to Work: The Case for a Cognitive Ethnography of
Design.”, International Journal of Human-Computer Studies, Vol. 53 No. 1, pp. 147–168.
https://doi.org/10.1006/ijhc.2000.0372.
Baur, N. (2014), “Comparing Societies and Cultures. Challenges of Cross-Cultural Survey Research as an
Approach to Spatial Analysis”, Historical Social Research (HSR), Vol. 39 No. 2, pp. 257–291.
Berggreen, L. and Kampf, C. (2016), “Stage-gate Project Management Processes as Professional
Communication Practice: Connecting Technical and Marketing Communication in New Product
Development’, IEEE ProComm 2016: International Professional Communication Conference: Austin,
Texas, October 2-5. Piscataway, NJ, IEEE.
Bernard, H. (2017), “Research methods in anthropology: Qualitative and quantitative approaches”, Rowman &
Littlefield, Lanham.
Butlewski, M., Misztal, A., and Belu, N. (2016), “An Analysis of the Benefits of Ethnography Design Methods
for Product Modelling.”, IOP Conference Series: Materials Science and Engineering, Vol. 145 No. 4,
paper 042023. https://doi.org/10.1088/1757-899X/145/4/042023.
Brymann, A. (2016), “Social Research Methods”. Oxford University Press, Oxford.
Byrne, D. (2009), “Complex Realist Configurational Approaches to Cases: A Radical Synthesis. In: Byrne, D.;
Ragin, C. (Eds.): Case-Bases Methods. Sage, London, pp. 101-112.
Creswell, J. and Poth, C. (2017), “Qualitative Inquiry and Research Design: Choosing Among Five Approaches,
Sage, London.
Davenport, T., Prusak, L. and Höhlein, H. (1999), “Wenn Ihr Unternehmen wüßte, was es alles weiß: Das
Praxishandbuch zum Wissensmanagement”, 2nd ed, Verlag Moderne Industrie, Landsberg/Lech.
2505

ICED19
Emerson, R., Fretz, R. and Shaw, Linda (2011), “Writing ethnographic fieldnotes (2nd ed.), University of
Chicago Press, Chicago.
Feufel, M. and Stahl S. (2012), “What do Web-Use Skill Differences Imply for Online Health Information
Searches?”, Journal of Medical Internet Research Vol. 14 No. 3, e87. DOI: 10.2196/jmir.2051.
Flick, Uwe (2009), “Qualitative Sozialforschung”, Eine Einführung
Flick, Uwe (2011), “Das Episodische Interview”, In: Oelerich, Gertrud; Otto, Hans-Uwe (Hrsg.): Empirische
Forschung und Soziale Arbeit, Wiesbaden, (273–280).
Gellner, D. and Hirsch, E. (Eds.) (2001), “Inside organizations. anthropologists at work, Berg, Oxford.
Holstein, J./Gubrium, F. (2016), “Narrative Practice and the Active Interview. In: Silvermann, D. (eds.):
Qualitative Research. Sage, London, pp. 83–100.
Höst, M., Regnell, B. and Wohlin, C. (2000), “Using Students as Subjects‐-A Comparative Study of Students
and Professionals in Lead-Time Impact Assessment’, Empirical Software Engineering, Vol. 5 No. 3, pp.
201–214.
Fetterman, D. (1998), “Ethnography”, In: Bickman, L. and Rog, D. (Ed.), Handbook of Applied Social Research
Methods, Sage, London, pp. 473–504.
Klein G., Calderwood R. and MacGregor, D. (1989), “Critical decision method for eliciting knowledge”, IEEE
Transactions on Systems, Man, and Cybernetics, Vol. 19 No. 3, pp.462–472. DOI: 10.1109/21.31053.
Lee, S. and Yan, J. (2016), “The impact of 3D CAD interfaces on user ideation: A comparative analysis using
SketchUp and Silhouette Modeler”, Design Studies, 44, pp. 52–73.
Madden, R. (2010), “Being ethnographic. a guide to the theory and practice of ethnography, Sage, Los Angeles.
Märten, A., Jenek, J. F. W., Wang, W. M., Fleck, C., Meyer, H., Stark, R. and Ammon, S. (2018), “Conceptual
Framework for Analysing Knowledge Dynamics in Engineering Science”, Proceedings of the DESIGN
2018 15th International Design Conference, May, 21-24, 2018, Faculty of Mechanical Engineering and
Naval Architecture, University of Zagreb, Croatia; The Design Society, Glasgow, UK, pp. 1631–1642.
Miller, J. and Glassner, B. (2016), “The `Inside`and the `Outside`: Finding Realisties in Interviews. In:
Silvermann, D. (eds.): Qualitative Research. Sage, London, pp. 51-66.
Mohedas, I., Sarvestani, A., Daly, S. and Sienko, K. (2015), “Applying Design Ethnograhy to Product
Evaluation: A Case Example of a Medical Device in a low-resource setting”, Proceedings of the 20th
International Conference on Engineering Design (ICED15), Vol. nn: Title of Volume, Milan, Italy, 27.-
30.07.2015.
Moser, C., and Kalton, G. (2017), “Survey Methods in Social Investigation”, Routledge, London.
Robson, C. and McCartan, K. (2016), “Real World Research”, John Wiley & Sons, Chichester.
Nunes, F. and Fitzpatrick, G. (2018), “Understanding the Mundane Nature of Self-care: Ethnographic Accounts
of People Living with Parkinson’s”, Proceedings of the 2018 CHI Conference on Human Factors in
Computing Systems (CHI’18), p. 402. https://doi.org/10.1145/3173574.3173976
Robinson, A. (2010), “Work sampling. Methodological advances and new applications. In: Human Factors and
Ergonomics in Manufacturing & Service Industires Vol. 20 No. 1, pp. 42–60.
Schoonenboom, J. and Johnson, B. (2017), “How to Construct a Mixed Methods Research Design”, Kölner
Zeitschrift für Soziologie und Sozialpsychologie, Vol. 69 No. 2, pp. 107-131.
Self, J. (2017), “Resolving Wicked Problems: Appositional Reasoning and Sketch Representation”, The Design
Journal, Vol. 20 No. 3, pp. 313–331, https://dx.doi/org/10.1080/14606925.2017.1301070
Suchman, L. (1985), “Plans and Situated Actions: The problem of human-machine communication”, Xerox, Palo
Alto.
Sudman, S., Bradburn, N. and Schwarz, N. (1996), “Thinking About Answers. The Application of Cognitive
Processes to Survey Methodology”, Jossey-Bass Publishers, San Francisco.
Thierbach, C. and Lorenz, A. (2014), “Exploring the Orientation in Space. Mixing Focused Ethnography and
Surveys in Social Experiment”, Historical Social Research Vol. 39 No. 2, pp.137–166.
https://dx.doi/org/10.12759/hsr.39.2014.2.137-166 Vinck, D. (ed.) (2003), Everyday Engineering: An
Ethnography of Design and Innovation. MIT Press, Cambridge, Massachusetts.
Wang, W. M., Lünnemann, P., Preidel, M. and Stark R. (2017), “Wissen in Produktentwicklungsprozessen – ein
Aktivitätenbasierter Analyseansatz”, In: Brökel, K., Grote, K.-H., Stelzer, R., Rieg F., Feldhusen, J.,
Müller, N. and Köhler, P. (eds), Vo. 15. Gemeinsames Kolloquium Konstruktionstechnik: Interdisziplinäre
Produktentwicklung, Essen, Universität Duisburg-Essen, pp. 183–192.
Wilkinson, Sue (2016), “Analysing Focus Group Data. In: Silvermann, D. (eds.): Qualitative Research. Sage,
London, pp. 83–100.
Yin, R. (2014), “Case Study Research”, Design and Methods. Sage, Thousand Oaks/London/New Delhi.
Yock, P. et al. (2015), “ Biodesign: The process of innovating medical technologies 2nd ed., United Kingdom:
Cambridge University Press.
2506