Conference PaperPDF Available

Approaching Knowledge Dynamics Across the Product Development Process with Methods of Social Research



Knowledge is a crucial factor in state-of-the-art product development. It is often provided by stakeholders from divers disciplinary and individual backgrounds and has to be integrated to create competitive products. Still, it is not fully understood, how knowledge is generated, transformed, transferred and integrated in complex product development processes. To investigate the dynamic interrelations between involved stakeholders, applied knowledge types and related artefacts, researchers at the TU Berlin conducted and evaluated a student experiment to study basic phenomena of development projects. In relation to research methods and instruments applied in this experiment, various improvement opportunities were identified. In this paper, the experimental setting and its results are critically analysed from a social science perspective in order to generate improved research design. Based on the results of this analysis, a first set of methods and instruments from social sciences are identified that can be applied in further experiments. The goal is to develop a methodological toolbox that can be used to approach research on knowledge dynamics in product development.
Cite this article: Wang, W.M., Mörike, F., Hergesell, J., Baur, N., Feufel, M., Stark, R. (2019) ‘Approaching Knowledge
Dynamics Across the Product Development Process with Methods of Social Research’, in Proceedings of the 22nd
International Conference on Engineering Design (ICED19), Delft, The Netherlands, 5-8 August 2019. DOI:10.1017/
Wang, Wei Min; Mörike, Frauke; Hergesell, Jannis; Baur, Nina; Feufel, Markus; Stark, Rainer
Technische Universität Berlin
Knowledge is a crucial factor in state-of-the-art product development. It is often provided by
stakeholders from divers disciplinary and individual backgrounds and has to be integrated to create
competitive products. Still, it is not fully understood, how knowledge is generated, transformed,
transferred and integrated in complex product development processes. To investigate the dynamic
interrelations between involved stakeholders, applied knowledge types and related artefacts, researchers
at the TU Berlin conducted and evaluated a student experiment to study basic phenomena of
development projects. In relation to research methods and instruments applied in this experiment,
various improvement opportunities were identified. In this paper, the experimental setting and its results
are critically analysed from a social science perspective in order to generate improved research design.
Based on the results of this analysis, a first set of methods and instruments from social sciences are
identified that can be applied in further experiments. The goal is to develop a methodological toolbox
that can be used to approach research on knowledge dynamics in product development.
Keywords: Design theory, Organisation of product development, Knowledge management, Knowledge
dynamics in product development, Evaluation with students
Wang, Wei Min
Technische Universität Berlin
Industrial Information Technology
The development of modern products increasingly demands the networking of different disciplines
due to the convergence of mechanics, electrics/electronics, software and services. To create innovative
and successful products, knowledge from stakeholders with heterogeneous backgrounds must be
integrated. In industrial practice, the control and management of such product development processes
(PDPs) requires the continuous exchange of domain-specific and technical as well as organizational
and process-related knowledge (Berggreen and Kampf 2016). As knowledge per se arises in
individuals or groups of persons and is applied by those, the daily development activities in teams
directed to the achievement of objectives leads to a constant and situational increase and negotiation of
knowledge. This knowledge can be based on data and information and represents a variable mixture of
experiences, individual beliefs and contextual information. (Davenport et al., 1999) While engineering
sciences have developed a rich body of insights on how designers process knowledge individually in
the course of the PDP (e.g. Ahmed and Wallace, 2004; Mohedas et al. 2015), the understanding of
knowledge dynamics within PDP teams across the entire PDP delineates a gap, especially with regard
to which knowledge is required at which point in the development process and by whom it must be
In the DFG-funded (German Research Foundation, DFG) project Knowledge in stage-gate driven
product development processes researchers at the Technical University Berlin (TUB) sought to gain a
deeper understanding about knowledge-related activities in PDPs and how they vary along the process.
The aim is to develop a model to identify the knowledge related to specific development phases and
their dynamic interactions with stakeholders, activities and artefacts in the course of a stage-gate
driven PDP (SGPDP). A first part of the project was a student experiment conducted as a project on
product development to investigate basic phenomena of knowledge dynamics in SGPDP. As the
insights into knowledge dynamics from this initial student experiment turned out to be unsatisfactory,
this article presents a critical evaluation of the experiment in relation to the research methods applied
as well as lessons learned for an improved research design. A new dimension on approaching the issue
will be introduced by including the analysis of the experiment from the methodological perspective of
human factors research and sociology to ideate how the complex topic of knowledge dynamics in
SGPDPs can be approached with interdisciplinary knowledge. In particular, we seek to answer the
following research questions (RQ):
RQ1: Which methods of social research are suitable for assessing knowledge dynamics in
RQ2: How can these methods be adopted and mixed to improve the quality of research findings
on knowledge dynamics?
RQ3: What are the opportunities and potential limitations of methods of social research in this
We like to emphasize, that the goal of this paper is not evaluation and discussion of the results of the
initial experiment, but to contribute to an improved research of knowledge dynamics by
conceptualizing an interdisciplinary research design. To enable full transparency, however, it is
necessary to present the original research design of the initial experiment as a case study in the
following section in order to consequently discuss possible error sources from the perspective of the
social research and answer the research questions.
2.1 Original goal of the experiment
As mentioned above, the initial experiment, conducted as a student project, was part of the DFG-
funded research project. The overall goal was to identify an initial set of knowledge related activities
in PDPs by evaluating an exemplary development project. The results were intended as a basis to
create a first draft of a generic model to describe relationships between development activities that
demand the processing of knowledge, the knowledge bearers involved in those activities (people and
IT-systems), the related types of knowledge as well as the artefacts representing the knowledge.
2.2 Research design of the initial experiment
The experiment was integrated into a masters course at the Department for Industrial Information
Technology (IIT) of the TUB. Two student teams consisting of seven persons each were tasked with
the development of a Product-Service System (PSS) concept for a single-axle tractor (see Figure 1).
The student groups role-played to be start-ups trying to convince an investor of their product. To
simulate organizational structures in actual industrial companies, each student was instructed to take
on one of the following project-specific roles with respective responsibilities: Project Manager (PM):
represents the team to external parties and is responsible for internal project planning and controlling;
Product Owner (PO): is responsible for the overall product view; the product should represent a good
overall package while adhering to the specifications; Sustainability Expert (SE): responsible for the
consideration (not implementation) of sustainability aspects; advises PO on sustainability aspects;
Product Developers (PD): responsible for technical feasibility.
The progress of the project was examined via three design reviews (DR) according to predefined
deliverables. During the DRs, the students had to present their results to a steering board consisting of
researchers from the IIT department. For DR1, students hat to develop and present three concepts for a
PSS. For DR2, they had to select and elaborate one of the three concepts. Among other things, this
included the development of an add-on module, a requirements list and a service blue-print for the
PSS concept. For DR3, they had to present finalized models (CAD models, system models, business
canvas etc.) of the PSS as well as marketing materials. At the end of the course, they had to deliver a
closing report summarizing the entire project.
Figure 1. Task for the student project - Development of a PSS concept including an add-on
2.3 Evaluation of the student project using semi-structured interviews und
documentary analysis
At the beginning of the course, the students were introduced to the research project and were informed
that they would participate in an experiment. The main evaluation method were semi-structured
interviews conducted as group interviews. This method was chosen in order to enable the students to
report on their activities in a narrative way, but also to steer the answers thematically through an
interview guide. (Flick, 2009, 2011) The method of individual interviews was discarded due to time
limitations of the course. In order to keep the interviews manageable, each of the student teams was
further split into two sub groups, one comprising roles with managerial tasks (i.e. PM, PO, SE) and the
other the PDs. It was assumed that the answers within these role-groups would be more homogeneous
due to similarities in the tasks and therefore facilitate the evaluation of the results. For the interviews, an
interview guide comprising 29 questions was developed. This interview guide was based on a conceptual
framework for analyzing knowledge dynamics in engineering science developed at the TUB. The
questions within the interview guide were organized along the key concepts of this framework:
knowledge acquisition, knowledge transformation, knowledge generation, knowledge transfer,
knowledge coordination and knowledge integration (cf. Märten et al., 2018). The interview guide was
pre-tested and refined with researchers of the IIT department. During the experiment, a total of three
interviews were conducted with each sub group, arranged one week before the respective DR. The
timing of the interviews was chosen as it was assumed that students would have to make all product
decisions in advance in order to present the final results in the DRs. The researchers who conducted the
interviews took part in the DRs, but were not involved in the rating process of the project results.
In addition to the interviews, both student teams were advised to save all project-related data (i.a.
CAD files, sketches, concept documentations, meeting minutes, presentations, reports, calculation
results) into a university-internal cloud-based shared folder (cf. Robinson, 2010). The students were
also instructed to follow a folder structure and a naming pattern, which allowed for the tracing of
version history, the respective editor and the date of a change. This facilitated the later analysis of
change logs and work patterns within the teams on the documents. The parallel observation of data
development in the shared folders was intended to provide information on which artefacts were
created in which project phase and to which tasks and roles they can be assigned. By triangulating the
interview results with the changes in the data folders in the corresponding period, the initial aim was to
establish a connection between the activities carried out and the artefacts involved.
The results from the interviews and documentary analysis were quantified, organized in a database and
statistically analyzed using R. Comparative analyses of response frequencies were performed with
regard to the following aspects:
Timing: Differences in knowledge activities in the course of the project, e.g. from where and how
is knowledge acquired and transformed in specific phases of the project?
Role: Role-specific differences in knowledge processing between the managerial roles and PDs,
e.g. are managerial roles looking for different kind of knowledge than PDs?
Group: Differences in knowledge processing between the two development teams, e.g. was one
team using different knowledge sources than the other?
Bivariate analyses were also carried out, e.g. to examine differences between roles at a particular
stage. The evaluation of the results allowed for first insights about how development activities are
organized and carried out on an operative team level. Based on these insights, a first theoretical
framework could be derived that can be used to describe knowledge processing activities in PDPs
grounding on stakeholders, activities and artefacts involved in the PDP (Wang et al., 2017).
However, the evaluation of the experiment results also revealed shortcomings regarding the overall
research design, sampling, data collection instruments and data quality that limited the validity of the
results in several respects. Therefore, the present student experiment is critically analyzed from a social
science perspective to identify improvement approaches. It is necessary to reflect these methodological
questions to improve the data quality and to achieve a higher level of detail of questioning in future
evaluations. At the beginning of such an interdisciplinary cooperation, methodological problems should
first be identified based on the social research`s experience. Subsequently, some suggestions for
improvement can be formulated and alternative methodical approaches can be named.
3.1 Critical analysis of the research design
A well-known problem concerning the overall research design is artificiality (Thierbach and Lorenz,
2014) because the circumstances of product development in a student course deviates from a real
industrial environment. Among other things, there was no organizational or personal knowledge e.g.
on product history, the product and the target market. In addition, there was neither an organizational
structure with defined decision-making authorities nor a specified development process. The student
project aimed at simulating a rudimental organizational structure with rules for the allocation of tasks
and competencies by assigning roles with defined tasks and responsibilities. The artificial
assignment of roles turned out to be a problem, as the students were not socialized into these roles
and therefore could not simulate them according to the projects goal. However, evaluation results
indicated that the students did not stick to their chosen roles during the course of the project (a, Figure
2). Thus, it could not be clearly determined whether there is a difference in knowledge processing
between roles.
The scenario of the experiment, in which the student teams represent start-ups that are seeking investor
money, plausibilises the framework conditions of the development project, but also results in decision
situations no longer coinciding with the DRs (b, Figure 2). In real life, decisions are actually made
during the review meetings (e.g. selecting a concept) and reviewers should play an active role in the
decision-making process, e.g. as a management board that determines the direction of development
(e.g. decides on alternative solutions based on the presentation) and approval for the next development
phase. Whereas in the evaluated student project, the students made product-related decisions before
the meeting (e.g. performing utility analyses) and merely presented and explained their decisions. The
reviewers took on the role of external evaluators instead of being part of the development project and
were not involved in the actual process of making product-related decision. Consequently, the
observation of the DR meetings could not give any hints about the actual decision making mechanisms
within the team.
3.2 Critical analysis of sampling
Due to the selected course, participants were rather homogenous: All of them were male students,
most of them being white middle-class of a similar age, mostly studying mechanical engineering in a
similar semester in a masters degree. All of them had prior experience in product development and
with their role-specific tasks, gained either at university or in internships and student part-time jobs.
Only for the role of the sustainability expert did the participants hardly have any previous knowledge.
In order to generalize results, sampling need to be either random samples (Baur, 2014) or purposeful
samples (Creswell and Poth 2017, pp. 156-159). However, in our case study, sampling was arbitrary,
resulting in a biased sample (c, Figure 2) (; Byrne 2009; Yin 2014). Therefore, generalization beyond
the single case is not possible.
3.3 Critical analysis of data collection instruments
The data collection instruments used in the evaluation need to be improved in order to be able to
record dynamic aspects of the knowledge processing activities as extensively as possible (d, Figure 2).
In the course of the evaluation, it was revealed that semi-structured group interviews did not allow for
the necessary level of detail of questioning. The acquisition of information and knowledge is also
intuitively based on ones own experience. To allocate applied or activated knowledge for certain
activities retrospectively, in-depth interviews or narrative techniques are more suitable. For a
replication of the evaluation in an industrial context, an interview method must be chosen that allows
in-depth questions to be put to a participant. It should also be asked critically whether interviews are
the most useful method for an exploratory analysis of knowledge dynamics. These issues could be
processed in an interdisciplinary cooperation with social research: The retrieval of experiences, e.g.,
often occurs unconsciously in the problem-solving process. A test interview with a member of the
scientific staff revealed that developers restrict the solution space according to their own experience or
make a preselection of solution principles without explicitly going through a formal selection process.
Hence, it could be useful to observe the knowledge processing situations, e.g. by participating in such
situations and using ethnographic methods.
3.4 Critical analysis of data quality
Concerning data quality of the semi-structured interviews, a first criticism is the interview situation (e,
Figure 2). In general, response behavior should be influenced as little as possible. Otherwise, a
distortion of the response behavior will lead to data bias. An unbiased interview setting could be
accomplished by interviewing respondents in a situation where they do not need to fear negative
consequences of their responses (e.g. bad marks). Although the interviewer in the present evaluation
explicitly announced that he would not be involved in the rating process of the course, his affiliation
IIT could have influenced the response behavior. The respondents were also interviewed in groups, so
that their responses could be biased by the social desirability of the group. For example, students may
seek to conceal within-group conflicts in order to protect individual students or aim at presenting their
group as successful. In addition, the interviews need to be anonymized. In the best case, no
conclusions can be drawn to the identity of the respondent, which makes it possible to give
unpleasant answers.
Furthermore, semi-structured interviews seek a compromise between open-ended narrative interviews
and surveys. However, in the case of this evaluation, this kind of compromise turned out to be more
detrimental: Surveys (Moser/Kalton 2017) usually have the advantage of being to process a higher
number of cases, fast data preparation and easy comparability. Due to the open-ended answers in the
semi-structured interviews, these advantages did not apply in the evaluation - instead, researchers had
to strongly interpret data, answers were not always comparable, and data preparation was rather time-
consuming. In contrast, both surveys and the semi-structured interviews used in the evaluation can
only achieve reliable results, if the relevant issues are already known in advance and can be
operationalized into precise, one-dimensional closed questions with a given set of equally precise and
exhaustive possible answers. However, the student evaluation pursued here had an exploratory
research interest. This means that in such an early phase of the research process, too little is known
about the relevant aspects of the student evaluation to formulate precisely closed questions.
Even if more about the topic was known, knowledge dynamics represent a complex social process.
This implies selecting a data collection method that allows access to semantic content and social
dynamics. In general, the best methodical approach for such objects are qualitative research designs
(Creswell and Poth, 2017), as standardized instruments limit respondents ability to answers in detail
and in their own words. As a result, respondents are unable to fully introduce their experiences into
their responses. As well, it is hard for the researchers to figure out new, unexpected aspects of their
research topic from respondents answers to closed questions. Therefore, the research process of the
student evaluation does not fully exploit the students potential to provide their knowledge. All in all,
the problems of the evaluation could also be a result of a lack of precision during the
operationalization of the questionnaire and the students very limited answer options (f, Figure 2).
Figure 2. Allocation of identified improvement potentials in the evaluation of the student experiment
The following discussion of error sources should not be understood as deterministic or strict
instruction for future experiments. Rather, it is a matter of showing suggestions for improvement from
the diverse range of social research approaches for the development of research designs, which of
course have to be adapted in individual cases for future student experiments.
4.1 Exploratory sequential mixed methods
For future evaluations, an exploratory sequential mixed methods design (Schoonenboom and Johnson,
2017) would be more suitable: (a) One would start with qualitative interviews. This procedure includes
inductive strategies in the research process and has the advantage that the respondents are very free in
their response behavior and so also previously unrecognized aspects of knowledge dynamics can be
included in the interview (Miller/Glassner 2016; Holstein/Gubrium 2016). Disadvantages of qualitative
interviews are a difficult comparability of the different interviews as well as an increased data analysis
effort. A less time-consuming qualitative methods are focus groups (Wilkinson 2016). In doing so, the
entire group of students would be interviewed together by asking them to discuss particular aspects of
their work. This procedure has the advantage that in addition to the concrete information that enables the
creation of a questionnaire, also negotiation processes and controversially discussed, possibly
problematic conflictive phases of the development process become accessible for the researchers. A
disadvantage of this approach is a greater data analysis effort and a possibly distorted response behavior,
as the group members did not want to speak openly before the other participants.
(b) After one has analyzed the qualitative data and developed first hypotheses, these hypotheses can be
tested in a survey - and even surveys would allow for integrating open-ended and standardized elements
can be combined. For instance, so-called open questions can be asked in a first short questionnaire
(Brymann 2016, 248). This means that respondents are stimulated by short but broad question and then
they can answer in their own words without being influenced by a suggestive posing of questions. These
answers can then be evaluated by the researchers. This approach has the advantage that the individual
answers of the respondents are well comparable with each other and the data analysis effort remains
rather low.
4.2 Ethnography in PDPs and knowledge as field of study
The previous section illustrated how the approach to gain insights into the role(s) knowledge plays in
PDPs through interviews could be improved within the logic of collecting interview data. As already
mentioned above, it would be also sensible to mix interviews with observation-based data collection
(Thierbach and Lorenz, 2014) in order to improve data quality. In this section, the methodological
toolbox is extended to include modes of in-situ data collection, more specifically ethnography.
Ethnography (Silverman, 2016) is a method originating from the realms of social and cultural sciences
with the approach to undertake research by systematically observing and participating (to a greater or
lesser degree) in the lives of the studied groups. The idea is to gain insights into practices through
presence in the same social setting, by walking a mile in the shoes of others(Madden, 2006).
Literally translated from Greek, ethnography means writing about alien groups of people,
highlighting its (stereo-)typical association with anthropological research in remote places (Gellner
and Hirsch, 2001). The application of ethnographic methods in design contexts garners a decade-long
tradition since its origins in the 1980s with the observations of ethnographer Lucy Suchman on how
people interacted with copy machines (Suchman, 1985). To date, design ethnography is an established
field in human-computer-interaction research (Nunez and Fitzpatrick, 2018), and increasingly applied
in PDPs (Butlewski et al., 2016). In engineering sciences, the use of ethnography is primarily focusing
on providing insights on users practices (Vinck, 2003), requirements and/or product evaluation
(Mohedas et al., 2015). In the suggested revised research design to approach knowledge dynamics,
ethnography is applied to gain insights into the PDP itself, which delineates a clear gap in current
research. This additional perspective should account for the situation that evaluations of knowledge
dynamics in student experiments seem to be coined by a number of perceived challenges, such as the
uncontrollable (and non-reproducible) dynamics of semester-length student work groups, the
inevitably hierarchical frame of interacting students and faculty staff, students heterogeneous
backgrounds on organizational knowledge about PDPs acquired in courses at university or at various
industry organizations, and, apparently numerous communication iterations within and outside course
room contexts. At the end of the first student experiment, hence, the decision-making mechanisms
within the teams and their knowledge base remained opaque to the research team. Another aspect is
that knowledge as a research topic is a concept which students, and most likely practitioners,
struggle to denominate and label when directly asked about, especially in the context of their
unreflexively everyday study or work life (Klein et al., 1989). The aim is to approach all those
seemingly challenging factors from a different angle and to turn them into a strength of the research
project. While such a setting is for the above reasons difficult to put into an evaluational research
context, the complexities are akin to those encountered in industry-based PDPs and therefore
worthwhile to include into the research design rather than to attempt to rule complexity out.
4.2.1 Participant observation: in-situ data collection
Participant observation stands at the center of ethnographic research and describes the seemingly
oxymoronic approach where a researcher takes part in the daily activities, interactions, and events of a
group of people as one of the means of learning the explicit and tacit aspects of their routines and
practices (Robson and McCartan 2016). For research on knowledge acquisition and negotiation
processes in student PDPs, the in-situ observation mode of ethnography depicts a promising method, as
it allows for a maximum level of flexibility to adapt data collection to the dynamics of project team
interactions. The ethnographer can be amongst the project teams to observe how tasks are distributed
amongst the team members and to witness role negotiations or decision-making processes in preparation
for milestone submissions. As long as the student team members accept the presence of the ethnographer
and are willing to provide insights into their ways of working as a team along the project cycle, the time
and context of data collection is independent from the actual course room setting. The communication
dynamics within the PDP teams might be even more active and less controlled by impression
management the farther away from pre-defined interaction formats such as milestone presentations and
course room locations. In short, participant observation puts you were the action is and lets you collect
data (Bernard, 2017), and precisely the documentation of this wide array of (inter-)action depicts the
backbone of ethnographic research. To achieve this, ethnographers take notes on relevant interactions
throughout the observation phase and consolidate these jottings shortly after or at the end of the day into
a set of detailed field notes (Emerson et al., 2011). Over the course of different observation phases, the
field notes allow for several layers of insights into the knowledge dynamics at play in stage-gate-driven
PDPs. To ensure rich sets of data for analysis and triangulation, ethnography can encompass the
collection of multiple additional types of data. Interviews, as detailed in the previous section three, are a
central data resource in the research process, as the students statements can ideally confirm data from
observation, add additional perspectives and/or suggest other foci for the ethnographer.
4.2.2 Tracing knowledge: cognitive methods of ethnography
While being engaged in participant observation ethnography can integrate a wide range of additional
methods to gain insights into their interlocutors practices. In the presented desideratum to find out
more about knowledge dynamics within PDPs, the research topic demands for methodological means
to elicit students ideas on their knowledge development and decision-making processes. An approach
to such complex and abstract research topics are cognitive methods of data collection as employed in
qualitative social research such as free lists, pile sorts or mental maps (Antweiler, 2008; Ball and
Ormerod, 2000). By asking the students for example to list and group different activities associated
with information handling and inquiring on their underlying strategies associated with that task, the
construction of knowledge as an abstract concept becomes more tangible. This input delineates a
relevant factor for data triangulation to verify in how far they coincide with statements from interviews
and effectively observed behaviors and where deviations can be seen. Due to the exploratory nature of
ethnography, findings from such a methodological approach depict a solidly informed basis to build a
framework of hypotheses for subsequent verification by wider-populated survey-based research.
As the case study on the initial experiment has shown, the applied approach to investigate knowledge
dynamics in SGPDPs encountered a number of issues, both in relation to the abstract nature of the
research topic as well as in relation to the methodological set up as student experiment. Based on the
critical reflection of the student experiments research design, these issues can be addressed with
social research methods and the research questions formulated at the beginning can be answered as
RQ1: The critical discussion of the employed interview method (semi-structured interviews)
revealed, that in order to investigate knowledge dynamics as complex social processes,
methodical approaches should be as open as possible in the early phases of the case study.
Qualitative interview strategies (such as in-depth interviews, narrative techniques or focus
groups) or ethnographic observations are necessary, which allow exploring work processes first
open, without limiting the students responses or the researchers interpretations by standardized
RQ2: By using qualitative interviews approaches or ethnographic observations, as many as
possible aspects of the complex construction of knowledge in development processes can first be
identified and then, as a second step in the research process, hypotheses can be formulated and
verified by precisely posed closed questions. Therefore, it is a future promising approach to use
exploratory sequential mixed methods designs, which mix e.g. qualitative interviews or
ethnographies with structured interviews.
RQ3: The potential opportunities of methods of social research for analyzing knowledge dynamics
in student experiments are to generate access to previously ignored semantic context and covert
social dynamics, which remained hidden by standardized research designs. Especially ethnographic
observations make it possible to identify non-articulated or reflected aspects of knowledge
dynamics. But also qualitative interview strategies, which avoid distortion or restriction of the
student`s possibility to provide informations, can lead to new findings. Further on, more flexible
research designs can be used to respond more effectively to problems arising during the research
process, such as insufficient sampling or data collection, and thus increase the validity of results.
However, the discussed qualitative or mixed methods approaches require high personnel and time
effort, which limits their use and has to be considered in future experiments. Another general
problem of student experiments arise from the artificiality of the experimental setting, as it became
apparent in the discussed case study in the assignment of roles: While the project teams structure
included the assignment of formal roles to individual student members, the findings suggested that
these roles were not played as consistently as one would expect in industry settings. On the other
hand, are several strong similarities between student PDPs and their counterparts in the industry, as
in both scenarios the pressure to deliver a defined output at the end of the assigned timeframe is
fundamentally existent. The complexities of interactions required to reach this goal are also highly
comparable and hence student projects in principle delineate an emulation of the real world
sufficiently close to research knowledge dynamics in such a setting.
The results of the critical analysis of the case study indicate how the research design, the setting and
data collection instruments can be improved by integrating of methods of social research and
ethnography (see Figure 3). By applying and testing the proposed methods in further evaluations of
student experiments, an interdisciplinary method toolbox can be established, which can be used to
approach research on knowledge dynamics in product development. Furthermore, such a toolbox can
be adopted for experiments and studies in the broader field of engineering research.
Figure 3. Research design suggestion to improve the validity of results from student
evaluations by integrating methods of social research
This research was funded by the German Research Foundation (DFG, STA 1112/15-1). We thank all
the student participants for their cooperation and patience.
Ahmed, S. and Wallace, K. M. (2004), Identifying and supporting the knowledge needs of novice designers
within the aerospace industry, Journal of Engineering Design, vol. 15, no. 5, pp. 475492.
Antweiler, C. (2008), Kognitive Methoden. In: Beer, B. (Ed.), Methoden ethnologischer Feldforschung,
Reimer, Berlin, pp. 233254.
Ball, L., and Ormerod, T. (2000), Putting Ethnography to Work: The Case for a Cognitive Ethnography of
Design., International Journal of Human-Computer Studies, Vol. 53 No. 1, pp. 147168.
Baur, N. (2014), Comparing Societies and Cultures. Challenges of Cross-Cultural Survey Research as an
Approach to Spatial Analysis, Historical Social Research (HSR), Vol. 39 No. 2, pp. 257291.
Berggreen, L. and Kampf, C. (2016), Stage-gate Project Management Processes as Professional
Communication Practice: Connecting Technical and Marketing Communication in New Product
Development, IEEE ProComm 2016: International Professional Communication Conference: Austin,
Texas, October 2-5. Piscataway, NJ, IEEE.
Bernard, H. (2017), Research methods in anthropology: Qualitative and quantitative approaches, Rowman &
Littlefield, Lanham.
Butlewski, M., Misztal, A., and Belu, N. (2016), An Analysis of the Benefits of Ethnography Design Methods
for Product Modelling., IOP Conference Series: Materials Science and Engineering, Vol. 145 No. 4,
paper 042023.
Brymann, A. (2016), Social Research Methods. Oxford University Press, Oxford.
Byrne, D. (2009), Complex Realist Configurational Approaches to Cases: A Radical Synthesis. In: Byrne, D.;
Ragin, C. (Eds.): Case-Bases Methods. Sage, London, pp. 101-112.
Creswell, J. and Poth, C. (2017), Qualitative Inquiry and Research Design: Choosing Among Five Approaches,
Sage, London.
Davenport, T., Prusak, L. and Höhlein, H. (1999), Wenn Ihr Unternehmen wüßte, was es alles weiß: Das
Praxishandbuch zum Wissensmanagement, 2nd ed, Verlag Moderne Industrie, Landsberg/Lech.
Emerson, R., Fretz, R. and Shaw, Linda (2011), Writing ethnographic fieldnotes (2nd ed.), University of
Chicago Press, Chicago.
Feufel, M. and Stahl S. (2012), What do Web-Use Skill Differences Imply for Online Health Information
Searches?, Journal of Medical Internet Research Vol. 14 No. 3, e87. DOI: 10.2196/jmir.2051.
Flick, Uwe (2009), Qualitative Sozialforschung, Eine Einführung
Flick, Uwe (2011), Das Episodische Interview, In: Oelerich, Gertrud; Otto, Hans-Uwe (Hrsg.): Empirische
Forschung und Soziale Arbeit, Wiesbaden, (273280).
Gellner, D. and Hirsch, E. (Eds.) (2001), Inside organizations. anthropologists at work, Berg, Oxford.
Holstein, J./Gubrium, F. (2016), Narrative Practice and the Active Interview. In: Silvermann, D. (eds.):
Qualitative Research. Sage, London, pp. 83100.
Höst, M., Regnell, B. and Wohlin, C. (2000), Using Students as Subjects-A Comparative Study of Students
and Professionals in Lead-Time Impact Assessment, Empirical Software Engineering, Vol. 5 No. 3, pp.
Fetterman, D. (1998), Ethnography, In: Bickman, L. and Rog, D. (Ed.), Handbook of Applied Social Research
Methods, Sage, London, pp. 473504.
Klein G., Calderwood R. and MacGregor, D. (1989), Critical decision method for eliciting knowledge, IEEE
Transactions on Systems, Man, and Cybernetics, Vol. 19 No. 3, pp.462472. DOI: 10.1109/21.31053.
Lee, S. and Yan, J. (2016), The impact of 3D CAD interfaces on user ideation: A comparative analysis using
SketchUp and Silhouette Modeler, Design Studies, 44, pp. 5273.
Madden, R. (2010), Being ethnographic. a guide to the theory and practice of ethnography, Sage, Los Angeles.
Märten, A., Jenek, J. F. W., Wang, W. M., Fleck, C., Meyer, H., Stark, R. and Ammon, S. (2018), Conceptual
Framework for Analysing Knowledge Dynamics in Engineering Science, Proceedings of the DESIGN
2018 15th International Design Conference, May, 21-24, 2018, Faculty of Mechanical Engineering and
Naval Architecture, University of Zagreb, Croatia; The Design Society, Glasgow, UK, pp. 16311642.
Miller, J. and Glassner, B. (2016), The `Inside`and the `Outside`: Finding Realisties in Interviews. In:
Silvermann, D. (eds.): Qualitative Research. Sage, London, pp. 51-66.
Mohedas, I., Sarvestani, A., Daly, S. and Sienko, K. (2015), Applying Design Ethnograhy to Product
Evaluation: A Case Example of a Medical Device in a low-resource setting, Proceedings of the 20th
International Conference on Engineering Design (ICED15), Vol. nn: Title of Volume, Milan, Italy, 27.-
Moser, C., and Kalton, G. (2017), Survey Methods in Social Investigation, Routledge, London.
Robson, C. and McCartan, K. (2016), Real World Research, John Wiley & Sons, Chichester.
Nunes, F. and Fitzpatrick, G. (2018), Understanding the Mundane Nature of Self-care: Ethnographic Accounts
of People Living with Parkinsons, Proceedings of the 2018 CHI Conference on Human Factors in
Computing Systems (CHI18), p. 402.
Robinson, A. (2010), Work sampling. Methodological advances and new applications. In: Human Factors and
Ergonomics in Manufacturing & Service Industires Vol. 20 No. 1, pp. 4260.
Schoonenboom, J. and Johnson, B. (2017), How to Construct a Mixed Methods Research Design, Kölner
Zeitschrift für Soziologie und Sozialpsychologie, Vol. 69 No. 2, pp. 107-131.
Self, J. (2017), Resolving Wicked Problems: Appositional Reasoning and Sketch Representation, The Design
Journal, Vol. 20 No. 3, pp. 313331, https://dx.doi/org/10.1080/14606925.2017.1301070
Suchman, L. (1985), Plans and Situated Actions: The problem of human-machine communication, Xerox, Palo
Sudman, S., Bradburn, N. and Schwarz, N. (1996), Thinking About Answers. The Application of Cognitive
Processes to Survey Methodology, Jossey-Bass Publishers, San Francisco.
Thierbach, C. and Lorenz, A. (2014), Exploring the Orientation in Space. Mixing Focused Ethnography and
Surveys in Social Experiment, Historical Social Research Vol. 39 No. 2, pp.137166.
https://dx.doi/org/10.12759/hsr.39.2014.2.137-166 Vinck, D. (ed.) (2003), Everyday Engineering: An
Ethnography of Design and Innovation. MIT Press, Cambridge, Massachusetts.
Wang, W. M., Lünnemann, P., Preidel, M. and Stark R. (2017), Wissen in Produktentwicklungsprozessen ein
Aktivitätenbasierter Analyseansatz, In: Brökel, K., Grote, K.-H., Stelzer, R., Rieg F., Feldhusen, J.,
Müller, N. and Köhler, P. (eds), Vo. 15. Gemeinsames Kolloquium Konstruktionstechnik: Interdisziplinäre
Produktentwicklung, Essen, Universität Duisburg-Essen, pp. 183192.
Wilkinson, Sue (2016), Analysing Focus Group Data. In: Silvermann, D. (eds.): Qualitative Research. Sage,
London, pp. 83100.
Yin, R. (2014), Case Study Research, Design and Methods. Sage, Thousand Oaks/London/New Delhi.
Yock, P. et al. (2015), Biodesign: The process of innovating medical technologies 2nd ed., United Kingdom:
Cambridge University Press.
... To do so, they, for instance, use project-based learning (PBL). Based on previous studies and conceptual considerations about knowledge dynamics in engineering teams [1] we report results of an ethnographic study to gain deeper insights into the everyday knowledge dynamics of students in PBL courses that is not captured in formal design reviews and graded project deliverables. We have distilled the findings into an improved educational concept to support working and learning in product development projects and will discuss the potential of ethnographic methods for research in and evaluation of PBL. ...
... As knowledge is generated and applied by individuals in groups, successful product development teams must be able to share and negotiate increasingly heterogenous knowledge [1]. To convey the necessary skillset and to increase students' job readiness [2], PBL, which engages students in meaningful projects and the development of real-world products, has become a central element of engineering education [3; 4]. ...
... The results yield three practical implications for an improved design of PBL courses: (1) In order to support the individual skill development of the students a balanced distribution of experience and skills across the project teams is necessary to shorten the group finding process and more clearly allocate roles and responsibilities within the teams. This could be achieved by lecturers assigning the students to a team based on each student's expectations, goals, experience and skills as stated prior to the beginning of the course. ...
Conference Paper
What is going on in student project teams? How do students acquire, exchange, and integrate the knowledge necessary to collectively perform a given task? Alt-hough knowledge dynamics play a decisive role in project-based learning (PBL), they may happen during and/or between classes and thus take on diverse forms that are often difficult to track for students and teachers. Based on ethnographic research with in-situ observations and interviews, this paper sheds light on the knowledge dynamics of engineering graduate students in PBL courses beyond for-mal design reviews and graded project deliverables. The overall plot of the PBL-course mirrored quasi-real industrial work processes where lecturers acted as po-tential investors from industry. Our data suggest that knowledge dynamics in this context are related to two factors: (1) the individual student’s learning aspirations and (2) the evolving salience of distinct (functional) roles and knowledge resources amongst the team members. We have distilled the findings into an improved educa-tional concept to support working and learning in product development projects and will discuss the potential of ethnographic methods for research in and evaluation of PBL. In reaction to the pandemic, we have furthermore re-designed the concept to support online learning.
... 28 Data is enriched to information and knowledge 29 and can be provided in the form of a data stream, parameters, and digital models on a formal or an informal level. 30 Several notations from different domains allow the modeling and analysis of data streams or processes. 31,32 Related domains are, in particular, business process and computer science. ...
Full-text available
A significant challenge of managing successful engineering projects is to know their status at any time. This paper describes a concept of automated project progress measurement based on data flow models, digital twins, and machine learning (ML) algorithms. The approach integrates information from previous projects by considering historical data using ML algorithms and current unfinished artifacts to determine the degree of completion. The information required to measure the progress of engineering activities is extracted from engineering artifacts and subsequently analyzed and interpreted according to the project's progress. Data flow models of the engineering process help understand the context of the analyzed artifacts. The use of digital twins makes it possible to connect plan data with actual data during the completion of the engineering project.
Full-text available
The current engineering environment demands for an increasing level of interdisciplinarity, innovation, creativity and cross-domain thinking as well as the consideration of sustainability aspects. New concepts, such as Digital Twins and complex product systems lead to the need for integrated product development approaches and new methods that put the user perspective in focus. This also needs to be an integral part in today's teaching concepts of the next generation of engineers. At the Department of Industrial Information Technology of the Technical University of Berlin, a case study was conducted by applying a concept of project-based learning in the engineering domain to address these challenges. In this paper, the case study as well as the method and its validation are presented. Students from different engineering disciplines had the task of developing virtual and physical prototypes for a sustainable, complex product system with a digital twin and respective sustainable business models. Within a structured survey, the teaching concept and the applied method were validated and lessons learned as well as further improvement measures are derived.
Conference Paper
Full-text available
Self-care technologies have been influenced by medical values and models. One of the values that was acritically incorporated was that self-care was medicalised, and, as a result, technologies were designed to afford use with clinicians and fit structured medical processes. This paper seeks to broaden the understanding of self-care in HCI, to acknowledge the mundane ways in which self-care is achieved. Drawing on in-depth interviews with patients and carers, and online ethnography of an online community, we describe how the self-care of Parkinson's is mundane. The fieldwork contrasts with more medicalised perspectives on self-care, thus we discuss the properties of a self-care concept that would acknowledge its mundane nature. Our hope is to sensitise designers to identify the mundane ways in which self-care is performed and, consequently, design technologies that better fit the complexities of everyday life with a chronic condition.
Full-text available
This article provides researchers with knowledge of how to design a high quality mixed methods research study. To design a mixed study, researchers must understand and carefully consider each of the dimensions of mixed methods design, and always keep an eye on the issue of validity. We explain the seven major design dimensions: purpose, theoretical drive, timing (simultaneity and dependency), point of integration, typological versus interactive design approaches, planned versus emergent design, and design complexity. There also are multiple secondary dimensions that need to be considered during the design process. We explain ten secondary dimensions of design to be considered for each research study. We also provide two case studies showing how the mixed designs were constructed.
Full-text available
The essence of modelling is to reflect the studied piece of reality in such a way that best describes the selected elements of the designed system. A model is used in design to optimize the structure and parameters of the constructed object and is a tool for assessing the quality of construction, eliminating weak links and ensuring adequate safety components. In view of the aim of modelling, it can be divided into functional modelling, showing the complexity of the object, and reliability modelling, specifying its states at variable threshold values. In design, modelling allows for significant savings in resources that would otherwise be spent because of problems appearing at the prototype stage, but also during production or in the course of using the product. In the practice of ergonomic design many problems could be avoided if early enough in the design process the values of parameters and their relations would be taken into account through modelling. On the other hand, the modelling process can be costly and time-consuming to carry out, and against the currently pervasive lean production it is a highly undesirable factor. Therefore, the modelling process should be supported with the use of appropriate cognitive techniques namely ethnography design, which would determine inadequacies of existing models as well as indicate the equivalent conditions for modelling. The justification of the use of this technique results both from the possibility of providing additional information, as well as the opportunity to "test" the phenomena affecting the design process. Ergonomic modelling tests developed solutions towards their adaptation to users' anthropometric, biomechanical and psychomotor characteristics, as well as behaviour patterns. However, knowledge of the latter and achieving a sufficient ergonomic and functional quality of proposed solutions often requires the use of the ethnography design approach. The aim of this article is to test the practical application of ethnography design methodology in product design and to analyse the benefits of its use. The analysis is based on effects of its application with the support of product design from various industries, along with a discussion of the method's limitations. Among benefits of ethnography design, the greatest proved to be providing knowledge of nonspecific user behaviour previously unknown to designers, which when rendered by models allowed to develop innovative solutions.
The influence of a design education upon reasoning in response to ill-defined design problems was examined through a comparative protocol study of design and non-design students. A statistical analysis compared distributions of and transitions between the activities naming, framing, moving, and reflecting. Design student protocols were characterized by significantly increased activity associated with reasoning between problem definition and solution ideation. In contrast, participants lacking any formal design education or experience indicated significantly increased reasoning towards problem definition, with little evidence of ideation. A subsequent qualitative comparison identifies sketching as a potential driver for both increased solution-focused activity and greater iteration between problem definition and solution ideation. Implications for design ability, sketching and appositional bridge-building between problem definition and solution ideation are discussed.