![DOR from Ebel et al. [54] applied in the PDSA framework](https://www.researchgate.net/profile/Hannes-Rothe/publication/317289806/figure/fig3/AS:500495332671488@1496338794744/DOR-from-Ebel-et-al-54-applied-in-the-PDSA-framework_Q320.jpg)
Content uploaded by Hannes Rothe
Author content
All content in this area was uploaded by Hannes Rothe on Nov 17, 2017
Content may be subject to copyright.
Knowledge Accumulation in Design-Oriented
Research
Developing and Communicating Knowledge
Contributions
Ana Paula Barquet
(&)
, Lauri Wessel, and Hannes Rothe
Freie Universität Berlin, Garystr. 21, 14195 Berlin, Germany
ana.barquet@fu-berlin.de
Abstract. In this paper, we problematize a relative absence of established ways
to develop and communicate knowledge contributions (KC) from Design-
oriented research (DOR) within information systems. This is problematic since it
hinders the potential for knowledge accumulation within the field. Thus, for
communicating KC, we propose a framework, dubbed PDSA (Prescriptive,
Descriptive, Situated, and Abstract). To develop KC especially from empirical
data, we suggest the use of qualitative process methods. The framework is
illustrated by revisiting a published DOR study. Finally, we show how the
PDSA framework serves as a template to establish firm KC in DOR. In addition,
we explore contributions generated from empirical data and suggest possibilities
to use qualitative process methods as means to increase transparency and rigor
of KC development and communication.
Keywords: Knowledge contribution Qualitative process methods Empirical
data Design theory Design-oriented research
1 Introduction
The design of information systems (IS) has been an important topic within the IS
research community for many years [1–3]. Studies in this domain have improved
means for organizations to confront challenging issues, such as managing diffuse
knowledge processes [2], aligning individual and organizational competencies [4], or
exploiting potentials of secondary design in emergent IS [5]. Such design-oriented
studies typically aim to generate prescriptive knowledge on how organizations design
IS, while the development of explanatory or predictive knowledge might assume a
secondary role [6,7]. Therefore, design-oriented works are relatively well geared for
contributions to solving important problems in organizational practice [8–10].
Research on how to design artifacts has indeed become such an important topic in
IS that it is now widely acknowledged as the ‘Design Science (DS) paradigm’[11]. It
has grown significantly in volume throughout the last decades, comprising different
streams of literature that we summarize by using the umbrella term ‘design-oriented
research’(DOR). The main aim in DOR is to understand how effective artifacts can be
designed and how design-oriented knowledge can be utilized for theorizing [12].
©Springer International Publishing AG 2017
A. Maedche et al. (Eds.): DESRIST 2017, LNCS 10243, pp. 398–413, 2017.
DOI: 10.1007/978-3-319-59144-5_24
To this end, DOR usually draws on design processes that unfold over different stages
such as problem identification, development and evaluation of artifacts [13–15]. In this
context, evaluation of the utility of artifacts is typically central [16–18]. Yet, this has
led to the criticism that DOR is too narrowly concerned with designing and evaluating
artifacts and circumventing questions about generalizable knowledge [19,20].
Therefore, while there is a wellspring of work on how to design and evaluate the
quality and utility of artifacts, little is known about how to develop and communicate
knowledge contributions (KC). This is problematic since one important aspect of KC in
DOR comprises gradual abstraction of knowledge about particular instantiations into
more general ‘design principles’and ‘design theories’[12]. Yet, this abstraction
demands careful attention to how researchers collected and used empirical data when
interacting with organizations and how this affected formulation of ‘design principles’
or ‘design theories.’
In this paper, we constructively engage with the aforementioned challenges and
discuss how usage of empirical data within a DOR project affects KC. In this regard,
we propose that an increasing importance of theorizing from DOR [21,22] demands
critical engagement with procedures for data collection and analysis, which are carried
out as part of DOR. As DOR projects are typically seen as processes where researchers
enact multiple cycles and stages, we promote that reliance on procedures for the
analysis of process-data, which are known from innovation research [23], increases
potentials to clearly communicate how KC were formulated. In this spirit, we follow
recent calls to extend use of these particular procedures within DOR [24]. Mandviwalla
[24] recently stressed that techniques for analyzing process-data could be fruitfully
used to build design theories. By extension, we propose that use of these methods is not
limited to develop design theory but also to other types of KC. Our first research
question is thus (RQ1): How do qualitative process methods help to develop knowledge
contributions in DOR?
Through engagement with this question, we gradually discovered a second related
issue that demands careful consideration, i.e., what ‘knowledge contributions’really
are. Gregor and Hevner [12] have provided first insights by introducing a differentia-
tion of three levels of generalizability of KC. They also propose a rather generic
typology of KC, which considers a relevant but not complete selection of KC. For
instance, it does not cover some types of KC, such as the ones depicted from empirical
data. Accordingly, we ask (RQ2): How do we present and communicate ‘knowledge
contributions’created in DOR?
By taking Gregor and Hevner [12] as a starting point, we elaborate on their con-
tribution by further detailing what KC are and how they can be developed during DOR
projects. Our study, thus, offers two main contributions. On the one hand, we offer a
framework, called PDSA (Prescriptive, Descriptive, Situated, Abstract), that is con-
ducive to communicate and capture the dynamic evolution of novel knowledge in DOR
projects over time. Furthermore, we contribute by showing that techniques for analysis
of qualitative process methods [23] have significant potentials to inform the develop-
ment of cumulative KC, especially when DOR covers empirical cases.
We proceed by a brief review of the fundamentals in DOR, followed by a pre-
sentation of dominant procedural models to carry out DOR. This sets the basis to define
the problem, which addresses the lack of prescriptions on how to develop and
Knowledge Accumulation in Design-Oriented Research 399
communicate KC from DOR. Subsequently, we introduce the PDSA framework to
support the communication of KC and show how techniques for analysis of qualitative
process data supports the development of KC in DOR. We indicate the potentials of
this idea through an illustrative case. In closing, we discuss our contributions, limita-
tions and further research opportunities.
2 Relevant Literature on Design-Oriented Research (DOR)
Design-oriented research within IS can broadly be seen as a problem solving paradigm
that aims to extend human and organizational capabilities by developing artifacts [11].
Thus, development of IS artifacts is of central concern to DOR [3,12,20,25], which
has its intellectual roots in engineering and architecture [3,24,25]. Various perspec-
tives from behavioral sciences have been used in DOR over time (see for example [2,5,
26]). While this undoubtedly increased the prominence of DOR within the IS discipline
[18,20,27], the broad label ‘design-oriented research’has also become sub-divided
into different branches of literature that, while mostly similar, differ in details [12,25,
27]. First, ‘design research’refers to constructing artifacts in order to solve a specific
class of problems [22,27]. Second, ‘design science’in the narrower sense is concerned
with general rigor standards for conducting research projects [20]. Thus, design science
aims at “explicitly organized, rational and wholly systematic approach[es] to design”
[28] (p. 53) of IS artifacts. Third, ‘design theory’refers to theorizations of knowledge
about how specific classes of artifacts should be designed [3,7,24]. This means that
design theories put a strong emphasis on how design-oriented knowledge can be for-
malized and made subject to replication [3,12,24].
2.1 The Roles of Artifacts and Knowledge Contributions in DOR
The outputs of DOR encompass IS artifacts and KC. Recently, KC in the form of
design theories have become increasingly important [12]. In contrast, earlier works
highlighted that contributions of DOR largely comprise ‘design artifacts’[29] like
constructs, models, methods, and instantiations [7,30,31]. Even though many of these
artifacts carry certain degrees of abstraction [25,31], scholars in DOR expressed their
concern that it is sometimes hard to identify abstract knowledge contributions which
arise from a DOR project [12]. In this context, Gregor and Hevner [12] categorized
DOR contributions by their level of abstractness. Instantiations can be seen as most
concrete and particular contributions (‘Level 1’), ‘Level 2’contributions comprise
abstractions of a ‘mid-range’, such as design principles [32,33]. They reach beyond a
particular application context, but are themselves insufficient to be seen as ‘design
theories’[24,26]. Finally, design theories would be the most general and abstract
contribution of DOR (‘Level 3’). Table 1reviews these contributions.
The introduction of these levels ‘1–3’coincides with a general concern to develop
theory through DOR [24,34–36]. The reason is that the levels are cumulative, i.e. level
1 contributions can be developed into level 2 contributions, which may be the basis for
building level 3 contributions [24,26,37–39]. Accumulation is a key idea in this regard
400 A.P. Barquet et al.
because moving from level 1 to level 3 will unlikely be possible within a single
research project or one paper [24]. Instead, systematic design theory-building is likely
to be a process that emerges across publications of different scholars interested in
related phenomena [3]. Therefore, if DOR is to exploit these potentials for systematic
KC development, it needs a toolkit to explicate how knowledge was developed as well
as a clear way to communicate it so that succeeding studies can carry on in a systematic
way. Next, we review procedure models in DOR to assess whether and how they
incorporate such thinking.
2.2 Procedure Models in DOR
In this section, some of the commonly accepted procedure models for DOR are
explored with the goal to highlight how they address KC. The model of Hevner et al.
[11] aims to support the understanding, execution and evaluation of DOR. This par-
ticular framework was later revised as a model comprising three cycles [14]: (i) the
“relevance cycle”draws on business’needs and introduces the artifact into the appli-
cation domain, (ii) the “design cycle”comprises artifact building and evaluation and,
(iii) the “rigor cycle”receives applicable knowledge as input and adds contributions to
the knowledge base as output. Hevner et al. [11] also emphasized guidelines for
creation of useful artifacts. They highlight research contributions in the form of
designed artifacts, foundations or methodologies as well as the importance to com-
municate results.
Peffers et al. [15] proposed a methodology within steps from problem identification
to the development of a solution, demonstration, evaluation and communication. The
authors illustrated the latter as a part of an iterative DOR process, however it is not
further explained how ‘communication’informs further design iterations. Nunamaker
et al. [13] developed an iterative and prototypical process for system development in
IS, which covers fives phases: construct of a conceptual framework, development of a
Table 1. Examples of contributions from DOR according to their level of abstractness
Contributions Definition Level
Instantiation The realization of an artifact in its environment Level
1
Constructs The conceptualization used to describe problems and solutions
within a certain domain
Level
2
Model Set of propositions or statements that express relationships
between constructs
Level
2
Method Set of steps used to perform a task. It is based on constructs and
the solution space representation
Level
2
Design
principles
Knowledge captured in the process of creating solutions and
building instances for same problems class
Level
2
Design
theory
Covers “explanatory, predictive, normative aspects”into a design
for achieving a specific goal
Level
3
Knowledge Accumulation in Design-Oriented Research 401
system architecture, analysis and design of the system, building of a (prototypical)
system, and observing and evaluating the system.
Mandviwalla [24] developed a set of processes to support the development of
design theory. Goals, kernel theory and existing artifacts inform the prototyping cycle,
which includes a concurrent iteration of design, evaluation, and appropriation/
generation. Last, Sein et al. [26] proposed a method called ‘action design research’,
which treats the artifact creation and evaluation as inseparable and interrelated activ-
ities, differently from traditional DOR, which separates artifact building and evaluation.
Action design research addresses four inter-related stages: problem formulation,
building, intervention and evaluation, reflection and learning and formalization of
learning.
Even though KC play a role for these procedure models, they provide limited
prescriptions on how to develop or communicate them. For example, the importance of
knowledge contributions is echoed in the method of Sein et al. [26] within “formal-
ization of the learning”; within Hevner et al. [11], in “Guideline 4: Knowledge
Contributions”; and in studies that stressed (partial) design theories as outcomes of
DOR [12,24], including theory building or refinement [13].
3 Towards Development and Communication of KC
3.1 Knowledge Contribution in DOR
Claims on how to develop and communicate knowledge contributions demand a
clarification of what knowledge represents in lieu of epistemological and ontological
considerations. The epistemological position of this study sees DOR knowledge con-
tributions as “knowing by making”[39] (p. 4). Thus, knowledge embraces creation of
artifacts as well as understanding the more abstract idea that guided the design of the
artifact, regardless if this is theory building or testing. Ontology addresses nature and
components of theory [7]. Our ontological position coincides with DOR (see [7,37,
40]), which separates theory from understanding of individuals through Popper’s[41]
three worlds classification. He discerns an objective/material (1), from a subjective/
mental (2), and an abstract world (3). The latter embraces human-made entities, e.g.
language, theories, models, and constructs. According to this view, artifacts instanti-
ations are part of world 1, abstract knowledge is part of world 3 and ideas and
experiences of design science researchers belong to world 2 [12].
3.2 Types of Knowledge Contribution in DOR
In order to structure and understand better KC from DOR, we considered publications
which addressed topics such as design knowledge, theory development or theorizing in
DOR as well as knowledge contribution itself [7,12,21,32,40,42–44]. Additionally,
we investigated DOR procedure models, as presented in Sect. 2.2, and possible KC
mentioned by them. In the end, a set of types of KC were identified, as exemplified in
Table 2.
402 A.P. Barquet et al.
The authors of this paper worked together in a set of discussion rounds to clarify the
similarities and differences of the identified KC. Relying on Gregor and Hevner [12],
our first insight was to classify KC according to descriptive and prescriptive knowl-
edge. While this was a feasible alternative, we also realized that some of the KC differ
according to its level of abstractness, i.e. some KC are context-related or data-driven
while others are more abstracts or theory-driven, e.g. theories [45,46]. Therefore, we
propose four dimensions for a KC typology: descriptive, prescriptive, situational and
abstract.
Descriptive vs. Prescriptive Knowledge. Based on Aristotle’s terms “episteme”and
“techne”, Gregor and Hevner [12] suggest to classify knowledge as descriptive and
prescriptive. Descriptive knowledge describes natural, artificial and human phenomena
as well as relationships among them. By classifying, observing, measuring and cata-
loging, these descriptions can be made accessible to the human mind [47]. Prescriptive
knowledge addresses artifacts created to improve reality. Gregor and Hevner [12] have
added design theories arguing that they are formed from prescriptive knowledge that
can also include other types of knowledge. In this sense, while prescriptive research
focuses on improvement through the “how”knowledge, descriptive research focus on
understanding via the “what”knowledge [12,29,36]. Yet, descriptive knowledge
might evolve into prescriptive knowledge, e.g., when explanatory statements are
combined with goals into prescriptive statements [48] or when little is known about the
phenomena and classification schema or taxonomies [7] prompt future research.
Situational vs. Abstract Knowledge. Goldkuhl and Lind [45] proposed to classify
DOR-related knowledge as abstract versus situational. Abstract knowledge refers to
general knowledge enhancing understanding phenomena so that this knowledge can be
used as foundation for DOR in a variety of contexts. Situational knowledge refers to
specific knowledge generated in specific contexts and produced during empirical
design practice. In this sense, a set of data about single facts are generated by not yet
considered theory, although they might be foundations for future theories [7]. Against
this background, situational outcomes are more empirical outcomes or exploratory
Table 2. Examples of knowledge contributions from DOR identified on the review
Examples of KC from DOR
Constructs, statements of relationship, causal explanations, testable propositions (hypothesis),
prescriptive statements, frameworks, classification schema, taxonomies [7]
Patterns, principles, laws of a phenomenon [12]
Observational, predictive and explanatory statements [37]
Instantiated artefacts, empirical data and data triggers (e.g. interview questions and observation
protocols); models that works only in a specific situation [45]
Constructs, models, methods and instantiations [29]
[29], evaluation methods and metrics [11]
[29], design principles and technological rules, design theory [12]
Instantiations, design principles, technological rules [24]
Descriptive knowledge, hypotheses, mechanisms, conceptual knowledge (ontologies, concepts
or constructs) [13]
Knowledge Accumulation in Design-Oriented Research 403
results. Abstract knowledge embraces design theories but also other knowledge con-
tributions developed throughout iterative cycles of (i) generation and validation of
knowledge and (ii) between different types of knowledge sources, such as empirical
data, design theory, other knowledge and theories [17,45]. Abstract knowledge can be
extracted from as well as empirically grounded in situational knowledge and adapted to
be applied in situational contexts, which might lead to modification of the abstract
knowledge. On the other hand, situational knowledge is grounded in abstract knowl-
edge and can also evolve into generalized abstract knowledge [45].
Both attempts to classify knowledge, i.e. the descriptive vs. prescriptive as well as
the situational vs. abstract dichotomies, share much in spirit yet differ in important
ways. In terms of similarity, both ideas allow that a focal project uses and generates
both types of knowledge per respective dichotomy and that cumulative research helps
to develop one type of knowledge out of the other. In terms of differences, the
dichotomies refer to different claims. Where prescriptive (P) vs. descriptive (D) ad-
dresses the knowledge base on DOR [7], situational (S) vs. abstract (A) is more
concerned with the knowledge reach (design knowledge of a specific context or more
generalizable) but less with whether that is prescriptive or descriptive. In this sense,
synthesis of both views helps to systematically understand and classify knowledge
generated in DOR, therefore supporting to answer our second research question.
After defining dimensions for a KC typology, we separately placed the set of
identified KC into the typology to be sure that the four dimensions could comprise all
types properly. In another round of discussion, we compared the individual classifi-
cations and discussed them until we found a common decision. Figure 1presents the
result of this discussion by illustrating examples of different knowledge contribution
types classified according to the KC typology.
3.3 Communication of Knowledge Contributions: The PDSA Framework
As KC in DOR emerge throughout a research process, it is important to consider time
when classifying and presenting them. To this end, we introduce a framework drawing
Fig. 1. Typology of KC from DOR
404 A.P. Barquet et al.
on the aforementioned typology and incorporate the time dimension, represented by the
phases of a DOR project. Figure 2shows the framework.
The procedure models for DOR cover different number of phases and labelling. For
Peffers et al. [15], phase 1 is “problem identification and motivation”, phase 2
“definition of the objectives for a solution”, phase 3 “design and development”, phase 4
“demonstration”, phase 5 “evaluation”and phase 6 “communication”. For Sein et al.
[26], phase 1 is “problem formulation”, phase 2 “building, intervention and evalua-
tion”, phase 3 “reflection and learning”and phase 4 “formalization of learning”.
Therefore, the three phases showed in Fig. 2have illustrative purposes.
Figure 2provides a backdrop to propose means to communicate KC in DOR. As
scholars generally see DOR as a process [13–15,24,26], KC likely emerge through
phases like those in Fig. 2. Moreover, as they emerge through these phases, they likely
fall into different quadrants over time. Providing transparency about these dynamics,
we believe, is central to clearly communicate KC in DOR and support the reuse and
accumulation of knowledge over time.
4 Development of Knowledge Contributions from Empirical
Data: Potentials of Qualitative Process Methods
In this section, we draw on templates for analyzing process data in order to suggest a
frame of reference for how DOR researchers could develop KC especially in DOR
covering empirical cases. In doing so, we draw on qualitative process methods [23,49],
prominent in behavioral IS and management studies [50,51] but comparatively
under-utilized in DOR [52]. By doing that, we follow the suggestion of Mandviwalla
[24], who proposed the use of these methods to develop design theory. Design theory
not only represents the prominent KC generated in DOR, but may also represent the
final outcome of knowledge accumulation steps throughout several projects and pub-
lications. Our intention is to show that such methods do not only help to develop design
theory, but also other types of KC.
Fig. 2. PDSA framework
Knowledge Accumulation in Design-Oriented Research 405
Process methods are methods to analyze data that has been collected over a series of
events [23,49,53–55]. Phases of DOR procedure models (see above) include many
potential events like ‘formalization’and ‘evaluation’[26]. Therefore, systematically
collecting and analyzing process data can help to increase data quality and, hence, the
overall rigor of the resulting KC. A general template for rigorous use of process
methods, proposed by Langley [23], can be used to develop KC in DOR. The seven
data analysis strategies are Grounded Theory, Alternative Templates, Narrative, Visual
Mapping, Temporal Bracketing, Quantification and Synthesis.
Several strategies can be seen as “sources for concepts”[23] (p. 707) because they
allow researchers to become grounded in the empirical phenomenon and to begin
theorizing from it. Langley [23] proposed that two strategies would be helpful in this
regard: (i) a grounded theory strategy as well as (ii) alternative templates strategy.
(i) Grounded theory allows the systematic and transparent development of conceptual
categories from the empirical data. The alternative templates strategy (ii) is more
deductive in that it proposes to use different pre-existing theoretical premises to explain
data and assess which premise performs best [23]. As such, these two strategies have a
lot in common with the process of formalizing problems in DOR.
Even though they did not use this language, Giessmann and Legner [56] seemed
close to the grounded theory approach since they engaged with the field to formalize
the design problem as prescribed by Sein et al. [26]. In contrast, Peters et al. [57]
surveyed existing literature, i.e. theoretical templates, to justify their solution.
Accordingly, the source chosen to formalize or ground a problem in DOR depends on
the individual study. This coincides with Gregor and Hevner’s[12] proposal that
different types of problems imply KC that differ in scope. However, how to assess that
scope is a relatively under-developed in DOR. Hence, reliance on a more standardized
procedure could help to increase validity and transparency in qualitative DOR.
The organization of data can be seen as a crucial step in DOR. With organizing
data, Langley [23] refers to means of “descriptively representing process data in a
systematic organized form.”We believe that such systematic engagement with the data
is important when researchers build artifacts, intervene in the field and evaluate out-
comes of this intervention. At this stage, numerous encounters with the field happen,
and empirical data gathered in this encounters affect the formulation of KC [17,20,36].
Thus, a transparent and organized way to report on the development of the processes of
building, intervening and evaluating could help external audiences to trace how
immersion with the field affected KC. Two other strategies could help here: (iii) a
narrative strategy as well as a (iv) visual mapping strategy [23].
The narrative strategy (iii) comprises writing a detailed narrative about the research
process to provide numerous contextual details about how a DOR process unfolded,
putting more focus on the situational knowledge. This level of detail can help to
disentangle which encounters affected the formulation of KC in a highly granular
manner. The visual mapping strategy (iv) is more reductionist. While narratives capture
many details in words, visual maps are abstract representations of the building, inter-
vention and evaluation processes that took place. Such maps should include clear
denominations, i.e. “arrows and boxes”, effects of one element on another (positive/
negative) as well as brief descriptions of the involved elements. Both strategies could
also be used in high and low n studies. For n > 1, it seems possible to compare
406 A.P. Barquet et al.
narratives and visual maps across cases in order to search for regularities. For n = 1
narratives and maps could be compared across design cycles in the form of within-case
analysis. Practically, narratives and maps could be made available as research sup-
plements, which would increase transparency over the process of developing KC.
Because increasing attention has been paid to formalizing KC and making them
replicable and testable [3,24], it is important to understand how qualitative methods
can serve this purpose. In this context, three of Langley’s[23] strategies can be helpful:
(v) temporal bracketing, (vi) quantification as well as (vii) synthesis. Temporal
bracketing (v) means to structure a DOR process into distinct phases, which arise due
to a “certain continuity in the activities within each period and there are certain dis-
continuities at its frontiers”[23] (p. 703). Temporal bracketing is in a sense evident in
DOR procedure models since labels such “building, intervention, evaluation”[23]
(p. 559) are used to structure the process. This provides a significant opportunity for
replicating DOR studies because if researchers document how each phase affected the
development of the KC (for example by visual maps), other researchers could replicate
studies or modify them by bringing them to other contexts or by holding certain factors
constant while variating others. This may not be a ‘hard control’in the statistical sense
but nonetheless an insightful inquiry into the maturity of KC that allows to assess
whether these are ‘design theories’or ‘design principles’[12].
The quantification strategy (vi) fosters quantitative analysis of the data and, hence,
formalization of design principles as hypotheses that enable testing and replication.
Specifically, this strategy involves systematically coding the data, for example visual
maps or narratives, according to sets of pre-defined codes, which could be results of the
formalization phase. For process theorists [23,49,53–55], one important coding in this
regard is to capture whether intended changes in each phase of the process really
occurred. This ties in nicely with high-level KC, i.e. design theory, because theoretical
predispositions about why a design should work can be coded as well as whether the
design really had such outcomes. This involves tests of the design propositions and
thus yields more robust propositions as outcomes. Therefore, this is a means to for-
malize qualitative DOR and make it conducive to replication.
The last strategy, synthesis (vii), is related to quantification, perhaps most reduc-
tionist and least suited for low n case. Synthesis “attempts to construct global measures
from the detailed event data”[23] (p. 704). The main goal of this approach is to identify
larger regularities across processes that allows the formulation of a more predictive
theory in the statistical sense. Thus, this approach aims to synthesize qualitative process
data into a more abstract statement on how certain independent variables affect
dependent variables. This could be done if sufficient information is available (like
narratives or visual from multiple cases). For this reason, this approach is the most
suitable in terms of making a qualitative design theory generalizable.
Through the description of these strategies, we attempted to unpack that qualitative
process methods [23] have potentials to inform how DOR develops KC. Next, in order
to illustrate how the PDSA framework can communicate DOR knowledge as well as
how qualitative processes methods can be used to develop KC from empirical data, we
present an illustrative case.
Knowledge Accumulation in Design-Oriented Research 407
5 Development and Communication of KC: Illustrative Case
To illustrate our contributions, a published DOR by Ebel et al. [58] was chosen after a
literature search on development of design theories from empirical cases. Search of
relevant literature was done using the template by von Brocke et al. [59] in the journals
listed in the AIS’“Basket of Eight”. Because of space constrains, we only mention the
11 papers selected: [5,21,34,35,56–58,60–63]. Ebel et al. [58] is an interesting
example of empirical data use in different phases of a DOR project as well as for the
communication of KC in the “formalization of learning”phase.
In order to illustrate the use of the framework in this case, each of the DOR phases
are briefly explained and a number is given for the sequence of activities carried out
within these phases. Subsequently, in order to understand the development of KC from
empirical data, an analysis of the situated knowledge is done with the goals to present
how deep this knowledge was addressed, how it was presented and similarities and
opportunities of developing it according to Langley’s[23] strategies.
5.1 Communication of KC: Application of the PDSA Framework
Drawing on action design research, Ebel et al. [58] developed a solution for system-
atically designing business models based on theoretical and empirical knowledge about
business models. Figure 3shows the application of the PDSA framework to their
study. Note that despite the fact that Sein et al. [26] suggest to carry out the phases
development and evaluation together, we separate them for illustrative purposes, i.e. to
better represent the time sequence of activities and the KC emerging from them.
Problem identification was done by reviewing the existing product portfolio of a
specific company (1) and investigating literature. Both led to the formulation of a set of
processes relevant to develop and manage business models (2) as well as identification
of gaps in the literature (3). Aiming to assess the processes they developed as well as to
contribute to the literature, interviews with experts were performed using semi-
structured questionnaires (4). Analysis of this data occurred in three phases: immersion,
Fig. 3. DOR from Ebel et al. [54] applied in the PDSA framework
408 A.P. Barquet et al.
reduction and interpretation. During immersion, data were transcribed and analyzed. In
the reduction phase, data was reduced to what was considered relevant to the research.
To reach that, a coding scheme and codebook were created in order to enable the
rearrangement of data into meaningful categories. The process of creating the codes are
explained in detail, however the data itself was not presented. During interpretation,
codes were then used to reassemble data in a coherently and concisely. From the
analysis, the authors stated they could confirm the processes they developed in (2).
In order to build their artifact, kernel theories used to solve similar problems were
investigated (5). Next steps concerned the creation of an alpha-version (6) and its
evaluation within an organizational setting (7) drawing on 27 test users. With the aim
of evaluating the usability of the artefact, the Questionnaire for User Interaction Sat-
isfaction (QUIS) was used and the data analyzed through an independent-samples t-test
(M > 5). Some insights of the evaluation are given: “… a major weakness of the
artifacts is that the used terminology does not relate well to the work situation…”;“[t]
he testers also criticized the system as being too dull (…) and too rigid to cope with
their needs”(p. 17). When reporting the refinement of the functionalities, the authors
described how they improved the tool (8) according to the weaknesses pointed out
during the first evaluation. The second evaluation (9) aimed at assessing the tool
efficacy. To this end, six project teams were formed with the goal of developing
business models with the tool. Based on literature, six dimensions (novelty, originality,
feasibility, acceptability, effectiveness and elaboration) were used to assess the results.
In “formalization of learning”, two aspects of the framework that extend the
existing literature were pointed out: the shared material sections and the community
Section (10). While the former includes guides on how people from outside the project
can contribute to the design of the business model, the first provides training material to
support the development of business models. Finally, by stating, “the artefact itself
produces knowledge as constructs and instantiations that may or may not lead to the
level of abstraction that constitutes a design theory”(p. 26), the authors concluded that
they created three major contributions to the knowledge base for the specific problem
domain. A conceptual classification (2), which is “…descriptive knowledge in the
problem domain”(p. 26), additional descriptive knowledge that extends current liter-
ature (10) and the development of a framework to create business models, a nascent
design theory (8), cover these contributions.
5.2 Development of KC from Empirical Data: Qualitative Process
Methods
Despite the sophisticated use of data in DOR, it seems sometimes difficult to build on
earlier studies because little is known about how data was used. In terms of this study
in particular, situated knowledge is generated in problem identification (1), through
expert interviews (4), as well as evaluation (7, 9) and it is here were use of Langley’s
[23] strategies could amplify transparency. For example, details of the problem iden-
tification (1) can be made more widely known through either using grounded theory or
alternative templates. Choice between the two likely varies with maturity of design
theorizing in a particular domain, as more mature domains are likely to offer more
Knowledge Accumulation in Design-Oriented Research 409
firmly established templates to identify and analyze problems. In case of developing
‘nascent’design theories, grounded theory is likely helpful as ‘nascence’of design
theories indicates that they were developed in partially unknown contexts that become
more known through the nascent design theory.
Similarly, regarding evaluation with experts (4), while this particular study explains
the coding process, it does provide little insights on data itself. While insights gathered
from interviews are presented, they lack links to the raw data. It is interesting that
provision of such links seems key for papers in behavioral IS, which implies that it
should also be seen as such in the DOR context. To facilitate the formulation of these
links, we propose the use of narratives or visual mapping. Narratives may accrue more
to cases that began with grounded theory while visual mapping may facilitate to show
how codes, which were developed from literature [58], can be related to each other on
the basis of the analysis of empirical material. This could help to show how comments
made in interviews converged towards KC. Concerning evaluation (7; 9), a statistical
analysis is presented, which suggests that linking earlier, more qualitative insights to
the ‘synthesis’strategy [23] could be helpful to also use mixed-methods in DOR.
6 Discussion
Within this article, we highlighted the role of designing and communicating KC in
DOR, a current gap in the literature [12,17,37]. Our first contribution is the PDSA
framework, suitable to systematically communicate KC from the entire DOR process.
PDSA goes beyond existing research by synthesizing somewhat isolated understand-
ings of knowledge contributions in DOR (e.g. [12,45]) and by providing a backdrop to
map and understand how KC emerge over time. In addition, by considering the time
dimension, the framework can be used complementary to different existing procedure
models in DOR. The second contribution of this study is to offer suggestions on how to
develop KC from empirical data. To this end, we leveraged the techniques for ana-
lyzing qualitative process data proposed by Langley [23] and pointed out that these can
be used to report how empirical data was collected and analyzed. Through a somewhat
more formalized approach to justify KC generated from empirical data, we hope to
offer a toolkit that enables researchers to more easily explicate and document what they
did, how they did it, and what the limitations of these approaches were. In turn, this will
enable audiences to better assess the rigor of KC emerging from DOR.
While hopefully thought provoking, our work is not without limitations. First, our
work has not been formally evaluated, a limitation that needs to be overcome in the
future. In addition, only one application of the framework is presented, despite the fact
that we applied it in several of the papers selected from the literature review. Therefore,
we see multiple options for future research. By increasing the number of illustrative
cases to which the framework is applied, we can find associations between types of KC
in different phases of DOR. For instance, we can depict how descriptive and situational
knowledge of one phase is linked to abstract and prescriptive knowledge in another.
Therefore, the framework may be helpful in specifying the role, validity, and boundary
conditions of these associations. Furthermore, it may enable us to explore patterns for
410 A.P. Barquet et al.
KC developed, methods applied, and quality standards in their evaluation in accor-
dance with different research aims, e.g. varying artifacts.
References
1. Walls, J.G., Widmeyer, G.R., El Sawy, O.A.: Building an information system design theory
for vigilant EIS. Inf. Syst. Res. 3,36–59 (1992)
2. Markus, M.L., Majchrzak, A., Gasser, L.: A design theory for systems that support emergent
knowledge processes. MIS Q. 26, 179–212 (2002)
3. Jones, D., Gregor, S., Jones, D., Gregor, S.: The anatomy of a design theory. J. Assoc. Inf.
Syst. 8, 312–335 (2007)
4. Lindgren, R., Henfridsson, O., Schultze, U.: Design principles for competence management
systems: a synthesis of an action research study. MIS Q. 28, 435–472 (2004)
5. Germonprez, M., Hovorka, D., Gal, U.: Secondary design: a case of behavioral design
science research. J. Assoc. Inf. Syst. 12, 662–683 (2011)
6. Davison, R.M., Martinsons, M.G., Ou, C.X.J.: The roles of theory in canonical action
research. MIS Q. 36, 763–786 (2012)
7. Gregor, S.: The nature of theory in information systems. MIS Q. 30, 611–642 (2006)
8. Zmud, B.: Editor’s comments. MIS Q. 20, 257 (1996)
9. Davison, R.M., Martinsons, M.G., Kock, N.: Principles of canonical action research. Inf.
Syst. J. 14,65–86 (2004)
10. Straub, D., Ang, S.: Editor ’s comments: rigor and relevance in IS research: redefining the
debate. MIS Q. 35,ii–xi (2011)
11. Hevner, A.R., March, S.T., Park, J., Ram, S.: Design science in information systems
research. MIS Q. 28,75–105 (2004)
12. Gregor, S., Hevner, A.R.: Positioning and presenting design science research for maximum
impact. MIS Q. 37, 337–355 (2013)
13. Nunamaker, J., Chen, M., Purdin, T.: Systems development in information systems research.
J. Manag. Inf. Syst. 7(3), 89–106 (1990)
14. Hevner, A.R.: A three cycle view of design science research. Scand. J. Inf. Syst. 19,87–92
(2007)
15. Peffers, K., Tuunanen, T., Rothenberger, M.A., Chatterjee, S.: A design science research
methodology for information systems research. J. Manag. Inf. Syst. 24,45–77 (2007)
16. Pries-Heje, J., Baskerville, R.L., Venable, J.R.: Strategies for design science research
evaluation. Eur. Conf. Inf. Syst. Pap. 87,1–13 (2008)
17. Venable, J., Pries-Heje, J., Baskerville, R.: A comprehensive framework for evaluation in
design science research. In: Peffers, K., Rothenberger, M., Kuechler, B. (eds.) DESRIST
2012. LNCS, vol. 7286, pp. 423–438. Springer, Heidelberg (2012). doi:10.1007/978-3-642-
29863-9_31
18. Venable, J., Pries-Heje, J., Baskerville, R.: FEDS: a framework for evaluation in design
science research. Eur. J. Inf. Syst. 25,77–89 (2016)
19. Chandra, L., Seidel, S., Gregor, S.: Prescriptive knowledge in IS research: conceptualizing
design principles in terms of materiality, action, and boundary conditions. In: 2015 48th
Hawaii International Conference on System Sciences (HICSS), pp. 4039–4048 (2015)
20. Winter, R.: Design science research in Europe. Eur. J. Inf. Syst. 17, 470–475 (2008)
21. Gregory, R.W., Muntermann, J.: Heuristic theorizing: proactively generating design theories.
Inf. Syst. Res. 25, 639–653 (2014)
Knowledge Accumulation in Design-Oriented Research 411
22. Niehaves, B., Ortbach, K.: The inner and the outer model in explanatory design theory: the
case of designing electronic feedback systems. Eur. J. Inf. Syst. 25, 303–316 (2016)
23. Langley, A.: Strategies for theorizing from process data. Acad. Manag. Rev. 24, 691–710
(1999)
24. Mandviwalla, M.: Generating and justifying design theory. J. Assoc. Inf. Syst. 16, 314–344
(2015)
25. Gregory, R.W.: Design science research and the grounded theory method: characteristics,
differences, and complementary uses. In: Theory-Guided Model Empiricism Information
Systems Research, pp. 111–127 (2011)
26. Sein, M.K., Henfridsson, O., Purao, S., Rossi, M., Lindgren, R.: Action design research.
MIS Q. 35,37–56 (2011)
27. Baskerville, R.: What design science is not. Eur. J. Inf. Syst. 17, 441–443 (2008)
28. Cross, N.: Designerly ways of knowing: design discipline versus design science. Des. Issues
17,49–55 (2001)
29. March, S.T., Smith, G.F.: Design and natural science research on information technology.
Decis. Support Syst. 15, 251–266 (1995)
30. Iivari, J.: Distinguishing and contrasting two strategies for design science research. Eur.
J. Inf. Syst. 24, 107–115 (2015)
31. Österle, H., Becker, J., Frank, U., Hess, T., Karagiannis, D., Krcmar, H., Loos, P.,
Mertens, P., Oberweis, A., Sinz, E.J.: Memorandum on design-oriented information systems
research. Eur. J. Inf. Syst. 20,7–10 (2011)
32. Kuechler, W., Vaishnavi, V.: A framework for theory development in design science
research: multiple perspectives. J. Assoc. Inf. Syst. 13, 395–423 (2012)
33. Merton, R.K.: Social Theory and Social Structure. The Free Press, New York (1968)
34. Kuechler, B., Vaishnavi, V.: On theory development in design science research: anatomy of
a research project. Eur. J. Inf. Syst. 17, 489–504 (2008)
35. Pries-Heje, J., Baskerville, R.: The design theory nexus. MIS Q. 32, 731–755 (2008)
36. Baskerville, R., Pries-Heje, J.: Explanatory design theory. Bus. Inf. Syst. Eng. 2, 271–282
(2010)
37. Iivari, J.: A paradigmatic analysis of information systems as a design science. Scand. J. Inf.
Syst. 19,39–64 (2007)
38. Bunge, M.: Scientific Research II. The Search for Truth. Springer, New York (1967)
39. Purao, S.: Design research in the technology of information systems: truth or dare.
Pennsylvania State University (2002)
40. Gregor, S., Jones, D.: The anatomy of a design theory. J. Assoc. Inf. Syst. 8, 313–335 (2007)
41. Popper, K.: Unended Quest an Intellectual Autobiography. Fontana, Glasgow (1986)
42. William, K., Vijay, V.: A framework for theory development in design science research:
multiple perspectives science research: multiple perspectives. J. Assoc. Inf. 13, 395–423
(2007)
43. Beck, R., Weber, S., Gregory, R.W.: Theory-generating design science research. Inf. Syst.
Front. 15, 637–651 (2013)
44. Holmstrom, J., Tuunanen, T., Kauremaa, J.: Logic for accumulation of design science
research theory. In: 2014 47th Hawaii International Conference on System Sciences,
pp. 3697–3706. IEEE (2014)
45. Goldkuhl, G., Lind, M.: A multi-grounded design research process. In: Winter, R.,
Zhao, J.L., Aier, S. (eds.) DESRIST 2010. LNCS, vol. 6105, pp. 45–60. Springer,
Heidelberg (2010). doi:10.1007/978-3-642-13335-0_4
46. Fischer, C., Gregor, S., Aier, S.: Forms of discovery for design knowledge. In: Proceedings
of the ECIS 2012, p. 64 (2012)
412 A.P. Barquet et al.
47. Nagel, E.: The Structure of Science Problems in the Logic of Scientific Explanation. Hackett
Publishing Co., Indianapolis (1979)
48. Goldkuhl, G.: Design theories in information systems –a need for multi-grounding. J. Inf.
Technol. Appl. 6,59–72 (2004)
49. Langley, A., Smallman, C., Tsoukas, H., Van de Ven, A.H.: Process studies of change in
organization and management: unveiling temporality, activity, and flow. Acad. Manag. J. 56,
1–13 (2013)
50. Pozzebon, M., Pinsonneault, A.: Challenges in conducting empirical work using structura-
tion theory: learning from it research. Organ. Stud. 26, 1353–1376 (2005)
51. Ortiz de Guinea, A., Webster, J.: Overcoming variance and process distinctions in
information systems research. In: Proceedings of the 35th ICIS, Auckland, NZ (2014)
52. Adomavicius, G., Bockstedt, J.C., Gupta, A., Kauffman, R.J.: Making sense of technology
trends in the information technology landscape: a design science approach. MIS Q. 32,
779–809 (2008)
53. Van de Ven, A.H., Poole, M.S.: Methods for studying innovation development in the
minnesota innovation research program. Organ. Sci. 1, 313–335 (1990)
54. Van de Ven, A.H., Poole, M.S.: Explaining development and change in organizations. Acad.
Manag. Rev. 20, 510–540 (1995)
55. Van de Ven, A.H., Poole, M.S.: Alternative approaches for studying organizational change.
Organ. Stud. 26, 1377–1404 (2005)
56. Giessmann, A., Legner, C.: Designing business models for cloud platforms. Inf. Syst. J. 26,
551–579 (2016)
57. Peters, C., Blohm, I., Leimeister, J.M.: Anatomy of successful business models for complex
services: insights from the telemedicine field. J. Manag. Inf. Syst. 32,75–104 (2015)
58. Ebel, P., Bretschneider, U., Leimeister, J.M.: Leveraging virtual business model innovation:
a framework for designing business model development tools. Inf. Syst. J. 26, 519–550
(2016)
59. von Brocke, J., Simons, A., Niehaves, B., Riemer, K., Plattfaut, R., Cleven, A.:
Reconstructing the giant: on the importance of rigour in documenting the literature search
process. In: 17th European Conference on Information Systems, vol. 9, pp. 2206–2217
(2009)
60. Gregor, S., Imran, A., Turner, T.: A “sweet spot”change strategy for a least developed
country: leveraging e-Government in Bangladesh. Eur. J. Inf. Syst. 23, 655–671 (2014)
61. Papas, N., O’Keefe, R.M., Seltsikas, P.: The action research vs design science debate:
reflections from an intervention in eGovernment. Eur. J. Inf. Syst. 21, 147–159 (2012)
62. Rosenkranz, C., Holten, R., Räkers, M., Behrmann, W.: Supporting the design of data
integration requirements during the development of data warehouses: a communication
theory-based approach. Eur. J. Inf. Syst. 0,1–33 (2015)
63. Spagnoletti, P., Resca, A., Sæbø,Ø.: Design for social media engagement: insights from
elderly care assistance. J. Strateg. Inf. Syst. 24, 128–145 (2015)
Knowledge Accumulation in Design-Oriented Research 413
