Data-driven Failure Management: An Ontology-based Speech Recognition App for Failure Capturing in Manufacturing Processes

Abstract

Manufacturing processes are characterized by an increasing complexity , making them susceptible to failures. An effective strategy to avoid such failures , or at least to minimize their impact, is data-driven failure management. However, for many small and medium sized manufacturers, this strategy is not feasible due to a paucity of relevant failure data, which can be explained by the severe limitations and shortcomings of available solutions: ranging from the error proneness and high efforts of manual solutions to the high costs and implementation efforts of automated solutions. Against this backdrop, our study follows a design science research approach to design, develop, and evaluate a novel ontol-ogy-based speech recognition app that addresses key shortcomings of currently available solutions. Main contributions of our study are the development of design requirements and principles, as well as their instantiation in an app prototype for collecting failure data in the context of manufacturing processes.

Full text

Data-driven Failure Management: An Ontology-based
Speech Recognition App for Failure Capturing in
Manufacturing Processes
Philipp Scharfe1, Heiner Ludwig2, Katja Bley3,4, Martin Wiener1, and Thorsten
Schmidt2
1 Business Information Systems, esp. Business Engineering, TU Dresden, Dresden, Germany
2 Institute of Material Handling and Industrial Engineering, TU Dresden, Dresden, Germany
3 Business Information Systems, esp. IS in Trade and Indsutry, TU Dresden, Dresden, Germany
{philipp.scharfe,heiner.ludwig,katja.bley,martin.wiener,
technische.logistik}@tu-dresden.de
4 Department of Computer Science, Norwegian University of Science and Technology,
Trondheim, Norway
katja.bley@ntnu.no
Abstract. Manufacturing processes are characterized by an increasing complex-
ity, making them susceptible to failures. An effective strategy to avoid such fail-
ures, or at least to minimize their impact, is data-driven failure management.
However, for many small and medium sized manufacturers, this strategy is not
feasible due to a paucity of relevant failure data, which can be explained by the
severe limitations and shortcomings of available solutions: ranging from the error
proneness and high efforts of manual solutions to the high costs and implemen-
tation efforts of automated solutions. Against this backdrop, our study follows a
design science research approach to design, develop, and evaluate a novel ontol-
ogy-based speech recognition app that addresses key shortcomings of currently
available solutions. Main contributions of our study are the development of de-
sign requirements and principles, as well as their instantiation in an app prototype
for collecting failure data in the context of manufacturing processes.
Keywords: Data-driven failure management, manufacturing-failure capturing,
speech recognition app, ontology, design science.
1 Introduction
Nowadays, manufacturing processes are characterized by a high complexity since they
need to meet rising demands in terms of productivity, quality, and flexibility [1, 2]. As
a result of this complexity, manufacturing processes are becoming increasingly prone
to failures [1, 2]. In a manufacturing context, a failure can be broadly defined “as an
unforeseen deviation from the planned manufacturing process” [2, p. 92]. Such failures
(e.g., a machine breakdown) can lead to serious consequences, such as missed delivery
dates, lost business, or significant expenditures for failure fixing [3]. To minimize the

2
negative consequences of manufacturing process failures, an effective data-driven fail-
ure management is necessary [4]. A critical requirement for, and first step toward, a
data-driven failure management pertains to the collection of accurate failure data, in-
cluding failure type, object, etc. In the short term, such collected failure data can help
workers to resolve failures faster (e.g., by using failure data to decide on the ‘right’
countermeasures). Further, in the mid to long term, manufacturing managers can use
collected failure data to develop models to predict future failures, as well as, to identify
and eliminate failure root causes [4]. However, even the first step (i.e., collection of
failure data) is already challenging for many manufacturing firms, given the partly se-
vere limitations of the solutions currently available.
Based on their automation level, existing data-collection solutions can be classified
into manual, semi-automated, and automated solutions. Generally, manual solutions
lead to high efforts, delays, and are error-prone [5]. In contrast, automated solutions
avoid key shortcomings of manual solutions, but tend to be very complex and expen-
sive, limiting the use of such solutions in the manufacturing industry, which is still
dominated by small and medium-sized enterprises (SMEs) [6]. In this regard, semi-
automated solutions represent a compromise between manual and automated solutions
[5]. Existing semi-automated solutions can be further classified into typing-based and
speech recognition-based solutions. Since the latter minimizes key shortcomings of the
former (e.g., worker distraction) [5, 7], speech recognition-based systems represent a
promising approach for the collection of failure data in the manufacturing industry.
Despite this potential, speech recognition-based data collection systems are not widely
used in the manufacturing domain yet [8]. Only a few studies propose the design of
speech recognition-based systems for the collection of failure data in this domain [7,
9]. However, these systems collect data along a fixed set of a few general failure attrib-
utes, which is problematic since relevant attributes can vary significantly across failure
types [4, 10]. To overcome this shortcoming and collect for each failure type specific
attributes a more complex underlying failure data structure is necessary, which is diffi-
cult to implement with the typically relational-based data structures used in existing
solutions. In this regard, an ontology-based data structure represents a well-suited al-
ternative to handle the complexity of manufacturing failure data [11].
Against this backdrop, the objective of this study is to design, instantiate, and eval-
uate an ontology-based speech recognition app as an assistance system for collecting
failure data in manufacturing processes. To do so, we follow the design science research
(DSR) cycle proposed by Sonnenberg and vom Brocke [12] and describe the respective
design and evaluation steps for the development of our solution artifact. In this context,
our study contributes design requirements and design principles for the instantiation of
such artifacts [13] and furthermore provides contributions to the body of knowledge in
IS. In this paper, we first outline the study foundations (section 2), followed by a presen-
tation of our DSR approach (3), the derived design requirements and principles (4), the
design of the main app artifact (5), the app artifact evaluation (6), and discussion (7).

3
2 Foundations and Related Work
2.1 Manufacturing Processes and Existing Solutions for Failure Data
Collection
Generally, manufacturing refers to processes that convert “raw materials into useful
products” [14, p. 69] by carrying out a set of activities. As already mentioned, manu-
facturing processes (e.g., casting or drilling) are becoming increasingly prone to fail-
ures (i.e., deviations from the optimal manufacturing process and its results [2]). In this
context, data-driven failure management is considered an effective strategy to avoid
such failures, or at least to minimize their negative impacts. As indicated above, exist-
ing solutions for the collection of manufacturing process (failure) data can be classified
into three main groups based on their level of automation [5].
Manual solutions rely on text- or spreadsheet-based forms and reports, or basic data-
entry terminals, and continue to be widely used for the collection of failure data, espe-
cially in the SME-dominated manufacturing industry. However, the high human de-
pendency of manual data collection procedures leads to high efforts and delays (e.g., if
the data has to be reentered into a software system later on), as well as to errors (result-
ing from subjectivity, negligence, or malice) [5]. In contrast to manual solutions, auto-
mated solutions (e.g., advanced sensor systems, camera-based vision systems) do not
rely on human-based data collection, thereby avoiding a key source of (future) data
problems. However, since automated solutions are often complex and difficult to setup
and maintain, manufacturing SMEs tend to lack the financial resources and internal
capabilities required to acquire and implement these solutions [5, 6]. Semi-automated
solutions still rely on manual data collection but use information technology to reduce
manual efforts by partly automating the data collection process (e.g., by offering menu-
based data entry) [5]. Since semi-automated solutions are not as difficult to implement
and maintain as automated ones (in terms of required capabilities and resources), they
tend to be well-suited for SMEs. Semi-automated solutions can be further divided into
typing-based and speech recognition-based solutions [5-7].
2.2 Typing-based vs. Speech Recognition-based Solutions
Generally, speech-recognition based solutions recognize spoken words, translate them
into character strings, and make these strings available for further data processing,
thereby avoiding several shortcomings of typing-based solutions. In particular, since
people talk faster than they type, speech recognition has a higher data collection speed.
Moreover, speech recognition avoids navigation through complex graphical menus and
instead provide the necessary information and parameters in just a few sentences [15].
Speech recognition is also more convenient, since keyboards for typing are often small
(e.g., on a smartphone screen). Keyboards also require visual attention and the use of
hands, leading to work interruptions and distractions. In addition, workers often wear
gloves, which makes typing difficult. Speech recognition avoids these shortcomings [7,
16]. As well, given their inherent flexibility, speech recognition systems are able to

4
collect very heterogeneous data (e.g., as opposed to sensors that can only measure pre-
defined values).
Given the above, earlier studies point to the use of speech recognition as a promis-
ing solution approach for the effective and efficient collection of failure data in the
context of manufacturing processes. For example, Camba et al. [17] present a system
that can record manufacturing product failure descriptions by speech and link them as
annotations to the relevant elements in computer-aided design (CAD) drawings. Their
system, however, just transforms the recorded failure descriptions into simple text
strings. To extract relevant data from the text strings, elaborated methods (e.g., text
mining) are necessary [18]. Also, the system does not verify that all relevant data for
the failure description is included in the annotation. Another example is Fischer et al.
[7] who propose a system where workers can record the location, cause, and handling
of each failure in a production line via speech recognition. Similarly, Sim et al. [9]
describe a system where workers can use speech recognition to collect data about ma-
chine failures (e.g., breakdown time, failure reason, etc.). In both systems [7, 9], only a
few, and always the same, attributes are captured for any failure (e.g., location, break-
down time, etc.). However, for an effective data-driven failure management, the man-
ufacturing process failure data collection should be tailored to the specific characteris-
tics of a given failure (i.e., for each failure, data about the failure type-specific attributes
should be collected) [4, 10]. For example, to describe a machine pump overheating,
other attributes will be relevant than for describing a “missing information” failure in
an assembly instruction. On this basis, we argue that speech recognition-based solutions
proposed in earlier studies fail to meet a key requirement of effective data-driven failure
management, as they lack the ability to collect failure type-specific data (and thus treat
any failure the same).
2.3 Ontologies as Data Model
For mapping data, a distinction can be made between relational and non-relational data
structures. Among the latter exist graph-based data structures that are well suited for
mapping complex data with many connections. Knowledge graphs that form the basis
of graph-based data structures consist of two nodes connected by a named edge (e.g.,
the named edge ‘is at’ connects the nodes ‘sawing machine 1’ and ‘workplace 1’). Sim-
plified, an ontology consists of multiple knowledge graphs [11, 19]. For ontology de-
scription, the Web Ontology Language (OWL) is widely used. OWL organizes data in
classes, individuals, and properties [11]. A class (e.g., ‘machine’) can have subclasses
(e.g., ‘sawing machine’) to which individuals (e.g., ‘sawing machine 1’) belong [19].
In terms of properties, a distinction can be made between object properties and data
properties. Object properties predefine how classes relate to each other (e.g., ‘worker’
work(s) on ‘machine’ and ‘machine’ is at ‘workplace’). Object property assertions then
connect the corresponding individuals within the classes to each other (e.g., ‘worker 1’
work(s) on ‘sawing machine 1’ and ‘sawing machine 1’ is at ‘workplace 1’). Data prop-
erties define the attributes that the individuals of a class can have, as well as, the attrib-
utes data type. Data property assertions then link the individuals and corresponding
attributes with their values [11, 19].

5
The mentioned named edges are useful for targeted querying, since nodes can have
multiple edges with different names. This enables inferring by querying along multiple
edges (e.g., worker 1 is at workplace 1). Moreover, ontologies enable reasoning, which
means initial rules for modelling the ontology can be defined and their compliance can
be checked (e.g. that certain classes must be connected by an object property). For on-
tologies exist the free modeling tool Protégé, which we used in this project and consid-
ered to be well usable also for novice users, since it allows graphical modelling (i.e.,
without coding) [11]. Moreover, from our experience, the SPARQL language, that we
used to query ontology data stored in Resource Description Framework (RDF) graph
databases, is well readable because the queries are based on the simple structure of
knowledge-graphs. Because of the described advantages, we decided to use ontologies
to model, store, and query the complex and highly networked manufacturing process
failure data structure within our system. Other researchers have also used ontologies as
data structure for their failure data collection tools for the same reasons (e.g., [20, 21]).
3 Research Approach
The overall objective of our study is to propose a new IT artifact for the collection of
process failure data, with a particular focus on the specific context of the SME-
dominated manufacturing industry. To do so, we follow a DSR approach, which is a
fundamental paradigm in IS research that concerns the construction of IT artifacts in
order to address organizational problems and deduce prescriptive design knowledge
[12, 13]. Specifically, our study follows the cyclic DSR process proposed by Sonnen-
berg and vom Brocke, which includes the activities problem identification, design, con-
struction, and use. Each activity is followed by an evaluation (see Fig. 1). Ex ante/ ex
post evaluations are conducted before/ after any artifact construction [12]. We choose
this DSR process since it provides an individual evaluation cycle after each design ac-
tivity. We consider these evaluation cycles of essential importance when developing a
design science artifact, since the artifact design is justified and validated even before it
is put into use. This ensures that the developed artifact continuously meets the user
needs and thereby provides solution for the problem class. Further, we followed a hu-
man risk & effectiveness evaluation strategy since the main design risk stems from the
perception of the artifact user [22]. We implemented this strategy with multiple forma-
tive evaluation cycles at the beginning and one summative evaluation workshop at the
end of our research project.
For this project, we collaborated closely with three SME manufacturers (SURF,
SHELF, and PROT) as part of a two-year, publicly-funded research project. The three
participating companies manufacture different products (SURF: surface coating ma-
chines; SHELF: store shelves; PROT: wood prototypes) and have a large number of
different failure types. Since these companies lack appropriate solutions to collect fail-
ure data, they were well suited for participating in our research project. In each com-
pany, there were two main project liaisons: the Heads of Production and Industry 4.0
at SHELF (referred to as SHELF1 and SHELF2); the Heads of Production and Assem-
bly at SURF; as well as the CEO and the CTO at PROT.

6
According to the design-evaluation cycle by Sonnenberg and vom Brocke [12] in
the first activity and prior to the artifact development itself, we identified problems of
existing solutions for manufacturing process failure data collection. This step is crucial
to be able to develop an artifact, which provides an improvement to existing ap-
proaches. Based on our identified shortcomings we were afterwards able to formulate
design requirements (DRs). To do so, we iteratively reviewed relevant literature (see
section 2), conducted a group workshop with all company representatives, and three
dedicated interviews with the liaisons of the three SME partner companies. This step is
considered as first evaluation (Eval. 1) and investigates whether the identified problems
are relevant and result from inabilities of existing solutions [12].
In the second design activity, we translated the identified and evaluated DRs into
design principles (DPs) for the IT artifact to be developed. For the DP formulation, we
reviewed relevant literature (e.g., [7, 15, 20]), conducted one interview per company,
as well as one group meeting with all company representatives. The formulation of DPs
was finalized when all company liaisons were satisfied. With this iterative procedure
we examined (Eval. 2) the clarity, feasibility, applicability, and internal consistency of
our DPs [12].
In the third activity, we constructed our app prototype based on the DPs. After this
construction process we initiated the ex post evaluation of our artifact (Eval. 3). Thus,
we conducted three interviews with the liaisons of each partner company to evaluate
the feasibility and suitability of specific app components and functions (e.g., of the final
ontology structure). Further, we conducted a workshop to investigate the practical ap-
plicability of our app prototype (see section 6).
The fourth activity – use – is not part of this research approach and will be conducted
in the future when one of the participating companies implement our app and provides
us with the respective results (Eval. 4). This will give us the opportunity for a long-term
evaluation of our app artifact in a naturalistic setting.
Fig. 1. Research process based on Sonnenberg and vom Brocke [12].

7
4 Design Requirements and Principles
In the first activity of our research process, we derived seven key DRs for our artifact,
based on the shortcomings of existing solutions.
DR 1: Minimize work interruptions for data collection. Our project liaisons de-
manded the to-be-developed solution should minimize work interruptions of existing
(in particular manual) solutions. To reach this, the time for the data collection on the
system itself should be minimized and it should be avoided that workers have to go to
a fixed terminal that is often not close to their actual work place [5].
DR 2: Increase correctness of collected data. Our project liaisons also explained
that due to high efforts for failure data collection, as well as, missing structures how
failures should be described in existing solutions, collected failure data is often not
correct, too superficial, or the failures are not even recorded at all [5].
DR 3: Limit implementation/ set-up costs and efforts. Automated solutions for col-
lecting manufacturing process failure data tend to be too expensive and require too high
implementation efforts for SME manufacturers [6] (e.g., SHELF was offered a software
solution that automatically collects machine failure data. However, this software is too
expensive).
DR 4: Enable capturing of failure type individual attributes. Existing solutions for
manufacturing process failure data collection (i.e., the speech recognition-based sys-
tems mentioned in section 2 and widely used standard enterprise resource planning
(ERP) systems [cf. 10]) usually lack the ability to capture failure type individual attrib-
utes (i.e., they collect for all failures the same few attributes).
DR 5: Minimize manual efforts for data retrieval. In existing solutions failure data
is often retrieved manually, leading to high efforts, delays, and errors. For example, at
PROT failures are noted in spreadsheets, from which information is manually retrieved.
DR 6: Operate in loud environment. Since we planned to use speech recognition in
manufacturing, the practitioners demanded that the to-be-developed artifact should
have a high accuracy also in noisy environments.
DR 7: Ensure a high data security and privacy level. A high data security and pri-
vacy level is crucial for manufacturing firms (e.g., to avoid industrial espionage) [23]
and should therefore be considered in the artifact design.
Based on the results of our first evaluation cycle, we were able to use the seven
identified DRs to derive six DPs, which meet the formulated DRs in the second activity
of our research process. For the DP formulation, we followed the recommendations of
Chandra et al. [24] and derived action and materiality oriented DPs that are both precise
and clear.
DP 1: Provide the artifact with a mobile device app to enable workplace-independ-
ent data collection and effortless, cost-effective setup. To reduce work interruptions
(DR 1), an app installed on mobile devices (e.g., smartphone or tablet) seems suitable,
since it avoids workers going to a fixed station. Further, mobile device apps limit set-
up efforts and costs (DR 3) since they can easily be installed, their handling is well
known from everyday life, and the required devices are financially affordable for
SMEs.

8
DP 2: Provide the artifact with an ontology-based data structure to map, model, and
query the complexity of manufacturing process failure data. To capture manufacturing
process failure data with their multiple failure (type individual) attributes (DR 4) a com-
plex data structure is necessary, which can be mapped by ontologies [cf. 20, 21]. Fur-
ther, ontology-based data structures can be modelled and queried with limited manual
efforts (see section 2; DR 3+5). Ontologies also enable inferring. For example, if the
failure location is captured, failure objects that are only possible for this location can
be inferred and presented to the user. Choosing only from such a limited number re-
duces work interruption and increases correctness (DR 1+2).
DP 3: Provide the artifact with a guided question-answer conversation flow to cap-
ture all failure type relevant attributes in a short time. A guided question-answer flow
enables the artifact to ask for the relevant attributes of the occurred failure type (DR 4).
Since no relevant attribute is missed out and the artifact only asks for relevant attributes,
correctness is increased while at the same time work interruption is minimized (DR
1+2).
DP 4: Provide the artifact with well selected components to ensure a high data se-
curity and privacy level. The artifact software and hardware components should be well
selected, to meet the high data security and privacy demands of manufacturing firms
(DR 7) [23].
DP 5: Provide the artifact with speech recognition backed by a Graphical User In-
terface (GUI) for data collection to reduce work interruptions but also enable data
collection in loud environments. To reduce work interruptions speech recognition
seems a suitable solution, given no/little visual attention is needed and the workers have
their hands free for their manual task (DR 1) [7, 15]. However, to account for risks of
speech recognition (e.g., reduced correctness in very loud environments), a GUI as
backup should be used (DR 6).
DP 6: Provide the artifact with report and query functions to minimize data retrieval
efforts. To do so (DR 5), query and report functions should automatically search for
relevant data and present large amounts of information in a standard format (e.g., in a
dashboard). These retrieval functions can be part of the app itself and/ or on another
device (e.g., a computer with a larger screen in a less noisy office off the shop floor).
5 Artifact Instantiation
Our design artifact is a blueprint for the instantiation of an ontology-based speech
recognition app prototype, whose key functions are the collection and storage of man-
ufacturing process failure data. In the following, we describe the instantiated artifact,
which is based on the derived DRs and DPs, with a particular focus on its two main
components, namely: the ontology-based data structure (data storage) and the speech
recognition-based user interface (data collection). In addition, we describe three tools
for data retrieval/analysis, which are embedded in our artifact. The developed app pro-
totype is Android-based, so it can be installed on different kinds of smartphones/tablets.
For the actual transformation of spoken words into machine-readable text, we decided
to use the Google speech-to-text API, since this API is free, has a high accuracy—also

9
in noisy environments (as indicated in relevant discussion forums), and used to be “of-
fline-capable” (mitigating data security and privacy concerns). Besides the speech in-
terface, the app has a GUI.
5.1 Data Storage: Ontology-based Data Structure
The ontology underlying our speech recognition app is used to structure relevant failure
data/ information. Our system contains an ‘initial’ ontology including 4 classes (failure
location, object, type, and collecting employee; see Fig. 2) and their connecting object
properties [20, 25]. This ‘initial’ ontology can be adapted to the particularities of the
using firm in the following ways. The four top classes of the initial ontology cannot be
changed. To the classes failure location and failure object can additional subclasses and
individuals be added in a hierarchical manner (e.g., the subclass drilling machine (DM)
with the individual DM DRB_0015). To the class failure type can only subclasses (e.g.,
cooling pump error; CPE) be added. To the class collecting employee can only individ-
uals be added, which each represent one employee. Moreover, classes that represent
attributes to describe specific failure types in a more detailed manner can also be created
(e.g., for CPE it is important to describe which project this failure type effects). In ad-
dition, object properties (assertions) connect the created (sub)classes and individuals.
Initially each failure type has to be connected to a failure object (e.g., a CPE can occur
at a DM), each failure object has to be connected to a failure location (e.g., DRB_0015
is at machine area 1), and failure type subclasses are connected to their individual at-
tribute subclasses (e.g., a CPE effects a project). We implemented the described ontol-
ogy structure requirements in reasoning rules. This ensures that the ontologies modeled
by the firms meet the requirements and that the app thus works.
If occurred failures are collected, this initial ontology is extended in the following
ways: For the failure type a new individual is created. Each individual represents one
occurred failure of the corresponding class/ failure type (e.g., Failure CPE1 is an oc-
curred CPE). The created failure type individual is connected to its failure object (for
Failure CPE1 this is DRB_0015); to its collecting employee (Failure CPE 1 was col-
lected by employer 1) and to its attribute that describes the corresponding failure type
in a more detailed manner (e.g., Failure CPE1 effects Project K001).
Fig. 2. Ontology-based data structure

10
5.2 Speech Recognition-based User Interface
Since the main task of the speech recognition-based user interface is the data collection,
we describe this component along the individual steps of the data-collection process.
Before the app can collect failures, the user has to choose his name and location. Then
the app is ready for the three-step data collection process. To start this process, the user
need to press a button on its headset. Then the app asks within the first step “What is
the failure object?”. The user’s response is processed by the speech-to-failure-object
algorithm, which compares the response with the failure objects that are theoretically
possible for the user’s location (i.e., objects whose individual names, class names, or
data property values are part of the preprocessed input words). Since objects in manu-
facturing often have abstract (technical) names, there exist data property values for ad-
ditional attributes. Therefore, our speech-to-failure-object algorithm searches not only
by individual or class names, but also by data property values (e.g., Homag as manu-
facturer). Ideally, the speech-to-failure-object algorithm will identify only (one)single
failure object; otherwise the speech-to-failure-object algorithm will generate a candi-
date list with all failure objects matching the provided information. The user can select
the appropriate failure object from that candidate list (by speech for a maximum of three
failure objects; otherwise using the GUI). If no failure object is identified, the user is
asked to use other words to describe the failure object. In the second step, the app asks
the user “What was the failure type?” and compares the user response with the failure
types possible for the failure object. If the app can identify the failure type, it will con-
tinue with the next step; otherwise, the user has to select the failure type via speech or
GUI (for more than three possible failure types). In the third and last step, the app asks
for individual attributes of the occurred failure type. In case of CPE1, the app would
ask the user which project was affected, and the user can then select ‘Project K001’.
5.3 Data Retrieval Tools
The collected data must be transmitted to a computer for retrieval. A company internal
Wi-Fi should be used to ensure data transmission in a fast, reliable, and secure way. On
the computer the data can be analyzed on a Power Bi dashboard or queried in an RDF
graph database using the SPARQL language. For short-term evaluation, the Power Bi
dashboard contains a list of all failures recorded on the current working day. For long-
term evaluation exist on the dashboard multiple diagrams (e.g., showing failure fre-
quency per failure object, location, type etc.). The querying enables a targeted data re-
trieval. For example, by inferring along the data property assertions it is possible to
query which failures on which machine areas effect a project or to query along class
hierarchies (e.g., all failures on individuals that belong to the class machine; Fig. 2).
Moreover, with the question "What happened?" the app enumerates to the user all fail-
ures recorded on the current day.

11
6 Artifact Evaluation
According to the build-evaluate cycle following Sonnenberg and vom Brocke [12] in
Fig. 1, we already evaluated the DRs (Evaluation 1) and the corresponding DPs (Eval-
uation 2) as ex ante evaluations based on previous findings, which were derived from
a systematic literature review and expert interviews. Based on these results, the current
research approach now focuses on the instantiation of the app prototype artifact, and
we further describe the ex post evaluation of the artifact against the DRs (Evaluation
3). To do so, we conducted a workshop with one of our company partners (SHELF).
The objective of this workshop was to evaluate the practical applicability of our app
prototype in a real-world setting [22]. Before the workshop, we developed a company-
specific ontology together with our main liaisons at SHELF. Three SHELF employees
participated in the workshop (hereafter referred to as SHELF 1-3), including one shop-
floor worker (SHELF 3), which is important, given that shopfloor workers are the des-
ignated app users. The workshop consisted of three parts: First, after explaining the
app’s technical components and their installation, we showed the workshop participants
how to use the Protégé tool for building a company-specific ontology, as well as how
to collect failure data with the app (see three-step process outlined in section 5.2). Fur-
thermore, we showed them the dashboard and how to query data using the SPARQL
language. In the second part, we asked each participant to use the app and to capture
multiple failures. As the participants got increasingly familiar with the app, the data
collection process (again, see section 5.2) worked fast and almost flawlessly. In the
third part, we visited SHELF's shop floor to assess whether the app can understand
speech input even in the presence of loud background noise. Throughout the workshop,
we kept asking the participants questions in relation to the different DRs and took de-
tailed notes of their answers/feedback (as well as of their app usage behaviors).
Within the workshop, the company participants confirmed that our app would enable
them to minimize work interruptions (DR 1). Here, they emphasized two aspects. First,
the app installed on a smartphone allows data collection directly at every workplace.
Therefore, workers no longer have to go to the fixed terminal in the production hall.
Second, SHELF 3 emphasized, that the developed app enables him almost hands-free
data collection. Therefore, he is able to continue tasks (e.g., load machine with material)
and collect failure data in parallel. The participants further highlighted the ability to
increase correctness and avoid incompleteness of collected data (DR 2), since instead
of writing in ‘free-text-fields’ only predefined attributes can be selected. Furthermore,
the participants confirmed that the implementation costs for our artifact are low since
they already have all required devices on-site (headsets) or they are affordable for them
(tablets/ smartphones). Therefore, SHELF is planning to use our app for machine fail-
ure tracking after the workshop since the “price of the [corresponding machine manu-
facturer] software alone is a criterion that leads to failure”. They also found that the
described set-up steps are doable with their IT expertise. The participants also con-
firmed the Protégé tool as useful for the modelling of ontologies since no coding is
required (DR 3). Further, the company participants highlighted that our app would en-
able them to capture failure type-specific data/information (DR 4), which represents a
significant improvement over their existing systems. For example, they can use our app

12
to specify the cause for the failure type “vacuum is not applied”, which could be “dia-
phragm defective, valve open, or vacuum pump defective” (SHELF 1). Further, the
participants confirmed that our artifact minimizes manual efforts and time delays to
retrieve and analyze collected data (DR 5). In the short run, SHELF 1 wants to monitor
actual occurred failures on the dashboard to assign workers to troubleshoot these fail-
ures. For long run evaluations, SHELF wants to query the collected failure data and
combine it with the tracked machine running times to “evaluate which failures typically
cause machine work interruptions” (SHELF 2). Moreover, we concluded that the
speech recognition of our app works well within noisy environments (DR 6) since the
app collected at the shopfloor volume test (fourth workshop step) 28 of 30 failures cor-
rectly (at 2 failures we spoke unclearly). For this test, we wore a noise cancellation
headset of the company BOSE and we measured a background noise level of 75 dB.
This type of headset is typically worn by the shop floor workers of SHELF. SHELF 2
also confirmed that since no internet connectivity is required and therefore no external
access to the data is possible our artifact ensures data security and privacy (DR 7).
Based on this feedback, we conclude that the instantiated app fulfills the seven DRs.
Thereby we were able to confirm these DRs for the intended artifact. Thereby, we fur-
thermore indirectly evaluated the six derived DPs since the instantiation, thus the de-
velopment of the app prototype, is based on them. Consequently, we concluded that if
the DRs are met in the instantiated artifact, the method of development, which relied
on the DPs, can be considered as being effective.
7 Discussion and Outlook
In this study, we followed a DSR approach [12] to present the design, development,
and evaluation of an SME tailored ontology-based speech recognition app as an assis-
tance system for the collection of failure data in manufacturing processes. Our study
contributes to the body of knowledge in IS in several ways. First, we contribute an
improved and relevant solution that solves shortcomings of existing solutions for man-
ufacturing process failure data collection (e.g., high efforts for failure data collection
and data retrieval) [13, 26]. In particular, we investigated the problem domain, formu-
lated seven DRs, derived six DPs and instantiated them in a software artifact. The de-
rived DRs and DPs can be transferred to other domains with a high proportion of man-
ual work and many different failure types (e.g., construction industry), that face the
same problem class of failure data collection. Second, since we instantiated the practi-
cal-oriented artifact build-evaluation design cycle [26] proposed by Sonnenberg and
vom Brocke [12] and furthermore followed the evaluation strategy presented by Vena-
ble et al. [22], as enhancements of traditional DSR approaches, we met a research gap
and the demand for “evidence-based design knowledge rooted in the design, implemen-
tation, and evaluation of real-world service systems” [27, p.74]. Third, we contribute
an ontology-structure that can be used by other researchers for the development of their
own failure ontology, in particular since our structure is able to map failure type specific
attributes. Overall, our study adds knowledge to the knowledge base with different lev-

13
els of inquiry: nomothetic design knowledge (the general design requirements and prin-
ciples), idiographic design knowledge (our individual artifact with all its components),
and nomothetic scientific knowledge (the general failure ontology structure with the
formalized relationships of failure characteristics) [28].
As with any research, our study has several limitations. First, the app evaluation is
limited to a set of rather generic criteria. Thus, further evaluations are needed, including
a more detailed comparison with existing (manual) solutions for the collection of man-
ufacturing process failure data (e.g., in terms of total collection efforts). Also, since the
evaluation only involved one workshop, our study would benefit large-scale evalua-
tions on the shop floor. As mentioned, SHELF is planning to introduce our app, which
would give us the opportunity for such an evaluation. We could get feedback from
multiple workers, who would use our artifact. For such evaluation could serve the Tech-
nology Acceptance Model as theoretical foundation [29]. Second, the evaluation scope
in our study is on technical and functional aspects. Further evaluations should also focus
on organizational aspects (e.g., app acceptance). In this context, we would like to en-
courage future studies on the topic of speech recognition in manufacturing in general.
For example, a longitudinal study might help clarify the specific characteristics of
speech recognition that influence its acceptance, continuous use, and efficiency.
Furthermore, all three case firms saw a high relevance in an additional function
where in particular new workers with little experience can access troubleshooting in-
structions via the app and follow them while solving the failure. Further, such a function
is relatively easy to realize based on the ontology-based data structure, since the in-
structions can be assigned to the respective failure types in the ontology.
In conclusion, we hope that our results can serve as a reference point for future re-
search on the design and use of ontology-based speech recognition apps for manufac-
turing process failure data collection.
References
1. Bokrantz, J., Skoogh, A., Ylipää, T., Stahre, J.: Handling of production disturbances in the
manufacturing industry, Journal of Manufacturing Technology Management 27 (8), 1054-
1075 (2016).
2. Bergers, T., Stauder, L., Grünbaum, T., Barth, S.: Adaptive design of manufacturing process
sequences in case of short-term disruptions in the production process. Manufacturing Letters
27, 92-95 (2021).
3. Servicemax, https://www.servicemax.com/de/unplanned-downtime, last accessed
2020/07/05.
4. Stich, V., Jordan, F., Birkmeier, M., Oflazgil, K., Reschken, J., Diews, A.: Big Data Tech-
nology for Resilient Failure Management in Production Systems. In: Umeda, S., Nakano,
M., Mizuyama, H., Hibino, N., Kiritsis, D., von Cieminski, G. (eds.) Advances in Production
Management Systems: Innovative Production Management Towards Sustainable Growth.
APMS 2015. IFIP Advances in Information and Communication Technology, vol 459,
pp.447-454. Springer, Cham (2015).
5. Cwikla, C.: Methods of manufacturing data acquisition for production management – a re-
view. Advanced Materials Research 837, 618-623 (2014).

14
6. Ghmire, S., Melo, R., Ferreira, J., Agostinho, C., Goncalves, R.: Continuous Data Collection
Framework for Manufacturing Industries. In: Ciuci, I., Panetto, H., Debruye, C., Aubry, A.
et al. (eds.) On the Move to Meaningful Internet Systems: OTM 2015 Workshops, LNCS,
vol. 9416, pp. 29-40, Springer, Cham (2015).
7. Fischer, J., Pantförder, D., Vogel-Heuser, B.: Improvement of Maintenance through Speech
Interaction in Cyber-Physical Production Systems. In: IEEE 15th International Conference
on Industrial Informatics (INDIN), pp. 290-295, IEEE, Emden, Germany (2017).
8. Cloudflight, https://de.cloudflight.io/presse/hallo-iot-warum-scheitert-die-konversation-
mit-dingen-der-industrie-35368/, last accessed: 2020/07/05
9. Sim, E.-S., Lee, H.-G., Lee, J.-C., Park, J.-W.: Efficient work measurement system of man-
ufacturing cells using speech recognition and digital image processing technology. The In-
ternational Journal of Advanced Manufacturing Technology 29(7), 772-785 (2006).
10. Westkämper, E., Löffler, C.: Strategien der Produktion. Technologien, Konzepte und Wege
in die Praxis, Springer, Berlin, Heidelberg (2016).
11. Stuckenschmidt, H.: Ontologien, Springer, Berlin, Heidelberg (2009).
12. Sonnenberg, C., vom Brocke, J.: Evaluations in the science of the artificial – reconsidering
the build-evaluate pattern in design science research. In: Peffers, K., Rothenberger, M.,
Kuechler, B. (eds.) DESRIST 2012, vol. 7286, pp. 381–397. Springer, Heidelberg (2012).
13. Gregor, S., Hevner, A.R.: Positioning and Presenting Design Science Research for Maxi-
mum Impact. MIS Quarterly: Management Information Systems 37, 337–355 (2013).
14. Dahotre, N.B., Harimkar, S.P.: Manufacturing Processes. In: Laser Fabrication and Machin-
ing of Materials, pp. 69-96. Springer, Boston (2008).
15. Kasper, G. M., Morris, A. H.: The Effect of Presentation Media on Recipient Performance
in Text-based Information Systems. Journal of Management Information Systems 4(4), 25–
43 (1988).
16. Udoka, S.J.: Automated Data Capture Techniques: A Prerequisite for Effective Integrated
Manufacturing Systems. Computers & Industrial Engineering 21, 217-221 (1991).
17. Camba, J., Naya, F., Perez-Lopez, D., Contero, M.: From Voice to Knowledge: A Proposal
for a Voice Annotation System to Support Collaborative Engineering Design Processes. In:
Proceedings of the 53th HICSS, pp. 385-394, IEEE, Maui, Hawaii, USA (2020)
18. Allahyari, M., Pouriyeh, S.A., Assefi, M., Safaei, S., Trippe, E.D., Guttierez, J., Kochut, K.
A brief survey of text mining: classification, clustering and extraction techniques In: Pro-
ceedings of KDD bidgas, Association for Computing Machinery, New York (2017).
19. W3C, https://www.w3.org/TR/owl-ref/#Individual, last accessed 2021/04/10.
20. Ali, N., Hong, J.-E.: Failure Detection and Prevention for Cyber-Physical Systems Using
Ontology-Based Knowledge Base, Computers 7(4), 1-16 (2018).
21. Park, C.-S., Lee, D.-Y., Kwon, O.-H., Wang, X.: A framework for proactive construction
defect management using BIM, augmented reality and ontology-based data collection tem-
plate, Automation in Construction 33, 61-67 (2018).
22. Venable. J., Pries-Heje, J., Baskerville, R.: FEDS: a Framework for Evaluation in Design
Science Research. European Journal of Information Systems 25(1), 77-89 (2016).
23. Deloitte, https://www2.deloitte.com/content/dam/Deloitte/us/Documents/manufacturing/
us-manu-cyber-risk-in-advanced-manufacturing.pdf, last accessed 2020/12/06.
24. Chandra, L., Seidel, S., Gregor, S.: Prescriptive knowledge in IS research: conceptualizing
design principles in terms of materiality, action, and boundary conditions. In: Proceedings
of the 48th HICSS, pp. 4039–4048. IEEE, Kauai, Hawaii, USA (2015).
25. Meyer, G., Knüppel, K., Busch, J., Jakob, M., Nyhuis, P.: Störgrößenmanagement-Systema-
tik. Zeitschrift für wirtschaftlichen Fabrikbetrieb 109(10), 704-707 (2014).

15
26. Hevner, A. A Three Cycle View of Design Science Research. Scandinavian Journal of In-
formation Systems 19(2), 87-92 (2007).
27. Böhmann, T., Leimeister, J.M., Möslein, K.: Service Systems Engineering. Business & In-
formation Systems Engineering 6(2), 73-79 (2014).
28. Baskerville, R., Kaul, M., Storey, V. Genres of Inquiry in Design-Science Research: Justi-
fication and Evaluation of Knowledge Production. MIS Quarterly: Management Information
Systems 39(3), 541-564 (2015).
29. Davis, F.D.: Perceived Usefulness, Perceived Ease of Use, and User Acceptance of Infor-
mation Technology. MIS Quarterly: Management Information Systems 13(3), 319-340
(1989).