Content uploaded by Alexander Maedche
Author content
All content in this area was uploaded by Alexander Maedche on Feb 26, 2016
Content may be subject to copyright.
RESEARCH PAPER
The Impact of Process Visibility on Process Performance
A Multiple Case Study of Operations Control Centers in ITSM
Martin Berner
•
Jino Augustine
•
Alexander Maedche
Springer Fachmedien Wiesbaden 2015
Abstract Successful monitoring is essential for managing
security-critical or business-critical processes. The paper
seeks to understa nd and empirically evaluate benefits of the
BPM use case ‘‘monitor’’ in the context of Operations
Control Centers (OCCs). OCCs create visibility about criti-
cal events and statuses in very sensitive processes. In IT
Service Management (ITSM) they support the event man-
agement process with real-time monitoring and event anal-
ysis of critical systems in complex system landscapes. This
special focus of OCCs on visibility is a promising context to
study fundamentals of process visibility. The paper develops
a Process Monitoring Benefits Framework that draws on the
Situation Awareness Theory and the Theory of Constraints.
The authors conceptualize process visibi lity and suggest that
it is positively related to process performanc e. A multiple
case study in seven organizations is carried out to examine
the framework and its propositions. The case study indicates
that the impact of process visibility on process performance
is mediated by the situation awareness of the proce ss par-
ticipants as well as the identification of bottlenecks in pro-
cesses. Moreover, factors are identified that potentially
influence process visibility outcome – namely continuous
improvement culture, outsourcing quality, and maturity of
the software tool used for monitoring.
Keywords BPM use case monitor Process visibility
Continuous improvement Situation awareness ITSM
event management
1 Intro duction
Huge benefits are expected from data assets created by
advanced information technologies that enable new ways
of data capturing, storing, managing, and analyzing
(Manyika et al. 2011). In business process management
(BPM) such data assets are most important for the use case
monitor which refers to data measurements for decision
support during process execution (van der Aalst 2013).
This monitoring of business processes is relevant to sup-
porting continuous improvement as well as day-to-day
operations. Accordingly, process monitoring is an essential
and common element in lifecycle models that define the
managerial practices of BPM, although in these models
process monitoring is sometimes also referred to as process
control, evaluation, or diagnosis (Morais et al. 2014).
Hence, BPM software vendors and analysts increasingly
focus on the monitor use case: for example Gartner (2012)
stresses the importance of integrating state-of-the-art ana-
lytics into operational processes under the label intelligent
Business Process Management Suites (iBPMS), and Rus-
som (2013) proposes the term Real-time Operational
Intelligence which describes ‘‘an emerging class of
Accepted after two revisions by the editors of the special issue.
Electronic supplementary material The online version of this
article (doi:10.1007/s12599-015-0414-0) contains supplementary
material, which is available to authorized users.
Dipl.-Hdl. M. Berner (&) J. Augustine A. Maedche
Institute of Enterprise Systems (InES), University of Mannheim,
L15, 1-6, 68131 Mannheim, Germany
e-mail: berner@es.uni-mannheim.de
J. Augustine
e-mail: jaugusti@mail.uni-mannheim.de
A. Maedche
Institute of Information Systems and Marketing (IISM),
Karlsruhe Institute of Technology (KIT), Fritz-Erler-Straße 23,
76131 Karlsruhe, Germany
e-mail: alexander.maedche@kit.edu
123
Bus Inf Syst Eng
DOI 10.1007/s12599-015-0414-0
analytics that provides visibility into business processes,
events, and operations as they are happening’’. The
underlying assumption in this discussion regarding
sophisticated process monitoring and next generation
intelligent BPM is that higher process visibility ultimately
leads to highe r process performance. However, currently it
remains vague how increased process visibility actually
contributes to process performance.
The performance impact of visibility is intensively
studied in the field of supply chain management (SCM).
Visibility is identified as an essential contributor to SCM
process performance and its degree depends on the level to
which the accessible information is relevan t, trustworthy,
and timely (Swaminathan and Tayur 2003; Barratt and Oke
2007).
Besides SCM, lean production literature stresses the
importance of making information visible during process
operation (Womack and Jones 2003). Visual controls that
create immediate transparency about abnor malities are a
crucial part of lean production systems (Shingo 1989), and
they are essential for banishing waste to continuously
improve processes (Womack and Jones 2003).
Recent research generalizes the visibility concepts of
SCM and lean production to a broader business process
context (Klotz et al. 2008; Pidun et al. 2011; Graupner
et al. 2014). Based on these foundations, this paper
understands process visibility as a characteristic of a pro-
cess that describes the quality of information to support
process operation and im provement.
For processes where visibility is of utmost importance,
we see the use of Operations Control Centers (OCCs).
Most prominent OCCs are the area control centers used to
manage the air-space in aviation and control rooms in the
energy sector.
To a large degree, visibility has the same importance for
IT Service Management (ITSM) because in a growing
digitalized world the IT infrastructure is the backbone for
every kind of business. ITSM is a process-oriented
approach to managing IT services, and the most important
element of monitoring in ITSM is referred to as end-to-end
visibility (OGC 2007a). System and service downtimes can
result in serious regulatory liabilities or accumulate up to
multi-million dollar costs (Martinez 2009). Therefore,
OCCs are increasingly implemented to support the ITSM
event management process by providing visibility via real-
time monitoring of business critical systems and processes
(EMC 2012; SAP 2013). Such OCCs are typically physical
rooms where IT operators jointly carry out this monitoring
and big screens show the operational status of the IT
environment and the managed processes with the objective
to detect and solve issues before business is affected.
The business process under investigation in our study
about OCCs is the ITSM event management process, which
deals with the monitoring and systematic management of
alerts originating from the observed IT infrastructure. It is a
‘‘loosely framed process’’ (van der Aalst 2013) where a
process model typically describes the standard way of
doing things, but actual executions can deviate.
OCCs with their special focus on monitoring in ITSM are
a promising research arena to study the impact of process
visibility on process performance and its influencing factors.
Hence, we empirically examined the ITSM event man age-
ment process of seven organizations that recently introduced
an OCC and implemented a new software package for it. Our
paper studies fundamentals of process visibility in the con-
text of OCC s, but is guided by a more general research
question regarding monitoring benefits:
How does process visibility influence process
performance?
Our work contributes to empirical BPM research in the
context of the outlined highly relevant use case monitor,
which is currently underrepresented in the BPM literature
(van der Aalst 2013). We develop a Process Monitoring
Benefits Framework that seeks to describe how process
visibility impacts process performance. It builds on a
conceptualization of process visibility, its impacts, and
influencing factors based on existing literature. To empir-
ically examine the proposed framework we carry out a
positivist multiple case study in several companies. The
results of our empirical examination lead to a refined and
extended framework.
2 Conc eptual Foundations
2.1 Process Monitoring Benefit Dimensions
As outlined before, the intended benefit of process moni-
toring is to gain process visibility with the ultimate
objective to increase process performance. Thus, the
dependent variable of our study is process performance.
We argue that the process as unit of analysis is favorable to
evaluate the net benefit of process monitoring, as it chooses
the unit that process monitoring affects directly and at
which its impact is best observable and measurable. There
are two classical approaches to define process performance
(Ray et al. 2005): First, based on productivity measures
such as throughput time, and secon d, based on the quality
of the process output. The latter is adopted in our study
since in OCCs the output is significantly more important
than the productivity of the event management. In our
context of ITSM event management the quality of the
process output is defined by the creation of a reliable ser-
vice asset and the minimization of system downtimes
(Cater-Steel and McBride 2007). In other words, process
performance in OCC context can be determined by the
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
service quality of ITSM and the system quality of the
managed systems (see online Appendix B for more details
about the conceptualization of these qualitie s in our study).
We argue that the impact of process visibility on process
performance is mediated by situation awareness of the
process operators as well as bottleneck identification for
continuous process improvement. Therefore, we differen-
tiate and introduce the constructs process visibility, situa-
tion awareness, and bottleneck identification in the
following.
2.1.1 Process Visibility
We already introduced process visibi lity as an information
quality in respect to operating and improving a process. In
our conceptualization we leverage information quality as
one of the key constructs of the D&M IS Success Model
(DeLone and McLean 2003). It is a multi-dimensional
construct determined by accuracy, completeness, currency,
and format of information (Nelson et al. 2005). In this
regard, visibility should not be confused with visualization
because the representation format of information is only
one aspect of it.
Information quality in the D&M IS Success Model is a
characteristic of an information system whereas process
visibility is defined as a characteristic of a process. Pro-
cesses as unit of analysis are beneficial because the orga-
nizational benefits of IT are mediated by business
processes (Melville et al. 2004 ). Therefore, we suggest to
derive the process visibility dimensions from inf ormation
quality dimensions by putting them in a process informa-
tion context. Information that plays a supporting role in
process operation and improvement is called process
information (Davenport 1993).
In addition, information quality is influenced and inter-
linked with system quality (Xu et al. 2013). Hence, also
several system quality dimensions of the monitoring sys-
tem itself may contribute to the level of process visibi lity –
namely accessibil ity, flexibility, and integration (Nelson
et al. 2005). Process monitoring systems intend to improve
these dimensions explicitly, while other system qualities
(reliability and response time) are of generic relevance and
therefore no specific dimensions of process visibility. The
quality of the monitoring system itself (which is defined
here as an antecedent of process visibility) should not be
mistaken with the quality of the systems that are monitored
by an OCC (which we defined above as a criteria for
process performance in the ITSM event management
context).
Table 1 summarizes all identified and relevant dimen-
sions of process visibility and defines them based on Nel-
son et al. (2005) and Berner et al. (2012).
2.1.2 Situation Awareness (SA)
In the moni toring use case the impact of process visibility
on process performance depends on the operators who do
the monitoring. They have to permanently classi fy and
understand situations, basically they need to know ‘‘what’s
going on’’ (Endsley 1995). Cognitive psychology identified
situation awareness (SA) as crucial concept for operators’
decision outcome: In the context of control rooms the
phenomenon of SA in highly dynamic environments is
intensively studied based on the SA Theory (Endsley 1995)
for the domains of air traffic cont rol (e.g., O’Brien and
O’Harea 2007) and nuclear power plants (e.g., Hogg et al.
1995). Similar to IT support team members, ‘‘the operator
of a nuclear power plant must have knowledge of the
current process state at all times, and the ability to use this
knowledge effectively in predicting future process states
and controlling the process to attain operational goals’’
(Hogg et al. 1995, p. 2394). SA is defined as ‘‘the per-
ception [Level 1] of the elements in the environment within
a volume of time and space, the comprehension [Level 2]
of their meaning and the projection [Level 3] of their status
in the near future’’ (Endsley 1995, p. 36). These coherent
levels of SA are outlined in more detail in Table 2.
Table 1 Dimensions of process visibility
Dimension Definition (based on Nelson et al. 2005; Berner et al.
2012)
Accuracy The degree to which process information is correct,
unambiguous, meaningful, consistent, and
trustable (perceived to be valid, reliable and objective
and a positive attitude is embraced towards the
source)
Completeness The degree to which all possible process states and
other information relevant for the process
participants are represented
Currency The degree to which process information is up-to-
date, or the degree to which the information precisely
reflects the current state of a process instance
Format The degree to which process information is presented
in a manner that is useful, readily useable,
analytically interpreted, and contextualized (centered
on process steps and is set into relation with previous
and adjacent process steps)
Accessibility The degree to which process information can be
accessed by the process participants with relatively
low effort
Flexibility The degree to which process information analysis
and representation can adapt to a variety of process
participants needs and to changing conditions
Integration The degree to which process information is available
for the entire process by facilitating the combination
of information from various sources to support
decisions
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
SA can be analyzed on individual as well as on team
level (Endsley 1995). The involved teams and individuals
in ITSM event management process are the operators and
managers of the IT support team who we call subsequently
process participants. SA research identified system factors,
particularly system design in terms of how the needed
information is provided, as an important driver of SA
(Endsley 1995). These system factors are reflected to a
large degree also in our conceptualization of process visi-
bility, and thus we analogously propose:
[P1] The higher the level of process visibility, the higher
the SA of the process participants.
SA describes a very important antecedent for operators
to make better decisions and take appropriate actions
(Endsley 1995). Therefore, we conclude
[P2] The higher the level of process participants’ SA, the
higher the process performance.
2.1.3 Bottleneck Identification
Besides supporting daily process operations, process
monitoring additionally aims to provide the informational
baseline to improve processes. Existi ng research recog-
nizes the importance of information visibility of business
processes for identification of process bottlenecks (Cot-
teleer and Bendoly 2006). The concept bottleneck identi-
fication of Cottele er and Bendoly (2006) is based on the
Theory of Constraints (Goldratt and Cox 1992), which
claims that process bottlenecks hinder highe r process per-
formance due to physical or managerial constraints
(Table 3).
Hence, the level of bottleneck identification of a process
is defined by the degree to which physical and managerial
constraints of a process are recognized by the process
participants. In summary, we sugges t:
[P3] The higher the level of process visibility, the higher
the level of bottleneck identification of a process.
[P4] The higher the level of bottleneck identification, the
higher the process performance.
2.2 Influencing Factors
Neither new monitoring tools nor pote ntially resulting
higher situation awareness or bottleneck identification can
guarantee better process performance. There are additional
influential factors in key areas where ‘‘things must go
right’’ (Iden and Eikebrokk 2013) in order to gain benefits
of process monitoring. In our context a systematic literature
review by Iden and Eikebrokk (2013) identified several
influencing factors for ITSM success. Particularly, staff’s
skills and knowledge as well as willingness to change
might be important for monitoring and are outlined in the
following, because they directly link to process operation
and improvement.
First, a crucial aspect in ITSM processes are skills and
knowledge of the IT professionals (Galup and Dattero
2010). Th us, skills and knowledge of operators is one
potential factor that moderates the impact of process visi-
bility o n process performance in OCC context. Skills are
commonly defined as acquired cognitive or metacognitive
competency that develops with training and/or practice
(McCombs and Marzano 1990). Likewise, SA theory rec-
ognizes experience and training as individual factors
influencing SA (Endsley 1995). In conclusion, we propose:
[P5] The lower the skills and knowledge of the operators
who monitor the process, the lower is the situation
awareness (whic h lowers the impact of process visibility on
process performance).
Second, successful ITSM requires a Continuous
Improvement (CI) culture that welcomes changes and
improvements (OGC 2007b). The culture of a group
manifests itself at three different levels (Schein 2004):
artifacts (e.g., structures), values (e.g., strategies), and
underlying basic assumptions. The shared basic assump-
tions on the deepest level are most difficult to observe, bu t
represent the biggest part of an organizational culture.
Table 2 Levels of SA
Level Definition (Endsley 1995)
Level 1 SA
(perception)
The degree to which an operator or operation
teams perceive the status, attributes, and
dynamic of relevant elements in the
environment
Level 2 SA
(comprehension)
The degree to which an operator or operation
teams are able to understand the significance of
elements in the environment in the light of his/
her goals based on his/her level 1 perception
Level 3 SA
(projection)
The degree to which an operator or operation
teams are able to project the (near) future based
on his/her level 2 comprehension
Table 3 Dimensions of bottleneck identification
Dimension Definition (based on Goldratt and Cox 1992)
Physical constraint
identification
The degree to which physical constraints
such as materials, machines, people and
demand that limit a process from achieving
higher performance versus its goal are
recognized
Managerial constraint
identification
The degree to which managerial constraints
in the form of policies, procedures, rules and
methods that limit a process from achieving
higher performance versus its goal are
recognized
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
Organizations with a strong CI culture are more likely to
seek out for new bottlenecks as others are solved (Goldratt
and Cox 1992). Thus, we conclude:
[P6] The lower the CI culture of the organization, the
lower is the bottleneck identification (which lowers the
impact of process visibility on process performance).
The derived propositions P1–6 are visualized in Fig. 1.
In the course of our study the initial conceptual framework
was enhanced by additional influencing factors and
propositions (P7–8) based on the results of our empirical
study which are described subsequently.
3 Research Methodology
3.1 Introduction
We follow a multiple case study research approach to
explore our propositions in multiple firms. This enables
us to treat each case as an empirical test of our proposed
framework and ensure generalizability by applying
replication logic (Yin 2003). We follow the widely
accepted positivist case study perspective of theory
testing (Dube
´
and Pare
´
2003). Additionally, the qualita-
tive approach enables adoption or po tential theory
extensions in an exploratory manner (e.g., Dibbern et al.
2008).
Our study is done in cooperation with the software
company SAP SE. Recently, the OCC concept has been
integrated into the RunSAP like a Factory methodology
which is SAP’s approach to operate and continuously
improve the operations of SAP and non-SAP IT landscapes
(SAP 2013). In this methodology the OCC is positioned as
a central IT support entity at the customer sites to monitor
the status of business processes and IT landscape compo-
nents. SAP implements OCCs based on their software tool
SAP Solution Manger and recommends to set up a physical
room for the OCC including large screens (SAP 2013).
SAP supported our study by providing assistance in
establishing contact to the organizations where such OCCs
have been implemented.
3.2 Case Selection
We began the case selection by classifying potential
companies with finished OCC implementations based on
data from a customer database of our industry partner –
including organization size, geographic locations, com-
plexity of the event management process, etc. (Table 4).
Additionally, we asked managers in the support orga-
nization of SAP SE, who have broad overview of different
OCC implementations, to give a rough estimate of the
OCC success in the potential cases. These managerial
perceptions helped to select a case mixture with more and
less successful OCC implementations – potentially result-
ing in lower and higher process visibility levels. Thus, we
applied literal and theoretic al replication strategies to
ensure external validity of our research (Yin 2003). First,
theoretical replication requires a selection of cases that
vary in their characteristics and thus in their proposed
impact. Therefore, our cases shall have different degrees of
process visibility. Second, literal replications refer to sim-
ilar cases and accordingly leading to similar proposed
outcomes. Thus, we need multiple cases with the same
process visibility level.
Process
Visibility
Process
Performance
Situation
Awareness
Process
Improvement
Bottleneck
Identification
Process
Operation
Process Monitoring Benefit Dimensions
Influencing Factors
Skills and
Knowledge of
Operators
Continuous
Improvement
(CI) Culture
Outsourcing
Quality
P1
P3
P2
P4
P5
(not confirmed)
P6 P7
Tool
Maturity
P8
Fig. 1 Process monitoring
benefits framework
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
3.3 Data Sources and Analysis
After the case selection based on the customer database and
the high-level assessment of the support managers, we
conducted 17 semi-structured interviews. Before the
interviews a questionnaire with open-ended questions
along our conceptual framework was created (online
Appendix D). The questionnaire was used as a generic
baseline for the interviews, and additional questions were
asked during the individual discussions (Myers and New-
man 2007).
We executed one comprehensive interview with the
OCC team leads (TL) from every case organization and
one further interview with the technical quality manager
(TQM) from our industry partner who support the respec-
tive case organizations. By doing so, we were able to get
perspectives on our cases from inside and outside the
affected companies. Additionally, for three cases operators
were interviewed in order to consi der als o the perspective
of the operational workers. All interviews were conducted
by the first two authors of the paper.
Beyond interview data, the authors had access to a
customer data base of our industry partner that contained
information of the OCC implementations such as imple-
mentation challenges, number of alert events, and hours of
unplanned downtimes of the monitored systems. These
information were used for the detailed preparation of the
individual interviews and to triangulate the interview
results.
Data analysis consists of examining, categorizing, tab-
ulating, testing quantitative and qualitative evidence to
address the propositions (Yin 2003). With this aim, we
applied open coding and axial coding techniques (Corbin
and Strauss 2008) supported by the coding software
MAXQDA.
In order to mitigate potential bias and improve coding
reliability, the authors encoded the interviews in an itera-
tive dual coding approach as follows: First, all transcrip ts
were encoded independently by two coders based on a
codebook that explained the code system and how the
codes should be applied. Second, always after 3–4 dual
encoded transcripts the mismatches were discussed by the
authors. If inter-coder reliability was below 85 %, the
codebook was adjusted and the affected transcripts were
encoded again. In total an inter-coder reliability according
to Holst i (1969) of 91 % was achieved (Table 5), which is
beyond the recommended reliability threshold for textual
content analysis of 85 % (Kassarjian 1977).
Hence, in multiple iterations we identified information
that is linked to our conceptual framework and adjusted the
code system if required. Additionally the weight feature of
MAXQDA was used to document whether a coded seg-
ment is an indication for a low, medium or high level of a
variable in our model (online Appendix C). This helped to
assess the overall level of a variable in a case relatively to
the other cases.
4 Results
The level of process visibility in the ITSM event man-
agement process of all the case sites was found to be low
prior to the OCC implementation. In fact the motivation for
these organizations to opt for an OCC was the high effort
required to monitor and assess the system statuses. To
compartmentalize the benefits of the enhanced process
monitoring, we specifically present results with regards to
process visibility, situation awareness, bottleneck identifi-
cation, and process performance after OCC implementa-
tion. Furthermore, we describe influential factors that can
explain differences between the cases.
4.1 Level of Process Visibility
The results show that there are positive impacts regarding
process visibility in the organizations as a result of OCC
implementation. It came to light in the process of the
interviews that cross-system monitoring was performed
manually and sporadically before OCC implementation.
With the automation of system monitoring the process
Table 4 Descriptive case data
Case
Company
Industry Number of
employees
Region Offshore outsourcing
partner
Months since
OCC go-live
Managerial perception of
OCC implementation
A Retail [40k Europe significant involvement 20 Expectations met
B Manufacturing [60k North America No 15 Expectations not met
C Finance [50k Europe No 22 Expectations not met
D Finance [20k Europe No 22 Expectations met
E Manufacturing [20k North America Significant involvement 23 Expectations not met
F Manufacturing [70k Europe Significant involvement 2 Expectations not met
G Energy [10k North America Minor involvement 18 Expectations exceeded
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
visibility in event management process increased. After
OCC implementation most cases (A, C, D, E, G) show high
degree of process visibility examined against dimensions
of accuracy, completeness, currency, format, accessibility,
flexibility and integration. This is exemplified by the fol-
lowing interv iew quote (The coding corresponding to the
process visibility dimension is added in squared brac kets to
the quote):
‘‘We are now able to discuss with our service pro-
vider on the same level. This information was not
available to us before OCC [Accessibility]… The
vendor worked in one direction based on the infor-
mation they had and we worked in another direction
based on the information we had [Integration]… This
is a huge gain for us… it was very unaligned and also
the root cause analysis deliveries coming from ser-
vice vendor were taking very long time, perhaps even
as long as one to two months, and this is of course not
good [Currency]’’ (TL Case A).
Cases B and F were identified to have a relative lower
level of process visibility as they showed more lacks in the
dimensions of process visibility. The TL of organization B
reported for example the following shortages:
I think there is more available than what we currently
have [Completeness], but then also we have some
underlying issues of the reliability of being able to
keep the system managed and sending us good
alerts… Some of the information on alerts has to be
created from manual interventions [Format]… There
are scenarios where we get alerts too late [Currency]’’
(TL Case B).
4.2 Level of Situation Awareness
With the OCC some process participants (cases A and G)
experienced substantial gains in situation awareness. The
information that was formerly not easily available can now
be accessed in real-time, which helps the IT support team
to raise their perception of critica l situations and mitigate
issues. Example interview excerpts in this regard are:
‘‘We have information on the performance, avail-
ability, database issues and what not. When you
receive an alert, we can investigate instantly to make
sure that it is a real alert [Comprehension]’’ (TL Case
G).
‘‘We can get total information about the alert [Per-
ception] and we can take action [Comprehension] in a
short span of time and resolve the issue… which is
the main comparison before and after OCC imple-
mentation… In the last half quarter the benefit came
back to us where we actually – before the system
went down, catch the iss ue… We are extremely
Table 5 Inter-coder reliability
Interview transcript Total number
of coded
segments in
agreement
Total number of
coded segments
Percentage
inter-coder
reliability
(Holsti 1969) (%)
1. Case A Interview TL 70 77 91
2. Case A Interview TQM 24 28 86
3. Case A Interview Operator 22 25 88
4. Case B Interview TL 64 73 88
5. Case B Interview TQM 8 9 89
6. Case C Interview TL 70 73 96
7. Case C Interview TQM 20 21 95
8. Case C Interview Operator 36 38 95
9.a) Case D Interview TL 38 41 93
9.b) Case D Written Response TL 32 35 91
10. Case D Interview TQM 52 58 90
11. Case D Interview Operator 32 35 91
12. Case E Interview TL 118 129 91
13. Case E Interview TQM 28 30 93
14 Case F Interview TL 62 71 87
15. Case F Interview TQM 38 44 86
16. Case G Interview TL 110 116 95
17. Case G Interview TQM 38 43 88
Total 862 946 91
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
pleased because these are proactive actions, not
reactive [Projection]’’ (Operator Case A).
Whereas in other cases still bigger deficiencies of situ-
ation awareness are reported, for example:
‘‘We have missed critical events… if that person
doesn’t look at that inbox within 15 minutes, that
event won’t be addressed [Perception]… How do I
know the cause … ? So, we have a lot of information
but putting it together and get one comprehensive
picture is not there [Comprehension]’’ (TL Case E).
4.3 Level of Bottleneck Identification
The examined organizations made information available
through the OCC which helped them identify issues in their
existing proce sses and streamline operations. Most bottle-
necks that were newly identified with help of the OCC
relate to physical constraints coming from misconfigura-
tions of the managed systems – for example:
‘‘We were able to identify some capacity issues…
and we were able to identify some configuration
issues’’ (TL Case G).
‘‘We got a lot of events around memory utiliza tion…
That of course means there’s something wrong,
something needs to be properly configured’’ (TL Case
E).
Benefits regarding the identification of managerial
constraints were reported only for the cases A and G, e.g.:
‘‘What we do… is: refini ng alerts, identifying new
alerts that might need to be created, reviewing and
refining standard operating procedures, eliminating
those things that we don’t need’’ (TL Case G).
For the other cases (B, C, D, E, F) we observed a lack in
the identification of managerial constraints by the IT sup-
port team. Therefore, these cases have a lower level of
bottleneck identification. Even though some of these
organizations have a strong emphasis on the creation of
policies and procedures to deal with the different event
types, they are still occupied with the initial creation of
these ‘‘guided procedures’’ and do not systematically
identify and improve managerial constraints.
4.4 Level of Process Performance
The performance of the ITSM event management process
improved since OCC implementation in most of the studied
cases. Our qualitative examination of the interview data
indicates that organizations A, D, and G reached relatively
high process performance. The exemplified quotes stand
testimony to that.
‘‘There are around 9 priority incidents handled
internally every month. Meaning we prevented 9
major breakdowns monthly… We had a lot of issues
last year with memory and we had three crashes. This
year we did not have any issue’’ (TL Case A).
‘‘The last major incident in production environment
was 12 months ago. So the systems have been very
stable’’ (TQM Case G).
Even though in cases B and E good system stability and
proactive incident resolution was reported, we rated their
overall process performance with medium, because they
show a weak perception of their service quality by their
stakeholders:
‘‘We struggle within our own management to pro-
mote the value of OCC’’ (TL Case G).
‘‘They [stakeholders] don’t remember what used to
be… and now it [high service quality] is just an
expectation’’ (TL Case E).
In cases C and F system and service quality still shows a
lot of flaws. The potential reasons for these deficiencies are
discussed in the next section. However, even cases with
low process performance after OCC implementation report
some first gains – for example:
‘‘There is already a shift. I would not say that it is
proactive now. But at least it has become real-time
now, for the reactive approach that was in place
earlier… We were able to identify some issues that
our service provider chose to ignore or postpone
earlier… We were also able to avert a major issue the
past weekend’’ (TL Case F).
4.5 Explaining Variations
Table 6 summarizes the high-level evaluation of the pro-
cess monitoring benefit dimensions of the different cases. It
shows that high process performance is only observed, if
beside high process visibility also high SA or high bottle-
neck identification is reached. This is line with the pro-
posed mediating effects of SA (P1–2) and bottleneck
identification (P3–4). However, the question remains why
there are different outcomes while all case organizations
implemented the same software package to realize an
OCC. Therefore, Table 6 outlines potential influential
factors that might impact process monitoring benefits. The
moderating effects of these factors on the observed impacts
are elaborated in the following.
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
4.5.1 Skills and Knowledge of Operators
Skill and knowledge of operators significantly differ
between the cases. In some cases (A, B, G) there are very
experienced and knowledgeable experts in the ITSM event
management domain – for example:
‘‘The OCC team comes from our historical basis
support team [with deep tec hnical knowledge]. After
a few years there, they may move on to the OCC
team’’ (TL Case B).
Whereas in other cases (C, D, E, F, G) the IT support
team is staffed by junior level employees. Interestingly the
OCC introduction was partially even the reason to assign
less skilled employees:
‘‘Before OCC the monitoring was done by senior
analysts. This was a humo ngous waste of resources…
It is part of our cost savings by getting these reall y
junior level resources with just basic SAP knowledge
from a two weeks training’’ (TL Case E).
Altogether, our cross case analysis shows that the
influence of skills and knowledge of the OCC team
members might be less important than expected. In case B
also the highly experienced operators could not reach high
process performance, whereas in case D less skilled IT
operators could reach high process performance. Thus, we
cannot confirm proposition P5 that low skills and knowl-
edge of process operators necessarily have negative impact
on process performance.
4.5.2 CI Culture
With CI culture there are strong differences between the
organizations. Some organizations (A, B, G) have dedi-
cated strategies, functions, and processes for CI. On the
other hand, some of the cases (C, F) did not have CI focus
at all. Resource issues and internal politics played a role in
case C not having any meaningful CI strategy:
‘‘But now, the next step will be to prevent them. Do
some root cause analysis and problem management.
For this, you need people. The way we are working
makes it impossible to get people’s time’’ (TL Case C).
One important artifact of a CI culture that we recognize
in the OCC context is the documentation and continuous
improvement of instructions about how to react on events:
‘‘We are trying to create more guided procedures. We
don’t have that many but it’s our aim to use more
guided procedures on alerts… [because] we hope to
work on a more efficient way, that’s to me also
continuous improvement’’ (TL Case D).
Furthermore, our data indicate that there is a relation
between CI culture and the level of bottleneck identifica-
tion. All cases with lower CI culture achieved also lower
levels of bottleneck identification. Or in other words, in an
environment where CI is not valued, process visibility is
also not leveraged for bottleneck identification. Thus we
confirm proposition P6 that low levels of CI culture leads
to lower levels of process visibility’s benefits regarding
bottleneck identification and ultimately to lower process
performance impact.
4.5.3 Further Influential Factors
In the course of our research we identified two further
influential factors that seem to affect the benefits of process
monitoring. First, open coding of the interviews showed
that outsourcing quality partially had strong impact on the
OCC outcome, e.g.:
‘‘It was quite a challenge because our service provi-
der did not have a motivation to change. They did not
want to use the new processes and tools to support
our busi ness’’ (TL Case F).
In some organizations (A, E, and F) the OCC is running
at external offshore serv ice providers which are operating
Table 6 High-level summary of case by case analysis
Case Process Monitoring Benefit Dimensions Influential Factors
Process
visibility
Situation
awareness
Bottleneck
identification
Process
performance
Skills and knowledge
of operators
CI
culture
Outsourcing (OS)
quality
Tool
maturity
A High High High High High High High High
B Medium Medium Medium Medium High High No OS Low
C High Medium Medium Low Medium Low No OS Low
D High High Medium High Medium Medium No OS High
E High Low Medium Medium Medium Medium Low Low
F Medium Low Low Low Medium Low Low Medium
G High High High High High High No OS High
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
major parts of the ITSM event management process. For
well managed outsourcing relationships we couldn’t iden-
tify a negative impact on process monitoring benefits (case
A). However, outsourcing in cases E and F required a lot of
coordination and controlling, which are recognized in IS
outsourcing research as potentially expensive activities in
labor-intensive offshoring relations (Dibbern et al. 2008).
This impacted the overall benefit realization to a large
extent and situation awareness as well as bottleneck iden-
tification were relatively low. Accordingly, we conclude
with an additional proposition:
[P7] The lower the quality of the relationship to out-
sourcing partners who are significantly involved in a
process, the lower is the situation awareness and bottle-
neck identification (which lowers the impact of process
visibility on process performance).
Second, the maturity of the monitoring tool in use was
found to be another important factor why organizations
reached lower performance gains than others as they faced
deficiencies in situation awareness and bottleneck identifi-
cation. We define tool maturity as the degree to what extent
a software tool is ready for use in its intended operational
environment to be validated against user requirements
(Tetlay and John 2009). In case C for example the sole focus
of the OCC implementation on efficiency resulted in with-
drawal of experienced IT professionals from the project
before the basic configuration of the monitoring tool was
finished. In cases B, E and F we identified issues with the
initial setup of the software tool which led to extra efforts in
the implementation and running of the solution. These
issues were coming from gaps in the implementation pro-
cedure, configuration errors, or from functional deficiencies
in early versions of the software:
‘‘One main issue is the overall OCC stability. Some
of these issues are related to our personal setup of not
having a quality test environment’’ (TL Case B).
‘‘There are already lots of things that we can only use
now, and, yes, and it’s a pity that we didn’t have
those earlier’’ (TQM Case C).
Therefore, regarding tool maturity we suggest:
[P8] The lower the maturity of the monitoring software
tool, the lower is the situation awareness and bottleneck
identification (which lowers the impact of process visibility
on process performance).
By the identification of this final proposition from our
empirical examination, we present the resulting Process
Monitoring Benefits Framework in Fig. 1. This conceptual
framework summarizes our suggested process monitoring
benefit dimensions, its relations, and influencing factor s.
5 Discussion and Conclusion
Our Process Monitoring Benefits Framework and its
propositions were empirically verified in a multiple case
study in 7 organizations that had implemented an OCC. An
OCC aims to improve monitorin g in the ITSM event
management process by increasing its process visibility.
We conceptualize process visibility as a multidimensional
construct on the process level. Drawing on the SA Theory
and the Theory of Constraints, our conceptual framework
suggests that process visibility increases situation aware-
ness in process operation and bottleneck identification for
process improvement. Both, situation awareness and bot-
tleneck identification are proposed to positively influence
process performance. Furthermore, we identified influential
factors for benefit realization of process monitoring in
ITSM event management based on existing literature and
our empirical investigation. Our multiple case study data
proposes that process visibility increases process perfor-
mance, mediated by situation awareness and bottleneck
identification. The potential benefits of process monitoring
in ITSM were influenced by three factors: CI culture,
outsourcing quality, and maturity of the monitoring tool.
Regarding skills and knowledge of the process operators, it
was found that process visibility seems to reduce the
impact of this factor on process performance.
Our study is subject to specific limitations: First, the
amount of qualitative data is limited as only 2–3 interviews
per case have been conducted. However, the interview data
were triangulated with information from a customer data-
base of our industry partner. Second, hindsight bias might
have influenced our findings as we could not observe
process participants inside concrete critical situations,
which particularly for SA assessments would have been
beneficial and should be considered for future research.
Likewise, changes attributed to the OCC impleme ntations
were evaluated only in retrospect. Third, for generaliz-
ability to a broader process context our Process Monitoring
Benefits Framework ought to be studied also outside ITSM
operations. Finally, although we acknowledge that the
close collaboration with one software vendor bears the risk
of being influenced by biases of the industry partner, we
also see it as an opportunity to ensure the relevance of our
work.
However, we believe to have made significant contri-
butions to theory and practice. From a theoretical per-
spective, this paper adds to the body of knowledge related
to empirical BPM research in the important domain of the
monitoring use case. The conceptualization of process
visibility offers a generalization of concepts coming from
SCM and lean production to a broader process context,
which is a promising foundation for more studies of the
process visibility phenomenon in and beyond ITSM. Our
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
suggested Process Monitoring Benefits Framework and its
propositions helps to guide future research about the
impact of process visibility on process performance. From
a practitioner perspective, our paper proposes several
anchors on how to increase benefits of process monitoring
in organizations. Particularly, it describes what influencing
factors should be considered while implementing new
software for process monitoring. Furthermore, it identifies
situation awareness and bottleneck identification as areas
where leveraging data assets is of utmost importance for
BPM.
References
Barratt M, Oke A (2007) Antecedents of supply chain visibility in
retail supply chains: a resource-based theory perspective. J Oper
Manag 25(6):1217–1233
Berner M, Graupner E, Maedche A, Mueller B (2012) Process
visibility – towards a conceptualization and research themes. In:
Proceedings of the 13th International Conference on Information
Systems (ICIS 2012), pp 1–13
Cater-Steel A, McBride N (2007) IT service management improve-
ment – actor network perspective. In: ECIS 2007 Proceedings.
In: Proceedings of the 15th European Conference on Information
Systems (ECIS 2007), pp 1202–1213
Corbin J, Strauss A (2008) Basics of qualitative research: techniques
and procedures for developing grounded theory. Sage, Thousand
Oaks
Cotteleer MJ, Bendoly E (2006) Order lead-time improvement
following enterprise information technology implementation:
an empirical study. MIS Q 30(3):643–660
Davenport TH (1993) Process innovation: reengineering work
through information technology. Havard Business School Press,
Cambridge
de Morais RM, Kazan S, de Pa
´
dua SID, Costa AL (2014) An analysis
of BPM lifecycles: from a literature review to a framework
proposal. Bus Process Manag J 20(3):412–432
DeLone WH, McLean ER (2003) The DeLone and McLean model of
information systems success: a ten-year update. J Manag Inf Syst
19(4):9–30
Dibbern J, Winkler J, Heinzl A (2008) Explaning variations in client
extra costs between software projects offshored to India. MIS Q
32(2):333–366
Dube
´
L, Pare
´
G (2003) Rigor in information systems positivist case
research: current practices, trends, and recommendations. MIS Q
27(4):597–635
EMC (2012) EMC Ionix ControlCenter. http://www.emc.com/collat
eral/TechnicalDocument/docu7670.pdf. Accessed 20 Oct 2015
Endsley MR (1995) Toward a theory of situation awareness in
dynamic systems. J Hum Factors 37(1):32–64
Galup SD, Dattero R (2010) A five-step method to tune your ITSM
processes. Inf Syst Manag 27(2):156–167
Gartner (2012) Gartner says intelligent business operations is the next
step for BPM programs. http://www.gartner.com/it/page.jsp?id=
1943514. Accessed 20 Oct 2015
Goldratt E, Cox J (1992) The goal: a process of ongoing improve-
ment, 2nd edn. North River, Great Barrington
Graupner E, Berner M, Maedche A, Jegadeesan H (2014) Assessing
the need for visibility of business processes – a process visibility
fit framework. In: Proceedings of Business Process Management
Conference BPM 2014. Springer LNCS 8659, pp 384–392
Hogg DN, Folles K, Strand-Volden F, Torralba B (1995) Develop-
ment of a situation awareness measure to evaluate advanced
alarm systems in nuclear power plant control rooms. Ergonomics
38(11):2394–2413
Holsti OR (1969) Content analysis for the social sciences and
humanities. Addison-Wesley, Boston
Iden J, Eikebrokk TR (2013) Implementing IT service management: a
systematic literature review. Int J Inf Manag 33(3):512–523
Kassarjian HH (1977) Content analysis in consumer research.
J Consum Res 4(1):8–18
Klotz L, Horman M, Bi HH, Bechtel J (2008) The impact of process
mapping on transparency. Int J Product Perform Manag
57(8):623–636
Manyika J, Chui M, Brown B et al (2011) Big data: the next frontier
for innovation, competition, and productivity. McKinsey Global
Institute, San Francisco
Martinez H (2009) How much does downtime really cost? InfoMan-
agement Direct, August 6, 2009. http://www.information-man
agement.com/infodirect/2009_133/downtime_cost-10015855-1.
html. Accessed 20 Oct 2015
McCombs BL, Marzano RJ (1990) Putting the self in self-regulated
learning: the self as agent in integrating will and skill. Educ
Psychol 25(1):51–69
Melville N, Kraemer K, Gurbaxani V (2004) Information technology
and organizational performance: an integrative model of IT
business value. MIS Q 28(2):283–322
Myers MD, Newman M (2007) The qualitative interview in IS
research: examining the craft. Inf Organ 17(1):2–26
Nelson RR, Todd PA, Wixom BH (2005) Antecedents of information
and system quality: an empirical examination within the context
of data warehousing. J Manag Inf Syst 21(4):199–235
O’Brien KS, O’Harea D (2007) Situational awareness ability and
cognitive skills training in a complex real-world task. Ergo-
nomics 50(7):1064–1091
OGC (2007a) ITIL Service strategy, Version 3. Office of Government
Commerce, London
OGC (2007b) Continual service improvement, Version 3. Office of
Government Commerce, London
Pidun T, Buder J, Felden C (2011) Optimizing process performance
visibility through additional descriptive features in performance
measurement. In: 15th IEEE International Enterprise Distributed
Object Computing Conference Workshops. IEEE Computer
Society, pp 204–212
Ray G, Muhanna WA, Barney JB (2005) Information technology and
the performance of the customer service process: a resource-
based analysis. MIS Q 29(4):625–652
Russom P (2013) Operational intelligence: real-time business analyt-
ics from big data. TDWI Checkl Rep 1–8
SAP (2013) Operations control center. White Paper.
https://docu
ments.wdf.sap.corp/share/proxy/alfresco//api/node/content/work
space/SpacesStore/94863204-455e-41a1-865e-4924d4aafe8e/
OCC%20White%20Paper%20v1.0.pdf?a=true. Accessed 20 Oct
2015
Schein EH (2004) Organizational culture and leadership. Jossey-Bass,
San Francisco
Shingo S (1989) A study of the Toyota production system from an
industrial engineering viewpoint. Productivity Press, New York
Swaminathan MJ, Tayur SR (2003) Models for supply chains in
e-business. Manag Sci 49(10):1387–1406
Tetlay A, John P (2009) Determining the lines of system maturity,
system readiness and capability readiness in the system devel-
opment lifecycle. In: 7th Annual Conference on Systems
Engineering Research (CSER 2009), pp 1–8
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
van der Aalst WMP (2013) Business process management: a
comprehensive survey. ISRN Softw Eng 2013:1–37
Womack JP, Jones DT (2003) Lean thinking: banish waste and create
wealth in your corporation. Free Press, New York
Xu J, Benbasat I, Cenfetelli RT (2013) Integrating service quality
with system and information quality: an empirical test in the
e-service context. MIS Q 37(3):777–794
Yin RK (2003) Case study research, 3rd edn. Sage, Thousand Oaks
123
M. Berner et al.: The Impact of Process Visibility on Process Performance, Bus Inf Syst Eng
- A preview of this full-text is provided by Springer Nature.
- Learn more
Preview content only
Content available from Business & Information Systems Engineering
This content is subject to copyright. Terms and conditions apply.
