Content uploaded by Wilhelm Hasselbring
Author content
All content in this area was uploaded by Wilhelm Hasselbring on May 18, 2019
Content may be subject to copyright.
Industrial DevOps
Wilhelm Hasselbring∗,S
¨
oren Henning∗,Bj
¨
orn Latte†, Armin M¨
obius‡,
Thomas Richter§, Stefan Schalk†, and Maik Wojcieszak†
∗Software Engineering Group, Kiel University, Department of Computer Science, 24098 Kiel
†wobe-systems GmbH, Wittland 2-4, 24109 Kiel, Germany
‡IBAK Helmut Hunger GmbH & Co. KG, Wehdenweg 122, 24148 Kiel, Germany
§Krause-Biagosch GmbH, Paul-Schwarze-Straße 5, 33649 Bielefeld, Germany
https://www.industrial-devops.org
Abstract—The visions and ideas of Industry 4.0 require a
profound interconnection of machines, plants, and IT systems in
industrial production environments. This significantly increases
the importance of software, which is coincidentally one of the
main obstacles to the introduction of Industry 4.0. Lack of
experience and knowledge, high investment and maintenance
costs, as well as uncertainty about future developments cause
many small and medium-sized enterprises hesitating to adopt
Industry 4.0 solutions. We propose Industrial DevOps as an
approach to introduce methods and culture of DevOps into
industrial production environments. The fundamental concept of
this approach is a continuous process of operation, observation,
and development of the entire production environment. This way,
all stakeholders, systems, and data can thus be integrated via
incremental steps and adjustments can be made quickly. Fur-
thermore, we present the Titan software platform accompanied
by a role model for integrating production environments with
Industrial DevOps. In two initial industrial application scenarios,
we address the challenges of energy management and predictive
maintenance with the methods, organizational structures, and
tools of Industrial DevOps.
I. INTRODUCTION
The digital transformation of the conventional, manufactur-
ing industry enables a new level of automation in production
processes. More and more technical machines and produc-
tion plants become increasingly intelligent and autonomous.
Equipped with network capabilities, they are able to consume
and supply data to others. This new trend, often referred to
as Industrial Internet of Things or Industry 4.0 [1], confronts
the production operator with the challenge of connecting and
monitoring the individual machines and devices [2]. While
software used to be only part of the production process,
nowadays software increasingly defines the production process
itself [3], so that the resulting software systems inevitably
become more complex. In order to ensure a smooth production
process these software systems have to be designed with a
special focus on reliability, scalability, and adaptability [4].
As is usual with complex systems, however, this complicates
the design, development, and maintenance of such a system.
In particular for small and medium-sized enterprises (SMEs),
this poses enormous challenges as these SMEs often do not
This research is funded by the Federal Ministry of Education and Research
(BMBF, Germany) in the Titan project (https://www.industrial-devops.org,
contract no. 01IS17084B).
have suitable software development departments. Instead, soft-
ware is often developed by domain experts (e.g., mechanical
engineers) with basic programming skills, but no education
in software engineering. Alternatives such as establishing
a dedicated software development department or delegating
software development to specialized external companies are
also risky. Business or domain requirements and their technical
implementations often diverge, so adaptations are cumber-
some, costly, and the time until they are released is long.
Agile and iterative principles, methods, and techniques that
are common in other fields of software engineering, such as
e-commerce systems [5], [6], can provide solutions to this.
Section II of this paper describes how these methods can
be transferred to the domain of industrial production envi-
ronments and highlights necessary changes in processes and
culture. Section III presents a software platform with which
these methods can be applied in the manufacturing industry.
Section IV describes a role model for applying the required
organizational and technical structures and Section V outlines
our two initial application scenarios. Section VI concludes this
paper and points out future work.
II. INDUSTRIAL DEVOPS
The traditional separation of software operation from its
development leads to several issues due to a lack of com-
munication, collaboration, and integration. DevOps [7] is a
movement to bridge this gap. We propose Industrial DevOps
as an approach for transferring DevOps values, principles,
and methods to industrial systems integration. In this way, we
expect that development and operation of such an integrating
software system will be improved and the discrepancy between
production operators and software developers will be reduced,
while ensuring high software quality. The following principles
are of crucial importance for Industrial DevOps.
1) Continuous Adaption and Improvement Process: The
core element of Industrial DevOps is a coherent, cyclic,
and continuous process as illustrated in Fig. 1. During its
operation, the software system as well as the production
itself is monitored and analyzed. Based on analysis results,
new requirements are identified. The implementation of these
requirements will either be done by adapting the software or by
reconfiguration in the IT operation or production. Adjustments
are tested automatically and after passing all tests, the new
1SFQSJOUPG
8)BTTFMCSJOHFUBM*OEVTUSJBM%FW0QT*&&&*OUFSOBUJPOBM$POGFSFODFPO4PGUXBSF"SDIJUFDUVSF
$PNQBOJPO*$4"$)BNCVSH(FSNBOZQQ
EPJ*$4"$
Fig. 1. The continuous adaption and improvement process of Industrial DevOps
software replaces the old one in production, where it is
monitored again.
Applying this iterative process aims to enable continuous
adaptions and improvements of the software. Thus, complex
systems are going to be created by starting from simple ones
that are extended in incremental steps. Another advantage
can be found in the maintenance of the systems. While IT
systems that run in production after a development phase are
provided with updates for a certain period of time, followed
by a certain period of service, the actual operating time is
often considerably longer. As a result, operation becomes
increasingly complex and risky, but at the same time migrating
to another system becomes more and more difficult. The
cyclic process of Industrial DevOps is intended to achieve
that development, updates, and service become continuous
processes that run permanently parallel to the operation of
the system.
2) Lean Organizational Structure: When applying external
software solutions, this often requires enterprises to adapt their
production processes to these solutions. Industrial DevOps
comes with an organizational structure to prevent this and in-
stead support the alignment of the software with the production
process.
A key factor of Industrial DevOps is that requirements are
discussed between all stakeholders and people from different
business units are brought together. This practice is referred
to as BizDevOps as a more general term [8]. Industrial
DevOps extends this by applying BizDevOps to integrating
systems and data of industrial production environments. For
the implementation of these requirements, domain experts and
developers work closely together. Moreover, the development
provides appropriate means, which allow domain experts to
solve the domain problems themselves to a certain extent. This
is accomplished by an extendable software platform, which is
owned by the user. Ownership can be direct or indirect if the
software is owned by an open source community.
Information from all levels is available for anyone in the
value stream through monitoring. Even if the information that
is required differs, the source of information is consistent
and allows investigation about cause and effect (culture of
causality). This should enable lean organizational learning [9].
3) Customer-Centric Value Generation: Organizations who
apply customer-centric value generation align all departments
to contribute to this goal [10]. All activities in the organi-
zation which do not directly or indirectly contribute to this
goal are abandoned. Some activities which are required for
organizational or compliance reasons are organized to stay
out of the way of the overall goal. Instead of wasting huge
amounts of money in market research, experiments are started
in a way that new features, functions, or products are offered
to customers and direct measurements are made by utilizing
IT to find out the customers response. These measurements
are not only done once, but continuously.
III. THE TITAN PLATFORM
Titan is a software platform for integrating and monitoring
industrial production environments using Industrial DevOps.
In order to integrate tools or infrastructure (e.g., machines, IoT
devices, software, or databases), modular software components
7RROV
,QIUDVWUXFWXUH
,7 ,QIUDVWUXFWXUH
7LWDQ
)ORZ (QJLQH
%ULFNV DQG
3OXJLQV
)ORZV
3ODWIRUP
'HYHORSHU
%ULFN
'HYHORSHU
7RRO
'HYHORSHU
,7 6HFXULW\
([SHUW
,7 2SHUDWRU
)ORZ 0DQDJHU
%XVLQHVV 8QLW
0DQDJHU
'RPDLQ
([SHUW
3URFHVV
2SHUDWRU 5HTXLUHPHQWV
3
3 7
3
%
%
7
%XVLQHVV DQG
3URGXFWLRQ
,7 DQG
3ODWIRUP
3URFHVV
0RQLWRULQJ
Fig. 2. The Titan role model
are created. These components, in Titan called bricks, serve
for integrating exactly one such system. This allows on the one
hand to exchange the underlying tool or infrastructure easily,
but on the other hand also to replace the integrating software
implementation. To connect bricks (and thus the underlying
systems of the production), Titan applies the principles of
flow-based programming [11] and calls a network of connected
bricks a flow.
Titan provides a graphical modeling language, which en-
ables the organization to graphically model the integration and,
based on this, to configure the integrated system. Hence, there
is no programmer required to perform these configurations or
(pre-configured) changes. Instead, a person that has received
training on the modeling language can work together with a
domain expert to do this.
Thus, this platform allows to react efficiently and effec-
tively to changing requirements in production. Furthermore, it
enables conducting experiments with little effort and to adjust
quickly in case of experiment failures. Titan integrates the
required monitoring, which includes resource consumption,
system utilization, and business data.
IV. THE TITAN ROLE MODEL
Industrial DevOps involves people from different domains,
business units, and organizations. The Titan role model (see
Fig. 2) defines roles to be assigned to people or groups that
use, operate, or develop the Titan platform and describes their
responsibilities as well as how they interact.
ADeveloper is a trained software engineer. This person has
learned to develop software in many programming languages.
Many different aspects of software development like code
quality or version management are covered. In Industrial
DevOps, many developers who are spread over many teams in
different organizations work together on a single system. The
role model introduces the following sub-roles: The Platform
Developer works on new features and improvements of the
Titan platform itself. The Tool Developer creates software
tools or interfaces for hardware that is integrated into the
system. The Brick Developer creates bricks and works closely
together with the Tool developers.
The IT Operator is trained in installing, configuring, and
running IT infrastructure. This person is also responsible
for installing, updating, configuring, and running software
applications.
The Flow Manager is a person that is closely related to
the business, but with a special training to master all aspects
of flow modeling. This person works as a connection between
business and technology. The Flow Manager implements new
flows and flow changes according to business requirements.
The Business Unit Manager can be located on all levels of
the hierarchy. This person knows about the business require-
ments and works together with the Flow Manager, the Domain
Expert, and the Process Operator to refine requirements.
The Domain Expert owns deep knowledge about the
domain at hand.
The Process Operator benefits from a new flow or a
change. This person is involved in utilizing the flow in the
actual production environment.
The IT Security Expert is responsible for the IT security
domain of the whole organization. The IT Security Expert
works together with development and operations through all
levels of the system.
As an extension to regular DevOps roles, the Titan role
model introduces business roles, domain experts, flow man-
agers, and process operators, which do not need to be devel-
opers.
V. I NITIAL INDUSTRIAL APPLICATION SCENARIOS
Enterprises, in particular in the manufacturing industry, face
the challenge of reducing and optimizing their energy con-
sumption for economic and ecological reasons. This requires
in-depth monitoring and analysis of the individual energy
consuming devices, machines, and plants. Doing this, also
referred to as energy management, serves as an initial appli-
cation scenario for Industrial DevOps and Titan. We designed
a monitoring infrastructure, which monitors, analyses, and
visualizes the electrical power consumption in industrial pro-
duction environments [12]. Employing the Titan platform, it
is able to integrate different kinds of sensors that use different
data schemata, formats, and protocols. Referring to the Titan
role model, this monitoring infrastructure enables Business
Unit Managers, Domain Experts, and Process Operators to
gain detailed insights into the energy usage and to identify
saving potentials. Thus, it is possible to perform and evaluate
optimization measures across departmental boundaries within
the company.
Predictive maintenance is another initial application sce-
nario we address with Industrial DevOps and Titan. Titan
can serve as a platform, which detects faults in machines
and production plants in advance on the basis of data from
integrated sensors and systems. An essential aspect here is
that not only data from a single machine and production plan
is considered for analysis. Instead, measurements and events
from of the entire production as well as environmental influ-
ences can be taken into account. The predictive maintenance
function thus has a much broader amount of data available
and can therefore also include non-obvious influences. With
the insights gained from the data, maintenance intervals can
be optimized. Additionally by returning analysis results to the
machines, they can optimize themselves.
VI. CONCLUSIONS AND FUTURE WORK
We propose Industrial DevOps as an approach to transfer
practices, methods, and culture of DevOps for integrating
systems in industrial production environments. It suggests an
organizational structure in which domain experts, business
managers, developers, and operators work together on solu-
tions. Production environments should be integrated, config-
ured, and operated in a validated learning loop to manage
changes and to control costs.
Future work lies in evaluating the proposed principles,
methods, and organizational structures of Industrial DevOps in
different industrial application scenarios. Therefore, we further
develop the Titan platform. To overcome the disadvantages of
proprietary integration systems, we will release it as free and
open source software. We are already using Clean Code ap-
proaches in the development to prevent software quality from
degrading over time [13]. Along with including quantitative
quality characteristics such as performance in the DevOps
cycle [14] as well as rigorous monitoring in operation, this
is supposed to contribute in creating a long-living software
platform [15].
REFERENCES
[1] S. Jeschke, C. Brecher, H. Song, and D. B. Rawat, Industrial Internet
of Things: Cybermanufacturing Systems. Springer, 2017.
[2] Y. Lu, “Industry 4.0: A survey on technologies, applications and open
research issues,” Journal of Industrial Information Integration, vol. 6,
pp. 1 – 10, 2017. doi: 10.1016/j.jii.2017.04.005
[3] J. Wan, S. Tang, Z. Shu, D. Li, S. Wang, M. Imran, and A. V. Vasilakos,
“Software-defined industrial internet of things in the context of Industry
4.0,” IEEE Sensors Journal, vol. 16, no. 20, pp. 7373–7380, Oct 2016.
doi: 10.1109/JSEN.2016.2565621
[4] B. Vogel-Heuser, A. Fay, I. Schaefer, and M. Tichy, “Evolution of
software in automated production systems: Challenges and research
directions,” Journal of Systems and Software, vol. 110, pp. 54 – 84,
2015. doi: 10.1016/j.jss.2015.08.026
[5] W. Hasselbring and G. Steinacker, “Microservice architectures for scala-
bility, agility and reliability in e-commerce,” in Proceedings 2017 IEEE
International Conference on Software Architecture Workshops, 2017.
doi: 10.1109/ICSAW.2017.11 pp. 243–246.
[6] H. Knoche and W. Hasselbring, “Drivers and barriers for microservice
adoption - a survey among professionals in germany,” Enterprise Mod-
elling and Information Systems Architectures (EMISAJ) - International
Journal of Conceptual Modeling, vol. 14, no. 1, pp. 1–35, Januar 2019.
doi: 10.18417/emisa.14.1
[7] L. Bass, I. Weber, and L. Zhu, DevOps: A Software Architect’s Perspec-
tive. Addison-Wesley, 2015.
[8] V. Gruhn and C. Sch¨
afer, “BizDevOps: Because DevOps is not the end of
the story,” in Intelligent Software Methodologies, Tools and Techniques,
H. Fujita and G. Guizzi, Eds. Springer, 2015. doi: 10.1007/978-3-319-
22689-7 30 pp. 388–398.
[9] J. Knuf, “Benchmarking the lean enterprise: Organizational learning at
work,” Journal of Management in Engineering, vol. 16, no. 4, pp. 58–71,
2000. doi: 10.1061/(ASCE)0742-597X(2000)16:4(58)
[10] S. Denning, The Age of Agile: How Smart Companies Are Transforming
the Way Work Gets Done. AMACOM, 2018.
[11] J. P. Morrison, Flow-Based Programming, 2nd Edition: A New Approach
to Application Development. Paramount, CA: CreateSpace, 2010.
[12] S. Henning, W. Hasselbring, and A. M¨
obius, “A scalable architecture for
power consumption monitoring in industrial production environments,”
in IEEE International Conference on Fog Computing, 2019, in press.
[13] B. Latte, S. Henning, and M. Wojcieszak, “Clean code: On the use
of practices and tools to produce maintainable code for long-living
software,” in Proceedings of the Workshops of the Software Engineering
Conference 2019. Stuttgart, Germany: CEUR Workshop Proceedings,
Februar 2019.
[14] J. Waller, N. C. Ehmke, and W. Hasselbring, “Including performance
benchmarks into continuous integration to enable DevOps,” SIGSOFT
Software Engineering Notes, vol. 40, no. 2, pp. 1–4, Mar. 2015. doi:
10.1145/2735399.2735416
[15] U. Goltz, R. Reussner, M. Goedicke, W. Hasselbring, L. M¨
artin, and
B. Vogel-Heuser, “Design for future: managed software evolution,”
Computer Science – Research and Development, vol. 30, no. 3, pp.
321–331, Aug. 2015. doi: 10.1007/s00450-014-0273-9
