Access to this full-text is provided by Springer Nature.
Content available from Scientific Reports
This content is subject to copyright. Terms and conditions apply.
1
Vol.:(0123456789)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports
Analysis of quantitative
metrics for assessing resilience
of human‑centered CPPS
workstations
Tanel Aruväli *, Matteo De Marchi & Erwin Rauch
Manufacturing companies’ preparedness level against external and internal disruptions is complex
to assess due to a lack of widely recognized or standardized models. Resilience as the measure to
characterize preparedness against disruptions is a concept with various numerical approaches, but still
lacking in the industry standard. Therefore, the main contribution of the research is the comparison of
existing resilience metrics and the selection of the practically usable quantitative metric that allows
manufacturers to start assessing the resilience in digitally supported human‑centered workstations
more easily. An additional contribution is the detection and highlighting of disruptions that potentially
inuence manufacturing workstations the most. Using ve weighted comparison criteria, the
resilience metrics were pairwise compared based on multi‑criteria decision‑making Analytic Hierarchy
Process analysis on a linear scale. The general probabilistic resilience assessment method Penalty of
Change that received the highest score considers the probability of disruptions and related cost of
potential changes as inputs for resilience calculation. Additionally, manufacturing‑related disruptions
were extracted from the literature and categorized for a better overview. The Frequency Eect Sizes
of the extracted disruptions were calculated to point out the most inuencing disruptions. Overall,
resilience quantication in manufacturing requires further research to improve its accuracy while
maintaining practical usability.
Disruptions and adverse events in the world may inuence and interrupt the production processes in a manu-
facturing company for months and even years. e events of severe disruptions are oen related to external
changes which are out of manufacturers’ sphere of inuence. Manufacturers are impressionable from external
changes, consequently without an ability to counteract the source of change. Recent severe disruptions such as
the spread of Covid-19, Suez Canal blockage by cargo ship, the war in Ukraine and geopolitical sanctions had
and still have severe inuences internationally. External disruptions are more complicated to predict as they oen
occur unexpectedly compared with manufacturing internal adverse events. e behavior of internal disruptions
is more predictable and controllable, but without a systematic approach to managing them, the consequences
can be even severer. Although many causes of internal disruptive events are known (worker absence, machinery
breakdown, misinformation, lack of information, outage of material or instruments, etc.) the overall prepared-
ness for their actual occurrence is complicated to estimate. Complexity is even increased by the fact that the list
of disruptive events is not nal, and it expands in time due to technological development and overall external
environment evolution. erefore, an unproperly evaluated preparedness level against potential disruptions may
have existential consequences for manufacturing companies.
To assess the preparedness level against potential disruptions, the measure of resilience is an important
indicator. Resilience is a concept historically more used in social sciences and it has been rather a qualitative
measure. In engineering, the feature of resilience has been taken more widely into consideration recently1. In
manufacturing, during the era of Industry 4.0, the widely used key performance indicator to evaluate everyday
production has been productivity which is relatively simple to measure quantitatively. Cyber-Physical Production
Systems (CPPS) retrieve various data from manufacturing shopoors which are mostly used for eciency-related
metrics. e ongoing transition to the era of Industry 5.0 contrarywise, highlights a wider and longer-term view
of the manufacturing shopoor health and the benets of sustainable manufacturing. erefore, CPPS have the
advantage compared with traditional shopoors for using retrieved data simultaneously for lling the cap in
OPEN
Free University of Bozen-Bolzano, Piazetta del Università 1, 39100 Bozen-Bolzano, Italy. *email: tanel.aruvaeli@
unibz.it
Content courtesy of Springer Nature, terms of use apply. Rights reserved
2
Vol:.(1234567890)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
resilience assessment. Resilience is to encompass all three pillars of Industry 5.0: economic, ecologic, and social
aspects. Even so, no widely recognized approach nor equation to measure resilience exists. erefore, for a spe-
cic application, a variety of studies must be reviewed and analyzed to select an optimal tool for the numerical
assessment of resilience.
e driver of this research is on one hand lack of a practically usable standard model to measure resilience
in manufacturing, on the other hand, a variety of models diering in their inputs, algorithms, and even units
of measure. To create more clarity, there is a need for a deeper understanding what are the causes that trigger
disruptions and how to quantitatively measure their potential impact on manufacturing resilience. e main
research question is formulated as follows:
RQ: How to quantitatively measure resilience in CPPS workstations by considering the most inuencing
disruptions?
e following three research sub-questions are investigated in this study:
RQ1: What are the disruptions potentially inuencing manufacturing workstations the most?
RQ2: Which quantitative metrics exist to measure the resilience of manufacturing workstations?
RQ3: What is the most practically valuable quantitative resilience metric to assess the level of resilience in
CPPS workstations?
is work aims to provide an overview of dierent resilience metrics and to extract the metrics which can be
used in the assessment of resilience in manufacturing. More specically, the quantitative resilience metrics are
pointed out which can be eciently used in the assessment of digital twin supported worker assistance system
in manufacturing and in the process of verifying those workstations based on resilience. Additionally, manufac-
turing-related disruptions are categorized and analyzed. e research is focused on resilience quantication of
existent manufacturing systems and excludes system design-related optimization where resilience is evaluated
during the design of systems. Further, supply chain related resilience is excluded.
e article is organized as follows: aer the overview of the concept of resilience in “Resilience background”;
the research questions are formulated and the used methods for literature review and Analytic Hierarchy Process
(AHP) analysis process are described in “Methodology”; thereaer in “Results”, the results of the research are
presented descriptively and in table format for easier trackability of processes and comparability; the paper is
concluded in “Discussion” where the results, future perspectives, and challenges are discussed in more detail.
Resilience background
Concept of resilience. Disruptions are the events that cause breakage of resilience. e concept of disrup-
tion comprises disturbances and failures2, in some contexts disruptions are called shocks3 or adverse events4. If
a deviation from a plan is suciently large that the plan must be changed substantially it is called a disrupted
situation or disruption5. For consistency, the word “disruption” is used in this article where no further specica-
tion is needed.
Resilience can be considered belonging to a category of ilities6. e ilities are engineering system proper-
ties that concern wider system impacts and are not considered primary functional requirements in contrast to
reliability, robustness, and durability. Resilience supports other ilities such as safety, sustainability, quality, and
exibility7.
e researchers have provided dierent denitions of the term “resilience”. In general, common positions in
denitions are that resilience includes three focal components: (i) an ability to absorb the impact of disruptions
(absorption), (ii) adaptation to disruptions (adaptation), and (iii) recovery to its normal regime (restoration).
e pathnder of dening the term of resilience as a property of a system is Holling who expressed, resilience
determines the persistence of relationships within a system and is a measure of the ability of these systems to
absorb changes of state variables, driving variables, and parameters, and still persist8. Gu etal.9 dened resilience
as the ability of a system to withstand potentially high-impact disruptions, and it is characterized by the capa-
bility of the system to mitigate or absorb the impact of disruptions, and quickly recover to normal conditions.
Whereas Gasser etal.10 pointed out modern understanding of resilience as a process under which the observed
system undergoes in response to a disruption quantied in terms of a measure of system performance and its
evolution over the system response time aer an event. Romero etal.11 combined the denitions of resilience and
smartness and dened the concept Smart Resilient Manufacturing System as an agile and exible/recongurable
system that uses smart sensor systems and descriptive, predictive, and prescriptive analytics techniques to collect
and analyze in real-time operational and environmental data to anticipate, react, and recover from a disruption.
While Yoon etal.12 proceeded with resilience denition in engineering as the ability of a component or a system
to maintain its required functionality by resisting and recovering from adverse events.
Hence, resilience is a multifactorial concept, it can be resolved by several factors which in a more focused way
characterize the features of it. According to Hu, the resilience of an engineering system consists of three key ele-
ments: reliability, vulnerability, and recoverability13, while Lim etal.14 have replaced the element of vulnerability
with redundancy. e system’s capability of maintaining its functions and structure in the situation of internal
or external changes is part of resilience15. If passively reliable equals vulnerable then adoptively reliable equals
resistant16. Requirements such as functionality, rapidity, and resourcefulness are also brought out as properties of
resilience17. Additionally, resilience engineering factors to consider in manufacturing are exibility, redundancy,
and fault-tolerant18.
ere are other engineering system properties as reliability and stability principally dierentiate and must be
distinguished from resilience. Reliability measures the continued success of a system, while resilience measures
the insensitivity of the system to disruptions19. Whereat fault isolation is an important aspect in achieving internal
reliability20, which in turn increases resilience. Holling pointed out a principal dierence between resilience and
Content courtesy of Springer Nature, terms of use apply. Rights reserved
3
Vol.:(0123456789)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
stability8, stability is an ability of a system to return to equilibrium with the least uctuation aer a temporary
disruption. A system can have high resilience, but still uctuate greatly and have low stability.
Preventive maintenance and redundancy are two main methods for disruptions management21. In manufac-
turing, redundancy is oen related to having a backup machineries and workforce. However, redundancy can
be also achieved at the system management level as informational redundancy, for instance in knowledgeable
implementation of Enterprise Resource Planning22. A high level of redundancy also increases a system’s life
cycle cost16 and cannot be a sustainable solution. Another widely practiced method for disruptions management
is quantitative risk assessment which mainly focuses on the pre-failure scenarios23 but can oer a base for the
assessment of resilience if supported with advanced analytics4. It is accepted that every risk cannot be foreseen
but it must be rather learned to adapt and manage risks in a way to minimize the impact on systems24.
Quantication of resilience. Resilience is a time-dependent phenomenon. In a system, aer an event of a
disruption (Tg), the performance of the system starts to decrease. e performance starts to increase again aer
the event of recovery action (Tr) and the performance increases until the system achieves its steady state (Tss).
Whereas the recovered performance can recover to its original level (P0), to have a shortfall (Pw) or to have a
growth (Pb). Pmin represents the minimum level of performance over disruption. In the time dimension, relative
to the occurrence of a disruptive event, three phases are distinguished: pre-disaster phase (− Tg), during disaster
phase (Tg–Tr) and post-disaster phase (Tr–∞)17. Maximum acceptable recovery time (Ta) and minimum accept-
able performance level (Pa) are proposed, below which operations are presumed to shut down25. e blue area
(Fig.1) represents total loss of performance due to disruption.
In manufacturing systems, resilience is mostly measured as a technical attribute as loss of productivity, it is
also named performance loss or loss of throughput. Resilience is an abstract concept and expressing its value by
loss of manufacturing throughput only is a rather simplied approach. Counterweight, Bruneau etal.27 proposed
four dimensions of community resilience—(1) technical, (2) organizational, (3) social, and (4) economic that
are so far not approached in a socio-technical system as human-centered workstation.
To understand the purposes and functions behind the measuring, the measuring framework has been devel-
oped. Linkov etal.4 have created a resilience matrix to provide guidelines based on what resilience metrics can
be developed to measure overall system resilience. In this matrix system domains (physical, information, cogni-
tive, social) across an event management cycle of resilience functions (plan/prepare, absorb, recover, adapt) are
mapped and described.
In a large scale, resilience assessment approaches can be divided into qualitative and quantitative (Fig.2).
e quantitative measures are typically not optimal, possible, or desirable. erefore, semi-quantitative meas-
ures are oen used in the assessment. Quantitative28 assessment is expressed in numerical values and is stated
in measurement-related specic units. In resilience assessment, these can be divided into general measures and
structural-based models. Semi-quantitative29 assessment uses qualitative categories that assign numeric values
which are thereaer calculated as indices, the assessment oen needs an expert opinion. e least precise method
is qualitative conceptual framework assessment which contributes notion of resilience.
e methods as observation, interview, expert opinion, and focus group can be considered as main qualitative
data collection methods. e data is oen collected in dierent formats as audio, video, and text. In supply chain
Po
Pmin
TrTg
Pa
Pb
Pw
TaTss
Time (T)
Pe
rformance (P)
Threshold
Disruptive event
Figure1. Performance dependence from disruptive event and recovery action (adapted from Refs.25 and 26).
Content courtesy of Springer Nature, terms of use apply. Rights reserved
4
Vol:.(1234567890)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
resilience assessment Nikookar etal.31 used questionnaire in their case study for suppliers evaluation. Garcia
etal.32 analyzed resilience qualitatively in ICT based network systems where they compared delity on dierent
emulation testbeds and recommended to observe system behavior and trend inside the system. For qualitative
data representation, quantitative analytics can be applied as a semi-quantitative approach. Such a scenario was
used by Eljaoued etal.33 where qualitative functional resonance analysis method with numerical approach was
used in socio-technical system resilience assessment.
Without any numerical basis for assessing resilience, it is complicated to monitor and track the improvements.
Numerical measuring allows targets to be established and set clear goals for improvement. Aer Youn etal.16
pioneered a way to measure the engineering design resilience quantitatively, more explicit models have been
developed which additionally consider various characteristics as explicit temporal aspects26 and sensor faults34.
Research of resilience metrics in manufacturing rather focuses on resilience design methods35,36, than assess-
ment and validation of existing systems. Even though, resilience metrics assessment methodology based on
experimental design and statistical analysis has been proposed to validate the metrics37. Still, some resilience
assessment case studies have been recently conducted in the industry overall. Hybrid simulation soware Any-
logic was used to evaluate the impact of disruptions in the cork industry, dierent disruption scenarios were
analyzed and numerically represented38. According to a recent (2021) review on resilience in Cyber-Physical
System39, from 390 relevant articles 32 papers were in the domain of manufacturing and 24 of the papers had
resilience metrics related approach. Still, none of the 32 papers in the domain of manufacturing had resilience
metrics approach. us, resilience metrics in CPPS have not been in the focus of recent research.
Discussion of existing literature. Resilience is abstract concept with many denitions. Which proper-
ties it exactly gathers and what is their allocation are still under scientic discussion. e related characteristics
mainly brought out in various approaches are reliability, redundancy, exibility, sustainability, and maintaining
of functionality. Resilience is rather imagined as dynamic temporal measure that needs constant input for its
quantication. What is agreed is that resilience is a valuable indicator, and its quantication enables companies
to assess their long-term success and to be better prepared for disruptions. Despite many approaches to quantify
resilience in manufacturing, no industry standard metric is recognized so far.
Applying of resilience metrics in CPPS have not been in the research focus. Although CPPS have higher
potential to contribute with numerical inputs to the resilience assessment, these have not been in standalone
observation in resilience perspective so far. However, as explained earlier, many overall engineering and manu-
facturing resilience metrics have been proposed. e further analyze of these metrics’ gains in recognition of
CPPS suitable metrics. eir deeper analyze may come up with most practically suitable solution to be considered
to become a recognized standard.
Methodology
is study adopts a descriptive literature review methodology (a) to nd the most common disruptions in manu-
facturing and (b) to nd and compare the resilience metrics which can be used in the resilience quantication
of manufacturing workstations. For the investigation of mentioned disruptions, a quantitative meta-summary
was used. For resilience metrics comparison, multi-criteria decision-making AHP analysis in linear scale was
applied. e main sequence of research methodologies is presented in Fig.3.
Literature search and evaluation for inclusion. e literature review was conducted to investigate
the research questions. Scopus database was selected as a source for literature as it is internationally recognized,
in addition, it contains a large number of publications from the area of engineering sciences. e search was
performed on the 9th of March 2022. Search query string (TITLE-ABS-KEY ("resilience assessment" OR "resil-
ience metric*" OR "resilience measure*" OR "resilience analysis" OR "resilience quantication" OR "evaluating
resilience") AND ALL ("manufacturing plant" OR "manufacturing system" OR "resilient production" OR "engi-
neering resilience") AND NOT ALL (ecosystem) AND NOT ALL (seismic)) OR (TITLE-ABS-KEY (resilience
Resilience assessment approaches
Qualitave assessment Quantave assessment
Conceptual frameworks Semi-quantave indices General measures Structural-based models
Probabilisc approaches Opmizaon models
Determinisc approachesSimulaon models
Fuzzy logic models
Figure2. Classication of resilience assessment methodologies (redrawn from Ref.30).
Content courtesy of Springer Nature, terms of use apply. Rights reserved
5
Vol.:(0123456789)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
AND (assessment OR metrics)) AND TITLE-ABS-KEY (manufacturing)) AND (LIMIT-TO (LANGUAGE,
"English")) AND (EXCLUDE (LANGUAGE, "Spanish") OR EXCLUDE (LANGUAGE, "Italian")) was chosen
for literature search. e query string was divided into three parts (divided by brackets). e rst part focuses
on publications regarding the resilience quantitative approach, while extra to manufacturing eld engineering
resilience (as manufacturing systems and workstations are part of it) was also included. Whereas manufacturing
and engineering-related keywords were searched from all text to receive a wider scope of results. In turn, it was
needed to exclude some query terms from environmental topics (ecosystem, seismic) to stay in scope. e sec-
ond part of the query string covers resilience in manufacturing overall, these were only searched from abstracts,
keywords, and titles as these keywords are more comprehensive. erefore, the terms “assessment” and “metrics”
were allowed to be separated from the term “manufacturing”. e third part limits results to the English language
only. Nevertheless, Italian and Spanish needed to be separately excluded to receive only results in English. To
cover possible disruptions over time, no time limit was set. Secondary documents and patents were excluded.
e rst screening included the reading of abstracts. If the abstract supplied no sucient information, the
content of a publication was also overlooked. Only publications with a focus on topics such as manufacturing,
industry resilience generally and engineering system resilience were approved for inclusion criteria. Manufactur-
ing-related publications were included, except food and oil industry. Additionally, manufacturing-related product
design, risk analysis, and supply chain publications were excluded. Whereas system engineering publications
were included only if these were focused on resilience metrics. In addition, full proceedings and duplicates were
excluded. Moreover, inaccessible publications were excluded as well.
e second round of screening consisted of reading the papers. Included were only papers mentioning
disruptions (including disruptive events, adverse events, shocks, and detrimental events) in manufacturing or
contributing resilience metrics for manufacturing. Resilience design related documents were excluded.
Disruptions. For the investigation of mentioned disruptions, a review methodology quantitative meta-
summary40 was applied. First, descriptive expressions which could be viewed as disruption causes or disruption
modes were extracted. ereaer, expressions describing the same disruption with dierent words were identi-
ed and rephrased to describe their common meaning. Frequency Eect Sizes of mentioned disruptions and
Intensity Eect Sizes41 of articles were calculated and analyzed.
As papers focusing on supply chain were excluded, also general supply chain disruptions pointed out in
included articles as a cause of disruptions were not separately categorized to avoid distortions in results. Supply
chain disruptions caused specic consequences more related to manufacturing processes were listed instead.
e list of mentioned disruptions was divided into external and internal disruptions. External disruptions
were divided into subcategories based on STEEPLE (Social, Technical, Economic, Environmental, Political, Legal,
and Ethical), which has been developed for the analysis of key system elements in manufacturing42. Still, some
STEEPLE categories were united having a close relationship with each other in the resilience context (social and
ethical; political and legal; technical and economic), while technical was replaced with technological.
For the classication of internal disruptions, the model of the Automation Pyramid43 was taken as a basis.
Although, modern workstations are highly automated, still human in the loop is oen present. erefore, the
human aspect was included for levels 2–4, whereas operator-related disruptions were covered in the second level.
us, internal disruptions were divided into ve subcategories as follows: eld, control, operator, planning, and
management.
Inial literature search
(Scopus search)
Screening 1
(Reading abstracts)
Screening 2
(Reading full papers)
Data
extracon
(resilience
metrics)
Comparave table (RQ2)
Inial no of included papers
No of excluded papers
No of excluded papers
No of included papers
Final no of included papers
Data
analysis
(AHP)
Priority table (RQ3)
Evaluaon criteria’s
Data
extracon
(menoned
distur-
bances)
List of most
menoned (RQ1)
Figure3. Main methodology of the research.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
6
Vol:.(1234567890)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
Some of the mentioned disruptions can reect both, external and internal disruptions. In this case, the fol-
lowing characteristics were analyzed. At rst, the category with stronger inuence was identied (e.g., internal
social unrest is with higher inuence). Secondly, it was asked if a company possesses direct and fast inuence
over the disruption. If the answer was negative, still external category was selected (e.g., availability of investment
capital). If mentioned disruption has consequences in at least two subcategories, the source subcategory was
detected (e.g., Covid-19 source category is social, although political and economic aspects were also present). In
categorizing internal disruptions, the automation pyramid lower level was preferred (e.g., the material shortage
was categorized as management-related disruption, while material delay and poor material supply inuence
post-planning activities and was categorized accordingly).
Dierent expressions for the same or similar disruption were counted as the same type of disruption (“dete-
rioration in quality output” and “quality aws” were both counted as “output quality aws”). If a more specic
reason was also brought out, the disruption was counted separately and not listed as a related general disruption
(“bias of the pallet (tire treads exceed on the side)” refers to quality but was listed separately).
Resilience metrics comparison. Multi-criteria decision-making tool, standard linear scale AHP44 was
used to compare extracted resilience metrics’ practical value in digital twin supported worker assistance system
in manufacturing. For AHP process parameters set up and calculation, an online tool AHP-OS45 was used.
Comparison criteria were chosen as the most important characteristics for real-life practical usability:
• Feasibility—not only theoretical, supported with case studies or examples, real-life tested and repeatable.
• Relevance—suitability for this specic application: digital twin supported human-centered assembly station.
• Accuracy—more measuring or experiment based and less probabilistic or expert opinion driven.
• Comprehensiveness—takes into account a wide variety of possible disruptions, including dierent agents
such as humans and technology-related components as well as the external and internal types of disruptions.
• Comparability—comparability levels: comparable with the same type of workstations, comparable with a
dierent type of workstations in the same plant, comparable between dierent companies.
One level hierarchy of AHP analysis was used. Comparison criteria weights (Table1) were derived by pair-
wise comparison of criteria on a ratio scale from 1 to 9 (scale dened by the soware). e scale of comparison
characterizes the intensity of importance as follows: 1—equal importance (two elements contribute equally to
the objective), 5—moderate importance (experience and judgment moderately favor one element over another,
9—strong importance (experience and judgment strongly favor one element over another). e values between
these numbers characterize intermediary levels accordingly.
For resilience metrics pairwise comparison the previously described AHP analysis in scale from 1 to 9 was
used again in the same way. e process of comparison started with the decision which metric is more valuable
under specic criterion and secondly how much more in given scale (Fig.4). e comparison was made by two
authors together on a consensus basis. In cases the consensus could not be found between these authors, an
additional expert (in the eld of sustainability and resilience) opinion was asked. In each pairwise comparison,
Table 1. Weights of comparison criteria for AHP.
Criterion Priority (%)
Feasibility 38.1
Relevance 38.1
Accuracy 3.2
Comprehensiveness 8.0
Comparability 12.6
Figure4. Pairwise comparison of Zhang etal. proposed resilience metric against other selected metrics under
criteria of relevance (screenshot from AHP-OS soware).
Content courtesy of Springer Nature, terms of use apply. Rights reserved
7
Vol.:(0123456789)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
consistency ratio was calculated for inconsistency assessment. As the comparison result was a collective decision,
some inconsistency is admissible. Aer the comparison of all pairs, the decision matrix was received (Fig.5).
Based on decision matrixes, resilience metrics consolidated priorities were calculated under each comparison
criterion (Fig.6). Subsequently, each resilience metric consolidated priority scores were summed to receive the
nal AHP analysis results. To illustrate the process, only criterion of relevance related gures are presented in
the article. e rest of the pairwise comparisons can be found in Supplementary Fig. S1.
Figure5. Decision matrix for the criterion of relevance (screenshot from AHP-OS soware). e row no. 1 and
the column no. 1 correspond with Fig.4 comparison results.
Figure6. Consolidated priority scores under the criterion of relevance comparison (screenshot from AHP-OS
soware).
Inial literature search: 247 arcles
Screening 1: 67 arcles le (based on abstracts: resilience in manufacturing)
Screening 2: 14 arcles le (based on full arcles: resilience metrics or
disturbances in manufacturing)
Menoned disturbances: 14 arcles Resilience metrics: 8 arcles
Figure7. Literature review in numbers.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
8
Vol:.(1234567890)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
Results
Literature search and screening. Based on the literature review, quantication of resilience in manufac-
turing (excluding design of manufacturing systems) processes was provided in 8 articles and specic disruption
causes were mentioned in 14 articles. Screening of review results is presented in Fig.7. More detailed screening
details can be found in Supplementary TableS1.
90% of initial literature search articles were published in 2013 or later, whereas over 50% of articles were
published within the last 3years. Hence state-of-the-art articles were found without limiting the year of publica-
tions. e rst screening excluded 180 articles, mostly from other engineering areas. Namely (listed decreasingly)
supply chain, infrastructure, power engineering, environment, medical engineering, economy, construction
engineering, material engineering, product design, and others (Supplementary TableS1). e second screening
excluded 53 articles proposing no relevant resilience metrics nor mentioning specic manufacturing-related
disruptions. e second screening revealed that relevant manufacturing-based resilience metrics were proposed
in 12% of manufacturing resilience related articles, while disruptions (disruption causes and disruption modes)
were mentioned in 21% of the articles. Whereas all articles in which resilience metrics were proposed, the dis-
ruptions were also mentioned.
Manufacturing inuencing disruptions. e ndings of disruption mentions were found in articles
published from the year 2015 to 2021. In newer articles the average number of ndings is smaller, resulting in
57% of ndings from the year 2016. A total of 86 disruption mentions were extracted (Table2). e list of dis-
ruptions is highly inuenced by two articles, a total of which provide 50% of the ndings. e article46 with the
highest intensity eect size of all ndings (22 ndings) is general smart manufacturing systems based and refers
to agile manufacturing when listing disruptions. e second highest intensity eect size of all ndings (21 nd-
ings) article47 is also general by focusing on re-distributed manufacturing.
Of 86 mentions, 58 disruptions were interpreted as unique: 27 as external and 31 as internal (Table3). Men-
tioned disruptions were sometimes general as machine fault, but oen specic as solenoid valve disfunction or
dye stripping fault. Whereby, in some articles, general disruptions were named and more specic reasons were
brought out in addition. Regardless of their comprehensiveness level, all mentioned disruptions were analyzed
equally to receive a natural view of mentioned disruptions.
A total of 7 disruptions (hereinaer eective disruptions) received mentions from > 20% of articles. e most
frequently mentioned disruptions were “natural disaster, “covid-19” and “changes in availability of materials and
parts” (Table4). From these disruptions, in some case-specic disruptions and their higher-level general disrup-
tions were represented (covid-19 belonging under pandemics and earthquake belonging under natural disaster).
Intensity eect size > 20% (Table2) characterizes the percentage of eective ndings (mentioned > 20% of
articles) in a certain article from all eective ndings. Intensity eect size shows the importance and trustfulness
of the article regarding the ndings.
Unique external and internal disruptions show approximately equal mentions, 27 and 31 accordingly. How-
ever, frequency eect size shows more disruptions in external than internal, 6 vs 1 disruption. is denotes, the
internal category includes more single-mentioned disruptions. It reects many internal disruptions being a com-
pany or narrow manufacturing eld or system specic whilst external disruptions are more common overall in
manufacturing companies. For external disruptions, the category economic and technological received the most
mentions and the most unique mentions. For internal disruptions, planning-related disruptions were the most
mentioned (both, total mentions and unique mentions). In some subcategory disruption lists, general and more
precise disruptions (belonging under the same general disruption) are both represented. is illustrates mostly
Table 2. List of articles with ndings of disruption mentions and their intensity eect sizes of all ndings (86
ndings) and intensity eect sizes of ndings with frequency eect sizes > 20% (7 ndings—Table4).
Authors of the article Year No. of ndings Intensity eect size of all ndings Intensity eect size > 20%
Peng etal.48 2021 1 0.01 0.14
Zhang etal.49 2021 12 0.14 0.57
Alexopoulos etal.50 2021 3 0.03 0.43
Latsou etal.36 2021 3 0.03 0.14
Song etal.19 2020 1 0.01 0
Okorie etal.51 2020 3 0.03 0.43
Diaz-Elsayed etal.52 2020 2 0.02 0.14
Caputo etal.53 2019 1 0.01 0.14
Li etal.54 2019 5 0.06 0.29
Yoon etal.12 2017 1 0.01 0
Kibira etal.46 2016 22 0.26 0.29
Jin etal.55 2016 6 0.07 0.57
Freeman etal.47 2016 21 0.24 0.29
Gu etal.92015 5 0.06 0.43
Tot a l 86 1
Content courtesy of Springer Nature, terms of use apply. Rights reserved
9
Vol.:(0123456789)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
the disruptions which were more oen mentioned (natural disasters, covid-19, machine faults/breakdown).
us, regardless of modest sampling data, more oen mentioned disruptions can be conrmed are trustful as
they were supported by both, general and specic disruption reasons.
Resilience metrics review and analysis. Eight articles with relevant resilience metrics (Table5) were
identied and researched to compare their features in ve categories using AHP analysis. As a result, the meth-
odology Penalty of Change (POC) was evaluated as the most suitable and practically usable quantitative resil-
ience assessment metric for digitally supported human-centric workstations in SMEs.
Descriptive review. Eight resilience metrics are further described below and specically dened in Table6 in
following related terms: metric symbol and name, formula and symbols denitions, denition of concept resil-
ience, and case study or example use.
Reference49 developed digital twin platform for resilience automatic analysis for recongurable electronic
assembly line by using a systematic method based on max-plus algebra. e solution was tested in a smartphone
assembly line where 6 disruptions were used to attack the system randomly. Two indicators were used for bot-
tleneck vulnerability estimation: Vulnerability Time Delay—the time interval between the occurrence moment
of a disruptive event and the moment of production stoppage at the bottleneck station and Vulnerability Time
Window—the time interval between the occurrence of a disruptive event and the time point where permanent
production losses occur. e resilience metric is calculated as loss of production. e calculation considers also
buers and historical information regarding potential fault modes and their repair time.
Reference50 assessed resilience in manufacturing plants by calculating a generic measure of POC. Inputs for
calculation are the cost of the potential change (equipment investment, labor training, reprogramming, oppor-
tunity cost, and others) and the probability of change, where a ‘change’ denotes a transition from a current ‘state’
of a manufacturing system to a new state. It is relatively easy to be applied to realistic manufacturing situations.
Table 3. Mentioned unique disruptions based on category.
Disruption category No. of unique disruptions Disruptions
External 27
Environmental 8 Fire, earthquake, ood, natural disaster, hurricane, extreme climatic events, shis in weather patterns, climate change
Political and legal 3 Changing regulations, sudden changes in political landscape, role of global non-prot and philanthropy organizations
Social and ethical 5 C ovid-19, pandemics, terrorism, changing demographics, shocks that change ethical stances
Economic and technological 11
Modications in the demanded volume of product(s), power or water outages, changes in availability of materi-
als or parts, changes in cost of materials or parts, availability of investment capital, economic downturns/upturns,
globalization, future markets, dynamic of technology and innovation, supplier bankrupt, uncertainty and dynamicity
environment
Internal 31
Field 6 Machine faults / machine breakdown, sensor faults, screwdriving device wear, vacuum absorption, solenoid valve
malfunction, dye stripping fault
Control 2 Control network fault, system connection failure
Operator 7 Fluctuation of processing time, labor tiredness, output quality aws, bias of the pallet, labor shortage, social unrest,
changes in workforce
Planning 9 Scheduled maintenance, delayed material supply, changed product routing, customer order reprioritization, rush
order, order change, mass customization, poor material supply, order cancellation
Management 7 Product returns, new equipment installation, new production line conguration, new soware installation, new prod-
uct introduction, changes in business ownership, changed product line
Tot a l 58
Table 4. Eective ndings (frequency eect size over 20%) of disruptions.
Subcategory Disruption No. of mentions Frequency eect size
External
Environmental Natural disaster 5 0.36
Environmental Earthquake 3 0.21
Social and ethical Covid-19 5 0.36
Social and ethical Pandemics 3 0.21
Social and ethical Terrorism 3 0.21
Economic and technological Changes in availability of materials or parts (shortage) 5 0.36
Internal
Field Machine faults/breakdown 3 0.21
Content courtesy of Springer Nature, terms of use apply. Rights reserved
10
Vol:.(1234567890)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
Metric was tested by a hypothetical case study in two dierent production systems: a 3-D printing farm and an
injection molding factory, during the Covid-19 pandemic.
Reference19 proposes a hybrid global optimization approach to assess a service composition and optimal
selection in cloud manufacturing, where resilience is one of the attributes. e inputs for resilience calculation
are dierent attributes of endogenous and exogenous equilibriums that are nally compared with resilience that
is expected from the service demander. e solution was tested by a series of experiments.
Reference53 developed a quantitative method for manufacturing company resilience assessment by evaluating
the initial capacity loss aer a disruptive event occurrence, time-dependent capacity recovery path, economic
loss due to capacity reconstruction, and business interruption. e process was divided into 7 steps as follows:
process ows mapping; construction of process Capacity Block Diagram; construction of Overall Reconstruction
Activities Network; damage scenario denition; computation of initial capacity loss; determination of capacity
recovery function; determination of economic loss.
Reference54 presented a quantitative resilience assessment architecture for a material handling system, includ-
ing material transportation, picking, and storage. e used methods were Disruption Mode and Eects Analysis
(DMEA) and the Monte Carlo method. is assessment basis on a comparison of simulation runs with and with-
out disruption. With the tool of the DMEA system response for each disruption is found. For material handling
system evaluation and modeling concerning resilience, the following input is needed: system conguration data
(equipment layout parameters, system composition, equipment functional parameters), simulation-related data
(iteration number, time duration, granularity of simulation), disruption-related data (disruption probability,
minimum acceptable value of resilience, the shape of performance degradation curve and recovery curve). It
was practically tested in a tire tread handling system, where 86 dierent disruption modes and disruption causes
were identied.
Reference12 proposes probabilistic resilience metric which considers false alarm (false fault and false health)
rates and reliability. A case study demonstration was carried out in an electro-hydrostatic actuator. e metric
allows for estimating a system resilience more rigorously and accurately by also considering sensor faults (false
alarms) in addition to the other factors.
Reference55 dened 3 resilience metrics: performance loss—system performance loss during the transients of
a disruptive event (it can be either loss of productivity, reliability, or available functions); performance restoration
time—the time the system takes to restore its throughput to a predetermined threshold and, total underper-
formance time—the period during which the system capacity is lower than a predetermined threshold. It charac-
terizes both, spatial and temporal characteristics. e metrics depend on the characteristic of a disruption, system
congurations, machine reliabilities, and buer capacities. Both, temporal and spatial aspects are considered. A
case study was conducted using a system comprising six production units and two variations in conguration.
Table 5. Comparison of resilience metrics based on quantication methods, strengths, and limitations.
Article Methods Strengths Limitations
Zhang etal.49 Based on max-plus algebra Uses a digital twin based automatic resilience
evaluation system
It focuses on system internal disruptions only.
Needs historical datasets regarding fault modes to
evaluate resilience
Alexopoulos etal.50 Generic algorithm (probabilistic)
It combines both technological and economical
terms and requires no large and complex amounts
of data for calculations. Disruptions are observed
as an ignition for system changes. Production-
related aspects, such as varying types of products,
operational status, and varying demand, can be
described and utilized in a common context
Dependent on disruptions occurrence probabilities
estimation
Song etal.19 Fuzzy logic and generic algorithm e resilience model can be also used to solve other
combination problems. Considers also cost and
reputation factors Focus on cloud manufacturing only
Caputo etal.53 Generic algorithm (deterministic)
Step by step description of the process explained.
Based on resilience, economic loss is calculated.
Manufacturing was observed as the quality of
service to estimate resilience
Addresses full plant processes and systems, rather
than one workstation. No experiment or case study
was included
Li etal.54 DMEA and Monte Carlo simulation based (deter-
ministic) Considers dierent types of resilience behaviors
based on specic disruption Only considers internal disruptions and needs
historical datasets for a bottom-up approach
Yoon etal.12 General algorithm (probabilistic) It is based on the existing resilience equation in
which restoration as one component is considered
Focus and description are on sensor false alarms.
Systems using prognostics and health management
techniques were considered only
Jin etal.55 Generic algorithms (probabilistic)
Expands the manufacturing resilience approach
by dening 3 resilience metrics: performance loss,
performance restoration time, and underper-
formance time
Case study set-up and resilient calculation not
described, but only mentioned
Gu etal.9Generic algorithms (probabilistic)
Expands the manufacturing resilience approach
by dening 3 resilience metrics: production loss,
throughput settling time, and total underpro-
duction time. Compares resilience to dierent
company policies
Addresses full plant processes and systems, rather
than one workstation
Content courtesy of Springer Nature, terms of use apply. Rights reserved
11
Vol.:(0123456789)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
Article Metric Formula and symbols Resilience denition Example
Zhang etal.46 Re
Re
=1/
TS
0
(TPE
−TP)
dt
where TS is the total working time of the system, TPĒ is throughput without
disruptions and TP is system throughput
Ability to maintain
production under
disruptions. It meas-
ures production losses
under disruptions
A digital twin testing
platform for smart phone
assembly was developed
for resilience control of
disruptions
Alexopoulos
etal.47 POC—penalty of possible
changes
POC
=
D
i=1Pn(Xi)
Pr
(Xi)
where D is the number of potential changes, Xi is the i-th potential change, Pn(Xi)
is the penalty (cost) of the i-th potential change and, Pr(Xi) is the probability of
the i-th potential change to occur
Changing the system
encompasses a poten-
tial penalty that may
include relevant costs:
equipment investment
(machines, tooling,
etc.), labor training,
reprogramming,
opportunity costs and
others
COVID-19 related pilot
case applied to two hypo-
thetical manufacturing
systems (3D printing farm
vs. injection molding) that
produce plastic products for
the automotive industry
Song etal.19 Q4(ψi,j)
max Q
4(ψi,j)=
N
j
=1ψi,j(
µ
i,j
M
M
o
=1σi,j,men
Nin,Nout
i
,
j
,
m
+νi,j
N
L
p
=1ρi,j,lex
Nin,Nout
i
,
j
,
l)
where Q4(ψi,j) is the resilience of manufacturing service of the i-th candidate of
the j-th sub-task; M and L represent the number of endogenous attributes and the
number of exogenous attributes, respectively; σi,j,m and ρi,j,l represent the weight of
each endogenous attributes and the weight of each exogenous attributes, respec-
tively; μi,j and νi,j represent the weight of total endogenous attributes and the
weight of total exogenous attributes, respectively; eni,j,m(Nin,Nout) and exi,j,m(Nin,Nout)
are calculated by:
E
A=
t
b
ta
VA−VE
A
dt
max (VA)−min (VA)
where EA refers to the measured value corresponding to equilibrium A; VA repre-
sents real-time measured value of equilibrium A; VA(E) represents measured value
of equilibrium A in consistent operation; ta and tb represent the establishment
time and nal time of the inspection. e denominator indicates the value of the
maximum change of measured value during the inspection period
Resilience is an
attribute of the
service, which is used
to measure the insen-
sitivity of the system
to disturbances
Resilience calculation was
one attribute of hybrid
resilience-aware global
optimization (HRGO)
approach. e other HRGO
attributes are cost, time,
and reliability. Based on
HRGO, service composition
and optimal selection is
tested in a company Chery
Automobile Co, where two
scenarios are considered
and compared
Caputo
etal.50 Resilience (Caputo etal.)
Resilience
=
1
tr−t0t
r
t0
C(t)dt
where C(t) is nominal capacity, t0 is time of disruption and (tr–t0) is recovery
interval
Resilience is a
performance measure
representing the sys-
tem ability to survive
disruptive events, and
the rapidity in restor-
ing system capacity
aer the disruptive
event has occurred
N/A
Li etal.54
RA
-Estimation of expected
system resilience
RA
=E(R
D
)and
R
A
=
n
i=1RD,i
n
where n is the number of the deterministic resilience, RA is the expectation of sys-
tem resilience and reects the average resilience of the system,
RA
is an estimate
of RA and is calculated by taking the average of the deterministic resilience under
dierent disturbances
Technical resilience
refers to the ability of
the system to perform
at an acceptable level
when the disturbance
occurs. Economic
resilience refers to the
capacity of the system
to reduce both direct
and indirect economic
losses resulting from
the disturbance
In automatic tire tread
handling system subjected
to random disturbances,
the resilience was evaluated
based on 1000 Monte Carlo-
based simulation runs and
proposed disturbance mode
and eects analysis
Yoon etal.12 ΨFA—resilience measure
that considers false alarms
ψFA =Pr(ˆ
HH)+Pr(Emr
FF)
where Pr(ĤH) is probabilities of “system normal” operation and Pr(EmrḞF) is
system restoration rate
Engineering resilience
is the ability of a com-
ponent or a system to
maintain its required
functionality by resist-
ing and recovering
from adverse events
Two case studies were
employed and resilience cal-
culated. e rst examines
numerical examples and the
second studies an electro‐
hydrostatic actuator
Jin etal.55 RM—resilience metric
Spatial characterization:
RMSub
j
i
=RM
Sub
j
i
ϕ
d
;ϕS;ϕM
1
,ϕM
2
,...,ϕM
M
;ϕB
1
,...,ϕB
B
Temporal characterization:
RM
[T]
i
=
N
T
l=1(ϕdl;ϕS;ϕM1,ϕM2,...,ϕMM;ϕB1,...,ϕBB
)
1
where
RM
Sub
j
i
is the ith resilience metrics for a subsystem j; φd is the set of param-
eters that describe the disruption (e.g., starting time, duration, location); φS is the
information related to the system conguration (e.g., serial, parallel, hybrid);
ϕMi
is a set of parameters that characterie component Mi (e.g., reliability);
ϕBi
repre-
sents the attributes of other connecting components Bi for i = 1,2,…,B;
RM[T]
i
is
the ith resilience metrics over a period of time T; N is the number of total number
of disruptions that may occurs during time T;
ϕdl
is the set of parameters for the
lth disruption; represents the additivity of multiple disruptions
Resilience is the ability
of a system to with-
stand potentially high-
impact disruptions,
and it is characterized
by the capability of
the system to mitigate
or absorb the impact
of disruption, quick
recover to normal
conditions
A case study was conducted
using a six -machine system
with two variations in
conguration
Continued
Content courtesy of Springer Nature, terms of use apply. Rights reserved
12
Vol:.(1234567890)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
Reference9 described three resilience metrics (throughput settling time, production loss, and total underpro-
duction time) and analyzed them using the Bernoulli reliability model. e proposed solution was tested by a case
study. e main authors overlap with55, therefore these two articles can be considered extensions of each other.
AHP analysis. e AHP multi-criteria decision-making was used on a standard linear scale to compare the
selected resilience metrics in ve criteria: feasibility, relevance, accuracy, comprehensiveness, and comparability
(Fig.8). In pairwise comparison, the highest consistency ratio received is 4.3%, this was received from pairwise
comparison under the criterion of comparability. In pairwise comparison, this level of inconsistency is allowed,
and it does not inuence the reliability of the results.
e article “A quantitative approach to resilience in manufacturing systems” written by Kosmas Alexopoulos,
Ioannis Anagiannis, Nikolaos Nikolakis and George Chryssolouris received an AHP analysis score of 24.3%
which is the highest score received. is is fresh research, published in the International Journal of Produc-
tion Research in the year 2022. It proposes a methodology called POC which received the best results under
the criteria of feasibility and comprehensiveness. While generic probabilistic algorithm extended with sensor
faults consideration (16.3%) and max-plus algebra based systematic approach (14.4%) also received relatively
higher scores compared with other metrics, whereas these articles received the highest score in the second most
inuencing criteria—relevance.
POC resilience metric equations. POC is practical for manufacturing companies as it is a generic algo-
rithm with relatively simple inputs and illustrated with a sample case study. It considers the changes related to
cost which is an important factor. As manufacturing systems are considered continuous systems, there is an
innite number of potential transitions, and the changing scenario is continuous. erefore, additionally to the
main POC formula (Table6), in a dynamic system, the POC can be calculated as follows50:
(1)
POC
=
X
2
X
1
Pn(X)Pr(X)dX
,
Table 6. Comparison of resilience metrics based on mathematical formulas, resilience denitions and
application examples.
Article Metric Formula and symbols Resilience denition Example
Gu etal.9
PLP—production loss;
TSTP
ε
—throughput set-
tling time;
TUTP
ε
—total under pro-
duction time.
PL
P=tD
To
I
(0)PRS−tD−t
P
k=tP+IPRP(k
)
+
∞
k=tD
To
l
(0)+1(PR
S
−PR
P
(kT
o
I(0
)))
where
tP=tR∗I{P=B}
;
TST
P
ε=max
k|k≥tD
To
I
(0),PRP(kTo
I(0)) < (1−ε)PRS
To
I(0)+To
I(0)−t
D
TUTP
ε=tD+
∞
k=tD
To
I
(0)+1I
PR
P
(kT
o
I(0)) < (1−ε) PR
S
T
o
I(0)
−
tD−2tP
TP
I(tP)
k=1I
PRP(tP+kTP
I(tP)≥(1−ε)PRS×TP
I(tP)
To
I
(0)
TP
I(tP
)
where PRP(k) is the production rate at time k under policy P (P = A, B, or O)
(hereaer superscript ‘P’ is the corresponding performance under policy P); I is
number of stages of the system; Ti(k) is cycle time for each machine in stage i at
time k; tD is duration of the disruption; tR is the time of reconguration; (1−ε) is
the steady-state value of production rate of the system; I{X}} is an indicator func-
tion, representing the true(1)/false(0) value of the statement X;
Resilience is the ability
of a system to with-
stand potentially high-
impact disruptions,
and it is characterized
by the capability of
the system to mitigate
or absorb the impact
of disruptions, and
quickly recover to
normal conditions
Numerical case studies were
conducted to investigate
how the system resilience
is aected by dierent
design factors, including
system conguration, level
of redundancy or exibility,
and buer capacities
Figure8. Decision matrix with comparison criteria weights, resilience metrics consolidated priority scores, and
nal AHP analysis results comparison (screenshot from AHP-OS soware).
Content courtesy of Springer Nature, terms of use apply. Rights reserved
13
Vol.:(0123456789)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
where X1 is the lowest value of the potential change X, X2 is the highest value of the potential change X, Pn(X) is
the cost distribution and Pr(X) yields the probability distribution of the potential change.
POC can be modied for dierent approaches. For measuring temporal penalty, the cost factor can be changed
with the time factor. e main weakness of the methodology is the dependency on probabilities and related costs
which decreases its accuracy.
Discussion
e research used a literature review methodology to provide an overview of dierent resilience metrics and
disruptions. Followed by the search string and screenings, 8 resilience metrics were identied that could be used
in the assessment of resilience in manufacturing workstations. Further, the multi-criteria decision-making tool
AHP pairwise comparison was applied under ve weighted comparison criteria to analyze the metrics practical
usage for manufacturing rms. As a result, the resilience assessment metric POC was selected as the metric with
the highest value in practical usage for human-centered CPPS workstations. It received the highest score in two
criteria: feasibility and comprehensiveness. A high feasibility score was the result of its clear structure and generic
equation. is supports its implementation even without higher mathematics skills, which favors its wide-scale
usage. High comprehensiveness score was earned by its generic structure that allows counting internal, external,
human-related, and machine-related disruptions. Additionally, frequency eect sizes for extracted disruptions
were found to highlight the most inuencing for manufacturing companies.
According to the results, sub-RQ-s are answered as follows:
RQ1. Natural disasters, Covid-19, changes in the availability of materials or parts (shortage), earthquakes,
terrorism, pandemics, and machine faults/breakdown are the disruptions potentially inuencing manufacturing
workstations the most.
RQ2. e existing quantitative metrics are presented in Table5 which allow to measure the resilience of exist-
ing manufacturing workstations.
RQ3. POC50 is the most practically valuable quantitative resilience metric to assess the level of resilience in
human-centered CPPS workstations in SMEs.
e research is not answering if the most mentioned disruptions have a higher rate of occurrence, more criti-
cal consequences, or the highest risk (probability multiplied by cost) for manufacturing companies. All these can
be reasons for their frequent highlighting in research papers. As the collected disruptions were collected from full
articles, thus many of the disruptions were collected from introductions where mostly general and topical disrup-
tions were brought out as a list of examples. While from case studies more specic disruptions were collected.
Overall, it can be still concluded that the highest frequency eect size disruptions have a relatively higher inu-
ence on manufacturing plants regardless of their core reason for mention. To generate more specic conclusions
concerning mentioned reasons and dynamics, an explicitly structured data collection process should be followed.
Publishing time aects more specic disruptions, for instance newly appeared technology or diseases related
disruptions. General disruptions are less inuenced by timing. For instance, Covid-19 as a specic pandemic was
rstly called in 2020 and received a high score, while pandemics were also mentioned in earlier years. erefore,
the list of disruptions should be considered as dynamically changing in time.
An interesting result is that Covid-19 as a specic disruption received a higher eect size than its general
equivalent, while earthquake and natural disaster eect sizes are vice versa. It shows a specic pandemic higher
inuence on manufacturers than pandemics overall. is can be concluded by a rare rate of occurrence of pan-
demics as well as the high level of potential consequences if one should emerge.
Resilience can be evaluated at dierent levels, such as company level, manufacturing system level, and work-
station level. Mostly, the compared resilience metrics were designed to use at the manufacturing system level.
Many of them concentrated to assess the rearrangement possibilities of current resources using redundancy of
workstations (machinery) and workers. Workstation level resilience assessment involves a more complex struc-
ture of possible solutions as it goes into more detail about specic components (sensors, actuators, etc.) while
servicing subsystems (information availability, warehouse, planning) are present at both levels. erefore, general
manufacturing system-related resilience metrics can be also applied at the workstation level. In the same way, bot-
tlenecks as critical resources can be viewed and managed at the manufacturing system level and workstation level.
POC received a high score in feasibility which reects its practical usability. It considers the cost of changes
which is an important factor as it helps to analyze and balance potential cost uctuations. e POC is a universal
metric as the cost factor can be easily changed to the time factor where needed. e weakest side of this metric
is its dependency on the probability of potential changes. e metric Re49 received the highest score under
comparison criteria relevance. It can be considered to be used in companies where digital twin is already imple-
mented, and historical machinery-based historical datasets are collected. e other reviewed resilience metrics
can be used as supportive tools for POC, for instance considering sensors false alarms or cloud manufactur-
ing. Considering all the benets as well as the feature that POC is a dimensionless quantity that is comparable
between various workstations in a company as well as between various companies, its potential applicability in
the manufacturing sector is high.
e main limitation is the accuracy of potential disruptions behavior prediction and their temporal factor.
Generally, disruptions can cause unavailability of current resources (machinery faults, material shortage), insta-
bility of current resources (uctuation of processing time, quality aws, poor material), or potential need for a
new type of resources (changing regulations, new equipment installation). Recovery of current resources covers
the knowledgeable zone, which is more accurately estimable, while implementing new resources may involve
an unknowledgeable zone. erefore, exibility is not only needed inside the manufacturing system but also in
terms of openness to changes in the business environment on a broader scale. e need for new types of resources
can be caused for instance by changes in legal regulations as an extra need for safety tests or by new equipment
Content courtesy of Springer Nature, terms of use apply. Rights reserved
14
Vol:.(1234567890)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
installation as a need for operators with specic knowledge and/or experience. e resilience of known resources
is relatively easier to assess, while readiness for implementation of unknown resources is a more abstract feature.
Based on historical datasets, estimation of probabilities of disruptions as a function of time is well predictable
at the eld level mostly but leaves higher tolerance in other subcategories. An alternative approach would be to
analyze the consequences of dierent combinations of cut-o or unstable resources (prioritizing bottlenecks),
instead of focusing on certain specic probabilistic disruptions. As every disruption inuences certain resources,
while the number of possible disruptions is unlimited, the analysis of a limited number of disruptions may pro-
vide noncomplete or even misleading results in a sense of resilience. erefore, research focusing on the modeling
of external disruptions’ potential occurrences and their temporal behavior in manufacturing is further needed
to maximize the accuracy of resilience assessment.
In our further research, POC will be used as functional input for Axiomatic Design based decomposition of
resilient CPPS. is generates design guidelines for monitoring system architecture for resilient manufacturing
system in digital twin perspective.
Data availability
e data that support the ndings of this study (Supplementary TableS1 and Supplementary Fig. S1) are openly
available for download in Figshare repository at https:// gsh are. com/ a rtic les/ datas et/ Analy sis_ of_ Quant itati ve_
Metri cs_ for_ Asses sing_ Resil ience_ of_ Human- Cente red_ CPPS_ Works tatio ns_ Suppl ement ary_ Table_ S1_ and_
Figure_ S2/ 19307 129 (DOI: https:// doi. org/ 10. 6084/ m9. gsh are. 19307 129).
Received: 28 September 2022; Accepted: 9 February 2023
References
1. Zhang, W. J. & van Luttervelt, C. A. Toward a resilient manufacturing system. CIRP Ann. 60, 469–472 (2011).
2. Galaske, N. & Anderl, R. Disruption management for resilient processes in cyber-physical production systems. Proc. CIRP 50,
442–447 (2016).
3. Häring, I. et al. Towards a Generic resilience management, quantication and development process: General denitions, require-
ments, methods, techniques and measures, and case studies. In Resilience and Risk (eds. Linkov, I. & Palma-Oliveira, J. M.) 21–80
(Springer Netherlands, 2017). https:// doi. org/ 10. 1007/ 978- 94- 024- 1123-2_2.
4. Linkov, I. et al. Measurable resilience for actionable policy. Environ. Sci. Technol. 47, 10108–10110 (2013).
5. Clausen, J., Larsen, J., Larsen, A. & Hansen, J. Disruption management—Operations research between planning and execution.
12 (2001) http:// www. imm. dtu. dk/ pubdb/p. php? 144.
6. Moradi, B., Daclin, N. & Chapurlat, V. Formalization and evaluation of non-functional requirements: Application to resilience.
IFIP Adv. Inf. Commun. Technol. 534, 124–131 (2018).
7. Weck, O. L. D., Ross, A. M. & Rhodes, D. H. Investigating relationships and semantic sets amongst system lifecycle properties
(ilities). In 3rd International Engineering Systems Symposium, CESUN 2012 (2012).
8. Holling, C. S. Resilience and stability of ecological systems. Annu. Rev. Ecol. Syst. 4, 1–23 (1973).
9. Gu, X., Jin, X., Ni, J. & Koren, Y. Manufacturing system design for resilience. Proc. CIRP 36, 135–140 (2015).
10. Gasser, P. et al. A review on resilience assessment of energy systems. Sustain. Resilient Infrastruct. 6, 273–299 (2021).
11. Romero, D. & Stahre, J. Towards the resilient operator 5.0: e future of work in smart resilient manufacturing systems. Proc. CIRP
104, 1089–1094 (2021).
12. Yoon, J. T., Youn, B. D., Yoo, M. & Kim, Y. A newly formulated resilience measure that considers false alarms. Reliab. Eng. Syst. Saf.
167, 417–427 (2017).
13. Hu, Z. & Mahadevan, S. Resilience assessment based on time-dependent system reliability analysis. J. Mech. Des. 138, 111404
(2016).
14. Lim, S., Kim, T. & Song, J. System-reliability-based disaster resilience analysis: Framework and applications to structural systems.
Struct. Saf. 96, 102202 (2022).
15. Allenby, B. & Fink, J. Toward inherently secure and resilient societies. Science 309, 1034–1036 (2005).
16. Youn, B. D., Hu, C. & Wang, P. Resilience-driven system design of complex engineered systems. J. Mech. Des. 133, 101011 (2011).
17. Ren, F., Zhao, T., Jiao, J. & Hu, Y. Resilience optimization for complex engineered systems based on the multi-dimensional resilience
concept. IEEE Access 5, 19352–19362 (2017).
18. Hosseini, Z., Navazi, F., Siadat, A., Memari, P. & Tavakkoli-Moghaddam, R. A tailored fuzzy simulation integrated with a fuzzy
DEA method for a resilient facility layout problem: A case study of a refrigerator injection process. IFAC-Pap. 52, 541–546 (2019).
19. Song, H., Lu, X., Zhang, X., Tang, X. & Zhang, Q. A HRGO approach for resilience enhancement service composition and optimal
selection in cloud manufacturing. Math. Biosci. Eng. 17, 6838–6872 (2020).
20. Granberg, D., Pinney, D. & Eldali, F. An optimal algorithmic approach to eciently automate fault isolation and service restoration
on an arbitrary distribution feeder system. IEEE Trans. Power Deliv. 37, 3006–3015 (2022).
21. Uday, P. & Marais, K. Exploiting stand-in redundancy to improve resilience in a system-of-systems (SoS). Proc. Comput. Sci. 16,
532–541 (2013).
22. Aruväli, T. & Rauch, E. ERP and digital planning in learning factories for increasing digital resilience. In Proceedings of the 15th
EPIEM Conference 2022 vol. 29 66–71 (Verlag der Technischen Universität Graz, 2022).
23. Tong, Q., Yang, M. & Zinetullina, A. A dynamic bayesian network-based approach to resilience assessment of engineered systems.
J. Loss Prev. Process Ind. 65, 104152 (2020).
24. Mohsen, G.-A., Mahban, A. From disaster to resilience; A way forward. In 2nd International Conference on Urban Development
Based on New Technologies (2016).
25. Tukamuhabwa, B. R., Stevenson, M., Busby, J. & Zorzini, M. Supply chain resilience: Denition, review and theoretical foundations
for further study. Int. J. Prod. Res. 53, 5592–5623 (2015).
26. Tran, H. T., Balchanos, M., Domerçant, J. C. & Mavris, D. N. A framework for the quantitative assessment of performance-based
system resilience. Reliab. Eng. Syst. Saf. 158, 73–84 (2017).
27. Bruneau, M. et al. A framework to quantitatively assess and enhance the seismic resilience of communities. Earthq. Spectra 19,
(2003).
28. Malik, A. A. & Tosh, D. K. Quantitative risk modeling and analysis for large-scale cyber-physical systems. In 2020 29th International
Conference on Computer Communications and Networks (ICCCN) 1–6 (2020). https:// doi. org/ 10. 1109/ ICCCN 49398. 2020. 92096
54.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
15
Vol.:(0123456789)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
29. Hirman, M., Benesova, A., Sima, K., Steiner, F. & Tupa, J. Design, fabrication and risk assessment of iot unit for products manu-
factured in industry 4.0 factory. Proc. Manuf. 51, 1178–1183 (2020).
30. Hosseini, S., Barker, K. & Ramirez-Marquez, J. E. A review of denitions and measures of system resilience. Reliab. Eng. Syst. Saf.
145, 47–61 (2016).
31. Nikookar, H., Takala, J., Sahebi, D. & Kantola, J. A qualitative approach for assessing resiliency in supply chains. Manag. Prod. Eng.
Rev. 5(4) (2014).
32. Garcia, A. P., Siaterlis, C. & Masera, M. Testing the delity of an emulab testbed. In 2010 IEEE 30th International Conference on
Distributed Computing Systems Workshops 307–312 (2010). https:// doi. org/ 10. 1109/ ICDCSW. 2010. 74.
33. Eljaoued, W., Yahia, N. B. & Ben Saoud, N. B. A qualitative-quantitative resilience assessment approach for socio-technical systems.
Proc. Comput. Sci. 176, 2625–2634 (2020).
34. Yoo, M. et al. A resilience measure formulation that considers sensor faults. Reliab. Eng. Syst. Saf. 199, 106393 (2020).
35. Papacharalampopoulos, A., Michail, C. K. & Stavropoulos, P. Manufacturing resilience and agility through processes digital twin:
Design and testing applied in the LPBF case. Proc. CIRP 103, 164–169 (2021).
36. Latsou, C., Erkoyuncu, J. A. & Farsi, M. A multi-objective approach for resilience-based system design optimisation of complex
manufacturing systems. 100, 536–541 (2021).
37. Najarian, M. & Lim, G. J. Design and assessment methodology for system resilience metrics. Risk Anal. 39, 1885–1898 (2019).
38. Ribeiro, D., Almeida, A., Azevedo, A. & Ferreira, F. Resilience in industry 4.0 digital infrastructures and platforms. In Advances
in Manufacturing Technology XXXIV (2021).
39. Colabianchi, S., Costantino, F., Di Gravio, G., Nonino, F. & Patriarca, R. Discussing resilience in the context of cyber physical
systems. Comput. Ind. Eng. 160, 107534 (2021).
40. Xiao, Y. & Watson, M. Guidance on conducting a systematic literature review. J. Plan. Educ. Res. 39, 93–112 (2019).
41. Sandelowski, M., Barroso, J. & Voils, C. I. Using qualitative metasummary to synthesize qualitative and quantitative descriptive
ndings. Res. Nurs. Health 30, 99–111 (2007).
42. Freeman, R., McMahon, C. & Godfrey, P. An exploration of the potential for re-distributed manufacturing to contribute to a
sustainable, resilient city. Int. J. Sustain. Eng. 10, 260–271 (2017).
43. Rahman, M. et al. A framework for learning system for complex industrial processes. In AI and Learning Systems—Industrial
Applications and Future Directions 1–29 (IntechOpen, 2021). https:// doi. org/ 10. 5772/ intec hopen. 92899.
44. Goepel, K. D. Implementation of an online soware tool for the analytic hierarchy process (AHP-OS). Int. J. Anal. Hierarchy Process
10, (2018).
45. Goepel, K. D. AHP online system—AHP-OS.
46. Kibira, D., Morris, K. C. & Kumaraguru, S. Methods and tools for performance assurance of smart manufacturing systems. J. Res.
Natl. Inst. Stand. Technol. 121, 282–313 (2016).
47. Freeman, R., McMahon, C. & Godfrey, P. Design of an integrated assessment of re-distributed manufacturing for the sustainable,
resilient city. Smart Innov. Syst. Technol. 52, 601–612 (2016).
48. Peng, T., He, Q., Zhang, Z., Wang, B. & Xu, X. Industrial internet-enabled resilient manufacturing strategy in the wake of covid-19
pandemic: a conceptual framework and implementations in China. Chin. J. Mech. Eng. Engl. Ed. 34, (2021).
49. Zhang, D., Xie, M., Yan, H. & Liu, Q. Resilience dynamics modeling and control for a recongurable electronic assembly line under
spatio-temporal disruptions. J. Manuf. Syst. 60, 852–863 (2021).
50. Alexopoulos, K., Anagiannis, I., Nikolakis, N. & Chryssolouris, G. A quantitative approach to resilience in manufacturing systems.
Int. J. Prod. Res. https:// doi. org/ 10. 1080/ 00207 543. 2021. 20185 19 (2022).
51. Okorie, O. et al. Manufacturing in the time of covid-19: An assessment of barriers and enablers. IEEE Eng. Manag. Rev. 48, 167–175
(2020).
52. Diaz-Elsayed, N., Morris, K. C. & Schoop, J. Realizing environmentally conscious manufacturing in the post–COVID-19 era.
Smart Sustain. Manuf. Syst. 4, (2020).
53. Caputo, A. C., Pelagagge, P. M. & Salini, P. A methodology to estimate resilience of manufacturing plants. IFAC-Pap. 52, 808–813
(2019).
54. Li, R., Tian, X., Yu, L. & Kang, R. A systematic disturbance analysis method for resilience evaluation: A case study in material
handling systems. Sustain. Switz. 11, (2019).
55. Jin, X. & Gu, X. Option-based design for resilient manufacturing systems. IFAC-Pap. 49, 1602–1607 (2016).
Acknowledgements
is project has received funding from the Autonomous Province of Bozen/Bolzano, Department innovation,
research, universities and museums (Project Title: ASSIST4RESILIENCE: Increasing Resilience in Manufactur-
ing—Development of a Digital Twin Based Worker Assistance).
Author contributions
T.A.: methodology, investigation, literature review, comparison of resilience metrics, AHP analysis, writing—
original dra, writing—review and editing, visualization. M.D.M.: literature review, investigation, review and
editing, visualization. E.R.: conceptualization, supervision, literature search, comparison of resilience metrics,
writing—review and editing.
Competing interests
e authors declare no competing interests.
Additional information
Correspondence and requests for materials should be addressed to T.A.
Reprints and permissions information is available at www.nature.com/reprints.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and
institutional aliations.
Content courtesy of Springer Nature, terms of use apply. Rights reserved
16
Vol:.(1234567890)
Scientic Reports | (2023) 13:2914 | https://doi.org/10.1038/s41598-023-29735-1
www.nature.com/scientificreports/
Open Access is article is licensed under a Creative Commons Attribution 4.0 International
License, which permits use, sharing, adaptation, distribution and reproduction in any medium or
format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the
Creative Commons licence, and indicate if changes were made. e images or other third party material in this
article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the
material. If material is not included in the article’s Creative Commons licence and your intended use is not
permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from
the copyright holder. To view a copy of this licence, visit http:// creat iveco mmons. org/ licen ses/ by/4. 0/.
© e Author(s) 2023
Content courtesy of Springer Nature, terms of use apply. Rights reserved
1.
2.
3.
4.
5.
6.
Terms and Conditions
Springer Nature journal content, brought to you courtesy of Springer Nature Customer Service Center GmbH (“Springer Nature”).
Springer Nature supports a reasonable amount of sharing of research papers by authors, subscribers and authorised users (“Users”), for small-
scale personal, non-commercial use provided that all copyright, trade and service marks and other proprietary notices are maintained. By
accessing, sharing, receiving or otherwise using the Springer Nature journal content you agree to these terms of use (“Terms”). For these
purposes, Springer Nature considers academic use (by researchers and students) to be non-commercial.
These Terms are supplementary and will apply in addition to any applicable website terms and conditions, a relevant site licence or a personal
subscription. These Terms will prevail over any conflict or ambiguity with regards to the relevant terms, a site licence or a personal subscription
(to the extent of the conflict or ambiguity only). For Creative Commons-licensed articles, the terms of the Creative Commons license used will
apply.
We collect and use personal data to provide access to the Springer Nature journal content. We may also use these personal data internally within
ResearchGate and Springer Nature and as agreed share it, in an anonymised way, for purposes of tracking, analysis and reporting. We will not
otherwise disclose your personal data outside the ResearchGate or the Springer Nature group of companies unless we have your permission as
detailed in the Privacy Policy.
While Users may use the Springer Nature journal content for small scale, personal non-commercial use, it is important to note that Users may
not:
use such content for the purpose of providing other users with access on a regular or large scale basis or as a means to circumvent access
control;
use such content where to do so would be considered a criminal or statutory offence in any jurisdiction, or gives rise to civil liability, or is
otherwise unlawful;
falsely or misleadingly imply or suggest endorsement, approval , sponsorship, or association unless explicitly agreed to by Springer Nature in
writing;
use bots or other automated methods to access the content or redirect messages
override any security feature or exclusionary protocol; or
share the content in order to create substitute for Springer Nature products or services or a systematic database of Springer Nature journal
content.
In line with the restriction against commercial use, Springer Nature does not permit the creation of a product or service that creates revenue,
royalties, rent or income from our content or its inclusion as part of a paid for service or for other commercial gain. Springer Nature journal
content cannot be used for inter-library loans and librarians may not upload Springer Nature journal content on a large scale into their, or any
other, institutional repository.
These terms of use are reviewed regularly and may be amended at any time. Springer Nature is not obligated to publish any information or
content on this website and may remove it or features or functionality at our sole discretion, at any time with or without notice. Springer Nature
may revoke this licence to you at any time and remove access to any copies of the Springer Nature journal content which have been saved.
To the fullest extent permitted by law, Springer Nature makes no warranties, representations or guarantees to Users, either express or implied
with respect to the Springer nature journal content and all parties disclaim and waive any implied warranties or warranties imposed by law,
including merchantability or fitness for any particular purpose.
Please note that these rights do not automatically extend to content, data or other material published by Springer Nature that may be licensed
from third parties.
If you would like to use or distribute our Springer Nature journal content to a wider audience or on a regular basis or in any other manner not
expressly permitted by these Terms, please contact Springer Nature at
onlineservice@springernature.com
Content uploaded by Tanel Aruväli
Author content
All content in this area was uploaded by Tanel Aruväli on Feb 23, 2023
Content may be subject to copyright.
