![Peterson's Simulation Model architecture. Components not used in this analysis are indicated in gray. Adapted with permission from [11]. Copyright 2021, copyright Peterson, Van Bossuyt, Giachetti, and Oriti.](https://www.researchgate.net/profile/Douglas-Van-Bossuyt/publication/366001011/figure/fig3/AS:11431281105212096@1670341447749/Petersons-Simulation-Model-architecture-Components-not-used-in-this-analysis-are_Q320.jpg)
Content uploaded by Douglas Lee Van Bossuyt
Author content
All content in this area was uploaded by Douglas Lee Van Bossuyt on Dec 06, 2022
Content may be subject to copyright.
Citation: He, J.J.; Van Bossuyt, D.L.;
Pollman, A. Experimental Validation
of Systems Engineering Resilience
Models for Islanded Microgrids.
Systems 2022,10, 245. https://
doi.org/10.3390/systems10060245
Academic Editor: William T. Scherer
Received: 15 October 2022
Accepted: 1 December 2022
Published: 6 December 2022
Publisher’s Note: MDPI stays neutral
with regard to jurisdictional claims in
published maps and institutional affil-
iations.
Copyright: © 2022 by the authors.
Licensee MDPI, Basel, Switzerland.
This article is an open access article
distributed under the terms and
conditions of the Creative Commons
Attribution (CC BY) license (https://
creativecommons.org/licenses/by/
4.0/).
systems
Article
Experimental Validation of Systems Engineering Resilience
Models for Islanded Microgrids
Justin J. He, Douglas L. Van Bossuyt * and Anthony Pollman
Department of Systems Engineering, Naval Postgraduate School, Monterey, CA 93943, USA
*Correspondence: douglas.vanbossuyt@nps.edu
Abstract:
Microgrids are used in many applications to power critical loads that have significant
consequences if they lose power. Losing power to medical centers, water treatment plants, data
centers, national defense installations, airports, and other critical infrastructure can cause loss of
money and loss of life. Although such microgrids are generally reliable at providing stable power,
their resilience to disruption can be poor. Common interruptions include natural disasters like
earthquakes, and man-made causes such as cyber or physical attacks. Previous research into mi-
crogrid resilience evaluation efforts centered on theoretical modeling of total electrical microgrid
loading, critical electrical load prioritization, assumed capacity of renewable energy sources and their
associated energy storage systems, and assumed availability of emergency generators. This research
assesses the validity of two microgrid resilience models developed for analyzing islanded microgrids
by using experimental data from a scaled microgrid system. A national defense context is provided
to motivate the work and align with the intended purpose two microgrid resilience models. The
results of this research validate that the simulation models are valid to use in some situations, and
highlight some areas for further model improvement.
Keywords: microgrid; resilience; systems analysis; validation; national defense
1. Introduction
Civilian infrastructure and national defense installations require resilient and reliable
electrical power to ensure operations, and meet operational and mission requirements.
While the power grid is cost effective in providing power for hospitals, ports, food storage
facilities, water treatment and sanitation plants, national defense organizations such as the
United States Department of Defense (DOD), and other critical infrastructure, the power
grid is susceptible to disruptions which may impact operations. Weather events are a major
contributor to power disruptions in the United States [
1
] and were found to cost billions an-
nually [
2
]. Within the realm of national defense, disruptions risk exposing operational and
strategic capabilities to interruption which are challenging to attach a monetary value to [
3
].
Recent extreme weather events have also exposed critical vulnerabilities in centralized
grid power generation to lengthy disruptions where users were left without power due
to extreme cold weather and wildfire risk. Examples include the Texas blackouts caused
by gas power plant shutdowns in the extremely cold winter storms in February 2021 [
4
]
and the California ’Public Safety Power Shut-Offs’ during periods of severe wildfire risk in
2019 [5].
There is sustained interest by many critical infrastructure sectors including the DOD
and other national defense organizations to improve energy resilience [
6
]. Organizations
such as the DOD also aim to consume more energy from renewable sources to substitute
current strategies of using diesel generators for backup power. Microgrids utilizing renew-
able energy sources are an emerging system of interest to improve resilience. In the event
of power interruption, the microgrid is able to continue to provide power in islanded mode
Systems 2022,10, 245. https://doi.org/10.3390/systems10060245 https://www.mdpi.com/journal/systems
Systems 2022,10, 245 2 of 24
operations using distributed energy resources [
7
] and without the need of diesel fuel resup-
ply [
8
]. As such, there has been sustained interest in the analysis of critical infrastructure
microgrids, including national defense microgrids, in order to improve resilience from a
systems engineering perspective. Although there are several commercial tools available for
microgrid design optimization, the main focus of the tools is to optimize the microgrid for
cost [
9
,
10
]. These tools do not fully address resilience analysis in the context of national
defense and other critical infrastructure settings where monetary damages due to an outage
are difficult or impossible to define. Recent research into systems engineering measures of
microgrid resilience for critical infrastructure and national defense installations has resulted
in the development of simulation models to conduct resilience analysis [
11
,
12
]. However,
these models have not been validated against real-world hardware data. To validate that the
models closely match reality and build trust in the model results, experimental validation
can be useful.
This article uses a scaled experimental microgrid setup to validate some of the simula-
tion models used for analyzing microgrid resilience in critical infrastructure applications
where monetary damages cannot be assigned to outages such as with national defense. Two
microgrid resilience simulation models are discussed and the scaled experimental micro-
grid parameters are used as simulation inputs. The results of the models are then compared
with data obtained from the experimental setup. A close result after comparison of the
experimental model and the simulation model is used to validate the simulation models.
2. Background and Related Research
Many critical infrastructure sectors have facilities that contain advanced systems that
require electrical and computing power to support complex system functions. The impor-
tance of electricity cannot be overemphasized as the lack of electrical power could disable
all modern communications, cause hospitals to be unable to treat complex cases, stop water
treatment and wastewater plants, spoil refrigerated and frozen food in warehouses, and
disable national defense installations that are involved in radar defenses, sever lines of com-
munication required for command and control of defense systems, and cause many other
issues. The requirement for resilient energy for national defense is emphasized in Title 10,
section 101 in the United States Code, which outlines the role of the armed forces defines en-
ergy resilience as “.. . the ability to avoid, prepare for, minimize, adapt to, and recover from
anticipated and unanticipated energy disruptions in order to ensure energy availability
and reliability sufficient to provide for mission assurance and readiness, including mission
essential operations related to readiness, and to execute or rapidly reestablish mission
essential requirements” [
13
]. Key points in the quote include the ability to minimize energy
disruption and to recover from it. Microgrids can support energy resilience requirements
for critical infrastructure including national defense. The remainder of this section provides
background and discussion on microgrid architecture and microgrid resilience measures in
a systems engineering context.
2.1. Microgrids
Electric utilities, regardless of ownership type (public or private), have been found to
have similar performance and only differ slightly in pricing structure and reliability [
14
].
Regardless of utility ownership type, utilities must provide a reliable supply of power
to customers at a reasonable cost to be viable. This balance between reliability and cost
compels utilities to build infrastructure to operate under typical historical conditions. As
such, occasional interruptions occur during abnormal weather which may inconvenience
civilian customers and lead to some monetary loss to corporate customers [
15
]. Power
interruptions at critical infrastructure facilities and installations including national defense
installations may impact national security and the consequences may be more severe,
and can be hard to quantify monetarily. National security requirements and the increas-
ing number of abnormal weather events have elevated interest in improving electrical
Systems 2022,10, 245 3 of 24
resilience though the deployment of microgrids. Recent publications have also asserted
that microgrids are successful and practical in improving resilience [16–18].
The U.S. Department of Energy has adopted the widely cited definition of a microgrid
which was developed by the Microgrid Exchange Group. It defines a microgrid as “a
group of interconnected loads and distributed energy resources within clearly defined
electrical boundaries that acts as a single controllable entity with respect to the grid. A
microgrid can connect and disconnect from the grid to enable it to operate in both grid-
connected or island-mode” [
19
]. Crucially, the microgrid acts as a single controllable entity
that can function regardless of whether it is connected to another grid. It requires the
microgrid to be distinctly identifiable from the grid, locally interconnected and controlled,
and lastly, functionally independent [
20
]. For the microgrid to be distinctly identifiable,
it must have clear physical and functional boundaries. This is defined by the hardware
and functional components that form the external interface for coupling to the utility grid,
also known as the Point of Common Coupling (PCC) [
21
]. The microgrid controller and its
local interconnection fulfill the requirement of being locally interconnected and controlled
to balance power availability with load demands [
22
]. Finally, the requirement of being
independent is fulfilled by the microgrid’s capacity to sufficiently cater to load demands
within its boundaries. Therefore, to meet the functional requirements of a microgrid, the
basic components include (1) Distribution System, (2) Distributed Generation Resources,
(3) Energy Storage Systems, and (4) a Control and Communication System [
23
]. A basic
microgrid system is presented in Figure 1.
Figure 1. Components of a basic microgrid architecture.
2.1.1. Microgrid Distribution System
Microgrids typically utilize AC and/or DC distribution systems that match with the
connected loads and generation sources. DC distribution systems have some advantages in
certain cases by providing lower losses and higher transmittable power, while AC distri-
bution has been a standard for a century and is widely used. DC distribution advantages
include that DC power need not be converted to AC for Distributed Generation sources
such as photovoltaic (PV) to the energy storage systems (ESS) such as batteries. There-
fore, research work on standardizing DC distribution and control is ongoing and, in the
meantime, hybrid AC/DC microgrids are sometimes used [24].
Systems 2022,10, 245 4 of 24
2.1.2. Distributed Generation Resources
There are a range of technologies that are available for microgrid power generation.
Widely used systems includes diesel generators, solar PV systems, wind turbines, and micro-
hydro [
20
]. Diesel generators consume fuel to generate electrical power; however, they
are not a renewable energy source and fuel resupply is required for continued operations
which can leave them vulnerable to disruption in certain scenarios [
8
]. Solar PV generates
DC electricity from solar energy. The performance of the PV system is dependent on the
system’s location which determines solar intensity and cloud cover. Solar PV is also not able
to generate electricity after nightfall and its performance is degraded in winter months [
25
].
Wind turbines harness kinetic energy from wind with rotor blades and transform it into
electric energy though a generator. Similar to the Solar PV system, their performance is
also location dependent and can only generate electricity when the weather allows. The
last common power generation technology is micro-hydro, which generates electricity from
the flow of water and is dependent on topography and rainfall of the area.
2.1.3. Energy Storage Systems
The rapid reduction in cost of ESS and its central role in many microgrids has driven
the development and the successful operation of microgrids with a significant percentage
of generation (or all generation) being handled by renewable energy. ESS allows for the
balancing of power and energy demands while providing uninterrupted transition from
utility supply to the microgrid supply. Further, ESS can store renewable energy that is
intermittently generated for later use. Essentially, the primary functions of an ESS allow
for some of the following within the microgrid:
•
Handle load fluctuations and power transients, and provide some time for generation
sources to respond to the fluctuations;
• Ensure power supply stability when the power source is unstable;
• Handle microgrid transition from utility connected to islanded operations.
There can be a slight change in the system AC frequency when loads are added or
removed from the system. This must be handled by sufficient ESS capacity to ensure
the microgrid with several power generation sources is able to balance energy demand
with generation following system loading adjustments. Common ESS for microgrids
that are practical include batteries, fuel cells, flywheels, and super-capacitors. Batteries
are the most common microgrid storage solution as they are the most affordable type of
system. The most common type of battery deployed for microgrids are lead-acid batteries
as they can support high currents in a very short period to handle power transients [
26
]
during microgrid decoupling and they are capable of saving reserve energy for future
demands although lithium chemistries are being increasingly used in some microgrids. The
next type of ESS, fuel cells, is rapidly rising in popularity [
27
] as the technology matures.
Fuel cells provide high efficiency by directly converting chemical energy from a fuel into
electricity through a chemical reaction. This improves the practical performance of the
microgrid [
28
] by reducing cost, improving energy efficiency, and microgrid reliability.
When a fuel cell is implemented with an electrolyser, it can supplement batteries for
energy storage as it has a high specific energy that can be used to soak up spare energy
generated on the microgrid [
29
]. The last two types of ESS—flywheel and super-capacitor—
are usually employed to improve power quality and as uninterruptible power supply for
small loads [30].
2.1.4. Control and Communication Systems
IEEE Std. 2030.7-20.7 [
31
] specifies the general functional requirements for microgrid
control to allow for standardization. The core functions described in the standard are the
“dispatch” and “transition” functions for the microgrid. The microgrid controller dispatch
function ensures balancing power generation and load when the system is in islanded mode,
rebalancing of generation and load when there are changes in profiles, and responding to
Systems 2022,10, 245 5 of 24
external control orders to meet interconnection agreement requirements. The microgrid
controller transition function enables the system to transit between grid-connected mode
and islanded mode without delay or power supply disruption to connected loads.
To achieve dispatch control, a unified rule-based control strategy with separate rules
for grid-connected mode and islanded mode operations can be adopted [
32
]. In grid-
connected mode, the reduction of power variation at the point of interconnection to the grid
is prioritized to meet interconnection agreement requirements. For dispatch in islanded
mode, the control objectives change to maintaining power balance while ensuring the
microgrid components are operating within limits defined by the predefined rules. This
ensures power balance, safe, and efficient operations of the microgrid. Transition between
grid connected and islanded mode is initiated by one of the following three processes with
steps described:
•
Planned Islanding—the microgrid controller receives the islanding command, pro-
ceeds to balance load and generation, configure local controllers, disconnect point of
interconnection, and achieve steady state islanded power dispatch on the microgrid;
•
Unplanned Islanding—the microgrid detects islanded conditions, disconnect point of
interconnection to create an island, configure local controllers, execute pre-configured
control commands such as load shedding and achieves steady state islanded power
dispatch on the microgrid;
•
Reconnection to the grid—the microgrid controller synchronizes to the grid power,
configures local controllers, reconnects the point of interconnection, and achieves
steady state grid-connected dispatch mode.
Microgrid controllers rely on a robust communication system to enable core control
functions. Centralized communication architectures were initially developed for microgrid
controls, implementation was straightforward, and it met the microgrid requirements [
33
].
2.2. Microgrid Resilience
Natural disasters which include flooding, earthquakes, and hurricanes among many
others [
34
,
35
] can cause severe power disruptions. Although such events are rare, the
disruptions are severe and continue to cause economic loss after the disaster. Power system
resilience has been studied and defined to improve the design of power systems to be
able to withstand external shock or damage events, and to recover quickly [
36
]. A recent
review of microgrid resilience found that accurate and realistic simulations are needed to
design microgrids with better resilience. The examination of more realistic and general
simulation frameworks would enable accurate comparison of different microgrid design
and employment strategies [37].
There are various performance measures for microgrids such as those proposed by
Lu, Wang, Zhang and Cheng which include reliability, economic (cost), practicality, and
environmental sustainability as performance indices [
38
]. The reliability index measures
the ratio of total unmet load to total electric load demand. The economic index measures the
system cost effectiveness by computing the ratio of annualized cost for power generation
to the total electrical load demand. Next, the practicality index computes the ratio of the
total microgrid system occupied area to the available area for the system. Finally, the
environmental sustainability index is computed from percentage of load demands met
by renewable sources. This may be a comprehensive matrix for microgrid performance
which incorporates design architecture elements like space practicality and environmental
sustainability. Most microgrid assessments optimize system performance by maximizing
reliability to meet an objective reliability value and minimizing system cost. This is therefore
achieved by computing a reliability–cost objective function with reliability as a constraint,
or with a predetermined investment amount [
39
,
40
]. Such methods focus on the reduction
of operational cost for historical normal loads and do not focus on microgrid resilience [
41
].
Systems 2022,10, 245 6 of 24
2.2.1. Energy Resilience Definitions
Energy resilience must first be understood and measured before microgrid resilience
can be analyzed. Various studies have reviewed quantitative measures of energy resilience
from different perspectives that include design, identified threats to the system, and from
different time periods. These measures typically use the resilience curve shown in Figure 2
where the disruption impacts the microgrid at time t
d
[
42
]. Ideally, a resilient microgrid
system would either have no or a small drop in performance at time t
d
at the onset of
the disruption event. The system would then need to maintain a stabilized supply before
it could recover and this period is the recovery time. The invulnerability (the drop in
performance at t
d
) and recovery time form the key measures of the microgrid resilience.
These measures are also depicted in Figure 2.
Figure 2.
Typical resilience curve showing the phases of disruption, and the key measures of
invulnerability and recovery time..
A common measure for resilience uses the ratio of the area under normal operational
performance and the actual degraded performance after the disruption [
43
]. Other re-
searchers include the definition of the components of resilience like absorption capacity or
invulnerability, and the time taken to recover from the disruption [42,44].
Resilience is also threat-dependent [
45
] and the threats can be widely categorized as
intentional attacks, including physical and cyber-attacks, and low-probability high-impact
events such as extreme weather. As such, some researchers have introduced operational
resilience and infrastructural resilience to analyze different operational and infrastructural
resilience strategies [
36
]. This also highlights the difference between the reliability and
resilience measures for the microgrid although both terms have frequently been used
interchangeably [46].
Clark-Ginsberg uses a cyber incident to illustrate the difference between power relia-
bility and resilience. When the cyber incident disrupts power supply, the system reliability
is impacted as reliability is generally measured by the power supply’s ability to meet load
demands. Resilience then measures how much the cyber incident disrupts (invulnerability
measure) and how quickly the system is able to recover (recovery time). He then argues that
by implementing rolling blackouts, the system maintains a high resilience while reliability
continues to degrade. However, this views resilience from a single-load perspective as the
recovery, and in this instance, when the lights go back on the system is considered to be
‘recovered’ prior to the next blackout in a rolling blackout scenario. However, should the
resilience measure account for the grid system supply, rolling blackouts are considered as
part of the stabilization and restoration phase [
47
]. To summarize, reliability provides a
quantified metric for a system to function as specified and does not assess the system’s
Systems 2022,10, 245 7 of 24
degraded functionality or ability to recover from failure or disruption. Resilience provides
a measure of functionality of the system when degraded from a disruption event and the
ability of the system to return to a state that is able to meet the functional requirements of
the system [46].
2.2.2. National Defense Energy Resilience
The deployment of microgrids in national defense installations can improve electrical
power security, reduce energy costs by incorporating renewable energy generation, meet
national defense mission objectives, and provide supply in remote installations [
48
]. There-
fore, national defense requirements differ from civilian microgrid application requirements
(including many other critical infrastructure sectors) which typically only define the value
of resilience in financial terms and only account for low-probability high-impact events. The
value of national defence microgrid resilience is national defense and this can make it an
attractive target for intentional attacks [
49
]. This widens the range of threats to the national
defense installation energy security and examples include grid disruptions, component
failure, damage due to disaster, and intentional physical or cyber attacks. Since national
defense microgrids are unable to use just monetary value to measure microgrid resilience,
there is no standard to define the value of energy resilience within the DOD [
41
]. Some
examples of methods used include attempts to quantify resilience for national defense
microgrids using the cost of implemented generators, the cost to relocate the mission for
the duration of the disruption, or developing a method to compute a damage function
based on the disruption duration [
50
,
51
]. Although there are differences in the computation
for the value of energy resilience, researchers agree that the implementation of microgrids
with distributed energy resources improves energy resilience [52,53].
2.2.3. Resilience Improvement
Mahzarnia, Moghaddam, Baboli, and Saino conducted a study to review measures
to enhance power system resilience and found that there was a lack of comprehensive
studies that considered power system resilience holistically. Although the impact of the
topology and employment of renewable systems on resilience needs to be better understood
for power distribution, they found that the investment of distributed energy resources,
development of smart grid technologies, and the employment of microgrids to be useful [
54
].
For distributed energy resources, a study for distributed energy storage systems has found
considerable improvements to power resilience [
55
], while smart grid technologies for fault
isolation and service restoration provide quick analysis and decision support [
56
] that also
enhances resilience. Microgrids have been shown to be invaluable in servicing connected
critical loads in Tokyo after major disruption caused by a tsunami in 2011 and are able to
support power restoration, network formation strategies, and power disruption prevention
measures [16] .
2.2.4. Resilience Assessment Methodologies
There is keen research interest in national defence microgrids to enhance power
resilience and the diverse studies have spanned from cost trade space, to assessing the
impact of power resilience on mission operations and cyber security [
11
,
12
,
57
,
58
]. To
conduct an assessment or analysis, simulation models are a common feature of most
studies. This article focuses on two methods gaining traction within the US Navy’s Naval
Facilities Command (NAVFAC) and elsewhere that are used for analysis of national defense
mission resilience impact, and the cost trade space for microgrids on islands, and they are
discussed next.
Peterson’s model developed for the analysis of microgrid resilience and demonstrated
on a national defense microgrid computes an electrical disruption mission impact metric
for a microgrid model by determining the power flow within the microgrid. The mission
impact metric accounts for periods when the load demands are not met and load shedding
occurs. This model uses reference building load models from the Department of Energy and
Systems 2022,10, 245 8 of 24
solar radiation data from the National Renewable Energy Laboratory together with user
input of the microgrid design parameters such as energy storage and generation capacity
to simulate the impact to mission when a disruption occurs [11].
Anderson’s model computes islanded microgrid resilience using invulnerability and
recoverability metrics [
12
]. A case study of a national defense microgrid is provided in
Anderson’s work. Invulnerability is computed from the ratio of power delivered and load
demand, and recoverability is the computed ratio of power demand that is not met after
the disruption event. The model varies the power generation from three power sources
(diesel, solar, and wind) together with the energy storage system capacity to generate the
metrics. Further details of both models are provided in subsequent sections.
2.3. Validation Process
The system engineering process comprehensively describes the key activities for a
structured development process to realize a successful system. It is an interdisciplinary
approach to integrating various disciplines and specializations that is initiated from system
conception to operation. The system engineering V-Model which was developed in the
1970s has roots in software engineering and some early researchers used it as a tool to
emphasize the importance of verification and validation [
59
]. The right side of the V-model
depicts the integration phase of the system and consists of various verification and vali-
dation processes alongside the integration processes. The INCOSE Systems Engineering
handbook describes both the verification and validation processes which have some simi-
larities. Although both processes aim to provide objective evidence that the system meets
requirements, the aims and scope differ. The verification process seeks to produce evidence
that the system meets system and technical requirements and is generally at the lower
right side of the V-model. This phase consists of the development phase where the systems
and sub-systems are tested for acceptance. The validation process then tests the system
in its operational environment after the system has been verified to meet system require-
ments. This validation process assesses if the system is suitable and meets operational
needs. In this paper, the focus is on initial experimental validation of systems engineering
resilience models.
3. Methodology and Simulation
This section introduces the systems engineering process and focus on the verification
and validation method used to validate Peterson’s and Anderson’s microgrid resilience
models. The systems engineering method for verification and validation is presented
together with the strategy, identified inputs, activities and outputs [
60
]. In short, the
verification and validation method used in this article follows these steps: (1) The simulation
models are first adapted to a scaled down experimental microgrid for direct comparison.
(2) Next, the results from the experimental microgrid are collected. (3) Then, the simulation
models are run using the data such as solar irradiance from the experimental microgrid.
(4) Finally, the experimental microgrid data and the simulation results are compared. The
simulation and experiment are set up to illustrate a 72 h power disruption to allow for
the microgrid to operate in islanded mode. This validation effort focuses on validation
of electrical energy resilience of the simulation models and does not assess additional
functions such as mission impact or cost effectiveness.
3.1. Peterson’s Microgrid Resilience Simulation Model
Peterson’s simulation model was developed to quantify microgrid resilience and
investigate the impact to national defense installation missions [
11
]. It uses an expected
electrical disruption mission impact metric to analyze mission impact (
MI
) and does not
account for peripheral issues such as power factor and phase imbalance. This allows for
the model to explore the high-level engineering trade space between power resilience and
mission impact.
Systems 2022,10, 245 9 of 24
Peterson suggests using the Mission Dependency Index (MDI) [
61
,
62
] to determine
the severity of impact to of a specific load suffering a power failure. MDI is one way to
prioritize loads on a 0–100 scale with 100 being the highest priority to maintain power.
However, there are criticisms of using MDI for such situations [63].
For the simulation to mimic real scenarios as closely as possible, hourly historical
models for solar energy from the National Solar Radiation Data Base [
64
] and facility load
demands from the US Department of Energy were used as model inputs [
65
]. The other
user-defined input variables include the mission impact assessment and the ratio of critical
loads of each facility modeled, a set of power interruption scenarios and the assessed
recovery time. The simulation could then be initiated to produce mission impact results for
a single run or utilize a series of Monte Carlo simulations to compute a mean for a more
representative general result. Peterson calculates the mission impact for a single scenario
(s) as the M I per unit time (T) of the entire duration of the scenario,
Ms=
T
∑
t=1
MIs,t(1)
Peterson further proposes that the total impact of disruption events over all considered
failure scenarios (the expected electrical disruption mission impact (
EEDM I
)) should be
calculated as
EEDM I =∑
s∈S
Pr(S=s)Ms(2)
which quantifies the resilience of the system for the analyzed microgrid disruptions.
If MDI is used for
MI
,
EEDM I
is unitless. However, a practitioner may choose
a different metric fro
MI
which can add units.
EEDM I
is meant to compare between
different microgrid architectures when identifying the most suitable candidate microgrid
architecture for a collection of loads that best serves the loads. For the purposes of the
analysis conducted in this paper, the authors have reduced Peterson’s calculations to focus
only on one load and have thus reduced the
MI
component to an assumed impact based on
the total load not served during a disruption. This simplification is appropriate to validate
Peterson’s power balance model and other model aspects.
The baseline microgrid system design used in the model consists of (1) Utility Grid
Connection, (2) Diesel Generators, (3) Photovoltaic Solar Arrays, (4) Energy Storage Systems,
and (5) Multiple Facility loads [
11
]. To compute mission impact, the model then categorizes
load priority based on assessed criticality and adds a mission impact figure if the microgrid
was unable to meet critical loads for the simulation. The summation of the mission impact
measure will then be the result of one simulation. It was assessed that conducting an
experiment to generate results for computing mission impact was not feasible as part of
this article because failure distributions for distribution line components and generating
systems were used in the Monte Carlo simulations. The experiment in this article instead
focuses on the validation of power flow and system battery charge results which is used by
the simulation to compute mission impact. The power flow and battery charge status graphs
will be compared with experimental result graphs for an assessment. This baseline model is
shown in Figure 3, where loads are indicated as EP with the respective power sources shown.
Peterson’s baseline model was adjusted to match the scaled down experimental set up
shown in Figure 3and was used to produce data to compare against the experiment system.
To facilitate a direct comparison, switches to loads and source components were not used
in the experiment model and were set open, a constant AC load was used, and the power
generation and storage system ratings were changed to match the experimental microgrid.
A simple power disruption scenario with indefinite utility grid power loss and with a
20 h loss of the diesel generator was simulated. This produced the simulated baseline result
graph for the model that mimics the experiment microgrid shown in Figure 4. The result is
consistent with expected power delivery behavior as the load has to draw from the battery
once the microgrid is islanded. In the initial 20 h with no diesel generator, the battery charge
Systems 2022,10, 245 10 of 24
is consumed until there is sufficient solar power to meet power demands. After the diesel
generator is recovered, demand is supplied by the diesel generator and does not further
drain battery charge. When solar energy is available in the day, the photovoltaic solar
system supplies power to the load and negative value power indicates power flow to the
batteries for charging. However, it is noted that in this model, although there was surplus
power from the 2 kW diesel generator after hour 20, the battery charge only increased when
there was solar power in excess of the load of 0.7 kW. In other words, the model does not
allow for ESS charging from the diesel generator. This becomes important in a subsequent
section with experiments conducted on the physical microgrid.
Figure 3.
Peterson’s Simulation Model architecture. Components not used in this analysis are
indicated in gray. Adapted with permission from [
11
]. Copyright 2021, copyright Peterson, Van
Bossuyt, Giachetti, and Oriti.
Figure 4.
Peterson’s Simulation Model adjusted architecture results for battery state of charge and
power flow.
Systems 2022,10, 245 11 of 24
3.2. Anderson’s Microgrid Resilience Simulation Model
Anderson’s microgrid resilience simulation model was developed to model resilience
and system cost assessment to provide a resilience and cost trade space for high-level
decision making [
12
]. The simulation model computes microgrid invulnerability and
recoverability metrics to determine the resilience measure, and uses a cost model to estimate
the cost of the modeled microgrid architecture. The model for resilience is stochastic
and includes distributions for probability of damage to the microgrid components and
resources. The recovery duration is also stochastic and based on the available repair
resources generated in the simulation [66].
This paper evaluates Anderson’s simulation resilience model for verification. The
model includes an equal weight for both invulnerability and recovery measures. The
measures are computed as positive ratios shown in Equation (3). The simulated micro-
grid resilience curve function is shown in Figure 5and it includes annotations for the
invulnerability and recovery measures.
resilience = 0.5 ( invulnerability + recovery) (3)
It should be noted that Anderson assigns equal weight to invulnerability and recovery.
It is possible that a practitioner may wish to adjust this relationship to develop a resilience
statistic that is more appropriate to their microgrid situation and priorities. Giving more
weight to invulnerability may, for instance, drive decision-makers to fund microgrid up-
grade projects that specifically address the depth of power loss during a power disruption.
However, this may come at the cost of an extended recovery timeline.
Figure 5.
Anderson’s Simulation Model resilience curve function for a power disruption with
annotations for invulnerability and recovery measures.
The invulnerability measure is computed with the reduction in power delivered
immediately after the disruption. The invulnerability measure is constructed by the ratio of
power delivered (
Pts
) to load demands (
Ptd
) shown in Equation (4). This is also described in
various research work by Francis and Bekra as absorptive capacity and by Yodo and Wang
as lost performance [42,67].
invulnerability =Pts
Ptd
(4)
It is important to note that in Anderson’s model, power generated is not the total rated
power available from all operating generation sources. Many microgrids that the authors
have worked with are over-provisioned with generation capacity. For instance, at one mi-
crogrid the authors have investigated, more than five times the needed generation capacity
is available and kept in reserve [
12
]. Thus, instead of potentially having invulnerability
>
1,
Anderson recommends using actual power generated which corresponds to Pts<=Ptd.
Recoverability is defined as the ratio of the area bounded by the demand and the
reduced post disruption power delivered shown in Equation (5). This method provides an
accurate indication of the microgrid’s ability to rapidly recover as it accounts for time after
the disruption until it is recovered.
Systems 2022,10, 245 12 of 24
recoverability =1−
∑t=tr
t=tdDt−Gt
∑t=tr
t=tdDt
(5)
Note that recoverability is only analyzed when
Gt<Dt
. Thus, recoverability
∈[
0, 1
]
where a microgrid that never recovers after a disturbance is represented by 0 and a disturbance
that never disrupts power delivery to all microgrid loads is represented by 1. In the first case,
this could represent when a microgrid on a remote island is completely destroyed and the
decision to abandon the island is made rather than attempt to repair the microgrid and other
damaged or destroyed critical infrastructure. In the second case, this could represent a microgrid
that is built to successfully ride through a disruptive event such as a thunderstorm without
dropping any attached loads.
The simulation resilience model requires 12 input variables which include energy gen-
eration resource parameters, energy storage capacity, probability of damage, and demand
profile. Random variables like damage to the generation resource and component mean
time to repair are generated by the simulation for use in the resilience model. The model
then computes the resilience metrics of recovery, invulnerability, and time to recover. The
simulation model includes a case study of a simplified national defense microgrid resem-
bling a microgrid at a facility at Naval Station Rota, Spain, which has a diesel generator,
a solar array, and an energy storage system to demonstrate the simulation model. This
demonstration used historical demand data and used the model to generate a simulation
of the loss of both solar and diesel generators in a tsunami event. Results of the system
demand, system power rating, and power delivered are shown in Figure 6. As both the
diesel generator and the solar generators have been damaged by the disruption, power can
only be supplied by the energy storage system for the initial 24 h. Thereafter, the recovery
of the solar and diesel generators allow the microgrid to meet demands at the time step of
65 h.
Figure 6.
Anderson’s Simulation Model resilience curve showing simulation of diesel generator and
solar power disruption. .
The baseline model inputs were then adjusted to match the scaled microgrid experi-
ment and allowed to generate results with probabilistic random variable input for system
damage and mean time to repair. This simulation model generates a 14-day scenario for
the islanded microgrid and a sample result of the resilience curve is shown in Figure 6. The
diesel generator in this scenario was damaged and was repaired at time 175 h. The battery
was able to sustain the load until hour 12 and was fully depleted at hour 19 while the solar
panels was only able to support part of the load requirements during the day. This solar
power cycling between day and night can be seen between hour 24 to 175 until the diesel
generator was recovered.
Systems 2022,10, 245 13 of 24
4. Scaled Microgrid Experimental System
The validation experiment was done on a scaled microgrid system with commercial
off the shelf systems consisting of an integrated controller, inverter, and charger system; a
photovoltaic array; a bank of batteries; and a generator. The system monitors the microgrid
power and generates an hourly log of the respective component power generation and
demands. This data log is then used for the experiment and to verify against the simulated
model data. Details of the main components of the system include:
•
2 kW Integrated controller, inverter and charger system (FXR2524A, from OutBack
Power, Bellingham, WA, USA);
•
1.2 kW Static array of twelve 100 watt solar panels are mounted on the roof with no
obstruction to the light from the sun. (RNG-100D-R-BK, from Renology, Ontario, CA,
UAS);
• 12 kWh Battery bank (SLR500-2, from GS Battery (USA) from Kyoto, Japan);
• 2 kW Gasoline generator (Hybrid Series H03651, from Firman, Peoria, AZ, USA).
The integrated controller, inverter, and charger are central to the system with solar,
grid, and generator power generation sources. The battery is also connected and can
be charged by any available source, and it can supply power when there is a disruption
causing power generated to fall below the load demands. The experimental system setup
diagram with power flow direction represented by arrows is shown in Figure 7below.
Figure 7.
Scaled Experimental Microgrid Diagram showing connected power generation and demand
components.
To mimic power disruption on the experimental microgrid, the system is configured
to consume battery power when grid power and solar power is insufficient to support load
power demands. The system will then attempt to utilize the generator to charge the battery
when its state of charge falls below a set threshold. It is important to note that microgrids
optimized for different outcomes such as reducing greenhouse gas emissions or providing
maximum resilience may use diesel generators differently with respect to battery charging.
In the case of optimizing for greenhouse gas emission reduction, the diesel generator would
not be used to charge the battery. However, a microgrid that is crucial to a national security
mission may be configured to use the generator to charge the battery when it falls below a
certain threshold to maximize the resilience of the system. The authors have observed both
operating configurations in microgrids currently in use at a variety of critical infrastructure
facilities including national defense installations. The experiment utilizes a grid power
disruption and hourly logs of system power are used to assess Peterson’s and Anderson’s
resilience models.
Systems 2022,10, 245 14 of 24
5. Experimental Results
The scaled microgrid experimental system was integrated and tested in the configura-
tion discussed in the methodology and simulation section above. This section discusses the
results from the experimental system described in the previous section, and assesses the
two microgrid resilience analysis models against the experimental system. The baseline
microgrid operational performance without any power disruptions is first discussed before
it is configured to simulate a power disruption. Key system parameters such as the battery
state of charge and power within the microgrid are presented.
5.1. Experimental Microgrid Baseline Results
Initially, the microgrid is connected to the utility grid and uses grid power to meet
power demands when grid power is available. With renewable energy resources connected,
the microgrid controller utilizes power generated by the renewable resource and sup-
plements it with utility grid power if power delivered by the renewable resource is not
sufficient to meet load demands. The baseline scenario was set up with a 0.7 kW load
demand over 32 h as shown in Figure 8. This experiment was conducted between 23 and
24 July 2022 in Monterey, CA, USA.
Figure 8.
Experimental microgrid baseline result for a clear day with no power disruption is shown in
this figure. It shows power delivered from energy resources, and power delivered to load demands.
As there were no power disruptions, the battery state of charge (SOC) was maintained
close to the full level and the load demand was supplied by the renewable resource and the
utility grid. In this baseline scenario, the power delivered matches or exceeds the power
demands. It can be seen in the data graph in Figure 8that the power delivered from the
utility grid is reduced when there is available power from the solar panels. In addition,
total power delivered exceeds load demands when solar power is available. This small
difference is due to inverter efficiency loss. This behavior of the microgrid under normal
power conditions is within expectations and consistent with deployed systems [68].
5.2. Experimental Microgrid Simulated Disruption Results
As the experimental microgrid system is installed in Monterey, California, USA, with
benign weather and a reliable utility grid, a simulated disruption is next conducted to carry
out the experiment to examine microgrid resilience and power behavior. The test scenario
used for the experiment was a power disruption of the utility grid power. To simulate the
scenario on the microgrid, the system was configured to prioritise power from the batteries,
Systems 2022,10, 245 15 of 24
and grid power was limited to 2 kW to simulate generator power. Data were collected
over a 70 h utility supply disruption with the microgrid operating within battery charge
constraints, with available solar power, and within simulated diesel generator constraints.
The test was conducted on 15 July 2022. The microgrid maintained delivery of power to
meet load demands for the whole test and charged the batteries when it fell below the 70%
(8.4 kWh) threshold. Solar power during daylight hours was able to support load demands
and reduce the rate of battery discharge, and this can be clearly seen in the results graph
in Figure 9. The battery charge takes a longer time compared to night hours to fall below
the threshold. Using the simulated generator to charge the battery is consistent with many
existing operational national defense microgrids that the authors have worked with.
Figure 9. Experimental microgrid simulated loss of utility grid disruption results.
To assess the resilience curve of this simulated utility grid power disruption on the
experimental microgrid, the overall power rating, demand, and power delivered is illus-
trated in Figure 10. Power demands were met for the length of the disruption as there was
sufficient power capacity from the batteries, solar power, and the simulated diesel gener-
ator. The system power rating shows the available microgrid power capacity during the
power disruption. This differs from the component power graph shown in Figure 9as the
microgrid controller balances power delivery to meet demands and only depicts the power
transferred within the system. The system resilience matrix can be computed from the
results between t
d
and t
r
, and this will be presented in the model result discussion sections
below. The minor differences between the power delivered and the constant 0.7 kW load
was also noted and this was assessed to be attributed to component power efficiency losses.
Figure 10. Experimental microgrid simulated loss of utility grid disruption resilience curve results.
Systems 2022,10, 245 16 of 24
The parameters used for the experimental microgrid are next used on Peterston’s
and Anderson’s simulation models for a comparison of results. The generator power is
controlled to toggle on and off in the same time interval as the experiment, and the load
demand is kept constant at 0.7 kW. This aims to generate simulations on the two models to
mimic the experimental results for assessment.
5.3. Peterson’s Simulation Model Results
Using the experiment parameters, the results of Peterson’s simulation model show
that the system was not able to meet load demands for 3 out of 71 h. This simulation result
was not expected as the simulated microgrid was assessed to have more than sufficient
remaining power resources to meet power demands. The battery charge results were
plotted and they showed that the battery continued to deplete even after diesel generator
power was available. Unlike the experimental results in Figure 9, where the battery
charges on excess power of the diesel generator, Peterson’s Simulation Model limits diesel
generator power delivery to load demands. Therefore, as shown in Figure 11 the battery
charge continues to deplete and at time 68 h, the system was not able to meet load demands
until solar power was available.
Figure 11.
Peterson’s Simulation Model battery charge and component power over length of simulation.
5.4. Anderson’s Simulation Model Results
The experiment parameters were also applied to Anderson’s Simulation Model and
it showed that the power generated was able to support the load for the duration of the
test. This result is similar to the experimental results. As this was a high-level, low-fidelity
simulation, minor effects such as power loss within the system were not included and
the power delivered matches the demand as shown in Figure 12. With the overlay of
the experimental data, it can be seen that the simulation model results are similar to the
experimental data.
Systems 2022,10, 245 17 of 24
Figure 12.
Anderson’s Simulation Model power rating, power demands and power delivered over
length of simulation overlaid with experimental data.
6. Result Analysis
The microgrid resilience simulation results from Peterson’s and Anderson’s Simulation
Models are now compared with the experimental results for assessment. Apart from com-
paring the computed metrics, the results for the identified parameters are also computed
and presented. This validates and highlights areas where the respective simulation model
results differ from the experimental results.
6.1. Peterson’s Simulation Model Result Analysis
Peterson’s Simulation Model is able to simulate the utility power disruption of 70 h.
The results report that the system was not able to handle load requirements for 3 h which
is not congruent with the experimental results where the experimental microgrid was
able to support load requirements for the duration of the disruption. Two observations of
the results are now discussed: (1) the reducing battery charge level in Peterson’s model
compared to the ability of the experimental microgrid to maintain battery charge, and
(2) the higher solar power generation in the simulation model. The results for Peterson’s
Simulation Model battery charge were computed and they show a clear positive trend as
shown in Figure 13. Upon investigation, it was found that Peterson’s Simulation Model was
not able to utilize generator power to charge the batteries and this continued to deplete the
battery charge. While many microgrids are configured to not use the generator to charge
the battery, the authors have observed a number of national defense microgrids that do
charge the battery in this manner. It was further observed that Peterson’s Simulation Model
does not have the ability to change this behavior, which indicates that additional work is
necessary to enable this feature.
The peaks for Peterson’s Simulation Model solar power generation was also found to
be higher than the experimental microgrid. The trend line for the residuals of Peterson’s
model solar power, as shown in Figure 14 is below 0, showing that the solar generation
in Peterson’s model was higher than the experimental results. This is caused by lower
solar power generation efficiency due to the position of the static solar panels and weather
conditions during the experiment versus the solar data used in Peterson’s model. However,
the residuals trend line is flat, and close to 0, showing that the model results for solar power
generation are similar to the experiment results. Peterson’s model could be improved with
further refinement of the solar power generation model although the authors suggest the
model is sufficient for high-level system resilience analysis.
Systems 2022,10, 245 18 of 24
Figure 13.
Peterson’s Simulation Model battery charge results show a positive trend because the
model batteries were not charged by excess diesel generator power.
Figure 14.
Peterson’s Simulation Model solar power residuals trend flat and slightly below 0 due to
difference in assumptions for solar efficiency and weather conditions.
The difference in battery charge conditions for Peterson’s Simulation Model is assessed
to only have an impact on specific conditions where there is a periodic need to draw power
from the batteries due to fluctuations in power generation. Although there are various
configuration schemes for the microgrid to ensure power delivery to meet demands, the
microgrid controller would be able to monitor the system conditions and maximize avail-
able power generation resources to meet demand. In the event that the batteries are unable
Systems 2022,10, 245 19 of 24
to charge, the controller would have allowed the diesel generator to run continuously to
meet power demands.
6.2. Anderson’s Simulation Model Result Analysis
The resilience measure for both the experiment and Anderson’s Simulation Model
was computed using Equation (3) and the results were 0.816 for the experimental microgrid
and 0.809 for Anderson’s Simulation Model. The difference of 0.7% in resilience measure is
assessed to be small and power demands are fully met in both the experiment and Ander-
son’s Simulation Model results. The results for Anderson’s Simulation Model power rating
were also computed and the trend line is flat and close to zero, as shown in
Figure 15
. This
shows that Anderson’s model power rating results are similar to the experimental results.
Figure 15. Anderson’s Simulation Model power rating residuals trend flat and close to 0.
7. Conclusions
The experimental microgrid results were useful in helping to validate the simulation
models used for microgrid resilience research. This validation effort for the microgrid
resilience models supports the effort to improve energy resilience in the increasing risk
environment from extreme weather events and adversarial threats. The two microgrid
resilience models (Peterson’s and Anderson’s) studied in this research have been used in
exploring the trade-off between energy resilience versus resource distribution and cost in
other research. Thus, it is useful to experimentally validate the models.
Although there were some differences in the power control for battery charging
between Peterson’s Simulation Model and the experimental microgrid, other aspects of
the model were found to be similar. One option to fix this variance is to include in the
model input an option to allow the user to indicate if the system is designed to allow for
diesel generator power to charge the batteries. In larger-capacity systems where connected
energy resources run continuously, the impact to the simulation results would be minimal
as the battery charge is utilized for bridging gaps in power delivery early in the disruption
phase. Thereafter, standby generators would supply power with minimal fluctuations in
supply. The model input parameters could also be improved to include the system solar
panel position and orientation to improve computation of solar power efficiency.
Anderson’s Simulation Model resilience results and associated metrics were found to
be similar to experimental results. This provides physical evidence that validates resilience,
invulnerability, and recoverability used in Anderson’s model. These experimental results
Systems 2022,10, 245 20 of 24
support the high-level microgrid architecture assessment simulation model that Anderson
developed to be used for design and cost trade-off analysis.
It is important to note that both Peterson’s Simulation Model and Anderson’s Simula-
tion Model are systems engineering tools. While they do analyze microgrids, they are not
detailed electrical engineering models. They are meant to be used early in a system design
process when making large architectural decisions about a microgrid. Later design efforts
can use models such as Fish’s electrical microgrid simulation tool [
69
] are more appropriate
for detailed design and simulation work.
One challenge with validating systems engineering models is the ability to achieve
some degree of statistical significance or confidence in validation results. The research
presented in this paper looked at one microgrid and one outage scenario conducted one
time. Attempting to conduct many scenarios hundreds of times across many microgrids is
at best impractical. There is an ongoing and open discussion in the systems engineering
community about how to validate systems engineering models and methods. During the
2019 National Science Foundation Engineering Design and Systems Engineering workshop
held at Purdue University, this issue was discussed extensively. Currently, the Validation
Square approach of deductive logic [
70
,
71
] is used by some, while others conduct limited
experiments such as the one presented in this paper. The authors view the research
conducted in this paper as a first step on a long road toward the goal of verification and
validation of microgrid resilience models fit for systems engineering purposes.
Battery energy storage can be used as a grid-forming asset for islanded microgrids.
However, the authors have personally observed several large battery systems that cannot
operate in grid-forming mode for a variety of bureaucratic reasons beyond the scope of this
article to describe. Especially for station black start situations, having grid-forming battery
systems is critical. The authors are currently working with several microgrid operators
to upgrade existing batteries or purchase new battery energy storage systems that can
support grid forming. In situations where grid-forming battery storage is unavailable,
either diesel generators need to be kept in spinning reserve (operational and idle or kept
spinning and hot via other means so they can start almost instantaneously) in order to
endure an initial grid disconnect event and keep renewable generation sources online. In
the authors’ experience, some microgrids contain very large diesel generators that can take
up to a half hour to come online and be ready to accept a load from a cold start, which
means that a microgrid may be completely offline for that entire period if a battery energy
storage system is not available and capable of grid forming.
While only one microgrid using a mix of PV and simulated diesel generator power
with a battery storage system was examined in this research, Anderson’s and Peterson’s
models are capable of examining many different microgrid configurations. Microgrids that
are fully reliant on diesel generation, such as is the case for some remote arctic communities,
can be simulated. Conversely, microgrids that are fully renewable can also be simulated.
While only common generation sources (diesel, wind, PV) are available in Anderson’s and
Peterson’s models, it is readily achievable to include other generation sources with small
modifications to the models.
Future Work
While this article investigated one scenario to validate Peterson’s and Anderson’s
models, there is a need for future studies to continue to utilize real-world data to improve
microgrid resilience simulation models. As this research was conducted on a scaled micro-
grid with commercial off-the-shelf components, it may not be able to highlight issues that
may be present in the operation of larger microgrid systems. Some recommendations for
future work include repeating the experiment on a scaled-up physical microgrid system
with a power capacity close to deployed microgrid system, or to collect power disruption
data from deployed systems when there is scheduled downtime.
For a more holistic result, more system factors could be included for the design of this
experiment. Possible factors include the number of renewable energy resources, energy
Systems 2022,10, 245 21 of 24
storage systems, and several repetitions with measured environment data. This will help
refine simulation models and identify significant factors that influence microgrid resilience.
The next experimental microgrid could also explore different power generation systems
and topology design for assessment with the simulation model results.
During a disruptive event and in the aftermath, a variety of contingency plans may
be executed by personnel associated with microgrid operations and recovery. Previous
work has investigated the time it takes line crews to rebuild microgrid infrastructure after
a disruptive event [
72
]. It may be useful to integrate contingency plans and planning
into resilience simulation models to gain a better understanding of how a microgrid can
either reduce the severity of an event (the invulnerability) or minimize the duration of
the recovery.
Author Contributions:
Conceptualization, D.L.V.B. and A.P.; methodology, J.J.H.; software, J.J.H.;
validation, J.J.H.; formal analysis, J.J.H.; investigation, J.J.H.; resources, D.L.V.B.; data curation, J.J.H.;
writing—original draft preparation, J.J.H. and D.L.V.B.; writing—review and editing, D.L.V.B and A.P.;
visualization, J.J.H.; supervision, D.L.V.B. and A.P.; project administration, D.L.V.B. and A.P.; funding
acquisition, D.L.V.B. All authors have read and agreed to the published version of the manuscript.
Funding:
This research was funded by the Navy Shore Energy Technology Transition and Integration
(NSETTI) Program under the direction of Naval Facilities Engineering Systems Command (NAVFAC)
and the office of the Deputy Assistant Secretary of the Navy of Operational Energy. The APC was
funded by the Naval Postgraduate School.
Data Availability Statement: Please contact the authors for data and software used in this study.
Acknowledgments:
The authors extend their thanks to Chris Frackell, Brandon Naylor, and Robert
Lindstrom who assisted at various stages of research and experimentation.
Conflicts of Interest: The authors declare no conflict of interest.
References
1.
Shield, S.A.; Quiring, S.M.; Pino, J.V.; Buckstaff, K. Major impacts of weather events on the electrical power delivery system in the
United States. Energy 2021,218, 119434. [CrossRef]
2.
Campbell, R.J.; Lowry, S. Weather-Related Power Outages and Electric System Resiliency; CRS Report No. R42696; Congressional
Research Service, Library of Congress: Washington, DC, USA, 2012.
3.
Sagunsky, D.L. Risk on the Horizon, Rig for Dark: Solutions to Mitigate DoD’s Reliance on the Fragile Electric Grid. Master ’s
Thesis, Joint Advanced Warfighting School, Norfolk, VA, USA, 2013.
4.
Douglas, E. Texas largely relies on natural gas for power. It wasn’t ready for the extreme cold. The Texas Tribune, 16 February 2021.
5.
Goss, M.; Swain, D.L.; Abatzoglou, J.T.; Sarhadi, A.; Kolden, C.A.; Williams, A.P.; Diffenbaugh, N.S. Climate change is increasing
the likelihood of extreme autumn wildfire conditions across California. Environ. Res. Lett. 2020,15, 094016. [CrossRef]
6.
Department of Defense. Department of Defense Annual Energy Management and Resilience Report (AEMRR) Fiscal Year 2020;
8-BC91200; Department of Defense: Washington, DC, USA, 2021.
7.
Padmanaban, S.; Nithiyananthan, K.; Karthikeyan, S.P.; Holm-Nielsen, J.B. Microgrids; Taylor and Francis: Boca Raton, FL,
USA, 2021.
8.
Anuat, E.; Van Bossuyt, D.L.; Pollman, A. Energy Resilience Impact of Supply Chain Network Disruption to Military Microgrids.
Infrastructures 2021,7, 4. [CrossRef]
9.
Stadler, M.; Cardorso, G.; Mashayekh, S.; DeForest, N. Distributed Energy Resources Customer Adoption Model Plus (DER-CAM+);
Version 1.0.0; Technical Report DER-CAM+; 004669MLTPL00; Lawrence Berkeley National Lab. (LBNL): Berkeley, CA, USA, 2016.
10.
XENDEE|Microgrid Design and Decision Support Platform. Available online: https://xendee.com/ (accessed on 16 July 2022).
11.
Peterson, C.J.; Van Bossuyt, D.L.; Giachetti, R.E.; Oriti, G. Analyzing Mission Impact of Military Installations Microgrid for
Resilience. Systems 2021,9, 69.
12.
Giachetti, R.E.; Van Bossuyt, D.L.; Anderson, W.; Oriti, G. Resilience and Cost Trade Space for Microgrids on Islands. IEEE Syst. J.
2021,16, 3939–3949. [CrossRef]
13.
United States House of Representatives. 10 USC. 1—DEFINITIONS; United States Government Publishing Office: Washington,
DC, USA, 2020.
14.
Pazzalia, J. Sparking Debate: Private or Public? The Effect of Ownership on Electric Utility Performance. Ph.D. Thesis, West
Chester University, West Chester, PA, USA, 2022.
15.
Schneider, K.P.; Tuffner, F.K.; Elizondo, M.A.; Liu, C.C.; Xu, Y.; Ton, D. Evaluating the feasibility to use microgrids as a resiliency
resource. IEEE Trans. Smart Grid 2016,8, 687–696.
Systems 2022,10, 245 22 of 24
16.
Bajwa, A.A.; Mokhlis, H.; Mekhilef, S.; Mubin, M. Enhancing power system resilience leveraging microgrids: A review. J. Renew.
Sustain. Energy 2019,11, 035503.
17.
Van Broekhoven, S.; Judson, N.; Galvin, J.; Marqusee, J. Leading the charge: Microgrids for domestic military installations. IEEE
Power Energy Mag. 2013,11, 40–45. [CrossRef]
18.
Hussain, A.; Bui, V.H.; Kim, H.M. Microgrids as a resilience resource and strategies used by microgrids for enhancing resilience.
Appl. Energy 2019,240, 56–72. [CrossRef]
19. Ton, D.T.; Smith, M.A. The US department of energy’s microgrid initiative. Electr. J. 2012,25, 84–94. [CrossRef]
20.
Hirsch, A.; Parag, Y.; Guerrero, J. Microgrids: A review of technologies, key drivers, and outstanding issues. Renew. Sustain.
Energy Rev. 2018,90, 402–411. [CrossRef]
21.
Giachetti, R.E.; Peterson, C.J.; Van Bossuyt, D.L.; Parker, G.W. Systems Engineering Issues in Microgrids for Military Installations.
INCOSE Int. Symp. 2020,30, 731–746.
22.
Caldognetto, T.; Tenti, P.; Costabeber, A.; Mattavelli, P. Improving Microgrid Performance by Cooperative Control of Distributed
Energy Sources. IEEE Trans. Ind. Appl. 2014,50, 3921–3930.
23.
Mariam, L.; Basu, M.; Conlon, M.F. Microgrid: Architecture, policy and future trends. Renew. Sustain. Energy Rev.
2016
,
64, 477–489. [CrossRef]
24.
Planas, E.; Andreu, J.; Gárate, J.I.; de Alegría, I.M.; Ibarra, E. AC and DC technology in microgrids: A review. Renew. Sustain.
Energy Rev. 2015,43, 726–749. [CrossRef]
25.
Mondol, J.D.; Yohanis, Y.; Smyth, M.; Norton, B. Long term performance analysis of a grid connected photovoltaic system in
Northern Ireland. Energy Convers. Manag. 2006,47, 2925–2947. [CrossRef]
26.
Venkataramanan, G.; Illindala, M. Microgrids and sensitive loads. In Proceedings of the 2002 IEEE Power Engineering Society
Winter Meeting (Cat. No.02CH37309), New York, NY, USA, 27–31 January 2002; Volume 1, pp. 315–322. [CrossRef]
27.
Wood, E. The Rise of Fuel Cell Microgrids: Special Report; Distributed Energy. 2017. Available online: https://www.
microgridknowledge.com/distributed-energy/article/11431516/the-rise- of-fuel-cell-microgrids-special- report (accessed on 16
July 2022).
28.
Li, Z.; Zheng, Z.; Xu, L.; Lu, X. A review of the applications of fuel cells in microgrids: Opportunities and challenges. BMC
Energy 2019,1, 8. [CrossRef]
29.
Akinyele, D.; Olabode, E.; Amole, A. Review of Fuel Cell Technologies and Applications for Sustainable Microgrid Systems.
Inventions 2020,5, 42.
30.
Ekanayake, J.B.; Jenkins, N.; Liyanage, K.; Wu, J.; Yokoyama, A. Smart Grid: Technology and Applications; John Wiley & Sons:
Hoboken, NJ, USA, 2012.
31. IEEE Std 2030.7-2017; IEEE Standard for the Specification of Microgrid Controllers. IEEE: New York, NY, USA, 2018; pp. 1–43.
32.
Sun, C.; Paquin, J.N.; Al Jajeh, F.; Joos, G.; Bouffard, F. Implementation and CHIL Testing of a Microgrid Control System. In
Proceedings of the 2018 IEEE Energy Conversion Congress and Exposition (ECCE), Portland, OR, USA, 23–27 September 2018;
pp. 2073–2080. [CrossRef]
33.
Fang, X.; Misra, S.; Xue, G.; Yang, D. Smart Grid—The New and Improved Power Grid: A Survey. IEEE Commun. Surv. Tutor.
2012,14, 944–980.
34.
Sancaktar, S.; Ng, C. Risk Assessment of Operational Events: External Events; United States Nuclear Regulatory Commission:
Rockville, MD, USA, 2017.
35.
Wielenberg, A.; Alzbutas, R.; Apostol, M.; Bareith, A.; Brac, P.; Burgazzi, L.; Cazzoli, E.; Cizelj, L.; Hage, M.; Hashimoto, K.; et al.
Methodology for Selecting Initiating Events and Hazards for Consideration in an Extended PSA; Technical Report; ASAMPSA E and
EURATOM: Fontenay-aux-Roses, France, 2016. [CrossRef]
36.
Panteli, M.; Mancarella, P.; Trakas, D.N.; Kyriakides, E.; Hatziargyriou, N.D. Metrics and Quantification of Operational and
Infrastructure Resilience in Power Systems. IEEE Trans. Power Syst. 2017,32, 4732–4742.
37.
Wang, Y.; Rousis, A.O.; Strbac, G. On microgrids and resilience: A comprehensive review on modeling and operational strategies.
Renew. Sustain. Energy Rev. 2020,134, 110313. [CrossRef]
38.
Lu, J.; Wang, W.; Zhang, Y.; Cheng, S. Multi-Objective Optimal Design of Stand-alone Hybrid Energy System Using Entropy
Weight Method Based on HOMER. Energies 2017,10, 1664.
39. Adefarati, T.; Bansal, R.C. Reliability, economic and environmental analysis of a microgrid system in the presence of renewable
energy resources. Appl. Energy 2019,236, 1089–1114. [CrossRef]
40.
Wang, S.; Li, Z.; Wu, L.; Shahidehpour, M.; Li, Z. New Metrics for Assessing the Reliability and Economics of Microgrids in
Distribution System. IEEE Trans. Power Syst. 2013,28, 2852–2861.
41.
Rusco, F.; Lepore, B.J.; Ludwigson, J.; Durland, L.; Barnes, T.; Biskup, E.; Ettaro, L.; Gerken, E.; Guinane, T.; Hanford, T.; et al.
DOD Renewable Energy Projects: Improved Guidance Needed for Analyzing and Documenting Costs and Benefits; Technical Report; U.S.
Government Accountability Office Washington United States: Washington, DC, USA, 2016.
42.
Francis, R.; Bekera, B. A metric and frameworks for resilience analysis of engineered and infrastructure systems. Reliab. Eng. Syst.
Saf. 2014,121, 90–103. [CrossRef]
43.
Gholami, A.; Shekari, T.; Amirioun, M.H.; Aminifar, F.; Amini, M.H.; Sargolzaei, A. Toward a Consensus on the Definition and
Taxonomy of Power System Resilience. IEEE Access 2018,6, 32035–32053.
Systems 2022,10, 245 23 of 24
44.
Vugrin, E.D.; Castillo, A.R.; Silva-Monroy, C.A. Resilience Metrics for the Electric Power System: A Performance-Based Approach;
Technical Report SAND2017-1493; Sandia National Lab. (SNL-NM): Albuquerque, NM, USA, 2017. [CrossRef]
45. Haimes, Y.Y. On the Definition of Resilience in Systems. Risk Anal. 2009,29, 498–501.
46.
Albasrawi, M.N.; Jarus, N.; Joshi, K.A.; Sarvestani, S.S. Analysis of Reliability and Resilience for Smart Grids. In Proceedings of
the 2014 IEEE 38th Annual Computer Software and Applications Conference, Vasteras, Sweden, 21–25 July 2014; pp. 529–534.
[CrossRef]
47.
Clark-Ginsberg, A. What’s the Difference between Reliability and Resilience? Stanford University: Palo Alto, CA, USA, 2016.
[CrossRef]
48.
Kashem, S.B.A.; De Souza, S.; Iqbal, A.; Ahmed, J. Microgrid in military applications. In Proceedings of the 2018 IEEE 12th
International Conference on Compatibility, Power Electronics and Power Engineering (CPE-POWERENG 2018), Doha, Qatar,
10–12 April 2018; pp. 1–5. [CrossRef]
49.
Venkataramanan, V.; Srivastava, A.K.; Hahn, A.; Zonouz, S. Measuring and Enhancing Microgrid Resiliency Against Cyber
Threats. IEEE Trans. Ind. Appl. 2019,55, 6303–6312.
50.
Rickerson, W.; Gillis, J.; Bulkeley, M. The Value of Resilience for Distributed Energy Resources: An Overview of Current Analytical
Practices; National Association of Regulatory Utility Commissioners: Washington, DC, USA, 2019.
51.
Giraldez, J.; Booth, S.; Anderson, K.; Massey, K. Valuing Energy Security: Customer Damage Function Methodology and Case Studies at
DoD Installations; Technical Report NREL/TP-7A30-55913; National Renewable Energy Lab. (NREL): Golden, CO, USA, 2012.
[CrossRef]
52.
Mishra, S.; Anderson, K.; Miller, B.; Boyer, K.; Warren, A. Microgrid resilience: A holistic approach for assessing threats,
identifying vulnerabilities, and designing corresponding mitigation strategies. Appl. Energy 2020,264, 114726. [CrossRef]
53.
Li, Z.; Shahidehpour, M.; Aminifar, F.; Alabdulwahab, A.; Al-Turki, Y. Networked Microgrids for Enhancing the Power System
Resilience. Proc. IEEE 2017,105, 1289–1310.
54.
Mahzarnia, M.; Moghaddam, M.P.; Baboli, P.T.; Siano, P. A Review of the Measures to Enhance Power Systems Resilience. IEEE
Syst. J. 2020,14, 4059–4070.
55.
Beaton, D.T. Testing Whether Distributed Energy Storage Results in Greater Resilience of Microgrids. Master’s Thesis, Naval
Postgraduate School, Monterey, CA, USA, 2021.
56.
Xu, Y.; Liu, C.C.; Schneider, K.P.; Ton, D.T. Toward a resilient distribution system. In Proceedings of the 2015 IEEE Power & Energy
Society General Meeting, Denver, CO, USA, 26–30 July 2015; pp. 1–5. [CrossRef]
57.
Castelo Branco, C.A.S.; Moraes, F.P.; Oliveira, H.A.; Neto, P.B.L.; Saavedra, O.R.; de Matos, J.G.; Oliveira, C.B.M.; Ribeiro, L.A.d.S.;
Oliveira, A.C.; Júnior, M.F.A.; et al. Mission Critical Microgrids: The Case of the Alcântara Space Center. Energies
2022
,15, 3226.
58.
Sarker, P.S.; Venkataramanan, V.; Cardenas, D.S.; Srivastava, A.; Hahn, A.; Miller, B. Cyber-Physical Security and Resiliency
Analysis Testbed for Critical Microgrids with IEEE 2030.5. In Proceedings of the 2020 8th Workshop on Modeling and Simulation
of Cyber-Physical Energy Systems, Sydney, NSW, Australia, 21–21 April 2020; pp. 1–6. [CrossRef]
59. Weilkiens, T.; Lamm, J.G.; Roth, S.; Walker, M. Model-Based System Architecture; John Wiley & Sons: Newark, NJ, USA, 2015.
60.
INCOSE; Wiley. INCOSE Systems Engineering Handbook: A Guide for System Life Cycle Processes and Activities; John Wiley & Sons:
New York, NY, USA, 2015.
61.
Al Antelman, R.; Miller, A. Mission Dependency Index Validation Report; Technical Report; Naval Facilities Engineering Service
Center: Port Hueneme, CA, USA, 2002.
62.
Smith, C.W. Mission Dependency Index of Air Force Built Infrastructure: Knowledge Discovery with Machine Learning. Master’s
Thesis, WRIGHT-PATTERSON AFB, Dayton OH, USA, 2016.
63.
Kujawski, E.; Miller, G. The mission dependency index: Fallacies and misuses. In Proceedings of the INCOSE International
Symposium; Wiley Online Library: Hoboken, NJ, USA, 2009; Volume 19, pp. 1565–1580.
64.
Wilcox, S.M. National Solar Radiation Database 1991–2010 Update: User’s Manual; Technical Report NREL/TP-5500-54824; National
Renewable Energy Lab. (NREL): Golden, CO, USA, 2012. [CrossRef]
65.
Deru, M.; Field, K.; Studer, D.; Benne, K.; Griffith, B.; Torcellini, P.; Liu, B.; Halverson, M.; Winiarski, D.; Rosenberg, M.; et al. U.S.
Department of Energy Commercial Reference Building Models of the National Building Stock; Technical Report NREL/TP-5500-46861;
National Renewable Energy Lab. (NREL): Golden, CO, USA, 2011. [CrossRef]
66.
Anderson, W.J. Resilience Assessment of Islanded Renewable Energy Microgrids. Ph.D. Thesis, Naval Postgraduate School,
Monterey, CA, USA, 2020.
67.
Yodo, N.; Wang, P. Engineering Resilience Quantification and System Design Implications: A Literature Survey. J. Mech. Des.
2016,138, 111408. [CrossRef]
68.
Khodayar, M.E.; Barati, M.; Shahidehpour, M. Integration of High Reliability Distribution System in Microgrid Operation. IEEE
Trans. Smart Grid 2012,3, 1997–2006.
69.
Anglani, N.; Oriti, G.; Fish, R.; Van Bossuyt, D.L. Design and Optimization Strategy to Size Resilient Stand-Alone Hybrid
Microgrids in Various Climatic Conditions. IEEE Open J. Ind. Appl. 2022,3, 237–246. [CrossRef]
70.
Pedersen, K.; Emblemsvåg, J.; Bailey, R.; Allen, J.K.; Mistree, F. Validating design methods and research: The validation square.
In Proceedings of the International Design Engineering Technical Conferences and Computers and Information in Engineering Conference;
American Society of Mechanical Engineers: New York, NY, USA, 2000; Volume 35142, pp. 379–390.
Systems 2022,10, 245 24 of 24
71.
Seepersad, C.C.; Pedersen, K.; Emblemsvåg, J.; Bailey, R.; Allen, J.K.; Mistree, F. The validation square: How does one verify and
validate a design method. In Decision Making in Engineering Design; Lewis, K.E., Chen, W., Schmidt, L.C., Eds.; American Society
of Mechanical Engineers Press: New York, NY, USA, 2006; pp. 303–314, ISBN 0791802469.
72.
HersterDudley, M.R. Building Resilience within DOD Microgrids by Considering Human Factors in Recovery Procedures.
Master’s Thesis, Naval Postgraduate School, Monterey, CA, USA, 2021.
