Content uploaded by Ja'Mar A. Watson
Author content
All content in this area was uploaded by Ja'Mar A. Watson on Jul 19, 2019
Content may be subject to copyright.
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
1!
Modeling for Concise Space Mission Utility Simulation with Apollo as Exemplar
Ja’Mar A. Watson
Watson Institute for Scientific Engineering Research
4201 Wilson Blvd Ste 110-444, Arlington, VA 22203, USA
jamar.watson@sciengresearch.org
Abstract
Presented is a stochastic modeling method enabling rapid yet comprehensive space
mission utility simulation. The method facilitates multivariate analysis with concurrent
tradespace exploration, risk assessment, and holistic design while simultaneously
exploring, assessing, and developing statistically validated concepts of prospective space
missions. Modeling is achieved through the synergistic integration of statistical
mechanics, blackbox, Bayesian, ansatz, and analytics techniques. The method is verified
for its ability to accurately depict a human spaceflight mission and validated for its ability
to perform mission utility analysis by backtesting the Apollo 11-17 missions to the Moon
through Monte Carlo simulation.
Keywords: Mission Utility Simulation; Space Mission Engineering; Surrogate Modeling;
Apollo Missions; Progspexion
1. Introduction
Modeling complex space missions introduces two competing traits: fidelity and
agility. The substance of simulations run on a conglomeration of high fidelity models,
particularly when utilizing stochastic methods inclusive of tradespace exploration, often
encounters one or more of the following problems: 1) hierarchies of high fidelity models
produce lengthy Monte Carlo simulation runtimes, 2) complex mission architectures
make it impractical to simulate their entirety at high fidelity across all concept levels with
a comprehensive mission phasespace, and 3) the literality of high fidelity lacks the
capability to model nascent technologies and callow mission operations. While these
models can be utilized for smaller simulation subsets such as analysis vignettes,
campaigns, or scenarios, the aforementioned problems arise when these same high-
fidelity models are used to simulate an entire mission with tradespace exploration.
Instead, the method seeks to optimize the trade of fidelity and agility by deriving new
lower fidelity models from their higher fidelity counterparts that are computationally
cheaper to execute during iterative Monte Carlo mission simulations used for mission
utility analysis. Additionally, this new surrogate model of the mission is able to
intrinsically incorporate the system-of-systems, emergent, and interoperable behaviors
present in both the mission architecture and concept of operations (CONOPS) that make
up the mission concept. The efficacy of this modeling method is demonstrated by
performing mission utility analysis of the historic Apollo 11-17 missions to the Moon.
2. Modeling
The only way to perform space mission utility analysis is with a simulation of the
mission (Wertz, 2008). Therefore, modeling methods are essential and inherently
influential to mission utility analysis. It is already documented in literature that high
fidelity models are costly and do not ensure simulations are highly effective or
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
2!
generalizable (Elliott, Wesley, & Schiflett, 2001), nor does the reduction of fidelity in
favor of increased agility necessarily produce unrealistic results in multiphysics
dynamical simulations (Williams & Alleyne, 2014). Particularly as it relates to future
space missions, this fidelity to agility ratio adjustment in mission architecture-level
analyses has been shown to be an acceptable alternative with the increased ability to
explore a broader mission tradespace (Moeller & et al, 2011). With these underlying
principles, the following modeling method is used to produce concise space mission
utility analysis for prospective space missions.
2.1. Mission Architecture
Modeling the mission architecture is accomplished by utilizing methods of
statistical mechanics. As such, subsystem and system capabilities are transferred to the
architectural level. The product is mission-level variables that mimic the system-of-
systems nature of mission architectures. The modeling fidelity implies that only the
utility affecting aspects of the mission are reflected in the model and the real overall
mission system topologies need not be represented (Eickhoff, 2009). In a holistic
perspective, the model developed only contains enough fidelity to showcase macroscopic
parameters that affect mission utility. Those variables do not depend on every intricate
detail of the space mission concept. For example, consider Figure 1.
!
Figure 1: Mission Systems Modeling Using Statistical Mechanics
The top illustrates a typical thermodynamic system under consideration, one with a gas in
an enclosed volume. To simulate this, one could consider trying to model every molecule
of the gas as an individual system, assign each a momentum vector, track energy transfers
and so forth. Or, more realistically, one can holistically capture these behaviors at a
higher level through a collection of macroscopic variables such as entropy, temperature,
pressure, volume, etc. Similarly, as shown underneath, it is possible to model a complex
!
!
!
!
!
!
!
!
!
!
!
!
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
3!
set of mission systems in a mission architecture by translating its behavior to a state
vector representative of macroscopic behaviors. In mission utility simulation, the mission
state vector is comprised of variables that mimic tradespace exploration, perturb the
mission phasespace trajectory, and/or alter mission utility, such as the examples of utility
obtainment (U) and time (t) shown in Figure 1. Additionally, the stochastic methods
present in statistical mechanics techniques makes it possible for the state vector variables
to contain distributions to account for uncertainty and/or variations in their values across
the multiple runs of Monte Carlo simulations. Therefore, this mathematical representation
completes the modeling of the mission architecture while enabling multivariate mission
utility analysis.
2.2. Concept of Operations
Prospective space mission concepts are often ambiguous with indistinct constituents.
However, the absence of omniscience does not prevent simulation of these concepts due
to the utility-centric interest. This is further benefited by implementation of Black Box
Theory, shown in Figure 2.
!
Figure 2: Modeling the Operational Concept Using Black Box Theory
Black Box Theory states: if it is possible to define the input and output, the behavior of
the black box can be inferred without knowing all or any of its internal components. The
consequences of this modeling approach means the unfolding of a space mission (black
box) can be characterized according to the mission outcome (output) and potential
mission concepts (input), even though the entirety of its nuances are impossible to
capture. Therefore, the modeling approach is to develop a sufficient black box surrogate
to replicate the behavior of a space mission in order to portray mission outcomes. It is
this black box model that enables mission utility simulations of otherwise unfathomable
prospective space mission concepts.
Once the black box is established, variation of the magnitude of the input mission
design vector is representative of the tradespace as a statistical ensemble. It therefore
includes variables associated with the alterable components of the mission design or
varying operational parameters for mission utility analysis. When a particular mission
design is selected, the mission design vector is used to quantify the values of the initial
mission state vector, which includes variables associated with the black box behavior.
Additionally, when the mission state vector comprises stochastic variables and the
mission utility analysis is accomplished through Monte Carlo simulations, the derived
statistical likelihoods and phasespace perturbations provide datasets for characterizing
performance of the mission.
2.3. Mission Concept
It is of utmost importance to simulate the mission in a manner that does not
suppress the agility of the developed models. Since the mission is represented as a
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
4!
phasespace, the derived mission state vector is considered canonical and the primary
mechanism for mission simulation is progression of the mission state variables along the
phasespace trajectory. Mathematically an operator that updates the state of the mission
throughout the mission phasespace constitutes simulation in this method. Incorporation of
aforementioned stochastic variables in mission propagation further approximates the
varying mission phasespace trajectories across almost all possible mission scenarios and
outcomes (Conway, 2015).
Mission outcome is dependent on the mission phasespace, which minimally
consists of operational execution, mission utility obtainment, constitutive relations, and
the status of mission systems. In performing Monte Carlo simulation, a broad set of
phasespace trajectories are determined which are representative of varying mission
possibilities. Analytics provides concurrently integrated risk assessment by producing
mission outcome likelihoods while pinpointing mission contingencies, aborts, and
interrupts. The simulation explores the tradespace by altering simulation inputs, assesses
the space mission through simulation output of mission utility (and any other desired
metrics included in the model), and develops the mission concept by analytics of the
simulation dataset. The totality of the aforementioned establishes holistic design of the
space mission by determining technological, systematic, architectural, and operational
deficiencies, risks, and gaps. When selecting constituents of the tradespace to represent a
space mission ensemble, the method quantifies the benefit, capability, and utility transfer
of mission design trades.
3. Backtesting
To clarify the implementation and to demonstrate the applicability of this
modeling and simulation (M&S) method for concise space mission utility simulation, the
Apollo 11-17 missions to the Moon are backtested using historic datasets. The
methodology for the mission utility simulation used in tandem with this modeling
approach has been discussed in detail in a previous publication (Watson, 2017). The
publication presents the Holistic Methodology for Stochastic Mission Utility Analysis,
which describes steps required to complete mission utility simulation. As such, the
publication’s 10-task framework is utilized for verification and validation (V&V) of this
modeling method. The exemplification of the Apollo missions verify the modeling
method for accurate depiction of a historic human spaceflight mission and validate the
method in its sufficient production of data to perform space mission utility analysis.
3.1. Verification
3.1.1 Mission Objectives and Metrics
In this mission utility analysis of the Apollo lunar missions, Task 1 of quantifying
the mission objectives is simply what president John F. Kennedy stated in his 1961
speech (Kennedy, 1961) – “achieving the goal, before this decade is out, of landing a man
on the moon and returning him safely to the Earth.” With such a singularly directional
load, Task 2 of defining the metrics provides only the measure of effectiveness (MoE) of
mission utility, in which the mission objective represents its entirety. To facilitate
validation, the figures of merit (FoM) of fragility, resiliency, risk, robustness, and
versatility are transferred. Fragility is the ability to result in loss of mission (LoM) or loss
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
5!
of crew (LoC) from loss of a system (LoS) or loss of vehicle (LoV). Resiliency is the
mission’s ability to maintain utility with LoS, LoV, and mission dynamics (MD). MD are
any events in the mission that perturb the mission phasespace trajectory. Risk is the
probability of LoM or LoC. Robustness is the ability to result in LoC or LoM from MD.
Versatility is the mission’s ability to operate in different mission designs with the same
mission systems (Watson, 2017).
3.1.2 Mission Modeling
Task 3 is to model the space mission arena, which includes the environments in
which the mission systems will operate. In M&S of the Apollo missions, the space
mission arena includes the Moon, the Sun, the Earth, and space weather. Since the
mission only partially takes place within the confines of the Earth’s magnetic field, it is
still possible to encounter space weather during the mission, and this operational weather
is included in the model.
Task 4 of modeling the mission architecture needs only to include the launch and
space segments. Launch was achieved utilizing the Saturn V (SV), while the latter
consisted of the Command Module (CM): the crew transit habitation of the mission, the
Service Module (SM): consisting of the subsystem components for performing the
mission, and the Lunar Module (LM): responsible for EDL, ascent, and surface habitation
of the crew during the lunar portion of the mission. When the CM and SM are connected,
it is referred to as the CSM. For missions with notional mission systems and/or nascent
technologies where known modeling methods are not established, the ansatz method can
be used. While this is not necessary for this V&V due to the historic nature of the Apollo
missions, the following is shown as a demonstration.
Mission architecture-level algorithms are developed via techniques similar to the
ansatz method. They may be derived by any means, such as an approximation of its
equations of motions, derivations from experimental datasets, comparative analysis of
analog mission systems, etc. These algorithms are generally run in analysis vignettes to
determine if the assumed modeling of the mission system behaves as desired for the
mission and that responses to environmental and operational stimuli are as expected for
the constitutive relations.
For example, a mission architecture-level algorithm can be developed for the
probability of successful launch based on the maturity of the SV. A simple regression of
historic SV performance can produce the desired algorithm (Equation 1).
!
"# $ %&'&()*+% , %&'-.)*% / %01'00%%
(1)
Here X denotes the maturity of the SV (1 = Low, 2 = Modest, 3 = High). If this algorithm
is being derived from high fidelity models, it may be possible to replace the qualitative
maturity with that of technology readiness level (TRL) as a definition for X in a similar
algorithm. To note, the algorithm itself may have no physical meaning or resemblance to
higher fidelity models. Instead, it is the desired inputs (in this case, just the maturity of
SV) and output (in this case, just the probability of successful launch) that are of
importance. The insertion of this derived equation constitutes a mathematical
representation for the behavior of its portion of the space mission model.
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
6!
3.1.3 Operational Concept
With the static modeling complete, the dynamics of the simulation are introduced.
For mission analyses like Apollo with extensive historic analogs and mission
architectures composed of mission systems with high TRLs, it is advantageous to reduce
the dynamic aspects of the mission to a single variable per mission event. In essence,
regardless of derivation, this is the incorporation of a possibilistic/probabilistic risk
assessment (PRA) dataset. However, although each data point may represent the
probability of event occurrence (probabilistic), it is important that these values be
represented possibilistically (the degree to which occurrence is viewed as being possible).
Possibilistic values allow for more realistic determination of varying mission outcomes
under uncertainty. Mathematically, this means the summation of individual data points
need not equal to one or 100% (Nikolaidis, Haftka, & Rosca, 1998). It is possible to
convert the aforementioned mission architecture-level algorithms, such as the example of
Equation 1, to stochastic datasets by tabulating performance metrics and outcomes over
analysis vignettes.
Due to the limited scope and historical nature of the mission, defining and
developing the CONOPS is also reduced to the PRA representation. Therefore Task 5 of
the methodology, defining the operational concept, consists of laying out the stochastic
portion of the mission simulation. In the absence of ‘actual’ PRA data for the historic
NASA mission, a sufficient model of operations is generated based on risk and reliability
analysis of the mission (Young & Wilhite, 2009), Bayesian inference (Sforza, 2016), and
research conducted for other potential lunar human spaceflight missions (National
Aeronautics and Space Administration, 2005) such as the cancelled Constellation
program (Prassinos & et al, 2006). This literature review effort culminates in the
following possibility of major event occurrences (Table 1).
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
7!
Table 1: Probabilities of Occurrence for Apollo Mission Events
EVENT
PO
Launch Success
93.33
CSM Loiter
98.69
CSM Avionics
85.95
CSM ECLSS
94.92
CSM Power
98.17
SM for TLI
99.25
TLI Burn
92.93
Correct Lunar Trajectory
98.69
CSM Habitation
85.71
SM for LOI
99.24
LOI Burn
96.91
LOI Navigation
99.90
ADCS for EDL
99.22
Lunar EDL
94.87
LM Avionics
96.35
LM ECLSS
94.92
LM Power
95.72
LM Habitation
99.93
Lunar Surface Operations
97.76
CM Loiter
99.99
Lunar Ascent
97.41
Rendezvous Maneuver
99.43
ADCS for Docking
99.93
SM for TEI
98.54
TEI
92.71
ADCS for Re-entry
99.22
Re-entry
99.26
β CCF Factor
00.01
γ CCF Factor
00.05
SPE LoC
1.5E-4
SPE LOM
1.5E-3
The table includes acronyms for trans-lunar injection (TLI), lunar orbit insertion (LOI),
trans-Earth injection (TEI), and entry, descent, and landing (EDL) events, as well as
attitude determination and control systems (ADCS) and environmental control and life
support systems (ECLSS) of the mission systems. Constitutive relations are solar particle
events (SPE) representing the operational weather of space, as well as common cause
failures (CCF) which determine multiple failure occurrences from a single event.
The derived mission state vector (X) therefore includes U and t as previously
introduced, as well as a variable for the status of each aforementioned mission system,
the crew, and a PO vector representing all of the probabilities of event occurrence in the
mission. When this is integrated stochastically, the operator propagating the mission state
vector therefore consists of a pseudorandom number generator to draw on each events
possibility of occurrence and represents the varying outcomes attributed to each mission
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
8!
phase or event.
3.1.4 Tradespace
A “baseline” of performance is inherent in the fact that the Apollo program
consists of historic space missions. Therefore Task 6 of establishing the baseline involves
gathering its historic mission outcomes. This is followed by Task 7, which defines the
tradespace. To make this applicable for V&V, the Apollo missions did consist of
numerous landing sites (Dunabr & Loff, 2015) (Table 2), and those can be “traded” to
represent a tradespace among the mission.
Table 2: Parameters of Apollo Landing Sites and Parking Orbits
SITE
LAT°
LON°
ALT(km)
Orbit ι°
Descartes Highlands
-08.97
15.51
173.00
32.54
Fra Mauro
-03.65
-17.48
190.79
31.12
Hadley-Apennine
26.08
03.66
160.45
29.68
Ocean of Storms
-03.04
-23.42
190.79
23.42
Sea of Tranquility
00.71
23.63
190.95
32.52
Taurus-Littrow
20.16
30.77
170.37
28.53
Additionally, redundancy was added to the Apollo designs to improve mission
performance. This also builds upon the tradespace and produces a redundant possibilistic
CONOPS through published PRA data with redundancy. This version of the model is
referred to as Model with Redundancy (Model w/ R).
As a final addition to the tradespace, Equation 1 is revisited. The risk of launch
failure prior to Apollo 11 was 93.33% (low SV maturity and a partial failure in the
Apollo 6 developmental test mission) but did not fail in any of the Apollo lunar landing
missions. Therefore, an additional trade on launch to high maturity SV is added to
provide a fully redundant mission design representative of Apollo 11-17. This is referred
to as Model with Redundancy and Launch (Model w/ R&L).
3.1.5 Mission Simulation
In performing Task 8, the mission utility simulations, approximately 1100 Monte
Carlo runs brought the mission results to convergence as shown in Figure 3.
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
9!
!
Figure 3: Apollo Simulation Convergence Determination
The results of data analytics are shown in Table 3. These compare the actual results of the
Apollo missions with simulation results of the three aforementioned models.
Table 3: Percent Apollo Simulation Results
ACTUAL
MODEL w/ R&L
MODEL w/ R
MODEL w/o R
Success
85.71
78.55
72.55
42.00
LoM
14.29
18.91
25.00
47.00
LoC
00.00
02.54
02.45
11.00
The simulation was therefore able to replicate the apparent mission utility of the historic
Apollo missions at 91.65% accuracy by performing mission utility simulations inclusive
of actual SV performance and redundancy implications modeling. Mission utility
progression of the models indicates greater accuracy may be possible if actual PRA
datasets can be utilized.
3.2. Validation
Showcasing the ability to quantify metrics from the simulation dataset completes
validation. Following aggregation of the data, the validation process can begin to
determine if performing space mission utility analysis with the modeling method
produces data to analyze metrics and perform analytics: Task 9 of the methodology. Due
to the binary nature of utility in this mission, its average value is equivalent to the percent
success outcome of the mission simulation. Utility of successful missions, therefore, is
always 100% and LoM/LoC always 0%. This held true across the three models.
Therefore, data production is sufficient to determine the MoE of mission utility.
3.2.1 Metrics Analysis
Analysis of select applicable FoM requiring post-processing of simulation data
assists in validation. The calculation of risk is already presented in Table 3. Recall, the
FoM of fragility determines the percentage of LoS/LoV directly attributing to LoC/LoM.
Conversely, robustness determines what percentage of mission dynamics directly results
in LoC/LoM. These FoM are successfully calculated and are presented in Table 4.
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
10!
Table 4: Apollo Mission Fragility and Robustness
!
FRAGILITY
ROBUSTNESS
LoM
LoC
LoM
LoC
Model w/ R&L
94.59
05.41
66.00
34.00
Model w/ R
96.48
03.52
53.85
46.15
Model w/o R
93.18
06.82
46.15
53.85
Due to the absence of partial satisfaction of mission utility and the results of fragility and
robustness, the metrics of resiliency and versatility simply compute a delta of the average
mission utility values presented in Table 3. This concludes the metrics analysis portion of
the validation.
3.2.2 Analytics
Contributions to mission utility of all three mission models indicated failure of the
CSM’s habitation as a major contributor to mission failures. This is demonstrated in the
failure contribution figures derived from the mission simulations.
For Model w/ R&L (fully redundant, Figure 4 ).
!
Figure 4: Proportion of Failure Contributions in Fully Redundant Apollo Missions
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
11!
Model w/ R (redundant, Figure 5).
!
Figure 5: Proportion of Failure Contributions in Redundant Apollo Missions
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
12!
Model w/o R (non-redundant, Figure 6).
!
Figure 6: Proportion of Failure Contributions in Non-Redundant Apollo Missions
The differential in these failure contribution proportionalities also additionally verifies
the effectiveness of alterations to the modeling input. Moreover, this was the reason for
failure of the actual Apollo 13 mission, reassuring the accuracy of mission failures in this
V&V process.
Additionally, the CSM as an entire mission system dominated the cause for LoV.
Figure 7 provides visual representation of this analysis with the fully redundant,
redundant, and non-redundant missions segmenting the outer, middle, and inner
concentric circles respectively.
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
13!
!
Figure 7: Ratio of Simulated Apollo Mission Vehicle Failures
In summary, for the model with redundancy and launch, the CSM accounted for over
74% of vehicle failures and the failure of its habitation tabulated roughly 61% of mission
failures, effectively replicating the source of the only failed Apollo mission (Apollo 13)
and the numerous CSM anomalies encountered in Apollo 16.
3.2.3 Discussion
To add context to the FoM calculations, the mission fragility performed well.
However, it failed to allow any successful mission outcomes. This is likely due to the
jettisoning of the LM prior to TEI, making the Apollo 13 solution impossible post lunar
landing. Additionally, the sensitivity of systems through the re-entry phase and the likely
aborting of the mission in case of vehicle failure prior to lunar EDL also contributed to no
successful mission outcomes with LoV occurrences. Once again, the CSM’s habitation
overwhelmed LoV fragility, with rare LoC outcomes likely occurring from CSM ECLSS
and LM power failures.
Similarly, robustness was unable to achieve successful missions when maneuver
events were failed and indicated the mission design was not favorable for avoiding LoC
in such a case. The low probability of failed events, however, made this effect minimal.
Major contributors to reduced robustness of the simulated Apollo missions were entry
into incorrect trans-lunar trajectories, failed TLI burns, and unsuccessful ascent from the
lunar surface.
As expected, the trade of landing site produced a coefficient of correlation to
mission utility of 0.03, showing it potentially had no effect on the mission outcome. This
was also apparent for the actual Apollo missions, mostly due to the lack of variation in
EDL among the sites and a near absence of CONOPS for surface operation to rely on
varying lunar topography. The MoE of mission utility and the FoM of fragility and
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
14!
robustness, along with the total number of failure events in 1100 Monte Carlo runs of
each of the three mission models, are used to provide correlational and intercorrelational
values of metrics (Table 5).
Table 5: Correlation and Intercorrelation Among Apollo Mission Metrics
!
Mision
Utility
Failure
Events
LoM
Fragility
LoC
Fragility
LoM
Robustness
LoC
Robustness
Mission
Utility
-0.99
-0.59
-0.72
-0.88
-0.88
Failure
Events
0.51
0.64
0.92
0.92
LoM
Fragility
-0.99
-0.13
-0.13
LoC
Fragility
0.29
0.29
LoM
Robustness
-1.00
LoC
Robustness
Although it is true that correlation does not equal causation in most instances, correlation
does provide a quantifiable metric for the behavior of causation. This is particularly
useful in stochastic, complex systems where interoperability and emergent behaviors are
not fully understood or cannot be comprehensively captured, thus making absolute
determination of causation impractical. It is also useful from a holistic designer’s or
decision-maker’s perspective because correlation can statistically implicate architectural
and/or operational components for alteration. Particularly in experimental design,
defining the underlying facet of causation is not a necessity since input manipulations can
be tabulated for output differentials even without understanding any of the process’
mechanisms. Recall the definition of Black Box Theory in Figure 2 that explains this
modeling phenomenon. Additionally, the integration of black box techniques enables
analysis of confounding variables that direct correlation and intercorrelation cannot
identify.
Table 5 quantifies the expected mission utility relationships of failure events and
the ability to result in LoC or LoM. The high negative correlation signifies these mission
concept properties as driving the decrease of mission utility. The actual failure events
themselves showed high correlation in increasing the ability to result in LoC/LoM for
mission maneuver failures, but only modest correlation for increasing LoC/LoM from
LoV/LoS. This is further supported by the fact that as mission system fragility increases,
the ability to result in LoC/LoM from failed mission operations has little to no
correlation. As expected and by definition, the LoC and LoM components of fragility are
purely negatively correlated. This is the same for the LoC/LoM components of
robustness. This is due to separate tabulation of LoC and LoM outcomes, despite the fact
that LoC technically also results in LoM.
Due to the historic nature of the Apollo missions in this M&S V&V effort, the
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
15!
predictive, prescriptive, and enterprise architecting portion of the analytics are omitted.
The methodology would continue into Task 10 for design synthesis and mission
architecting of new mission concepts but these are not applicable to this mission utility
analysis of the Apollo 11-17 missions. Therefore, validation has successfully been
achieved.
4. Conclusion
Following a 91.65% accurate depiction of the Apollo 11-17 missions to the Moon,
demonstrated capturing of data for analytics, and production of desired metrics, the
stochastic modeling method is considered verified and validated for concise space
mission utility simulation. The modeling method enables modeling of the mission,
inclusive of the mission architecture, CONOPS, operational weather, and uncertainty at
the payload, system, architectural, and mission levels. It enables variable inputs of the
mission concept across a tradespace without the need to alter the models or simulators
themselves. The stochastic modeling enables analysis of mission event likelihoods across
almost all possible mission outcomes. Analytics of the simulations’ dataset provides
mathematical input for concept development and experimentation, as well as architecting
statistically validated concepts. This and a comprehensive replication of the Apollo 11-17
missions make the stochastic modeling method viable for analyzing space mission utility
via Monte Carlo space mission simulation. The return of human spaceflight missions in
the future will enable further V&V of this modeling philosophy.
Acknowledgements
This research did not receive any specific grant from funding agencies in the
public, commercial, or not-for-profit sectors.
The author would like to thank all reviewers for their recommendations to this
article for publication.
References
Conway, E. (2015). Exploration and Engineering: The Jet Propulsion Laboratory and
the Quest for Mars. Baltimore, M.D.: John Hopkins University Press.
Dunabr, B., & Loff, S. (2015). The Apollo Missions. (National Aeronautics and Space
Administration) Retrieved from Apollo Overview.
Eickhoff, J. (2009). Simulating Spacecraft Systems. Heidelberg: Springer.
Elliott, L. R., Wesley, R. J., & Schiflett, S. G. (2001). Scaling Scenarios for Synthetic
Task Environments: Issues Related to Fidelity and Validity. Human Factors and
Ergonomics Society, 1, pp. 377-381. Santa Monica, CA.
Kennedy, J. F. (1961). Address to Joint Session of Congress May 25, 1961. Presidential
Library and Museum. Washington, D.C.: Columbia Broadcasting System.
Moeller, R. C., & et al. (2011). Assessment Using the JPL Rapid Mission Architecture
(RMA) Team Approach, IEEE Aerospace Conference, (pp. 1-11). Big Sky, MT.
National Aeronautics and Space Administration. (2005). NASA's Exploration Systems
Architecture Study. NASA-TM-2005-214062.
Nikolaidis, E., Haftka, R., & Rosca, R. (1998). Comparison of Probabilistic and
Possibility-Based Methods for Design Against Catastrophic Failure Under
Pre-Print:)J.A.)Watson,)Modeling)for)Concise)Space)Mission)Utility)Simulation)with)
Apollo)as)Exemplar,)Journal)of)the)Astronautical)Sciences,)In)Press,)2017)
!
16!
Uncertainty . Virginia Polytechnic Institute and State University and the
University of Florida.
Prassinos, P., & et al. (2006). Constellation Probabilistic Risk Assessment (PRA): Design
Consideration for CEV. OSMA-PRA-07-01, National Aeronautics and Space
Administration, Safety Assurance and Requirements Division, Washington, D.C.
Sforza, P. M. (2016). Safety, Reliability, and Risk Assessment. In Manned Spacecraft
Design Principles. Waltham, M.A.: Elsevier.
Watson, J. A. (2017). Holistic Methodology for Stochastic Mission Utility Analysis. Int.
J. System of Systems Engineering, 8(2), 174-188.
Wertz, J. R. (2008). ORS Mission Utility and Measures of Effectiveness. 6th Responsive
Space Conference. Los Angeles, C.A.
Williams, M. A., & Alleyne, A. G. (2014). Switched-Fidelity Modeling and Optimization
for Multi-Physics Dynamical Systems. IEEE American Control Conference, (pp.
3104-3109). Portland, OR.
Young, D. A., & Wilhite, A. (2009). A Methodology for Achieving Optimal Reliability
and Cost in a Lunar Architecture. American Institute of Aeronautics and
Astronautics.
!
