Content uploaded by Daniel Straub
Author content
All content in this area was uploaded by Daniel Straub on Apr 12, 2015
Content may be subject to copyright.
Bayesian updating with structural reliability methods 1/34
Accepted for publication in Journal of Engineering Mechanics, Trans. ASCE
May 2014
Bayesian Updating with Structural Reliability Methods
Daniel Straub, Iason Papaioannou
Engineering Risk Analysis Group, Technische Universität München (straub@tum.de)
Abstract
Bayesian updating is a powerful method to learn and calibrate models with data and
observations. Because of the difficulties involved in computing the high-dimensional integrals
necessary for Bayesian updating, Markov Chain Monte Carlo (MCMC) sampling methods have
been developed and successfully applied for this task. The disadvantage of MCMC methods is
the difficulty of ensuring the stationarity of the Markov chain. We present an alternative to
MCMC that is particularly effective for updating mechanical and other computational models,
termed BUS: Bayesian Updating with Structural reliability methods. With BUS, structural
reliability methods are applied to compute the posterior distribution of uncertain model
parameters and model outputs in general. We propose an algorithm for the implementation of
BUS, which can be interpreted as an enhancement of the classical rejection sampling algorithm
for Bayesian updating. This algorithm is based on the subset simulation and its efficiency is not
dependent on the number of random variables in the model. The method is demonstrated by
application to parameter identification in a dynamic system, Bayesian updating of the material
parameters of a structural system, and Bayesian updating of a random-field-based FE model of
a geotechnical site.
Keywords
Bayesian updating; structural reliability; sampling; measurements; monitoring; FEM.
Bayesian updating with structural reliability methods 2/34
Introduction
With advances in information and sensor technology, increasing amounts of data on
engineering systems are collected and stored; examples include data on deformations and
dynamic properties of structural systems, and data on ambient factors influencing deterioration
of engineering structures. This information can – and should – be used to reduce the uncertainty
in engineering models and optimize the management of these systems. As an example, a smart
structure should use sensor information to automatically trigger actions like detailed inspections
or system shut-downs.
A consistent and effective framework for combining new information with existing models is
provided by Bayesian analysis, in which prior probabilistic models are updated with data and
observations. The Bayesian framework enables the combination of uncertain and incomplete
information with models from different sources and it provides probabilistic information on the
accuracy of the updated model. The latter is of particular relevance, since system predictions
typically remain uncertain even with new information. For this reason, important decisions on
engineering systems should be made on the basis of reliability and risk assessments, and
Bayesian analysis is a cornerstone of such assessments.
Bayesian updating of engineering and mechanical models has been considered since the 1960s.
Benjamin and Cornell (1970) described the use of Bayesian updating for improved engineering
decision making through examples from material testing and geotechnical site investigation.
Tang (1973) recognized the potential of Bayesian updating for updating the probabilistic
description of material imperfections and flaws with inspection information, an idea that later
formed the basis for reliability- and risk-based planning of inspections using Bayesian
principles (Yang and Trapp 1974; Straub and Faber 2005). Bayesian analysis has also been used
extensively for structural identification, i.e. the task of identifying dynamic properties of
structural systems based on vibration measurements (Natke 1988; Beck and Katafygiotis 1998).
In hydrology, Bayesian analysis has been frequently applied for model calibration with
measurements, e.g. of rainfall and discharge measurements (Kavetski et al. 2006; Beven 2008).
The topic has also attracted the attention of the mathematical community (Kennedy and
O'Hagan 2001). Overall, countless applications of Bayesian updating of mechanical,
engineering and computational models are reported in the literature. Its popularity is further
increasing as computational limitations are becoming less of a concern due to increased
computing power and enhanced algorithms.
Bayesian updating with structural reliability methods 3/34
Bayesian updating requires the evaluation of the posterior probabilistic model given the prior
model and the likelihood function describing observation data. With few exceptions, the
posterior model must be evaluated numerically. For cases where the information contained in
the data is much stronger than the prior model, the posterior can be approximated in terms of
an asymptotic expression (Beck and Katafygiotis 1998; Papadimitriou et al. 2001). However,
this expression requires determination of the possibly multiple local maxima of the likelihood
function as well as evaluation of the Hessian of the likelihood at each corresponding parameter
set. Most commonly, Bayesian updating is performed through sampling methods. The Markov
Chain Monte Carlo (MCMC) method is particularly popular, since it allows direct sampling
from the posterior distribution without the need to solve the potentially high-dimensional
integral in the Bayesian formulation (Gilks et al. 1998; Gelman 2004). Many authors have
applied and adopted MCMC to Bayesian updating of mechanical models, including (Beck and
Au 2002; Cheung and Beck 2009; Sundar and Manohar 2013). A main problem of MCMC
methods is that it cannot generally be ensured that the samples have reached the stationary
distribution of the Markov chain, i.e. the posterior distribution (Plummer et al. 2006). Various
alternatives to MCMC exist, which are mostly based on rejection sampling and importance
sampling, e.g. the adaptive rejection sampling from a log-concave envelope distribution, which
is effective for updating single random variables (Gilks and Wild 1992), or generalized
sequential particle filter methods for updating arbitrary static or dynamic systems (Chopin 2002;
Ching and Chen 2007). These methods are often combined with MCMC.
An increasingly popular approach to represent probabilistic models in engineering is the
Bayesian network (BN) framework, which allows decomposing joint probability distributions
into local conditional distributions. As its name suggests, the BN modeling framework is
particularly suitable for Bayesian updating. Under certain conditions, mechanical models can
be effectively represented in a BN with discrete or discretized random variables and efficient
algorithms are then available (Straub and Der Kiureghian 2010b). Examples of BN applied to
updating of mechanical models are presented e.g. in (Mahadevan et al. 2001; Straub 2009;
Straub and Der Kiureghian 2010a). The BN framework may also be combined with continuous
random variables, e.g. it is ideally suited for applications of MCMC algorithms based on Gibbs
sampling (Gilks et al. 1994). Unfortunately, the conditions for the effectiveness of BN, namely
conditional independence among random variables, are often not given in mechanical models.
In this paper, a novel approach to Bayesian updating of engineering models is presented, which
combines a rejection sampling strategy with structural reliability methods. We term the method
BUS (Bayesian Updating with Structural reliability methods). BUS is based on the approach
Bayesian updating with structural reliability methods 4/34
described in (Straub 2011) for Bayesian updating of the reliability of engineering systems. In
this paper, the approach is extended to computing the posterior distribution of uncertain model
parameters and model outputs in general. Key advantages of the BUS approach are its
simplicity and that it enables using the whole portfolio of structural reliability methods and
associated software for Bayesian updating. Here, we implement the BUS approach through
subset simulation (Au and Beck 2001), which can efficiently handle models with many random
variables. Three illustrative examples are included, which demonstrate the application of the
method to structural identification, Bayesian updating of material parameters in a statically
loaded structure, and updating of a random-field-based finite-element (FE) model.
Bayesian updating of mechanical models
Mechanical models consist of a set of equations and boundary conditions that describe the
geometry, material properties and loading conditions. In many applications, some of the
parameters of the model are uncertain. These parameters are modeled as random variables ,
characterized through their joint probability density function (PDF) . The mechanical
model itself is often uncertain, which can be reflected by introducing one or more additional
random variables into to represent model errors.
When new observations or data related to a mechanical model are available, these can be used
to learn the mechanical model. As an example, observations of deformations of a structure
under known loading provide information on the material properties of the structure. Since the
uncertainty in the mechanical model and its parameters is reflected through , learning from
the observations is tantamount to updating the distribution of . Bayes’ rule enables updating a
prior probability distribution with observations or data to a posterior probability
distribution :
(1)
Throughout this paper we use the convention
. The direct
evaluation of the -fold integral in Eq. (1) is not feasible in the general case, which has
motivated the introduction of MCMC and other sampling techniques for Bayesian analysis.
in Eq. (1) is the so-called likelihood function and describes observations and data (Fisher
1922). It is defined as
Bayesian updating with structural reliability methods 5/34
(2)
When observations are made, they often correspond to outcomes of the mechanical models.
Therefore, the likelihood function must include the mechanical models to relate the observation
to the model parameters . As an example, if deformations of a structure are measured, the
model predictions of these deformations for given values of are required. Let denote
such a model prediction. Furthermore, let denote the corresponding observed deformation
and let denote the deviation of the model prediction from the observation. This deviation is
due to measurement errors and model errors (those not modeled explicitly through ); it is
modelled through the PDF . The following relationship holds: . The
likelihood function describing this observation is therefore
(3)
Generally, one can distinguish two classes of observations (Madsen et al. 1985; Straub 2011):
(a) Observations providing equality information. These are measurements of continuous
quantities, for which equalities similar to can be formulated. The likelihood
function for these observations is
(4)
is the PDF of the measured quantity given . Typically, is defined by the PDF
of the observation error , which is often assumed to follow a Gaussian distribution. In case of
an additive observation error , is defined as in Eq. (3). In case of a multiplicative
observation error , the relationship between measurement and model prediction is
; solving for and inserting into , the likelihood function is obtained as
(5)
(b) Observations providing inequality information. These are observations that are
characterized by a finite probability of occurrence, e.g. observations of categorical values or
observations of system performances such as failure/survival, but also censored data.
Commonly, it is possible to formulate a model outcome such that the observation event
is defined through
Bayesian updating with structural reliability methods 6/34
(6)
A model outcome of this form is known in structural reliability as limit state function.
The corresponding likelihood function is then
(7)
is the indicator function. This likelihood thus takes on values 0 or 1.
In the general case, observations are available, of which are of the equality type and
are of the inequality type. Under the common assumption that the measurement/observations
are statistically independent given the model parameters , the combined likelihood of all
observations is
(8)
In case of statistically dependent observation errors , the combined likelihood must be
formulated as a function of the joint PDF of all . An example of statistically dependent
observation errors is included in the applications presented later in the paper.
Through Eq. (8), all observations are combined in the single expression . For the
computations of the posterior distribution presented in the remainder of the paper, it is irrelevant
whether the observations are of the equality type, the inequality type, or combinations thereof.
It should be clear from the preceding discussion that every computation of the likelihood
function requires up to model evaluations to determine . Luckily, for most
applications, one model call is sufficient to compute all , so the necessary number of model
evaluations is equal to the number of likelihood function calls. For advanced numerical models,
such as the geotechnical finite element model presented later in the numerical applications,
these evaluations are computationally costly. Our aim is therefore to perform Bayesian updating,
Eq. (1), with a minimum number of likelihood function calls.
Sampling algorithm
The goal is an efficient algorithm for sampling from the posterior distribution . If direct
evaluation of the integral in Eq. (1) is not feasible, the posterior PDF is known only up to a
proportionality constant, i.e.
Bayesian updating with structural reliability methods 7/34
(9)
Samples from can be generated through a simple rejection sampling algorithm, which is
introduced in the following.
Let be a standard uniform random variable in . Consider the augmented outcome space
and define the domain
(10)
where is a positive constant that ensures for all . The selection of this constant
is discussed in Annex A.
The posterior distribution of Eq. (1) can be written as
(11)
The validity of this result can be demonstrated as follows. The denominator in (11) corresponds
to a structural reliability problem with limit state function ,. It is equal to
the probability of being in under the prior PDF , i.e.
(12)
The numerator in (11) is equal to
(13)
Inserting (12) and (13) in (11), the original Bayesian updating formulation of Eq. (1) is obtained.
The posterior cumulative distribution function (CDF) is obtained by integrating Equation (11)
on both sides:
Bayesian updating with structural reliability methods 8/34
(14)
Generating samples from and samples from the standard uniform distribution,
, the following Monte Carlo approximation to the posterior CDF is obtained:
(15)
It follows that samples generated from and falling into the domain are distributed
according to the posterior . This leads to the following rejection sampling algorithm,
which generates samples of the posterior . Note that this algorithm is equivalent to a
classical rejection sampling where the prior distribution is applied as an envelope distribution
and the likelihood as a filter (Smith and Gelfand 1992).
Simple rejection sampling algorithm
1. .
2. Generate a sample from .
3. Generate a sample from the standard uniform distribution in [0,1].
4. If
a. Accept
b.
5. Stop if , else go to 2.
The rate of acceptance is equal to the denominator in Eq. (11),
(16)
For a fixed number of initial samples , the number of accepted samples is binomial distributed
with parameters and . To generate a fixed number of accepted samples , as in the
above algorithm, the number of samples that must be generated has the negative binomial
distribution:
(17)
Bayesian updating with structural reliability methods 9/34
The mean value of is . Consequently, the expected number of samples to
generate, and thus the expected number of evaluations of , is proportional to .
Let us consider a simple example to clarify the simple rejection sampling algorithm and its
limitations.
Illustration: Updating of a single random variable
Let be a random variable with standard normal prior, i.e.
. A
measurement of is made, resulting in a value of . The measurement is associated with an
additive error , which is normal distributed with zero mean and standard deviation .
The likelihood function is thus
. The parameter is chosen as
. For this example, an analytical solution can be obtained for
comparison: The posterior is the normal distribution with mean 1.6 and standard deviation
.
The domain , together with samples of and , is shown in Figure 1a. In Figure 1b, the
empirical frequency plot of the accepted samples is shown, together with the exact posterior
CDF . The sample mean and standard deviation of the accepted samples are and
; these values are close to the true solution.
In this example, 25 out of 200 samples are accepted. The exact mean acceptance rate can be
computed as
. Although not ideal, a mean acceptance rate of
would be acceptable for most problems. However, the acceptance rate decreases
quickly with increasing number of observations . If measurements are made of in the
above example, all with independent identically distributed (iid) observation errors , the
average acceptance rate is proportional to
(see Annex B).
Bayesian updating with structural reliability methods 10/34
Figure 1. Illustration of the simple rejection sampling algorithm: Updating a single random variable
with the observation , using 200 samples.(a) Joint samples of and drawn from the
prior distribution; accepted samples are indicated by a cross. The shaded area is the
observation/acceptance domain . (b) Resulting empirical frequency plot of the accepted samples,
together with the analytical solution.
Note that the acceptance rate is directly proportional to the constant of Eq. (10). Therefore,
should be selected as large as possible, while still ensuring that for all . In the above
example, it is chosen as , which is the optimal choice. However, in some real
applications can be difficult to evaluate. Strategies for selecting an optimal value of
in such cases are discussed in Annex A.
Bayesian updating with structural reliability methods (BUS)
The above rejection sampling algorithm quickly becomes inefficient with increasing number of
observations due to the large rejection rate, unless the posterior is close to the prior .
This problem is inherent to any direct rejection sampling algorithm (Gelman 2004; Bolstad
2011) and has motivated the development of MCMC techniques for simulating from the
posterior (Gilks et al. 1998). However, the advantage of the simple rejection sampling
Bayesian updating with structural reliability methods 11/34
algorithm over MCMC is that it is straightforward to implement and it is guaranteed to give
exact, uncorrelated samples of the posterior. Therefore, in the following a method is proposed
that maintains partly the advantages of the simple rejection sampling algorithm but has much
higher efficiency. This method, which we call BUS, deals with the inefficiency problem of the
rejection sampling by combining it with structural reliability methods.
As pointed out earlier, the integral in the denominator of Eq. (11) corresponds to a structural
reliability problem. In structural reliability, the domain describes the event of interest,
typically a failure event; in the context of the Bayesian updating of Eq. (11), it describes an
observation event, which we denote by . The integral, and therefore , is equal to the
acceptance rate of the simple rejection sampling algorithm, which decreases with the
square root of the number of observations. In many applications, will be too small for
Monte Carlo simulation to be efficient. What is needed are methods that allow to more
efficiently explore the domain and compute even when it is very small. Structural
reliability methods such as FORM/SORM or importance sampling methods have been
specifically developed to evaluate small probabilities (Ditlevsen and Madsen 1996; Rackwitz
2001). They approximate either the limit state function or the PDF of in the neighborhood of
the domain with the highest probability density, which in the context of Bayesian updating
corresponds to the region of the highest posterior probability density. These methods are
therefore ideally suited to enhance the efficiency of the rejection sampling algorithm.
When applying structural reliability methods, it is convenient to transform the problem from
the outcome space of the original random variables and to a space with independent
standard normal random variables . Since and are
independent, they can be transformed separately. The transformation from to and is
(18)
where standard normal CDF, and
(19)
one of the classical transformations used in structural reliability methods. Both the
Rosenblatt transformation (Hohenbichler and Rackwitz 1981) or the marginal transformation
based on the Nataf model (Der Kiureghian and Liu 1986) can be applied.
The domain describing the observation can now be transformed to an equivalent domain
in standard normal space:
Bayesian updating with structural reliability methods 12/34
(20)
For convenience, we define in terms of a function ,
(21)
so that . is a limit state function describing the observations in standard
normal space, as may be evident from the correspondence of to the definition of
the inequality observation in Eq. (6).
For illustration, Figure 2 shows the transformation of the observation domain and the samples
of shown in Figure 1a to the corresponding domain and samples in standard normal space.
In order to obtain samples from the posterior, the samples falling into the domain must be
transformed to the original space using Eq. (19).
Figure 2. Observation domain and samples of Figure 1a transformed to standard normal space.
A main advantage of BUS is that it can potentially be combined with a large number of available
structural reliability methods. It is not the intention of this paper to identify the most optimal
method, since this choice will depend on problem as well as on the preferences and experience
of the analyst. The strength of expressing the observations and data through limit state functions
is exactly the flexibility that it provides.
Among the available structural reliability methods, importance sampling (IS) methods are
suitable for generating (weighted) samples from the observation domain . Such IS methods
Bayesian updating with structural reliability methods 13/34
may be based on the design point (the Most Likely Failure Point MLFP) identified by means of
the first-order reliability method (FORM). Note that in the context of Bayesian updating, the
design point corresponds to the mode of the posterior distribution. Axis parallel IS based on the
design point (also known as line sampling) was applied in an example presented in (Straub
2011) for the updating of the failure probability; it would also be efficient for updating the
probability distribution of using BUS. Since the identification of the design point is an
optimization problem, alternative IS methods that do not require the design point may be
preferable. In addition to classical adaptive IS methods (e.g. Bucher 1988), newly proposed
approaches such as the one of (Kurtz and Song 2013) appear to be promising for the application
with BUS. When applying IS concepts, the resulting samples are weighted. To obtain
unweighted samples, a resampling scheme must be applied: In an additional step, independent
samples are drawn from a discrete probability distribution defined through the original samples
and their weights (Doucet et al. 2001).
In principle it is possible to use FORM (or SORM) in the context of the BUS approach. If
FORM was applied, the posterior distribution in -space would be approximated by a censored
standard multi-normal distribution; the censoring is defined through the hyperplane given by
, where is the design point and is the gradient vector of the
observation limit state of Eq. (21). However, the use of FORM or SORM is not advocated
without further investigations on its accuracy. As can be observed from Figure 2, the shape of
the observation domain differs from the failure domains typically encountered in structural
reliability.
An alternative to FORM/SORM and importance sampling methods is subset simulation (SuS)
proposed by (Au and Beck 2001), which is particularly efficient for structural reliability
problems where the number of random variables is large. For the considered implementation
in BUS, the SuS has the additional advantage that in its final step it directly produces samples
from the posterior distribution.
BUS algorithm based on subset simulation
In the following, we introduce an algorithm that uses SuS to generate samples in the observation
domain . It assumes that the observation event is transformed to -space by means of Eq.
(21). The algorithm is applied in the numerical examples. We restrict the presentation of SuS
to a summary of its main principles. For details on its implementation in the examples presented
later in the paper, the reader is referred to (Papaioannou et al. 2014).
Bayesian updating with structural reliability methods 14/34
SuS, originally proposed by Au and Beck (2001), evaluates the probability of an event
associated with a limit state function and here defined as .
SuS is based on expressing the event as the intersection of intermediate events that are
nested, i.e. it holds . The probability is then expressed as
(22)
where is the certain event. Thus the possibly small probability is expressed as the
product of larger conditional probabilities. The intermediate events ,, are defined as
, where . The values of can be chosen
adaptively, so that the estimates of the conditional probabilities correspond to a chosen value
. To this end, samples are simulated from the random vector conditional on each
intermediate domain . For each sample, the limit state function is evaluated and the
samples are ordered in increasing order of magnitude of their limit state function values. The
threshold is then set to the -percentile of the ordered samples. This procedure is repeated
until the maximum level is reached, for which . The samples conditional on the
certain event are obtained by crude Monte Carlo sampling. The samples conditional on the
events , for , are computed by simulating states of Markov chains through
MCMC starting from the samples conditional on for which . It is noted that the
seeds of the Markov chains always follow the target distribution and hence the applied MCMC
does not suffer a convergence (burn-in) problem (Zuev et al. 2012; Papaioannou et al. 2014).
The probability of can be approximated by:
(23)
where is the estimate of the conditional probability and is given by the ratio
of the number of samples for which over the total number of samples simulated
conditional on . The value of the intermediate probabilities and the number of samples
in each intermediate step are chosen by the analyst. Au and Beck (2001) suggested using
, whereas Zuev et al. (2012) showed that a choice of leads to similar
efficiency. should be selected large enough to give an accurate estimate of .
The SuS algorithm can be slightly modified for application to Bayesian updating, where
is not the main result. For Bayesian updating, one is interested in obtaining samples that fall
into the domain , i.e. samples conditional on . Therefore, we add a final step, which is the
Bayesian updating with structural reliability methods 15/34
generation of such samples. In this final step, samples are generated, where can be freely
chosen. The resulting algorithm is presented in the following.
SuS-based algorithm for BUS
Define: (number of samples in each intermediate step), (number of final samples),
(probability of intermediate subsets).
Sample from the original distribution:
1. Generate samples
, , from the -variate independent standard
normal distribution, .
2. Define the domain , wherein is chosen as the -percentile of the
samples
, .
3.
Sample from the conditional distributions:
4. Repeat while .
a.
b. Generate conditional samples , , from the -variate
independent standard normal distribution conditional on , ,
using a MCMC algorithm, e.g. (Au and Beck 2001) or (Papaioannou et al. 2014).
c. Define the domain , wherein is chosen as the -percentile of
the samples or 0, whichever is larger.
Sample from the posterior distribution:
5. Identify all samples from that are in the domain . Set equal to
the number of these samples
6. Generate conditional samples , , from the -variate
independent standard normal distribution conditional on , . This uses a
MCMC algorithm where the seeds are the samples identified in 5.
7. Transform the samples to the original space to obtain samples from the posterior
distribution:
, .
Estimate the acceptance probability:
8.
, following Eq. (23), with .
Bayesian updating with structural reliability methods 16/34
As pointed out earlier, the necessary computational effort is determined mainly by the number
of likelihood function calls. In the above algorithm, the first and each intermediate step require
likelihood function calls; the final step (sampling from the posterior) requires an additional
likelihood function calls. The total number of calls associated with the above algorithm
is thus equal to , where is the total number of steps. It is
, hence the computational effort is approximately proportional to .
Note that the number of likelihood function calls in the BUS approach is approximately equal
to the number of limit state function calls in the traditional use of SuS (or other structural
reliability methods), when evaluating the probability of an event with probability . For BUS,
the fact that accuracy in computing is typically not crucial can motivate a reduction in the
number of samples in each step as compared to the original SuS, for which values of
and were proposed by (Au and Beck 2001). However, we do not investigate such
further optimization in this paper.
Bayesian inference
Once samples from the posterior distribution are obtained, any property of this
distribution can be estimated using a MCS approach. When the are generated following
the SuS algorithm outlined above, it must be taken into account that the samples can be
correlated (this is in analogy to MCMC). While this does not affect the estimates below, it does
affect their accuracy.
The expected value of any function of is estimated as
(24)
Accordingly, an estimate of the distribution of is obtained as
(25)
Alternatively, kernel density approximations may be used for the posterior of or
(Turlach 1993).
Bayesian updating with structural reliability methods 17/34
If the interest is in computing the posterior probability of a failure event , defined through a
limit state function as , the corresponding function is
. A MCS estimate from the posterior samples following Eq. (24) is inefficient
when the posterior probability of failure is small. In such cases, which are common in structural
reliability problems, it is beneficial to follow the procedure in (Straub 2011). Thereby, the
intersection of with the observation event is computed directly by means of structural
reliability methods.
Illustrative applications
Three applications are presented. The first one demonstrates the applicability of BUS to
problems that are not globally identifiable. The second demonstrates its applicability to
problems for which a larger number of measurements with correlated errors are available. The
third example demonstrates the applicability of the method to FE-based analysis, including a
random field modeling of material (soil) properties.
Parameter identification in a two DoF system
This example is due to (Beck and Au 2002). The mechanical model, the prior distributions as
well as the data are taken from the original reference, so that the results can be compared.
The example is a two degree of freedom (DoF) system, whose uncertain spring coefficients are
to be determined based on measurements of the first two eigenfrequencies. The problem is not
globally identifiable, i.e. there are multiple combinations of values of the model parameters that
can well explain the measured eigenfrequencies. In (Beck and Au 2002), the problem was
solved through Bayesian updating using a MCMC approach. Here, we employ the proposed
BUS method implemented through the SuS-based algorithm.
A two-story frame-structure is modeled through a shear building model with two DoF (Figure
3). The story masses, which include the mass contributions from the columns, are taken as
deterministic values with kg and kg. The inter-story
stiffness values are modeled as and , where N/m is the
nominal value and are correction factors. Damping is not considered in the analysis.
Observations of the first two eigenfrequencies and are used to update the prior distribution
of to its posterior distribution. Following (Beck and Au 2002), the likelihood
function is
Bayesian updating with structural reliability methods 18/34
(26)
where
(27)
is a modal measure-of-fit function. is the th eigenfrequency predicted with the model
with parameters and is the corresponding measurement. are the means and
is the standard deviation of the prediction error. The measured eigenfrequencies are
and
.
Figure 3. Two DoF shear building model.
The prior probability distributions of are uncorrelated lognormal distributions with
modes 1.3 and 0.8 and standard deviations . The proportionality constant in the
formulation of the likelihood function, Eq. (26), is selected as 1 and the constant in the
observation limit state function is taken as 1. (The largest possible value of the likelihood
function occurs for , in which case it is . Therefore, with it is
for all .)
The samples obtained with the SuS-based rejection sampling algorithm are shown in Figure 4,
including the samples from the intermediate steps. The parameters of the algorithm were
selected as , and , without attempting any optimization. The number
of subset simulation steps is and the number of additional samples in the last step is
. It follows that the total number of samples, and hence the total number of likelihood
function evaluations, is 1849.
Bayesian updating with structural reliability methods 19/34
Figure 4. Bayesian updating of the correction factors . Samples from the prior
distribution, the intermediate subsets and the final posterior distribution, shown in the outcome space
of . In each step, 500 samples were generated, but some samples are coinciding.
In Figure 5, the resulting posterior marginal CDF of is shown for several randomly selected
runs of the algorithm. The bi-modal nature of the posterior distribution, which is already evident
from Figure 4, can be clearly observed also in the marginal CDF. The comparison with the
exact result reveals that the shape of the distribution around the two modes is well captured by
the samples. The probability of the two modes is estimated with less accuracy, as evident from
the scatter in the value of the CDF between 0.5 and 1.5. The same observation is made in (Beck
and Au 2002). This scatter could be reduced by increasing the number of samples; alternatively,
additional importance sampling evaluations may help to reduce this scatter. For most practical
applications the accuracy of the presented results would be sufficient. The more important result
is that the proposed method can successfully identify both modes of the posterior distribution.
Bayesian updating with structural reliability methods 20/34
Figure 5. Posterior CDF of obtained from 6 repeated runs of the proposed algorithm, together with
the exact result.
The results shown in Figure 4 and Figure 5 are consistent with the results given in (Beck and
Au 2002), where further details and discussion of the example can be found.
Bayesian updating of the flexibility of a cantilever beam
This example demonstrates the application of the proposed method to learning a random field
with measurements whose errors are correlated. The example has an analytical solution, which
allows a validation of the method.
We update the spatially variable flexibility of a cantilever beam based on measurements
of the beam deflections. The beam has length and is subjected to a deterministic point
load at the free end (Figure 6). The flexibility is defined as the reciprocal of
the bending rigidity of the beam, so that
, where is the Young’s modulus
at location and is the moment of inertia. We assume that the prior distribution of is
described by a homogeneous Gaussian random field with mean and
exponential auto-covariance function
, where
is the standard deviation and the correlation length.
Bayesian updating with structural reliability methods 21/34
Figure 6. Beam subjected to point load ; measured deflection.
The second derivative of the vertical deflection is a function of and the bending
moment :
(28)
Taking the double integral of the above, it is:
(29)
Using the prior information on , one can evaluate the prior mean and auto-covariance
function of . Moreover, because is Gaussian and is a linear function of ,
the prior distribution of will also be Gaussian. The mean of reads:
(30)
The auto-covariance function of is:
(31)
The above integral can be solved analytically, however the resulting expression is lengthy and
hence omitted here.
Bayesian updating with structural reliability methods 22/34
Measurements of the deflection are made at 50 points (0.1m, 0.2m, …, 5m) along the beam
using optic measurements. The measurements are subjected to additive errors, which are
described by a joint normal PDF with zero mean and covariance matrix , whose elements
are determined from the auto-covariance function
with
. The standard deviation of the measurement error is . The simulated
measurements are shown in Figure 6, together with the true (but in real applications unknown)
deflection of the beam.
The likelihood function describing these measurements is
(32)
where is a vector describing the flexibility at the 50
discretization points and is the vector of deflections at these 50 locations computed from
following Eq. (29).
Bayesian updating of the flexibility at the locations is performed
with BUS using the SuS-based algorithm with , and . The
constant is selected as the inverse of the likelihood function value at the MLE following
Annex A. The resulting number of subsets is
Estimates of the flexibility based on the deflection measurements are presented in Figure 7. The
results show the advantage of a Bayesian analysis over MLE for this problem. Due to the fact
that 50 (correlated) measurements are available for estimating 50 parameters, the problem is
ill-posed and MLE leads to overfitting. The Bayesian analysis with informative prior
distribution regularizes the problem and provides a good approximation to the true values.
Figure 7. Posterior credible interval of the flexibility computed with BUS, together with the true value
and the maximum likelihood estimate (MLE).
Bayesian updating with structural reliability methods 23/34
Figure 8 presents the deflection of the beam, comparing the true value, the measured value, the
MLE and the Bayesian estimate. Figure 8a illustrates that the measured values are very similar
in absolute terms. To enable an appraisal of the differences in the estimates, Figure 8b presents
the difference of the deflections from the ones computed with the prior mean of . The
Bayesian analysis provides a good estimate of the actual deflections, whereas the MLE leads to
an overfitting to the measurements in agreement with the results of Figure 7.
Since the prior distribution of is Gaussian and the measurement errors are additive and
also jointly Gaussian, an analytical solution of the posterior joint PDF of can be obtained
for validating the results obtained with BUS. The results in Figure 8 show that the posterior
credible intervals obtained with BUS coincide almost perfectly with those calculated
analytically. Further proof of the accuracy of BUS is available from Figure 9, which shows a
comparison of the analytically calculated posterior CDF of the deflection at the free end with
the BUS solution.
Figure 8. (a) Posterior credible interval of the deflection, together with the true value, the
measurements and the maximum likelihood estimate (MLE). (b) As in (a), but showing the difference
of the deflection to its prior mean.
Bayesian updating with structural reliability methods 24/34
Figure 9. Marginal CDF of the deflection at the end of the cantilever . Prior CDF, together
with the posterior CDF computed using the proposed method and the analytical solution.
Finite element model updating of a geotechnical construction
This example is based on previous reliability analysis and updating presented in (Papaioannou
and Straub 2012). We update the material properties of the soil surrounding a geotechnical site
based on a deformation measurement performed in situ.
The site consists of a deep trench with cantilever sheet piles in a homogeneous soil layer
of dense cohesionless sand with uncertain spatially varying mechanical properties (see Figure
10). The soil is modelled in 2D with plane-strain finite elements. For simplicity, neither
groundwater nor external loading is considered. Additionally, we take advantage of the
symmetry of the trench and model just half of the soil profile, although this implies an
approximation when randomness in the soil material is taken into account. The material model
used is an elasto-plastic model with a prismatic yield surface according to the Mohr-Coulomb
criterion and a non-associated plastic flow. The sheet pile is modelled using beam elements and
the interaction between the retaining structure and the surrounding soil is modelled using
nonlinear interface elements. The corresponding FE model is implemented in the SOFiSTiK
program (SOFiSTiK 2012). Further details on the mechanical model and the simulation of the
excavation process can be found in (Papaioannou and Straub 2012).
Bayesian updating with structural reliability methods 25/34
Figure 10. Sheet pile wall in sand.
Homogeneous non-Gaussian random fields describe the prior distributions of the uncertain
material properties: Young’s modulus , friction angle and specific weight . The joint
distribution at each pair of locations is modelled by the Nataf distribution (Kiureghian and Liu
1986) with marginal distributions according to Table 1. The auto-correlation coefficient
function is given by a separable exponential model
, where ,
are the absolute distances in the (horizontal) and (vertical) directions. The correlation
lengths are and for all uncertain soil material properties. Cross-correlation
between the different material properties is not included. The random fields are discretized by
the midpoint method (Der Kiureghian and Ke 1988) using a stochastic mesh, consisting of
deterministic FE patches. The stochastic discretization resulted in a total of
basic random variables gathered in a vector . In Figure 11, the stochastic and deterministic FE
meshes are shown. Figure 11c shows the deformed configuration at the final excavation stage
computed with the mean values of the random fields.
Table 1. Prior marginal distributions of the material properties of the soil.
Parameter Distribution Mean COV
Specific weight Normal 19.0 5%
Young’s modulus [MPa] Lognormal 125.0 25%
Poisson’s ratio - 0.35 -
Friction angle Beta(0.0,45.0) 35.0 10%
Cohesion [MPa] - 0.0 -
Dilatancy angle - 5.0 -
Bayesian updating with structural reliability methods 26/34
Figure 11. (a) Stochastic and (b) deterministic finite element mesh of the geotechnical site, shown for
the situation prior to the excavation; (c) deformed configuration at full excavation for the mean values
of the material properties.
We assume that a measurement of the horizontal displacement at the top of the trench
is made at full excavation. The measurement is subjected to an additive error , which
is described by a normal PDF with zero mean and standard deviation =. The
likelihood function describing the measurement is
(33)
where describes the material properties at the midpoints of the stochastic elements and
is the displacement evaluated by the FE program. Bayesian updating of the vector is
performed with BUS using the SuS-based algorithm with , and
. The constant is selected as , which satisfies the condition .
The prior mean of is , which indicates that the prior model underestimates the
measured tip displacement. Figure 12 and Figure 13 show the posterior mean of the Young’s
modulus and friction angle , respectively. The posterior means of the elements in the
vicinity of the trench are smaller than the prior mean, which reflects the effect of the measured
displacement. The effect is local, since the values of the stiffness and strength of the soil farther
away from the trench have limited influence on the deformation at the location of the
measurement. Moreover, the results show the influence of the auto-correlation of the prior
distribution. The change of the posterior means is steeper in the vertical than in the horizontal
direction, which is due to the fact that the prior correlation length in the horizontal direction is
larger than in the vertical direction. The low values of the Young’s modulus observed in the
bottom right of Figure 12 cannot be explained by the measurement and hence are attributed to
sampling error.
Bayesian updating with structural reliability methods 27/34
Figure 12. Posterior mean of the Young’s modulus of the soil.
Figure 13. Posterior mean of the friction angle of the soil.
Discussion
Bayesian updating using structural reliability methods (BUS) has been developed for learning
and calibrating engineering and computer models. By interpreting Bayesian updating as a
structural reliability problem, the existing suite of methods for solving such problems is
available, including importance sampling, subset simulation and first- and second-order
reliability methods (FORM/SORM). For analysts with experience in these methods, it is
straightforward to implement the BUS approach.
The presented implementation of BUS through the subset simulation (SuS) algorithm is ready-
to-use for many practically relevant applications. The presented numerical applications as well
as further numerical investigations not reported in this paper showed that the SuS-based
algorithm works robustly and efficiently for a broad range of problems. It also has the advantage
of being applicable in high dimensions – at least theoretically its performance does not decrease
with increasing number of random variables. It is pointed out that the algorithm uses MCMC
Bayesian updating with structural reliability methods 28/34
to generate the conditional samples required by SuS. Since the initial samples of these Markov
chains are already from the target distribution, it is ensured that all samples follow this target
distribution and convergence of the chain is not an issue. However, the MCMC procedure does
introduce a correlation among the resulting samples, which must be taken into account when
evaluating the posterior statistics.
BUS has originally been developed for problems with an informative prior, i.e. when relevant
information on the uncertain variables is available a-priori. This is a common situation when
learning mechanical models, where significant prior knowledge on uncertain parameters is
often available. But BUS is a viable alternative to existing methods also for problems with
weakly or non-informative prior distributions (e.g. in parameter identification), as demonstrated
by application 1 in this paper.
In this work, no attempt was made to optimize the structural reliability methods for their use in
BUS. We are currently investigating further optimizations of SuS targeted towards its
implementation in BUS, including an adaptive choice of the constant . In addition, we believe
that there is a significant potential in identifying alternative structural reliability methods that
can be more efficient than SuS in combination with BUS for specific types of applications. As
an example, the use of importance sampling methods around the design point (which in BUS
corresponds to the mode of the posterior distribution) appears promising, but has not been
considered here.
In the applications presented in this paper, the number of model evaluations required for
Bayesian updating is on the order of , which is similar to existing state-of-the-art
methods such as transitional MCMC proposed in (Ching and Chen 2007) or the hybrid MCMC
approach of (Cheung and Beck 2009). While the presented implementation of BUS certainly
offers room for a further enhancement of the computational efficiency, we do not believe that
this number can be significantly reduced with any purely sampling-based approach. For
expensive computational models, the BUS approach can be combined with surrogate models
(response surfaces) to further reduce the number of model evaluations. Experience with
adaptive response surfaces available in the structural reliability community may be used for this
purpose.
In many instances, one is interested in updating the reliability of a mechanical system, or more
generally in updating the probability of a rare event with observations. Unlike other methods
presented in the literature, BUS does not require that one first updates the joint probability
distribution of the relevant variables and then computes the updated probability of based
Bayesian updating with structural reliability methods 29/34
on an approximation of the posterior PDF of . With BUS, the observation is represented by
the event and the posterior probability of is equal to , see
(Straub 2011). Therefore, Bayesian updating reduces to solving two structural reliability
problems.
Conclusion
A novel approach to Bayesian updating of mechanical and general engineering models was
proposed, termed BUS. It is based on interpreting the Bayesian updating problem as a structural
reliability problem. This enables one to use the whole portfolio of existing structural reliability
methods for performing Bayesian updating. In this paper, subset simulation was used to obtain
samples of the posterior distribution. Three application examples were included to demonstrate
the versatility and efficiency of BUS.
Bayesian updating with structural reliability methods 30/34
Annex A: How to select the constant defining the observation domain?
BUS requires the selection of the constant in Eq. (10). On the one hand, it is required that
for any . On the other hand, the acceptance rate is directly proportional to ; i.e.
the larger , the larger , which is beneficial for the MCS-based rejection
sampling as well as for BUS in combination with subset simulation. It follows that an optimal
choice of is
(34)
In some instances, is directly available. In case of pure inequality observations, it is
and it follows that . In case of a single measurement with error ,
is equal to maximum of the PDF of , which is readily available. However, it is not always
straightforward to evaluate Eq. (34) when there are several observations with corresponding
likelihoods . occurs at the location of the maximum likelihood, so
if it is possible to determine the maximum likelihood estimator using an optimization
algorithm, it is . In case is not available, an alternative suboptimal
choice is
(35)
For large , i.e. when many individual observations are available, this choice is inefficient. In
these situations, one can get an estimate of the statistical properties of to obtain a more
efficient value of , as demonstrated through an example in the following.
Consider a case where all measurements have random errors that are independent
identically distributed (iid) following a normal distribution with mean zero and standard
deviation . The likelihood function is
(36)
Bayesian updating with structural reliability methods 31/34
Let us assume that the model is perfect if the true parameters are used, i.e. are the
true values of the measured quantities. In this case, the actual measurements are
, i.e. they are normal iid random variables with means and standard
deviation . The maximum of the log-likelihood is then approximately
(37)
where are independent standard normal random variables. The approximation becomes
better with increasing . It follows from Eq. (37) that {} is approximately chi-
squared distributed with degrees of freedom. Therefore, the distribution of the maximum
likelihood is independent of the observation error and depends only on the number of
independent measurements. One can thus select a value of , for which it holds with
given probability , as
(38)
where
is the inverse CDF of the chi-squared distribution with degrees of freedom. E.g.
if the number of independent measurements is , a value will ensure that
for all with probability .
In reality, the model will not be perfect, which will lead to additional deviations of the model
prediction from the observations. Therefore, the maximal log-likelihood will be lower than
predicted by Eq. (37), and the probability that is too large will be smaller than .
Bayesian updating with structural reliability methods 32/34
Annex B: Probability of accepting samples in the simple rejection
sampling algorithm
To illustrate the dependence of the average acceptance rate of samples on the number of
observations, we consider measurements of the quantity , all with iid observation errors.
This is in analogy to the illustration of the simple rejection sampling algorithm provided in the
paper.
The likelihood function with observations , , with iid normal additive error
with zero mean is:
(39)
The measurement values can be written as , where is the true
(but unknown) value of , is the observation error and is the distance between
and the unknown true value of . Inserting this relationship in (39), setting the proportionality
constant to 1 and taking logarithms, one obtains
(40)
The expected value with respect to the observation errors is
(41)
The first-order approximation of the expected value of the likelihood is
(42)
The width of this likelihood function is proportional to . It follows that the expected
number of samples falling into this domain, and consequently , is also proportional to
.
Bayesian updating with structural reliability methods 33/34
References
Au, S.-K. and J. L. Beck (2001). "Estimation of small failure probabilities in high dimensions by subset
simulation." Probabilistic Engineering Mechanics 16(4): 263-277.
Beck, J. and L. Katafygiotis (1998). "Updating Models and Their Uncertainties. I: Bayesian Statistical
Framework." Journal of Engineering Mechanics 124(4): 455-461.
Beck, J. L. and S. K. Au (2002). "Bayesian updating of structural models and reliability using Markov chain Monte
Carlo simulation." Journal of Engineering Mechanics-Asce 128(4): 380-391.
Benjamin, J. and C. A. Cornell (1970). Probability, Statistics, and Decision for Civil Engineers, McGraw-Hill,
New York.
Beven, K. (2008). Environmental modelling: an uncertain future?, Taylor & Francis.
Bolstad, W. M. (2011). Understanding computational Bayesian statistics, Wiley.
Bucher, C. G. (1988). "Adaptive sampling — an iterative fast Monte Carlo procedure." Structural Safety 5(2):
119-126.
Cheung, S. H. and J. L. Beck (2009). "Bayesian Model Updating Using Hybrid Monte Carlo Simulation with
Application to Structural Dynamic Models with Many Uncertain Parameters." Journal of Engineering
Mechanics-Asce 135(4): 243-255.
Ching, J. and Y. Chen (2007). "Transitional Markov Chain Monte Carlo Method for Bayesian Model Updating,
Model Class Selection, and Model Averaging." Journal of Engineering Mechanics 133(7): 816-832.
Chopin, N. (2002). "A sequential particle filter method for static models." Biometrika 89(3): 539-552.
Der Kiureghian, A. and J.-B. Ke (1988). "The stochastic finite element method in structural reliability."
Probabilistic Engineering Mechanics 3(2): 83-91.
Der Kiureghian, A. and P.-L. Liu (1986). "Structural reliability under incomplete probability information." Journal
of Engineering Mechanics 112(1): 85-104.
Ditlevsen, O. and H. O. Madsen (1996). Structural reliability methods. Chichester [u.a.], Wiley.
Doucet, A., N. De Freitas and N. Gordon (2001). Sequential Monte Carlo methods in practice, Springer New York.
Fisher, R. A. (1922). "On the Mathematical Foundations of Theoretical Statistics." Philosophical Transactions of
the Royal Society of London. Series A 222(594-604): 309-368.
Gelman, A. (2004). Bayesian data analysis. Boca Raton, Fla., Chapman & Hall/CRC.
Gilks, W. R., S. Richardson and D. J. Spiegelhalter (1998). Markov chain Monte Carlo in practice. Boca Raton,
Fla., Chapman & Hall.
Gilks, W. R., A. Thomas and D. J. Spiegelhalter (1994). "A language and program for complex Bayesian
modelling." The Statistician: 169-177.
Gilks, W. R. and P. Wild (1992). "Adaptive rejection sampling for Gibbs sampling." Applied Statistics 41(2): 337-
348.
Hohenbichler, M. and R. Rackwitz (1981). "Non-Normal Dependent Vectors in Structural Safety." Journal of the
Engineering Mechanics Division-Asce 107(6): 1227-1238.
Kavetski, D., G. Kuczera and S. W. Franks (2006). "Bayesian analysis of input uncertainty in hydrological
modeling: 1. Theory." Water Resources Research 42(3): n/a-n/a.
Kennedy, M. C. and A. O'Hagan (2001). "Bayesian calibration of computer models." Journal of the Royal
Statistical Society: Series B (Statistical Methodology) 63(3): 425-464.
Kiureghian, A. D. and P. L. Liu (1986). "Structural Reliability under Incomplete Probability Information." Journal
of Engineering Mechanics-Asce 112(1): 85-104.
Kurtz, N. and J. Song (2013). "Cross-entropy-based adaptive importance sampling using Gaussian mixture."
Structural Safety 42(0): 35-44.
Bayesian updating with structural reliability methods 34/34
Madsen, H. O., S. Krenk and N. C. Lind (1985). Methods of structural safety. Englewood Cliffs, NJ, Prentice-
Hall.
Mahadevan, S., R. Zhang and N. Smith (2001). "Bayesian networks for system reliability reassessment." Structural
Safety 23(3): 231-251.
Natke, H. G. (1988). "Updating computational models in the frequency domain based on measured data: a survey."
Probabilistic Engineering Mechanics 3(1): 28-35.
Papadimitriou, C., J. L. Beck and L. S. Katafygiotis (2001). "Updating robust reliability using structural test data."
Probabilistic Engineering Mechanics 16(2): 103-113.
Papaioannou, I., W. Betz, K. Zwirglmaier and D. Straub (2014). "MCMC algorithms for subset simulation."
Manuscript, Engineering Risk Analysis Group, Technische Universität München, Germany.
Papaioannou, I. and D. Straub (2012). "Reliability updating in geotechnical engineering including spatial
variability of soil." Computers and Geotechnics 42: 44-51.
Plummer, M., N. Best, K. Cowles and K. Vines (2006). "CODA: Convergence diagnosis and output analysis for
MCMC." R news 6(1): 7-11.
Rackwitz, R. (2001). "Reliability analysis - a review and some perspectives." Structural Safety 23(4): 365-395.
Smith, A. F. M. and A. E. Gelfand (1992). "Bayesian Statistics without Tears: A Sampling-Resampling
Perspective." The American Statistician 46(2): 84-88.
SOFiSTiK (2012). SOFiSTiK analysis programs. Version 2012. Oberschleissheim, SOFiSTiK AG.
Straub, D. (2009). "Stochastic Modeling of Deterioration Processes through Dynamic Bayesian Networks."
Journal of Engineering Mechanics-Asce 135(10): 1089-1099.
Straub, D. (2011). "Reliability updating with equality information." Probabilistic Engineering Mechanics 26(2):
254-258.
Straub, D. and A. Der Kiureghian (2010a). "Bayesian Network Enhanced with Structural Reliability Methods:
Application." Journal of Engineering Mechanics - ASCE 136(10): 1259-1270.
Straub, D. and A. Der Kiureghian (2010b). "Bayesian Network Enhanced with Structural Reliability Methods:
Methodology." Journal of Engineering Mechanics-ASCE 136(10): 1248-1258.
Straub, D. and M. H. Faber (2005). "Risk based inspection planning for structural systems." Structural Safety
27(4): 335-355.
Sundar, V. S. and C. S. Manohar (2013). "Updating reliability models of statically loaded instrumented structures."
Structural Safety 40(0): 21-30.
Tang, W. (1973). "Probabilistic updating of flaw information." Journal of Testing and Evaluation 1: 459-467.
Turlach, B. A. (1993). Bandwidth selection in kernel density estimation: A review. Discussion Paper 9307, Institut
für Statistik und Ökonometrie, Humboldt-Universität zu Berlin.
Yang, J. and W. Trapp (1974). "Reliability Analysis of Aircraft Structures under Random Loading and Periodic
Inspection." Aiaa Journal 12: 1623-1630.
Zuev, K. M., J. L. Beck, S.-K. Au and L. S. Katafygiotis (2012). "Bayesian post-processor and other enhancements
of Subset Simulation for estimating failure probabilities in high dimensions." Computers & Structures
92: 283-296.
