Content uploaded by Daniel Straub

Author content

All content in this area was uploaded by Daniel Straub on Apr 12, 2015

Content may be subject to copyright.

Bayesian updating with structural reliability methods 1/34

Accepted for publication in Journal of Engineering Mechanics, Trans. ASCE

May 2014

Bayesian Updating with Structural Reliability Methods

Daniel Straub, Iason Papaioannou

Engineering Risk Analysis Group, Technische Universität München (straub@tum.de)

Abstract

Bayesian updating is a powerful method to learn and calibrate models with data and

observations. Because of the difficulties involved in computing the high-dimensional integrals

necessary for Bayesian updating, Markov Chain Monte Carlo (MCMC) sampling methods have

been developed and successfully applied for this task. The disadvantage of MCMC methods is

the difficulty of ensuring the stationarity of the Markov chain. We present an alternative to

MCMC that is particularly effective for updating mechanical and other computational models,

termed BUS: Bayesian Updating with Structural reliability methods. With BUS, structural

reliability methods are applied to compute the posterior distribution of uncertain model

parameters and model outputs in general. We propose an algorithm for the implementation of

BUS, which can be interpreted as an enhancement of the classical rejection sampling algorithm

for Bayesian updating. This algorithm is based on the subset simulation and its efficiency is not

dependent on the number of random variables in the model. The method is demonstrated by

application to parameter identification in a dynamic system, Bayesian updating of the material

parameters of a structural system, and Bayesian updating of a random-field-based FE model of

a geotechnical site.

Keywords

Bayesian updating; structural reliability; sampling; measurements; monitoring; FEM.

Bayesian updating with structural reliability methods 2/34

Introduction

With advances in information and sensor technology, increasing amounts of data on

engineering systems are collected and stored; examples include data on deformations and

dynamic properties of structural systems, and data on ambient factors influencing deterioration

of engineering structures. This information can – and should – be used to reduce the uncertainty

in engineering models and optimize the management of these systems. As an example, a smart

structure should use sensor information to automatically trigger actions like detailed inspections

or system shut-downs.

A consistent and effective framework for combining new information with existing models is

provided by Bayesian analysis, in which prior probabilistic models are updated with data and

observations. The Bayesian framework enables the combination of uncertain and incomplete

information with models from different sources and it provides probabilistic information on the

accuracy of the updated model. The latter is of particular relevance, since system predictions

typically remain uncertain even with new information. For this reason, important decisions on

engineering systems should be made on the basis of reliability and risk assessments, and

Bayesian analysis is a cornerstone of such assessments.

Bayesian updating of engineering and mechanical models has been considered since the 1960s.

Benjamin and Cornell (1970) described the use of Bayesian updating for improved engineering

decision making through examples from material testing and geotechnical site investigation.

Tang (1973) recognized the potential of Bayesian updating for updating the probabilistic

description of material imperfections and flaws with inspection information, an idea that later

formed the basis for reliability- and risk-based planning of inspections using Bayesian

principles (Yang and Trapp 1974; Straub and Faber 2005). Bayesian analysis has also been used

extensively for structural identification, i.e. the task of identifying dynamic properties of

structural systems based on vibration measurements (Natke 1988; Beck and Katafygiotis 1998).

In hydrology, Bayesian analysis has been frequently applied for model calibration with

measurements, e.g. of rainfall and discharge measurements (Kavetski et al. 2006; Beven 2008).

The topic has also attracted the attention of the mathematical community (Kennedy and

O'Hagan 2001). Overall, countless applications of Bayesian updating of mechanical,

engineering and computational models are reported in the literature. Its popularity is further

increasing as computational limitations are becoming less of a concern due to increased

computing power and enhanced algorithms.

Bayesian updating with structural reliability methods 3/34

Bayesian updating requires the evaluation of the posterior probabilistic model given the prior

model and the likelihood function describing observation data. With few exceptions, the

posterior model must be evaluated numerically. For cases where the information contained in

the data is much stronger than the prior model, the posterior can be approximated in terms of

an asymptotic expression (Beck and Katafygiotis 1998; Papadimitriou et al. 2001). However,

this expression requires determination of the possibly multiple local maxima of the likelihood

function as well as evaluation of the Hessian of the likelihood at each corresponding parameter

set. Most commonly, Bayesian updating is performed through sampling methods. The Markov

Chain Monte Carlo (MCMC) method is particularly popular, since it allows direct sampling

from the posterior distribution without the need to solve the potentially high-dimensional

integral in the Bayesian formulation (Gilks et al. 1998; Gelman 2004). Many authors have

applied and adopted MCMC to Bayesian updating of mechanical models, including (Beck and

Au 2002; Cheung and Beck 2009; Sundar and Manohar 2013). A main problem of MCMC

methods is that it cannot generally be ensured that the samples have reached the stationary

distribution of the Markov chain, i.e. the posterior distribution (Plummer et al. 2006). Various

alternatives to MCMC exist, which are mostly based on rejection sampling and importance

sampling, e.g. the adaptive rejection sampling from a log-concave envelope distribution, which

is effective for updating single random variables (Gilks and Wild 1992), or generalized

sequential particle filter methods for updating arbitrary static or dynamic systems (Chopin 2002;

Ching and Chen 2007). These methods are often combined with MCMC.

An increasingly popular approach to represent probabilistic models in engineering is the

Bayesian network (BN) framework, which allows decomposing joint probability distributions

into local conditional distributions. As its name suggests, the BN modeling framework is

particularly suitable for Bayesian updating. Under certain conditions, mechanical models can

be effectively represented in a BN with discrete or discretized random variables and efficient

algorithms are then available (Straub and Der Kiureghian 2010b). Examples of BN applied to

updating of mechanical models are presented e.g. in (Mahadevan et al. 2001; Straub 2009;

Straub and Der Kiureghian 2010a). The BN framework may also be combined with continuous

random variables, e.g. it is ideally suited for applications of MCMC algorithms based on Gibbs

sampling (Gilks et al. 1994). Unfortunately, the conditions for the effectiveness of BN, namely

conditional independence among random variables, are often not given in mechanical models.

In this paper, a novel approach to Bayesian updating of engineering models is presented, which

combines a rejection sampling strategy with structural reliability methods. We term the method

BUS (Bayesian Updating with Structural reliability methods). BUS is based on the approach

Bayesian updating with structural reliability methods 4/34

described in (Straub 2011) for Bayesian updating of the reliability of engineering systems. In

this paper, the approach is extended to computing the posterior distribution of uncertain model

parameters and model outputs in general. Key advantages of the BUS approach are its

simplicity and that it enables using the whole portfolio of structural reliability methods and

associated software for Bayesian updating. Here, we implement the BUS approach through

subset simulation (Au and Beck 2001), which can efficiently handle models with many random

variables. Three illustrative examples are included, which demonstrate the application of the

method to structural identification, Bayesian updating of material parameters in a statically

loaded structure, and updating of a random-field-based finite-element (FE) model.

Bayesian updating of mechanical models

Mechanical models consist of a set of equations and boundary conditions that describe the

geometry, material properties and loading conditions. In many applications, some of the

parameters of the model are uncertain. These parameters are modeled as random variables ,

characterized through their joint probability density function (PDF) . The mechanical

model itself is often uncertain, which can be reflected by introducing one or more additional

random variables into to represent model errors.

When new observations or data related to a mechanical model are available, these can be used

to learn the mechanical model. As an example, observations of deformations of a structure

under known loading provide information on the material properties of the structure. Since the

uncertainty in the mechanical model and its parameters is reflected through , learning from

the observations is tantamount to updating the distribution of . Bayes’ rule enables updating a

prior probability distribution with observations or data to a posterior probability

distribution :

(1)

Throughout this paper we use the convention

. The direct

evaluation of the -fold integral in Eq. (1) is not feasible in the general case, which has

motivated the introduction of MCMC and other sampling techniques for Bayesian analysis.

in Eq. (1) is the so-called likelihood function and describes observations and data (Fisher

1922). It is defined as

Bayesian updating with structural reliability methods 5/34

(2)

When observations are made, they often correspond to outcomes of the mechanical models.

Therefore, the likelihood function must include the mechanical models to relate the observation

to the model parameters . As an example, if deformations of a structure are measured, the

model predictions of these deformations for given values of are required. Let denote

such a model prediction. Furthermore, let denote the corresponding observed deformation

and let denote the deviation of the model prediction from the observation. This deviation is

due to measurement errors and model errors (those not modeled explicitly through ); it is

modelled through the PDF . The following relationship holds: . The

likelihood function describing this observation is therefore

(3)

Generally, one can distinguish two classes of observations (Madsen et al. 1985; Straub 2011):

(a) Observations providing equality information. These are measurements of continuous

quantities, for which equalities similar to can be formulated. The likelihood

function for these observations is

(4)

is the PDF of the measured quantity given . Typically, is defined by the PDF

of the observation error , which is often assumed to follow a Gaussian distribution. In case of

an additive observation error , is defined as in Eq. (3). In case of a multiplicative

observation error , the relationship between measurement and model prediction is

; solving for and inserting into , the likelihood function is obtained as

(5)

(b) Observations providing inequality information. These are observations that are

characterized by a finite probability of occurrence, e.g. observations of categorical values or

observations of system performances such as failure/survival, but also censored data.

Commonly, it is possible to formulate a model outcome such that the observation event

is defined through

Bayesian updating with structural reliability methods 6/34

(6)

A model outcome of this form is known in structural reliability as limit state function.

The corresponding likelihood function is then

(7)

is the indicator function. This likelihood thus takes on values 0 or 1.

In the general case, observations are available, of which are of the equality type and

are of the inequality type. Under the common assumption that the measurement/observations

are statistically independent given the model parameters , the combined likelihood of all

observations is

(8)

In case of statistically dependent observation errors , the combined likelihood must be

formulated as a function of the joint PDF of all . An example of statistically dependent

observation errors is included in the applications presented later in the paper.

Through Eq. (8), all observations are combined in the single expression . For the

computations of the posterior distribution presented in the remainder of the paper, it is irrelevant

whether the observations are of the equality type, the inequality type, or combinations thereof.

It should be clear from the preceding discussion that every computation of the likelihood

function requires up to model evaluations to determine . Luckily, for most

applications, one model call is sufficient to compute all , so the necessary number of model

evaluations is equal to the number of likelihood function calls. For advanced numerical models,

such as the geotechnical finite element model presented later in the numerical applications,

these evaluations are computationally costly. Our aim is therefore to perform Bayesian updating,

Eq. (1), with a minimum number of likelihood function calls.

Sampling algorithm

The goal is an efficient algorithm for sampling from the posterior distribution . If direct

evaluation of the integral in Eq. (1) is not feasible, the posterior PDF is known only up to a

proportionality constant, i.e.

Bayesian updating with structural reliability methods 7/34

(9)

Samples from can be generated through a simple rejection sampling algorithm, which is

introduced in the following.

Let be a standard uniform random variable in . Consider the augmented outcome space

and define the domain

(10)

where is a positive constant that ensures for all . The selection of this constant

is discussed in Annex A.

The posterior distribution of Eq. (1) can be written as

(11)

The validity of this result can be demonstrated as follows. The denominator in (11) corresponds

to a structural reliability problem with limit state function ,. It is equal to

the probability of being in under the prior PDF , i.e.

(12)

The numerator in (11) is equal to

(13)

Inserting (12) and (13) in (11), the original Bayesian updating formulation of Eq. (1) is obtained.

The posterior cumulative distribution function (CDF) is obtained by integrating Equation (11)

on both sides:

Bayesian updating with structural reliability methods 8/34

(14)

Generating samples from and samples from the standard uniform distribution,

, the following Monte Carlo approximation to the posterior CDF is obtained:

(15)

It follows that samples generated from and falling into the domain are distributed

according to the posterior . This leads to the following rejection sampling algorithm,

which generates samples of the posterior . Note that this algorithm is equivalent to a

classical rejection sampling where the prior distribution is applied as an envelope distribution

and the likelihood as a filter (Smith and Gelfand 1992).

Simple rejection sampling algorithm

1. .

2. Generate a sample from .

3. Generate a sample from the standard uniform distribution in [0,1].

4. If

a. Accept

b.

5. Stop if , else go to 2.

The rate of acceptance is equal to the denominator in Eq. (11),

(16)

For a fixed number of initial samples , the number of accepted samples is binomial distributed

with parameters and . To generate a fixed number of accepted samples , as in the

above algorithm, the number of samples that must be generated has the negative binomial

distribution:

(17)

Bayesian updating with structural reliability methods 9/34

The mean value of is . Consequently, the expected number of samples to

generate, and thus the expected number of evaluations of , is proportional to .

Let us consider a simple example to clarify the simple rejection sampling algorithm and its

limitations.

Illustration: Updating of a single random variable

Let be a random variable with standard normal prior, i.e.

. A

measurement of is made, resulting in a value of . The measurement is associated with an

additive error , which is normal distributed with zero mean and standard deviation .

The likelihood function is thus

. The parameter is chosen as

. For this example, an analytical solution can be obtained for

comparison: The posterior is the normal distribution with mean 1.6 and standard deviation

.

The domain , together with samples of and , is shown in Figure 1a. In Figure 1b, the

empirical frequency plot of the accepted samples is shown, together with the exact posterior

CDF . The sample mean and standard deviation of the accepted samples are and

; these values are close to the true solution.

In this example, 25 out of 200 samples are accepted. The exact mean acceptance rate can be

computed as

. Although not ideal, a mean acceptance rate of

would be acceptable for most problems. However, the acceptance rate decreases

quickly with increasing number of observations . If measurements are made of in the

above example, all with independent identically distributed (iid) observation errors , the

average acceptance rate is proportional to

(see Annex B).

Bayesian updating with structural reliability methods 10/34

Figure 1. Illustration of the simple rejection sampling algorithm: Updating a single random variable

with the observation , using 200 samples.(a) Joint samples of and drawn from the

prior distribution; accepted samples are indicated by a cross. The shaded area is the

observation/acceptance domain . (b) Resulting empirical frequency plot of the accepted samples,

together with the analytical solution.

Note that the acceptance rate is directly proportional to the constant of Eq. (10). Therefore,

should be selected as large as possible, while still ensuring that for all . In the above

example, it is chosen as , which is the optimal choice. However, in some real

applications can be difficult to evaluate. Strategies for selecting an optimal value of

in such cases are discussed in Annex A.

Bayesian updating with structural reliability methods (BUS)

The above rejection sampling algorithm quickly becomes inefficient with increasing number of

observations due to the large rejection rate, unless the posterior is close to the prior .

This problem is inherent to any direct rejection sampling algorithm (Gelman 2004; Bolstad

2011) and has motivated the development of MCMC techniques for simulating from the

posterior (Gilks et al. 1998). However, the advantage of the simple rejection sampling

Bayesian updating with structural reliability methods 11/34

algorithm over MCMC is that it is straightforward to implement and it is guaranteed to give

exact, uncorrelated samples of the posterior. Therefore, in the following a method is proposed

that maintains partly the advantages of the simple rejection sampling algorithm but has much

higher efficiency. This method, which we call BUS, deals with the inefficiency problem of the

rejection sampling by combining it with structural reliability methods.

As pointed out earlier, the integral in the denominator of Eq. (11) corresponds to a structural

reliability problem. In structural reliability, the domain describes the event of interest,

typically a failure event; in the context of the Bayesian updating of Eq. (11), it describes an

observation event, which we denote by . The integral, and therefore , is equal to the

acceptance rate of the simple rejection sampling algorithm, which decreases with the

square root of the number of observations. In many applications, will be too small for

Monte Carlo simulation to be efficient. What is needed are methods that allow to more

efficiently explore the domain and compute even when it is very small. Structural

reliability methods such as FORM/SORM or importance sampling methods have been

specifically developed to evaluate small probabilities (Ditlevsen and Madsen 1996; Rackwitz

2001). They approximate either the limit state function or the PDF of in the neighborhood of

the domain with the highest probability density, which in the context of Bayesian updating

corresponds to the region of the highest posterior probability density. These methods are

therefore ideally suited to enhance the efficiency of the rejection sampling algorithm.

When applying structural reliability methods, it is convenient to transform the problem from

the outcome space of the original random variables and to a space with independent

standard normal random variables . Since and are

independent, they can be transformed separately. The transformation from to and is

(18)

where standard normal CDF, and

(19)

one of the classical transformations used in structural reliability methods. Both the

Rosenblatt transformation (Hohenbichler and Rackwitz 1981) or the marginal transformation

based on the Nataf model (Der Kiureghian and Liu 1986) can be applied.

The domain describing the observation can now be transformed to an equivalent domain

in standard normal space:

Bayesian updating with structural reliability methods 12/34

(20)

For convenience, we define in terms of a function ,

(21)

so that . is a limit state function describing the observations in standard

normal space, as may be evident from the correspondence of to the definition of

the inequality observation in Eq. (6).

For illustration, Figure 2 shows the transformation of the observation domain and the samples

of shown in Figure 1a to the corresponding domain and samples in standard normal space.

In order to obtain samples from the posterior, the samples falling into the domain must be

transformed to the original space using Eq. (19).

Figure 2. Observation domain and samples of Figure 1a transformed to standard normal space.

A main advantage of BUS is that it can potentially be combined with a large number of available

structural reliability methods. It is not the intention of this paper to identify the most optimal

method, since this choice will depend on problem as well as on the preferences and experience

of the analyst. The strength of expressing the observations and data through limit state functions

is exactly the flexibility that it provides.

Among the available structural reliability methods, importance sampling (IS) methods are

suitable for generating (weighted) samples from the observation domain . Such IS methods

Bayesian updating with structural reliability methods 13/34

may be based on the design point (the Most Likely Failure Point MLFP) identified by means of

the first-order reliability method (FORM). Note that in the context of Bayesian updating, the

design point corresponds to the mode of the posterior distribution. Axis parallel IS based on the

design point (also known as line sampling) was applied in an example presented in (Straub

2011) for the updating of the failure probability; it would also be efficient for updating the

probability distribution of using BUS. Since the identification of the design point is an

optimization problem, alternative IS methods that do not require the design point may be

preferable. In addition to classical adaptive IS methods (e.g. Bucher 1988), newly proposed

approaches such as the one of (Kurtz and Song 2013) appear to be promising for the application

with BUS. When applying IS concepts, the resulting samples are weighted. To obtain

unweighted samples, a resampling scheme must be applied: In an additional step, independent

samples are drawn from a discrete probability distribution defined through the original samples

and their weights (Doucet et al. 2001).

In principle it is possible to use FORM (or SORM) in the context of the BUS approach. If

FORM was applied, the posterior distribution in -space would be approximated by a censored

standard multi-normal distribution; the censoring is defined through the hyperplane given by

, where is the design point and is the gradient vector of the

observation limit state of Eq. (21). However, the use of FORM or SORM is not advocated

without further investigations on its accuracy. As can be observed from Figure 2, the shape of

the observation domain differs from the failure domains typically encountered in structural

reliability.

An alternative to FORM/SORM and importance sampling methods is subset simulation (SuS)

proposed by (Au and Beck 2001), which is particularly efficient for structural reliability

problems where the number of random variables is large. For the considered implementation

in BUS, the SuS has the additional advantage that in its final step it directly produces samples

from the posterior distribution.

BUS algorithm based on subset simulation

In the following, we introduce an algorithm that uses SuS to generate samples in the observation

domain . It assumes that the observation event is transformed to -space by means of Eq.

(21). The algorithm is applied in the numerical examples. We restrict the presentation of SuS

to a summary of its main principles. For details on its implementation in the examples presented

later in the paper, the reader is referred to (Papaioannou et al. 2014).

Bayesian updating with structural reliability methods 14/34

SuS, originally proposed by Au and Beck (2001), evaluates the probability of an event

associated with a limit state function and here defined as .

SuS is based on expressing the event as the intersection of intermediate events that are

nested, i.e. it holds . The probability is then expressed as

(22)

where is the certain event. Thus the possibly small probability is expressed as the

product of larger conditional probabilities. The intermediate events ,, are defined as

, where . The values of can be chosen

adaptively, so that the estimates of the conditional probabilities correspond to a chosen value

. To this end, samples are simulated from the random vector conditional on each

intermediate domain . For each sample, the limit state function is evaluated and the

samples are ordered in increasing order of magnitude of their limit state function values. The

threshold is then set to the -percentile of the ordered samples. This procedure is repeated

until the maximum level is reached, for which . The samples conditional on the

certain event are obtained by crude Monte Carlo sampling. The samples conditional on the

events , for , are computed by simulating states of Markov chains through

MCMC starting from the samples conditional on for which . It is noted that the

seeds of the Markov chains always follow the target distribution and hence the applied MCMC

does not suffer a convergence (burn-in) problem (Zuev et al. 2012; Papaioannou et al. 2014).

The probability of can be approximated by:

(23)

where is the estimate of the conditional probability and is given by the ratio

of the number of samples for which over the total number of samples simulated

conditional on . The value of the intermediate probabilities and the number of samples

in each intermediate step are chosen by the analyst. Au and Beck (2001) suggested using

, whereas Zuev et al. (2012) showed that a choice of leads to similar

efficiency. should be selected large enough to give an accurate estimate of .

The SuS algorithm can be slightly modified for application to Bayesian updating, where

is not the main result. For Bayesian updating, one is interested in obtaining samples that fall

into the domain , i.e. samples conditional on . Therefore, we add a final step, which is the

Bayesian updating with structural reliability methods 15/34

generation of such samples. In this final step, samples are generated, where can be freely

chosen. The resulting algorithm is presented in the following.

SuS-based algorithm for BUS

Define: (number of samples in each intermediate step), (number of final samples),

(probability of intermediate subsets).

Sample from the original distribution:

1. Generate samples

, , from the -variate independent standard

normal distribution, .

2. Define the domain , wherein is chosen as the -percentile of the

samples

, .

3.

Sample from the conditional distributions:

4. Repeat while .

a.

b. Generate conditional samples , , from the -variate

independent standard normal distribution conditional on , ,

using a MCMC algorithm, e.g. (Au and Beck 2001) or (Papaioannou et al. 2014).

c. Define the domain , wherein is chosen as the -percentile of

the samples or 0, whichever is larger.

Sample from the posterior distribution:

5. Identify all samples from that are in the domain . Set equal to

the number of these samples

6. Generate conditional samples , , from the -variate

independent standard normal distribution conditional on , . This uses a

MCMC algorithm where the seeds are the samples identified in 5.

7. Transform the samples to the original space to obtain samples from the posterior

distribution:

, .

Estimate the acceptance probability:

8.

, following Eq. (23), with .

Bayesian updating with structural reliability methods 16/34

As pointed out earlier, the necessary computational effort is determined mainly by the number

of likelihood function calls. In the above algorithm, the first and each intermediate step require

likelihood function calls; the final step (sampling from the posterior) requires an additional

likelihood function calls. The total number of calls associated with the above algorithm

is thus equal to , where is the total number of steps. It is

, hence the computational effort is approximately proportional to .

Note that the number of likelihood function calls in the BUS approach is approximately equal

to the number of limit state function calls in the traditional use of SuS (or other structural

reliability methods), when evaluating the probability of an event with probability . For BUS,

the fact that accuracy in computing is typically not crucial can motivate a reduction in the

number of samples in each step as compared to the original SuS, for which values of

and were proposed by (Au and Beck 2001). However, we do not investigate such

further optimization in this paper.

Bayesian inference

Once samples from the posterior distribution are obtained, any property of this

distribution can be estimated using a MCS approach. When the are generated following

the SuS algorithm outlined above, it must be taken into account that the samples can be

correlated (this is in analogy to MCMC). While this does not affect the estimates below, it does

affect their accuracy.

The expected value of any function of is estimated as

(24)

Accordingly, an estimate of the distribution of is obtained as

(25)

Alternatively, kernel density approximations may be used for the posterior of or

(Turlach 1993).

Bayesian updating with structural reliability methods 17/34

If the interest is in computing the posterior probability of a failure event , defined through a

limit state function as , the corresponding function is

. A MCS estimate from the posterior samples following Eq. (24) is inefficient

when the posterior probability of failure is small. In such cases, which are common in structural

reliability problems, it is beneficial to follow the procedure in (Straub 2011). Thereby, the

intersection of with the observation event is computed directly by means of structural

reliability methods.

Illustrative applications

Three applications are presented. The first one demonstrates the applicability of BUS to

problems that are not globally identifiable. The second demonstrates its applicability to

problems for which a larger number of measurements with correlated errors are available. The

third example demonstrates the applicability of the method to FE-based analysis, including a

random field modeling of material (soil) properties.

Parameter identification in a two DoF system

This example is due to (Beck and Au 2002). The mechanical model, the prior distributions as

well as the data are taken from the original reference, so that the results can be compared.

The example is a two degree of freedom (DoF) system, whose uncertain spring coefficients are

to be determined based on measurements of the first two eigenfrequencies. The problem is not

globally identifiable, i.e. there are multiple combinations of values of the model parameters that

can well explain the measured eigenfrequencies. In (Beck and Au 2002), the problem was

solved through Bayesian updating using a MCMC approach. Here, we employ the proposed

BUS method implemented through the SuS-based algorithm.

A two-story frame-structure is modeled through a shear building model with two DoF (Figure

3). The story masses, which include the mass contributions from the columns, are taken as

deterministic values with kg and kg. The inter-story

stiffness values are modeled as and , where N/m is the

nominal value and are correction factors. Damping is not considered in the analysis.

Observations of the first two eigenfrequencies and are used to update the prior distribution

of to its posterior distribution. Following (Beck and Au 2002), the likelihood

function is

Bayesian updating with structural reliability methods 18/34

(26)

where

(27)

is a modal measure-of-fit function. is the th eigenfrequency predicted with the model

with parameters and is the corresponding measurement. are the means and

is the standard deviation of the prediction error. The measured eigenfrequencies are

and

.

Figure 3. Two DoF shear building model.

The prior probability distributions of are uncorrelated lognormal distributions with

modes 1.3 and 0.8 and standard deviations . The proportionality constant in the

formulation of the likelihood function, Eq. (26), is selected as 1 and the constant in the

observation limit state function is taken as 1. (The largest possible value of the likelihood

function occurs for , in which case it is . Therefore, with it is

for all .)

The samples obtained with the SuS-based rejection sampling algorithm are shown in Figure 4,

including the samples from the intermediate steps. The parameters of the algorithm were

selected as , and , without attempting any optimization. The number

of subset simulation steps is and the number of additional samples in the last step is

. It follows that the total number of samples, and hence the total number of likelihood

function evaluations, is 1849.

Bayesian updating with structural reliability methods 19/34

Figure 4. Bayesian updating of the correction factors . Samples from the prior

distribution, the intermediate subsets and the final posterior distribution, shown in the outcome space

of . In each step, 500 samples were generated, but some samples are coinciding.

In Figure 5, the resulting posterior marginal CDF of is shown for several randomly selected

runs of the algorithm. The bi-modal nature of the posterior distribution, which is already evident

from Figure 4, can be clearly observed also in the marginal CDF. The comparison with the

exact result reveals that the shape of the distribution around the two modes is well captured by

the samples. The probability of the two modes is estimated with less accuracy, as evident from

the scatter in the value of the CDF between 0.5 and 1.5. The same observation is made in (Beck

and Au 2002). This scatter could be reduced by increasing the number of samples; alternatively,

additional importance sampling evaluations may help to reduce this scatter. For most practical

applications the accuracy of the presented results would be sufficient. The more important result

is that the proposed method can successfully identify both modes of the posterior distribution.

Bayesian updating with structural reliability methods 20/34

Figure 5. Posterior CDF of obtained from 6 repeated runs of the proposed algorithm, together with

the exact result.

The results shown in Figure 4 and Figure 5 are consistent with the results given in (Beck and

Au 2002), where further details and discussion of the example can be found.

Bayesian updating of the flexibility of a cantilever beam

This example demonstrates the application of the proposed method to learning a random field

with measurements whose errors are correlated. The example has an analytical solution, which

allows a validation of the method.

We update the spatially variable flexibility of a cantilever beam based on measurements

of the beam deflections. The beam has length and is subjected to a deterministic point

load at the free end (Figure 6). The flexibility is defined as the reciprocal of

the bending rigidity of the beam, so that

, where is the Young’s modulus

at location and is the moment of inertia. We assume that the prior distribution of is

described by a homogeneous Gaussian random field with mean and

exponential auto-covariance function

, where

is the standard deviation and the correlation length.

Bayesian updating with structural reliability methods 21/34

Figure 6. Beam subjected to point load ; measured deflection.

The second derivative of the vertical deflection is a function of and the bending

moment :

(28)

Taking the double integral of the above, it is:

(29)

Using the prior information on , one can evaluate the prior mean and auto-covariance

function of . Moreover, because is Gaussian and is a linear function of ,

the prior distribution of will also be Gaussian. The mean of reads:

(30)

The auto-covariance function of is:

(31)

The above integral can be solved analytically, however the resulting expression is lengthy and

hence omitted here.

Bayesian updating with structural reliability methods 22/34

Measurements of the deflection are made at 50 points (0.1m, 0.2m, …, 5m) along the beam

using optic measurements. The measurements are subjected to additive errors, which are

described by a joint normal PDF with zero mean and covariance matrix , whose elements

are determined from the auto-covariance function

with

. The standard deviation of the measurement error is . The simulated

measurements are shown in Figure 6, together with the true (but in real applications unknown)

deflection of the beam.

The likelihood function describing these measurements is

(32)

where is a vector describing the flexibility at the 50

discretization points and is the vector of deflections at these 50 locations computed from

following Eq. (29).

Bayesian updating of the flexibility at the locations is performed

with BUS using the SuS-based algorithm with , and . The

constant is selected as the inverse of the likelihood function value at the MLE following

Annex A. The resulting number of subsets is

Estimates of the flexibility based on the deflection measurements are presented in Figure 7. The

results show the advantage of a Bayesian analysis over MLE for this problem. Due to the fact

that 50 (correlated) measurements are available for estimating 50 parameters, the problem is

ill-posed and MLE leads to overfitting. The Bayesian analysis with informative prior

distribution regularizes the problem and provides a good approximation to the true values.

Figure 7. Posterior credible interval of the flexibility computed with BUS, together with the true value

and the maximum likelihood estimate (MLE).

Bayesian updating with structural reliability methods 23/34

Figure 8 presents the deflection of the beam, comparing the true value, the measured value, the

MLE and the Bayesian estimate. Figure 8a illustrates that the measured values are very similar

in absolute terms. To enable an appraisal of the differences in the estimates, Figure 8b presents

the difference of the deflections from the ones computed with the prior mean of . The

Bayesian analysis provides a good estimate of the actual deflections, whereas the MLE leads to

an overfitting to the measurements in agreement with the results of Figure 7.

Since the prior distribution of is Gaussian and the measurement errors are additive and

also jointly Gaussian, an analytical solution of the posterior joint PDF of can be obtained

for validating the results obtained with BUS. The results in Figure 8 show that the posterior

credible intervals obtained with BUS coincide almost perfectly with those calculated

analytically. Further proof of the accuracy of BUS is available from Figure 9, which shows a

comparison of the analytically calculated posterior CDF of the deflection at the free end with

the BUS solution.

Figure 8. (a) Posterior credible interval of the deflection, together with the true value, the

measurements and the maximum likelihood estimate (MLE). (b) As in (a), but showing the difference

of the deflection to its prior mean.

Bayesian updating with structural reliability methods 24/34

Figure 9. Marginal CDF of the deflection at the end of the cantilever . Prior CDF, together

with the posterior CDF computed using the proposed method and the analytical solution.

Finite element model updating of a geotechnical construction

This example is based on previous reliability analysis and updating presented in (Papaioannou

and Straub 2012). We update the material properties of the soil surrounding a geotechnical site

based on a deformation measurement performed in situ.

The site consists of a deep trench with cantilever sheet piles in a homogeneous soil layer

of dense cohesionless sand with uncertain spatially varying mechanical properties (see Figure

10). The soil is modelled in 2D with plane-strain finite elements. For simplicity, neither

groundwater nor external loading is considered. Additionally, we take advantage of the

symmetry of the trench and model just half of the soil profile, although this implies an

approximation when randomness in the soil material is taken into account. The material model

used is an elasto-plastic model with a prismatic yield surface according to the Mohr-Coulomb

criterion and a non-associated plastic flow. The sheet pile is modelled using beam elements and

the interaction between the retaining structure and the surrounding soil is modelled using

nonlinear interface elements. The corresponding FE model is implemented in the SOFiSTiK

program (SOFiSTiK 2012). Further details on the mechanical model and the simulation of the

excavation process can be found in (Papaioannou and Straub 2012).

Bayesian updating with structural reliability methods 25/34

Figure 10. Sheet pile wall in sand.

Homogeneous non-Gaussian random fields describe the prior distributions of the uncertain

material properties: Young’s modulus , friction angle and specific weight . The joint

distribution at each pair of locations is modelled by the Nataf distribution (Kiureghian and Liu

1986) with marginal distributions according to Table 1. The auto-correlation coefficient

function is given by a separable exponential model

, where ,

are the absolute distances in the (horizontal) and (vertical) directions. The correlation

lengths are and for all uncertain soil material properties. Cross-correlation

between the different material properties is not included. The random fields are discretized by

the midpoint method (Der Kiureghian and Ke 1988) using a stochastic mesh, consisting of

deterministic FE patches. The stochastic discretization resulted in a total of

basic random variables gathered in a vector . In Figure 11, the stochastic and deterministic FE

meshes are shown. Figure 11c shows the deformed configuration at the final excavation stage

computed with the mean values of the random fields.

Table 1. Prior marginal distributions of the material properties of the soil.

Parameter Distribution Mean COV

Specific weight Normal 19.0 5%

Young’s modulus [MPa] Lognormal 125.0 25%

Poisson’s ratio - 0.35 -

Friction angle Beta(0.0,45.0) 35.0 10%

Cohesion [MPa] - 0.0 -

Dilatancy angle - 5.0 -

Bayesian updating with structural reliability methods 26/34

Figure 11. (a) Stochastic and (b) deterministic finite element mesh of the geotechnical site, shown for

the situation prior to the excavation; (c) deformed configuration at full excavation for the mean values

of the material properties.

We assume that a measurement of the horizontal displacement at the top of the trench

is made at full excavation. The measurement is subjected to an additive error , which

is described by a normal PDF with zero mean and standard deviation =. The

likelihood function describing the measurement is

(33)

where describes the material properties at the midpoints of the stochastic elements and

is the displacement evaluated by the FE program. Bayesian updating of the vector is

performed with BUS using the SuS-based algorithm with , and

. The constant is selected as , which satisfies the condition .

The prior mean of is , which indicates that the prior model underestimates the

measured tip displacement. Figure 12 and Figure 13 show the posterior mean of the Young’s

modulus and friction angle , respectively. The posterior means of the elements in the

vicinity of the trench are smaller than the prior mean, which reflects the effect of the measured

displacement. The effect is local, since the values of the stiffness and strength of the soil farther

away from the trench have limited influence on the deformation at the location of the

measurement. Moreover, the results show the influence of the auto-correlation of the prior

distribution. The change of the posterior means is steeper in the vertical than in the horizontal

direction, which is due to the fact that the prior correlation length in the horizontal direction is

larger than in the vertical direction. The low values of the Young’s modulus observed in the

bottom right of Figure 12 cannot be explained by the measurement and hence are attributed to

sampling error.

Bayesian updating with structural reliability methods 27/34

Figure 12. Posterior mean of the Young’s modulus of the soil.

Figure 13. Posterior mean of the friction angle of the soil.

Discussion

Bayesian updating using structural reliability methods (BUS) has been developed for learning

and calibrating engineering and computer models. By interpreting Bayesian updating as a

structural reliability problem, the existing suite of methods for solving such problems is

available, including importance sampling, subset simulation and first- and second-order

reliability methods (FORM/SORM). For analysts with experience in these methods, it is

straightforward to implement the BUS approach.

The presented implementation of BUS through the subset simulation (SuS) algorithm is ready-

to-use for many practically relevant applications. The presented numerical applications as well

as further numerical investigations not reported in this paper showed that the SuS-based

algorithm works robustly and efficiently for a broad range of problems. It also has the advantage

of being applicable in high dimensions – at least theoretically its performance does not decrease

with increasing number of random variables. It is pointed out that the algorithm uses MCMC

Bayesian updating with structural reliability methods 28/34

to generate the conditional samples required by SuS. Since the initial samples of these Markov

chains are already from the target distribution, it is ensured that all samples follow this target

distribution and convergence of the chain is not an issue. However, the MCMC procedure does

introduce a correlation among the resulting samples, which must be taken into account when

evaluating the posterior statistics.

BUS has originally been developed for problems with an informative prior, i.e. when relevant

information on the uncertain variables is available a-priori. This is a common situation when

learning mechanical models, where significant prior knowledge on uncertain parameters is

often available. But BUS is a viable alternative to existing methods also for problems with

weakly or non-informative prior distributions (e.g. in parameter identification), as demonstrated

by application 1 in this paper.

In this work, no attempt was made to optimize the structural reliability methods for their use in

BUS. We are currently investigating further optimizations of SuS targeted towards its

implementation in BUS, including an adaptive choice of the constant . In addition, we believe

that there is a significant potential in identifying alternative structural reliability methods that

can be more efficient than SuS in combination with BUS for specific types of applications. As

an example, the use of importance sampling methods around the design point (which in BUS

corresponds to the mode of the posterior distribution) appears promising, but has not been

considered here.

In the applications presented in this paper, the number of model evaluations required for

Bayesian updating is on the order of , which is similar to existing state-of-the-art

methods such as transitional MCMC proposed in (Ching and Chen 2007) or the hybrid MCMC

approach of (Cheung and Beck 2009). While the presented implementation of BUS certainly

offers room for a further enhancement of the computational efficiency, we do not believe that

this number can be significantly reduced with any purely sampling-based approach. For

expensive computational models, the BUS approach can be combined with surrogate models

(response surfaces) to further reduce the number of model evaluations. Experience with

adaptive response surfaces available in the structural reliability community may be used for this

purpose.

In many instances, one is interested in updating the reliability of a mechanical system, or more

generally in updating the probability of a rare event with observations. Unlike other methods

presented in the literature, BUS does not require that one first updates the joint probability

distribution of the relevant variables and then computes the updated probability of based

Bayesian updating with structural reliability methods 29/34

on an approximation of the posterior PDF of . With BUS, the observation is represented by

the event and the posterior probability of is equal to , see

(Straub 2011). Therefore, Bayesian updating reduces to solving two structural reliability

problems.

Conclusion

A novel approach to Bayesian updating of mechanical and general engineering models was

proposed, termed BUS. It is based on interpreting the Bayesian updating problem as a structural

reliability problem. This enables one to use the whole portfolio of existing structural reliability

methods for performing Bayesian updating. In this paper, subset simulation was used to obtain

samples of the posterior distribution. Three application examples were included to demonstrate

the versatility and efficiency of BUS.

Bayesian updating with structural reliability methods 30/34

Annex A: How to select the constant defining the observation domain?

BUS requires the selection of the constant in Eq. (10). On the one hand, it is required that

for any . On the other hand, the acceptance rate is directly proportional to ; i.e.

the larger , the larger , which is beneficial for the MCS-based rejection

sampling as well as for BUS in combination with subset simulation. It follows that an optimal

choice of is

(34)

In some instances, is directly available. In case of pure inequality observations, it is

and it follows that . In case of a single measurement with error ,

is equal to maximum of the PDF of , which is readily available. However, it is not always

straightforward to evaluate Eq. (34) when there are several observations with corresponding

likelihoods . occurs at the location of the maximum likelihood, so

if it is possible to determine the maximum likelihood estimator using an optimization

algorithm, it is . In case is not available, an alternative suboptimal

choice is

(35)

For large , i.e. when many individual observations are available, this choice is inefficient. In

these situations, one can get an estimate of the statistical properties of to obtain a more

efficient value of , as demonstrated through an example in the following.

Consider a case where all measurements have random errors that are independent

identically distributed (iid) following a normal distribution with mean zero and standard

deviation . The likelihood function is

(36)

Bayesian updating with structural reliability methods 31/34

Let us assume that the model is perfect if the true parameters are used, i.e. are the

true values of the measured quantities. In this case, the actual measurements are

, i.e. they are normal iid random variables with means and standard

deviation . The maximum of the log-likelihood is then approximately

(37)

where are independent standard normal random variables. The approximation becomes

better with increasing . It follows from Eq. (37) that {} is approximately chi-

squared distributed with degrees of freedom. Therefore, the distribution of the maximum

likelihood is independent of the observation error and depends only on the number of

independent measurements. One can thus select a value of , for which it holds with

given probability , as

(38)

where

is the inverse CDF of the chi-squared distribution with degrees of freedom. E.g.

if the number of independent measurements is , a value will ensure that

for all with probability .

In reality, the model will not be perfect, which will lead to additional deviations of the model

prediction from the observations. Therefore, the maximal log-likelihood will be lower than

predicted by Eq. (37), and the probability that is too large will be smaller than .

Bayesian updating with structural reliability methods 32/34

Annex B: Probability of accepting samples in the simple rejection

sampling algorithm

To illustrate the dependence of the average acceptance rate of samples on the number of

observations, we consider measurements of the quantity , all with iid observation errors.

This is in analogy to the illustration of the simple rejection sampling algorithm provided in the

paper.

The likelihood function with observations , , with iid normal additive error

with zero mean is:

(39)

The measurement values can be written as , where is the true

(but unknown) value of , is the observation error and is the distance between

and the unknown true value of . Inserting this relationship in (39), setting the proportionality

constant to 1 and taking logarithms, one obtains

(40)

The expected value with respect to the observation errors is

(41)

The first-order approximation of the expected value of the likelihood is

(42)

The width of this likelihood function is proportional to . It follows that the expected

number of samples falling into this domain, and consequently , is also proportional to

.

Bayesian updating with structural reliability methods 33/34

References

Au, S.-K. and J. L. Beck (2001). "Estimation of small failure probabilities in high dimensions by subset

simulation." Probabilistic Engineering Mechanics 16(4): 263-277.

Beck, J. and L. Katafygiotis (1998). "Updating Models and Their Uncertainties. I: Bayesian Statistical

Framework." Journal of Engineering Mechanics 124(4): 455-461.

Beck, J. L. and S. K. Au (2002). "Bayesian updating of structural models and reliability using Markov chain Monte

Carlo simulation." Journal of Engineering Mechanics-Asce 128(4): 380-391.

Benjamin, J. and C. A. Cornell (1970). Probability, Statistics, and Decision for Civil Engineers, McGraw-Hill,

New York.

Beven, K. (2008). Environmental modelling: an uncertain future?, Taylor & Francis.

Bolstad, W. M. (2011). Understanding computational Bayesian statistics, Wiley.

Bucher, C. G. (1988). "Adaptive sampling — an iterative fast Monte Carlo procedure." Structural Safety 5(2):

119-126.

Cheung, S. H. and J. L. Beck (2009). "Bayesian Model Updating Using Hybrid Monte Carlo Simulation with

Application to Structural Dynamic Models with Many Uncertain Parameters." Journal of Engineering

Mechanics-Asce 135(4): 243-255.

Ching, J. and Y. Chen (2007). "Transitional Markov Chain Monte Carlo Method for Bayesian Model Updating,

Model Class Selection, and Model Averaging." Journal of Engineering Mechanics 133(7): 816-832.

Chopin, N. (2002). "A sequential particle filter method for static models." Biometrika 89(3): 539-552.

Der Kiureghian, A. and J.-B. Ke (1988). "The stochastic finite element method in structural reliability."

Probabilistic Engineering Mechanics 3(2): 83-91.

Der Kiureghian, A. and P.-L. Liu (1986). "Structural reliability under incomplete probability information." Journal

of Engineering Mechanics 112(1): 85-104.

Ditlevsen, O. and H. O. Madsen (1996). Structural reliability methods. Chichester [u.a.], Wiley.

Doucet, A., N. De Freitas and N. Gordon (2001). Sequential Monte Carlo methods in practice, Springer New York.

Fisher, R. A. (1922). "On the Mathematical Foundations of Theoretical Statistics." Philosophical Transactions of

the Royal Society of London. Series A 222(594-604): 309-368.

Gelman, A. (2004). Bayesian data analysis. Boca Raton, Fla., Chapman & Hall/CRC.

Gilks, W. R., S. Richardson and D. J. Spiegelhalter (1998). Markov chain Monte Carlo in practice. Boca Raton,

Fla., Chapman & Hall.

Gilks, W. R., A. Thomas and D. J. Spiegelhalter (1994). "A language and program for complex Bayesian

modelling." The Statistician: 169-177.

Gilks, W. R. and P. Wild (1992). "Adaptive rejection sampling for Gibbs sampling." Applied Statistics 41(2): 337-

348.

Hohenbichler, M. and R. Rackwitz (1981). "Non-Normal Dependent Vectors in Structural Safety." Journal of the

Engineering Mechanics Division-Asce 107(6): 1227-1238.

Kavetski, D., G. Kuczera and S. W. Franks (2006). "Bayesian analysis of input uncertainty in hydrological

modeling: 1. Theory." Water Resources Research 42(3): n/a-n/a.

Kennedy, M. C. and A. O'Hagan (2001). "Bayesian calibration of computer models." Journal of the Royal

Statistical Society: Series B (Statistical Methodology) 63(3): 425-464.

Kiureghian, A. D. and P. L. Liu (1986). "Structural Reliability under Incomplete Probability Information." Journal

of Engineering Mechanics-Asce 112(1): 85-104.

Kurtz, N. and J. Song (2013). "Cross-entropy-based adaptive importance sampling using Gaussian mixture."

Structural Safety 42(0): 35-44.

Bayesian updating with structural reliability methods 34/34

Madsen, H. O., S. Krenk and N. C. Lind (1985). Methods of structural safety. Englewood Cliffs, NJ, Prentice-

Hall.

Mahadevan, S., R. Zhang and N. Smith (2001). "Bayesian networks for system reliability reassessment." Structural

Safety 23(3): 231-251.

Natke, H. G. (1988). "Updating computational models in the frequency domain based on measured data: a survey."

Probabilistic Engineering Mechanics 3(1): 28-35.

Papadimitriou, C., J. L. Beck and L. S. Katafygiotis (2001). "Updating robust reliability using structural test data."

Probabilistic Engineering Mechanics 16(2): 103-113.

Papaioannou, I., W. Betz, K. Zwirglmaier and D. Straub (2014). "MCMC algorithms for subset simulation."

Manuscript, Engineering Risk Analysis Group, Technische Universität München, Germany.

Papaioannou, I. and D. Straub (2012). "Reliability updating in geotechnical engineering including spatial

variability of soil." Computers and Geotechnics 42: 44-51.

Plummer, M., N. Best, K. Cowles and K. Vines (2006). "CODA: Convergence diagnosis and output analysis for

MCMC." R news 6(1): 7-11.

Rackwitz, R. (2001). "Reliability analysis - a review and some perspectives." Structural Safety 23(4): 365-395.

Smith, A. F. M. and A. E. Gelfand (1992). "Bayesian Statistics without Tears: A Sampling-Resampling

Perspective." The American Statistician 46(2): 84-88.

SOFiSTiK (2012). SOFiSTiK analysis programs. Version 2012. Oberschleissheim, SOFiSTiK AG.

Straub, D. (2009). "Stochastic Modeling of Deterioration Processes through Dynamic Bayesian Networks."

Journal of Engineering Mechanics-Asce 135(10): 1089-1099.

Straub, D. (2011). "Reliability updating with equality information." Probabilistic Engineering Mechanics 26(2):

254-258.

Straub, D. and A. Der Kiureghian (2010a). "Bayesian Network Enhanced with Structural Reliability Methods:

Application." Journal of Engineering Mechanics - ASCE 136(10): 1259-1270.

Straub, D. and A. Der Kiureghian (2010b). "Bayesian Network Enhanced with Structural Reliability Methods:

Methodology." Journal of Engineering Mechanics-ASCE 136(10): 1248-1258.

Straub, D. and M. H. Faber (2005). "Risk based inspection planning for structural systems." Structural Safety

27(4): 335-355.

Sundar, V. S. and C. S. Manohar (2013). "Updating reliability models of statically loaded instrumented structures."

Structural Safety 40(0): 21-30.

Tang, W. (1973). "Probabilistic updating of flaw information." Journal of Testing and Evaluation 1: 459-467.

Turlach, B. A. (1993). Bandwidth selection in kernel density estimation: A review. Discussion Paper 9307, Institut

für Statistik und Ökonometrie, Humboldt-Universität zu Berlin.

Yang, J. and W. Trapp (1974). "Reliability Analysis of Aircraft Structures under Random Loading and Periodic

Inspection." Aiaa Journal 12: 1623-1630.

Zuev, K. M., J. L. Beck, S.-K. Au and L. S. Katafygiotis (2012). "Bayesian post-processor and other enhancements

of Subset Simulation for estimating failure probabilities in high dimensions." Computers & Structures

92: 283-296.