Content uploaded by Martin Käser

Author content

All content in this area was uploaded by Martin Käser on Mar 05, 2022

Content may be subject to copyright.

UNCERTAINTY QUANTIFICATION FOR SEISMIC RISK

ASSESSMENT USING LATIN HYPERCUBE SAMPLING AND QUASI

MONTE CARLO SIMULATION

Christoph SCHEINGRABER

1

, Martin KÄSER

2

ABSTRACT

In the insurance industry Seismic Risk Assessment is commonly used for modeling loss to a spatially

distributed portfolio. Best practice not only involves the computation of expected loss, but also requires

treatment of the uncertainty of all components of the modeling chain. Because the dimensionality is high,

this is typically performed with a Monte Carlo simulation of a large number of scenario realizations. In

this study, we first compare the computational efficiency of uncorrelated pseudo-random sampling to

variance reduction techniques for scenario loss uncertainty treatment. We observe that Latin Hypercube

sampling as well as Quasi Monte Carlo simulation using low-discrepancy sequences can improve the error

convergence from to in many cases. We then perform a global sensitivity analysis to

quantify the contribution of different modeling parameters and their uncertainties to the overall loss

uncertainty. To this end we use three scenarios in Indonesia and explore uncertainty in the geographical

distribution of portfolio items, structural properties such as building height and quality, as well as ground

motion and damage models. We find that depending on the portfolio a significant fraction of the output

variance can be attributed to uncertain factors in the exposure and vulnerability models, revealing the

importance of their thorough treatment in seismic risk analysis.

Keywords: Seismic Risk Analysis; Uncertainty Quantification; LHC; QMC; Global Sensitivity Analysis.

1. INTRODUCTION

Probabilistic Seismic Risk Assessment (PSRA), building upon Probabilistic Seismic Hazard

Assessment (PSHA; Cornell 1968, SSHAC 1997), is widely used in the insurance industry to model

the frequency and severity of losses to a geographically distributed portfolio from the occurrence of

earthquake events. In this context it is not sufficient to only compute expected loss, but the treatment

of uncertainty in all parts of the modeling chain is of immense importance (Crowley, 2014). For

practical purposes, model uncertainty can be categorized into being either epistemic or aleatory. The

term epistemic describes uncertainty due to limited knowledge or data and is commonly treated with a

logic tree combining multiple alternative models. Weights of the logic tree represent the degree of

belief in the correctness of a branch. Aleatory refers to variability inherent to natural processes which

is assumed to be irreducible and usually captured with a probability distribution.

Once uncertainty treatment is integrated, result uncertainty can be visualized and communicated to

decision makers. Moreover, the contribution of individual factors to the output variation can be

quantified using sensitivity analysis (SA). This allows to e.g. identify areas where additional research

or effort to reduce the associated uncertainty might be worthwhile. In contrast to local methods which

investigate the impact of incremental input perturbations at a base case, global SA aims at exploring

the entire space of uncertain input factors and thereby allows to take factor interactions into account.

Global SA is a computationally demanding technique, because in general a high-dimensional input

space needs to be sampled. This motivates our investigation of the efficiency of various sampling

1

PhD student, Department of Earth and Environmental Sciences, Ludwig-Maximilians-Universität,

Munich, Germany, scheingraber@geophysik.uni-muenchen.de.

2

Senior NatCat Risk Analyst, Corporate Underwriting, Munich Re, Munich, Germany,

mkaeser@munichre.com.

2

schemes in the first part of this paper before we perform a global SA for seismic risk analysis using

three loss scenarios in Indonesia in the second part.

2. METHODOLOGY

2.1 Modeling Framework and Uncertainty Treatment

2.1.1 Ground Motion Model

Ground Motion Prediction Equations (GMPEs) are used to model the distribution of ground motion at

a site given the occurrence of an earthquake event of magnitude . Ground motion levels are

expressed by intensity measures such as Peak Ground Acceleration, Peak Ground Velocity, or Pseudo-

Spectral Acceleration at a given frequency. A simplified form for expected ground motion is generally

given by

(1)

with coefficients and .

Most GMPEs capture the associated aleatory variability using a log-normal probability distribution.

The standard deviation of this distribution is part of the GMPE. In this study, we use five different

GMPEs (see Section 2.3) and sample the total ground motion residual.

2.1.2 Portfolio Location Uncertainty

In the context of Seismic Risk Assessment for insurance purposes, a wide range of portfolio

uncertainty exists. For many portfolios, risk items are only known to be located within some

administrative zone but exact coordinates are unknown. This can be caused by inaccurate geocoding

as well as reduced information accuracy between brokers and reinsurance companies or risk modelers.

We treat portfolio location uncertainty in a stochastic manner by sampling risk locations within their

respective administrative zone on a weighted irregular grid, which acts as a proxy for insured exposure

density so that e.g. residential buildings are preferentially located in areas of high population (see

Figure 1).

2.1.3 Vulnerability Function and Damage Uncertainty

In addition, it is common that building properties such as the number of stories as well as building age

and the related construction quality – due to updated building codes – remain unknown to the modeler.

For this study, we assume that building height and construction quality are unknown for all buildings.

The associated epistemic uncertainty is treated stochastically with a logic tree approach. To model

damage given a ground motion level at a site we use a zero-one-inflated Kumaraswamy distribution.

This is a mixture distribution of a Bernoulli distribution on and a Kumaraswamy distribution in

the open interval , which allows the use of discrete probability masses and to denote the

likelihood that a building suffers no damage or is completely destroyed, respectively. The remaining

probability mass is then used to scale the Kumaraswamy distribution to model partial

damage.

We employ the inverse transform method to sample building damage. Conveniently, the inverse

distribution function of the zero-one-inflated Kumaraswamy distribution has a closed-form expression.

2.2 Sampling Strategies and Error Estimation

Because a large number of model evaluations is necessary for sensitivity analysis, it is worth to

explore the efficiency of different sampling strategies for our study.

3

Figure 1. Sunda straight with the urban areas of Palembang and Jakarta. Transparent black markers depict the

vertices of the weighted irregular grid employed in this study. Color represents population density (residents per

km²; Gaughan et al. 2015), which we use as insurance density proxy.

2.2.1 Simple Monte Carlo

Stochastic sampling was first used on an electronic computer (ENIAC) during the Manhattan Project

at Los Alamos Laboratory, where the still widely used name Monte Carlo (MC) was also coined. The

theory was further developed and first published by Metropolis and Ulam (1949). Simple MC is based

on uniform random sampling of the domain and can be used for numerical integration instead of

deterministic quadrature rules. We create uncorrelated pseudo-random numbers using the Mersenne

Twister pseudo-random number generator (Matsumoto and Nishimura 1998). Given a sample of size

, the expectation of some function , for example mean scenario loss or values along a

probabilistic loss curve, is given by the unbiased estimator

(2)

Simple MC has a slow error convergence of . However, in contrast to deterministic

quadrature schemes such as the trapezoidal rule, the convergence order of simple MC is independent

of the number of parameters and only depends on the variance of the estimate

. For this reason, the

method is well suited for high-dimensional integrals such as seismic risk assessment with a large

portfolio size.

4

2.2.2 Latin Hypercube Sampling

Uncorrelated random sampling points tend to cluster, which is not ideal because there is little

information gain when sampling a point which is close to previous ones. Latin Hypercube

Sampling (LHS; McKay et al. 1979) is a variance reduction technique which aims to improve this by

stratifying the domain along each dimension. With a sample size of , for each dimension strata are

created such that every projection to one of the dimensions is itself a stratified sample with strata.

This can result in better asymptotic error convergence, particularly when the function is additive or

dominated by one parameter (Owen 1994). While convergence is often observed in practice,

theoretically this order could so far only be shown for special cases.

A problem with LHS is that while the domain is stratified in each separate dimension,

multidimensional combinations are not. This can be enhanced to some extent by additional design

criterions. We use the maximin distance criterion (Johnson et al. 1990), which maximizes the

minimum distance between two points. While LHS with sample size never performs worse than

simple MC with sample size (Owen 1997), the advantage of LHS can decrease for high-

dimensional problems. Thus the performance of LHS for our study needs to be evaluated.

2.2.3 Quasi Monte Carlo

With Quasi Monte Carlo (QMC) methods, deterministic low-discrepancy sequences are used instead

of quasi-random numbers to generate sampling points. Low-discrepancy sequences are designed to

avoid previous points and fill space evenly. In this study, we use the sequence introduced by Sobol

(1967). The Koksma-Hlawka inequality states that the QMC integration error of a function in

the -dimensional unit cube is bounded by where is the variation in

the Hardy-Krause sense which is finite if the integrand is smooth (Moskowitz and Caflisch 1996).

is the discrepancy of the sequence which is for large , although it can be worse for

intermediate (Morokoff and Caflisch 1994). This can potentially be improved with randomized

QMC, where a deterministic sequence is scrambled randomly. We use scrambling as described in

Owen 1997 and Matoušek 1998.

In practice, many MC simulations involve decisions or functions that are not smooth, like the

epistemic uncertainties and the zero-one-inflated loss distribution in this study. This motivates our

investigation of empirical error convergence.

2.2.4 Estimation of Standard Error and Confidence Intervals

We use repeated simulation for MC standard error estimation of the different sampling schemes.

Denoting a set of estimations of a statistic obtained from repeated simulations by

and the

variance by , then the standard error is given by

(3)

In this study, we normalize the standard error of the mean by the sample mean to obtain relative

standard error . For the estimation of confidence intervals of , we use bootstrapping. This

method assumes that the original sample holds all information about the underlying population, and

can be used to estimate the sampling distribution of

by resampling with replacement. Specifically,

we employ the bias-corrected accelerated percentile method (Efron and Tibshirani 1986).

2.3 Sensitivity Analysis

Seismic Risk Assessment relies on many uncertain parameters. Awareness of model uncertainties and

knowledge of the extent to which certain factors drive output uncertainty under specific circumstances

is important for risk modelers as well as end users. With sensitivity analysis (SA), it is possible to

quantify the influence of uncertain model input factors. Regulatory documents and official guidelines

5

of the European Commission and the United States Environmental Protection Agency recommend the

use of SA and stress the importance to consider factor interactions (Saltelli et al. 2010).

Local SA methods use first-order partial derivatives

to evaluate the sensitivity of model

output against input at a predefined base case of the input space. If the modeling code itself does

not return derivatives, they can e.g. be estimated with finite differences. Another very powerful

approach is algorithmic differentiation, which is the automated differentiation of an entire model

source code via application of the chain rule. While local methods are relatively cheap, they give only

limited insight into the sensitivity of a model with respect to a variable, because they only provide

valid information close to the base case where non-linear response can be neglected. They provide no

information about other regions of the input space. By contrast, global SA methods explore the entire

input space. This allows to quantify the overall sensitivity of the model output with respect to input

factors as well as interactions between factors.

For the global SA in this study, we use the variance decomposition introduced by Sobol (2001) based

on his Analysis of Variances (ANOVA) decomposition. The total variance of a scalar model

output dependent on a model input vector can be decomposed into components

(4)

where denotes the number of input factors, a first-order variance term dependent only on the th

input factor and a second-order term dependent on the th and th input. is the highest-order

variance term dependent on all input factors. Higher-order terms represent variance that cannot be

explained by lower-order terms, but is caused by some interaction of the involved factors. For

example, the variance in cannot be expressed by .

Sobol sensitivity indices express the fraction of the total variance due to a subset of the variance

components. In this study, we use the first order (or main effect) indices which quantify the fraction of

the variance caused by an input factor to the total output variance without interactions over the whole

input space:

(5)

denotes the th input factor and is the partial variance taken over factor . denotes all input

factors but the th, which can be thought of as the “non- direction” (Saltelli et al. 2010). is the

conditional expectation taken over all factors but , which means that variations in other parameters

are “averaged” and the variance is taken over these averages.

In addition, we use the total effect indices introduced by Homma and Saltelli (1996) to quantify the

contribution of the first order effect of the th input factor together with all higher-order interactions

with other factors to the total output variance:

(6)

is called the total effect index of the th input factor. Note that because individual interaction

components of the variance decomposition are reused for the computation of several total effect

indices (of all input factors involved in this interaction), the sum of all total effect indices exceeds 1

unless the model itself is purely additive. Because they are normalized by their variances, the exact

values of and are in the interval . Estimating and is usually performed via MC

simulation, which can be computationally demanding because convergence of the indices is often only

achieved after a very large number of model evaluations (Sarrazin et al. 2016). Numerous studies are

devoted to deriving efficient sampling designs for the joint estimation of main and total effects. In this

study, the design proposed by Jansen (1999) is employed.

6

2.4 Earthquake Scenarios and Portfolios

We study scenario loss variations based on two hypothetical earthquake events and several synthetic

portfolios in Indonesia. For this purpose we use a proprietary seismic risk assessment framework

developed by MunichRe running on MATLAB R2016B.

We use three “portfolio sets”, each is only known to be distributed in a different administrative zone

corresponding to an Indonesian province. Each portfolio set consists of several portfolios with a

varying number of buildings (1, 100, 5000 and 10000). For simplicity we use a total sum insured (TSI)

of and a flat value distribution for all portfolios, i.e. losses can be interpreted directly as

percentage of the TSI and the TSI is distributed equally among all risk items. For each model

evaluation, locations, heights and qualities are sampled independently for all buildings, but we assume

a fixed construction type (reinforced concrete with unreinforced masonry infill). To compute mean

damage ratios, our model then uses customized vulnerability functions reflecting different structural

performance due to varying building heights and construction qualities. On top of this, we sample

either the ground motion residuals or the damage residuals. Note that we do not sample ground motion

and damage residuals jointly, because in the reinsurance industry damage distributions are fitted to

include variation due to aleatory ground motion uncertainty. It might be possible to correct for this

effect in the future when more detailed loss and ground motion data become available, but currently

this approach is common practice. The aleatory ground motion uncertainty model is part of each

GMPE, while the parameters of the zero-one-inflated Kumaraswamy distribution used to treat aleatory

damage uncertainty (see Section 2.1.3) are part of the aforementioned risk assessment framework and

depend on mean damage ratios as well as building heights and qualities.

For the SA, we use the SAFE toolbox (Pianosi et al. 2015), which provides workflows for several SA

methods. For a variance-based global SA as performed in this study, SAFE provides functions to

approximate and visualize main and total effect indices.

Figure 2. Footprints of expected ground motion of two hypothetical scenarios employed for this study. Color and

isolines denote Peak Ground Acceleration (PGA) in . Figure 2a (left) shows the event on the

Sumatra subduction fault near the urban areas of Palembang and Jakarta using the GMPE by Zhao et al. (2006).

The provinces of Sumatera Selatan on the island of Sumatra and Daerah Khusus Ibukota Jakarta on the island

of Java in which we sample location uncertainty are outlined by their boundaries in blue color. Figure 2b (right)

shows the event on a northern segment of the Sumatra Fault Zone near the city of Medan using the

GMPE by Chiou and Youngs (2008). The province Sumatera Utara is outlined by its boundary in blue color.

7

2.4.1 Southern Sumatra Subduction Fault Event

The first scenario is a hypothetical event on the Sumatra subduction fault near the urban areas

of Palembang and Jakarta on the islands of Sumatra and Java, respectively. For this event, we use a

three-dimensional representation of the subduction zone based on the Slab 1.0 model (Hayes et al.

2012) and sample from two GMPEs with equal weights: Zhao et al. (2006), and Youngs et al. (1997).

We use two sets of portfolios: the first, hereafter labeled “Palembang portfolio set”, is only known to

be distributed within the province Sumatera Selatan on Sumatra; the second, hereafter labeled “Jakarta

portfolio set”, in the province Daerah Khusus Ibukota Jakarta on Java. Portfolio locations are sampled

onto the weighted irregular grid inside their respective zones (see Section 2.1.2 and Figure 1).

Figure 2a shows a footprint of the event obtained using the GMPE by Zhao et al. (2006) and the

outline of the two administrative zones.

2.4.2 Northern Sumatra Fault Zone Event

The second scenario is a hypothetical event on a northern segment of the Sumatra fault zone

near the urban area of Medan. For this event, we sample from three different GMPEs with equal

weights: Boore and Atkinson (2008), Campbell and Bozorgnia (2008), and Chiou and Youngs (2008).

We use one portfolio set for this event, hereafter referred to as “Medan portfolio set”, for which risk

items are only known to be located somewhere in the administrative zone Sumatera Utara. For each

model run, risk item locations are sampled onto the weighted irregular grid inside this area (see

Section 2.1.2). Figure 2b shows a footprint of this event obtained using the GMPE by Chiou and

Youngs (2008) and the outline of the administrative zone.

3. RESULTS

3.1 Error Convergence of Sampling Strategies

In this section, we analyze the performance of different sampling schemes described in Section 2.2 for

the scenarios and portfolios described in Section 2.4. The discontinuous zero-one-inflated damage

distribution (see Section 2.1.3) as well as the high dimensionality of large portfolios provide an

interesting and challenging test case.

Figure 3 shows estimated event losses

for the Palembang portfolio with 1 and 5 risk items against

ten sample sizes with for 50repeated simulations for each sample size All three

sampling schemes converge to the same solution, but the estimations obtained with Latin Hypercube

Sampling (LHS) with the maximin design criterion and Quasi Monte Carlo with the scrambled Sobol

sequence (SSobol) converge faster with less variable loss estimates. The portfolio with 5 risk items has

less variation than the portfolio with 1 risk item due to the diversification induced by uncorrelated

sampling of individual building losses. A correlation model such as a spatial ground motion

correlation model (e.g. Jayaram and Baker 2009) with a spatially clustered portfolio or any type of

damage correlation model would act to lessen this effect. Other portfolio value distributions than flat

(see Section 2.4) would also show relatively higher variability.

To analyze the convergence order of different sampling schemes, Figure 4 shows logarithmic plots of

the relative standard error of

against sample size for the Medan portfolio set, obtained from

repeated simulations ( for the portfolio with 1 risk item, for 100 risk items, and

for 5000 and 10000 risk items). The thin blue and red lines indicate theoretical and

) convergence given the initial at . As expected, simple MC converges slowest with

for all portfolios independently of the dimensionality. For the small portfolios with 1 and

100 risk items, SSobol and LHS perform about equally well and achieve linear convergence. For the

larger portfolio sizes (5000 and 10000 risk items), LHS does not achieve convergence but

retains some advantage over simple MC. We do not use the Sobol sequence for very large portfolios,

because the employed algorithm only supports up to 1111 dimensions (Joe and Kuo 2003).

8

Figure 3. Event loss

versus sample size for the Sumatra subduction zone event and the Palembang

portfolio set with 1 risk item (left) and 5 risk items (right). Semi-transparent circles depict 50 repeated

simulations for each sample size and sampling scheme, with solid lines highlighting one individual repetition.

The transparently shaded background indicates the entire range for each sampling scheme. Estimations obtained

using Latin Hypercube Sampling (LHS; red) and Quasi Monte Carlo using a scrambled Sobol sequence (SSobol;

green) scatter less than those obtained with simple MC (blue).

Figure 4. Logarithmic plot of relative standard errors versus sample size obtained from repeated

simulations and bootstrapped upper confidence intervals of event loss for the Medan portfolio set with 1,

100, 5000 and 10000 risk items and the Sumatra fault zone scenario. Latin Hypercube

Sampling (LHS; red) and Quasi Monte Carlo using a scrambled Sobol sequence (SSobol; green) achieve

convergence for the small portfolios. While is not achieved for larger portfolios, LHS still retains some

advantage over simple sampling (blue).

9

3.2 Results of Global Sensitivity Analysis

Using the same hypothetical scenarios and synthetic portfolios in Indonesia, we performed a global

SA to investigate the effect of uncertain input factors on event loss estimation.

To obtain a first impression of sensitivities, scatter plots are a simple and powerful tool. This graphical

global SA technique allows to quickly assess the first-order effect of varying each factor over its entire

range while also taking the global input space of other factors into account. Figure 5 shows scatter

plots of event losses

versus five different input factors for the Sumatra subduction zone

event and a portfolio in DKI Jakarta with 1 building, obtained using LHS with size .

Each plot is a one-dimensional projection of the entire sample, in which one factor ( is varied

systematically while all other factors () are taken unconstrained over their full range. The red lines

approximate ) by computing mean values of

inside a sliding window with length

of the range of the respective input factor. The steep slope and large range (i.e., large variance) of the

red lines of the building quality and damage residual suggest that these uncertainty types have a strong

effect for this scenario and portfolio.

Figure 5. Scatter plot of event loss

versus individual input factors for the Sumatra subduction zone

event and a portfolio in DKI Jakarta with 1 building obtained using Latin Hypercube Sampling. In each plot all

other input factors are sampled over their entire range, which corresponds to a projection to one dimension.

Semi-transparent blue markers depict individual event loss realizations. The red curves correspond to the

conditional mean obtained using a sliding window with a length of of the total input interval, thereby

approximating ).

Figure 6 shows the results of a variance-based global SA (see Section 2.3) for the same portfolio. To

ensure convergence of the sensitivity indices, for this portfolio we computed model

evaluations using simple MC, corresponding to a base sample size (see Sarrazin et

al. 2016, Equation 13). Main effects (first order Sobol sensitivity indices are depicted as orange

boxes and total effects (total sensitivity indices ) as blue boxes for each input factor. The MC

estimation of each effect is indicated by a thin black line inside the corresponding box, while the

extent of the boxes depicts confidence intervals obtained using bootstrapping. Confirming the

impression obtained from the scatter plot, the GMPE, the building quality and the damage distribution

all have an important first order effect. All three are subject to significant higher order interactions

with other factors, resulting in larger total sensitivity indices. This underlines the notion that

uncertainty quantification for seismic risk analysis should not merely consider first-order effects or

local sensitivities, but consider factor interactions and explore the global uncertainty space. For this

scenario location uncertainty has little effect, which can be explained by the relatively small extent of

the administrative zone DKI Jakarta (see Figure 2a).

Figure 7 shows the equivalent plot for the Sumatra fault zone event and the Medan portfolio

with 100 risk items. The main and total effects for the same uncertainty types as before are

investigated. However, in this case the 500 individual input factors are organized into five uncertainty

groups (one per uncertainty type), each containing 100 independently sampled factors corresponding

to the 100 risk items. Due to the larger portfolio size, model evaluations were necessary

to achieve convergence of the indices, corresponding to a base sample size .

For this scenario, location uncertainty has a stronger effect because the province of Sumatera Utara

has a large spatial extent (see Figure 2b) even though it is on the same administrative level as

DKI Jakarta.

10

Figure 6. Main effects (orange) and total effects (blue) for the Sumatra subduction zone event and

a portfolio in DKI Jakarta with 1 building. While in this case the building location and height account for a

negligible fraction of event loss variance, the GMPE, the building quality and the damage distribution all have a

sizable effect, in particular in interaction with other factors (. The vertical extent of the boxes corresponds to

bootstrapped confidence intervals.

Figure 7. Main effects (orange) and total effects (blue) for the Sumatra fault zone event and a

portfolio in Medan consisting of 100 buildings. While building heights accommodate a small fraction of the

output variance, building locations and qualities, the GMPE as well as the damage distributions have substantial

influence. The vertical extent of the boxes corresponds to bootstrapped confidence intervals.

11

5. CONCLUSIONS

The results shown in this study indicate that LHS as well as QMC have the potential to increase the

computational efficiency of seismic risk analysis. We observe that error convergence is improved from

to ) for many loss scenarios. While ) convergence is not fully achieved for very

large portfolios, it still remains advantageous to use advanced sampling strategies over simple MC.

We have furthermore investigated the impact of uncertainties in the ground motion model as well as in

the exposure and vulnerability models. Like many other uncertainty types in the exposure model,

uncertainties in building location and building properties are so far often neglected. This study has

shown that – depending on the loss scenario – a large fraction of the output variance can be attributed

to these factors. Although due care must be exercised when transferring the results to other models,

they highlight the importance of investigating the uncertainty associated with different factors.

Decision makers may then incorporate this knowledge into e.g. regulation, disaster management and

response plans, as well as risk mitigation measures and insurance pricing policies.

This work represents a progressive step towards a more comprehensive understanding of uncertainty

in seismic risk analysis. Nevertheless, the integration of more factors remains an important task. The

results of this study could also be tested using other methods than a variance-based SA, such as the

elementary effects test (Morris 1991) or density based methods (e.g. Pianosi and Wagener 2015).

Another powerful alternative is derivative based global SA (Sobol and Kucherenko 2009). This

approach is particularly efficient in combination with algorithmic differentiation, which has already

been successfully performed for PSHA (Molkenthin et al. 2017).

6. ACKNOWLEDGMENTS

We thank Francesca Pianosi and Thorsten Wagener for providing access to the SAFE toolbox.

7. REFERENCES

Boore DM, Atkinson GM (2008). Ground-motion prediction equations for the average horizontal component of

PGA, PGV, and 5%-damped PSA at spectral periods between 0.01 s and 10.0 s. Earthquake Spectra, 24(1): 99-

138.

Campbell KW, Bozorgnia Y (2008). NGA ground motion model for the geometric mean horizontal component

of PGA, PGV, PGD and 5% damped linear elastic response spectra for periods ranging from 0.01 to 10

s. Earthquake Spectra, 24(1): 139-171.

Chiou BSJ, Youngs RR (2008). An NGA model for the average horizontal component of peak ground motion

and response spectra. Earthquake Spectra, 24(1): 173–215.

Cornell CA (1968). Engineering seismic risk analysis. Bulletin of the Seismological Society of America, 58(5):

1583–1606.

Crowley H (2014). Earthquake risk assessment: Present shortcomings and future directions. In: Ansal A. (eds)

Perspectives on European earthquake engineering and seismology. Geotechnical, Geological and Earthquake

Engineering 34, Springer Cham, pp. 515–532.

Efron B, Tibshirani R (1986). Bootstrap methods for standard errors, confidence intervals, and other measures of

statistical accuracy. Statistical Science, 1: 54-75.

Gaughan AE, Stevens FR, Linard C, Jia P, Tatem AJ (2015). High resolution population distribution maps for

southeast Asia in 2010 and 2015. PLoS ONE, 8(2).

Hayes GP, Wald DJ, Johnson, RL (2012). Slab1.0: A threedimensional model of global subduction zone

geometries. Journal of Geophysical Research: Solid Earth, 117(B1).

Homma T, Saltelli A (1996). Importance measures in global sensitivity analysis of nonlinear models. Reliability

Engineering & System Safety, 52(1): 1–17.

Jansen MJ (1999). Analysis of variance designs for model output. Computer Physics Communications, 117(1-2):

35-43.

12

Jayaram N, Baker JW (2009). Correlation model for spatially distributed groundmotion intensities. Earthquake

Engineering & Structural Dynamics, 38(15): 1687-1708.

Joe S, Kuo FY (2003). Remark on algorithm 659: Implementing Sobol's quasirandom sequence generator. ACM

Transactions on Mathematical Software, 29(1): 49-57.

Johnson M, Moore L, Ylvisaker D (1990). Minimax and maximin distance design. Journal of Statistical

Planning and Inference, 26(2): 131-148.

Matoušek J (1998). On the L2-discrepancy for anchored boxes. Journal of Complexity, 14(4): 527–556.

Matsumoto M, Nishimura T (1998). Mersenne twister: a 623-dimensionally equidistributed uniform pseudo-

random number generator. ACM Transactions on Modeling and Computer Simulation, 8(1), 3–30.

McKay MD, Beckman RJ, Conover WJ (1979). A comparison of three methods for selecting values of input

variables in the analysis of output from a computer code, Technometrics, 21(2): 239–245.

Metropolis N, Ulam S (1949). The Monte Carlo method. Journal of the American Statistical Association,

44(247): 335–341.

Molkenthin C, Scherbaum F, Griewank A, Leovey H, Kucherenko S, Cotton F (2017). Derivative-based global

sensitivity analysis: Upper bounding of sensitivities in seismic-hazard assessment using automatic

differentiation. Bulletin of the Seismological Society of America, 107(2): 984–1004.

Morokoff WJ, Caflisch RE (1994). Quasi-random sequences and their discrepancies. SIAM Journal on Scientific

Computing, 15(6): 1251–1279.

Morris MD (1991). Factorial sampling plans for preliminary computational experiments. Technometrics, 33(2):

161–174.

Moskowitz B, Caflisch RE (1996). Smoothness and dimension reduction in Quasi-Monte Carlo methods.

Mathematical and Computer Modelling, 23(8–9): 37–54.

Owen, AB (1994). Controlling correlations in Latin hypercube samples. Journal of the American Statistical

Association, 89(428): 1517–1522.

Owen AB (1997). Monte Carlo variance of scrambled net quadrature. SIAM Journal on Numerical Analysis,

34(5): 1884–1910.

Pianosi F, Sarrazin F, Wagener T (2015). A Matlab toolbox for global sensitivity analysis. Environmental

Modelling and Software, 70: 80–85.

Pianosi F, Wagener T (2015). A simple and efficient method for global sensitivity analysis based on cumulative

distribution functions. Environmental Modelling and Software, 67: 1–11.

Saltelli A, Annoni P, Azzini I, Campolongo F, Ratto M, Tarantola S (2010). Variance based sensitivity analysis

of model output. Design and estimator for the total sensitivity index. Computer Physics Communications,

181(2): 259–270.

Sarrazin F, Pianosi F, Wagener T (2016). Global sensitivity analysis of environmental models: convergence and

validation. Environmental Modelling and Software, 79: 135-152.

Senior Seismic Hazard Committee (SSHAC) (1997). Recommendations for probabilistic seismic hazard

analysis: guidance on uncertainty and use of experts. NUREG/CR-6372. Vol. 1.

Sobol’ IM (1967). On the distribution of points in a cube and the approximate evaluation of integrals. Zhurnal

Vychislitel’noi Matematiki I Matematicheskoi Fiziki, 7(4): 784–802.

Sobol’ IM (2001). Global sensitivity indices for nonlinear mathematical models and their Monte Carlo

estimates. Mathematics and Computers in Simulation, 55(1): 271-280.

Sobol’ IM, Kucherenko S (2009). Derivative based global sensitivity measures and their link with global

sensitivity indices. Mathematics and Computers in Simulation, 79(10): 3009–3017.

Youngs RR, Chiou SJ, Silva WJ, Humphrey JR (1997). Strong ground motion attenuation relationships for

subduction zone earthquakes. Seismological Research Letters, 68(1): 58-73.

Zhao JX, Zhang J, Asano A, Ohno Y, Oouchi T, Takahashi T, ..., Fukushima Y (2006). Attenuation relations of

strong ground motion in Japan using site classification based on predominant period. Bulletin of the

Seismological Society of America, 96(3): 898-913.