Figure 1 - available via license: Creative Commons Attribution 4.0 International
Content may be subject to copyright.
Source publication
Comparison studies of global sensitivity analysis (GSA) approaches are limited in that they are performed on a single model or a small set of test functions, with a limited set of sample sizes and dimensionalities. This work introduces a flexible ‘metafunction’ framework to benchmarking which randomly generates test problems of varying dimensionali...
Similar publications
Science education has acknowledged the need to consider modelling as a key process in teaching and learning. Despite scientific modelling has been gaining traction within science education studies, there is relatively little research on the modelling of biological diversity. Our goal was to better understand teacher and students? modelling-related...
Citations
... However, closed analytical formulas for calculating the Sobol' indices exist just for some classes of models, such as Polynomial Chaos Expansion (PCE) (Sudret, 2008), and some simple and explicit models, generally used as benchmark cases (Becker, 2020). In the majority of practical cases, the Sobol' indices are estimated by sampling-based estimators, which consists of obtaining a finite sample of model evaluations, typically with size on the order of thousands, and applying a discrete estimation technique (Helton et al., 2006;Saltelli et al., 2008). ...
Global sensitivity analysis (GSA) aims at quantifying the contribution of input variables over the variability of model outputs. In the frame of functional outputs, a common goal is to compute sensitivity maps (SM), i.e sensitivity indices at each output dimension (e.g. time step for time series, or pixels for spatial outputs). In specific settings, some works have shown that the computation of Sobol' SM can be speeded up by using basis expansions employed for dimension reduction. However, how to efficiently compute such SM in a general setting has not received too much attention in the GSA literature.In this work, we propose fast computations of Sobol' SM using a general basis expansion, with a focus on statistical estimation. First, we write a closed-form expression of SM in function of the matrix-valued Sobol' index of the vector of basis coefficients. Secondly, we consider pick-freeze (PF) estimators, which have nice statistical properties (in terms of asymptotical efficiency) for Sobol' indices of any order. We provide similar basis-derived formulas for the PF estimator of Sobol' SM in function of the matrix-valued PF estimator of the vector of basis coefficients. We give the computational cost, and show that, compared to a dimension-wise approach, the computational gain is substantial and allows to calculate both SM and their associated bootstrap confidence bounds in a reasonable time. Finally, we illustrate the whole methodology on an analytical test case and on an application in non-Newtonian hydraulics, modelling an idealized dam-break flow.
... If the model runs fast, then perhaps up to a dozen inputs can be considered. Otherwise, owing to the sparsity-ofeffects principle-aka Pareto principle (Becker, 2020)-it is reasonable to expect that a sufficiently large number of total indices will be close to zero. Thus, we may still apply the inference method proposed, but to a well-chosen selection of inputs. ...
Although there is a plethora of methods to estimate sensitivity indices associated
with individual inputs, there is much less work on interaction effects of every order,
especially when it comes to make inferences about the true underlying values of
the indices. To fill this gap, a method that allows one to make such inferences
simultaneously from a Monte Carlo sample is given. One advantage of this method
is its simplicity: it leverages the fact that Shapley effects and Sobol indices are only
linear transformations of total indices, so that standard asymptotic theory suffices to
get confidence intervals and to carry out statistical tests. To perform the numerical
computations efficiently, Möbius inversion formulas are used, and linked to the fast
Möbius transform algorithm. The method is illustrated on two dynamical systems,
both with an application in life sciences: a Boolean network modeling a cellular
decision-making process involving 12 inputs, and a system of ordinary differential
equations modeling some population dynamics involving 10 inputs.
... One of the main SA methods is Sobol' analysis, which studies how the dispersion of individual components of the input data (and their combinations) affects the dispersion of output data [16][17][18][19]. Improvements to this method can be found in recent works [20,21]. ...
In designing accurate constitutive models, it is important to investigate the stability of the response obtained by means of these models to perturbations in operator and input data because the properties of materials at different structural-scale levels and thermomechanical influences are stochastic in nature. In this paper, we present the results of an application of the method developed by the authors to a numerical study of the stability of multilevel models to different perturbations: perturbations of the history of influences, initial condition perturbations, and parametric operator perturbations. We analyze a two-level constitutive model of the alpha-titanium polycrystal with a hexagonal closed packed lattice under different loading modes. The numerical results obtained here indicate that the model is stable to perturbations of any type. For the first time, an analytical justification of the stability of the considered constitutive model by means of the first Lyapunov method is proposed, and thus the impossibility of instability in models with modified viscoplastic Hutchinson relations is proved.
... The Park function is defined in, e.g., Cox et al. (2001). The Becker functions are from Becker (2020). Figure 2 displays the boxplots of the empirical coverage and the average width of the intervals (over the 40 repetitions) when the regularity parameter varies, in the case of the Goldstein-Price function. ...
This article advocates the use of conformal prediction (CP) methods for Gaussian process (GP) interpolation to enhance the calibration of prediction intervals. We begin by illustrating that using a GP model with parameters selected by maximum likelihood often results in predictions that are not optimally calibrated. CP methods can adjust the prediction intervals, leading to better uncertainty quantification while maintaining the accuracy of the underlying GP model. We compare different CP variants and introduce a novel variant based on an asymmetric score. Our numerical experiments demonstrate the effectiveness of CP methods in improving calibration without compromising accuracy. This work aims to facilitate the adoption of CP methods in the GP community.
... With over 50 years of development, modelers have access to various GSA procedures and a rich literature guiding method selection for specific SA situations (Becker 2020;). One of the most widespread routines are variance-based methods, which decompose the output variance of a d-dimensional model into contributions from individual inputs, pairs of inputs, triplets, etc., up to the dth term. ...
... To minimize the influence of the benchmarking design on the results of the analysis, we randomize the main factors that condition the accuracy of sensitivity estimators: the sampling method τ , base sample size N s , model dimensionality d, form of the test function and distribution of model inputs φ (Becker 2020;. We describe these factors with probability distributions selected to cover a wide range of sensitivity analysis settings, from low-dimensional, computationally inexpensive designs to complex, high-dimensional problems formed by inputs whose uncertainty is described by dissimilar mathematical functions (Figure 3). ...
... Our metafunction, whose functional form is defined by (i) , is based on the Becker (2020) metafunction and randomizes over 13 univariate functions representing common responses in physical systems and in classic SA functions (from cubic, exponential or periodic to sinusoidal, see Figure S2). A detailed explanation of the metafunction can be found in Becker (2020) and in . 4. We use y J to produce a vector with the total-order indices T, calculated with the Jansen (1999) estimator, which reads as follows: ...
... La fonction d'Ishigami [15,91] est largement utilisée comme fonction test de référence dans l'ASG en raison de son comportement fortement non linéaire et non monotone [16,189]. En outre, sa dépendance à l'égard de la troisième variable est assez particulière, page 58 Manuscrit de thèse 2.3 Problème analytique de référence : la fonction d'Ishigami ...
The more complex the problem, the greater the amount of computational resources needed to simulate it. On the other hand, the need for accuracy in the results of a system will not be the same depending on its design phase and the domain studied. The goal of this thesis is to propose a fast, low-cost multi-fidelity modeling strategy. To meet this need, a hybrid modeling approach is developed that combines Model-Based System Engineering (MBSE) and a metamodel based on Non-Uniform Rational Basis-Spline (NURBS) hypersurfaces. More specifically, the scientific challenge of this work is to develop a metamodel based on NURBS entities to simulate the behavior of highly nonlinear systems that require high fidelity modeling but are capable of providing results in real time to be compatible with the MBSE approach. In this context, the NURBS entity-based metamodel is obtained as a solution to an optimization problem solved with a gradient algorithm. In addition, a smoothing term is included in the problem formulation, not only to reduce the influence of any spurious nonlinearities in the training database, but also to limit the phenomenon of overfitting. The technical and scientific challenge of this work is to couple the general MBSE approach with the NURBS-based metamodel.
... Hence, we study the relation between the involved input parameters corresponding to the inflation pressure and the reduced axial force by means of an SA, namely the FAST method. For the implementation of the FAST method, there are some functions that can be applied, e.g., the Bratley function [28] , the Oakley function [29] , the metafunctions [30] , and the Ishigami function [31] . The metafunctions and the Ishigami function will be considered in this work as they are applicable to non-linear and non-monotonic problems. ...
... There exist various analyses that apply the FAST method to a set of data. In the framework of this article, we focus on the application of the metafunctions and the Ishigami function [30,45] . These methods are considered to be important quantification techniques within the FAST framework of SAs. ...
... They gain the random functions through designating the basis functions that happen in real models and follow the true specifications of these models, i.e., the metafunctions make random functions with various dimensions to carry out an SA. The method is proposed by Becker [30] , and is shown as follows: ...
This paper is dedicated to applying the Fourier amplitude sensitivity test (FAST) method to the problem of mixed extension and inflation of a circular cylindrical tube in the presence of residual stresses. The metafunctions and the Ishigami function are considered in the sensitivity analysis (SA). The effects of the input variables on the output variables are investigated, and the most important parameters of the system under the applied pressure and axial force such as the axial stretch and the azimuthal stretch are determined.
... The Sobol method is carried by means of the Saltelli-Janson estimator to determine the first and total Sobol indices [42]. The application of the FAST method invokes the use of so-called metafunctions, which is proposed by Becker [9]. For these two methods, we considered three types of distributions of the input parameters, namely, the uniform, gamma, and normal distributions. ...
... For the residual stresses, we apply a form that satisfies these conditions (9) and (10) ...
... There exist a variety of options to apply the FAST method to a certain problem. The latest approach applies so-called metafunctions, which have been proposed in [9], and we apply the metafunctions in the basic form ...
This paper deals with applying two main sensitivity analysis (SA) methods, namely, the Sobol method and the Fourier Amplitude Sensitivity Test (FAST) method on the problem of mixed extension, inflation, and torsion of a circular cylindrical tube in the presence of residual stress. The mechanical side of the problem was previously proposed by Merodio & Ogden (2016). The input parameters in the form of the initial cylinder geometry, the amount of the residual stress, the azimuthal stretch, the axial elongation, and the torsional strain are distributed according to three probability distribution methods, namely the uniform, the gamma, and the normal distribution. In the present work, through applying Sobol and FAST methods, the most influential factors among input parameters on the output variable have been determined. The assessment of our results is then determined by the computation of bias and standard deviation of Sobol and FAST indices for each input parameter in the model.
... After more than 50 years of development, modelers dispose of several SA procedures and of a rich literature informing on which methods are most efficient in each specific SA setting [7,8]. We briefly mention here some of these routines without further description and direct the reader to existing references: ...
... To minimize the influence of the benchmarking design on the results of the analysis, we randomize the main factors that condition the accuracy of sensitivity estimators: the sampling method τ , base sample size N s , model dimensionality d, form of the test function and distribution of model inputs φ [7,8]. We describe these factors with probability distributions selected to cover a wide range of sensitivity analysis settings, from low-dimensional, computationally inexpensive designs to complex, high-dimensional problems formed by inputs whose uncertainty is described by dissimilar mathematical functions (Fig. 3). ...
... 3. We run a metafunction rowwise through both the Jansen and the Discrepancy matrix and produce two vectors with the model output, which we refer to as y J and y D respectively. Our metafunction, whose functional form is defined by (i) , is based on the Becker metafunction [8] and randomizes over 13 univariate functions representing common responses in physical systems and in classic sensitivity analysis functions (from cubic, exponential or periodic to sinusoidal, see Fig. S2). A detailed explanation of the metafunction can be found in Becker [8] and in Puy et al. [7]. ...
While sensitivity analysis improves the transparency and reliability of mathematical models, its uptake by modelers is still scarce. This is partially explained by its technical requirements, which may be hard to understand and implement by the non-specialist. Here we propose a sensitivity analysis approach based on the concept of discrepancy that is as easy to understand as the visual inspection of input-output scatterplots. Firstly, we show that some discrepancy measures are able to rank the most influential parameters of a model almost as accurately as the variance-based total sensitivity index. We then introduce an ersatz-discrepancy whose performance as a sensitivity measure matches that of the best-performing discrepancy algorithms, is simple to implement, easier to interpret and orders of magnitude faster.
... For example, Sudret (2008) and Crestaux et al. (2009) have shown that polynomial chaos-based estimators of Sobol' indices are much more efficient than Monte Carlo or quasi-Monte Carlo-based estimators (for smooth models and dimensions up to 20). Recently, Becker (2020) has shown that certain sample-based approaches can be more efficient than metamodel-based ones for screening with total Sobol' indices. However, the screening performance metrics of Becker (2020) are only based on input ranking. ...
... Recently, Becker (2020) has shown that certain sample-based approaches can be more efficient than metamodel-based ones for screening with total Sobol' indices. However, the screening performance metrics of Becker (2020) are only based on input ranking. In contrary, our practical purpose is to perform a so-called quantitative screening which aims at providing a correct screening and a good estimation of Sobol' indices. ...
Variance-based global sensitivity analysis, in particular Sobol' analysis, is widely used for determining the importance of input variables to a computational model. Sobol' indices can be computed cheaply based on spectral methods like polynomial chaos expansions (PCE). Another choice are the recently developed Poincaré chaos expansions (PoinCE), whose orthonormal tensor-product basis is generated from the eigenfunctions of one-dimensional Poincaré differential operators. In this paper, we show that the Poincaré basis is the unique orthonormal basis with the property that partial derivatives of the basis form again an orthogonal basis with respect to the same measure as the original basis. This special property makes PoinCE ideally suited for incorporating derivative information into the surrogate modelling process. Assuming that partial derivative evaluations of the computational model are available, we compute spectral expansions in terms of Poincaré basis functions or basis partial derivatives, respectively, by sparse regression. We show on two numerical examples that the derivative-based expansions provide accurate estimates for Sobol' indices, even outperforming PCE in terms of bias and variance. In addition, we derive an analytical expression based on the PoinCE coefficients for a second popular sensitivity index, the derivative-based sensitivity measure (DGSM), and explore its performance as upper bound to the corresponding total Sobol' indices.
