Figure 2 - available via license: Creative Commons Attribution 4.0 International

Content may be subject to copyright.

# Properties of the metafunction over 1000 realisations: distribution of the fraction of active inputs (left); distribution of Si (right).

Source publication

Comparison studies of global sensitivity analysis (GSA) approaches are limited in that they are performed on a single model or a small set of test functions, with a limited set of sample sizes and dimensionalities. This work introduces a flexible ‘metafunction’ framework to benchmarking which randomly generates test problems of varying dimensionali...

## Similar publications

Science education has acknowledged the need to consider modelling as a key process in teaching and learning. Despite scientific modelling has been gaining traction within science education studies, there is relatively little research on the modelling of biological diversity. Our goal was to better understand teacher and students? modelling-related...

## Citations

... One of the main SA methods is Sobol' analysis, which studies how the dispersion of individual components of the input data (and their combinations) affects the dispersion of output data [16][17][18][19]. Improvements to this method can be found in recent works [20,21]. ...

In designing accurate constitutive models, it is important to investigate the stability of the response obtained by means of these models to perturbations in operator and input data because the properties of materials at different structural-scale levels and thermomechanical influences are stochastic in nature. In this paper, we present the results of an application of the method developed by the authors to a numerical study of the stability of multilevel models to different perturbations: perturbations of the history of influences, initial condition perturbations, and parametric operator perturbations. We analyze a two-level constitutive model of the alpha-titanium polycrystal with a hexagonal closed packed lattice under different loading modes. The numerical results obtained here indicate that the model is stable to perturbations of any type. For the first time, an analytical justification of the stability of the considered constitutive model by means of the first Lyapunov method is proposed, and thus the impossibility of instability in models with modified viscoplastic Hutchinson relations is proved.

... The Park function is defined in, e.g., Cox et al. (2001). The Becker functions are from Becker (2020). Figure 2 displays the boxplots of the empirical coverage and the average width of the intervals (over the 40 repetitions) when the regularity parameter varies, in the case of the Goldstein-Price function. ...

This article advocates the use of conformal prediction (CP) methods for Gaussian process (GP) interpolation to enhance the calibration of prediction intervals. We begin by illustrating that using a GP model with parameters selected by maximum likelihood often results in predictions that are not optimally calibrated. CP methods can adjust the prediction intervals, leading to better uncertainty quantification while maintaining the accuracy of the underlying GP model. We compare different CP variants and introduce a novel variant based on an asymmetric score. Our numerical experiments demonstrate the effectiveness of CP methods in improving calibration without compromising accuracy. This work aims to facilitate the adoption of CP methods in the GP community.

... If the model runs fast, then perhaps up to a dozen inputs can be considered. Otherwise, owing to the sparsity-ofeffects principle-aka Pareto principle (Becker, 2020)-it is reasonable to expect that a sufficiently large number of total indices will be close to zero. Thus, we may still apply the inference method proposed, but to a well-chosen selection of inputs. ...

Although there is a plethora of methods to estimate sensitivity indices associated
with individual inputs, there is much less work on interaction effects of every order,
especially when it comes to make inferences about the true underlying values of
the indices. To fill this gap, a method that allows one to make such inferences
simultaneously from a Monte Carlo sample is given. One advantage of this method
is its simplicity: it leverages the fact that Shapley effects and Sobol indices are only
linear transformations of total indices, so that standard asymptotic theory suffices to
get confidence intervals and to carry out statistical tests. To perform the numerical
computations efficiently, Möbius inversion formulas are used, and linked to the fast
Möbius transform algorithm. The method is illustrated on two dynamical systems,
both with an application in life sciences: a Boolean network modeling a cellular
decision-making process involving 12 inputs, and a system of ordinary differential
equations modeling some population dynamics involving 10 inputs.

... With over 50 years of development, modelers have access to various GSA procedures and a rich literature guiding method selection for specific SA situations (Becker 2020;). One of the most widespread routines are variance-based methods, which decompose the output variance of a d-dimensional model into contributions from individual inputs, pairs of inputs, triplets, etc., up to the dth term. ...

... To minimize the influence of the benchmarking design on the results of the analysis, we randomize the main factors that condition the accuracy of sensitivity estimators: the sampling method τ , base sample size N s , model dimensionality d, form of the test function and distribution of model inputs φ (Becker 2020;. We describe these factors with probability distributions selected to cover a wide range of sensitivity analysis settings, from low-dimensional, computationally inexpensive designs to complex, high-dimensional problems formed by inputs whose uncertainty is described by dissimilar mathematical functions (Figure 3). ...

... Our metafunction, whose functional form is defined by (i) , is based on the Becker (2020) metafunction and randomizes over 13 univariate functions representing common responses in physical systems and in classic SA functions (from cubic, exponential or periodic to sinusoidal, see Figure S2). A detailed explanation of the metafunction can be found in Becker (2020) and in . 4. We use y J to produce a vector with the total-order indices T, calculated with the Jansen (1999) estimator, which reads as follows: ...

... La fonction d'Ishigami [15,91] est largement utilisée comme fonction test de référence dans l'ASG en raison de son comportement fortement non linéaire et non monotone [16,189]. En outre, sa dépendance à l'égard de la troisième variable est assez particulière, page 58 Manuscrit de thèse 2.3 Problème analytique de référence : la fonction d'Ishigami ...

The more complex the problem, the greater the amount of computational resources needed to simulate it. On the other hand, the need for accuracy in the results of a system will not be the same depending on its design phase and the domain studied. The goal of this thesis is to propose a fast, low-cost multi-fidelity modeling strategy. To meet this need, a hybrid modeling approach is developed that combines Model-Based System Engineering (MBSE) and a metamodel based on Non-Uniform Rational Basis-Spline (NURBS) hypersurfaces. More specifically, the scientific challenge of this work is to develop a metamodel based on NURBS entities to simulate the behavior of highly nonlinear systems that require high fidelity modeling but are capable of providing results in real time to be compatible with the MBSE approach. In this context, the NURBS entity-based metamodel is obtained as a solution to an optimization problem solved with a gradient algorithm. In addition, a smoothing term is included in the problem formulation, not only to reduce the influence of any spurious nonlinearities in the training database, but also to limit the phenomenon of overfitting. The technical and scientific challenge of this work is to couple the general MBSE approach with the NURBS-based metamodel.

... Hence, we study the relation between the involved input parameters corresponding to the inflation pressure and the reduced axial force by means of an SA, namely the FAST method. For the implementation of the FAST method, there are some functions that can be applied, e.g., the Bratley function [28] , the Oakley function [29] , the metafunctions [30] , and the Ishigami function [31] . The metafunctions and the Ishigami function will be considered in this work as they are applicable to non-linear and non-monotonic problems. ...

... There exist various analyses that apply the FAST method to a set of data. In the framework of this article, we focus on the application of the metafunctions and the Ishigami function [30,45] . These methods are considered to be important quantification techniques within the FAST framework of SAs. ...

... They gain the random functions through designating the basis functions that happen in real models and follow the true specifications of these models, i.e., the metafunctions make random functions with various dimensions to carry out an SA. The method is proposed by Becker [30] , and is shown as follows: ...

This paper is dedicated to applying the Fourier amplitude sensitivity test (FAST) method to the problem of mixed extension and inflation of a circular cylindrical tube in the presence of residual stresses. The metafunctions and the Ishigami function are considered in the sensitivity analysis (SA). The effects of the input variables on the output variables are investigated, and the most important parameters of the system under the applied pressure and axial force such as the axial stretch and the azimuthal stretch are determined.

... The Sobol method is carried by means of the Saltelli-Janson estimator to determine the first and total Sobol indices [42]. The application of the FAST method invokes the use of so-called metafunctions, which is proposed by Becker [9]. For these two methods, we considered three types of distributions of the input parameters, namely, the uniform, gamma, and normal distributions. ...

... For the residual stresses, we apply a form that satisfies these conditions (9) and (10) ...

... There exist a variety of options to apply the FAST method to a certain problem. The latest approach applies so-called metafunctions, which have been proposed in [9], and we apply the metafunctions in the basic form ...

This paper deals with applying two main sensitivity analysis (SA) methods, namely, the Sobol method and the Fourier Amplitude Sensitivity Test (FAST) method on the problem of mixed extension, inflation, and torsion of a circular cylindrical tube in the presence of residual stress. The mechanical side of the problem was previously proposed by Merodio & Ogden (2016). The input parameters in the form of the initial cylinder geometry, the amount of the residual stress, the azimuthal stretch, the axial elongation, and the torsional strain are distributed according to three probability distribution methods, namely the uniform, the gamma, and the normal distribution. In the present work, through applying Sobol and FAST methods, the most influential factors among input parameters on the output variable have been determined. The assessment of our results is then determined by the computation of bias and standard deviation of Sobol and FAST indices for each input parameter in the model.

... After more than 50 years of development, modelers dispose of several SA procedures and of a rich literature informing on which methods are most efficient in each specific SA setting [7,8]. We briefly mention here some of these routines without further description and direct the reader to existing references: ...

... To minimize the influence of the benchmarking design on the results of the analysis, we randomize the main factors that condition the accuracy of sensitivity estimators: the sampling method τ , base sample size N s , model dimensionality d, form of the test function and distribution of model inputs φ [7,8]. We describe these factors with probability distributions selected to cover a wide range of sensitivity analysis settings, from low-dimensional, computationally inexpensive designs to complex, high-dimensional problems formed by inputs whose uncertainty is described by dissimilar mathematical functions (Fig. 3). ...

... 3. We run a metafunction rowwise through both the Jansen and the Discrepancy matrix and produce two vectors with the model output, which we refer to as y J and y D respectively. Our metafunction, whose functional form is defined by (i) , is based on the Becker metafunction [8] and randomizes over 13 univariate functions representing common responses in physical systems and in classic sensitivity analysis functions (from cubic, exponential or periodic to sinusoidal, see Fig. S2). A detailed explanation of the metafunction can be found in Becker [8] and in Puy et al. [7]. ...

While sensitivity analysis improves the transparency and reliability of mathematical models, its uptake by modelers is still scarce. This is partially explained by its technical requirements, which may be hard to understand and implement by the non-specialist. Here we propose a sensitivity analysis approach based on the concept of discrepancy that is as easy to understand as the visual inspection of input-output scatterplots. Firstly, we show that some discrepancy measures are able to rank the most influential parameters of a model almost as accurately as the variance-based total sensitivity index. We then introduce an ersatz-discrepancy whose performance as a sensitivity measure matches that of the best-performing discrepancy algorithms, is simple to implement, easier to interpret and orders of magnitude faster.

... For example, Sudret (2008) and Crestaux et al. (2009) have shown that polynomial chaos-based estimators of Sobol' indices are much more efficient than Monte Carlo or quasi-Monte Carlo-based estimators (for smooth models and dimensions up to 20). Recently, Becker (2020) has shown that certain sample-based approaches can be more efficient than metamodel-based ones for screening with total Sobol' indices. However, the screening performance metrics of Becker (2020) are only based on input ranking. ...

... Recently, Becker (2020) has shown that certain sample-based approaches can be more efficient than metamodel-based ones for screening with total Sobol' indices. However, the screening performance metrics of Becker (2020) are only based on input ranking. In contrary, our practical purpose is to perform a so-called quantitative screening which aims at providing a correct screening and a good estimation of Sobol' indices. ...

Variance-based global sensitivity analysis, in particular Sobol' analysis, is widely used for determining the importance of input variables to a computational model. Sobol' indices can be computed cheaply based on spectral methods like polynomial chaos expansions (PCE). Another choice are the recently developed Poincaré chaos expansions (PoinCE), whose orthonormal tensor-product basis is generated from the eigenfunctions of one-dimensional Poincaré differential operators. In this paper, we show that the Poincaré basis is the unique orthonormal basis with the property that partial derivatives of the basis form again an orthogonal basis with respect to the same measure as the original basis. This special property makes PoinCE ideally suited for incorporating derivative information into the surrogate modelling process. Assuming that partial derivative evaluations of the computational model are available, we compute spectral expansions in terms of Poincaré basis functions or basis partial derivatives, respectively, by sparse regression. We show on two numerical examples that the derivative-based expansions provide accurate estimates for Sobol' indices, even outperforming PCE in terms of bias and variance. In addition, we derive an analytical expression based on the PoinCE coefficients for a second popular sensitivity index, the derivative-based sensitivity measure (DGSM), and explore its performance as upper bound to the corresponding total Sobol' indices.

... In addition to comparing the two SA methods, the accuracy of sensitivity analysis methods also needs to be discussed considering their numerical implementations, as this factor has a significant influence on the efficiency and accuracy of method. A recent study (Becker, 2020) Furthermore, the majority of reliability estimates of factor rankings of VARS (i.e., especially IVARS50, the most comprehensive index) are higher than Sobol, giving more trust to VARS results compared to Sobol method. Our findings on comparative evaluation of both results are consistent with previous studies (Alipour et al., 2022;Razavi & Gupta, 2016a, where they also recognized VARS better than Sobol or other global sensitivity analysis methods. ...

Bioretention cell (BC) design variation significantly changes its hydrological dynamics at unit-scale leading to major changes in its target design goals when designed unit replicated at the catchment-scale. Improved understanding of the behaviors of BC design parameters under different rainfall conditions is critical for effective implementation of BCs to achieve design goals. This paper illustrates and compares two global sensitivity analysis methods (i.e., VARS and Sobol) for the identification of influential parameters and their variations in hydrological dynamics (model responses) under different rainfall conditions and perturbation scales, so as to quantify their uncertainty and reliability for influential factors ranking. From the application results of both sensitivity analysis methods, six parameters out of the total seventeen including conductivity, berm height, vegetation volume, suction head, porosity and wilting point were indicated for their significant sensitivities to surface infiltration, surface outflow and peak flow. In addition, soil thickness, conductivity slope and field capacity (in a total of nine) were categorized as influential to storage outputs. Conductivity and vegetation volume were ranked as the most influential parameters, followed by berm height and suction head, porosity and wilting point. The results analysis also demonstrated that the behavior of design parameters towards model response significantly changes with different rainfall conditions and perturbation scales, as well as the used sensitivity analysis methods. In particular, it is indicated from this study that the VARS is preferred over other sensitivity analysis approaches including the Sobol method (i.e., variance-based method) because of its higher accuracy, reliability, and computational efficiency.