Content uploaded by Qingyun Duan
Author content
All content in this area was uploaded by Qingyun Duan on Dec 02, 2017
Content may be subject to copyright.
A preview of the PDF is not available
Optimal Use of the SCE-UA Global Optimization Method for Calibrating Watershed Models
Abstract and Figures
The difficulties involved in calibrating conceptual watershed models have, in the past, been partly attributable to the lack of robust optimization tools. Recently, a global optimization method known as the SCE-UA (shuffled complex evolution method developed at The University of Arizona) has shown promise as an effective and efficient optimization technique for calibrating watershed models. Experience with the method has indicated that the effectiveness and efficiency of the algorithm are influenced by the choice of the algorithmic parameters. This paper first reviews the essential concepts of the SCE-UA method and then presents the results of several experimental studies in which the National Weather Service river forecast system-soil moisture accounting (NWSRFS-SMA) model, used by the National Weather Service for river and flood forecasting, was calibrated using different algorithmic parameter setups. On the basis of these results, the recommended values for the algorithmic parameters are given. These values should also help to provide guidelines for other users of the SCE-UA method.
Figures - uploaded by Qingyun Duan
Author content
All figure content in this area was uploaded by Qingyun Duan
Content may be subject to copyright.
... The code minimizes the RMSE (the objective function) by the shuffled complex evolution algorithm (SCE). A detailed description of the algorithm and discussions of its parameters are provided by Duan et al. (1992Duan et al. ( , 1993Duan et al. ( , 1994. ...
... The number of dimensions of the parameter space (denoted NrOfDimensions in the code and below) is the number of parameters whose values are actually fitted, as opposed to being set to a fixed value by the user. The number 230 of complexes (sets of points in the parameter space) adheres to Duan et al. (1994) for any number of fitting parameters, but with a minimum of two, but only if input variable FewComplexes is set to 'T' on input. This results in two complexes https://doi.org/10.5194/egusphere-2024-3487 ...
... Test calculations showed the quality of the fit occasionally improves. Per the guidelines of Duan et al. (1994), the size of the complexes and the number of evolution 235 steps between shuffles equal (2 NrOfDimensions) + 1. The size of the subcomplexes is NrOfDimensions + 1. ...
Several current models for the unsaturated soil hydraulic conductivity curve consider the conductivity of the domains of capillary water in water–filled pores and adsorbed water in films on soil grains, as well as an equivalent conductivity for water vapour diffusion. These models rely on unrealistic configuration of the domains. A junction model is introduced that sidesteps this problem by assigning all liquid water to films (dry range), or to capillaries (wet range). Combined with a sigmoidal junction model for the soil water retention curve, it has up to six fitting parameters, one less than the other multidomain models. Tests on data for 13 soils show that the junction model and an additive model (that adds all domain conductivities) often produce good fits. Models with six or more parameters may be overparameterized for many soils, giving the more parsimonious junction model an advantage, but for some soils, the extra parameter of the additive model is needed to achieve a good fit. This paper and a User Manual document a Fortran code (KRIAfitter) that uses the Shuffled Complex Evolution algorithm to fit the junction, additive, and four other conductivity models for any combination of fixed and fitting parameters or their log–transforms. KRIAfitter either maps the Root Mean Square Error in the entire parameter space in order to then constrain the parameter space around the likely global minimum, or it generates many fits and uses those to calculate statistics for individual parameters, as well as the covariance and correlation matrices.
... To provide a lower limit of the general LAI-AET performance of SWAT-T, a random sampling approach of the LAI-AET parameters (lower benchmark) is applied. The LAI-AET parameter optimization is conducted using the shuffled complex evolution algorithm (SCE-UA) (Duan et al., 1994). Figure 1 gives an overview of the methods applied in this study to evaluate the significance of the LAI for AET estimation in SWAT-T. ...
... To avoid an arbitrary good or bad model response from a single parameter set, like the default model parameters, Seibert et al. (2018) propose using random parameter samples for the lower benchmark. We apply the SCE-UA algorithm (Duan et al., 1994) to optimize the LAI-AET parameters. SCE-UA is a genetic algorithm by which samples of the parameters are stochastically generated first with respect to the lower and upper bounds of the parameter values. ...
... We use the KGE to compare the simulated model output with the observed data. The algorithm application divides the initial sample into several subsamples (complexes) (Duan et al., 1994). In each complex, varying combinations of parameter values are embedded. ...
Evapotranspiration (ET) is pivotal in the terrestrial water cycle in subhumid and tropical regions. In the water cycle, the contribution of plant transpiration can be distinctively more significant than soil evaporation. The seasonal dynamics of plant phenology, commonly represented as the vegetation attribute leaf area index (LAI), closely correlates with actual ET (AET). Hence, addressing the reciprocal LAI–AET interaction is essential for practitioners and researchers to comprehensively quantify the hydrological processes in water resources management, particularly in the perennially vegetated regions of West Africa. However, due to a lack of field measurements, evaluation of the LAI–AET interaction still remains challenging. Hence, our study aims to improve the understanding of the role of the LAI in AET estimation by investigating characteristic regions of West Africa. We set up ecohydrological models (using the Soil and Water Assessment Tool for the tropics – SWAT-T) for two homogeneous land cover types (forest and grassland) to guarantee the representativeness of field measurements for the LAI and AET. We apply different potential ET methods (the Hargreaves; Penman–Monteith – PET-PM; and Priestley–Taylor methods) to evaluate the LAI–AET interaction in SWAT-T. Further, the elementary effects method quantifies the parameter sensitivity for 27 relevant LAI–AET parameters. The comprehensive parameter set is then optimized using the shuffled complex evolution algorithm. Finally, we apply a benchmarking test to assess the performance of SWAT-T with respect to the simulation of AET and to determine the relevance of detailed LAI modeling. The results show that SWAT-T is capable of accurately predicting the LAI and AET at the footprint scale. While all three PET methods facilitate adequate modeling of the LAI and AET, the PET-PM technique outperforms the other methods for AET, independent of the land cover type. Moreover, the benchmarking highlights that, if it only accounts for the LAI but disregards AET data, an optimization process's prediction of AET still yields an adequate performance with SWAT-T for all PET methods and land cover types. Our findings demonstrate that the significance of detailed LAI modeling for the AET estimation is more pronounced in the forested than in the grassland region.
... The SCE was proposed by Duan et al. (1994) for calibrating rain-runoff models and identifying aquifer formation parameters (Bek and Kosolapov 1986). Thse effectiveness of the SCE in hydrology has led to an extensive research for broader engineering optimization applications, solidifying its position as a prominent technique in this field (Chu et al. 2010). ...
One of the main concerns of underground engineering works, such as subsurface structures and mining in rock formations, is ensuring their safety. The objective of this work is to present the stability analysis of trapdoors in Hoek-Brown (HB) rock masses, and to propose an innovative soft-computing approach utilizing optimized ANN-based surrogate models for evaluating the stability of trapdoors. The stability factor serves as a key parameter in formulating both lower bound (LB) and upper bound (UB) solutions for two-dimensional trapdoor through the finite element limit analysis (FELA). Furthermore, this paper introduces hybrid machine learning models that integrate artificial neural networks (ANNs) with diverse optimization algorithms (OAs), such as the ant lion optimizer (ALO), imperialist competitive algorithm (ICA), shuffled complex evolution algorithm (SCE), and teaching learning-based optimization (TLBO). Rigorous optimization ensures the accuracy and efficiency of these models in capturing the intricate dynamics of stability investigation. The performance of the proposed models is rigorously evaluated using metrics, convergence curves, regression plot, Taylor diagram, and rank analysis. Consequently, The ANN-SCE model achieved the highest performance (Testing Set), with R² of 0.9630, MAE of 2.7416, RMSE of 0.3696, VAF(%) of 96.2834, IOS of 0.0269, and RSR of 0.0172, respectively. These results demonstrate the accuracy and efficiency of the proposed models in capturing the complex dynamics of stability investigations. This research provides practical tools for engineers to assess road stability, plan mitigation for sinkholes, and account for rock strength using the Hoek-Brown criterion.
... The determination of six soil parameters (bin, Ds, Dmax, Ws, D2, D3) is a challenging task, given their conceptual nature and the lack of observable, physical quantities [24,25]. To address this issue, a calibration procedure was developed that utilizes an autocalibration script and the Shuffled Complex Evolution method, which was chosen for its ability to identify global optima with the right algorithmic parameters [26]. The calibration framework follows the VIC hydrological model [27], except for the parameter generation algorithm shown in Figure 2. The autocalibration script was run using a spatial resolution of 0.05°, which was a suitable resolution to mimic the Da River basin. ...
... The calibration was performed using the shuffledcomplex-evolution algorithm (SCE-UA) of Duan et al. (1994) and following a single-objective optimization approach for the nine selected parameters (five soil parameters, two routing parameters, and two vegetation parameters) to minimize a composite function that aggregates the performance metrics for streamflow and evaporation: ...
The simultaneous incorporation of streamflow and evaporation data into sensitivity analysis and calibration approaches has great potential to improve the representation of hydrologic processes in modelling frameworks. This work aims to investigate the capabilities of the Variable Infiltration Capacity (VIC) model in a large-sample application focused on the joint integration of streamflow and evaporation data for 189 headwater catchments located in Spain. The study has been articulated into three parts: (1) a regional sensitivity analysis for a total of 20 soil, routing, and vegetation parameters to select the most important parameters conducive to an adequate representation of the streamflow and evaporation dynamics; (2) a two-fold calibration approach against daily streamflow and monthly evaporation data based on the previous parameter selection for VIC; and (3) an evaluation of model performance based on a benchmark comparison against a well-established hydrologic model for the Spanish domain and a cross-validation test using multiple meteorological datasets to assess the generalizability of the calibrated parameters. The regional sensitivity analysis revealed that only two vegetation parameters – namely, the leaf area index and the minimum stomatal resistance – were sufficient to improve the performance of VIC for evaporation. These parameters were added to the soil and routing parameter during the calibration stage. Results from the two calibration experiments suggested that, while the streamflow performance remained close in both cases, the evaporation performance was highly improved if the objectives for streamflow and evaporation were combined into a single composite function during optimization. The VIC model outperformed the reference benchmark, and the independent meteorological datasets yielded a slight to moderate loss in model performance depending on the calibration experiment considered. Results from this investigation provide valuable insights into VIC parameter sensitivities, with a particular focus on large-sample applications, and highlight the importance of integrating multiple datasets into model calibration as a measure to reduce model equifinality.
В статье рассматривается методика прогнозирования уровней (расходов) воды с помощь автоматизированной информационной системы краткосрочных гидрологических прогнозов (АИС), разработанная в Государственном гидрологическом институте (ГГИ) для рек Ленинградской области (реки Тихвинка, Луга, Тосна и Тигода) и Камчатского края (реки Большая Воровская и Амчигача). В настоящее время технология прогнозирования АИС внедрена в оперативную практику Северо―Западного УГМС. АИС является готовым решением в области информационных технологий, требующим минимального количества исходной информации и времени для развертывания системы. Она имеет хорошие перспективы дальнейшей адаптации для различных речных бассейнов Российской Федерации.
This study developed a land use-based parameter identification method for a distributed rainfall-runoff model 1K-DHM, by calibrating the soil parameters of all land use types to multi-event and multi-site discharge observations simultaneously. The developed method was applied to the Kikuchi River basin, indicating that the identified parameter sets with the 10 highest Nash coefficient shared the similar value within the same land use type, while they differed among land-use categories. The parameter set with the highest Nash coefficient was validated in 15 basins including Kikuchi River, showing overall high performance with the Nash coefficient between 0.5 to 0.8 at 43 stations and over 0.8 at 44 stations, especially at river basins more than 500 km². Throughout the calibration and validation, this study realized the identification of robust model parameter values that reflect the physical properties of catchment surfaces based on land use types.
A stochastic parameter estimation procedure to be applied to large conceptual rainfall-runoff models is proposed. The procedure is based on a maximum likelihood approach, which is enhanced to allow for the use of prior information about some of the parameters. A stochastic model of base flow is proposed, and an algorithm for automatic identification of the time interval of base flow activity is developed. This algorithm, which permits the estimation of prior values for the base flow discharge coefficients, is successfully applied to two basins in the United States. The application of the maximum likelihood estimation procedure to large conceptual rainfall-runoff models is divided into two stages. In the first stage the procedure is applied to a simplified version of the National Weather Service River Forecasting System model, the parameters of which are estimated by using synthetically generated data. This stage produces valuable information about the performance of stochastic parameter estimation procedures in large conceptual rainfall-runoff models. In the second stage the maximum likelihood procedure is applied to estimating the parameters of the National Weather Service River Forecasting System model for the Bird Creek and for the Cohocton River basins, respectively. In the first case a severe structural error arising from a deficient formulation of the channel causes some of the parameters to converge to unrealistic values. In the second case the stochastic model performs in a mathematically correct but hydrologically unappealing fashion. The reasons for this problems are attributed to the non-identifiability of the threshold parameter of the upper zone tension water element and to interaction between several of the percolation function parameters.
The results of using different criteria for automatically fitting a nonlinear catchment model to real data are examined. One criterion used is ad hoc while the remainder, which are drawn from recent literature, are based on the maximum likelihood principle subject to different simplifying assumptions about the nature of the calculated flow residuals (the differences between measured and computed daily flows). For each criterion the model is fitted using seven different sets of initial parameter values. Final parameter values are examined for consistency. For each criterion the seven sets of final parameter values are averaged and used to predict river flows for data not used in the fitting. The reliabilities of predicted flows are then compared. The maximum likelihood criterion that assumes uncorrelated but heteroscedastic residuals is found to be the best criterion for the model and data used. Examination of 2-dimensional sections through the model's response surface shows that it is better conditioned for the maximum likelihood criteria than for the ad hoc criterion.
The National Weather Service River Forecast System consists of a set of interrelated computer programs developed to provide continuous hydrologic forecasting. The catchment model is conceptual in design and generally must be calibrated for each watershed. Current calibration procedures consist of a combination of trial-and-error and automatic-parameter optimization tech-niques. Initial parameter values are determined from analyses of the hydro-meteorological data base of a catchment. Soil moisture storage components, depletion coefficients, and other appropriate parameters are adjusted based on the results of trial-and-error simulations. An automatic-parameter optimization program utilizing a direct-search technique is used to make final modifications to the parameter values. The calibration procedures and results of a case study are presented in the paper to show how the techniques are applied. Current and important future research areas, including initial parameter estimation, physically based automatic parameter optimization, interactive computer graphics applications, and estimation theory techniques such as maximum likelihood and stochastic approximations, are discussed.
Firstly, initial parameter estimates were made from recession analysis of observed runoff. Secondly, the parameters were calibrated individually in an iteration loop starting with the snow routine, over the soil routine and finally the runoff-response function. This was done by minimizing different objective functions for different parameters and only over subperiods where the parameters were active. Approximately 350 objective function evaluations were needed to find the optimal parameter set, which resulted in a computer time of about 17 hours on a 386 processor PC for a 10-yr calibration period. -from Author
EPA’S Storm Water Management Model (SWMM) simulates all aspects of the hydrologic and quality cycles. Using expert system technology, an interactive user-support framework has been developed to automate the calibration of the runoff block. It acts as a front end to assist the user in the initial estimation of the parameter values and in building the SWMM input files. It interprets the simulation results and suggests some useful adjustments in the value of significant parameters thus reducing the user’s time and effort. For the interpretation of simulation results, production rules are employed to help the user decide what parameters need to be adjusted. Some heuristics have been developed to evaluate the new parameter values. The combination of simulation techniques and expert system methodologies facilitates the use of sophisticated models such as SWMM.
A case study is presented which illustrates some of the error analysis,
sensitivity analysis, and parameter estimation procedures reviewed in
the first part of this paper. It is shown that those procedures, most of
which come from statistical nonlinear regression theory, are invaluable
in interpreting errors in precipitation-runoff modeling and in
identifying appropriate calibration strategies.