No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
GA-PARSIMONY: A GA-SVR approach with feature selection and parameter optimization to obtain parsimonious solutions for predicting temperature settings in a continuous annealing furnace
Abstract
This article proposes a new genetic algorithm (GA) methodology to obtain parsimonious support vector regression (SVR) models capable of predicting highly precise setpoints in a continuous annealing furnace (GA-PARSIMONY). The proposal combines feature selection, model tuning, and parsimonious model selection in order to achieve robust SVR models. To this end, a novel GA selection procedure is introduced based on separate cost and complexity evaluations. The best individuals are initially sorted by an error fitness function, and afterwards, models with similar costs are rearranged according to model complexity measurement so as to foster models of lesser complexity. Therefore, the user-supplied penalty parameter, utilized to balance cost and complexity in other fitness functions, is rendered unnecessary. GA-PARSIMONY performed similarly to classical GA on twenty benchmark datasets from public repositories, but used a lower number of features in a striking 65% of models. Moreover, the performance of our proposal also proved useful in a real industrial process for predicting three temperature setpoints for a continuous annealing furnace. The results demonstrated that GA-PARSIMONY was able to generate more robust SVR models with less input features, as compared to classical GA.
... In solar radiation estimate problems, the employment of a GA parsimonious algorithm in association with ML approaches is scarce in the literature. Only a few researchers (Sanz-Garcia et al. 2015;Martinez-de-Pison et al. 2019, 2021 have combined regression techniques with the parsimonious algorithm. To build robust SVR models, (Sanz-Garcia et al. 2015) combines feature selection, model tuning, and parsimonious model selection. ...
... Only a few researchers (Sanz-Garcia et al. 2015;Martinez-de-Pison et al. 2019, 2021 have combined regression techniques with the parsimonious algorithm. To build robust SVR models, (Sanz-Garcia et al. 2015) combines feature selection, model tuning, and parsimonious model selection. For this purpose, an innovative GA selection process tied to specific cost and complexity assessments is presented. ...
... Finally, the population is exposed to a random mutation operator, with the best two individuals being excluded. Figure 8 portrays the flowchart referring to the H-GAPRFR optimization of the machine learning model tuning (Sanz-Garcia et al. 2015). P 2 is increased (P 2 = P 2 + 1 = 4), but the algorithm switches back positions in the following way: P 1 = P 1 + 1 = 2 and P 2 = P 1 + 1 = 3 as P 2 hits G 2 . ...
Machine learning has sparked a wide set of solar prediction experiments due to its recent success, which is one of the most common solutions for solar irradiance forecasting problems specifically. However, while using machine learning regression algorithms, additional attention must be paid to feature selection as well as effective parameter optimization. As a result, this work provides a parsimonious based genetic algorithm that incorporates feature selection integrated with random forest regression. The impact of wind speed on the solar irradiance problem has also been investigated. The performance of the proposed H-GAPRFR algorithm is tested for the location Madurai, India, with eight meteorological data variables over a year and has been validated through statistical metrics such as RMSE, MAE, and coefficient of Determination. In comparison to conventional Support Vector Regression (SVR) and Random Forest (RF) regression techniques, the suggested H-GAPRFR model reduced RMSE by 64.18% and 7.43%, respectively. Second, the suggested H-GAPRFR algorithm was used to investigate the effect of wind speed. From the analysis, it is clear that the proposed model considering wind speed improves the prediction accuracy by further reducing RMSE and MAE by 10.2% and 6.5%, respectively. As a result, it is indicated that in the problem of solar irradiance estimation, the parsimonious model with feature selection can produce improved prediction accuracy.
... In this context, we present GAparsimony [7], a public R package for searching robust and parsimonious models. This library implements previously published GA-PARSIMONY methodology [10,13] that uses genetic algorithms (GA) to search robust and parsimonious models with FS, HO. and parsimonious model selection (PMS). The objective of this paper is to describe the use of this new package for searching parsimonious models. ...
... GAparsimony [7] is a R package for implementing GA-PARSIMONY methodology [10,13] to search accurate parsimonious models (PM) by combining feature selection (FS), hyperparameter optimization (HO), and parsimonious model selection (PMS). ...
... GAparsimony [10,13] uses a GA-based optimization method with a similar flowchart to other classical GA methods. The main difference is that method selects best individuals in two separated cost and complexity evaluations. ...
Nowadays, there is an increasing interest in automating KDD processes. Thanks to the increasing power and costs reduction of computation devices, the search of best features and model parameters can be solved with different meta-heuristics. Thus, researchers can be focused in other important tasks like data wrangling or feature engineering. In this contribution, GAparsimony R package is presented. This library implements GA-PARSIMONY methodology that has been published in previous journals and HAIS conferences. The objective of this paper is to show how to use GAparsimony for searching accurate parsimonious models by combining feature selection, hyperparameter optimization, and parsimonious model search. Therefore, this paper covers the cautions and considerations required for finding a robust parsimonious model by using this package and with a regression example that can be easily adapted for another problem, database or algorithm.
... First, a GP (with UCB as an acquisition function) is used to obtain the best HPO setting (according to the RMSE), considering the full set of features. Next, a variant of GA (GA-PARSIMONY, Sanz-García et al. 2015) is used to select the best features of the problem, given the hyperparameters obtained in the first step. In this way, the final model has high accuracy and lower complexity (i.e., fewer features), and optimization time is significantly reduced. ...
... However, using a metamodel to reduce the number of HP configurations that need to be evaluated does not ensure a lower execution time. For instance, the hybrid algorithm GP + GA_Parsimony (Sanz-García et al. 2015) tries to optimize both hyperparameters and features used to train the ML model; the running time remains high, however, as the feature selection is performed in a separate phase after the HPO has been performed: this leads to a drastic increase in the number of HP configurations evaluated, compared with other algorithms such as NSGA-II and GPbased optimization. ...
Hyperparameter optimization (HPO) is a necessary step to ensure the best possible performance of Machine Learning (ML) algorithms. Several methods have been developed to perform HPO; most of these are focused on optimizing one performance measure (usually an error-based measure), and the literature on such single-objective HPO problems is vast. Recently, though, algorithms have appeared that focus on optimizing multiple conflicting objectives simultaneously. This article presents a systematic survey of the literature published between 2014 and 2020 on multi-objective HPO algorithms, distinguishing between metaheuristic-based algorithms, metamodel-based algorithms and approaches using a mixture of both. We also discuss the quality metrics used to compare multi-objective HPO procedures and present future research directions.
... Machine learning methods, such as support vector machine [15], random forest [16], extreme learning machine [2], XGBoost [17], and deep learning [1], are widely used in various prediction problems in iron and steel production process. They are also combined with evolutionary algorithms [18] to improve their prediction performance. Ge [19] and Yin et al. [20] have reviewed data-driven methods used in the steel industry. ...
... If too few ones are selected, the relationship between input and response variables cannot be fully described and thus causing poor prediction. If too many ones are selected, the interpretability (as introduced in Section I) and generalization capability (due to overfitting) of a model would be weakened [18]. Besides, collinearity caused by redundant variables may reduce the accuracy of regression coefficient estimates in a prediction model. ...
Width deviation is an important metric for evaluating the quality of a hot-rolled strip in steel production systems. This paper considers a width deviation prediction problem and proposes a Machine-learning and Genetic-algorithm-based Hybrid method named MGH to obtain a prediction model. Existing work mainly focuses on high prediction accuracy, while ignoring interpretability. This work aims to build a prediction model that can make a good trade-off between two industry-required criteria, i.e., prediction accuracy and interpretability. It first collects some process variables in a hot rolling process and includes them as well as some constructed variables in a feature pool. Then we propose MGH to find representative variables from it and build a prediction model. MGH results from the integration of hierarchical clustering, genetic algorithm, and generalized linear regression. In detail, hierarchical clustering is applied to divide variables into clusters. Genetic algorithm and generalized linear regression are innovatively combined to select a representative variable from each cluster and develop a prediction model. The computational experiments conducted on both industrial and public datasets show that the proposed method can effectively balance prediction accuracy and interpretability of its resulting model. It has better overall performance than the compared state-of-the-art models.
... On way to solve the above problems of FTS is to generate a parsimonious FTS model. As claimed in Sanz-García, Fernández-Ceniceros, Antonanzas-Torres, Pernia-Espinoza, and Martinez-de Pison (2015) , the less complex model tends to perform better on test data. Some researchers apply genetic algorithm (GA) to generate parsimonious models and optimize hyper-parameters at the same time ( Sanz-García et al., 2015;Sanz-Garcia, Fernández-Ceniceros, Fernández-Martínez, & Martínez-De-Pisón, 2014 ). ...
... As claimed in Sanz-García, Fernández-Ceniceros, Antonanzas-Torres, Pernia-Espinoza, and Martinez-de Pison (2015) , the less complex model tends to perform better on test data. Some researchers apply genetic algorithm (GA) to generate parsimonious models and optimize hyper-parameters at the same time ( Sanz-García et al., 2015;Sanz-Garcia, Fernández-Ceniceros, Fernández-Martínez, & Martínez-De-Pisón, 2014 ). However, there is still no work being done in designing parsimonious FTS model. ...
This paper proposes a novel modelling structure to ensure the parsimony of fuzzy time series (FTS) models while retaining certain level of out-of-sample accuracy. A parsimonious FTS model requires multiple optimizations of hyper-parameters such as time lags and partitioning which consists of the number of fuzzy sets, the partitioning type and the membership functions. In the vast literature of fuzzy time series, hyper-parameter optimization is usually ignored. In addition to that, optimization process for the hyper-parameters is also not presented properly. In this study, a parsimonious FTS modelling approach is introduced by using genetic algorithm (GA). Three major innovations are proposed: (1) Hyper-parameters of FTS structure are optimized to eliminate subjective preferences with the help of GA. Some of those parameters are never optimized or simply ignored in the past research. (2) The set of hyper-parameters is optimized subject to highest accuracy in validation set data and model’s complexity. (3) For achieving sparsification and accuracy simultaneously at reasonable computation time, a two-stage GA optimization is run to search for higher accuracy and lower complexity consecutively. Empirical studies are conducted on two types of datasets. Prices of liquid bulk cargo carriers (i.e. tanker) and secondhand ship have been predicted using the proposed approach. Potential benchmarks as well as a simple Nave forecast have been compared to the proposed model for validation based on mean absolute scaled error and root mean squared error.
... Hence, its value has to be manually set prior the execution of the optimization methodology. In this context, we introduced a new GA-based optimization methodology, named GA-PARSIMONY [22] . Our aim is to automate the optimization process when the complexity of the model is taken into account by getting rid of the penalty parameter . ...
... Specifically, feature selection and parameter tuning are simultaneously conducted in order to obtain accurate but parsimonious models. The methodology is referred by authors as GA-PARSIMONY [22] , as it combines the traditional GA structure (see Fig. 1 ) for FS and PT, with the selection of parsimonious models. Here, the main novelty compared to existing proposals is the elimination of the penalty parameter from the fitness function. ...
Most proposed metaheuristics for feature selection and model parameter optimization are based on a two-termed function. Their main drawback is the need of a manual set of the parameter that balances between the loss and the penalty term. In this paper, a novel methodology referred as the GA-PARSIMONY and specifically designed to overcome this issue is evaluated in detail in thirteen public databases with five regression techniques. It is a GA-based meta-heuristic that splits the classic two-termed minimization functions by making two consecutive ranks of individuals. The first rank is based solely on the generalization error, while the second (named ReRank) is based on the complexity of the models, giving a special weight to the complexity entailed by large number of inputs.
... In this context, we proposed the GA-PARSIMONY [23], a Genetic Algorithm (GA) methodology whose main objective is to obtain accurate parsimonious models. It optimizes HO, DT, and FS with a new model selection process based on a double criteria that considers accuracy and complexity in two steps. ...
... GA-PARSIMONY is a SC methodology based on Genetic Algorithms (GA) to automatically obtain precise overall parsimonious models [23]. Basically, this proposal includes HO, FS, and DT in the GA optimization process. ...
This paper presents a hybrid methodology that combines Bayesian Optimization (BO) with a constrained version of the GA-PARSIMONY method to obtain parsimonious models. The proposal is designed to reduce the computational efforts associated to the use of GA-PARSIMONY alone. The method is initialized with BO to obtain favorable initial model parameters. With these parameters, a constrained GA-PARSIMONY is implemented to generate accurate parsimonious models using feature reduction, data transformation and parsimonious model selection. Finally, a second BO is run again with the selected features. Experiments with Extreme Gradient Boosting Machines (XGBoost) and six UCI databases demonstrate that the hybrid methodology obtains analogous models than the GA-PARSIMONY but with a significant reduction on the execution time in five of the six datasets.
... The last one is what differentiates GAparsimony from other similar multi-objective methods. Indeed, previous experiments with other common techniques have shown that the simultaneous optimization of both performance and complexity usually produces an evolution of solutions that are not optimal [27]. In GAparsimony, the selection of the best individuals or solutions in each generation is carried out following a parsimony search principle that consists of two consecutive steps: a pre-selection of the most accurate models and, among those with a similar cost, a promotion to higher positions of those with less complexity. ...
The evaluation of student projects is a difficult task, especially when they involve both a technical and a creative component. We propose an AI-based methodology to help in the evaluation of complex projects in Engineering and Computer Science courses. This methodology is intended to evaluate the assessment process itself allowing to analyze the influence of each variable in the final grade, to discover possible biases, inconsistencies and discrepancies, and to generate appropriate rubrics that help to avoid them. As an example of its application, we consider the evaluation of the projects submitted in an undergraduate introductory course on Computer Science. Using data collected from the evaluation during five academic years, we follow the proposed methodology to create AI models and analyze the main variables which are involved in the assessment of the projects. The proposed methodology can be applied to other courses and degrees, where both technical and creative components are considered to evaluate the projects.
... Previous studies have shown that the hyperparameters of SVR (kernel function, penalty factor, etc) have significant influence on the performance of any SVR-based model. The SVR model with the proper hyperparameters selected by using particle swarm optimization (PSO) and genetic algorithm (GA) search approaches significantly contributes to the robustness of the models (Sanz-Garcia et al., 2015;Hasanipanah et al., 2017). Thus, before applying SVR model, other algorithms, e.g., genetic algorithm (GA) and particle swarm optimization (PSO), are needed to optimize the SVR model parameters. ...
Coastal soil is an important reserve land resource. The accurate prediction of soil hydraulic properties plays an important role in understanding the improvement of soil properties after reclamation in coastal areas. With the development of modern mathematical theory, the pore-solid fractal (PSF) model has become an important basis for simulating soil hydraulic properties. The accuracy of PSF model depends on the accurate acquisition of changepoints and fractal dimensions. Previous studies have found that micro-CT scanning combined with image processing could accurately obtain fractal dimensions. However, we still need soil water retention data to determine changepoints, which greatly limits the application of fractal models for estimating soil hydraulic properties. In this case, we determined the relationship between changepoints and soil physical and chemical properties and established a genetic algorithm-support vector regression (GA-SVR) model to predict changepoints. Then an improved PSF model was adopted to predict soil water retention curve in coastal areas of Jiangsu Province based on image processing. The results showed that soil physical and chemical properties changed the soil water movement and changepoints by affecting soil pore distribution. In coastal saline soil, four parameters (BD, silt, EC, SOM) were selected to predict changepoints. The mass fractal dimension (Dm) was mainly influenced by the porosity and the heterogeneity of pore space. The porosity, pore diameter and specific surface area were the determining factors of DS value. Reclamation activities in coastal reclamation area changed Dm value but had no clear effect on DS value. By combining predicted changepoints, measured water content at the suction of changepoints and fractal dimensions, the accuracy of the PSF model had been greatly improved.
... In our experiments, 10 repeated twofold cross validation is used in the implementation of RFE, GAFS and GAP for maximum similarity with the sampling settings in SOFS. We use the RFE and GAFS available with the caret package of R (Kuhn 2008) and GAP with the package GAparsimony (Sanz- Garcia et al. 2015). It is also worth noting that the learning algorithms that can be used within the existing RFE and GAFS are limited. ...
Non-informative or redundant features with big data can significantly reduce the performance of any machine learning problem. They render the model training costly and the model interpretability weak. Traditional feature selection methods, particularly wrapper methods, often performed using greedy search, are susceptible to suboptimal solutions, selection bias, and high variability due to noise in the data. Our simulation optimization framework seeks to identify the best subset of features by utilizing resamples of the training and test set, where the random holdout errors produce the simulation outputs. The resulting feature subsets are more reliable since they perform well on several resampled datasets. Our experiments on four actual and simulated datasets indicate the fixed sampling approach’s competitive advantages in various performance metrics. Further, we develop adaptive sampling strategies for large enough datasets, where the number of training and test resamples vary for each solution. Adaptive sample sizes reach the same quality level of recommended feature subsets but significantly faster than the fixed sample size version.
... Similar to our previous works, in previous works we also used GA-PARSIMONY [14], a method to search for parsimonious solutions with GA by optimizing HO, FS, and parsimonious model selection. ...
We present PSO-PARSIMONY, a new methodology to search for parsimonious and highly accurate models by means of particle swarm optimization. PSO-PARSIMONY uses automatic hyperparameter optimization and feature selection to search for accurate models with low complexity. To evaluate the new proposal, a comparative study with Multilayer Perceptron algorithm was performed by applying it to predict three important parameters of the force-displacement curve in T-stub steel connections: initial stiffness, maximum strength, and displacement at failure. Models optimized with PSO-PARSIMONY showed an excellent trade-off between goodness-of-fit and parsimony. Then, the new proposal was compared with GA-PARSIMONY, our previously published methodology that uses genetic algorithms in the optimization process. The new method needed more iterations and obtained slightly more complex individuals, but it performed better in the search for accurate models.
... The GAparsimony methodology used in this study is a genetic algorithm (GA) that conducts parsimonious model selection (PMS) [23]. It has been successfully applied in a range of contexts such as steel industrial processes [24], hotel room booking forecasting [25], mechanical design [26], and solar radiation forecasting [27]. ...
Healthcare facilities consume massive amounts of energy. This study outlines a methodology to enhance energy efficiency and solve common problems in hospital cooling-water systems, since hospitals are the most energy- intensive type of building. Building Management Systems (BMS) are a widely used technique to control and monitor all the different energy facilities contained in hospitals. Proper setup and upgrades can resolve inefficien- cies and existing problems. The methodology described herein addresses the general cooling system adjustments in three main areas: control system (CS), data acquisition system (DAS), and physical system (PS). An innovative feature incorporated in this methodology is the cooling demand model integrated into the CS, which is capable of forecasting and transmitting a schedule for maximum thermal energy requirements to the BMS a day in ad- vance, thereby anticipating decisions and scheduling energy generation and maintenance operations. During the process of developing the cooling demand model, various machine learning models were trained. This process consisted of searching for low-complexity models using a methodology called GAparsimony. This methodology uses genetic algorithms to search for highly precise, robust models that use a low input. The final model consisted of a weighted combination of Artificial Neural Network (ANN) and Support Vector Regression (SVR) models. The energy savings obtained thanks to this methodology are estimated to be between 7% and 10% per year. The energy plant improved its performance and chiller starts were reduced by 82.5%. It should also be noted that this study was affected by the recommendations for increased ventilation due to the COVID-19 pandemic, which en- tailed at 22.4% increase in energy consumption in 2020. The methodology was developed and tested successfully in a real hospital BMS between 2017 and 2019; the model was finally integrated in 2020.
... Moreover, there is no unique method to select the best values of SVR parameters. At present, empirical formula, grid search Fayed and Atiya 2019) and swarm intelligent optimization algorithms such as particle swarm optimization (PSO) (Hasanipanah et al. 2016;Liu et al. 2017) and genetic algorithm (GA) (Oyehan et al. 2018;Sanz-Garcia et al. 2015) are generally used to select SVR parameters, but there are some shortcomings such as easy to fall into local optimal value or poor performance existed in these algorithms to solve practical problems. Therefore, the IGS algorithm is proposed to solve the problem of the SVR parameters selection for predicting SPAD values in rice leaves. ...
Parameter optimization is an important step for support vector regression (SVR), since its prediction performance greatly depends on values of the related parameters. To solve the shortcomings of traditional grid search algorithms such as too many invalid search ranges and sensitivity to search step, an improved grid search algorithm is proposed to optimize SVR for prediction. The improved grid search (IGS) algorithm is used to optimize the penalty parameter and kernel function parameter of SVR by automatically changing the search range and step for several times, and then SVR is trained for the optimal solution. The available of the method is proved by predicting the values of soil and plant analyzer development (SPAD) in rice leaves. To predict SPAD values more quickly and accurately, some dimension reduction methods such as stepwise multiple linear regressions (SMLR) and principal component analysis (PCA) are processed the training data, and the results show that the nonlinear fitting and prediction performance of accuracy of SMLR-IGS-SVR and PCA-IGS-SVR are better than those of IGS-SVR.
... Inza et al. [22] Genetic algorithms and estimation of distribution algorithms 2001 Zhang and Sun [23] Tabu search and Pattern classifier 2002 Chen [24] Improved branch and bound algorithm 2003 Zhu and Guan [25] Genetic algorithm 2004 Tsymbal et al. [26] Search strategy and dynamic integration of classifiers 2005 Paterlini and Krink [27] Differential evolution and particle swarm optimization 2006 Wang et al. [28] GA and PSO 2007 Huang and Dun [29] PSO-SVM 2008 Nemati et al. [30] ACO-GA Hybrid and Hierarchical classification 2009 Vieira et al. [31] Ant colony optimization 2010 Pourhabibi et al. [32] GA and Guided Evolutionary Simulated Annealing (GESA) 2011 Kabir et al. [33] Ant colony optimization and Neural network 2012 Vieira et al. [34] PSO and SVM 2013 Nazir et al. [35] GA-PSO and gender classification 2014 Sanz-García et al. [36] GA-SVM 2015 Hancer et al. [37] Artificial bee colony and particle swarm optimization 2015 Ghaemi and Feizi-Derakhshi [38] Forest Optimization Algorithm (FOA) and KNN classifier 2016 Sheikhpour et al. [39] PSO and kernel density classifier 2016 ...
Land cover classification is one of the most important applications of POLSAR images. In this paper, a hybrid biogeography-based optimization support vector machine (HBBOSVM) has been introduced to classify POLSAR images of RADARSAT 2 in band C acquired from San Francisco, USA. The main purpose of this classification is to minimize the number of features and maximize classification accuracy. The proposed method consists of three main steps: preprocessing, feature selection and classification. As preprocessing, radiometric calibration, speckle reduction and feature extraction have been performed. In the proposed HBBO, the combination of onlooker bee of artificial bee colony (ABC) and migration operator of biogeography-based optimization has been applied in order to optimal feature selection. Then, SVM has been used to classify the pixels into specific labels of land-covers. The ground truth samples have been generated by google earth image, Pauli RGB image, high resolution image and national land cover database (NLCD 2006). The performance of HBBOSVM has been compared with BBOSVM, ABCSVM, particle swarm optimization support vector machine (PSOSVM) and the results of previous studies. In addition, the performance of HBBO is evaluated upon 20 well-known benchmark problems. According to the obtained results, the overall accuracy and average accuracy of HBBOSVM are 96.01% and 93.37% respectively which is the best result in comparison with other results. The HBBOSVM has better performance than other algorithms in terms of overall accuracy, kappa coefficient, average accuracy, convergence trend, and stability. In addition, the HBBO can be considered as a successful meta-heuristic for benchmark problems. This paper displays that the combined approach of optimization and machine learning methods provides powerful results.
... The GA-PARSIMONY optimization technique was used to generate parsimonious models by simultaneously preprocessing skewed data, tuning the necessary parameters and proceeding to a feature selection (Sanz-Garcia et al., 2015;Javier Martinez-de-Pison et al., 2016;Urraca et al., 2018). The "GAparsimony" package was utilized (Martinez-de-Pison, 2018) to tune the parameters of the NN-SGD model (introduced by the "caret" package (Kuhn, 2017)), including the number of processing units in the hidden layer, the learning rate, the learning rate decay, the momentum and the batch size, whereas the l2 regularization and the RMSE gradient scaling were kept constant. ...
... In order to select the best structure of the forecasting model, an optimization methodology was used based on the genetic algorithm (GA) with advanced generalization capabilities. This methodology is the GA-PARSIMONY [71], which allows the selection of parsimonious models. The main difference of this methodology with respect to the conventional GAs is a rearrange in the ranking of the individuals based on their complexities, so that individuals with less complexity (in this case, models with a less complex structure) are promoted to the best position of each generation. ...
The development of Short-Term Forecasting Techniques has a great importance for power system scheduling and managing. Therefore, many recent research papers have dealt with the proposal of new forecasting models searching for higher efficiency and accuracy. Several kinds of artificial intelligence (AI) techniques have provided good performance at predicting and their efficiency mainly depends on the characteristics of the time series data under study. Load forecasting has been widely studied in recent decades and models providing mean absolute percentage errors (MAPEs) below 5% have been proposed. On the other hand, short-term generation forecasting models for photovoltaic plants have been more recently developed and the MAPEs are in general still far from those achieved from load forecasting models. The aim of this paper is to propose a methodology that could help power systems or aggregators to make up for the lack of accuracy of the current forecasting methods when predicting renewable energy generation. The proposed methodology is carried out in three consecutive steps: (1) short-term forecasting of energy consumption and renewable generation; (2) classification of daily pattern for the renewable generation data using Dynamic Time Warping; (3) application of Demand Response strategies using Physically Based Load Models. Real data from a small town in Spain were used to illustrate the performance and efficiency of the proposed procedure.
... In recent years, there is an increasing tendency to create methods to automate modeling processes with hyperparameter optimization (HO), and feature selection (FS), in order to reduce the human effort involved in these timeconsuming tasks [19] [20]. Among the currently available methods, GAparsimony [21] is a genetic algorithm (GA) methodology for searching for parsimonious models. It is designed specifically to work with small datasets. ...
Hospitals are massive consumers of energy, and their cooling systems for HVAC and sanitary uses are particularly energy-intensive. Forecasting the thermal cooling demand of a hospital facility is a remarkable method for its potential to improve the energy efficiency of these buildings. A predictive model can help forecast the activity of water-cooled generators and improve the overall efficiency of the whole system. Therefore, power generation can be adapted to the real demand expected and adjusted accordingly. In addition, the maintenance costs related to power-generator breakdowns or ineffective starts and stops can be reduced. This article details the steps taken to develop an optimal and efficient model based on a genetic methodology that searches for low-complexity models through feature selection, parameter tuning and parsimonious model selection. The methodology, called GAparsimony, has been tested with neural networks, support vector machines and gradient boosting techniques. This new operational method employed herein can be replicated in similar buildings with comparable water-cooled generators, regardless of whether the buildings are new or existing structures.
... Optimization, computational intelligence, and genetic algorithms have been applied in other furnace problem situations. For example, the use in the context of reheating furnace scheduling can be pointed out [17][18][19] to predict highly precise set points in a continuous annealing furnace [20] and to adjust the temperatures of the inner zones of a pusher furnace [21]. ...
Metallurgy industries often use steel billets, at a proper temperature, to achieve the desired metallurgical, mechanical, and dimensional properties of manufactured products. Optimal operation of steel billet reheating furnaces requires the minimization of fuel consumption while maintaining a homogeneous material thermal soak. In this study, the operation of a reheating furnace is modeled as a nonlinear optimization problem with the goal of minimizing fuel cost while satisfying a desired discharge temperature. For this purpose, a genetic algorithms approach is developed. Computational simulation results show that it is possible to minimize costs for different charge temperatures and production rates using the implemented method. Additionally, practical results are validated with actual data, in a specific scenario, showing a reduction of 3.36% of fuel consumption.
... This paper tries to get the predictive value of hardness of wheat samples based on the support vector regression method, and in order to obtain the root mean square error as the performance evaluation criteria of the feature subset of the evaluation. ACO algorithm combined with the support vector regression applies in wheat hardness by the optimized selection of characteristics wave band of the NIR hyperspectral analysis, which can realize global search of spectral variables and set up the multivariate calibration model with the characteristics of high precision and strong stability [8,9]. ...
This paper presents a new and improved method that ant colony optimization (ACO) algorithm is combined with the support vector regression for band selection. The method is applied to the prediction research of wheat grain hardness, and tries to detect the feasibility of the forecasting ability. The optimized selection of characteristic wave band is the key link of the near infrared (NIR) hyperspectral analysis technology of wheat hardness. Experimental results showed that eleven characteristic wave band sub-intervals were selected from thirty spectral intervals by the algorithm, including 86 wave points. The selected wave band sub-interval were respectively 902.1 to 931.8 nm, 968.7 to 1027.5 nm, 1119.0 to 1143.4 nm, 1174.1 to 1275.5 nm, 1174.1 to 1275.5 nm, 1626.0 to 1647.6 nm and 1626.0 to 1647.6 nm. After using the optimized parameter in the spectral information forecasts and analyzes by the support vector regression. Prediction performances of regression models are assessed by calculating the estimated root mean square errors of cross-validation(RMSECV) the root mean square errors of prediction (RMSEP) and the correlation coefficient(R). The results showed that the estimated RMSECV and Rcv values were respectively 0.0382, and 0.9810 for the training set, the estimated RMSEP and RP values were respectively 0.0590, and 0.9544 for the validation set. Compared with the full spectrum of partial least squares (PLS), interval partial least squares (IPLS) algorithm, it simultaneously reduces the number of certain variables used in the model and increases in the prediction ability and the precision, and it can better reflect optimization model of the wave band. It is confirmed that the ACO method applied to the prediction research of the grain kernels is feasible.
... Several authors have reported SC applications to different real fields where feature selection (FS), model parameters optimization (MPO) or data transformation (DT) are optimized with several bioinspired optimization methods [12,8,20,5,9,1,4,25]. In this context, we proposed a SC methodology named GA-PARSIMONY in [19] to automatically obtain good overall parsimonious models. This methodology uses Genetic Algorithms (GA) and generates parsimonious models, while performing the preprocessing of skewed data, parameter tuning and feature selection at a time. ...
EXtreme Gradient Boosting (XGBoost) has become one of the most successful techniques in machine learning competitions. It is computationally efficient and scalable, it supports a wide variety of objective functions and it includes different mechanisms to avoid over-fitting and improve accuracy. Having so many tuning parameters, soft computing (SC) is an alternative to search precise and robust models against classical hyper-tuning methods. In this context, we present a preliminary study in which a SC methodology, named GA-PARSIMONY, is used to find accurate and parsimonious XGBoost solutions. The methodology was designed to optimize the search of parsimonious models by feature selection, parameter tuning and model selection. In this work, different experiments are conducted with four complexity metrics in six high dimensional datasets. Although XGBoost performs well with high-dimensional databases, preliminary results indicated that GA-PARSIMONY with feature selection slightly improved the testing error. Therefore, the choice of solutions with fewer inputs, between those with similar cross-validation errors, can help to obtain more robust solutions with better generalization capabilities.
... Typically in the literature, the proposed algorithm is applied to forecast the actual data based on the historical data reserved for testing to show the benefits and effectiveness of the proposed algorithm as done in [50,38,29]. As such, the FPNN is applied to forecast OPEC petroleum consumption based on the data reserved for testing to show the forecast ability of the FPNN. ...
A New Approach for Forecasting OPEC Petroleum Consumption Based on Neural Network Train by using Flower Pollination Algorithm
... Typically in the literature, the proposed algorithm is applied to forecast the actual data based on the historical data reserved for testing to show the benefits and effectiveness of the proposed algorithm as done in Sanz-Garcia et al., 2015;Mollaiy-Berneti, 2015). As such, the FPNN is applied to forecast OPEC petroleum consumption based on the data reserved for testing to show the forecast ability of the FPNN. ...
Petroleum is the live wire of modern technology and its operations, with economic development being positively linked to petroleum consumption. Many meta-heuristic algorithms have been proposed in literature for the optimization of Neural Network (NN) to build a forecasting model. In this paper, as an alternative to previous methods, we propose a new flower pollination algorithm with remarkable balance between consistency and exploration for NN training to build a model for the forecasting of petroleum consumption by the Organization of the Petroleum Exporting Countries (OPEC). The proposed approach is compared with established meta-heuristic algorithms. The results show that the new proposed method outperforms existing algorithms by advancing OPEC petroleum consumption forecast accuracy and convergence speed. Our proposed method has the potential to be used as an important tool in forecasting OPEC petroleum Accepted for Publication in Applied Soft Computing – Elsevier, 2.8 Impact Factor 2 consumption to be used by OPEC authorities and other global oil-related organizations. This will facilitate proper monitoring and control of OPEC petroleum consumption.
The feature selection problem has become a key undertaking within machine learning. For classification problems, it is known to reduce the computational complexity of parameter estimation, but also it adds an important contribution to the explainability aspects of the results. In this paper, a genetic algorithm for feature selection is proposed. The importance, as well as the effectiveness of features selected by each individual, is evaluated by using decision trees. The feature importance indicated by the decision tree is used during selection and recombination. The tree inducted by the best individual in the population is used for classification. Numerical experiments illustrate the behavior of the approach.Keywordsgenetic algorithmsdecision treesfeature selection
Studying the distribution patterns and controlling mechanisms of soil organic carbon (SOC) based on the comprehensive performance of vegetation restoration and check dams at the watershed scale is important for understanding carbon cycling processes in nature. Two typical watersheds (Xinshui River and Zhujiachuan) of the Loess Plateau were selected to evaluate the factors affecting the change in SOC content, and then the key factors were considered in the genetic algorithm‐support vector regression (GA‐SVR) model to predict SOC content. The results showed that the topography, vegetation and soil characteristics had significant effects on the SOC content in the upland hillslopes, while the SOC content in the check dams was significantly affected by depth and soil characteristics. The soil organic carbon storage (TSOC) in the check dams could be evaluated and predicted by the vegetation index (NGRDI) and area of the subwatershed. The GA‐SVR model had good prediction accuracy and stable performance in predicting SOC content. According to the model simulation results, bulk density (BD), mean weight diameter (MWD), elevation, NGRDI, clay ratio (CR) and slope could be used to predict the surface SOC content of the Loess Plateau. Furthermore, depth, CR, MWD, BD and median particle size (D50) could be applied in the model to predict the SOC content at different depths in the check dams. This study explored the potential control factors of SOC content and predicted SOC content from multiple angles, which can provide basic support for the study of the carbon sequestration on the Loess Plateau. This article is protected by copyright. All rights reserved.
Hyperparameter optimization (HPO) is a necessary step to ensure the best possible performance of Machine Learning (ML) algorithms. Several methods have been developed to perform HPO; most of these are focused on optimizing one performance measure (usually an error-based measure), and the literature on such single-objective HPO problems is vast. Recently, though, algorithms have appeared which focus on optimizing multiple conflicting objectives simultaneously. This article presents a systematic survey of the literature published between 2014 and 2020 on multi-objective HPO algorithms, distinguishing between metaheuristic-based algorithms, metamodel-based algorithms, and approaches using a mixture of both. We also discuss the quality metrics used to compare multi-objective HPO procedures and present future research directions.
Ammonium perchlorate (AP) occupies an important position in solid propellants. However, the increase of moisture content in AP will seriously affect its combustion performance and mechanical properties, resulting in the abnormal use of solid propellants, which may bring serious security risks. There are very few studies on the accurate measurement of AP moisture content, which makes this study of great value and significance. In this paper, the optical parameters of aqueous AP samples were measured by terahertz time domain spectroscopy. Combined with chemometrics, five quantitative prediction models were constructed. The results show that the absorption coefficient based genetic algorithm optimized extreme learning machine model has the best prediction results. This method can achieve accurate prediction for moisture content below 0.1%, and its absolute error is less than 0.001%, and the prediction error of the external verification model of different groups of samples is less than 0.06%. This study provides a reliable basis for rapid, accurate and non-destructive detection of the moisture content of ammonium perchlorate and similar substances.
Synthetic aperture radar (SAR) features has been demonstrated to have the potential to improve forest above ground biomass (AGB) estimation accuracy using polarimetric information. Genetic algorithms (GAs) have been successfully implemented in optimal feature identification, while support vector regression (SVR) has great robustness in parameter estimation. The use of combined GAs and SVR can improve the accuracy of forest AGB estimation through simultaneously identifying the optimal SAR features and the SVR model parameters. In this paper, 14 SAR polarimetric features were extracted from C-band and L-band full-polarization SAR images as input SAR features, respectively. C-band data was acquired on GaoFen-3 mission, we also call it GF-3 image. L-band data was ALOS-2 PALSAR-2 data. Both feature subsets used in the estimation and SVR hyper parameters were optimized by a GA processing, in which the SVR hyper parameters searching field were include 8 different settings of 3 kinds of parameters in total, it means there are 512 different combinations. The results of GA-SVR performance used in the two datasets were presented and compared with two traditional methods: the algorithm of GA feature selection companied with default SVR parameters (GA +Default SVR), and the algorithm of GA feature selection companied with grid searching for SVR parameter selection (GA+Grid SVR). The results showed that the proposed GA-SVR algorithm improved the forest AGB estimation accuracy with cross-validation coefficient (CVC) 80.21% for GF-3 and 70.41% for ALOS-2 PALSAR-2 data.
The use of data-driven models to forecast streamflow has received substantial attention from scholars in recent years. However, systematic studies have not been performed to examine binary metaheuristic wrapper-based input variable selection (BMWIVS) in real-world streamflow forecasting. In this study, we explored binary metaheuristic-based shallow machine learning wrappers for one-step monthly streamflow forecasting using local weather information and global climate indices from three catchments with different hydroclimatic conditions. First, the maximal information coefficient (MIC) was employed to investigate the correlations among the forecasting target, streamflow and candidate input variables, which included both local and global climate information. Then, the BMWIVS models obtained by combining eight binary metaheuristic algorithms, five commonly used shallow machine learning algorithms, two combined filter-based input variable selection (FIVS) methods, and two forecasting methods were examined. Finally, the performance of each model was compared with the performance of typical benchmark models, including the univariate seasonal autoregressive integrated moving average model, five machine learning algorithms with no input variable selection, and five machine learning algorithms that use five different FIVS methods. The experimental results emphasized three significant findings. First, an appropriate input variable selection method should be selected in practice because several examined wrappers were inferior to the benchmark models. Second, the BMWIVS model that combined the regularized extreme learning machine method, binary gray wolf optimizer, FIVS results-based initialization method, and forecasted values averaged over multiple runs yielded the best performance in the three cases studied. Third, the correlations in terms of the MIC between the global climate indices and streamflow were lower than those between local weather information and streamflow, and the best wrapper and FIVS would select more local weather information variables than global climate index variables, which suggests that global climate information can be complementary to local weather information for one-step monthly streamflow forecasting. These findings have remarkable practical applications for forecasting monthly streamflow.
Of all the different types of public buildings, hospitals are the biggest energy consumers. Cooling systems for air conditioning and healthcare uses are particularly energy intensive. Forecasting hospital thermal-cooling demand is a remarkable and innovative method capable of improving the overall energy efficiency of an entire cooling system. Predictive models allow users to forecast the activity of water-cooled generators and adapt power generation to the real demand expected for the day ahead, while avoiding inefficient subcooling. In addition, the maintenance costs related to unnecessary starts and stops and power-generator breakdowns occurring over the long term can be reduced. This study is based on the operations of a real hospital facility and details the steps taken to develop an optimal and efficient model based on a genetic methodology that searches for low-complexity models through feature selection, parameter tuning and parsimonious model selection. The methodology, called GAparsimony, has been tested with neural networks, support vector machines and gradient boosting techniques. Finally, a weighted combination of the three best models was created. The new operational method employed herein can be replicated in similar buildings with similar water-cooled generators.
Nowadays, interest is growing in automating KDD processes. Thanks to the increasing power and decreasing costs of computation devices, the search for the best features and model parameters can be conducted with different meta-heuristics. Thus, researchers can focus on other important tasks like data wrangling or feature engineering. This article details a comparative study of a GAparsimony R package with six model complexity metrics. The objective was to identify an adequate model complexity measure for searching for accurate parsimonious solutions by combining feature selection, hyperparameter optimization, and parsimonious evaluation. This study also includes a regression code example to address some recommended precautions and considerations to find robust parsimonious models. This code can be easily adapted to other problems, databases, or algorithms.
Due to the instability of α type HMX at low concentrations, it belongs to the impurity crystal form. To ensure the functional effectiveness, operational reliability and management safety of HMX, it is necessary to quantify the low content of the unstable α-HMX crystal form in the composite explosive. In this study, low-concentration α-HMX is quantitatively analyzed in a mixture of α- and β-HMX. First, terahertz time-domain spectroscopy (THz-TDS) is used to obtain the absorption spectrum of the α/β-HMX element in the frequency range of 0.2–2.0 THz, and the characteristic frequency is selected. The absorption coefficient data in the frequency band of 0.7–1.3 THz are considered as the sample data for quantitative analysis. Finally, support vector machine (SVM) algorithm is used to establish a regression model, and principal component analysis (PCA) is employed for feature extraction. Grid search (GS), genetic algorithm (GA) and particle swarm optimization (PSO) are utilized for parameter optimization in support vector regression (SVR). These algorithms are combined to establish six regression models, and their effectiveness is assessed. The experimental results show that all the six methods can predict the content of α-HMX components with a small error and a high prediction accuracy. Compared to GA-SVR and PSO-SVR models, the PCA-GA-SVR and PCA-PSO-SVR models exhibit higher prediction accuracy and stability. The test set of the PCA-GA-SVR model reveals an average absolute error of 0.880%. It has the highest prediction accuracy, and the coefficient of determination (R²) reaches 0.9996. This indicates that PCA and SVR can be effectively used in the detection of low-concentration HMX components and can serve as a reliable basis for the quantitative analysis of other explosives.
In this paper, we propose a novel support vector regression (SVR) approach for time series analysis. An efficient forward feature selection strategy has been designed for dealing with high-frequency time series with multiple seasonal periods. Inspired by the literature on feature selection for support vector classification, we designed a technique for assessing the contribution of additional covariates to the SVR solution, including them in a forward fashion. Our strategy extends the reasoning behind Auto-ARIMA, a well-known approach for automatic model specification for traditional time series analysis, to kernel machines. Experiments on well-known high-frequency datasets demonstrate the virtues of the proposed method in terms of predictive performance, confirming the virtues of an automatic model specification strategy and the use of nonlinear predictors in time series forecasting. Our empirical analysis focus on the energy load forecasting task, which is arguably the most popular application for high-frequency, multi-seasonal time series forecasting.
Predicting the delay in servicing incoming ships to ports is crucial for maritime transportation. In this study, we use support vector regression (SVR) in order to accurately predict this delay for ships arriving to the terminal No. 1 of Shahid Rajaee's port in Bandar Abbas. To achieve this goal, a combination of Clonal Selection and Grey Wolf Optimization algorithms (named as CLOGWO) is used for two purposes: (i) selecting the most important features among the features that affect prediction of this delay and (ii) optimizing SVR parameters for a more accurate prediction. Performance of the proposed method was compared with Genetic Algorithm (GA), Clonal Selection (CS), Grey Wolf Optimization (GWO), and Particle Swarm Optimization (PSO) algorithms on the following metrics: correlation, rate of feature reduction, root mean square error (RMSE), and normalized RMSE (NRMSE). Evaluations on Shahid Rajaee dataset showed that the mean value of these metrics in 10 independent runs of the proposed method were 0.867, 74.45%, 0.080, and 9.02, respectively. These results and evaluations on standard datasets indicate that the proposed method provides competitive results with other evolutionary algorithms.
Calculating the water absorption in sublayers based on the on-site monitored water injection profile is the most accurate way to reflect the actual amount of water absorbed by subalyers. However, access to this profile data is scarce and limited due to its high cost of testing. As a result, some well blocks have tested some discontinuous profiles, but other well blocks do not have any profiles. Traditional machine learning can be applied to construct an intelligent surrogate model by training with historical injection profile. This well-trained model can be only used to make a good prediction for well blocks which contribute profile data, not applicable to well blocks without injection profile. In this study, a Joint Distribution Adaption based Extreme Gradient boosting transfer learning approach is presented to predict water absorption of sublayers in water injection well which do not have any injection profile. A handful of observations are obtained from source well block which has tested sufficient injection profiles. Joint Distribution Adaption is first conducted to transfer knowledge from source well block to the target well block which have no injection profile. The transferred dataset with new feature representation is used to constitute the training dataset for target well block. This transferred training dataset then can be feed into the Extreme Gradient Boosting model to construct a water absorption predictive model. The well-trained model can be applied to predicting water absorption of sublayers in injectors which do not have any water injection profile. The proposed approach is demonstrated by applying to two well blocks from SL oilfield, China. Demonstrated results imply that the proposed transfer learning method can be used to dividing water absorption of subalyers in injectors which have no water injection profile data.
In this work, a strategy for automatic lag selection in time series analysis is proposed. The method extends the ideas of feature selection with support vector regression, a powerful machine learning tool that can identify nonlinear patterns effectively thanks to the introduction of a kernel function. The proposed approach follows a backward variable elimination procedure based on gradient descent optimisation, iteratively adjusting the widths of an anisotropic Gaussian kernel. Experiments on four electricity demand forecasting datasets demonstrate the virtues of the proposed approach in terms of predictive performance and correct identification of relevant lags and seasonal patterns, compared to well-known strategies for time series analysis designed for energy load forecasting and state-of-the-art strategies for automatic model selection.
This article presents a hybrid methodology that combines Bayesian optimization (BO) with a constrained version of the GA-PARSIMONY method to obtain parsimony models. The proposal is designed to reduce the sizeable computational effort associated with the use of GA-PARSIMONY alone. The method begins with BO to obtain favorable initial model parameters. Then, with these parameters, a constrained GA-PARSIMONY is implemented to generate accurate parsimony models by using feature reduction, data transformation and parsimonious model selection. Experiments with extreme gradient boosting machines (XGBoost) and ten UCI databases demonstrated that the hybrid methodology obtains models analogous to those of GA-PARSIMONY while achieving significant reductions in elapsed time in eight out of ten datasets.
This article presents a hybrid methodology in which a KDD scheme is optimized to build accurate parsimonious models. The methodology tries to find the best model by using genetic algorithms to optimize a KDD scheme formed with the following stages: feature selection, transformation of the skewed input and the output data, parameter tuning and parsimonious model selection. The results obtained demonstrated the optimization of these steps that significantly improved the model generalization capabilities in some UCI databases. Finally, this methodology was applied to create room demand parsimonious models using booking databases from a hotel located in a region of Northern Spain. The results proved that the proposed method created models with higher generalization capacity and lower complexity compared to those obtained with classical KDD process.
Wind turbine noise is important to environment and fault diagnosis and operation condition judgment. Aiming at complicated process of wind turbines noise detection, paper studied blade aerodynamic and the nacelle noise.To nacelle noise, put forward forecast noise sound pressure level by vibration parameters acceleration, simulate four kinds of typical operation conditions and consider influence of wind speed increase. Support vector machine regression based on genetic algorithm (GA-SVR) of the proposed ideas are verified. To blade aerodynamic noise, use non-acoustic parameters to predict noise A-weighted sound pressure level. The paper analyzed shortcomings of GA-SVR, and improvement was put forward. Termination conditions of GA was balanced. To predict noise by regression analysis combining with improved GA-SVR, prediction results is accurate and possess practical application value by the experiment verification.
This paper describes the process for optimising the annealing cycle on a hot dip galvanising line based on a combination of the techniques of artificial intelligence and genetic algorithms for creating two types of regression models. The first model can predict the furnace operating temperature for each coil and is trained to learn from the experience of the plant operators when the process has been correctly adjusted in 'manual mode' and from the control system when it has been properly operated in 'automatic mode'. Once the scheduling has been optimised, and using the two predictive models, a computer simulation is made of the galvanising process in order to optimise the target settings when there are sudden transitions in the steel strip. This substantially improves the thermal treatment, as these sudden transitions may occur when there are two welded coils differing in size and type of steel, whereby a drastic change in strip specifications leads to irregular thermal treatments that may affect the steel's coating or properties in that part of the coil.
Developing better prediction models is crucial for the steelmaking industry to improve the continuous hot dip galvanising line (HDGL). This paper presents a genetic based methodology whereby a wrapper based scheme is optimised to generate overall parsimony models for predicting temperature set points in a continuous annealing furnace on an HDGL. This optimisation includes a dynamic penalty function to control model complexity and an early stopping criterion during the optimisation phase. The resulting models (multilayer perceptron neural networks) were trained using a database obtained from an HDGL operating in the north of Spain. The number of neurons in the unique hidden layer, the inputs selected and the training parameters were adjusted to achieve the lowest validation and mean testing errors. Finally, a comparative evaluation is reported to highlight our proposal's range of applicability, developing models with lower prediction errors, higher generalisation capacity and less complexity than a standard method.
The prediction of the set points for continuous annealing furnaces on hot dip galvanising lines is essential if high product quality is to be maintained and energy consumption and related emissions into the atmosphere are to be reduced. Owing to the global and evolving nature of the galvanising industry, plant engineers are currently demanding better overall prediction models that maintain accuracy while working with continual changes in the production cycle. This paper presents three promising prediction models based on ensemble methods (additive regression,bagging and dagging) and compares them with models based on artificial intelligence to highlight how good ensembles are at creating overall models with lower generalisation errors. The models are trained using coil properties, chemical compositions of the steel and historical data from a galvanising process operating in Spain. The results show that the potential benefits from such ensemble models, once configured properly, include high performance in terms of both prediction and generalisation capacity, as well as reliability in prediction and a significant reduction in the difficulty of setting up the model.
Tuyere core drillings give a unique opportunity to probe the blast furnace and detect changes in both phys- ical and chemical conditions of its high-temperature region. In this paper the findings from drill cores taken from a blast furnace are used to characterize the internal state of the furnace hearth, quantified by an ero- sion model estimating the available hearth volume. The complex relation is studied by entertaining neural network models using different combinations of inputs consisting of the extent of the distinct tuyere-level zones (raceway, bird's nest, dead man, etc.) of the core samples. The resulting model can be used to gain knowledge of the relation between tuyere level conditions and hearth states, and to classify the findings from future core drillings. The results also throw light on possible reasons for thermal cycles observed in the hearth of the furnace studied.
Controlling the annealing cycle in a hot dip galvanising line (HDGL) is vital if each coil treated is to be properly galvanised and the steel is to have the right properties. Current HDGL furnace control models usually take into account the dimensions of the coil to be dipped and, in some cases, the type of steel. This paper presents a new model for monitoring furnace temperature settings, which considers not just the coil dimensions but also the chemical composition of the steel. This enables the model to be adjusted more suitably to each type of steel to be dipped, so that the HDGL annealing cycle is optimised and rendered more efficient in dealing with new products. The ultimate aim is to find a model that is equally efficient for new types of steel coil that have not been processed before and whose dimensions and chemical compositions are different from coils processed previously. To find the best model, this paper compares various new and classical algorithms for developing a precise and efficient prediction model capable of determining the three temperature settings for heating on an HDGL located in Aviles (Spain) on the basis of the physical and chemical characteristics of the coils to be processed and the preset process conditions.
The performance of most of the classification algorithms on a particular dataset is highly dependent on the learning parameters used for training them. Different approaches like grid search or genetic algorithms are frequently employed to find suitable parameter values for a given dataset. Grid search has the advantage of finding more accurate solutions in general at the cost of higher computation time. Genetic algorithms, on the other hand, are able to find good solutions in less time, but the accuracy of these solutions is usually lower than those of grid search.
This paper uses ideas from meta-learning and case-based reasoning to provide good starting points to the genetic algorithm. The presented approach reaches the accuracy of grid search at a significantly lower computational cost. We performed extensive experiments for optimizing learning parameters of the Support Vector Machine (SVM) and the Random Forest classifiers on over 100 datasets from UCI and StatLib repositories. For the SVM classifier, grid search achieved an average accuracy of 81 % and took six hours for training, whereas the standard genetic algorithm obtained 74 % accuracy in close to one hour of training. Our method was able to achieve an average accuracy of 81 % in only about 45 minutes. Similar results were achieved for the Random Forest classifier. Besides a standard genetic algorithm, we also compared the presented method with three state-of-the-art optimization algorithms: Generating Set Search, Dividing Rectangles, and the Covariance Matrix Adaptation Evolution Strategy. Experimental results show that our method achieved the highest average accuracy for both classifiers. Our approach can be particularly useful when training classifiers on large datasets where grid search is not feasible.
This study proposed a novel PSO–SVM model that hybridized the particle swarm optimization (PSO) and support vector machines (SVM) to improve the classification accuracy with a small and appropriate feature subset. This optimization mechanism combined the discrete PSO with the continuous-valued PSO to simultaneously optimize the input feature subset selection and the SVM kernel parameter setting. The hybrid PSO–SVM data mining system was implemented via a distributed architecture using the web service technology to reduce the computational time. In a heterogeneous computing environment, the PSO optimization was performed on the application server and the SVM model was trained on the client (agent) computer. The experimental results showed the proposed approach can correctly select the discriminating input features and also achieve high classification accuracy.
A new regression technique based on Vapnik's concept of support vectors is introduced. We compare support vector regression (SVR) with a committee regression technique (bagging) based on regression trees and ridge regression done in feature space. On the basis of these experiments, it is expected that SVR will have advantages in high dimensionality space because SVR optimization does not depend on the dimensionality of the input space.
A new regression technique based on concept of support vectors is introduced. We compare support vector regression with a committee regression technique (bagging) based on regression trees and ridge regression done in feature space. On the basis of these experiments, it is expected that SVR will have advantages in high dimensionality space because SVR optimization does not depend on the dimension&y of input space. This is a longer version of the paper appear in Advances in Neural Processing Systems 9 (proceedings of the 1996 conference)
Spectral band selection is a fundamental problem in hyperspectral classification. This paper addresses the problem of band selection for hyperspectral remote sensing image and SVM parameter optimization. First, we propose an evolutionary classification system based on particle swarm optimization (PSO) to improve the generalization performance of the SVM classifier. For this purpose, we have optimized the SVM classifier design by searching for the best value of the parameters that tune its discriminant function, and upstream by looking for the best subset of features that feed the classifier. Second, for making use of wavelet signal feature of pixels of hyperspectral image,we investigate the performance of the selected wavelet features based on wavelet approximate coefficients at the third level.The PSO algorithm is performed to optimize spectral feature and wavelet-based approximate coefficients to select the best discriminant features for hyperspectral remote imagery.The experiments are conducted on the basis of AVIRIS 92AV3C dataset. The obtained results clearly confirm the superiority of the SVM approach as compared to traditional classifiers, and suggest that further substantial improvements in terms of classification accuracy can be achieved by the proposed PSO-SVM classification system.
In the history of research of the learning problem one can extract four periods that can be characterized by four bright events: (i) Constructing the first learning machines, (ii) constructing the fundamentals of the theory, (iii) constructing neural networks, (iv) constructing the alternatives to neural networks.
This paper presents a performance enhancement scheme for the recently developed extreme learning machine (ELM) for classifying power system disturbances using particle swarm optimization (PSO). Learning time is an important factor while designing any computational intelligent algorithms for classifications. ELM is a single hidden layer neural network with good generalization capabilities and extremely fast learning capacity. In ELM, the input weights are chosen randomly and the output weights are calculated analytically. However, ELM may need higher number of hidden neurons due to the random determination of the input weights and hidden biases. One of the advantages of ELM over other methods is that the parameter that the user must properly adjust is the number of hidden nodes only. But the optimal selection of its parameter can improve its performance. In this paper, a hybrid optimization mechanism is proposed which combines the discrete-valued PSO with the continuous-valued PSO to optimize the input feature subset selection and the number of hidden nodes to enhance the performance of ELM. The experimental results showed the proposed algorithm is faster and more accurate in discriminating power system disturbances.
Solar global irradiation is barely recorded in isolated rural areas around the world. Traditionally, solar resource estimation has been performed using parametric-empirical models based on the relationship of solar irradiation with other atmospheric and commonly measured variables, such as temperatures, rainfall, and sunshine duration, achieving a relatively high level of certainty. Considerable improvement in soft-computing techniques, which have been applied extensively in many research fields, has lead to improvements in solar global irradiation modeling, although most of these techniques lack spatial generalization.
This new methodology proposes support vector machines for regression with optimized variable selection via genetic algorithms to generate non-locally dependent and accurate models. A case of study in Spain has demonstrated the value of this methodology. It achieved a striking reduction in the mean absolute error (MAE) – 41.4% and 19.9% – as compared to classic parametric models; Bristow & Campbell and Antonanzas-Torres et al., respectively.
Ant Colony Optimization is a population-based meta-heuristic that exploits a form of past performance memory that is inspired by the foraging behavior of real ants. The behavior of the Ant Colony Optimization algorithm is highly dependent on the values defined for its parameters. Adaptation and parameter control are recurring themes in the field of bio-inspired optimization algorithms. The present paper explores a new fuzzy approach for diversity control in Ant Colony Optimization. The main idea is to avoid or slow down full convergence through the dynamic variation of a particular parameter. The performance of different variants of the Ant Colony Optimization algorithm is analyzed to choose one as the basis to the proposed approach. A convergence fuzzy logic controller with the objective of maintaining diversity at some level to avoid premature convergence is created. Encouraging results on several traveling salesman problem instances and its application to the design of fuzzy controllers, in particular the optimization of membership functions for a unicycle mobile robot trajectory control are presented with the proposed method.
In this paper we describe the design of a fuzzy logic controller for the ball and beam system using a modified Ant Colony Optimization (ACO) method for optimizing the type of membership functions, the parameters of the membership functions and the fuzzy rules. This is achieved by applying a systematic and hierarchical optimization approach modifying the conventional ACO algorithm using an ant set partition strategy. The simulation results show that the proposed algorithm achieves better results than the classical ACO algorithm for the design of the fuzzy controller.
The accuracy of the component-based method relies heavily on the characteristic response of their constitutive elements. To properly assess the deformation capacity of the whole connection, modelling the complete force–displacement curves of the components, from the initial stiffness to fracture, is necessary. This paper presents a numerical-informational method for calculating the ductile response of the T-stub component. In order to reduce the intensive computation of the finite element (FE) method, the results of numerical simulations are used to train a set of metamodels based on soft-computing (SC) techniques. These metamodels are capable of predicting, with a high degree of accuracy, the key parameters that define the force–displacement curve of the T-stub. In addition, a feature selection (FS) scheme based on genetic algorithms (GAs) is included in the training process to select the most influential input variables. This scheme leads to overall and parsimonious metamodels that improve the method’s generalisation capacity.The mean absolute error (MAE) in the prediction of each key parameter reports values below 5% for both validation and test results. This demonstrates the strong performance of the SC-based metamodels when comparing them with the FE simulations. Finally, this hybrid method constitutes a suitable tool to be implemented in non-linear steel connections software.
Solar radiation estimates with clear sky models require estimations of aerosol data. The low spatial resolution of current aerosol datasets, with their remarkable drift from measured data, poses a problem in solar resource estimation. This paper proposes a new downscaling methodology by combining support vector machines for regression (SVR) and kriging with external drift, with data from the MACC reanalysis datasets and temperature and rainfall measurements from 213 meteorological stations in continental Spain.
The SVR technique was proven efficient in aerosol variable modeling. The Linke turbidity factor (TL) and the aerosol optical depth at 550 nm (AOD 550) estimated with SVR generated significantly lower errors in AERONET positions than MACC reanalysis estimates. The TL was estimated with relative mean absolute error (rMAE) of 10.2% (compared with AERONET), against the MACC rMAE of 18.5%. A similar behavior was seen with AOD 550, estimated with rMAE of 8.6% (compared with AERONET), against the MACC rMAE of 65.6%.
Kriging using MACC data as an external drift was found useful in generating high resolution maps (0.05° × 0.05°) of both aerosol variables. We created high resolution maps of aerosol variables in continental Spain for the year 2008.
The proposed methodology was proven to be a valuable tool to create high resolution maps of aerosol variables (TL and AOD 550). This methodology shows meaningful improvements when compared with estimated available databases and therefore, leads to more accurate solar resource estimations. This methodology could also be applied to the prediction of other atmospheric variables, whose datasets are of low resolution.
Metaheuristic optimization algorithms have become a popular choice for solving complex problems which are otherwise difficult to solve by traditional methods. However, these methods have the problem of the parameter adaptation and many researchers have proposed modifications using fuzzy logic to solve this problem and obtain better results than the original methods. In this study a comprehensive review is made of the optimization techniques in which fuzzy logic is used to dynamically adapt some important parameters in these methods. In this paper, the survey mainly covers the optimization methods of Particle Swarm Optimization (PSO), Gravitational Search Algorithm (GSA), and Ant Colony Optimization (ACO), which in the last years have been used with fuzzy logic to improve the performance of the optimization methods.
In classification, feature selection is an important data pre-processing technique, but it is a difficult problem due mainly to the large search space. Particle swarm optimisation (PSO) is an efficient evolutionary computation technique. However, the traditional personal best and global best updating mechanism in PSO limits its performance for feature selection and the potential of PSO for feature selection has not been fully investigated. This paper proposes three new initialisation strategies and three new personal best and global best updating mechanisms in PSO to develop novel feature selection approaches with the goals of maximising the classification performance, minimising the number of features and reducing the computational time. The proposed initialisation strategies and updating mechanisms are compared with the traditional initialisation and the traditional updating mechanism. Meanwhile, the most promising initialisation strategy and updating mechanism are combined to form a new approach (PSO(4-2)) to address feature selection problems and it is compared with two traditional feature selection methods and two PSO based methods. Experiments on twenty benchmark datasets show that PSO with the new initialisation strategies and/or the new updating mechanisms can automatically evolve a feature subset with a smaller number of features and higher classification performance than using all features. PSO(4-2) outperforms the two traditional methods and two PSO based algorithm in terms of the computational time, the number of features and the classification performance. The superior performance of this algorithm is due mainly to both the proposed initialisation strategy, which aims to take the advantages of both the forward selection and backward selection to decrease the number of features and the computational time, and the new updating mechanism, which can overcome the limitations of traditional updating mechanisms by taking the number of features into account, which reduces the number of features and the computational time.
The uncontrolled firing of neurons in brain leads to epileptic seizures in the patients. A novel scheme to detect epileptic seizures from back ground electroencephalogram ( EEG) is proposed in this paper. This scheme is based on discrete wavelet packet transform with energy, entropy, kurtosis, skewness, mean, median and standard deviation as the properties for creating features of signals for classification. Optimal features are selected using genetic algorithm ( GA) with support vector machine as a classifier for creating objective function values for the GA. Clinical EEG data from epileptic and normal subjects are used in the experiment. The knowledge of neurologist ( medical expert) is utilized to train the system. To evaluate the efficacy of the proposed scheme, a 10 fold cross-validation is implemented, and the detection rate is found 100% accurate with 100% of sensitivity and specificity for the data under consideration. The proposed GA-SVM scheme is a novel technique using a hybrid approach with wavelet packet decomposition, support vector machine and GA. It is novel in terms of selection of features sub set, use of SVM classifier as objective function for GA and improved classification rate. The proposed model can be used in the developing and the third world countries where the medical facilities are in acute shortage and qualified neurologists are not available. This system can be helpful in assisting the neurologists in terms of automated observation and saving valuable human expert time.
An intelligent identification system for mixed anuran vocalizations is developed in this work to provide the public to easily consult online. The raw mixed anuran vocalization samples are first filtered by noise removal, high frequency compensation, and discrete wavelet transform techniques in order. An adaptive end-point detection segmentation algorithm is proposed to effectively separate the individual syllables from the noise. Six features, including spectral centroid, signal bandwidth, spectral roll-off, threshold-crossing rate, spectral flatness, and average energy, are extracted and served as the input parameters of the classifier. Meanwhile, a decision tree is constructed based on several parameters obtained during sample collection in order to narrow the scope of identification targets. Then fast learning neural-networks are employed to classify the anuran species based on feature set chosen by wrapper feature selection method. A series of experiments were conducted to measure the outcome performance of the proposed work. Experimental results exhibit that the recognition rate of the proposed identification system can achieve up to 93.4%. The effectiveness of the proposed identification system for anuran vocalizations is thus verified.
This paper proposes a modified binary particle swarm optimization (MBPSO) method for feature selection with the simultaneous optimization of SVM kernel parameter setting, applied to mortality prediction in septic patients. An enhanced version of binary particle swarm optimization, designed to cope with premature convergence of the BPSO algorithm is proposed. MBPSO control the swarm variability using the velocity and the similarity between best swarm solutions. This paper uses support vector machines in a wrapper approach, where the kernel parameters are optimized at the same time. The approach is applied to predict the outcome (survived or deceased) of patients with septic shock. Further, MBPSO is tested in several benchmark datasets and is compared with other PSO based algorithms and genetic algorithms (GA). The experimental results showed that the proposed approach can correctly select the discriminating input features and also achieve high classification accuracy, specially when compared to other PSO based algorithms. When compared to GA, MBPSO is similar in terms of accuracy, but the subset solutions have less selected features.
This paper deals with the constant problem of establishing a usable and reliable evolutionary algorithm (EA) characterization procedure so that final users like engineers, mathematicians or physicists can have more specific information to choose the most suitable EA for a given problem. The practical goal behind this work is to provide insights into relevant features of fitness landscapes and their relationship to the performance of different algorithms. This should help users to minimize the typical initial stage in which they apply a well-known EA, or a modified version of it, to the functions they want to optimize without really taking into account its suitability to the particular features of the problem. This trial and error procedure is usually due to a lack of objective and detailed characterizations of the algorithms in the literature in terms of the types of functions or landscape characteristics they are well suited to handle and, more importantly, the types for which they are not appropriate. Specifically, the influence of separability and modality of the fitness landscapes on the behaviour of EAs is analysed in depth to conclude that the typical binary classification of the target functions into separable/non-separable and unimodal/multimodal is too general, and characterizing the EAs’ response in these terms is misleading. Consequently, more detailed features of the fitness landscape in terms of separability and modality are proposed here and their relevance in the EAs’ behaviour is shown through experimentation using standardized benchmark functions that are described using those features. Three different EAs, the genetic algorithm, the Covariance Matrix Adaptation Evolution Strategy and Differential Evolution, are evaluated over these benchmarks and their behaviour is explained in terms of the proposed features
In this day and age, galvanised coated steel is an essential product in several key manufacturing sectors because of its anticorrosive properties. The increase in demand has led managers to improve the different phases in their production chains. Among the efforts needed to accomplish this task, process modelling can be identified as the one with the most powerful outputs in spite of its non-trivial development. In many fields, such as industrial modelling, multilayer feedforward neural networks are often proposed as universal function approximators. These supervised neural networks are commonly trained by the traditional, back-propagation learning format, which minimises the mean squared error (mse) of the training data. However, in the presence of corrupted or extremely deviated samples (outliers), this training scheme may produce incorrect models, and it is well known that industrial data sets frequently contain outliers. The process modelled is a steel coil annealing furnace in a galvanising line, which shares characteristics with most of the furnaces used in galvanised lines all over the world. This paper reports the effectiveness of robust learning algorithms compared to the classical mse-based learning algorithm for the modelling of a real industry process. From this model an adequate line velocity (the velocity set point) for a coil, depending on its characteristics and the furnace condition to receive this coil (temperature set points), can be obtained. With this set point generation model the operator could set strategies to manage the line, i.e. set the order of the coil to be treated or preview the line's speed conditions for the transitory situations.
The efficient and reliable control of a reheating furnace is a challenging problem, due to: (a) the many different types of billets to be processed, (b) the strong intercorrelation among process variables, (c) the large dimension of the input and output space, (d) the strong interaction among process variables, (e) a large time delay, and (f) highly nonlinear behavior. Thus, conventional reheating furnace operation has been heavily dependent upon look-up tables which list the optimal set points. This paper describes a modified modular neural network for the supervisory control of a reheating furnace. Based on the divide-and-conquer concept, a modular network is capable of dividing a complex task into subtasks, and modeling each subtask with an expert network. To model such activities, a gating network is used for the classification and allocation of the input data to the corresponding expert network. To overcome the correlation effects among process variables and the problem of dimensionality, principal component analysis (PCA) has been employed to remove the correlation and reduce the problem dimension. From PCA analysis, it was possible to decide on the optimal dimension for the problem, to describe the dynamic behavior of the furnace. The proposed neural network has been trained and tested using operational data from the reheating furnace and has been implemented on the wire rod mill process of POSCOTM.
Roll load is a critical design parameter in steel rolling operation and mill setup. In this work a parsimonious roll load prediction model was developed using a neural network (NN). Design techniques based on orthogonal arrays were adopted for the allocation of the rolling process conditions, while a validated finite element (FE) code was used to generate the roll load data based on the process conditions specified by the orthogonal array. The rolling data obtained were then analysed using the traditional statistical techniques, such as level (mean response) analysis and ANOVA (analysis of variances), in order to find the critical input variables. A double-loop interactive training procedure was adopted in order to prevent over-fitting, with the resulting NN model balanced between the training and validation performance. Model performance analysis was conducted on the initial NN model to find if weak prediction regions exist, and further rolling data to cover these weak regions were generated and the extended data were used for re-training. The resulting model was then applied to new rolling data for testing, and the roll load prediction was satisfactory. The NN model prediction can be implemented for online application such as rolling schedule optimisation and dynamic roll gap control, due to its fast calculation ability. Post-model analyses such as model responses have been conducted to enhance the understanding of the behaviour of the neural network model, which is vital to increase the confidence in using the NN model. Model sensitivity derived from the NN model is consistent with the statistical analysis of the rolling data.
In this paper we report on the use of evolutionary algorithms for optimizing the identification of classification models for selected tumor markers. Our goal is to identify mathematical models that can be used for classifying tumor marker values as normal or as elevated; evolutionary algorithms are used for optimizing the parameters for learning classification models. The sets of variables used as well as the parameter settings for concrete modeling methods are optimized using evolution strategies and genetic algorithms. The performance of these algorithms is analyzed as well as the population diversity progress. In the empirical part of this paper we document modeling results achieved for tumor markers CA 125 and CYFRA using a medical data base provided by the Central Laboratory of the General Hospital Linz; empirical tests are executed using HeuristicLab.
Ensemble methods have been called the most influential development in Data Mining and Machine Learning in the past decade. They combine multiple models into one usually more accurate than the best of its components. Ensembles can provide a critical boost to industrial challenges -- from investment timing to drug discovery, and fraud detection to recommendation systems -- where predictive accuracy is more vital than model interpretability. Ensembles are useful with all modeling algorithms, but this book focuses on decision trees to explain them most clearly. After describing trees and their strengths and weaknesses, the authors provide an overview of regularization -- today understood to be a key reason for the superior performance of modern ensembling algorithms. The book continues with a clear description of two recent developments: Importance Sampling (IS) and Rule Ensembles (RE). IS reveals classic ensemble methods -- bagging, random forests, and boosting -- to be special cases of a single algorithm, thereby showing how to improve their accuracy and speed. REs are linear rule models derived from decision tree ensembles. They are the most interpretable version of ensembles, which is essential to applications such as credit scoring and fault diagnosis. Lastly, the authors explain the paradox of how ensembles achieve greater accuracy on new data despite their (apparently much greater) complexity.
Support Vector Machines (SVMs) have achieved very good performance on different learning problems. However, the success of SVMs depends on the adequate choice of the values of a number of parameters (e.g., the kernel and regularization parameters). In ...
A parsimonious genetic algorithm guided neural network ensemble modelling strategy is presented. Each neural network candidate model to participate in the ensemble model is structurally selected using a genetic algorithm. This provides an effective route to improve the performance of the individual neural network models as compared to more traditional neural network modelling approaches, whereby the neural network structure is selected through some trial-and-error methods or heuristics. The parsimonious neural network ensemble modelling strategy developed in this paper is highly efficient and requires very little extra computation for developing the ensemble model, thus overcoming one of the major known obstacles for developing an ensemble model. The key techniques behind the implementation of the ensemble model, include the formulation of the fitness function, the generation of the qualified neural network candidate models, as well as the specific definitions of the assemble strategies. A case study is presented which exploits a complex industrial data set relating to the Charpy impact energy for heat-treated steels, which was provided by Tata Steel Europe. Modelling results show a significant performance improvement over the previously developed models for the same data set.
The prediction of bankruptcy is of significant importance with the present-day increase of bankrupt companies. In the practical applications, the cost of misclassification is worthy of consideration in the modeling in order to make accurate and desirable decisions. An effective prediction system requires the integration of the cost preference into the construction and optimization of prediction models. This paper presents an evolutionary approach for optimizing simultaneously the complexity and the weights of learning vector quantization network under the symmetric cost preference. Experimental evidences on a real-world data set demonstrate the proposed algorithm leads to significant reduction of features without the degradation of prediction capability.
Traditional methods often employed to solve complex real world problems tend to inhibit elaborate exploration of the search space. They can be expensive and often results in sub-optimal solutions. Evolutionary computation (EC) is generating considerable interest for solving real world engineering problems. They are proving robust in delivering global optimal solutions and helping to resolve limitations encountered in traditional methods. EC harnesses the power of natural selection to turn computers into optimisation tools. The core methodologies of EC are genetic algorithms (GA), evolutionary programming (EP), evolution strategies (ES) and genetic programming (GP). This paper attempts to bridge the gap between theory and practice by exploring characteristics of real world problems and by surveying recent EC applications for solving real world problems in the manufacturing industry. The survey outlines the current status and trends of EC applications in manufacturing industry. For each application domain, the paper describes the general domain problem, common issues, current trends, and the improvements generated by adopting the GA strategy. The paper concludes with an outline of inhibitors to industrial applications of optimisation algorithms.
In this tutorial we give an overview of the basic ideas underlying Support Vector (SV) machines for function estimation. Furthermore, we include a summary of currently used algorithms for training SV machines, covering both the quadratic (or convex) programming part and advanced methods for dealing with large datasets. Finally, we mention some modifications and extensions that have been applied to the standard SV algorithm, and discuss the aspect of regularization from a SV perspective.
An optimal design of support vector machine (SVM)-based classifiers for prediction aims to optimize the combination of feature selection, parameter setting of SVM, and cross-validation methods. However, SVMs do not offer the mechanism of automatic internal relevant feature detection. The appropriate setting of their control parameters is often treated as another independent problem. This paper proposes an evolutionary approach to designing an SVM-based classifier (named ESVM) by simultaneous optimization of automatic feature selection and parameter tuning using an intelligent genetic algorithm, combined with k-fold cross-validation regarded as an estimator of generalization ability. To illustrate and evaluate the efficiency of ESVM, a typical application to microarray classification using 11 multi-class datasets is adopted. By considering model uncertainty, a frequency-based technique by voting on multiple sets of potentially informative features is used to identify the most effective subset of genes. It is shown that ESVM can obtain a high accuracy of 96.88% with a small number 10.0 of selected genes using 10-fold cross-validation for the 11 datasets averagely. The merits of ESVM are three-fold: (1) automatic feature selection and parameter setting embedded into ESVM can advance prediction abilities, compared to traditional SVMs; (2) ESVM can serve not only as an accurate classifier but also as an adaptive feature extractor; (3) ESVM is developed as an efficient tool so that various SVMs can be used conveniently as the core of ESVM for bioinformatics problems.
The improvement of the performances of a complex production process such as the Sollac hot dip galvanizing line of Florange (France) needs to integrate various approaches, including quality monitoring, diagnosis, control, optimization methods, etc. These techniques can be grouped under the term of intelligent control and aim to enhance the operating of the process as well as the quality of delivered products. The first section briefly describes the plant concerned and presents the objectives of the study. These objectives are mainly reached by incorporating the skill of the operators in neural models, at different levels of control. The low-level supervision of measurements and operating conditions are briefly presented. The control of the coating process, highly nonlinear, is divided in two parts. The optimal thermal cycle of alloying is determined using a radial basis function neural network, from a static database built up from recorded measurements. The learning of the weights is carried out from the results of a fuzzy C-means clustering algorithm. The control of the annealing furnace, the most important equipment, is achieved by mixing a static inverse model of the furnace based on a feedforward multilayer perceptron and a regulation loop. Robust learning criteria are used to tackle possible outliers in the database. The neural network is then pruned in order to enhance the generalization capabilities.
UCI machine learning repository
- D N A Asuncion
D.N.A. Asuncion, UCI machine learning repository (2007). http://www.ics.uci.
edu/∼mlearn/MLRepository.html