No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Temporal distribution-based prediction strategy for dynamic multi-objective optimization assisted by GRU neural network
... Li Shengwang and Duan Xin et al. used a flexible BP (back-propagation) method in their neural network model to predict and control the opening temperature of the decomposition furnace [20], but its training speed is slow and it easily falls into local minima, which makes it difficult to meet the precise temperature control requirements of methanol synthesis [21][22][23]. The gated recurrent unit (GRU) neural network is a variant of the long short-term memory (LSTM) network, which is able to process time series data with fewer parameters and faster computational speed [24,25]. Guo Jun et al. used GRU neurons to study the linear relationship between indicator gas and temperature, and optimized the parameters of the GRU model using a particle swarm algorithm to successfully predict the temperature of the coal body [26]. ...
The most important control parameters in the methanol distillation process, which are directly related to product quality and yield, are the temperature, pressure and water content of the finished product at the top of the column. In order to adapt to the development trend of modern industrial technology to be more accurate, faster and more stable, the fusion of multi-sensor data puts forward higher requirements. Traditional control methods, such as PID control and fuzzy control, have the disadvantages of low heterogeneous data processing capability, poor response speed and low control accuracy when dealing with complex industrial process detection and control. For the control of tower top temperature and pressure in the methanol distillation industry, this study innovatively combines generative artificial intelligence and a type II fuzzy neural network, using a GAN for data preprocessing and a type II fuzzy neural network for steady-state inverse prediction to construct the GAN-T2FNN temperature and pressure control model for an atmospheric pressure tower. Comparison experiments with other neural network models and traditional PID control models show that the GAN-T2FNN model has a better performance in terms of prediction accuracy and fitting effect, with a minimum MAE value of 0.1828, which is more robust, and an R² Score of 0.9854, which is closer to 1, for the best overall model performance. Finally, the SHAP model was used to analyze the influence mechanism of various parameters on the temperature and pressure at the top of the atmospheric column, which provides a more comprehensive reference and guidance for the precise control of the methanol distillation process.
... Zhang et al. [29] proposed a transfer learningbased surrogate-assisted evolutionary algorithm, utilizes historical optimal knee solutions to augment the training data for building Gaussian process models, effectively improving the quality of solutions. HOU et al. [30] by creating a time series and utilizing GRU neural network to maximize distribution features and minimize losses, the network model is trained to ultimately form the initial population for the next moment. YAO et al. [31] combining clustering difference strategy with transfer learning technology to improve the quality of the solution and the convergence speed. ...
This paper introduces a special points and neural network- based dynamic multi-objective optimization algorithm (SPNN-DMOA) for solving dynamic multi-objective optimization problems (DMOPs) with an irregularly changing pareto set. In the stage of population initialization, the algorithm employs a feedforward neural network (FNN) along with special points to generate an initial population. The FNN is trained with historical special points (knee point, boundary point, center point), and the current special points are generated by the FNN when an environmental change is detected. Then the decision variables are classified into convergence variables and diversity variables. The convergence variables of special points are locally searched to form a new population and the best individuals of this population are selected. Finally, a portion of the initial population is generated by conducting a local search on the diversity variables of best individuals, while the remaining portion is produced using random strategies. SPNN-DMOA only needs to maintain non-dominated solutions in proximity to special points, which reduces the computational complexity in the dynamic evolution process. The proposed algorithm has been compared with other state-of-the-art algorithms on a series of benchmark problems, demonstrating its superior performance in optimizing DMOPs.
... This strategy constructs differential models using central points from different subregions and initializes a new population based on these models. Hou [42] developed a time series using central points from historical environments and employed a gated-recurrent-unit neural network model to predict the central point at the next time step, thereby generating a new population. Overall, the effectiveness of prediction-based methods is contingent upon the accuracy of the prediction models. ...
The primary challenge in addressing dynamic multi-objective optimization problems (DMOPs) is the rapid tracking of optimal solutions. Although methods based on transfer learning have shown remarkable performance in tackling DMOPs, most existing methods overlook the potential relationships between individuals within the population and those from historical environments. Consequently, they fail to adequately exploit historical information. To this end, this study proposes a dynamic multi-objective optimization algorithm based on probability-driven prediction and correlation-guided individual transfer (PDP&CGIT), which consists of two strategies: probability-driven prediction (PDP) and correlation-guided individual transfer (CGIT). Specifically, the PDP strategy analyzes the distribution of population characteristics and constructs a discriminative predictor based on a probability-annotation matrix to classify high-quality solutions from numerous randomly generated solutions within the decision space. Moreover, from the perspective of individual evolution, the CGIT strategy analyzes the correlation between current elite individuals and those from the previous moment. It learns the dynamic change pattern of the individuals and transfers this pattern to new environments. This is to maintain the diversity and distribution of the population. By integrating the advantages of these two strategies, PDP&CGIT can efficiently respond to environmental changes. Extensive experiments were performed to compare the proposed PDP&CGIT with five state-of-the-art algorithms across the FDA, F, and DF test suites. The results demonstrated the superiority of PDP&CGIT.
... In addition to directly outputting optimal solutions, neural networks have also been used to assist the utilization of other optimization methods especially evolutionary algorithms. These include population initialization [47], mating selection [48], offspring generation [49], evolutionary direction learning [50], environmental selection [51], local search [52], surrogate models [53], dimensionality reduction [39], search space conversion [54], parameter tuning [55], operator selection [56], algorithm recommendation [57], and algorithm transfer [58]. Although many of these ideas have only been applied to continuous optimization in the literature, they can be easily extended to binary optimization due to the high flexibility of evolutionary algorithms. ...
Binary optimization assumes a pervasive significance in the context of practical applications, such as knapsack problems, maximum cut problems, and critical node detection problems. Existing techniques including mathematical programming, heuristics, evolutionary computation, and neural networks have been employed to tackle binary optimization problems (BOPs), however, they grapple with the challenge of optimizing a large number of binary variables. In this paper, we propose a dimensionality reduction method to assist evolutionary algorithms in solving large-scale BOPs, which is achieved based on neural networks. The proposed method converts the optimization of a large number of binary variables into the optimization of a small number of network weights, resulting in a significant reduction in search space dimensionality. Crucially, the proposed method obviates the necessity for a training process, which eliminates the requirement for a priori knowledge and enhances the search efficiency. On six types of single-and multi-objective BOPs with up to 10 000 000 variables, the proposed method demonstrates superiority over top-tier evolutionary algorithms and neural network-based methods.
Harnessing hydrogen through photocatalytic water splitting represents a pivotal stride towards carbon neutrality. Evaluating ongoing experiments under natural light conditions is critical for scaling this technology. Given the inherent instability of natural light, devising precise hydrogen production forecasts has emerged as a core focus to streamline project timelines and financial governance. This research introduces a robust system for continuous photocatalytic water splitting, leveraging natural light for hydrogen production. Analysis and prognostication of hydrogen generation are conducted using experimental data from 21 distinct groups. Critical factors, including radiation intensity, reaction temperature, reactant concentration, solar concentrator specifics, and hydrogen production rate (HPR), undergo thorough correlation scrutiny. Employing a deep-learning framework, gated recurrent units (GRU) neural network is utilized for nuanced feature apprehension, enhancing hydrogen output forecasting precision. The results illustrate that the mean daily solar-to-hydrogen (STH) energy conversion efficiency across the 21 groups is approximately 0.25 %. Radiation intensity prevails as a significant influencer, with reaction temperature also playing a substantial role. Solar concentrator utilization proves beneficial for augmenting HPR. Nevertheless, the unevenness of solar concentrator presents adverse effects, necessitating consideration during system conceptualization. The optimized prediction model's outcomes align closely with experimental observations, and with a test loss of 2.61 × 10−3 and 682 epochs, the model's predictive superiority is affirmed. The continuous hydrogen production experiments leveraging natural light provide the feasible reference for the large-scale application of this technology. The correlation analysis of critical factors provides the optimization direction for practical engineering design. The hydrogen production prediction model aims to provide scientific basis for the decision-making of this technology.
Energy forecasting plays an important role in effective power grid management. The widespread adoption of emerging technologies and the increased reliance on renewable sources of energy have created a need for a robust and accurate system for energy forecasting. This demand is becoming increasingly relevant due to the ongoing 2022 energy crisis. Modern power systems are very complex with many complicated correlations between various forecasting factors and parameters. Furthermore, renewable energy is often dependent on weather conditions, which complicates the process of forecasting. This work presents a novel artificial intelligence (AI) driven energy forecasting tuned deep learning framework. By formatting predictors as a time series, two variations of recurrent neural networks (RNN)s have been implemented: long-short-term memory (LSTM) and gated recurrent unit (GRU) neural networks. However, both approaches present several hyperparameters that require adequate tuning to attain desirable performance. Therefore, this work also proposes an improved version of a well know swarm intelligence algorithm, the sine cosine algorithm (SCA), tasked with tackling hyperparameter tuning for both approaches. To demonstrate the improvements made, three datasets have been constructed for evaluation from publicly available real-world data that contain relevant solar, wind, and power-grid load parameters alongside weather data. The proposed metaheuristic algorithm has been subjected to a comparative analysis with several contemporary metaheuristic algorithms to showcase the improvements made. The introduced metaheuristic demonstrated the best performance with a mean square error (MSE) rate for solar generation of only 0.0132 with LSTM methods and 0.0134 with GRU. Similar performance was observed for wind power generation forecasting with a MSE of 0.00292 with LSTM and 0.00287. When tackling power grid load forecasting a median MSE of 0.0162 was attained with LSTM and 0.01504 with GRU. Therefore there is great potential for tackling these tasks using the proposed approach. The best-performing models have been analyzed using SHapley Additive exPlanations (SHAP) to determine the factors that have the highest influence on energy generation and demand.
Evolutionary dynamic multi-objective optimisation (EDMO) is a relatively young but rapidly growing area of investigation. EDMO employs evolutionary approaches to handle multi-objective optimisation problems that have time-varying changes in objective functions, constraints and/or environmental parameters. Due to the simultaneous presence of dynamics and multi-objectivity in problems, the optimisation difficulty for EDMO has a marked increase compared to that for single-objective or stationary optimisation. After nearly two decades of community effort, EDMO has achieved significant advancements on various topics, including theoretic research and applications. This paper presents a broad survey and taxonomy of existing research on EDMO. Multiple research opportunities are highlighted to further promote the development of the EDMO research field.
The goal of this paper is to develop the optimal fuzzy controllers for nonlinear processed represented by electromagnetic actuated clutch systems in order to improve the control system performance. A Takagi-Sugeno type-1 fuzzy logic controller and a Takagi-Sugeno type-2 fuzzy logic controller are designed here to enhance position tracking performance of electromagnetic actuated clutch systems, which are subjected to parametric uncertainties. The parameters of membership functions and the rules of the proposed fuzzy controllers are optimally tuned using a recent metaheuristic optimization algorithm, namely the Grey Wolf Optimizer. A comparison of Takagi-Sugeno type-1 and Takagi-Sugeno type-2 fuzzy control systems is carried out in various simulation scenarios, which are discussed and analyzed. The results highlight that the type-2 fuzzy controller performs efficiently for these complex processes.
Many real-world optimization problems involve multiple objectives, constraints, and parameters that may change over time. These problems are often called dynamic multiobjective optimization problems (DMOPs). The difficulty in solving DMOPs is the need to track the changing Pareto-optimal front efficiently and accurately. It is known that transfer learning (TL)-based methods have the advantage of reusing experiences obtained from past computational processes to improve the quality of current solutions. However, existing TL-based methods are generally computationally intensive and thus time consuming. This article proposes a new memory-driven manifold TL-based evolutionary algorithm for dynamic multiobjective optimization (MMTL-DMOEA). The method combines the mechanism of memory to preserve the best individuals from the past with the feature of manifold TL to predict the optimal individuals at the new instance during the evolution. The elites of these individuals obtained from both past experience and future prediction will then constitute as the initial population in the optimization process. This strategy significantly improves the quality of solutions at the initial stage and reduces the computational cost required in existing methods. Different benchmark problems are used to validate the proposed algorithm and the simulation results are compared with state-of-the-art dynamic multiobjective optimization algorithms (DMOAs). The results show that our approach is capable of improving the computational speed by two orders of magnitude while achieving a better quality of solutions than existing methods.
The machine learning algorithms exploit a given dataset in order to build an efficient predictive or descriptive model. Multi-objective evolutionary optimization assists machine learning algorithms to optimize their hyper-parameters, usually under conflicting performance objectives and selects the best model for a given task. In this paper, recent multi-objective evolutionary approaches for four major data mining and machine learning tasks, namely: (a) data preprocessing, (b) classification, (c) clustering, and (d) association rules, are surveyed.
This work proposes a model-based methodology to produce recipes for Muscat wine distillations in Charentais alembics. The recipes were obtained by dynamic multi-objective optimisation of a multi-component first-principles model. Two multi-objective functions were developed considering chemical markers that define the aromatic characteristic of the distillates. The first function included three chemical markers characteristic of each distillation cut (head, heart and tail). The second considered a principal component analysis (PCA)applied to the chemical profile of Chilean commercial distillates. The optimal recipes were experimentally validated in an automatic Charentais alembic using a synthetic Muscat wine consisting of ethanol and water, plus six congeners. The experimental results indicated that by using model-based optimisation techniques, it is possible to obtain spirits with a chemical composition like some Chilean commercial distillates. In addition, the chemical composition of many commercial distillates was close to a Pareto front that defines a compromise between linalool, a characteristic Muscat floral aroma, and acetaldehyde, a head contaminant marker.
Dynamic multi-objective optimization problems (DMOPs) have been rapidly attracting the interest of the research community. Although static multi-objective evolutionary algorithms have been adapted for solving the DMOPs in the literature, some of those extensions may have high running time and may be inefficient for the given set of test cases. In this paper, we present a new hybrid strategy by integrating the memory concept with the NSGA-II algorithm, called the MNSGA-II algorithm. The proposed algorithm utilizes an explicit memory to store a number of non-dominated solutions using a new memory updating technique. The stored solutions are reused in later stages to reinitialize part of the population when an environment change occurs. The performance of the MNSGA-II algorithm is validated using three test functions from a framework proposed in a recent study. The results show that performance of the MNSGA-II algorithm is competitive with the other state-of-the-art algorithms in terms of tracking the true Pareto front and maintaining the diversity.
Accurate and real-time traffic flow prediction is important in Intelligent Transportation System (ITS), especially for traffic control. Existing models such as ARMA, ARIMA are mainly linear models and cannot describe the stochastic and nonlinear nature of traffic flow. In recent years, deep-learning-based methods have been applied as novel alternatives for traffic flow prediction. However, which kind of deep neural networks is the most appropriate model for traffic flow prediction remains unsolved. In this paper, we use Long Short Term Memory (LSTM) and Gated Recurrent Units (GRU) neural network (NN) methods to predict short-term traffic flow, and experiments demonstrate that Recurrent Neural Network (RNN) based deep learning methods such as LSTM and GRU perform better than auto regressive integrated moving average (ARIMA) model. To the best of our knowledge, this is the first time that GRU is applied to traffic flow prediction.
One of the major distinguishing features of the dynamic multiobjective optimization problems (DMOPs) is the optimization objectives will change over time, thus tracking the varying Pareto-optimal front becomes a challenge. One of the promising solutions is reusing the "experiences" to construct a prediction model via statistical machine learning approaches. However most of the existing methods ignore the non-independent and identically distributed nature of data used to construct the prediction model. In this paper, we propose an algorithmic framework, called Tr-DMOEA, which integrates transfer learning and population-based evolutionary algorithm for solving the DMOPs. This approach takes the transfer learning method as a tool to help reuse the past experience for speeding up the evolutionary process, and at the same time, any population based multiobjective algorithms can benefit from this integration without any extensive modifications. To verify this, we incorporate the proposed approach into the development of three well-known algorithms, nondominated sorting genetic algorithm II (NSGA-II), multiobjective particle swarm optimization (MOPSO), and the regularity model-based multiobjective estimation of distribution algorithm (RM-MEDA), and then employ twelve benchmark functions to test these algorithms as well as compare with some chosen state-of-the-art designs. The experimental results confirm the effectiveness of the proposed method through exploiting machine learning technology.
This paper investigates how to use prediction strategies to improve the performance of multiobjective evolutionary optimization algorithms in dealing with dynamic environments. Prediction-based methods have been applied to predict some isolated points in both dynamic single objective optimization and dynamic multiobjective optimization. We extend this idea to predict a whole population by considering the properties of continuous dynamic multiobjective optimization problems. In our approach, called population prediction strategy (PPS), a Pareto set is divided into two parts: a center point and a manifold. A sequence of center points is maintained to predict the next center, and the previous manifolds are used to estimate the next manifold. Thus, PPS could initialize a whole population by combining the predicted center and estimated manifold when a change is detected. We systematically compare PPS with a random initialization strategy and a hybrid initialization strategy on a variety of test instances with linear or nonlinear correlation between design variables. The statistical results show that PPS is promising for dealing with dynamic environments.
The insertion of atypical solutions (immigrants) in Evolutionary Algorithms populations is a well studied and successful strategy to cope with the difficulties of tracking optima in dynamic environments in single-objective optimization. This paper studies a probabilistic model, suggesting that centroid-based diversity measures can mislead the search towards optima, and presents an extended taxonomy of immigration schemes, from which three immigrants strategies are generalized and integrated into NSGA2 for Dynamic Multiobjective Optimization (DMO). The correlation between two diversity indicators and hypervolume is analyzed in order to assess the influence of the diversity generated by the immigration schemes in the evolution of non-dominated solutions sets on distinct continuous DMO problems under different levels of severity and periodicity of change. Furthermore, the proposed immigration schemes are ranked in terms of the observed offline hypervolume indicator.
Decomposition is a basic strategy in traditional multiobjective optimization. However, it has not yet been widely used in multiobjective evolutionary optimization. This paper proposes a multiobjective evolutionary algorithm based on decomposition (MOEA/D). It decomposes a multiobjective optimization problem into a number of scalar optimization subproblems and optimizes them simultaneously. Each subproblem is optimized by only using information from its several neighboring subproblems, which makes MOEA/D have lower computational complexity at each generation than MOGLS and nondominated sorting genetic algorithm II (NSGA-II). Experimental results have demonstrated that MOEA/D with simple decomposition methods outperforms or performs similarly to MOGLS and NSGA-II on multiobjective 0-1 knapsack problems and continuous multiobjective optimization problems. It has been shown that MOEA/D using objective normalization can deal with disparately-scaled objectives, and MOEA/D with an advanced decomposition method can generate a set of very evenly distributed solutions for 3-objective test instances. The ability of MOEA/D with small population, the scalability and sensitivity of MOEA/D have also been experimentally investigated in this paper.
Real-world multi-objective optimization problems encounter different types of uncertainty that may affect the quality of solutions. One common type is the stochastic noise that contaminates the objective functions. Another type of uncertainty is the different forms of dynamism including changes in the objective functions. Although related work in the literature targets only a single type, in this paper, we study Dynamic Multi-objective Optimization problems (DMOPs) contaminated with stochastic noises by dealing with the two types of uncertainty simultaneously. In such problems, handling uncertainty becomes a critical issue since the evolutionary process should be able to distinguish between changes that come from noise and real environmental changes that resulted from different forms of dynamism. To study both noisy and dynamic environments, we propose a flexible mechanism to incorporate noise into the DMOPs. Two novel techniques called Multi-Sensor Detection Mechanism and Welford-Based Detection Mechanism are proposed to differentiate between real change points and noise points. The proposed techniques are incorporated into a set of Dynamic Multi-objective Evolutionary Algorithms (DMOEAs) to analyze their impact. Our empirical study reveals the effectiveness of the proposed techniques for isolating noise from real dynamic changes and diminishing the noise effect on performance.
Wind speed forecasting plays a crucial role in reducing the risk of wind power uncertainty, which is vital for power system planning, scheduling, control, and operation. However, it is challenging to obtain accurate wind speed forecasting results since wind speed series contain complex fluctuations. In this paper, a novel wind speed forecasting model is proposed by using a genetic algorithm (GA) and long short-term memory neural network (LSTM), where GA is used for evolving the architectures and hyper-parameters of the LSTM, called EvLSTM, because there is no clear knowledge to determine these crucial parameters. In the proposed EvLSTM
forecasting model, a flexible gene encoding strategy, crossover operation, and mutation operation are proposed to describe the different architectures and hyper-parameters of the LSTM during the evolutionary process. In addition, to overcome the weakness of the single method for wind speed forecasting, the ensemble EvLSTM (EnEvLSTM) is proposed by using the no negative constraint theory ensemble learning strategy, whose weight coefficients are determined by the differential evolution algorithm. The proposed EvLSTM and EnEvLSTM forecasting models are evaluated on two real-world wind farms located in Inner Mongolia, China and Sotavento Galicia, Spain. Experimental results on two forecasting horizons demonstrate the superiority of EvLSTM and EnEvLSTM in terms of three performance indices and three statistical tests.
Dynamic multi-objective optimization problems involve multiple conflicting and time-dependent objectives that change continuously depending on the environment. Therefore, effectively tracking the movement of the Pareto-optimal front and the Pareto-optimal set (PS) using a single strategy is difficult. In this study, we propose an ensemble method called EMCD, which is based on the characterization of dynamism. EMCD selects different strategies to generate new populations depending on the characteristics of environmental changes. In our proposed algorithm, historical information is used to locate new populations when the current and historical environmental changes are similar. Otherwise, EMCD establishes whether the PS has changed. If the PS changes, individuals from the population of the new environment are generated by using a set of knee points based on the two previous consecutive populations. If not, new populations are generated by using the perturbation-based approach. In addition, we use an adaptive diversity introduction strategy to ensure the solvability of our proposed algorithm in dynamic environments. To verify the effectiveness of EMCD, we compare its performance to that of three state-of-the-art algorithms on fourteen benchmark problems. The experimental results show that EMCD outperforms the comparison algorithms on most of the benchmark problems.
Recently, dynamic multi-objective optimization has received growing attention due to its popularity in real-world applications. Inspired by polynomial fitting, this paper proposes a polynomial fitting-based prediction algorithm (PFPA) and incorporates it into the model-based multi-objective estimation of distribution algorithm (RM-MEDA) for solving dynamic multi-objective optimization problems. When an environment change is detected, the main mission of PFPA is to predict high-quality search populations for tracking the moving Pareto-optimal set effectively. Firstly, the non-dominated solutions obtained in past environments are utilized to predict high-quality solutions based on a multi-step movement strategy. Secondly, a polynomial fitting-based strategy is designed to fit the distribution of variables according to the obtained search populations, and capture the relationship between variables in the new search environment. Thirdly, some effective search agents are generated for improving population convergence and diversity based on characteristics of variables. To evaluate the performance of the proposed algorithm, experimental results on a set of benchmark functions, with a variety of different dynamic characteristics and difficulties, and two classical dynamic engineering design problems show that PFPA is competitive with some state-of-the-art algorithms.
Phase change material (PCM) is employed to enhance the thermal energy storage capacity of a building integrated photovoltaic thermal (BIPV/T) system. For this purpose, the best thickness of PCM is found using the dynamic multi-objective optimization (MOO), in which the performance throughout a year with real-time data is taken into account. In addition to the annual thermal energy storage capacity (AES), the annual energy (elec-tricity) production (AEP), as well as payback period (PBP), the levelized cost (LCOE), and annual carbon-dioxide emission reduction (ACDR) are considered as the objective function to obtain a favorable condition from all the energy, economic, and environmental (3E) perspectives. MOO is done for a residential building in Tehran, Iran, by employing numerical modeling for system simulation and the combination of TOPSIS and NSGA-II techniques to determine the final optimal solution. According to the results, the best PCM thickness is 77.2 mm. Compared to the base condition, in which air is used as the material for energy storage, using PCM with the optimum thickness is accompanied by 22.24%, 9.93%, 17.69%, and 9.59% improvement in AES, AEP, ACDR, and LCOE, respectively. Moreover, PBP in the optimum condition is 3.321 years, which shows that utilizing the results of MOO is economically justifiable.
The "curse of dimensionality" and the high computational cost have still limited the application of the evolutionary algorithm in high-dimensional feature selection (FS) problems. This article proposes a new three-phase hybrid FS algorithm based on correlation-guided clustering and particle swarm optimization (PSO) (HFS-C-P) to tackle the above two problems at the same time. To this end, three kinds of FS methods are effectively integrated into the proposed algorithm based on their respective advantages. In the first and second phases, a filter FS method and a feature clustering-based method with low computational cost are designed to reduce the search space used by the third phase. After that, the third phase applies oneself to finding an optimal feature subset by using an evolutionary algorithm with the global searchability. Moreover, a symmetric uncertainty-based feature deletion method, a fast correlation-guided feature clustering strategy, and an improved integer PSO are developed to improve the performance of the three phases, respectively. Finally, the proposed algorithm is validated on 18 publicly available real-world datasets in comparison with nine FS algorithms. Experimental results show that the proposed algorithm can obtain a good feature subset with the lowest computational cost.
This paper presents a novel population prediction algorithm based on modular neural network (PA-MNN) for handling dynamic multi-objective optimization. The proposed algorithm consists of three mechanisms. First, we set up a modular neural network (MNN) and train it with historical population information. Some of the initial solutions are generated by the MNN when an environmental change is detected. Second, some solutions are predicted based on forward-looking center points. Finally, some solutions are generated randomly to maintain the diversity. With these mechanisms, when the new environment has been encountered before, initial solutions generated by MNN will have the same distribution characteristics as the final solutions that were obtained in the same environment last time. Because the initialization mechanism based on the MNN does not need the solutions in recent time, the proposed algorithm can also solve dynamic multi-objective optimization problems with a dramatically and irregularly changing Pareto set. The proposed algorithm is tested on a variety of test instances with different dynamic characteristics and difficulties. The comparisons of experimental results with other state-of-the-art algorithms demonstrate that the proposed algorithm is promising for dealing with dynamic multi-objective optimization.
Dynamic Multi-objective Optimization Problem (DMOP) is emerging in recent years as a major real-world optimization problem receiving considerable attention. Tracking the movement of Pareto front efficiently and effectively over time has been a central issue in solving DMOPs. In this paper, a reinforcement learning-based dynamic multi-objective evolutionary algorithm, called RL-DMOEA, which seamlessly integrates reinforcement learning framework and three change response mechanisms, is proposed for solving DMOPs. The proposed algorithm relocates the individuals based on the severity degree of environmental changes, which is estimated through the corresponding changes in the objective space of their decision variables. When identifying different severity degree of environmental changes, the proposed RL-DMOEA approach can learn better evolutionary behaviors from environment information, based on which apply the appropriate response mechanisms. Specifically, these change response mechanisms including the knee-based prediction, center-based prediction and indicator-based local search, are devised to promote both convergence and diversity of the algorithm under different severity of environmental changes. To verify this idea, the proposed RL-DMOEA is evaluated on CEC 2015 test problems involving various problem characteristics. Empirical studies on chosen state-of-the-art designs validate that the proposed RL-DMOEA is effective in addressing the DMOPs.
Deep neural networks are able to learn powerful representations from large quantities of labeled input data, however they cannot always generalize well across changes in input distributions. Domain adaptation algorithms have been proposed to compensate for the degradation in performance due to domain shift. In this paper, we address the case when the target domain is unlabeled, requiring unsupervised adaptation. CORAL [18] is a simple unsupervised domain adaptation method that aligns the second-order statistics of the source and target distributions with a linear transformation. Here, we extend CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep neural networks (Deep CORAL). Experiments on standard benchmark datasets show state-of-the-art performance. Our code is available at: https://github.com/VisionLearningGroup/CORAL.
This article proposes an approach based on experiments to teach optimization technique (OT) courses in the Systems Engineering curricula at undergraduate level. Artificial intelligence techniques in terms of nature-inspired optimization algorithms and neural networks are inserted in the lecture and laboratory parts of the syllabus. The experiments are included in the laboratory part of the syllabus by first involving controlled process analysis and modeling, control structures and algorithms, real-time laboratory experiments, and their assessment. These experiments are focused on the representative case of the pendulum-cart system control, and the genetic algorithm-based optimal tuning of proportional-integral-derivative and state-feedback controllers is carried out. The laboratory part of the syllabus deals next with the development of neural network-based models for the prediction of financial time series. An analysis of the grades obtained by representative groups of students that attended the OT course at the Politehnica University of Timisoara, Romania, and their effects on the process control structures and algorithms course, which continues the OT course in the next semester, is performed. The analysis also discusses the situation prior to using the proposed approach. The results of this analysis demonstrate the efficiency of our approach based on complex systems optimization, modeling, and control targeting real-world practical applications, and a numerical outcome of the approach is given. This allows students to gain a better understanding of the theoretical aspects acquired during the lectures in comparison with the situation prior to using the proposed approach.
Dynamic multiobjective optimization requires the robust tracking of varying Pareto-optimal solutions (POS) in a changing environment. When a change is detected in the environment, prediction mechanisms estimate the POS by utilizing information from previous populations to accelerate search toward the true POS. To achieve a robust prediction of POS, a mixture-of-experts-based ensemble framework is proposed. Unlike existing approaches, the framework utilizes multiple prediction mechanisms to improve the overall prediction. A gating network is applied to manage switching among the various predictors based on performance of the predictors at different time intervals of the optimization process. The efficacy of the proposed framework is validated through experimental studies based on 13 dynamic multiobjective benchmark optimization problems. The simulation results show that the proposed framework improves the dynamic optimization performance significantly, particularly for: 1) problems with distinct dynamic POS in decision space over time and 2) problems with highly nonlinear decision variable linkages.
The expectation–maximization (EM) algorithm is a common approach for parameter estimation in the context of cluster analysis using finite mixture models. This approach suffers from the well-known issue of convergence to local maxima, but also the less obvious problem of overfitting. These combined, and competing, concerns are illustrated through simulation and then addressed by introducing an algorithm that augments the traditional EM with the nonparametric bootstrap. Further simulations and applications to real data lend support for the usage of this bootstrap augmented EM-style algorithm to avoid both overfitting and local maxima.
Many real-world multi-objective optimization problems (MOPs) are dynamic in which variables of search space and/or objective space change over time. Hence the optimization algorithms should can quickly and efficiently track the Pareto front in dealing with dynamic environments. In this paper, a hybrid population prediction strategy based on fuzzy inference and one-step prediction (FIOPPS) is presented to extrapolate ahead the trajectory (position and/or orientation) of the new Pareto optimal solution set from the previous Pareto optimal solution sets and ensure the algorithm to respond quickly and effectively when the environment changes thus tracking the changing Pareto front. In our algorithm, the fuzzy inference model based on the Maximum Entropy Principle is extracted automatically from the previously found Pareto optimal solution sets to predict the Pareto solution sets at the beginning of the next time. Moreover, a new one-step prediction model is proposed to improve the prediction accuracy for environmental changes from motion state to static state and vice versa. Furthermore, a new variant of teaching–learning-based optimization algorithm with decomposition is first proposed as the MOEA optimizer for solving dynamic multi-objective optimization problems (DMOPs). In the proposed MOTLBO/D variant, the multi-objective decomposition mechanism is adopted and neighbor strategy is introduced into teaching–learning-based optimization algorithm (TLBO) to maintain the diversity of population and avoid the algorithm trapping into the local areas. Finally, to verify the performance of the proposed methods, ten benchmark test functions are simulated and evaluated. The statistical results indicate that the proposed FIOPPS strategy is promising for dealing with DMOPs.
Characterization of dynamism is an important issue for utilizing or tailoring of several dynamic multi-objective evolutionary algorithms (DMOEAs). One such characterization is the change detection, which is based on proposing explicit schemes to detect the points in time when a change occurs. Additionally, detecting severity of change and incorporating with the DMOEAs is another attempt of characterization, where there is only a few related works presented in the literature. In this paper, we propose a type-detection mechanism for dynamic multi-objective optimization problems, which is one of the first attempts that investigate the significance of type detection on the performance of DMOEAs. Additionally, a hybrid technique is proposed which incorporates our type detection mechanism with a given DOMEA. We present an empirical evaluation by using seven test problems from all four types and five performance metrics, which clearly validate the motivation of type detection as well as significance of our hybrid technique.
The additions of oxygen and zinc oxide for the goethite process determine the cost and efficiency of the iron precipitation process. As the two production targets (cost and efficiency) are conflicting and the chemical reaction is a continuous process that changes over time, the amounts of additive need to be dynamically optimized to satisfy the requirement of industrial application. In this paper, a discretization method based on control variables and control intervals is proposed to transform the dynamic optimization problem to a nonlinear mathematical programming problem. Then, a multi-objective optimization approach based on the state transition algorithm and constrained nondominated sorting is proposed to find the Pareto optimal solutions. Finally, an evaluation mechanism is proposed to obtain the best solution for industrial applications. The results from a series of simulation experiments show the effectiveness of the proposed approach, e.g. the daily average additions of oxygen and zinc oxide are decreased by 778.0854 m³ and 4.9013 t, respectively.
In tracking the moving Pareto front of dynamic multi-objective optimization problem as soon as possible, a new algorithm based on reference point prediction (PDMOP) is proposed. Firstly, PDMOP distributes the past individuals to different time series according to the information of reference point association. Then for these time series, a linear regression model is used to predict the new environment population. At the same time, historical prediction error is added to the current prediction to enhance prediction accuracy, and a Gauss noise is added to every new individual to increase the initialized population diversity. In this way, the algorithm can speed up convergence in the new environment. The results of four benchmark problems and the comparison with other two existing dynamic multi-objective algorithms indicate that the proposed algorithm can maintain better performance in dealing with dynamic multiobjective problems.
Minimising the ongoing impact of train delays has benefits to both the users of the railway system and the railway stakeholders. However, the efficient rescheduling of trains after a perturbation is a complex real-world problem. The complexity is compounded by the fact that the problem may be both dynamic and multi-objective. The aim of this research is to investigate the ability of ant colony optimisation algorithms to solve a simulated dynamic multi-objective railway rescheduling problem and, in the process, to attempt to identify the features of the algorithms that enable them to cope with a multi-objective problem that is also dynamic. Results showed that, when the changes in the problem are large and frequent, retaining the archive of non-dominated solution between changes and updating the pheromones to reflect the new environment play an important role in enabling the algorithms to perform well on this dynamic multi-objective railway rescheduling problem.
Deep neural networks are able to learn powerful representations from large quantities of labeled input data, however they cannot always generalize well across changes in input distributions. Domain adaptation algorithms have been proposed to compensate for the degradation in performance due to domain shift. In this paper, we address the case when the target domain is unlabeled, requiring unsupervised adaptation. CORAL is a "frustratingly easy" unsupervised domain adaptation method that aligns the second-order statistics of the source and target distributions with a linear transformation. Here, we extend CORAL to learn a nonlinear transformation that aligns correlations of layer activations in deep neural networks (Deep CORAL). Experiments on standard benchmark datasets show state-of-the-art performance.
This paper presents a new algorithm, called steady-state and generational evolutionary algorithm, which combines the fast and steadily tracking ability of steady-state algorithms and good diversity preservation of generational algorithms, for handling dynamic multiobjective optimization. Unlike most existing approaches for dynamic multiobjective optimization, the proposed algorithm detects environmental changes and responds to them in a steady-state manner. If a change is detected, it reuses a portion of outdated solutions with good distribution and relocates a number of solutions close to the new Pareto front based on the information collected from previous environments and the new environment. This way, the algorithm can quickly adapt to changing environments and thus is expected to provide a good tracking ability. The proposed algorithm is tested on a number of bi- and three-objective benchmark problems with different dynamic characteristics and difficulties. Experimental results show that the proposed algorithm is very competitive for dynamic multiobjective optimization in comparison with state-of-the-art methods.
As the name suggests, multi-objective optimisation involves optimising a number of objectives simultaneously. The problem becomes challenging when the objectives are of conflicting characteristics to each other, that is, the optimal solution of an objective function is different from that of the other. In the course of solving such problems, with or without the presence of constraints, these problems give rise to a set of trade-off optimal solutions, popularly known as Pareto-optimal solutions. Because of the multiplicity in solutions, these problems were proposed to be solved suitably using evolutionary algorithms using a population approach in its search procedure. Starting with parameterised procedures in early 90s, the so-called evolutionary multi-objective optimisation (EMO) algorithms is now an established field of research and application with many dedicated texts and edited books, commercial softwares and numerous freely downloadable codes, a biannual conference series running successfully since 2001, special sessions and workshops held at all major evolutionary computing conferences, and full-time researchers from universities and industries from all around the globe. In this chapter, we provide a brief introduction to its operating principles and outline the current research and application studies of evolutionary multi-objective optmisation (EMO).
Evolutionary algorithms are effective in solving static multiobjective optimization problems resulting in the emergence of a number of state-of-the-art multiobjective evolutionary algorithms (MOEAs). Nevertheless, the interest in applying them to solve dynamic multiobjective optimization problems has only been tepid. Benchmark problems, appropriate performance metrics, as well as efficient algorithms are required to further the research in this field. One or more objectives may change with time in dynamic optimization problems. The optimization algorithm must be able to track the moving optima efficiently. A~prediction model can learn the patterns from past experience and predict future changes. In this paper, a new dynamic MOEA using Kalman filter (KF) predictions in decision space is proposed to solve the aforementioned problems. The predictions help to guide the search toward the changed optima, thereby accelerating convergence. A scoring scheme is devised to hybridize the KF prediction with a random reinitialization method. Experimental results and performance comparisons with other state-of-the-art algorithms demonstrate that the proposed algorithm is capable of significantly improving the dynamic optimization performance.
This work describes a forward-looking approach for the solution of dynamic (time-changing) problems using evolutionary algorithms. The main idea of the proposed method is to combine a forecasting technique with an evolutionary algorithm. The location, in variable space, of the optimal solution (or of the Pareto optimal set in multi-objective problems) is estimated using a forecasting method. Then, using this forecast, an anticipatory group of individuals is placed on and near the estimated location of the next optimum. This prediction set is used to seed the population when a change in the objective landscape arrives, aiming at a faster convergence to the new global optimum. The forecasting model is created using the sequence of prior optimum locations, from which an estimate for the next location is extrapolated. Conceptually this approach encompasses advantages of memory methods by making use of information available from previous time steps. Combined with a convergence/diversity balance mechanism it creates a robust algorithm for dynamic optimization. This strategy can be applied to single objective and multi-objective problems, however in this work it is tested on multi-objective problems. Initial results indicate that the approach improves algorithm performance, especially in problems where the frequency of objective change is high.
A novel dynamic multi-objective optimization evolutionary algorithm is proposed in this paper to track the Pareto-optimal set of time-changing multi-objective optimization problems. In the proposed algorithm, to initialize the new population when a change is detected, a modified prediction model utilizng the historical optimal sets obtained in the last two times is adopted. Meantime, to improve both convergence and diversity, a self-adaptive differential evolution crossover operator is used. We conducted two experiments: the first one compares the proposed algorithm with the other three dynamic multiobjective evolutionary algorithms, and the second one investigates the performance of the two proposed operators. The statistical results indicate that the proposed algorithm has better conergence speed and diversity and it is very promising for dealing with dynamic environment.
We introduce Adam, an algorithm for first-order gradient-based optimization
of stochastic objective functions. The method is straightforward to implement
and is based an adaptive estimates of lower-order moments of the gradients. The
method is computationally efficient, has little memory requirements and is well
suited for problems that are large in terms of data and/or parameters. The
method is also ap- propriate for non-stationary objectives and problems with
very noisy and/or sparse gradients. The method exhibits invariance to diagonal
rescaling of the gradients by adapting to the geometry of the objective
function. The hyper-parameters have intuitive interpretations and typically
require little tuning. Some connections to related algorithms, on which Adam
was inspired, are discussed. We also analyze the theoretical convergence
properties of the algorithm and provide a regret bound on the convergence rate
that is comparable to the best known results under the online convex
optimization framework. We demonstrate that Adam works well in practice when
experimentally compared to other stochastic optimization methods.
Dynamic multi-objective optimization problem (DMOP) is quite challenging and it dues to that there are multiple conflicting objects changing over with time or environment. In this paper, a novel cooperative coevolutionary dynamic multi-objective optimization algorithm (PNSCCDMO) is proposed. The main idea of a new cooperative coevolution based on non-dominated sorting is that it allows the decomposition process of the optimization problem according to the search space of decision variables, and each species subcomponents will cooperate to evolve for better solutions. This way derives from nature and can improve convergence significantly. A modified linear regression prediction strategy is used to make rapid response to the new changes in the environment. The effectiveness of PNSCCDMO is validated against various of DMOPs compared with the other four algorithms, and the experimental result indicates PNSCCDMO has a good capability to track the Pareto front as it is changed with time in dynamic environments.
The supply trajectory of electric power for submerged arc magnesia furnace determines the yields and grade of magnesia grain during the manufacture process. As the two production targets (i.e., the yields and the grade of magnesia grain) are conflicting and the process is subject to changing conditions, the supply of electric power needs to be dynamically optimized to track the moving Pareto optimal set with time. A hybrid evolutionary multiobjective optimization strategy is proposed to address the dynamic multiobjective optimization problem. The hybrid strategy is based on two techniques. The first one uses case-based reasoning to immediately generate good solutions to adjust the power supply once the environment changes, and then apply a multiobjective evolutionary algorithm to accurately solve the problem. The second one is to learn the case solutions to guide and promote the search of the evolutionary algorithm, and the best solutions found by the evolutionary algorithm can be used to update the case library to improve the accuracy of case-based reasoning in the following process. Due to the effectiveness of mutual promotion, the hybrid strategy can continuously adapt and search in dynamic environments. Two prominent multiobjective evolutionary algorithms are integrated into the hybrid strategy to solve the dynamic multiobjective power supply optimization problem. The results from a series of experiments show that the proposed hybrid algorithms perform better than their component multiobjective evolutionary algorithms for the tested problems.
Most real-world optimization problems involve objectives, constraints, and parameters which constantly change with time. Treating
such problems as a stationary optimization problem demand the knowledge of the pattern of change a priori and even then the
procedure can be computationally expensive. Although dynamic consideration using evolutionary algorithms has been made for
single-objective optimization problems, there has been a lukewarm interest in formulating and solving dynamic multi-objective
optimization problems. In this paper, we modify the commonly-used NSGA-II procedure in tracking a new Pareto-optimal front,
as soon as there is a change in the problem. Introduction of a few random solutions or a few mutated solutions are investigated
in detail. The approaches are tested and compared on a test problem and a real-world optimization of a hydro-thermal power
scheduling problem. This systematic study is able to find a minimum frequency of change allowed in a problem for two dynamic
EMO procedures to adequately track Pareto-optimal frontiers on-line. Based on these results, this paper also suggests an automatic
decision-making procedure for arriving at a dynamic single optimal solution on-line.
As the research of dynamic optimization arising, memory-based strategy has gained public attention recently. However, few studies on developing dynamic multi-objective optimization algorithms and even fewer studies on multi-objective memory-based strategy were reported previously. In this paper, we try to address such an issue by proposing several memory-based multi-objective evolutionary algorithms and experimentally investigating different multi-objective dynamic optimization schemes, which include restart, explicit memory, local search memory and hybrid memory schemes. This study is to provide pre-trial research of how to appropriately organize and effectively reuse the changed Pareto-optimal decision values (i.e., Pareto-optimal solutions: POS) information.
In addition to the need for satisfying several competing objectives, many real-world applications are also dynamic and require the optimization algorithm to track the changing optimum over time. This paper proposes a new coevolutionary paradigm that hybridizes competitive and cooperative mechanisms observed in nature to solve multiobjective optimization problems and to track the Pareto front in a dynamic environment. The main idea of competitive-cooperative coevolution is to allow the decomposition process of the optimization problem to adapt and emerge rather than being hand designed and fixed at the start of the evolutionary optimization process. In particular, each species subpopulation will compete to represent a particular subcomponent of the multiobjective problem, while the eventual winners will cooperate to evolve for better solutions. Through such an iterative process of competition and cooperation, the various subcomponents are optimized by different species subpopulations based on the optimization requirements of that particular time instant, enabling the coevolutionary algorithm to handle both the static and dynamic multiobjective problems. The effectiveness of the competitive-cooperation coevolutionary algorithm (COEA) in static environments is validated against various multiobjective evolutionary algorithms upon different benchmark problems characterized by various difficulties in local optimality, discontinuity, nonconvexity, and high-dimensionality. In addition, extensive studies are also conducted to examine the capability of dynamic COEA (dCOEA) in tracking the Pareto front as it changes with time in dynamic environments.
This work describes a forward-looking approach for the solution of dynamic (time-changing) problems using evolutionary algorithms. The main idea of the proposed method is to combine a forecasting technique with an evolutionary algorithm. The location, in variable space, of the optimal solution (or of the Pareto optimal set in multi-objective problems) is estimated using a forecasting method. Then, using this forecast, an anticipatory group of individuals is placed on and near the estimated location of the next optimum. This prediction set is used to seed the population when a change in the objective landscape arrives, aiming at a faster convergence to the new global optimum. The forecasting model is created using the sequence of prior optimum locations, from which an estimate for the next location is extrapolated. Conceptually this approach encompasses advantages of memory methods by making use of information available from previous time steps. Combined with a convergence/diversity balance mechanism it creates a robust algorithm for dynamic optimization. This strategy can be applied to single objective and multi-objective problems, however in this work it is tested on multi-objective problems. Initial results indicate that the approach improves algorithm performance, especially in problems where the frequency of objective change is high.