Multiobjective hybrid optimization and training of recurrent neural networks.

Department of Computer Science and Artificial Intelligence, University of Grenada, Grenada, Spain.
IEEE Transactions on Systems Man and Cybernetics Part B (Cybernetics) (Impact Factor: 3.24). 05/2008; 38(2):381-403. DOI: 10.1109/TSMCB.2007.912937
Source: PubMed

ABSTRACT The application of neural networks to solve a problem involves tasks with a high computational cost until a suitable network is found, and these tasks mainly involve the selection of the network topology and the training step. We usually select the network structure by means of a trial-and-error procedure, and we then train the network. In the case of recurrent neural networks (RNNs), the lack of suitable training algorithms sometimes hampers these procedures due to vanishing gradient problems. This paper addresses the simultaneous training and topology optimization of RNNs using multiobjective hybrid procedures. The proposal is based on the SPEA2 and NSGA2 algorithms for making hybrid methods using the Baldwinian hybridization strategy. We also study the effects of the selection of the objectives, crossover, and mutation in the diversity during evolution. The proposals are tested in the experimental section to train and optimize the networks in the competition on artificial time-series (CATS) benchmark.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: This paper presents a new multiobjective evolutionary algorithm applied to a radial basis function (RBF) network design based on multiobjective particle swarm optimization augmented with local search features. The algorithm is named the memetic multiobjective particle swarm optimization RBF network (MPSON) because it integrates the accuracy and structure of an RBF network. The proposed algorithm is implemented on two-class and multiclass pattern classification problems with one complex real problem. The experimental results indicate that the proposed algorithm is viable, and provides an effective means to design multiobjective RBF networks with good generalization capability and compact network structure. The accuracy and complexity of the network obtained by the proposed algorithm are compared with the memetic non-dominated sorting genetic algorithm based RBF network (MGAN) through statistical tests. This study shows that MPSON generates RBF networks coming with an appropriate balance between accuracy and simplicity, outperforming the other algorithms considered.
    Information Sciences 12/2013; 239:165–190. · 3.64 Impact Factor
  • Source
    Life Science Journal 12/2013; · 0.17 Impact Factor
  • [Show abstract] [Hide abstract]
    ABSTRACT: Pareto dominance is one of the most basic concepts in multi-objective optimization. However, it is inefficient when the number of objectives is large because in this case it leads to an unmanageable number of Pareto solutions. In order to solve this problem, a new concept of logic dominance is defined by considering the number of improved objectives and the quantity of improvement simultaneously, where probabilistic logic is applied to measure the quantity of improvement. Based on logic dominance, the corresponding logic nondominated solution is defined as a feasible solution which is not dominated by other ones based on this new relationship, and it is proved that each logic nondominated solution is also a Pareto solution. Essentially, logic dominance is an extension of Pareto dominance. Since there are already several extensions for Pareto dominance, some comparisons are given in terms of numerical examples, which indicates that logic dominance is more efficient. As an application of logic dominance, a house choice problem with five objectives is considered.
    Applied Mathematics and Computation. 01/2009; 215:3045-3056.