Multiobjective hybrid optimization and training of recurrent neural networks.

Department of Computer Science and Artificial Intelligence, University of Grenada, Grenada, Spain.
IEEE TRANSACTIONS ON CYBERNETICS (Impact Factor: 3.47). 05/2008; 38(2):381-403. DOI: 10.1109/TSMCB.2007.912937
Source: PubMed

ABSTRACT The application of neural networks to solve a problem involves tasks with a high computational cost until a suitable network is found, and these tasks mainly involve the selection of the network topology and the training step. We usually select the network structure by means of a trial-and-error procedure, and we then train the network. In the case of recurrent neural networks (RNNs), the lack of suitable training algorithms sometimes hampers these procedures due to vanishing gradient problems. This paper addresses the simultaneous training and topology optimization of RNNs using multiobjective hybrid procedures. The proposal is based on the SPEA2 and NSGA2 algorithms for making hybrid methods using the Baldwinian hybridization strategy. We also study the effects of the selection of the objectives, crossover, and mutation in the diversity during evolution. The proposals are tested in the experimental section to train and optimize the networks in the competition on artificial time-series (CATS) benchmark.

  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Evolutionary Algorithms (EAs) are population based algorithms, which allow for simultaneous exploration of different parts in the Pareto optimal set. This paper presents Memetic Elitist Pareto Evolutionary Algorithm of Three-Term Backpropagation Network for Classification Problems. This memetic elitist Pareto evolutionary algorithm is called METBP and used to evolve Three-term Backpropagation (TBP) network, which are optimal with respect to connection weight, error rates and architecture complexity simultaneously. METPB is based on NSGA-II benefit from the local search algorithm that used to enhance the individuals in the population of the algorithm. The numerical results of METPB show the advantages of the combination of the local search algorithm, and it is able to obtain a TBP network with better classification accuracy and simpler structure when compared with a multiobjective genetic algorithm based TBP network (MOGATBP) and some methods found in the literature, the results indicate that the proposed method is a potentially useful classifier for enhancing classification process ability.
  • Source
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: A space-time adaptive decision feedback (DF) receiver using recurrent neural networks (RNNs) is proposed for joint equalization and interference suppression in direct-sequence code-division multiple-access (DS-CDMA) systems equipped with antenna arrays. The proposed receiver structure employs dynamically driven RNNs in the feedforward section for equalization and multiaccess interference (MAI) suppression and a finite impulse response (FIR) linear filter in the feedback section for performing interference cancellation. A data selective gradient algorithm, based upon the set-membership (SM) design framework, is proposed for the estimation of the coefficients of RNN structures and is applied to the estimation of the parameters of the proposed neural receiver structure. Simulation results show that the proposed techniques achieve significant performance gains over existing schemes.
    IEEE Transactions on Neural Networks 12/2008; 19(11):1887-95. DOI:10.1109/TNN.2008.2003286 · 2.95 Impact Factor