An efficient hidden layer training method for the multilayer perceptron

FastVDO LLC, 21046, Columbia, MD, USA; Department of Electrical Engineering, University of Texas at Arlington, 76019, TX; Department of Radiology, Clinical Center National Institutes of Health, 20892, Bethesda, MD, USA
Neurocomputing (Impact Factor: 2.01). 12/2006; 70:525-535. DOI: 10.1016/j.neucom.2005.11.008

ABSTRACT A u t h o r ' s p e r s o n a l c o p y Abstract The output-weight-optimization and hidden-weight-optimization (OWO–HWO) training algorithm for the multilayer perceptron alternately solves linear equations for output weights and reduces a separate hidden layer error function with respect to hidden layer weights. Here, three major improvements are made to OWO–HWO. First, a desired net function is derived. Second, using the classical mean square error, a weighted hidden layer error function is derived which de-emphasizes net function errors that correspond to saturated activation function values. Third, an adaptive learning factor based on the local shape of the error surface is used in hidden layer training. Faster learning convergence is experimentally verified, using three training data sets.

  • [Show abstract] [Hide abstract]
    ABSTRACT: This paper proposes a constructing-and-pruning (CP) approach to optimise the structure of a feedforward neural network (FNN) with a single hidden layer. The number of hidden nodes or neurons is determined by their contribution ratios, which are calculated using a Fourier decomposition of the variance of the FNN's output. Hidden nodes with sufficiently small contribution ratios will be eliminated, while new nodes will be added when the FNN cannot satisfy certain design objectives. This procedure is similar to the growing and pruning processes observed in biological neural networks. The performance of the proposed method is evaluated using a number of examples: real-life date classification, dynamic system identification, and the key variables modelling in a wastewater treatment system. Experimental results show that the proposed method effectively optimises the network structure and performs better than some existing algorithms.
    Neurocomputing. 01/2013; 99:347–357.
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: When we apply MLPs(multilayer perceptrons) to pattern classification problems, we generally allocate one output node for each class and the index of output node denotes a class. On the contrary, in this paper, we propose to increase the number of output nodes per each class for performance improvement of MLPs. For theoretical backgrounds, we derive the misclassification probability in two class problems with additional outputs under the assumption that the two classes have equal probability and outputs are uniformly distributed in each class. Also, simulations of 50 isolated-word recognition show the effectiveness of our method.
    The Journal of the Korea Contents Association. 01/2009; 9(1).
  • Source
    [Show abstract] [Hide abstract]
    ABSTRACT: Functional near infrared spectroscopy (fNIRS) was used to explore hemodynamic responses in the human frontal cortex to noxious thermal stimulation over the right temporomandibular joint (TMJ). fNIRS experiments were performed on nine healthy volunteers under both low-pain stimulation (LPS) and high-pain stimulation (HPS), using a temperature-controlled thermal stimulator. By analyzing the temporal profiles of changes in oxy-hemoglobin concentration (HbO) using cluster-based statistical tests, several regions of interest in the prefrontal cortex, such as the dorsolateral prefrontal cortex and the anterior prefrontal cortex, were identified, where significant differences ( p < .05) between HbO responses to LPS and HPS were shown. In order to classify these two levels of pain, a neural network-based classification algorithm was utilized. With leave-one-out crossvalidation, the two levels of pain were identified with 99% mean accuracy to high pain. Furthermore, the “internal mentation hypothesis” and the default-mode network were introduced to explain our observations of the contrasting trend, as well as the rise and fall of HbO responses to HPS and LPS.
    Journal of Applied Biobehavioral Research 09/2013; 18(3):134-155.

Full-text (2 Sources)

Available from
May 17, 2014