An efficient hidden layer training method for the multilayer perceptron

Department of Electrical Engineering, University of Texas at Arlington, Arlington, Texas, United States
Neurocomputing (Impact Factor: 2.08). 12/2006; 70(1):525-535. DOI: 10.1016/j.neucom.2005.11.008


A u t h o r ' s p e r s o n a l c o p y Abstract The output-weight-optimization and hidden-weight-optimization (OWO–HWO) training algorithm for the multilayer perceptron alternately solves linear equations for output weights and reduces a separate hidden layer error function with respect to hidden layer weights. Here, three major improvements are made to OWO–HWO. First, a desired net function is derived. Second, using the classical mean square error, a weighted hidden layer error function is derived which de-emphasizes net function errors that correspond to saturated activation function values. Third, an adaptive learning factor based on the local shape of the error surface is used in hidden layer training. Faster learning convergence is experimentally verified, using three training data sets.

Download full-text


Available from: Pramod Lakshmi Narasimha, Oct 03, 2015
33 Reads
  • Source
    • "Then, these selected features were the input to a feature-extraction algorithm. In order to find near-optimal feature subsets, the 63 features were evaluated using the piecewise linear orthonormal floating search (PLOFS) algorithm (Li, Manry, Narasimha, & Yu, 2006). PLOFS comprises: (1) the floating search subset generation algorithm; and (2) a piecewise linear network's mean-squared error (MSE) to calculate the subset evaluation function (Chandrasekaran & Kim, 1999; Fukunaga, 1990; Pudil, Novovicová, & Kittler, 1994; Yu & Manry, 2007). "
    [Show abstract] [Hide abstract]
    ABSTRACT: Functional near infrared spectroscopy (fNIRS) was used to explore hemodynamic responses in the human frontal cortex to noxious thermal stimulation over the right temporomandibular joint (TMJ). fNIRS experiments were performed on nine healthy volunteers under both low-pain stimulation (LPS) and high-pain stimulation (HPS), using a temperature-controlled thermal stimulator. By analyzing the temporal profiles of changes in oxy-hemoglobin concentration (HbO) using cluster-based statistical tests, several regions of interest in the prefrontal cortex, such as the dorsolateral prefrontal cortex and the anterior prefrontal cortex, were identified, where significant differences ( p < .05) between HbO responses to LPS and HPS were shown. In order to classify these two levels of pain, a neural network-based classification algorithm was utilized. With leave-one-out crossvalidation, the two levels of pain were identified with 99% mean accuracy to high pain. Furthermore, the “internal mentation hypothesis” and the default-mode network were introduced to explain our observations of the contrasting trend, as well as the rise and fall of HbO responses to HPS and LPS.
    Journal of Applied Biobehavioral Research 09/2013; 18(3):134-155. DOI:10.1111/jabr.12009
  • Source
    • "An improved version of back propagation algorithm is used for breast cancer detection [11]. Changhua [12] implemented the multilayer feed forward network with a modified training algorithm. A different version of back propagation neural network is employed for pattern recognition [13]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Abnormal Magnetic Resonance (MR) brain image classification is amandatory but challenging task in the medical field. Accurate identification ofthe nature of the disease is highly essential for the successful treatmentplanning. Automated systems are highly preferred for image classificationbecause of its high accuracy. Artificial neural networks are one of the widelyused automated techniques. Though they yield high accuracy, most of theneural networks are computationally heavy due to their iterative nature. Lowspeed neural classifiers are least preferred since they are practically nonfeasible.Hence, there is a significant requirement for a neural classifier whichis computationally efficient and highly accurate. To satisfy these criterions, amodified Counter Propagation Neural Network (CPN) is proposed in this workwhich proves to be much faster than the conventional network. For furtherenhancement of the performance of the classifier, Particle Swarm Optimization(PSO) technique is used in conjunction with the modified CPN. Experimentsare conducted on these classifiers using real-time abnormal images collectedfrom the scan centres. These three types of classifiers are analyzed in terms ofclassification accuracy and convergence time period. Experimental results showpromising results for the PSO based modified CPN classifier in terms of theperformance measures.
  • Source
    • "또한, 학습속도를 빠르게 하기 위한 방법들도 제안되 었는데, 학습률을 변동시키는 방법으로는 Bold Driver[12]나 Delta-Bar-Dela[13] 등이 있다. 또한, MLP의 각 층을 분리해서 학습시키는 LBL (Layer-By-Layer)방법[14][15]과 이를 위하여 중간층 의 오차함수를 일반적인 MSE가 아니라 시그모이드 함 수 기울기를 고려하여 변경시키는 방법도 제안되었다 [16]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: When we apply MLPs(multilayer perceptrons) to pattern classification problems, we generally allocate one output node for each class and the index of output node denotes a class. On the contrary, in this paper, we propose to increase the number of output nodes per each class for performance improvement of MLPs. For theoretical backgrounds, we derive the misclassification probability in two class problems with additional outputs under the assumption that the two classes have equal probability and outputs are uniformly distributed in each class. Also, simulations of 50 isolated-word recognition show the effectiveness of our method.
    01/2009; 9(1). DOI:10.5392/JKCA.2009.9.1.123
Show more