Conference Paper

Opposite Transfer Functions and Backpropagation Through Time

Department of Systems Design Engineering, University of Waterloo, Ватерлоо, Ontario, Canada
DOI: 10.1109/FOCI.2007.371529 Conference: Proceedings of the IEEE Symposium on Foundations of Computational Intelligence, FOCI 2007, part of the IEEE Symposium Series on Computational Intelligence 2007, Honolulu, Hawaii, USA, 1-5 April 2007
Source: DBLP


Backpropagation through time is a very popular discrete-time recurrent neural network training algorithm. However, the computational time associated with the learning process to achieve high accuracy is high. While many approaches have been proposed that alter the learning algorithm, this paper presents a computationally inexpensive method based on the concept of opposite transfer functions to improve learning in the backpropagation through time algorithm. Specifically, we will show an improvement in the accuracy, stability as well as an acceleration in learning time. We will utilize three common benchmarks to provide experimental evidence of the improvements

Download full-text


Available from: Hamid R. Tizhoosh, May 10, 2014
  • Source
    • "As a potential future work, one may also investigate the incorporation of the new barcode technology [18] into retrievaloriented classification combined with optimization techniques that employ the concept of opposite entities [25], [26], [27]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Good results on image classification and retrieval using support vector machines (SVM) with local binary patterns (LBPs) as features have been extensively reported in the literature where an entire image is retrieved or classified. In contrast, in medical imaging, not all parts of the image may be equally significant or relevant to the image retrieval application at hand. For instance, in lung x-ray image, the lung region may contain a tumour, hence being highly significant whereas the surrounding area does not contain significant information from medical diagnosis perspective. In this paper, we propose to detect salient regions of images during training and fold the data to reduce the effect of irrelevant regions. As a result, smaller image areas will be used for LBP features calculation and consequently classification by SVM. We use IRMA 2009 dataset with 14,410 x-ray images to verify the performance of the proposed approach. The results demonstrate the benefits of saliency-based folding approach that delivers comparable classification accuracies with state-of-the-art but exhibits lower computational cost and storage requirements, factors highly important for big data analytics.
    The 14th International Conference on Machine Learning and Applications (IEEE ICMLA’15), Miami, Florida, USA; 12/2015
  • Source
    • "In order to implement a complete solution, we use LBP features and SVM to classify the images. One can, in future works, investigate the use of opposites as already reported in iterative for learning and optimization in order to see the effect of using a network incorporating opposites [46], [47], [48]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Content-based image retrieval (CBIR) of medical images is a crucial task that can contribute to a more reliable diagnosis if applied to big data. Recent advances in feature extraction and classification have enormously improved CBIR results for digital images. However, considering the increasing accessibility of big data in medical imaging, we are still in need of reducing both memory requirements and computational expenses of image retrieval systems. This work proposes to exclude the features of image blocks that exhibit a low encoding error when learned by a $n/p/n$ autoencoder ($p\!<\!n$). We examine the histogram of autoendcoding errors of image blocks for each image class to facilitate the decision which image regions, or roughly what percentage of an image perhaps, shall be declared relevant for the retrieval task. This leads to reduction of feature dimensionality and speeds up the retrieval process. To validate the proposed scheme, we employ local binary patterns (LBP) and support vector machines (SVM) which are both well-established approaches in CBIR research community. As well, we use IRMA dataset with 14,410 x-ray images as test data. The results show that the dimensionality of annotated feature vectors can be reduced by up to 50% resulting in speedups greater than 27% at expense of less than 1% decrease in the accuracy of retrieval when validating the precision and recall of the top 20 hits.
    International Conference on Image Processing, Tools, Theory and Applications (IPTA); 07/2015
  • Source
    • "The results demonstrate that ORL outperforms the reinforcement learning. Similarly, by considering the opposite transfer functions and opposite weights, the opposition-based neural networks were also proposed to improve their learning speed and accuracy [23]– [25]. "
    [Show abstract] [Hide abstract]
    ABSTRACT: Opposition-based Learning (OBL) has been reported with an increased performance in enhancing various optimization approaches. Instead of investigating the opposite point of a candidate in OBL, this study proposed a partial opposition-based learning (POBL) schema that focuses a set of partial opposite points (or partial opposite population) of an estimate. Furthermore, a POBL-based adaptive differential evolution algorithm (POBL-ADE) is proposed to improve the effectiveness of ADE. The proposed algorithm is evaluated on the CEC2014's test suite in the special session and competition for real parameter single objective optimization in IEEE CEC 2014. Simulation results over the benchmark functions demonstrate the effectiveness and improvement of the POBL-ADE compared with ADE.
    2014 IEEE Congress on Evolutionary Computation (CEC), Beijing, China; 07/2014
Show more