-
Kunihiko Fukushima
[show abstract]
[hide abstract]
ABSTRACT: The neocognitron is a hierarchical multi-layered neural network capable of robust visual pattern recognition. It has been demonstrated that recent versions of the neocognitron exhibit excellent performance for recognizing handwritten digits. When characters are written on a noisy background, however, recognition rate was not always satisfactory. To find out the causes of vulnerability to noise, this paper analyzes the behavior of feature-extracting S-cells. It then proposes the use of subtractive inhibition to S-cells from V-cells, which calculate the average of input signals to the S-cells with a root-mean-square. Together with this, several modifications have also been applied to the neocognitron. Computer simulation shows that the new neocognitron is much more robust against background noise than the conventional ones.
Neural networks: the official journal of the International Neural Network Society 03/2011; 24(7):767-78. · 1.88 Impact Factor
-
Kunihiko Fukushima
[show abstract]
[hide abstract]
ABSTRACT: The neocognitron, which was proposed by Fukushima (1980), is a hierarchical multi-layered neural network capable of robust visual pattern recognition. It acquires the ability to recognize patterns through learning. This paper proposes a new rule for competitive learning, named winner-kill-loser, and apply it to the neocognitron. The winner-kill-loser rule resembles the winner-take-all rule. Every time when a training stimulus is presented, non-silent cells compete with each other. The winner, however, not only takes all, but also kills losers. In other words, the winner learns the training stimulus, and losers are removed from the network. If all cells are silent, a new cell is generated and it learns the training stimulus. Thus feature-extracting cells gradually come to distribute uniformly in the feature space. The use of winner-kill-loser rule is not limited to the neocognitron. It is useful for various types of competitive learning, in general. This paper also proposes several improvements made on the neocognitron: such as, disinhibition to the inhibitory surround in the connections to C-cells (or complex cells) from S-cells (or simple cells); and square root shaped saturation in the input-to-output characteristics of C-cells. As a result of these improvements, the recognition rate of the neocognitron has been largely increased.
Neural networks: the official journal of the International Neural Network Society 09/2010; 23(7):926-38. · 1.88 Impact Factor
-
Kunihiko Fukushima
[show abstract]
[hide abstract]
ABSTRACT: When some parts of a pattern are occluded by other objects, the visual system can often estimate the shape of occluded contours from visible parts of the contours. This paper proposes a neural network model capable of such function, which is called amodal completion. The model is a hierarchical multi-layered network that has bottom-up and top-down signal paths. It contains cells of area V1, which respond selectively to edges of a particular orientation, and cells of area V2, which respond selectively to a particular angle of bend. Using the responses of bend-extracting cells, the model predicts the curvature and location of the occluded contours. Missing contours are gradually extrapolated and interpolated from the visible contours. Computer simulation demonstrates that the model performs amodal completion to various stimuli in a similar way as observed by psychological experiments.
Neural networks: the official journal of the International Neural Network Society 10/2009; 23(4):528-40. · 1.88 Impact Factor
-
Kunihiko Fukushima
[show abstract]
[hide abstract]
ABSTRACT: In an earlier paper [Fukushima, K., & Tohyama, K. (2005). Analysis of optic flow: a neural network model. In ICONIP 2005, International Conference on Neural Information Processing (pp. 312-317); Tohyama, K., & Fukushima, K. (2005). Neural network model for extracting 98 optic flow. Neural Networks, 18(5-6), 549-556], we proposed a neural network extracting optic flow based on vector field hypothesis. We started discussion there, however, from the stage where local velocities of visual objects have already been extracted. This paper proposes a new mechanism of extracting local velocity from retinal images, and adds it to the previous network. The network has a hierarchical multilayered architecture. X- and Y-cells of the retina extract spatial and temporal contrast of brightness. The network contains several types of V1 cells, namely, S- C- and V-cells. S- and C-cells extract orientated edges. V-cells extract local velocities, based on the signals from Y- and S-cells. MT cells extract relative velocities between adjoining small visual fields. MST cells add the responses of MT cells to extract a specific optic flow, such as rotation and expansion/contraction. The difference in type of optic flow extracted by MST cells can be created simply by the difference in relative locations of inhibitory to excitatory areas in the receptive fields of MT cells.
Neural Networks 06/2008; 21(5):774-85. · 2.18 Impact Factor
-
Kunihiko Fukushima
[show abstract]
[hide abstract]
ABSTRACT: This paper proposes a powerful algorithm for pattern recognition, which uses interpolating vectors for classifying patterns. Labeled reference vectors in a multi-dimensional feature space are first produced by a kind of competitive learning. We then assume a situation where virtual vectors, called interpolating vectors, are densely placed along line segments connecting all pairs of reference vectors of the same label. From these interpolating vectors, we choose the one that has the largest similarity to the test vector. Its label shows the result of pattern recognition. In practice, we can get the same result with a simpler process. We applied this method to the neocognitron for handwritten digit recognition and reduced the error rate from 1.52% to 1.02% for a blind test set of 5000 digits.
Neural Networks 11/2007; 20(8):904-16. · 2.18 Impact Factor
-
Neurocomputing. 01/2006; 69:1827-1836.
-
Kunihiko Fukushima
[show abstract]
[hide abstract]
ABSTRACT: This paper shows that the introduction of non-uniform blur is very useful for comparing images, and proposes a neural network model that extracts axes of symmetry from visual patterns. The blurring operation greatly increases robustness against deformations and various kinds of noise, and largely reduces computational cost. Asymmetry between two groups of signals can be detected in a single action by the use of non-uniform blur having a cone-shaped distribution. The proposed model is a hierarchical multi-layered network, which consists of a contrast-extracting layer, edge-extracting layers (simple and complex types), and layers extracting symmetry axes. The model extracts oriented edges from an input image first, and then tries to extract axes of symmetry. The model checks conditions of symmetry, not directly from the oriented edges, but from a blurred version of the response of edge-extracting layer. The input patterns can be complicated line drawings, plane figures or gray-scaled natural images taken by CCD cameras.
Neural Networks 02/2005; 18(1):23-32. · 2.18 Impact Factor
-
Kunihiko Fukushima
[show abstract]
[hide abstract]
ABSTRACT: This paper proposes a neural network model that has an ability to restore missing portions of partly occluded patterns. It is a multi-layered hierarchical neural network, in which visual information is processed by interaction of bottom-up and top-down signals. Memories of learned patterns are stored in the connections between cells. Occluded parts of a pattern are reconstructed mainly by top-down signals from higher stages of the network, while the unoccluded parts are reproduced mainly by signals from lower stages. The restoration progresses successfully, even if the occluded pattern is a deformed version of a learned pattern. The model tries to complete even an unlearned pattern by interpolating and extrapolating visible edges. Resemblance of local features to other learned patterns are also utilized for the restoration.
Neural Networks 02/2005; 18(1):33-43. · 2.18 Impact Factor
-
Kunihiko Fukushima
[show abstract]
[hide abstract]
ABSTRACT: This paper proposes a new neocognitron that accepts incremental learning, without giving a severe damage to old memories or reducing learning speed. The new neocognitron uses a competitive learning, and the learning of all stages of the hierarchical network progresses simultaneously. To increase the learning speed, conventional neocognitrons of recent versions sacrificed the ability of incremental learning, and used a technique of sequential construction of layers, by which the learning of a layer started after the learning of the preceding layers had completely finished. If the learning speed is simply set high for the conventional neocognitron, simultaneous construction of layers produces many garbage cells, which become always silent after having finished the learning. The proposed neocognitron with a new learning method can prevent the generation of such garbage cells even with a high learning speed, allowing incremental learning.
Neural Networks 02/2004; 17(1):37-46. · 2.18 Impact Factor
-
Knowledge-Based Intelligent Information and Engineering Systems, 7th International Conference, KES 2003, Oxford, UK, September 3-5, 2003, Proceedings, Part II; 01/2003
-
Kunihiko Fukushima
[show abstract]
[hide abstract]
ABSTRACT: This paper proposes a new learning rule by which cells with shift-invariant receptive fields are self-organized. With this learning rule, cells similar to simple and complex cells in the primary visual cortex are generated in a network. To demonstrate the new learning rule, we simulate a three-layered network that consists of an input layer (or the retina), a layer of S-cells (or simple cells), and a layer of C-cells (or complex cells). During the learning, straight lines of various orientations sweep across the input layer. Here both S- and C-cells are created through competition. Although S-cells compete depending on their instantaneous outputs, C-cells compete depending on the traces (or temporal averages) of their outputs. For the self-organization of S-cells, only winner S-cells increase their input connections in a similar way to that for the neocognitron. In other words, the winner S-cells have LTP (long term potentiation) in their input connections. For the self-organization of C-cells, however, loser C-cells decrease their input connections (LTD=long term depression), while winners increase their input connections (LTP). Here both S- and C-cells are accompanied by inhibitory cells. Modification of inhibitory connections together with excitatory connections is important for creation of C-cells as well as S-cells.
Neural networks: the official journal of the International Neural Network Society 08/1999; 12(6):791-801. · 1.88 Impact Factor
-
The Fifth International Conference on Neural Information Processing, ICONIP'R98, Kitakyushu, Japan, October 21-23, 1998, Proceedings; 01/1998
-
[show abstract]
[hide abstract]
ABSTRACT: This paper offers a neural network model that can memorize and recall spatial maps. When driving through a place we have been before, we can recall and imagine the scenery that we cannot see yet but shall see soon. Triggered by the newly recalled image, we can also recall other scenery further ahead of us. The model emulates such a chain process of recalling using a correlation matrix memory. A correlation matrix memory by itself, however, does not accept shifts in location of stimulus patterns, and each stimulus pattern has to be placed accurately at the location of one of the memorized patterns. We propose adjusting the location of the stimulus pattern using the cross-correlation between the stimulus pattern and the "piled pattern", which is the sum of all patterns memorized in the correlation matrix. A map of Europe is divided into a number of overlapping segments, and these segments are memorized in the proposed model. Triggered by an input image, say a map around Scotland, the model can recall maps of other parts of Europe sequentially up to Italy, for example. Copyright 1997 Elsevier Science Ltd.
Neural networks: the official journal of the International Neural Network Society 09/1997; 10(6):971-979. · 1.88 Impact Factor
-
[show abstract]
[hide abstract]
ABSTRACT: We propose a neural network model of the visual system of the brain which processes different kinds of attributes such as form and motion in parallel. The model has two separate channels: a channel processing form and a channel processing motion. Each channel has both forward and backward connections, and exhibits selective attention. The selective attention mechanism, however, does not work independently because of the interaction of the two channels, which occurs at their lower layers. Both channels always focus attention on the same object even when many objects are presented simultaneously to the input layer of the model. The model was simulated on a computer: several objects made of moving random dots were applied to the input layer. At first the model focused attention on one of the objects, and detected its form and motion. It then processed the rest of the objects in turn by switching attention. Copyright 1996 Elsevier Science Ltd.
Neural networks: the official journal of the International Neural Network Society 12/1996; 9(8):1417-1427. · 1.88 Impact Factor
-
Artificial Neural Networks - ICANN 96, 1996 International Conference, Bochum, germany, July 16-19, 1996, Proceedings; 01/1996
-
[show abstract]
[hide abstract]
ABSTRACT: We propose a neural network model of the 2D invariant pattern recognition including a mechanism of saccadic eye movement. The model extracts every spatial relation between two primitive features (feature-relation) from a stimulus. The mechanism of saccadic eye movement enables the network to obtain all feature-relations present in the stimulus. This set of feature-relations describe uniquely the shape of pattern independent of position, scale, and deformation. Recognition of the presented pattern is achieved by classifying the obtained set of feature-relations. The ability of the model is confirmed by computer-simulation.
Neurocomputing.
-
[show abstract]
[hide abstract]
ABSTRACT: When we travel in an environment, we have an optic flow on the retina. Neurons in the area MST of macaque monkeys are reported to have a very large receptive field and analyze optic flows on the retina. Many MST-cells respond selectively to rotation, expansion/contraction and planar motion of the optic flow. Many of them show position-invariant responses to optic flow, that is, their responses are maintained during the shift of the center of the optic flow. It has long been suggested mathematically that vector-field calculus is useful for analyzing optic flow field. Biologically, plausible neural network models based on this idea, however, have little been proposed so far. This paper, based on vector-field hypothesis, proposes a neural network model for extracting optic flows. Our model consists of hierarchically connected layers: retina, V1, MT and MST. V1-cells measure local velocity. There are two kinds of MT-cell: one is for extracting absolute velocities, the other for extracting relative velocities with their antagonistic inputs. Collecting signals from MT-cells, MST-cells respond selectively to various types of optic flows. We demonstrate through a computer simulation that this simple network is enough to explain a variety of results of neurophysiological experiments.
Neural Networks 18(5-6):549-56. · 2.18 Impact Factor
