Content uploaded by Thomas Martinetz
Author content
All content in this area was uploaded by Thomas Martinetz on Mar 05, 2017
Content may be subject to copyright.
A "Neural-Gas" Network Learns Topologies
... The Neural Gas Network (NGN) [17] is a type of artificial neural network that adapts to input data without a predetermined network structure, efficiently organizing itself to reflect the topology of the data it processes. This flexibility makes NGN particularly useful in applications such as vector quantization [15], clustering, dimensionality reduction 2 , image segmentation [12], and feature extraction [13]. ...
... The Neural Gas Network (NGN) [17] is a type of artificial neural network known for its adaptability and self-organizing capabilities. Unlike traditional neural networks, NGN does not require a pre-defined network structure. ...
Data scarcity remains a significant challenge in the field of emotion recognition using physiological signals, as acquiring comprehensive and diverse datasets is often prevented by privacy concerns and logistical constraints. This limitation restricts the development and generalization of robust emotion recognition models, making the need for effective synthetic data generation methods more critical. Emotion recognition from physiological signals such as EEG, ECG, and GSR plays a pivotal role in enhancing human-computer interaction and understanding human affective states. Utilizing these signals, this study introduces an innovative approach to synthetic data generation using a Supervised Neural Gas (SNG) network, which has demonstrated noteworthy speed advantages over established models like Conditional VAE, Conditional GAN, diffusion model, and Variational LSTM. The Neural Gas network, known for its adaptability in organizing data based on topological and feature-space proximity, provides a robust framework for generating real-world-like synthetic datasets that preserve the intrinsic patterns of physiological emotion data. Our implementation of the SNG efficiently processes the input data, creating synthetic instances that closely mimic the original data distributions, as demonstrated through comparative accuracy assessments. In experiments, while our approach did not universally outperform all models, it achieved superior performance against most of the evaluated models and offered significant improvements in processing time. These outcomes underscore the potential of using SNG networks for fast, efficient, and effective synthetic data generation in emotion recognition applications.
... The Neural Gas Network (NGN) [24] is a machine learning algorithm designed to learn topologies in an unsupervised manner, similar to Self-Organizing Maps. It excels in clustering and visualizing highdimensional data by iteratively adapting to a set of input vectors, reducing the dimensions while preserving the topological properties of the input space. ...
In the domain of emotion recognition using body motion, the primary challenge lies in the scarcity of diverse and generalizable datasets. Automatic emotion recognition uses machine learning and artificial intelligence techniques to recognize a person's emotional state from various data types, such as text, images, sound, and body motion. Body motion poses unique challenges as many factors, such as age, gender, ethnicity, personality, and illness, affect its appearance, leading to a lack of diverse and robust datasets specifically for emotion recognition. To address this, employing Synthetic Data Generation (SDG) methods, such as Generative Adversarial Networks (GANs) and Variational Auto Encoders (VAEs), offers potential solutions, though these methods are often complex. This research introduces a novel application of the Neural Gas Network (NGN) algorithm for synthesizing body motion data and optimizing diversity and generation speed. By learning skeletal structure topology, the NGN fits the neurons or gas particles on body joints. Generated gas particles, which form the skeletal structure later on, will be used to synthesize the new body posture. By attaching body postures over frames, the final synthetic body motion appears. We compared our generated dataset against others generated by GANs, VAEs, and another benchmark algorithm, using benchmark metrics such as Fréchet Inception Distance (FID), Diversity, and a few more. Furthermore, we continued evaluation using classification metrics such as accuracy, precision, recall, and a few others. Joint-related features or kinematic parameters were extracted, and the system assessed model performance against unseen data. Our findings demonstrate that the NGN algorithm produces more realistic and emotionally distinct body motion data and does so with more synthesizing speed than existing methods.
... The first one is knowledge representation learning and refinement. TOPIC [1] incorporates a neural gas network [20] to obtain and store the topology structure of the feature space. Zhang et al. [2] proposed the Continually Evolved Classifier (CEC) which decoupled representation learning and classification and employed a graph model as the representation learner to propagate the global context between old and new sessions. ...
This paper introduces a novel method for Semi-Supervised Few-Shot Class Incremental Learning (SSFSCIL) that exhibits virtually no catastrophic forgetting. The method uses a generic feature extractor that was pretrained without supervision on a large image dataset, and a classifier based on a Probabilistic PCA (PPCA) model for each class instead of the standard fully connected layer usually employed as the projection head. The PPCA models are localized around the class means and the models for existing classes are not retrained when new classes are added. The learning algorithm is a modified k-Means that freezes the models on the existing classes and only updates models for the new classes. This makes the approach both computationally efficient and accurate. Extensive experiments on CUB200, CIFAR100, and miniImageNet show the effectiveness of the proposed approach. Additionally, experiments on the ImageNet-1k dataset, which previous methods have avoided due to its size, demonstrate its applicability to large-scale datasets.
... In the most general form, the algorithm sequentially grows the nodes and adjusts the graph to the input data. In this way, each node of the graph has assigned a neural weight in the input space, and the algorithm sequentially adds or/and removes nodes based on cumulative error measurements between the nodes and the data [43,44]. An important aspect of the GNG is the position of the first two nodes. ...
Wearable cameras stand out as one of the most promising devices for the upcoming years, and as a consequence, the demand of computer algorithms to automatically understand the videos recorded with them is increasing quickly. An automatic understanding of these videos is not an easy task, and its mobile nature implies important challenges to be faced, such as the changing light conditions and the unrestricted locations recorded. This paper proposes an unsupervised strategy based on global features and manifold learning to endow wearable cameras with contextual information regarding the light conditions and the location captured. Results show that non-linear manifold methods can capture contextual patterns from global features without compromising large computational resources. The proposed strategy is used, as an application case, as a switching mechanism to improve the hand-detection problem in egocentric videos.
... Input-driven self-organization is an unsupervised mechanism that learns the input probability distribution through a finite set of prototype vectors. Unlike traditional vector quantization (VQ) methods, self-organizing neural networks such as the SOM [22], the neural gas (NG) [49] as well as their growing extensions such as the growing neural gas (GNG) [50] and the GWR algorithm [17], associate these prototype vectors with neurons that adaptively form topology preserving maps of the input space in an unsupervised fashion, i.e., similar inputs are mapped to neurons that are near to each other on the map. This process of topology preservation is motivated by similar neural mechanisms found in multiple cortical areas of the brainx [15]. ...
The visual recognition of transitive actions comprising human-object interactions is a key component for artificial systems operating in natural environments. This challenging task requires jointly the recognition of articulated body actions as well as the extraction of semantic elements from the scene such as the identity of the manipulated objects. In this paper, we present a self-organizing neural network for the recognition of human-object interactions from RGB-D videos. Our model consists of a hierarchy of Grow-When-Required (GWR) networks that learn prototypical representations of body motion patterns and objects, accounting for the development of action-object mappings in an unsupervised fashion. We report experimental results on a dataset of daily activities collected for the purpose of this study as well as on a publicly available benchmark dataset. In line with neurophysiological studies, our self-organizing architecture exhibits higher neural activation for congruent action-object pairs learned during training sessions with respect to synthetically created incongruent ones. We show that our unsupervised model shows competitive classification results on the benchmark dataset with respect to strictly supervised approaches.
In contrast to batch learning where all training data is available at once, continual learning represents a family of methods that accumulate knowledge and learn continuously with data available in sequential order. Similar to the human learning process with the ability of learning, fusing and accumulating new knowledge acquired at different time steps, continual learning is considered to have high practical significance. Hence, continual learning has been studied in various artificial intelligence tasks. In this paper, we present a comprehensive review of the recent progress of continual learning in computer vision. In particular, the works are grouped by their representative techniques, including regularisation, knowledge distillation, memory, generative replay, parameter isolation and a combination of the above techniques. For each category of these techniques, both its characteristics and applications in computer vision are presented. At the end of this overview, several subareas, where continuous knowledge accumulation is potentially helpful while continual learning has not been well studied, are discussed.
The concept and design of High Temperature Heat Pumps (HTHP) including their components for specific temperature needs is a time consuming and interdisciplinary task. Especially, the design of compressor geometries has a big impact on the overall performance and the initial costs of the system. For this reasoning, in this work an automated aerodynamic gradient-free optimization including structural constraints for the geometry of a radial compressor impeller blade as well as diffusor vane geometry for water steam, that is applied in a reverse Rankine cycle based HTHP, is presented. Objective of the optimization is the isentropic efficiency in the aerodynamic design point (ADP) of the compressor. Constraints for the pressure ratio, mass flow rate and limits for stresses in blade and disk geometry satisfy requirements of the cycle simulation of the whole HTHP system and structural needs. The optimization method is based on evolutionary algorithms and stochastical surrogate models. Additionally, a highly throttled operating point is regarded to achieve an acceptable distance to the surge line. These types of optimization problems are often characterized by lots of unconverged iterations due to unstable computational fluid dynamic simulations (CFD). To encounter this, a study of the optimization process with different surrogate models is presented. The results are discussed with respect to convergence history as well as objective and constraint improvement.
This chapter is concerned with the unsupervised learning algorithms which are approaches to text clustering. Machine learning is divided into supervised learning and unsupervised learning as mentioned in Chap. 6. We use unsupervised learning algorithms as approaches to text clustering. We mention k-means algorithm, its variants, and Kohonen networks as main approaches to the task. In this chapter, we describe unsupervised learning algorithms and neural networks as the approaches to text clustering.
A simple linear neuron model with constrained Hebbian-type synaptic modification is analyzed and a new class of unconstrained learning rules is derived. It is shown that the model neuron tends to extract the principal component from a stationary input vector sequence.
This work contains a theoretical study and computer simulations of a new self-organizing process. The principal discovery is that in a simple network of adaptive physical elements which receives signals from a primary event space, the signal representations are automatically mapped onto a set of output responses in such a way that the responses acquire the same topological order as that of the primary events. In other words, a principle has been discovered which facilitates the automatic formation of topologically correct maps of features of observable events. The basic self-organizing system is a one- or two-dimensional array of processing units resembling a network of threshold-logic units, and characterized by short-range lateral feedback between neighbouring units. Several types of computer simulations are used to demonstrate the ordering process as well as the conditions under which it fails.
