Article

Genetic Algorithm-based Optimization of Deep Neural Network Ensemble

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Personal identification is the task of authenticating a person using individual biological features. Deep neural networks (DNNs) have demonstrated an impressive performance in this field. Since no general algorithm is available for the design of network structures and the parameters adopted in DNNs for every application problem, DNNs should be determined according to the programmers’ experiments and know-how. For a new application task, it is very time-consuming for non-experts to design network structure, hyperparameters and an ensemble of base models adequately and effectively. In this paper, we present a genetic algorithm (GA)-based approach to construct network structures, tune their hyperparameters, and generate base models for the ensemble algorithm. The ensemble is constructed from base models with different network structures according to the voting ensemble algorithm. Our original personal identification dataset is employed as the numerical example to illustrate the performance of the proposed method. The results show that the prediction accuracy of the ensemble model is better than that of the base models and that the prediction of walking behavior toward the Kinect at 90 degrees and 225 degrees is more difficult than other walking behaviors.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... It's crucial to note that, in order to guarantee the predictive performance of the ensemble learning model, the selection of basic learners must consider both model accuracy as well as model diversity. GA [14,31] mimics the natural process of chromosomal recombination evolution and has been demonstrated to be wellsuited for optimization problems related to genomics. To streamline and automate this process, we utilize the "GA" package to optimize the selected basic learners. ...
Article
Full-text available
Recent studies have increasingly revealed the connection between metabolic reprogramming and tumor progression. However, the specific impact of metabolic reprogramming on inter-patient heterogeneity and prognosis in lung adenocarcinoma (LUAD) still requires further exploration. Here, we introduced a cellular hierarchy framework according to a malignant and metabolic gene set, named malignant & metabolism reprogramming (MMR), to reanalyze 178,739 single-cell reference profiles. Furthermore, we proposed a three-stage ensemble learning pipeline, aided by genetic algorithm (GA), for survival prediction across 9 LUAD cohorts (n = 2066). Throughout the pipeline of developing the three stage-MMR (3 S-MMR) score, double training sets were implemented to avoid over-fitting; the gene-pairing method was utilized to remove batch effect; GA was harnessed to pinpoint the optimal basic learner combination. The novel 3 S-MMR score reflects various aspects of LUAD biology, provides new insights into precision medicine for patients, and may serve as a generalizable predictor of prognosis and immunotherapy response. To facilitate the clinical adoption of the 3 S-MMR score, we developed an easy-to-use web tool for risk scoring as well as therapy stratification in LUAD patients. In summary, we have proposed and validated an ensemble learning model pipeline within the framework of metabolic reprogramming, offering potential insights for LUAD treatment and an effective approach for developing prognostic models for other diseases. Supplementary Information The online version contains supplementary material available at 10.1186/s12967-024-05138-2.
... Through the comparison of simulation experiments, the results show that the waiting time and execution error of the IEDF algorithm are smaller than those of the ordinary queuing algorithm within a sufficient time limit, and it has strong feasibility. Feng and other researchers [5] built a network structure based on GA, and continuously optimized the parameters to generate a basic model. The model consists of an integrated algorithm that utilizes the initial feature recognition data of individuals. ...
Article
Full-text available
Mobile operation belongs to the innovation of a business model. At present, the management form of mobile operation is still in manual management, and there are many problems in manual management. In order to solve the current situation, an optimized scheduling model for power mobile jobs based on improved genetic algorithm (GA) is proposed. The model’s objective is to enhance scheduling efficiency and accuracy of power mobile jobs while minimizing automation, scheduling costs, and fault impact. The research conducts simulation experiments to validate the model’s efficacy. In the model considering the total task completion time and the cost of idle hours, the algorithm performance of the model is significantly proposed. When the model has completed around 70 iterations, it converges and maintains a fitness value in the range of 600 to 800. In the task assignment of the model, the total task completion time is shortened by 3 hours, the task assignment of each team is more uniform, and the path planning of each team is more reasonable. The research combines the GA to intelligently schedule human resources, so that the scheduling work can achieve the purpose of automation, so as to achieve the lowest cost to complete the work.
... For example, Ayan et al [36] used a GA to perform a weighted integration of three CNN for crop pest classification tasks. Feng et al [37] applied GAs to construct a CNN structure and optimize the hyperparameter settings of the model. ...
Article
Full-text available
Multi-label classification of aerial images is a crucial task in remote sensing image analysis. Traditional image classification methods have limitations in image feature extraction, leading to an increasing use of deep learning models, such as Convolutional Neural Networks (CNN) and Vision Transformers (ViT). However, the standalone use of these models may have limitations when dealing with multi-label classification. To enhance the generalization performance of multi-label classification of aerial images, this paper combines two CNN and two ViT models, comparing four single deep learning models, a manually weighted ensemble learning method, and a GA-based weighted ensemble method. The experimental results using two public multi-label aerial image datasets show that the classification performance of ViT models is better than CNN models, the traditional weighted ensemble learning model performs better than a single deep learning model, and the GA-based weighted ensemble method performs better than the manually weighted ensemble learning method. The GA-based weighted ensemble method proposed in this study can achieve better multi-label classification performance of aerial images than previous results.
... Personal identity information is employed as the original data for arithmetic learning in the optimised model, which uses the integrated algorithm as its core. The experimental findings demonstrated that the personal recognition mode in biological features was significantly improved (Feng et al., 2021), and that the accuracy of the prediction function of this model was superior to that of the original model before modification. To improve the optimization efficiency of the network structure, Nikbakht's team uses the GA algorithm to optimize all hidden layers and related neurons in the network structure. ...
Article
Developing a safety evaluation model for construction is of utmost importance due to the increasing prevalence of safety issues on construction sites in a rapidly growing sector. Consequently, this research integrates Clonal Genetic Algorithm (CGA) and Bayesian Network (BN) into the current BIM technology for building construction to establish a comprehensive safety evaluation model for building construction. To develop a framework for assessing building safety, the study initially filters the factors impacting building safety through an advanced evolutionary algorithm. Subsequently, a Bayesian network is employed to understand the structure and parameters of the model. When compared to both the backpropagation neural network (BPNN) model and the genetic algorithm (GA) optimized neural network model, the CGA-BPNN model showed a network training error of approximately 0.09%. Additionally, the target error value was observed to be around 0.02%, and the genetic crossover probability of the CGA-BPPN model amounted to 0.6629. These results indicate small algorithm error and appropriate training time of the model, as well as higher accuracy. The CGA-BPNN model filters the evaluation indexes in the Bayesian network and assigns appropriate weights to accurately assess the safety status of the construction project.
... bias is an additional introduced quantity, which is used to describe the bias of the first neuron in the layer i þ 1 of the neural network. It is independent of the weight, but it can improve the ability of the network to fit data [10]. For the data transmission process between different levels in a neural network, a matrix algorithm can be introduced to accurately describe it. ...
Article
Full-text available
To improve cartoon art style rendering systems, increase their adaptability to different front-end rendering requirements, and shorten the waiting time of simultaneous multiuser online renderings, the author proposes an improved algorithm based on deep learning. In this paper, the advantages and disadvantages of the existing neural network rendering model of deep learning machines are comprehensively analyzed, and the design of the rendering algorithm is improved on this basis. The algorithm training link adopts the TensorFlow framework. In the generation link, the algorithm first reads the cartoon picture and carries out the corresponding convolution operation through a convolutional neural network in forward transmission mode. This effect can be achieved by setting the stripe displacement distance to 1 and the convolution kernel size to 3 × 3 and filling 1 layer of 0 at the edge. The analysis results show that the background of the whole website follows the operation law of the asynchronous process pool, which can better improve the rendering response speed. After performing runs and tests, it is found that the website can basically achieve high-efficiency interactions, the application of the new algorithm greatly shortens the development cycle, and the coupling degree between the modules of the rendering webpages is low, which can guarantee the smooth operation and maintenance of the system.
... Feng and other researchers proposed a network structure based on genetic algorithm, which generates a basic model dominated by an integrated algorithm by adjusting parameters and takes the original personal identification data as the numerical value. The results show that the prediction accuracy of the integrated model is better than the basic model, making personal identification occupy an important position in biological feature recognition [7]. Nikbakht's team applied the genetic algorithm to the neural network to find the optimal solution and optimized the hidden layer, integration point, and neuron number of each layer to achieve the highest accuracy, so as to predict the stress distribution of the structure. ...
Article
Full-text available
With the continuous development of urbanization, the urban population is becoming more and more dense, and the demand for land is becoming more and more tense. Urban expansion has become an indispensable part of urban development. This paper studies the optimization of neural network structure by genetic algorithm, puts forward the prediction model of urban scale expansion based on a genetic algorithm optimization neural network, and compares the performance of the model with the basic model. A genetic algorithm BP neural network (GA-BP) optimized by the genetic algorithm is used to shorten the running time of the algorithm and improve the prediction accuracy, but it is easy to fall into local solution. The genetic algorithm is improved by immune cloning algorithm, and the CGA-BP neural network model is established to obtain the global optimal solution. Compared with the BP neural network model and GA-BP neural network model, the CGA-BP neural network model converges faster, and the training times reach the error condition after 79 times, while the BP neural network model and GA-BP neural network model need 117 times and 100 times, respectively, and the fitness value corresponding to the number of iterations of the model is larger. Therefore, the CGA-BP neural network algorithm can make prediction more accurately and quickly and predict the expansion of urban scale through urban conditions.
... en, based on the relationship between the ability level of subjects and the amount of project information, Lord proposed the maximum Fisher information (MFI). e hypothesis of the maximum information method is that the greater the amount of information in the project, the closer the evaluation results of the subjects are to the real results [14]. ...
Article
Full-text available
In web design, we need to make full use of computer multimedia technology, which can effectively improve the quality of web design and provide greater convenience. The neural network needs to calculate a large amount of training data in image optimization, and the calculation speed cannot keep up with real-time data processing, resulting in the problem of poor quality of optimized images. This paper analyzes the problems existing in the traditional optimized BP neural network algorithm and puts forward an optimized BP neural network image optimization method which combines the increase of momentum term with the adaptive adjustment of learning rate. This method can speed up the iteration speed and jump out of the situation of premature local minimum. The test results show that the user satisfaction of the text visual effect of the web interface optimized by this method is more than 98% and the user satisfaction of other methods is only about 90%. The visual satisfaction of the web interface optimized by this method is significantly higher than that of the comparative method. The web interface visual optimization effect of the method in this article is better, and it can meet the satisfaction requirements of most users.
... Up to now, the combination direction of genetic algorithm and neural network can be roughly divided into the following three types [18,19]. ...
Article
Full-text available
Compared with the past questionnaire survey, this paper applies the intelligent algorithm developed rapidly in recent years to identify the tendency of customers to buy financial products in the market. In addition, for the single state customer classification indicators based on the previous demographic information and action information, it is proposed to combine the action of market activities with demographic information; that is, the static integrated customer classification index is further combined with the improved neural network model to study the classification and preference of enterprise financial customers. Firstly, the enterprise financial customer classification model based on neural network algorithm is studied. Aiming at the shortcomings of easy falling into the local optimal solution of neural network algorithm, slow convergence speed of algorithm, and difficult setting of network structure, combined with the characteristics of genetic algorithm, the concept of adaptive genetic neural network algorithm is proposed. Then, the design of adaptive genetic neural network model is studied. Secondly, combined with the customer data of a financial enterprise and the characteristics of enterprise finance, this paper analyzes the risk influencing factors of enterprise financial customers, analyzes the customer data, evaluates the enterprise financial customers through the adaptive genetic neural network model, and realizes the classification of enterprise financial customers. Through an example, it is proved that the enterprise financial customer classification and preference model based on the adaptive genetic neural network algorithm discussed in this paper has better customer classification accuracy and can provide better method support for enterprise financial customer management.
... The conventional manual parameter settings and grid search method requires domain knowledge and is error-prone, consumes time, and is computationally expensive [43]. Meanwhile, genetic algorithm (GA) have been used to optimize some neural networks, including autoencoders [44,45]. However, PSO is a less complex algorithm compared to GA, and it converges faster. ...
Article
Full-text available
Heart disease is the leading cause of death globally. The most common type of heart disease is coronary heart disease, which occurs when there is a build-up of plaque inside the arteries that supply blood to the heart, making blood circulation difficult. The prediction of heart disease is a challenge in clinical machine learning. Early detection of people at risk of the disease is vital in preventing its progression. This paper proposes a deep learning approach to achieve improved prediction of heart disease. An enhanced stacked sparse autoencoder network (SSAE) is developed to achieve efficient feature learning. The network consists of multiple sparse autoencoders and a softmax classifier. Additionally, in deep learning models, the algorithm's parameters need to be optimized appropriately to obtain efficient performance. Hence, we propose a particle swarm optimization (PSO) based technique to tune the parameters of the stacked sparse autoencoder. The optimization by the PSO improves the feature learning and classification performance of the SSAE. Meanwhile, the multilayer architecture of autoencoders usually leads to internal covariate shift, a problem that affects the generalization ability of the network; hence, batch normalization is introduced to prevent this problem. The experimental results show that the proposed method effectively predicts heart disease by obtaining a classification accuracy of 0.973 and 0.961 on the Framingham and Cleveland heart disease datasets, respectively, thereby outperforming other machine learning methods and similar studies.
Article
Full-text available
The considerable amount of energy utilized by buildings has led to various environmental challenges that adversely impact human existence. Predicting buildings' energy usage is commonly acknowledged as encouraging energy efficiency and enabling well-informed decision-making, ultimately leading to decreased energy consumption. Implementing eco-friendly architectural designs is paramount in mitigating energy consumption, particularly in recently constructed structures. This study utilizes clustering analysis on the original dataset to capture complex consumption patterns over various periods. The analysis yields two distinct subsets that represent low and high consumption patterns and an additional subset that exclusively encompasses weekends, attributed to the specific behavior of occupants. Ensemble models have become increasingly popular due to advancements in machine learning techniques. This research utilizes three discrete algorithms, namely Artificial Neural Network (ANN), K-nearest neighbors (KNN), and Decision Trees (DT). In addition, the application employs three more machine learning algorithms bagging and boosting: Random Forest (RF), Extreme Gradient Boosting (XGB), and Gradient Boosting Trees (GBT). To augment the accuracy of predictions, a stacking ensemble methodology is employed, wherein the forecasts generated by many algorithms are combined. Given the obtained outcomes, a thorough examination is undertaken, encompassing the techniques of stacking, bagging, and boosting, to conduct a comprehensive comparative study. It is pertinent to highlight that the stacking technique consistently exhibits superior performance relative to alternative ensemble methodologies across a spectrum of heterogeneous datasets. Furthermore, using a genetic algorithm enables the optimization of the combination of base learners, resulting in a notable enhancement in prediction accuracy. After implementing this optimization technique, GA-Stacking demonstrated remarkable performance in Mean Absolute Percentage Error (MAPE) scores. The improvement observed was substantial, surpassing 90 percent for all datasets. In addition, in subset-1, subset-2, and subset-3, the achieved R2 scores were 0.983, 0.985, and 0.999, respectively. This represents a substantial advancement in forecasting the energy consumption of residential buildings. Such progress underscores the potential advantages of integrating this framework into the practices of building designers, thereby fostering informed decision-making, design management, and optimization prior to construction.
Article
In this letter, attempt has been made to successfully design a pipelined VLSI architecture for the computation of genetic algorithm (GA). The concept of stochastic computing is uniquely exploited in the proposed pipelined GA architecture to attain significant area and power efficiency with reasonably high speed of operation. The prototype 8-bit fixed point GA architecture is realised using VHDL on Xilinx Vivado 2020.3 and implemented on Zynq Ultrascale+ MPSoC (XCZU7EV-2FFVC1156) to train an arbitrary 4:3:2 fully connected neural network in real-time. The performance of the prototype GA architecture in case of real-time training of the neural network outshines the software and other existing GA architectures. The proposed GA-trained 4:3:2 network exhibits 6X reduction in training time and 720X increase in power efficiency, only at the cost of 0.06%0.06\% reduction in accuracy with respect to other existing works and software in case of the image classification of MNIST data-set.
Article
Beam structure is the main load-bearing structure of engineering projects. High-order shear beams are widely used in engineering. Therefore, the damage identification of beam structure is an important guarantee for project quality and life safety. To identify the location and depth of cracks in beam structure, genetic algorithm (GA) and damage identification model are combined. This method optimizes the back-propagation neural network (BPNN) by using the ability of GA to find the global optimal solution. The natural frequency (NF) of the cracked beam is obtained through the finite element analysis, and the NF is taken as the input of the model, and the crack location and depth are taken as the outputs of the model. In the experiment, the predicted output value of the model has a high coincidence with the real value through regression analysis, and its regression coefficient reaches 0.99842. Through an example analysis, the sum of squares of the prediction error of the model is 5.6. The average relative errors of beam crack location and crack depth are 0.54% and 4.15%, respectively. The experimental results show that the proposed model has high prediction accuracy and can accurately identify the damage of beam structure.
Article
Full-text available
Roulette-wheel selection is a frequently used method in genetic and evolutionary algorithms or in modeling of complex networks. Existing routines select one of N individuals using search algorithms of O(N) or O(log(N)) complexity. We present a simple roulette-wheel selection algorithm, which typically has O(1) complexity and is based on stochastic acceptance instead of searching. We also discuss a hybrid version, which might be suitable for highly heterogeneous weight distributions, found, for example, in some models of complex networks. With minor modifications, the algorithm might also be used for sampling with fitness cut-off at a certain value or for sampling without replacement.
Article
Full-text available
Conventionally, we identify ourselves and gain access by physically carrying passports, keys, badges, tokens, and access cards or by remembering passwords, secret codes, and personal identification numbers (PINs). Unfortunately, passports, keys, badges, tokens, and access cards can be lost, duplicated, stolen, or forgotten; and passwords, secret codes, and PINs can easily be forgotten, compromised, shared, or observed. Such loopholes or deficiencies of conventional personal identification techniques have caused major problems to all concerned. For example, hackers often disrupt computer networks; credit card fraud is estimated at 2billionperyearworldwide;andintheUSA,welfarefraud(bydoubledippers)isbelievedtobeinexcessof2 billion per year worldwide; and in the USA, welfare fraud (by double dippers) is believed to be in excess of 4 billion a year. Robust, reliable, and foolproof personal identification solutions must be sought to address the deficiencies of conventional techniques.
Article
Full-text available
Several means for improving the performance and training of neural networks for classification are proposed. Crossvalidation is used as a tool for optimizing network parameters and architecture. It is shown that the remaining residual generalization error can be reduced by invoking ensembles of similar networks
Article
Full-text available
We study the characteristics of learning with ensembles. Solving exactly the simple model of an ensemble of linear students, we find surprisingly rich behaviour. For learning in large ensembles, it is advantageous to use under-regularized students, which actually over-fit the training data. Globally optimal performance can be obtained by choosing the training set sizes of the students appropriately. For smaller ensembles, optimization of the ensemble weights can yield significant improvements in ensemble generalization performance, in particular if the individual students are subject to noise in the training process. Choosing students with a wide range of regularization parameters makes this improvement robust against changes in the unknown level of noise in the training data. 1 INTRODUCTION An ensemble is a collection of a (finite) number of neural networks or other types of predictors that are trained for the same task. A combination of many different predictors can ofte...
Article
This article focuses on a new approach for personal identification by exploring the features of pedestrian behavior. The recent progress of a motion capture sensor system enables personal identification using human behavioral data observed from the sensor. Kinect is a motion sensing input device developed by Microsoft for Xbox 360 and Xbox One. Personal identification using the Microsoft Kinect sensor (hereafter referred to as Kinect) is presented in this study. Kinect is used to estimate body sizes and the walking behaviors of pedestrians. Body sizes such as height and width, and walking behavior such as joint angles and stride lengths, for example, are used as explanatory variables for personal identification. An algorithm for the personal identification of pedestrians is defined by a traditional neural network and by a support vector machine. In the numerical experiments, pictures of body sizes and the walking behaviors are captured from fifteen examinees through Kinect. The walking direction of pedestrians was specified as 0°, 90°, 180°, and 225°, and then the accuracies were compared. The results indicate that identification accuracy was best when the walking direction was 180°. In addition, the accuracy of the vector machine was better than that of the neural network.
Article
Gradient descent optimization algorithms, while increasingly popular, are often used as black-box optimizers, as practical explanations of their strengths and weaknesses are hard to come by. This article aims to provide the reader with intuitions with regard to the behaviour of different algorithms that will allow her to put them to use. In the course of this overview, we look at different variants of gradient descent, summarize challenges, introduce the most common optimization algorithms, review architectures in a parallel and distributed setting, and investigate additional strategies for optimizing gradient descent.
Chapter
As the name suggests, multi-objective optimisation involves optimising a number of objectives simultaneously. The problem becomes challenging when the objectives are of conflicting characteristics to each other, that is, the optimal solution of an objective function is different from that of the other. In the course of solving such problems, with or without the presence of constraints, these problems give rise to a set of trade-off optimal solutions, popularly known as Pareto-optimal solutions. Because of the multiplicity in solutions, these problems were proposed to be solved suitably using evolutionary algorithms using a population approach in its search procedure. Starting with parameterised procedures in early 90s, the so-called evolutionary multi-objective optimisation (EMO) algorithms is now an established field of research and application with many dedicated texts and edited books, commercial softwares and numerous freely downloadable codes, a biannual conference series running successfully since 2001, special sessions and workshops held at all major evolutionary computing conferences, and full-time researchers from universities and industries from all around the globe. In this chapter, we provide a brief introduction to its operating principles and outline the current research and application studies of evolutionary multi-objective optmisation (EMO).
Chapter
In this chapter we consider bounds on the rate of uniform convergence. We consider upper bounds (there exist lower bounds as well (Vapnik and Chervonenkis, 1974); however, they are not as important for controlling the learning processes as the upper bounds).
Chapter
In the history of research of the learning problem one can extract four periods that can be characterized by four bright events: (i) Constructing the first learning machines, (ii) constructing the fundamentals of the theory, (iii) constructing neural networks, (iv) constructing the alternatives to neural networks.
Article
Training Deep Neural Networks is complicated by the fact that the distribution of each layer's inputs changes during training, as the parameters of the previous layers change. This slows down the training by requiring lower learning rates and careful parameter initialization, and makes it notoriously hard to train models with saturating nonlinearities. We refer to this phenomenon as internal covariate shift, and address the problem by normalizing layer inputs. Our method draws its strength from making normalization a part of the model architecture and performing the normalization for each training mini-batch}. Batch Normalization allows us to use much higher learning rates and be less careful about initialization. It also acts as a regularizer, in some cases eliminating the need for Dropout. Applied to a state-of-the-art image classification model, Batch Normalization achieves the same accuracy with 14 times fewer training steps, and beats the original model by a significant margin. Using an ensemble of batch-normalized networks, we improve upon the best published result on ImageNet classification: reaching 4.9% top-5 validation error (and 4.8% test error), exceeding the accuracy of human raters.
Article
We introduce Adam, an algorithm for first-order gradient-based optimization of stochastic objective functions. The method is straightforward to implement and is based an adaptive estimates of lower-order moments of the gradients. The method is computationally efficient, has little memory requirements and is well suited for problems that are large in terms of data and/or parameters. The method is also ap- propriate for non-stationary objectives and problems with very noisy and/or sparse gradients. The method exhibits invariance to diagonal rescaling of the gradients by adapting to the geometry of the objective function. The hyper-parameters have intuitive interpretations and typically require little tuning. Some connections to related algorithms, on which Adam was inspired, are discussed. We also analyze the theoretical convergence properties of the algorithm and provide a regret bound on the convergence rate that is comparable to the best known results under the online convex optimization framework. We demonstrate that Adam works well in practice when experimentally compared to other stochastic optimization methods.
Article
Any procedure of social choice makes use of some types of information and ignores others. For example, the method of majority decision concentrates on people's votes, but pays no direct attention to, say, their social standings, or their prosperity or penury, or even the intensities of their preferences. The differences between distinct procedures lie, to a substantial extent, on the kind of information that each procedure uses and what it has to ignore. The informational bases of the different social choice procedures tell us a great deal about how they respectively work and what they can or cannot achieve.
Book
Setting of the learning problem consistency of learning processes bounds on the rate of convergence of learning processes controlling the generalization ability of learning processes constructing learning algorithms what is important in learning theory?.
Article
We present a novel per-dimension learning rate method for gradient descent called ADADELTA. The method dynamically adapts over time using only first order information and has minimal computational overhead beyond vanilla stochastic gradient descent. The method requires no manual tuning of a learning rate and appears robust to noisy gradient information, different model architecture choices, various data modalities and selection of hyperparameters. We show promising results compared to other methods on the MNIST digit classification task using a single machine and on a large scale voice dataset in a distributed cluster environment.
Article
This paper reports the first phase of a research program on visual perception of motion patterns characteristic of living organisms in locomotion. Such motion patterns in animals and men are termed here as biological motion. They are characterized by a far higher degree of complexity than the patterns of simple mechanical motions usually studied in our laboratories. In everyday perceptions, the visual information from biological motion and from the corresponding figurative contour patterns (the shape of the body) are intermingled. A method for studying information from the motion pattern per se without interference with the form aspect was devised. In short, the motion of the living body was represented by a few bright spots describing the motions of the main joints. It is found that 10–12 such elements in adequate motion combinations in proximal stimulus evoke a compelling impression of human walking, running, dancing, etc. The kinetic-geometric model for visual vector analysis originally developed in the study of perception of motion combinations of the mechanical type was applied to these biological motion patterns. The validity of this model in the present context was experimentally tested and the results turned out to be highly positive.
Chapter
This article deals with the use of genetic algorithms to optimize the architecture of a neural network. After a brief recall of our original neural network (named Yprel network), we show that a simulated-annealing-like technique has been advantageously replaced by genetic operators. Indeed, tests on character recognition (NIST handwritten database) have shown that the generalization rate has been improved, the mean network size has been reduced by a factor 3 and the learning speed has been significantly increased. Moreover, a portable adaptive mutation probability has been introduced which enables a parameter-free learning.
Conference Paper
This paper propose a biometric personal identification method based on a pair of right and left sole pressure distribution change. We acquire the sole pressure distribution change by load distribution sensor and use it for a personal identification. We employ twelve features based on shape of footprint, and twenty seven features based on movement of weight during walking for each sole pressure data. We make these fuzzy if-then rules. We calculate a fuzzy degree of a pair of right and left sole pressure data for one person, and identify person by this fuzzy degree. We evaluated our method by five-hold cross validation method. The low false rejection and acceptance rates are evaluated from 20 to 90 persons.
Conference Paper
This paper introduces a new algorithm which uses a genetic algorithm (GA) to determine the topology and link weights of a neural network. If the genetic algorithm fails to find a satisfactory solution network, the best network developed by the GA is used to try to find a solution via back-propagation. In this way, each algorithm is used to its greatest advantage: the GA (with its global search) determines a (sub-optimal) topology and weights to solve the problem, and back-propagation (with its local search) seeks the best solution in the area of the weight and topology spaces found by the GA. The intent is to develop an algorithm which can be used as a first attempt to solve unknown problems. If a solution is not found immediately by the algorithm, at least an appreciable amount of information about the solution can be gleaned from its results, and a lot of the initial guesswork that currently exists in finding a neural network solution to a problem can be eliminated. Some of the features of the GANNet algorithm are:1. The networks produced are feed-forward layered networks, which means that there exist many techniques for further refinement and analysis of the resulting networks. 2. The network nodes can have fan-in and fan-out limits, permitting relatively simple hardware implementation using modular units. 3. Both Gaussian and sigmoid node transfer functions are available. 4. No initial knowledge about a solution network is needed, other than the obvious (the number of inputs and outputs).
Conference Paper
We present a new family of subgradient methods that dynamically incorporate knowledge of the geometry of the data observed in earlier iterations to perform more informative gradient-based learning. Metaphorically, the adaptation allows us to find needles in haystacks in the form of very predictive but rarely seen features. Our paradigm stems from recent advances in stochastic optimization and online learning which employ proximal functions to control the gradient steps of the algorithm. We describe and analyze an apparatus for adaptively modifying the proximal function, which significantly simplifies setting a learning rate and results in regret guarantees that are provably as good as the best proximal function that can be chosen in hindsight. We give several efficient algorithms for empirical risk minimization problems with common and important regularization functions and domain constraints. We experimentally study our theoretical analysis and show that adaptive subgradient methods outperform state-of-the-art, yet non-adaptive, subgradient algorithms.
Conference Paper
Multilayered feedforward neural networks possess a number of properties which make them particu­ larly suited to complex pattern classification prob­ lems. However, their application to some real- world problems has been hampered by the lack of a training algonthm which reliably finds a nearly globally optimal set of weights in a relatively short time. Genetic algorithms are a class of optimiza­ tion procedures which are good at exploring a large and complex space in an intelligent way to find values close to the global optimum. Hence, they are well suited to the problem of training feedfor­ ward networks. In this paper, we describe a set of experiments performed on data from a sonar image classification problem. These experiments both 1) illustrate the improvements gained by using a ge­ netic algorithm rather than backpropagation and 2) chronicle the evolution of the performance of the genetic algorithm as we added more and more domain-specific knowledge into it.
Book
From the Publisher: An Introduction to Nueral Networks will be warmly welcomed by a wide readership seeking an authoritative treatment of this key subject without an intimidating level of mathematics in the presentation.
Conference Paper
ANN design is usually thought as a training problem to be solved for some predefined ANN structure and connectivity. Training methods arc very problem and ANN dependent. They are sometimes very accurate procedures but they work in narrow and restrictive domains. Thus the designer is faced to a wide diversity of multimodal and different training mechanisms. We have selected Genetic Algorithms as training procedures because of their robustness and their potential application to any ANN type training. Furthermore we have addressed the connectivity and structure definition problems in order to accomplish a full genetic ANN design. These three levels of design can work in parallel, thus achieving multilevel relationships to yield better ANNs. GRIAL is the tool used to test several new and known genetic techniques and operators. PARLOG is the Concurrent Logic Language used for the implementation in order to introduce new models for the genetic work and attain an intralevel distributed search as well as to parallelize any ANN evaluation.
Article
Thesis (Ph. D.)--University of Michigan, 1975. Includes bibliographical references (leaves 253-256). Photocopy.
Conference Paper
This paper proposes an automatic gait recognition approach for analyzing and classifying human gait by computer vision techniques. The approach attempts to incorporate knowledge of the static and dynamics of human gait into the feature extraction process. The width vectors of the binarized silhouette of a walking person contain the physical structure of the person, the motion of the limbs and other details of the body are chosen as the basic gait feature. Different from the model-based approaches, the limb angle information is extracted by analyzing the variation of silhouette width without needing the human body model. Discrete cosine analysis is used to analyze the shape and dynamic characteristic and reduce the gait features. And this paper uses the multi-class support vector machines to distinguish the different gaits of human. The performance of the proposed method is tested using different gait databases. Recognition results show this approach is efficient.
Conference Paper
Our goal is to establish a simple baseline method for human identification based on body shape and gait. This baseline recognition method provides a lower bound against which to evaluate more complicated procedures. We present a viewpoint-dependent technique based on template matching of body silhouettes. Cyclic gait analysis is performed to extract key frames from a test sequence. These frames are compared to training frames using normalized correlation, and subject classification is performed by nearest-neighbor matching among correlation scores. The approach implicitly captures biometric shape cues such as body height, width, and body-part proportions, as well as gait cues such as stride length and amount of arm swing. We evaluate the method on four databases with varying viewing angles, background conditions (indoors and outdoors), walking styles and pixels on target.
Conference Paper
Motion-based recognition deals with the recognition of objects or motions directly from the motion information extracted from a sequence of images. There are two main steps in this approach. The first consists of finding an appropriate representation for the objects or motions, from the motion cues of the sequence, and then organize them into useful representations. The second step consists of the matching of some unknown input with a model. This paper provides a review of recent developments in motion-based recognition
Conference Paper
Shows how to find both the weights and architecture for a neural network, including the number of layers, the number of processing elements per layer, and the connectivity between processing elements. This is accomplished by using a recently developed extension to the genetic algorithm which genetically breeds a population of LISP symbolic expressions of varying size and shape until the desired performance by the network is successfully evolved. The novel `genetic programming' paradigm is applied to the problem of generating a neural network for a one-bit adder
Article
Research on potential interactions between connectionist learning systems, i.e., artificial neural networks (ANNs), and evolutionary search procedures, like genetic algorithms (GAs), has attracted a lot of attention recently. Evolutionary ANNs (EANNs) can be considered as the combination of ANNs and evolutionary search procedures. This paper first distinguishes among three kinds of evolution in EANNs, i.e., the evolution of connection weights, of architectures and of learning rules. Then it reviews each kind of evolution in detail and analyses critical issues related to different evolutions. The review shows that although a lot of work has been done on the evolution of connection weights and of architectures, few attempts have been made to understand the evolution of learning rules. Interactions among different evolutions are seldom mentioned in current research. However, the evolution of learning rules and its interactions with other kinds of evolution play a vital role in EANNs. As t...
Article
Motion-based recognition deals with the recognition of objects or motions directly from the motion information extracted from a sequence of images. There are two main steps in this approach. The first consists of finding an appropriate representation for the objects or motions, from the motion cues of the sequence, and then organize them into useful representations. The second step consists of the matching of some unknown input with a model. This paper provides a review of recent developments in motion-based recognition. 1 Introduction Motion perception plays an important role in the human visual system. It helps us recognize different objects and their motion in a scene, infer their relative depth, rigidity, etc. We tend to focus our attention on moving objects, while motionless objects are not as easily detectable. Our sensitivity and ease of perception and interpretation of motion suggests that our visual system is well adapted to temporal information. Motion perception has been s...
A biochemical invariant for gait perception and performance
  • J E Cutting
  • D R Proffitt
  • L Kozlowski