Article

Support-vector networks

Authors:
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

Thesupport-vector network is a new learning machine for two-group classification problems. The machine conceptually implements the following idea: input vectors are non-linearly mapped to a very high-dimension feature space. In this feature space a linear decision surface is constructed. Special properties of the decision surface ensures high generalization ability of the learning machine. The idea behind the support-vector network was previously implemented for the restricted case where the training data can be separated without errors. We here extend this result to non-separable training data.High generalization ability of support-vector networks utilizing polynomial input transformations is demonstrated. We also compare the performance of the support-vector network to various classical learning algorithms that all took part in a benchmark study of Optical Character Recognition.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... Note that the proposed algorithm changes the sequential updating manner in [28] to a parallel way. As observed from (6), no terms related to x k+1 j are needed to update x k+1 i . Hence The state x i updates in a sequential manner. ...
... where the relation (14) is obtained according to the updating rule (6). ...
... where in (22) the relation of the saddle point of Lagrangian function relation F (x * ) − T Ax * ≤ F (x * ) − ( * ) T Ax * has been used. From (6) and ...
Article
Full-text available
Alternating direction method of multipliers (ADMM) has been widely used for solving the distributed optimisation problems. This paper proposes a novel distributed ADMM algorithm to solve the distributed optimisation problems consisting of convex cost functions under an undirected connected graph. The proposed algorithm adopts the concepts of predecessors and successors in the distributed sequential ADMM algorithm, but changes the sequential updating manner to a parallel one, which allows the agents to update their local states and dual variables in a completely distributed and parallel manner. This brings some benefits when solving large‐scale optimisation problems. Variational inequality is applied to analyse the convergence of agents' states. It is proved that the states of all the agents converge to the optimal point, and the global cost function converge to the optimal value at a rate of O(1/k)$O(1/k)$. Numerical experiments are given to show the effectiveness and suitability of the proposed algorithm.
... Resting-State fMRI sequence needs an additional EPI BOLD sequence that requires Brain wave software, available only with 8 Channel coil. It was resting-state fMRI which was 10 min long and the scan parameters were settled as (Size 96 × 96 in-plane, 3 mm thickness, repetition time (TR) 3000 ms, slices 39) [16,17]. All Subjects closed their eyes and rested while the conducting fMRI experiments. ...
... We carried out the experiments in Python with the libraries Tensorflow v.1.4 [16] and Keras v.2.1 (https://keras.io/). We used the same validation division for every set of experiments. ...
... In this study, we applied k-fold cross-validation to classify the difference in brain structure between normal patients and schizophrenia patients by using eleven different classification methods (Linear and Nonlinear SVM, QDA, Gaussian Process, Nearest Neighbors, Neural Net, NaiveBayese, XgBoost, Adaboost, Decision Tree, and Random Forest). We used an independent sample t-test to determine functional activation, and correlation coefficients to verify functional connectivity in regions of interest (ROIs) and feature ranking method LASSO [16]. ...
Article
Full-text available
Recently, advances in neuroscience have attracted attention to the diagnosis, treatment, and damage to schizophrenia-associated brain regions using resting-state functional magnetic resonance imaging (rs-fMRI). This research is immersed in the endowment of machine learning approaches for discriminating schizophrenia patients to provide a viable solution. Toward these goals, firstly, we implemented a two sample t -tests to find the activation difference between schizophrenia patients and healthy controls. The average activation in control is higher than the average activation of the patient. Secondly, we implemented the correlation technique to find variations on presumably hidden associations between brain structure and its associated function. Moreover, current results support the viewpoint that the resting-state function integration is helpful to gain insight into the pathological mechanism of schizophrenia. Finally, Lasso regression is used to find a low-dimensional integration of the rs-fMRI and their experimental results showed that SVM classifier surpasses nine algorithms provided the best results with good accuracy of 94%.
... Support vector machines (SVMs) are inductive learning methods originally developed by Cortes and Vapnik for supervised classification [39], but they have been applied in regression tasks as well [40]. The classification using SVMs is formalised as searching for an optimal hyperplane (or set of hyperplanes) having a maximal functional margin hyperplane that separates the training data points. ...
... For obtaining non-linear regression surfaces, kernel functions [39] (such as Radial Basis Function-RBF, polynomial, sigmoid, etc.) are used for mapping the input data space into a higher dimension [39]. In the linear case, methods such as stochastic gradient descent (SGD) [35] are used to solve the SVR problem faster. ...
... For obtaining non-linear regression surfaces, kernel functions [39] (such as Radial Basis Function-RBF, polynomial, sigmoid, etc.) are used for mapping the input data space into a higher dimension [39]. In the linear case, methods such as stochastic gradient descent (SGD) [35] are used to solve the SVR problem faster. ...
Article
Full-text available
Flash floods are a major weather-related risk, as they cause more than 5000 fatalities annually, according to the World Meteorological Organization. Quantitative Precipitation Estimation is a method used to approximate the rainfall over locations where direct field observations are not available. It represents one of the most valuable information employed by meteorologists and hydrologists for issuing early warnings concerning flash floods. The current study is in line with the efforts to improve radar-based rainfall estimates through the use of machine learning techniques applied on radar data. With this aim, as a proof of concept, six machine learning models are evaluated to make estimations of the radar-based hourly accumulated rainfall using reflectivity data collected on the lowest radar elevation angles, and we employ a new data model for representing these radar data. The data were collected by a WSR-98D weather radar of the Romanian Meteorological Administration, located in the central region of Romania, during 30 non-consecutive days of the convective seasons, between 2016 and 2021. We obtained encouraging results using a stacked machine learning model. In terms of the Root Mean Squared Error evaluation metric, the results of the proposed stacked regressor are better than the radar estimated accumulated rainfall by about 33% and also outperform the baseline computed using the Z-R relationship by about 13%.
... In practice, heuristic methods such as the OVO and OVR approaches are mostly used than other multiclass SVM implementations. There are several online software packages available that efficiently solve the binary SVM, such as (Cortes & Vapnik, 1995). ...
... The SVM is a popular supervised learning method for classification and regression (Cortes & Vapnik, 1995;Vapnik, 2013). corresponding approximation function can be found in (Cortes & Vapnik, 1995). ...
... The SVM is a popular supervised learning method for classification and regression (Cortes & Vapnik, 1995;Vapnik, 2013). corresponding approximation function can be found in (Cortes & Vapnik, 1995). The mathematical formulas of the C-SVC optimization algorithm and its corresponding approximation function shows below (C.-C. ...
Thesis
Full-text available
The use of speech recognition technology has increased considerably in the last three decades. In the real world, the performance of well-trained speech recognizers is usually degraded by different types of noise and distortions such as background noise, reverberation and telephone channels. In particular, speech signal is extremely difficult to recognize due to the interference created by reverberation and bandwidth of transmission channels. The accuracy of traditional speech recognition systems in noisy environments is much lower than the recognition accuracy of an average human being. Robustness of speech recognition systems must be addressed for practical applications. Although many successful techniques have been developed for dealing with clean signal and noise, particularly uncorrelated noise with simple spectral characteristics (e.g., white noise), the problem of sound reverberation and channel distortions has remained essentially unsolved. This problem hampers the wider use of acoustic interfaces for many applications. Unlike traditional methods in which features are extracted from the properties of the acoustic signal, this study proposes a phoneme classification technique using neural responses from a physiologically-based computational model of the auditory periphery. The 2-D neurograms were constructed from the simulated responses of the auditorynerve fibers to speech phonemes. The features of the neurograms were extracted using the Radon transform and used to train the classification system using a support vector machine classifier. Classification performances were evaluated for phonemes extracted from the TIMIT and HTIMIT databases. Experiments were performed in mismatched train/test conditions where the test data in these experiments consist of speech corrupted by variety of real world additive noises at different signal-to-noise ratios (SNRs), convolutive distortions introduced by different room impulse response functions, and v multiple telephone channel speech recordings with different frequency characteristics. Performances of the proposed method were compared to those of Mel-Frequency Cepstral Coefficients (MFCC), Gamma-tone Frequency Cepstral Coefficients (GFCC), and Frequency Domain Linear Prediction (FDLP)-based phoneme classifiers. Based on simulation results, the proposed method outperformed most of the traditional acousticproperty- based phoneme classification methods for both in quiet and under noisy conditions. This approach is accurate, easy to implement, and can be used without any knowledge about the type of distortion in the signal, i.e., it can handle any type of noise. Using (support vector machine/ hidden Markov model) hybrid classifiers, the proposed method could be extended to develop an automatic speech recognition system.
... Considering each layer of deep networks can be modeled as a random projection process [2], our finding is helpful to understand why sparse ternary or binary networks perform comparably and even better than their full-precision counterparts [8]. To verify the quantization gain both on sparse features and their sparse random projections, we conduct extensive classification experiments on four benchmark image databases, including YaleB [9], Fashion-MNIST [10], Cifar10 [11] and ImageNet [12], with several popular classifiers known as SVM [13], kNN [14], sparse representation classifier (SRC) [15], local subspace classifier (LSC) [16] and a simplified variant of LSC termed kNNC. ...
... We test several popular classifiers, including SVM [13], kNN [14], LSC [16] and SRC [15]. Except kNN, the others all hold polynomial complexity, unsuitable for large-scale classification. ...
Preprint
p> Large-scale classification poses great challenges to storage and computation. There are two major solutions to address the problem: data compression and quantization. In the paper, we study the method of first reducing data dimension by random projection and then quantizing the projections to ternary and binary codes, which has been widely applied in practice. Often, the extreme quantization would degrade the accuracy of classification due to high quantization errors. Interestingly, however, we observe that the quantization could result in performance improvement, rather than degradation, if the data for quantization are preprocessed by sparse transform. Also, the quantization gain could be obtained with the random projections of the data, if both the data and random projection matrices are sparse enough, such that the resulting projections remain sparse. The intriguing performance is verified and analyzed with extensive experiments. </p
... To assess the classification potential of the αSA, ML models were used to differentiate each molecule within its own class (Supplementary Note and Supplementary Figs. 10 and 11). Briefly, six algorithms were tested-Gaussian naïve Bayes, k-nearest neighbors 40 , linear discriminant analysis (LDA), an AdaBoost 41 classifier, and two supportvector classifiers with a linear kernel (linear SVC) or with a radial basis function kernel (SVC) 42 -with the aim of selecting the simplest model with the best performance. Training used nested stratified crossvalidation and the average accuracy across all folds was calculated. ...
... Data outputs are generated again for visual inspection once outliers have been removed. Six ML algorithms-Gaussian Naïve Bayes, K-nearest neighbors 40,65 , linear discriminant analysis, support vector classification (linear and radial basis function kernel) 42 and an AdaBoost classifier 41 -were trained using nested stratified k-folds cross-validation and compared to two dummy classifiers (which mimic random guessing). Feature importance analysis (KBest analysis, an ExtraTrees classifier and permutation analysis) was performed for all datasets. ...
Article
Full-text available
Differential sensing attempts to mimic the mammalian senses of smell and taste to identify analytes and complex mixtures. In place of hundreds of complex, membrane-bound G-protein coupled receptors, differential sensors employ arrays of small molecules. Here we show that arrays of computationally designed de novo peptides provide alternative synthetic receptors for differential sensing. We use self-assembling α-helical barrels (αHBs) with central channels that can be altered predictably to vary their sizes, shapes and chemistries. The channels accommodate environment-sensitive dyes that fluoresce upon binding. Challenging arrays of dye-loaded barrels with analytes causes differential fluorophore displacement. The resulting fluorimetric fingerprints are used to train machine-learning models that relate the patterns to the analytes. We show that this system discriminates between a range of biomolecules, drink, and diagnostically relevant biological samples. As αHBs are robust and chemically diverse, the system has potential to sense many analytes in various settings. Differential sensing aims to mimic senses such as taste and smell through the use of synthetic receptors. Here, the authors show that arrays of de novo designed peptide assemblies can be used as sensor components to distinguish various analytes and complex mixtures.
... S UPPORT vector machine (SVM) [1] achieves relatively good performance in many applications, which aims at constructing a hyperplane that can classify samples with the structure risk minimization principle. In recent decades, several variants for SVM have been proposed, such as least squares SVM (LSSVM) [2], pinball SVM (Pin-SVM) [3], robust support vector classifiers (RSVC) [4] and so on. ...
... In this section, we make experiments on UCI datasets to validate the effectiveness of our proposed method and com-pare it with other algorithms including SVM [1], TSVM [8], NPHSVM [25], least squares recursive projection twin support vector machine (LSPTSVM) [26] and DSVM [23]. Our experiments are implemented on a Windows 10 computer with 3.6GHz Intel Core i7-9700K with 32GB RAM and RTX 2080Ti. ...
Article
Full-text available
In the last few decades, deep learning based on neural networks has become popular for the classification tasks, which combines feature extraction with the classification tasks and always achieves the satisfactory performance. Non-parallel hyperplane support vector machine (NPHSVM) aims at constructing two non-parallel hyperplanes to classify data and extracted features are always used to be input data for NPHSVM. As for NPHSVM, extracted features will greatly influence the performance of the model to some extent. Therefore, in this paper, we propose a novel DNHSVM for classification, which combines deep feature extraction with the generation of hyperplanes seamlessly. Each hyperplane is close to its own class and as far as possible to other classes, and deep features are friendly for classification and samples are easy to be classified. Experiments on UCI datasets show the effectiveness of our proposed method, which outperforms other compared state-of-the-art algorithms.
... Notice that the trivial prototypes are located as far as possible from the classification boundary in the feature space. Similar to ProtoPNet, the support vector machine (SVM) [6] classifier is trained by minimising a loss function that learns a set of support vectors. Different from the ProtoPNet's prototypes, these support vectors are located as close as possible to the classification boundary, as shown in Fig. 2(c). ...
... To better understand the optimality of prototypes, we consider the support vector machine (SVM) [6] classifier that finds support vectors to represent classes. More specifically, SVM learns the maximum-margin classifier defined by a classification boundary that maximises the distance to the closest training samples, which are the support vectors for the classes. ...
Preprint
Full-text available
Prototypical part network (ProtoPNet) methods have been designed to achieve interpretable classification by associating predictions with a set of training prototypes, which we refer to as trivial (i.e., easy-to-learn) prototypes because they are trained to lie far from the classification boundary in the feature space. Note that it is possible to make an analogy between ProtoPNet and support vector machine (SVM) given that the classification from both methods relies on computing similarity with a set of training points (i.e., trivial prototypes in ProtoPNet, and support vectors in SVM). However, while trivial prototypes are located far from the classification boundary, support vectors are located close to this boundary, and we argue that this discrepancy with the well-established SVM theory can result in ProtoPNet models with suboptimal classification accuracy. In this paper, we aim to improve the classification accuracy of ProtoPNet with a new method to learn support prototypes that lie near the classification boundary in the feature space, as suggested by the SVM theory. In addition, we target the improvement of classification interpretability with a new model, named ST-ProtoPNet, which exploits our support prototypes and the trivial prototypes to provide complementary interpretability information. Experimental results on CUB-200-2011, Stanford Cars, and Stanford Dogs datasets demonstrate that the proposed method achieves state-of-the-art classification accuracy and produces more visually meaningful and diverse prototypes.
... One feature for each power channel 2 and 4 were computed by averaging the rectified signal output over the course of a trial. Together, these features were used to produce an M × N (M = number of trials, N = 16 features were) that was used as input to train a range of distinct classifiers including Random Forests (Breiman, 2001), K-Nearest Neighbors (Cover and Hart, 1967), linear discriminant analysis, a linear support vector machine (Cortes and Vapnik, 1995), and an artificial neural network (Tshitoyan, 2021). ...
... Iterative blocks of data were used to train and test an array of motor-imagery classifiers including Random Forests (Breiman, 2001), K-Nearest Neighbors (Cover and Hart, 1967), linear discriminant analysis, a linear support vector machine (Cortes and Vapnik, 1995), and an artificial neural network (Tshitoyan, 2021). These models were trained on data from the extracted features using 5-fold cross validation. ...
Article
Full-text available
Introduction: Most spinal cord injuries (SCI) result in lower extremities paralysis, thus diminishing ambulation. Using brain-computer interfaces (BCI), patients may regain leg control using neural signals that actuate assistive devices. Here, we present a case of a subject with cervical SCI with an implanted electrocorticography (ECoG) device and determined whether the system is capable of motor-imagery-initiated walking in an assistive ambulator. Methods: A 24-year-old male subject with cervical SCI (C5 ASIA A) was implanted before the study with an ECoG sensing device over the sensorimotor hand region of the brain. The subject used motor-imagery (MI) to train decoders to classify sensorimotor rhythms. Fifteen sessions of closed-loop trials followed in which the subject ambulated for one hour on a robotic-assisted weight-supported treadmill one to three times per week. We evaluated the stability of the best-performing decoder over time to initiate walking on the treadmill by decoding upper-limb (UL) MI. Results: An online bagged trees classifier performed best with an accuracy of 84.15% averaged across 9 weeks. Decoder accuracy remained stable following throughout closed-loop data collection. Discussion: These results demonstrate that decoding UL MI is a feasible control signal for use in lower-limb motor control. Invasive BCI systems designed for upper-extremity motor control can be extended for controlling systems beyond upper extremity control alone. Importantly, the decoders used were able to use the invasive signal over several weeks to accurately classify MI from the invasive signal. More work is needed to determine the long-term consequence between UL MI and the resulting lower-limb control.
... There were no subjective factors involved in the manual labeling process because each accident report recorded the direct and indirect reasons of the fatal accident. The data labeling process is shown in Figure 2. Figure 3 describes the ML modeling processes of DT, SVM [62], k-nearest neighbor (KNN) [63], RF [64], AdaBoost [65], and gradient boosting decision tree (GBDT) [66]. DT, SVM, and KNN are examples of basic and classical ML models, while RF, AdaBoost, and GBDT are examples of ensemble learning methods. ...
Article
Full-text available
The construction industry is fraught with danger. The investigation of the causes of occupational accidents receives considerable attention. The purpose of this research is to determine the hierarchical relationship and critical combination of the fatal causes of accidents on construction sites. The framework for fatal cause attribute was established. Machine learning technologies were developed to predict the different types of accidents. Using feature importance, the hierarchical relationship of fatal causes was extracted. An iterative analysis algorithm was created to quantify the cause combinations. The F1 prediction score was 92.93%. The results revealed that combinations existed in fatal causes analysis, even if they were hierarchical. Furthermore, this study made recommendations for improving safety management and preventing occupational accidents. The findings of this study guide construction participants in providing early warning signs of fatal and unsafe factors, ultimately assisting in the prevention of fatalities.
... Video Multi-method Assessment Fusion (VMAF) [25,26] is another main stream evaluation metric in the real-world industry where lots of famous commercial companies like Netflix [26], Meta [34], Tiktok [48], Intel [22] etc., and standardization such as AO-Media [12] adopt it for video codec evaluation. VMAF combines three quality features: Visual Information Fidelity (VIF) [36], Detail Loss Metric (DLM) [23], and Motion, to train a Support Vector Machine (SVM) regressor [15] to predict subjective score of video quality. Lot of studies have demonstrated that VMAF is remarkably more correlated to the Mean Opinion Score (MOS) than SSIM and PSNR [5,33,47]. ...
Preprint
Full-text available
In the past decades, lots of progress have been done in the video compression field including traditional video codec and learning-based video codec. However, few studies focus on using preprocessing techniques to improve the rate-distortion performance. In this paper, we propose a rate-perception optimized preprocessing (RPP) method. We first introduce an adaptive Discrete Cosine Transform loss function which can save the bitrate and keep essential high frequency components as well. Furthermore, we also combine several state-of-the-art techniques from low-level vision fields into our approach, such as the high-order degradation model, efficient lightweight network design, and Image Quality Assessment model. By jointly using these powerful techniques, our RPP approach can achieve on average, 16.27% bitrate saving with different video encoders like AVC, HEVC, and VVC under multiple quality metrics. In the deployment stage, our RPP method is very simple and efficient which is not required any changes in the setting of video encoding, streaming, and decoding. Each input frame only needs to make a single pass through RPP before sending into video encoders. In addition, in our subjective visual quality test, 87% of users think videos with RPP are better or equal to videos by only using the codec to compress, while these videos with RPP save about 12% bitrate on average. Our RPP framework has been integrated into the production environment of our video transcoding services which serve millions of users every day.
... In our approach, we used a Support Vector Machine (SVM) [5] model to classify the skill level of athletes. SVM is a supervised machine learning method that can solve binary classification problems. ...
Preprint
Full-text available
Bowling is a target sport that is popular among all age groups with professionals and amateur players. Delivering an accurate and consistent bowling throw into the lane requires the incorporation of motion techniques. Consequently, this research presents a novel IoT-Cloud based system for providing real-time monitoring and coaching services to bowling athletes. The system includes two inertial measurement units (IMUs) sensors for capturing motion data, a mobile application and a cloud server for processing the data. First, the quality of each phase of a throw is assessed using a Dynamic Time Wrapping (DTW) based algorithm. Second, an on device-level technique is proposed to identify common bowling errors. Finally, an SVM classification model is employed for assessing the skill level of bowler athletes. We recruited nine right-handed bowlers to perform 50 throws wearing the two sensors and using the proposed system. The results of our experiments suggest that the proposed system can effectively and efficiently assess the quality of the throw, detect common bowling errors and classify the skill level of the bowler.
... This manuscript adopts a support vector machine (SVM) with a Gaussian kernel function, which is implemented in the LibSVM library (Cortes and Vapnik, 1995;Chang and Lin, 2000). The way to achieve multi-class classification is to use a one-to-one strategy. ...
Article
Full-text available
Background As a medium for developing brain-computer interface systems, EEG signals are complex and difficult to identify due to their complexity, weakness, and differences between subjects. At present, most of the current research on sleep EEG signals are single-channel and dual-channel, ignoring the research on the relationship between different brain regions. Brain functional connectivity is considered to be closely related to brain activity and can be used to study the interaction relationship between brain areas. Methods Phase-locked value (PLV) is used to construct a functional connection network. The connection network is used to analyze the connection mechanism and brain interaction in different sleep stages. Firstly, the entire EEG signal is divided into multiple sub-periods. Secondly, Phase-locked value is used for feature extraction on the sub-periods. Thirdly, the PLV of multiple sub-periods is used for feature fusion. Fourthly, the classification performance optimization strategy is used to discuss the impact of different frequency bands on sleep stage classification performance and to find the optimal frequency band. Finally, the brain function network is constructed by using the average value of the fusion features to analyze the interaction of brain regions in different frequency bands during sleep stages. Results The experimental results have shown that when the number of sub-periods is 30, the α (8–13 Hz) frequency band has the best classification effect, The classification result after 10-fold cross-validation reaches 92.59%. Conclusion The proposed algorithm has good sleep staging performance, which can effectively promote the development and application of an EEG sleep staging system.
... For adversarial attack detection, we extract features based on multiple techniques, including statistical features, chaos-theoretic measures, and stacked denoising auto encoder. We then train two novelty detection algorithms: one-class support vector machine (OCSVM) [8] and local outlier factor (LOF) [4]. The results show that LOF predictions are significantly better than OCSVM where LOF has 89% average F 2 score while OCSVM can reach 63%. ...
Preprint
Full-text available
Industrial Internet of Things (I-IoT) is a collaboration of devices, sensors, and networking equipment to monitor and collect data from industrial operations. Machine learning (ML) methods use this data to make high-level decisions with minimal human intervention. Data-driven predictive maintenance (PDM) is a crucial ML-based I-IoT application to find an optimal maintenance schedule for industrial assets. The performance of these ML methods can seriously be threatened by adversarial attacks where an adversary crafts perturbed data and sends it to the ML model to deteriorate its prediction performance. The models should be able to stay robust against these attacks where robustness is measured by how much perturbation in input data affects model performance. Hence, there is a need for effective defense mechanisms that can protect these models against adversarial attacks. In this work, we propose a double defense mechanism to detect and mitigate adversarial attacks in I-IoT environments. We first detect if there is an adversarial attack on a given sample using novelty detection algorithms. Then, based on the outcome of our algorithm, marking an instance as attack or normal, we select adversarial retraining or standard training to provide a secondary defense layer. If there is an attack, adversarial retraining provides a more robust model, while we apply standard training for regular samples. Since we may not know if an attack will take place, our adaptive mechanism allows us to consider irregular changes in data. The results show that our double defense strategy is highly efficient where we can improve model robustness by up to 64.6% and 52% compared to standard and adversarial retraining, respectively.
... • Support vector machines (SVM) [15]: SVM maps the input vectors non-linearly to a high-dimension feature space, and then uses linear decision surfaces in the feature space to separate the data samples. We evaluate linear SVM (which removes the non-linear mapping) and radial basis function (RBF) SVM (which uses the radian basis function as the non-linear mapping). ...
Article
Full-text available
Chronic heart failure, pulmonary hypertension, acute respiratory distress syndrome (ARDS), coronavirus disease (COVID), and kidney failure are leading causes of death in the U.S. and across the globe. The cornerstone for managing these diseases is assessing patients’ volume fluid status in lungs. Available methods for measuring fluid accumulation in lungs are either expensive and invasive, thus unsuitable for continuous monitoring, or inaccurate and unreliable. With the recent COVID-19 epidemic, the development of a non-invasive, affordable, and accurate method for assessing lung water content in patients became utmost priority for controlling these widespread respiratory related diseases. In this paper, we propose a novel approach for non-invasive assessment of lung water content in patients. The assessment includes quantitative baseline assessment of fluid accumulation in lungs (normal, moderate edema, edema), as well as continuous monitoring of changes in lung water content. The proposed method is based on using a pair of chest patch radio frequency (RF) sensors and measuring the scattering parameters (S-parameters) of a 915-MHz signal transmitted into the body. To conduct an extensive computational study and validate our results, we utilize a National Institute of Health (NIH) database of computerized tomography (CT) scans of lungs in a diverse population of patients. An automatic workflow is proposed to convert CT scan images to three-dimensional lung objects in High-Frequency Simulation Software and obtain the S-parameters of the lungs at different water levels. Then a personalized machine learning model is developed to assess lung water status based on patient attributes and S-parameter measurements. Decision trees are chosen as our models for the superior accuracy and interpretability. Important patient attributes are identified for lung water assessment. A “cluster-then-predict” approach is adopted, where we cluster the patients based on their ages and fat thickness and train a decision tree for each cluster, resulting in simpler and more interpretable decision trees with improved accuracy. The developed machine learning models achieve areas under the receiver operating characteristic curve of 0.719 and 0.756 for 115 male and 119 female patients, respectively. These results suggest that the proposed “Chest Patch” RF sensors and machine learning models present a promising approach for non-invasive monitoring of patients with respiratory diseases.
... In addition, we train an ARIMA [31] model for each relationship and use the fitted parameters as a baseline to compare with the predictive performance of Hawkes features. Next, we use the Hawkes, ARIMA, and concatenated Hawkes+ARIMA feature sets to train three off-the-shelve classification algorithms: Random Forests [2], Support Vector Machines (SVM) [8], and XGBoost [5]. We compute the prediction performance of each algorithm using two nested cross-validation loops. ...
Article
Full-text available
Mobile phones contain a wealth of private information, so we try to keep them secure. We provide large-scale evidence that the psychological profiles of individuals and their relations with their peers can be predicted from seemingly anonymous communication traces – calling and texting logs that service providers routinely collect. Based on two extensive longitudinal studies containing more than 900 college students, we use point process modeling to describe communication patterns. We automatically predict the peer relationship type and temporal dynamics, and assess user personality based on the modeling. For some personality traits, the results are comparable to the gold-standard performances obtained from survey self-report data. Findings illustrate how information usually residing outside the control of individuals can be used to reconstruct sensitive information.
... To clarify experimentally the possibility of diagnosing SS using the color of the tongue, we extracted information on the tongue color from the dataset and trained a machine learning classifier. For the first validation, three well-known algorithms: logistic regression (LR), support vector machine (SVM) 11 , and random forest (RF) 12 , were selected. LR is the oldest classification algorithm, and it uses a non-linear sigmoid function, which enables a slightly more complex division of images into classes compared to linear classification, which is based on a threshold. ...
Article
Full-text available
Sjögren's syndrome (SS) is an autoimmune disease characterized by dry mouth. The cause of SS is unknown, and its diverse symptoms make diagnosis difficult. The Saxon test, an intraoral examination, is used as the primary diagnostic method for SS, however, the risk of salivary infection is problematic. Therefore, we investigate the possibility of diagnosing SS by non-contact and imaging observation of the tongue surface. In this study, we obtained tongue photographs of 60 patients at the Tsurumi University School of Dentistry outpatient clinic to clarify the relationship between the features of the tongue and SS. We divided the tongue into four regions, and the color of each region was transformed into CIE1976L*a*b* space and statistically analyzed. To clarify experimentally the possibility of SS diagnosis using tongue color, we employed three machine-learning models: logistic regression, support vector machine, and random forest. In addition, we constructed diagnostic prediction models based on the Bagging and Stacking methods combined with three machine-learning models for comparative evaluation. This analysis used dimensionality compression by principal component analysis to eliminate redundancy in tongue color information. We found a significant difference between the a* value of the rear part of the tongue and the b* value of the middle part of the tongue in SS and non-SS patients. In addition to the principal component scores of tongue color, the support vector machine was trained using age, and achieved high accuracy (71.3%) and specificity (78.1%). The results indicate that the prediction of SS diagnosis by tongue color reaches a level comparable to machine learning models trained using the Saxon test. This is the first study using machine learning to predict SS diagnosis by non-contact tongue observation. Our proposed method can potentially support early SS detection simply and conveniently, eliminating the risk of infection at diagnosis, and it should be validated and optimized in clinical practice.
... It is widely used in various fields, such as land use and cover statistics, environmental monitoring, geological disaster investigation, Geographic Information System updates, precision agriculture urban planning, military defense, etc. In recent years, technologies, such as the watershed segmentation algorithm [2], visual saliency algorithm [3], canny edge detection algorithm [4], and classification algorithm based on SVM [5], have been applied to the object detection of RGB images and have achieved good results against simple backgrounds. However, in optical remote sensing imaging, the shooting angle is not horizontal, and most captured objects are tiny. ...
Article
Full-text available
The object detection technologies of remote sensing are widely used in various fields, such as environmental monitoring, geological disaster investigation, urban planning, and military defense. However, the detection algorithms lack the robustness to detect tiny objects against complex backgrounds. In this paper, we propose a Multiple Attention Mechanism Enhanced YOLOX (MAME-YOLOX) algorithm to address the above problem. Firstly, the CBAM attention mechanism is introduced into the backbone of the YOLOX, so that the detection network can focus on the saliency information. Secondly, to identify the high-level semantic information and enhance the perception of local geometric feature information, the Swin Transformer is integrated into the YOLOX’s neck module. Finally, instead of GIOU loss, CIoU loss is adopted to measure the bounding box regression loss, which can prevent the GIoU from degenerating into IoU. The experimental results of three publicly available remote sensing datasets, namely, AIBD, HRRSD, and DIOR, show that the algorithm proposed possesses better performance, both in relation to quantitative and qualitative aspects.
... In each tree of (30). The variable selection is implemented using the minimal depth of a respective variable, determined in each decision tree of a RSF as the distance from the root node to the closest node (31). ...
Article
Full-text available
The aim of this study was to evaluate the contribution of macro- and micronutrients intake to mortality in patients with gastrointestinal cancer, comparing the classical statistical approaches with a new generation algorithm. In 1992, the ONCONUT project was started with the aim of evaluating the relationship between diet and cancer development in a Southern Italian elderly population. Patients who died of specific death causes (ICD-10 from 150.0 to 159.9) were included in the study (n = 3,505) and survival analysis was applied. This cohort was used to test the performance of different techniques, namely Cox proportional-hazards model, random survival forest (RSF), Survival Support Vector Machine (SSVM), and C-index, applied to quantify the performance. Lastly, the new prediction mode, denominated Shapley Additive Explanation (SHAP), was adopted. RSF had the best performance (0.7653711 and 0.7725246, for macro- and micronutrients, respectively), while SSVM had the worst C-index (0.5667753 and 0.545222). SHAP was helpful to understand the role of single patient features on mortality. Using SHAP together with RSF and classical CPH was most helpful, and shows promise for future clinical applications.
... Based on maximum margin criterion, several models were first built for linearly separable data sets (Vapnik 1999). In order to classify the linearly nonseparable data sets, Cortes and Vapnik (1995) introduced slack variables for misclassification samples and proposed a soft margin method based on the principle of structural risk minimization. Instead of searching an optimal hyperplane, Boser et al. (1992) introduced the kernel method to search a nonlinear surface in the input space by projecting a data set into a higher dimensional space. ...
Article
Full-text available
Support vector machines have been widely applied in binary classification, which are constructed based on crisp data. However, the data obtained in practice are sometimes imprecise, in which classical support vector machines fail in these situations. In order to handle such cases, this paper employs uncertain variables to describe imprecise observations and further proposes a hard margin uncertain support vector machine for the problem with imprecise observations. Specifically, we first define the distance from an uncertain vector to a hyperplane and give the concept of a linearly α-separable data set. Then, based on maximum margin criterion, we propose an uncertain support vector machine for the linearly α-separable data set, and derive the corresponding crisp equivalent forms. New observations can be classified through the optimal hyperplane derived from the model. Finally, a numerical example is given to illustrate the uncertain support vector machine.
... Proposed by Cortes and Vapnik, SVM is a binary classification tool that can be extended for multiclass problems [57]. In the present study, it was used for a tripartite classification as positive, negative, and neutral. ...
Article
Full-text available
COVID-19, a novel virus from the coronavirus family, broke out in Wuhan city of China and spread all over the world, killing more than 5.5 million people. The speed of spreading is still critical as an infectious disease, and it causes more and more deaths each passing day. COVID-19 pandemic has resulted in many different psychological effects on people’s mental states, such as anxiety, fear, and similar complex feelings. Millions of people worldwide have shared their opinions on COVID-19 on several social media websites, particularly on Twitter. Therefore, it is likely to minimize the negative psychological impact of the disease on society by obtaining individuals’ views on COVID-19 from social media platforms, making deductions from their statements, and identifying negative statements about the disease. In this respect, Twitter sentiment analysis (TSA), a recently popular research topic, is used to perform data analysis on social media platforms such as Twitter and reach certain conclusions. The present study, too, proposes TSA using convolutional neural network optimized via arithmetic optimization algorithm (TSA-CNN-AOA) approach. Firstly, using a designed API, 173,638 tweets about COVID-19 were extracted from Twitter between July 25, 2020, and August 30, 2020 to create a database. Later, significant information was extracted from this database using FastText Skip-gram. The proposed approach benefits from a designed convolutional neural network (CNN) model as a feature extractor. Thanks to arithmetic optimization algorithm (AOA), a feature selection process was also applied to the features obtained from CNN. Later, K-nearest neighbors (KNN), support vector machine, and decision tree were used to classify tweets as positive, negative, and neutral. In order to measure the TSA performance of the proposed method, it was compared with different approaches. The results demonstrated that TSA-CNN-AOA (KNN) achieved the highest tweet classification performance with an accuracy rate of 95.098. It is evident from the experimental studies that the proposed approach displayed a much higher TSA performance compared to other similar approaches in the existing literature.
... Support vector machines (SVM) have been developed in the framework of statistical learning theory [31,32], and have been successfully applied to a number of applications ranging from time series prediction [33], to face recognition [34], to data processing for medical diagnosis [35,36]. SVM is a supervised learning technique based on the methods of separating hyper-planes. ...
Article
Full-text available
In the context of Industry 4.0, condition-based maintenance (CBM) for complex systems is essential in order to identify failures and mitigate them. After the identification of a sensor set that guarantees the system monitoring, three main problems must be addressed for effective CBM: (i) collection of the right data; (ii) choice of the optimal technique to identify the specific dataset; (iii) correct classification of the results. The solutions currently used are typically data driven and, therefore, the results are variable, as it is sometimes challenging to identify a pattern for all specific failures. This paper presents a solution that combines a data driven approach with an in-depth knowledge of the mechanical system’s behaviour. The choice of the right sensor set is calculated with the aid of the software MADe (Maintenance Aware Design environment), whereas the optimal dataset identification technique is pursued with a second tool called Syndrome Diagnostics. After an overview of such methodology, this work also presents RSGWPT (redundant second-generation wavelet packaged transform) analysis to show different possible outcomes depending on the available sensor data and to tailor a detection technique to a given dataset. Supervised and unsupervised learning techniques are tested to obtain either an anomaly detection or a failure identification depending on the chosen sensor set. By using the described method, it is possible to identify potential failures in the system so to awarely implement the optimal maintenance actions.
... KNN is a non-parametric supervised machine learning algorithm used to solve both classification and regression problem and classifies the new data point based on a similarity measure [27]. The KNN procedure is described by figure 2. Fig. 2. KNN Procedure Support Vector Machine: Support Vector Machine is a supervised machine learning algorithm used to both classification and regression problem [28]. The algorithm separates a data point into class attribute using hyperplane and the goal of the line is to maximizing the margin between the points on either side of the so called decision line. ...
Article
The purpose of this project is to develop the mobile application, by applied Machine learning, for analyzing, collecting, monitoring, and retrieving information between patients with diabetes especially diabetes type 2 and village public health volunteers and to study the impact of using mobile application based on self- learning and self-management in diabetes information. This is a research and development mobile application and the sample consisted of 30 diabetes patients and 5 village health volunteers participated in this research. The project has demonstrated the effectiveness of using mobile application to support patients and village health volunteers. The results showed that user satisfaction has a high level.
... The support-vector machine (SVM) [7] was initially created to solve logistic or classification problems. This method generalizes the Random Forest one and its purpose is to find the most optimal hyper-plane that divide the data into two recognizable and well-defined classes. ...
Preprint
Full-text available
In this study, we approached the Hit Song Prediction problem, which aims to predict which songs will become Billboard hits. We gathered a dataset of nearly 18500 hit and non-hit songs and extracted their audio features using the Spotify Web API. We test four machine-learning models on our dataset. We were able to predict the Billboard success of a song with approximately 86\% accuracy. The most succesful algorithms were Random Forest and Support Vector Machine.
... 1.1.2, https://scikit-learn.org, accessed on 30 July 2021) [23]. Due to the reduced sample size, the leave-one-out method was used as a cross-validation strategy to divide the normalized gene expression values into train and test subsets. ...
Article
Full-text available
Candida albicans is one of the most commonly found species in fungal infections. Due to its clinical importance, molecular aspects of the host immune defense against the fungus are of interest to biomedical sciences. Long non-coding RNAs (lncRNAs) have been investigated in different pathologies and gained widespread attention regarding their role as gene regulators. However, the biological processes in which most lncRNAs perform their function are still unclear. This study investigates the association between lncRNAs with host response to C. albicans using a public RNA-Seq dataset from lung samples of female C57BL/6J wild-type Mus musculus with induced C. albicans infection. The animals were exposed to the fungus for 24 h before sample collection. We selected lncRNAs and protein-coding genes related to the host immune response by combining the results from different computational approaches used for gene selection: differential expression gene analysis, co-expression genes network analysis, and machine learning-based gene selection. Using a guilt by association strategy, we inferred connections between 41 lncRNAs and 25 biological processes. Our results indicated that nine up-regulated lncRNAs were associated with biological processes derived from the response to wounding: 1200007C13Rik, 4833418N02Rik, Gm12840, Gm15832, Gm20186, Gm38037, Gm45774, Gm4610, Mir22hg, and Mirt1. Additionally, 29 lncRNAs were related to genes involved in immune response, while 22 lncRNAs were associated with processes related to reactive species production. These results support the participation of lncRNAs during C. albicans infection, and may contribute to new studies investigating lncRNA functions in the immune response.
... [47,[78][79][80]. When the distributions are not known, several classifiers such as logistic regression, support vector machines, naive Bayes, have been proposed in the literature [81][82][83][84]. However, none of these classifiers ensures a guarantee on the type-I error probability resulting in the possibility of a large type-I errors. ...
Thesis
Statistical testing is one of the main problems in statistics and finds applications in a number of fields, including engineering, signal processing, medicine, and finance among others. Traditionally in hypothesis testing problem, the hypothesis distributions subject to testing are known. However, in practice, true distributions are difficult to obtain. In this thesis, we study the hypothesis testing problem with unknown distribution. In the first part of this thesis, we consider the problem of mismatched binary hypothesis testing between i.i.d. distributions, and between Markov sources. We analyze the tradeoff between the pairwise error probability exponents when the actual distributions generating the observation are different from the distributions used in the likelihood ratio test, sequential probability ratio test, and Hoeffding's generalized likelihood ratio test in the composite setting. When the real distributions are within a small divergence ball of the test distributions, we define the worst-case error exponent of each test with respect to the matched error exponent. In addition, we consider the case where an adversary tampers with the observation, again within a divergence ball of the observation type. We show that the tests are more sensitive to distribution mismatch than to adversarial observation tampering. In the next part of the thesis, we propose a composite hypothesis test in the Neyman-Pearson setting where the null distribution is known and the alternative distribution belongs to a certain family of distributions. The proposed test interpolates between Hoeffding's test and the likelihood ratio test and achieves the optimal error exponent tradeoff for every distribution in the family. In addition, the proposed test is shown to attain the type-\RNum{1} error probability prefactor of $n^{\frac{\bar{d}-1}{2}}$, where $\bar d$ is the dimension of the family of distributions projected onto a relative entropy ball centered at the null distribution. This can be significantly smaller than the prefactor $n^{\frac{d-2}{2}}$ achieved by the Hoeffding's test where $d$ is the dimension of the probability simplex. In addition, the proposed test achieves the optimal type-\RNum{2} error probability prefactor for every distribution in the family. Finally, we consider the universal classification for the binary Neyman-Pearson classification where the null distribution is known while only a training sequence is available for the alternative distribution. The proposed classifier interpolates between Hoeffding's classifier and the likelihood ratio test and attains the same error probability prefactor as the likelihood ratio test, i.e., the same prefactor as if both distributions were known. In addition, like Hoeffding's universal classifier, the proposed classifier is shown to attain the optimal error exponent tradeoff attained by the likelihood ratio test whenever the ratio of training to observation samples exceeds a certain value. We propose upper and lower bounds to the training to observation ratio. In addition, we propose a sequential classifier that attains the optimal error exponent tradeoff. We also consider the classification problem in the minimax setting when both distributions are unknown in the Neyman-Pearson setting. We propose classifiers that can asymptotically achieve a predetermined ratio between type-\RNum{2} and type-\RNum{1} error exponents.
... Our pipeline simplifies analysis by focusing only on the color range of the target feature; it therefore requires minimal manual labor and can be performed quickly on consumer-grade computers using large numbers of medium-resolution images. Although we showed two simple methods of predetermined classification in our examples, the data generated from the first three scripts can be readily fed into other more complex classification methods such as support vector machines or random forests (Cortes and Vapnik, 1995;Breiman, 2001) with a predetermined number of color bins. If there is no prior knowledge or preference for the number of bins, one can use unsupervised machine learning techniques such as OPTICS or DBSCAN (Ester et al., 1996;Ankerst et al., 1999) to create color bins using the HSV data collected in the Data Collector script. ...
Article
Full-text available
Petal color is an ecologically important trait, and uncovering color variation over a geographic range, particularly in species with large distributions and/or short bloom times, requires extensive fieldwork. We have developed an alternative method that segments images from citizen science repositories using Python and k‐means clustering in the hue‐saturation‐value (HSV) color space. Our method uses k‐means clustering to aggregate like‐color pixels in sample images to generate the HSV color space encapsulating the color range of petals. Using the HSV values, our method isolates photographs containing clusters in that range and bins them into a classification scheme based on user‐defined categories. We demonstrate the application of this method using two species: one with a continuous range of variation of pink‐purple petals in Geranium maculatum, and one with a binary classification of white versus blue in Linanthus parryae. We demonstrate results that are repeatable and accurate. This method provides a flexible, robust, and easily adjustable approach for the classification of color images from citizen science repositories. By using color to classify images, this pipeline sidesteps many of the issues encountered using more traditional computer vision applications. This approach provides a tool for making use of large citizen scientist data sets.
... It is suitable for supervised classification problems. 39 Random forest (RF) is an ensemble learning classifier based on the decision tree. It contains a specified number of decision tree classifiers, and the output category is decided by the output results of these decision trees. ...
Article
Full-text available
Objective: At present, there is still a lack of reliable biomarkers for ovarian cancer (OC) to guide prognosis prediction and accurately evaluate the dominant population of immunotherapy. In recent years, the relationship between peripheral blood markers and tumor-infiltrating immune cells (TICs) with cancer has attracted much attention. However, the relationship between the survival of OC patients and intratumoral- or extratumoral-associated immune cells remains controversial. Methods: In this study, four machine-learning algorithms were used to predict overall survival in OC patients based on peripheral blood indicators. To further screen out immune-related gene and molecular targets, we systematically explored the correlation between TICs and OC patient survival based on The Cancer Genome Atlas database. Using the TICs score method, patients were divided into a low immune infiltrating cell group and a high immune infiltrating cell group. Results: The results showed that there was a significant statistical significance between the peripheral blood indicators and the survival prognosis of OC patients. Survival analysis showed that TICs play a crucial role in the survival of OC patients. Four core genes, CXCL9, CD79A, MS4A1, and MZB1, were identified by cross-PPI and COX regression analysis. Further analysis found that these genes were significantly associated with both TICs and survival in OC patients. Conclusions: These results suggest that both peripheral blood markers and TICs can be used as prognostic predictors in patients with OC, and CXCL9, CD79A, MS4A1, and MZB1 may be potential therapeutic targets for OC immunotherapy.
... In classification problems, a classifier is a function that mimics the relationship between the data vectors and their class labels. Support vector machine (SVM) is a popular classifier, which was proposed by Cortes and Vapnik [1] as a maximum margin classifier. The success of the SVM has encouraged further research into extensions to the more general multiclass cases, which has been an active topic of research interest [2][3][4]. ...
Article
Full-text available
Support vector machines (SVM) is one of the well known supervised machine learning model. The standard SVM models are dealing with the situation where the exact values of the data points are known. This paper studies the SVM model when the data set contains uncertain or mislabelled data points. To ensure the small probability of misclassification for the uncertain data, a chance constrained conic-segmentation SVM model is proposed for multiclass classification. Based on the data set, a mixed integer programming formulation for the chance constrained conic-segmentation SVM is derived. Kernelization of chance constrained conic-segmentation SVM model is also exploited for nonlinear classification. The geometric interpretation is presented to show how the chance constrained conic-segmentation SVM works on uncertain data. Finally, experimental results are presented to demonstrate the effectiveness of the chance constrained conic-segmentation SVM for both artificial and real-world data.
... However, simple statistical rules are prone to producing more false negatives and false positives. Conventional distribution-based methods for anomaly detection, such as SVM [13,14], one-class SVM [15], and kernel density estimation [16][17][18] are fragile when dealing with high-dimensionality data. The drawbacks of distribution-based techniques spawned a considerably more robust method using deep learning for anomaly detection. ...
Article
Full-text available
Anomalies are a set of samples that do not follow the normal behavior of the majority of data. In an industrial dataset, anomalies appear in a very small number of samples. Currently, deep learning-based models have achieved important advances in image anomaly detection. However, with general models, real-world application data consisting of non-ideal images, also known as poison images, become a challenge. When the work environment is not conducive to consistently acquiring a good or ideal sample, an additional adaptive learning model is needed. In this work, we design a potential methodology to tackle poison or non-ideal images that commonly appear in industrial production lines by enhancing the existing training data. We propose Hierarchical Image Transformation and Multi-level Features (HIT-MiLF) modules for an anomaly detection network to adapt to perturbances from novelties in testing images. This approach provides a hierarchical process for image transformation during pre-processing and explores the most efficient layer of extracted features from a CNN backbone. The model generates new transformations of training samples that simulate the non-ideal condition and learn the normality in high-dimensional features before applying a Gaussian mixture model to detect the anomalies from new data that it has never seen before. Our experimental results show that hierarchical transformation and multi-level feature exploration improve the baseline performance on industrial metal datasets.
... To better appreciate dataset distillation, we briefly introduce other cognate methods in dataset reduction. For the support vector machine (SVMs), its hyperplane is solely determined by "support vectors", and removing all other points in the training dataset does not have an influence on the convergence result [Cortes and Vapnik, 1995]. Therefore, the selection of "support vectors" is a favourable method to reduce the training burden for SVMs [Nalepa and Kawulok, 2019]. ...
Preprint
Deep learning technology has unprecedentedly developed in the last decade and has become the primary choice in many application domains. This progress is mainly attributed to a systematic collaboration that rapidly growing computing resources encourage advanced algorithms to deal with massive data. However, it gradually becomes challenging to cope with the unlimited growth of data with limited computing power. To this end, diverse approaches are proposed to improve data processing efficiency. Dataset distillation, one of the dataset reduction methods, tackles the problem via synthesising a small typical dataset from giant data and has attracted a lot of attention from the deep learning community. Existing dataset distillation methods can be taxonomised into meta-learning and data match framework according to whether explicitly mimic target data. Albeit dataset distillation has shown a surprising performance in compressing datasets, it still possesses several limitations such as distilling high-resolution data. This paper provides a holistic understanding of dataset distillation from multiple aspects, including distillation frameworks and algorithms, disentangled dataset distillation, performance comparison, and applications. Finally, we discuss challenges and promising directions to further promote future studies about dataset distillation.
... However, in practice the data points are not necessarily linearly separable. To allow mislabelling, the concept of soft margin SVM was introduced in [22]. Let {(φ i , ζ i ) ∈ R m × {−1, +1}|i = 1, . . . ...
Preprint
Full-text available
Quantum linear system algorithms (QLSAs) have the potential to speed up algorithms that rely on solving linear systems. Interior Point Methods (IPMs) yield a fundamental family of polynomial-time algorithms for solving optimization problems. IPMs solve a Newton linear system at each iteration to find the search direction, and thus QLSAs can potentially speed up IPMs. Due to the noise in contemporary quantum computers, such quantum-assisted IPM (QIPM) only allows an inexact solution for the Newton linear system. Typically, an inexact search direction leads to an infeasible solution. In our work, we propose an Inexact-Feasible QIPM (IF-QIPM) and show its advantage in solving linearly constrained quadratic optimization problems. We also apply the algorithm to $\ell_1$-norm soft margin support vector machine (SVM) problems and obtain the best complexity regarding dependence on dimension. This complexity bound is better than any existing classical or quantum algorithm that produces a classical solution.
... The support vector machine is a nonparametric supervised statistical learning technique developed by Cortes and Vapnik (Cortes and Vapnik 1995). This algorithm is used for classification and regression problems that have no assumptions about the distribution of basic data. ...
Conference Paper
Full-text available
In recent decades, global warming and sea level rise, population growth, and intensification of human activities, have directly affected the coasts and as such, their monitoring for the accretion and retreat are among the issues that are considered by the coastal countries. This study, compares two supervised classification algorithms for classifying Sentinel-2 satellite imagery for shoreline extraction. Median monthly images from 2020/01 to 2021/12 are taken and classified by Random Forest (RF) and Support Vector Machine (SVM) algorithms. By validating the maps, it is found that the RF algorithm has better accuracy and as such by averaging the accuracy of all maps, the overall accuracy (OA) values of 97.18% and the kappa coefficient (KC) of 0.97, and the mean overall accuracy and kappa coefficient of maps from SVM algorithm of 85.15% and 0.79, respectively, is obtained. After extracting the shorelines, the Digital Shoreline Analysis System (DSAS) is used to calculate the displacement rate. By calculating the Linear Regression Rate (LRR) factor, it is found that in 91% of transects (166 transects) we see the shoreline retreat to land. In 54% of them, the average rate of the retreat is 5.42 meters per year and in only 9% (16 transects) we see the accretion towards the sea.
... This section discusses the experimental results obtained during the recognition of nine mudstone lithofacies belonging to Kansas oil and gas fields. The performance of Stacking and Voting ensembles was compared with four popular classifiers, namely GB [80], RF [81], SVM [82], and MLP [30,31]. Stacking and Voting are two HEMs that were implemented to predict complex lithofacies. ...
Thesis
Full-text available
In the oil and gas industry, a huge amount of data is generated through sensory measurements during exploration to production phases of the reservoir. Uncertainties and inexactness are present in all the reservoir measurements due to heterogeneity and stochastic distribution of reservoir characteristics. Conventionally, field data are interpreted by experienced experts to extract useful information. However, with the advent of measurements-while-drilling and smart-well technologies, there is a significant increase in the volume of data generated and to be analyzed. Therefore, processing and analysis of this huge data pose a significant challenge to the prevailing technologies used in the oil and gas industry. In this study, the intelligent modeling approach has been investigated to provide cost-effective solutions for three major problems of petroleum domain viz. (a) Lithofacies identification (b) Drilling optimization (c) Production rate estimation.
... Theoretically, SVM is very good at finding the margin and hyperplane for classification, and it is very robust for high dimensional dataZhang et al. (2006). However, SVM model is sensitive to noise, for example, if there is a noise in background or a visible object in one image is occluded or partially blocked by scenes in other, it will have a negative impact on the performance of the classification model Cortes and Vapnik (1995). Moreover, since one-vs-all involves training a binary classifier for all classes, computation time can be very expensive. ...
Preprint
With the rise of internet technology amidst increasing rates of urbanization, sharing information has never been easier thanks to globally-adopted platforms for digital communication. The resulting output of massive amounts of user-generated data can be used to enhance our understanding of significant societal issues particularly for urbanizing areas. In order to better analyze protest behavior, we enhanced the GSR dataset and manually labeled all the images. We used deep learning techniques to analyze social media data to detect social unrest through image classification, which performed good in predict multi-attributes, then also used map visualization to display protest behaviors across the country.
... Also, if we compare state-of-the-art algorithms with the simple machine learning algorithms, we can admit that intermediate steps are not readable by humans as modern algorithms are composed of many convolution layers that work in the visual domain. Still, additional effort can be made by using some of the nonsupervised algorithms, like density clustering [73] or SVMs (support vector machine) [74] to try to extract different categories of data without initial pretraining. ...
... The work of Tarjoman et al. (2013) proposes a simple feature extraction method using Support Vector Machine (SVM) (Cortes and Vapnik, 1995) and the Grey Level Cooccurrence Matrix as the main input of each image. A similar approximation was followed in Kumar et al. (2016) where 1 https://purl.com/mocae_brats ...
Preprint
Full-text available
The present work proposes a Multi-Output Classification Autoencoder (MOC-AE) algorithm to extract features from brain tumour images. The proposed algorithm is able to focus on both the normal features of the patient and the pathological features present in the case, resulting in a compact and significant representation of each image. The architecture of MOC-AE combines anatomical information from the patients scan using an Autoencoder (AE) with information related to a specific pathology using a classification output with the same image descriptor. This combination of goals forces the network to maintain a balance between anatomical and pathological features of the case while maintaining the low cost of the labels being used. The results obtained are compared with those of similar studies and the strengths and limitations of each approach are discussed. The results demonstrate that the proposed algorithm is capable of achieving state-of-the-art results in terms of both the anatomical and tumor characteristics of the recommended cases.
... In comparison to the model without realized skewness and the model that contains realized skewness and realized kurtosis, our algorithm containing only realized skewness was applicable to acquire the intraday price and shows better pricing performance. Moreover, to verify the performance of the LSTM model when considering realized skewness, benchmark models were exercised for pricing options [40][41][42][43][44][45]. Several evaluating metrics, such as mean square error (MSE), root mean square error (RMSE), mean absolute error (MAE) and mean absolute percentage error (MAPE), were used to test the pricing accuracy of our proposed model. ...
Article
Full-text available
Deep learning has drawn great attention in the financial field due to its powerful ability in nonlinear fitting, especially in the studies of asset pricing. In this paper, we proposed a long short-term memory option pricing model with realized skewness by fully considering the asymmetry of asset return in emerging markets. It was applied to price the ETF50 options of China. In order to emphasize the improvement of this model, a comparison with a parametric method, such as Black-Scholes (BS), and machine learning methods, such as support vector machine (SVM), random forests and recurrent neural network (RNN), was conducted. Moreover, we also took the characteristic of heavy tail into consideration and studied the effect of realized kurtosis on pricing to prove the robustness of the skewness. The empirical results indicate that realized skewness significantly improves the pricing performance of LSTM among moneyness states except for in-the-money call options. Specifically, the LSTM model with realized skewness outperforms the classical method and other machine learning methods in all metrics.
Article
Full-text available
Economic complexity methods, and in particular relatedness measures, lack a systematic evaluation and comparison framework. We argue that out-of-sample forecast exercises should play this role, and we compare various machine learning models to set the prediction benchmark. We find that the key object to forecast is the activation of new products, and that tree-based algorithms clearly outperform both the quite strong auto-correlation benchmark and the other supervised algorithms. Interestingly, we find that the best results are obtained in a cross-validation setting, when data about the predicted country was excluded from the training set. Our approach has direct policy implications, providing a quantitative and scientifically tested measure of the feasibility of introducing a new product in a given country.
Article
Full-text available
Phenotyping approaches have been considered as a vital component in crop breeding programs to improve crops and develop new high-yielding cultivars. However, traditional field-based monitoring methods are expensive, invasive, and time-intensive. Moreover, data collected using satellite and airborne platforms are either costly or limited by their spatial and temporal resolution. Here, we investigated whether low-cost unmanned/unoccupied aerial systems (UASs) data can be used to estimate winter wheat ( Triticum aestivum L.) nitrogen (N) content, structural traits including plant height, fresh and dry biomass, and leaf area index (LAI) as well as yield during different winter wheat growing stages. To achieve this objective, UAS-based red–green–blue (RGB) and multispectral data were collected from winter wheat experimental plots during the winter wheat growing season. In addition, for each UAS flight mission, winter wheat traits and total yield (only at harvest) were measured through field sampling for model development and validation. We then used a set of vegetation indices (VIs), machine learning algorithms (MLAs), and structure-from-motion (SfM) to estimate winter wheat traits and yield. We found that using linear regression and MLAs, instead of using VIs, improved the capability of UAS-derived data in estimating winter wheat traits and yield. Further, considering the costly and time-intensive process of collecting in-situ data for developing MLAs, using SfM-derived elevation models and red-edge-based VIs, such as CIre and NDRE, are reliable alternatives for estimating key winter wheat traits. Our findings can potentially aid breeders through providing rapid and non-destructive proxies of winter wheat phenotypic traits.
Article
Full-text available
Cybersecurity is one of the most important concerns associated with ever expanding internet based technologies, products, services and networks. If cybersecurity is prevention then cyber forensics is the cure. Both are equally important pillars of digital security. This paper presents an extensive bibliometric analysis of cybersecurity and cyberforensic research published in Web of Science during the past decade (2011–2021). The analysis included yearly publications, publication types and trends across different verticals such as publishing sources, organizations, researchers, countries and keywords. Full counting method was used for citation analysis, whereas fractional counting method was implemented to analyze co-citation, co-author collaborations as well as keyword co-occurrences across all these verticals. Furthermore, timeline and burst detection analyses were carried out to unravel significant topic trends and citations in the last decade. The study presents bibliometric results in terms of the authors, organizations, countries, keywords, sources and documents with the highest collaborative link strengths worldwide in the field of cybersecurity and forensics. Latest trends, under-investigated topics and future directions are also presented.
Article
Cyberbullying refers to bullying and harassment of defenseless or vulnerable people such as children, teenagers, and women through any means of communication (e.g., e-mail, text messages, wall posts, tweets) over any online medium (e.g., social media, blogs, online games, virtual reality environments). The effect of cyberbullying may be severe and irreversible and it has become one of the major problems of cyber-societies in today’s electronic world. Prevention of cyberbullying activities as well as the development of timely response mechanisms, require automated and accurate detection of cyberbullying acts. This study focuses on the problem of cyberbullying detection over Facebook activity content written in Turkish. Through extensive experiments with the various machine and deep learning algorithms, the best estimator for the task is chosen and then employed for both cross-domain evaluation and profiling of cyber-aggressive users. The results obtained with 5-fold cross-validation are evaluated with an average-macro F1 score. These results show that BERT is the best estimator with an average macro F1 of 0.928, and employing it on various datasets collected from different OSN domains produces highly satisfying results. This paper also reports detailed profiling of cyber-aggressive users by providing even more information than what is visible to the naked eye.
Preprint
Full-text available
The assessment of regional low carbon development level can fully reflect the current situation of low carbon development, and the carbon emission forecasting can reveal the future trend of carbon emission and pressure of emission reduction. Therefore, it is of great significance to carry out the assessment of low carbon development level and carbon emission forecasting in Gansu, Qinghai and Shaanxi provinces for regional green economy development. Therefore, this research constructs an index system to objectively reflect the low carbon development level of Gansu, Qinghai and Shaanxi provinces, and proposes IE-TOPSIS comprehensive assessment method to assess the regional low carbon development level from 2010 to 2020. It is showed that low carbon development level of the three provinces over the past decade has an increasing trend. Furthermore, this research establishes a GRA-DPC-MAPE combined forecasting model for carbon emission forecasting of Gansu, Qinghai and Shaanxi. GRA-DPC-MAPE model has higher forecast accuracy than individual model SVR, PSO-SVR, ELM and Elman. It is showed that carbon emission of Gansu and Shaanxi provinces will continue to increase from 2021 to 2025 under the current development.
Preprint
p>Multi-feature SAR ship classification aims to build models that can process, correlate, and fuse information from handcrafted features and deep features. Although handcrafted ones provide rich expert knowledge, current fusion methods do not thoroughly investigate the relatively important of handcrafted features with deep features, feature contribution imbalance, and the way features learn collaboratively. In this paper, a novel multi-feature collaborative fusion network with deep supervision (MFCFNet) is proposed to effectively realize handcrafted feature and deep feature fusion in SAR ship classification task. Specifically, our framework mainly includes two types of feature extraction branches, a knowledge supervision and collaboration module, and a feature fusion and contribution assignment module. The former module improves the feature map quality learned by each branch through auxiliary feature supervision, and introduces synergy loss to facilitate the information interaction between deep features and handcrafted features. The latter utilizes an attention mechanism to adaptively balance the importance among various features, and to assign the corresponding feature contribution to the total loss function based on the generated feature weights. We conduct extensive experimental and ablation studies on two public OpenSARShip-1.0 and FUSAR-Ship datasets, and the results show that MFCFNet is effective and outperforms single deep feature and multi-feature models based on previous Internal FC Layer and Terminal FC Layer fusion, and exhibits better performance than the current state-of-the-art methods.</p
Article
Applying ensemble feature selection (EFS) models in various problems has not been actively discussed, and there has been a lack of effort to make it applicable in the situations such as distributed environments. Due to restrictions of centralized algorithms such as their poor scalability in the high dimension data and also distributed nature of some data, using the traditional centralized computing for dealing with such problems may be inevitable. This paper aims to develop a homogenous distributed ensemble feature selection (HMDE‐FS) framework through a distributed resampling approach rather than a centralized one. The homogenous ensembles mainly operate along with a resampling process, so applying various methods to resampling can affect the performance of the model. Among various strategies, those with and without replacement are two of the main technique families, hence we investigated the efficiency of two well‐known with/without replacement techniques: bootstrapping (BS) and cross‐validation (CV) inspired method that we named crisscross (CC). The proposed HMDE‐FS approaches are tested on eight datasets, and the heavy experimental results illustrate that these methods considerably reduce runtime, while classification accuracy maintains its competitiveness.
Article
Full-text available
The performance of a state-of-the-art neural network classifier for hand-written digits is compared to that of a k-nearest-neighbor classifier and to human performance. The neural network has a clear advantage over the k-nearest-neighbor method, but at the same time does not yet reach human performance. Two methods for combining neural-network ideas and the k-nearest-neighbor algorithm are proposed. Numerical experiments for these methods show an improvement in performance.
Conference Paper
Full-text available
This paper compares the performance of several classifier algorithms on a standard database of handwritten digits. We consider not only raw accuracy, but also training time, recognition time, and memory requirements. When available, we report measurements of the fraction of patterns that must be rejected so that the remaining patterns have misclassification rates less than a given threshold
Article
Full-text available
A training algorithm that maximizes the margin between the training patterns and the decision boundary is presented. The technique is applicable to a wide variety of classifiaction functions, including Perceptrons, polynomials, and Radial Basis Functions. The effective number of parameters is adjusted automatically to match the complexity of the problem. The solution is expressed as a linear combination of supporting patterns. These are the subset of training patterns that are closest to the decision boundary. Bounds on the generalization performance based on the leave-one-out method and the VC-dimension are given. Experimental results on optical character recognition problems demonstrate the good generalization obtained when compared with other learning algorithms. 1
Article
We describe a new learning procedure, back-propagation, for networks of neurone-like units. The procedure repeatedly adjusts the weights of the connections in the network so as to minimize a measure of the difference between the actual output vector of the net and the desired output vector. As a result of the weight adjustments, internal 'hidden' units which are not part of the input or output come to represent important features of the task domain, and the regularities in the task are captured by the interactions of these units. The ability to create useful new features distinguishes back-propagation from earlier, simpler methods such as the perceptron-convergence procedure.
Article
Linear procedures for classifying an observation as coming from one of two multivariate normal distributions are studied in the case that the two distributions differ both in mean vectors and covariance matrices. We find the class of admissible linear procedures, which is the minimal complete class of linear procedures. It is shown how to construct the linear procedure which minimizes one probability of misclassification given the other and how to obtain the minimax linear procedure; Bayes linear procedures are also discussed.
We present an application of back-propagation networks to handwritten digit recognition. Minimal preprocessing of the data was required, but architecture of the network was highly constrained and specifically designed for the task. The input of the network consists of normalized images of isolated digits. The method has 1% error rate and about a 9% reject rate on zipcode digits provided by the U.S. Postal Service. 1 INTRODUCTION The main point of this paper is to show that large back-propagation (BP) networks can be applied to real image-recognition problems without a large, complex preprocessing stage requiring detailed engineering. Unlike most previous work on the subject (Denker et al., 1989), the learning network is directly fed with images, rather than feature vectors, thus demonstrating the ability of BP networks to deal with large amounts of low level information. Previous work performed on simple digit images (Le Cun, 1989) showed that the architecture of the network s...
Cognitiva 85: A la Frontiere de l'Intelligence Artificielle des Sciences de la Connaissance des Neurosciences
  • Y Lecun
LeCun, Y. (1985). Une procedure d'apprentissage pour reseau a seuil assymetrique. Cognitiva 85: A la Frontiere de l'Intelligence Artificielle des Sciences de la Connaissance des Neurosciences, 599-604, Paris.
Comparison of classifier methods: A case study in handwritten digit recognition
  • L Bottou
  • C Cortes
  • J S Denker
  • H Drucker
  • I Guyon
  • L D Jackel
  • Y Lecun
  • E Sackinger
  • P Simard
  • V Vapnik
  • U A Miller
Bottou, L., Cortes, C., Denker, J.S., Drucker, H., Guyon, I., Jackel, L.D., LeCun, Y., Sackinger, E., Simard, P., Vapnik, V., & Miller, U.A. (1994). Comparison of classifier methods: A case study in handwritten digit recognition. Proceedings of 12th International Conference on Pattern Recognition and Neural Network.
  • R Courant
  • D Hilbert
Courant, R., & Hilbert, D. (1953). Methods ofMathematical Physics, Interscience, New York.