Figure 3 - uploaded by Asa Ben-Hur
Content may be subject to copyright.
3.3. The effect of the soft-margin constant, C, on the decision boundary. A smaller value of C (right) allows to ignore points close to the boundary and increases the margin. The decision boundary between negative examples (circles) and positive examples (crosses) is shown as a thick line. The lighter lines are on the margin (discriminant value equal to –1 or +1). The grayscale level represents the value of the discriminant function, dark for low values and a light shade for high values.
Source publication
The Support Vector Machine (SVM) is a widely used classifier in bioinformatics. Obtaining the best results with SVMs requires an understanding of their workings and the various ways a user can influence their accuracy. We provide the user with a basic understanding of the theory behind SVMs and focus on their use in practice. We describe the effect...
Similar publications
In recent work, it was shown that combining multi-kernel based support vector machines (SVMs) can lead to near state-of-the-art performance on an action recognition dataset (HMDB-51 dataset). This was 0.4\% lower than frameworks that used hand-crafted features in addition to the deep convolutional feature extractors. In the present work, we show th...
The negative impact of reduced visibility on driver performance has been recognized as one of the major causes of motor vehicle crashes. Proper assessment of real-time visibility condition is therefore crucial for safe driving, especially during adverse weather including fog. Although many studies have investigated various visibility detection meth...
With the emergence of data streaming applications that produce large data in motion, anomaly detection in non-stationary environments has become a major research focus. Unknown and unstable behaviour of data over time, limits the application of traditional anomaly detection methods that have been designed for stationary data. Moreover, basic assump...
Detecting and monitoring of abnormal movement behaviors in patients with Parkinson’s Disease (PD) and individuals with Autism Spectrum Disorders (ASD) are beneficial for adjusting care and medical treatment in order to improve the patient’s quality of life. Supervised methods commonly used in the literature need annotation of data, which is a time-...
Time series classification (TSC) arises in many fields and has a wide range of applications. Here, we adopt the bag-of-words (BoW) framework to classify time series. Our algorithm first samples local subsequences from time series at feature-point locations when available. It then builds local descriptors, and models their distribution by Gaussian m...
Citations
... 36 Nonetheless, it can be computationally expensive, particularly with large datasets. 37 RF provides robust feature selection through its ensemble of DTs, offering resistance to overfitting and better interpretability of the selected features. However, its complexity can lead to longer training times. ...
This study aimed to develop an automated classification framework for distinguishing between cervical cancer tumor and normal uterine tissue, leveraging CT images for radiomics feature extraction. We retrospectively analyzed CT images from 117 cervical cancer patients. To distinguish between cancerous and healthy tissue, we segmented gross tumor volume and normal uterine tissue as distinct regions of interest (ROIs) using manual segmentation techniques. Key radiomic parameters were extracted from these ROIs. To bolster model's predictive capability, the data was stratified into train data (70%) and validation data (30%). During feature selection phase, we applied Least Absolute Shrinkage and Selection Operator regression algorithm to identify most relevant features. Subsequently, we built classification models using five state-of-the-art machine learning algorithms: Support Vector Machine (SVM), Random Forest (RF), K-Nearest Neighbors (KNN), Extreme Gradient Boosting (XGBoost), and Decision Tree (DT). Ultimately, the performance of each model was evaluated. Through stringent feature selection process, we identified 18 pivotal radiomic features for classification of cervical cancer and normal uterine tissue. When applied to test data, all five models achieved excellent performance, with area under the curve (AUC) values ranging from 0.8866 to 0.9190 (SVM: 0.9144, RF: 0.9078, KNN: 0.9051, DT: 0.8866, XGBoost: 0.9190), all surpassing threshold of 0.8. In terms of test data, all five models had high sensitivity; accuracy of SVM, RF, and XGBoost models was comparable; and specificity of five models was similar. XGBoost model outperformed the others in terms of diagnostic accuracy, achieving an AUC of 0.8737 (95% CI: 0.8198-0.9277) for train data and 0.9190 (95% CI: 0.8525-0.9854) for test data. Our findings underscore the potential of CT radiomics combined with machine learning algorithms for accurately classifying cervical cancer tumors and normal uterine tissue with high recognition capabilities. This approach holds significant promise for clinical diagnostics.
... A quadratic programming optimization problem obtains the parameters of the solution hyperplane. Kernel methods prevent data from being mapped into a feature space with various dimensions to deal with a memory increase problem that follows a quadratic trend to storing features and the time required to compute the discriminant function of the classifier 50 . By considering the data mentioned above set consisting of pairs x 1 , y 1 , x 2 , y 2 , x 3 , y 3 , . . . ...
Valvular Heart Disease (VHD) is a globally significant cause of mortality, particularly among aging populations. Despite advancements in percutaneous and surgical interventions, there are still uncertainties that remain regarding the risk factors that significantly contribute to this condition within the domain of cardiovascular disease. This study investigates these uncertainties and the role of machine learning in categorizing VHD based on cardiovascular risk factors. It follows a two-part investigation comprising feature extraction and classification phases. Feature extraction is initially performed using a wrapping approach and refined further with binary logistic regression. The second phase employs five classifiers: Artificial Neural Network (ANN), XGBoost, Random Forest (RF), Naïve Bayes, and Support Vector Machine (SVM), along with advanced methods such as SVM combined with Principal Component Analysis (PCA) and a majority-voting ensemble method (MV5). Data on VHD cases were collected from DHQ Hospital Faisalabad using simple random sampling. Various statistical measures, such as the ROC curve, F-measure, sensitivity, specificity, accuracy, MCC, and Kappa are applied to assess the results. The findings reveal that the combination of SVM with PCA achieves the highest overall performance while the MV5 ensemble method also demonstrates high accuracy and balance in sensitivity and specificity. The variation in VHD prevalence linked to specific risk factors highlights the importance of a comprehensive approach to reduce this disease’s burden. The Exceptional performance of SVM + PCA and MV5 highlights their significance in diagnosing VHD and advancing knowledge in biomedicine.
... Some of the commonly used kernels include: The choice of kernel in classification problems is application-dependent. Typically, the Gaussian kernel outperforms the polynomial kernel in terms of accuracy and convergence time [30]. The Gaussian kernel is notable for its interpolation ability and effectiveness in capturing local properties, but it has computational and scalability issues, particularly with large datasets. ...
As particle physics experiments evolve to achieve higher energies and resolutions, handling the massive data volumes produced by silicon pixel detectors, which are used for charged particle tracking, poses a significant challenge. To address the challenge of data transport from high resolution tracking systems, we investigate a support vector machine (SVM)-based data classification system designed to reject low-momentum particles in real-time. This SVM system achieves high accuracy through the use of a customized mixed kernel function, which is specifically adapted to the data recorded by a silicon tracker. Moreover, this custom kernel can be implemented using highly efficient, novel van der Waals heterojunction devices. This study demonstrates the co-design of circuits with applications that may be adapted to meet future device and processing needs in high-energy physics (HEP) collider experiments.
... The complex interplay of factors influencing malaria transmission, including climatic conditions, vector behavior, human demographics, and socioeconomic determinants, underscores the need for advanced analytical approaches to predict disease outbreaks accurately (Ben-Hur & Weston, 2010;Tay & Cao, 2001). Traditional methods of surveillance and prediction often fall short in capturing the dynamic nature of malaria transmission dynamics. ...
Malaria continues to pose a significant public health challenge globally, with Indonesia being among the countries most affected by the disease. Despite extensive efforts to control malaria transmission, the disease remains endemic in various regions, leading to substantial morbidity and mortality. Accurate prediction of malaria cases is crucial for guiding effective prevention and control strategies, particularly in resource-limited settings. This study investigates the application of machine learning (ML) techniques to predict malaria incidence in Indonesia, leveraging climatic, epidemiological, and socioeconomic data. Three ML algorithms, namely Random Forest, Support Vector Machine (SVM), and Artificial Neural Networks (ANN), are employed and evaluated for their predictive capabilities. The study spans from 2010 to 2021, incorporating diverse datasets from the Indonesian Meteorological, Climatological, and Geophysical Agency (BMKG), the Ministry of Health of Indonesia, and the Indonesian Bureau of Statistics (BPS). Results indicate that the ML models exhibit strong predictive performance, with Random Forest demonstrating the highest accuracy. The integration of multidimensional data sources enhances the robustness of the predictive models, enabling the identification of spatiotemporal patterns in malaria transmission dynamics. The findings underscore the potential of ML-based approaches in improving malaria surveillance and control efforts in Indonesia, offering valuable insights for public health decision-makers and stakeholders. Moreover, the study highlights the importance of data quality, model refinement, and interdisciplinary collaboration in addressing complex public health challenges such as malaria. By harnessing the power of advanced analytics and innovative methodologies, this research contributes to the ongoing efforts to combat malaria and alleviate its burden on communities and healthcare systems in Indonesia and beyond.
... Hyperparameter values are typically set before the learning process starts and influence the value of the decision function. Generally, there are fewer hyperparameters in SVM than in other ML classification algorithms [53]. Specificity, sensitivity, and accuracy are commonly used to measure the performance of SVM, providing insights into the accuracy and reproducibility of the SVM hyperplane that differentiates classes. ...
... A kernel function is a technique for transforming input data into the format needed for data processing. Both non-linear (a statistical method called nonlinear regression is used to model non-linear relationships between independent and dependent variables) and linear regression could be carried out with SVMs based on the kernel function that was applied (Ben-Hur et al., 2008;Ben-Hur and Weston, 2010;Kircher et al., 2014). To quantify, the best idea is to train an SVM through a kernel of a radial basis function and a linear SVM can be used from a non- (Bzdok et al., 2018). ...
Crop improvement and production domains encounter large amounts of expanding data with multi-layer complexity that forces researchers to use machine-learning approaches to establish predictive and informative models to understand the sophisticated mechanisms underlying these processes. All machine-learning approaches aim to fit models to target data; nevertheless, it should be noted that a wide range of specialized methods might initially appear confusing. The principal objective of this study is to offer researchers an explicit introduction to some of the essential machine-learning approaches and their applications, comprising the most modern and utilized methods that have gained widespread adoption in crop improvement or similar domains. This article explicitly explains how different machine-learning methods could be applied for given agricultural data, highlights newly emerging techniques for machine-learning users, and lays out technical strategies for agri/crop research practitioners and researchers.
... Hyperparameter values are typically set before the learning process starts and influence the value of the decision function. Generally, there are fewer hyperparameters in SVM than in other ML classification algorithms [53]. Specificity, sensitivity, and accuracy are commonly used to measure the performance of SVM, providing insights into the accuracy and reproducibility of the SVM hyperplane that differentiates classes. ...
Machine learning (ML) has transformed numerous fields, but understanding its foundational research is crucial for its continued progress. This paper presents an overview of the significant classical ML algorithms and examines the state-of-the-art publications spanning twelve decades through an extensive bibliometric analysis study. We analyzed a dataset of highly cited papers from prominent ML conferences and journals, employing citation and keyword analyses to uncover critical insights. The study further identifies the most influential papers and authors, reveals the evolving collaborative networks within the ML community, and pinpoints prevailing research themes and emerging focus areas. Additionally, we examine the geographic distribution of highly cited publications, highlighting the leading countries in ML research. This study provides a comprehensive overview of the evolution of traditional learning algorithms and their impacts. It discusses challenges and opportunities for future development, focusing on the Global South. The findings from this paper offer valuable insights for both ML experts and the broader research community, enhancing understanding of the field's trajectory and its significant influence on recent advances in learning algorithms.
... Some popular kernel functions include polynomial kernel, Gaussian kernel (known as the radial basis function), and Sigmoid kernel. The Gaussian kernel usually outperforms the polynomial kernel in both accuracy and convergence time [8]. Gaussian kernel has interpolation ability and is effective at identifying local properties, and Sigmoid kernel is better suited for identifying global characteristics but has a relatively weak interpolation ability [11]. ...
... how to optimize hyperparameters in SVMs and kernels? Uninformed choices may result in severely reduced accuracy [8], [15], however, there is little insight about the uninformed choices. Most approaches for SVM research and applications focused on tuning three main ratios -mixed kernel ratio, Sigmoid ratio, and Gaussian ratio -to find the best combination for the highest accuracy of the mixed-kernel SVMs, however, the selection of hyperparameters in the SVMs and the kernels may greatly vary for different applications and datasets, and choosing the optimal kernel is critical for a high classification accuracy of the mixed-kernel SVMs [11]. ...
... Nevertheless, little research has been reported in the SVM literature on both of these parameters. In [8], a smaller value of C (=10) allows to ignore points close to the boundary and increases the margin, but a large value of C (=100) decreases the margin. In [15], the performance impact of C with the values of 1 and 16 was discussed. ...
Support Vector Machine (SVM) is a state-of-the-art classification method widely used in science and engineering due to its high accuracy, its ability to deal with high dimensional data, and its flexibility in modeling diverse sources of data. In this paper, we propose an autotuning-based optimization framework to quantify the ranges of hyperparameters in SVMs to identify their optimal choices, and apply the framework to two SVMs with the mixed-kernel between Sigmoid and Gaussian kernels for smart pixel datasets in high energy physics (HEP) and mixed-kernel heterojunction transistors (MKH). Our experimental results show that the optimal selection of hyperparameters in the SVMs and the kernels greatly varies for different applications and datasets, and choosing their optimal choices is critical for a high classification accuracy of the mixed kernel SVMs. Uninformed choices of hyperparameters C and coef0 in the mixed-kernel SVMs result in severely low accuracy, and the proposed framework effectively quantifies the proper ranges for the hyperparameters in the SVMs to identify their optimal choices to achieve the highest accuracy 94.6\% for the HEP application and the highest average accuracy 97.2\% with far less tuning time for the MKH application.
... 30 • Researchers explored various ML techniques, including neural networks, genetic algorithms, and statistical methods, to develop AI systems that could learn from data. 31 • Key developments during this period included the back propagation algorithm for training neural networks, the development of support vector machines (SVMs), and the rise of Bayesian methods in ML. 32 Computational Advances ...
... SVMs examine and group labelled data into classes, split by the widest plane (support vector). They are often employed when there is a nonlinear correlation among data, and, as such, a separation line is not easily recognizable [98]. In their study, Raji et al. identified three white matter regions distinguishing mild TBI from controls using edge density imaging maps. ...
The dawn of Artificial intelligence (AI) in healthcare stands as a milestone in medical innovation. Different medical fields are heavily involved, and pediatric emergency medicine is no exception. We conducted a narrative review structured in two parts. The first part explores the theoretical principles of AI, providing all the necessary background to feel confident with these new state-of-the-art tools. The second part presents an informative analysis of AI models in pediatric emergencies. We examined PubMed and Cochrane Library from inception up to April 2024. Key applications include triage optimization, predictive models for traumatic brain injury assessment, and computerized sepsis prediction systems. In each of these domains, AI models outperformed standard methods. The main barriers to a widespread adoption include technological challenges, but also ethical issues, age-related differences in data interpretation, and the paucity of comprehensive datasets in the pediatric context. Future feasible research directions should address the validation of models through prospective datasets with more numerous sample sizes of patients. Furthermore, our analysis shows that it is essential to tailor AI algorithms to specific medical needs. This requires a close partnership between clinicians and developers. Building a shared knowledge platform is therefore a key step.
