No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Hyperopt: A Python library for optimizing the hyperparameters of machine learning algorithms
... This study has also adopted the Python Hyperopt library [72] to deduce optimal hyperparameters, shown in Table 3 for BILSTM, LSTMCNN, DNN, MARS, MLP, KRR and GPR benchmark models. In this way, users can select their models or optimize their parameters simultaneously in Python programming environment. ...
... In this way, users can select their models or optimize their parameters simultaneously in Python programming environment. In fact, Hyperopt operates as a black box system in which the users can provide an evaluation function and parameter space to attain the best values based on the inputs [72]. When selecting an optimization algorithm through the Hyperopt, the distribution over the choice ('Adagrad', 'Adam', 'SGD', and 'RMSprop') is used. ...
... TPE, as named by Bergstra et al. [18], is the technique of using Bayesian optimization heuristics for guiding and speeding up the hyperparameter configuration optimization process. Its innovative approach involves the creation of a binary tree-like model that adeptly maps out the probability distributions for various hyperparameters. ...
... Its innovative approach involves the creation of a binary tree-like model that adeptly maps out the probability distributions for various hyperparameters. This model is particularly adept at navigating the complex landscapes often encountered in high-dimensional spaces, where objective functions can be costly to evaluate, thereby streamlining the optimization trajectory towards optimal solutions with efficiency and precision [18]. Mathematically, TPE optimizes by iteratively selecting hyperparameters based on the following principle: θ * = arg min θ P (Objective better than current best | θ) P (Objective worse than current best | θ) ...
In this paper, we explore the optimization of hyperparameters for the Soft Actor-Critic (SAC) and Proximal Policy Optimization (PPO) algorithms using the Tree-structured Parzen Estimator (TPE) in the context of robotic arm control with seven Degrees of Freedom (DOF). Our results demonstrate a significant enhancement in algorithm performance, TPE improves the success rate of SAC by 10.48 percentage points and PPO by 34.28 percentage points, where models trained for 50K episodes. Furthermore, TPE enables PPO to converge to a reward within 95% of the maximum reward 76% faster than without TPE, which translates to about 40K fewer episodes of training required for optimal performance. Also, this improvement for SAC is 80% faster than without TPE. This study underscores the impact of advanced hyperparameter optimization on the efficiency and success of deep reinforcement learning algorithms in complex robotic tasks.
... The researcher starts with pipeline A in Fig. 1: 1 conversion of raw data from a sequencer into count matrices, 2 frequency threshold filtering for quality control, 3 data normalization, 4 principal component analysis for dimensionality reduction, and 5 correlation analysis to calculate gene co-expression scores. 6 Then, they create graph embedding A that preserves relations between co-expressed genes [32]. By analyzing the difference in the embedding between the "healthy" and the "exposed to a disease" conditions, the domain expert is able to identify key genes that might be associated with that particular disease. ...
... AutoML. Related AutoML solutions [6,21,23,26,27,39,46,47] consider the problem at hand an optimization task by iteratively generating and evaluating pipelines. To evaluate the generated candidate, these solutions require defining an objective loss function. ...
When designing data science (DS) pipelines, end-users can get overwhelmed by the large and growing set of available data preprocessing and modeling techniques. Intelligent discovery assistants (IDAs) and automated machine learning (AutoML) solutions aim to facilitate end-users by (semi-)automating the process. However, they are expensive to compute and yield limited applicability for a wide range of real-world use cases and application domains. This is due to (a) their need to execute thousands of pipelines to get the optimal one, (b) their limited support of DS tasks, e.g., supervised classification or regression only, and a small, static set of available data preprocessing and ML algorithms; and (c) their restriction to quantifiable evaluation processes and metrics, e.g., tenfold cross-validation using the ROC AUC score for classification. To overcome these limitations, we propose a human-in-the-loop approach for the assisteddesignofdatasciencepipelines using previously executed pipelines. Based on a user query, i.e., data and a DS task, our framework outputs a ranked list of pipeline candidates from which the user can choose to execute or modify in real time. To recommend pipelines, it first identifies relevant datasets and pipelines utilizing efficient similarity search. It then ranks the candidate pipelines using multi-objective sorting and takes user interactions into account to improve suggestions over time. In our experimental evaluation, the proposed framework significantly outperforms the state-of-the-art IDA tool and achieves similar predictive performance with state-of-the-art long-running AutoML solutions while being real-time, generic to any evaluation processes and DS tasks, and extensible to new operators.
... This will stop the process if the present loss is lower than 90% of the mean value of the last 10 losses. In contrast to the trial-and-error method, we used grid search [29] to decrease the range of hyper-parameters, and then used the Python Hyperopt library [30] to find the best values. The Hyperopt library provides a parallel solution for model selection and parameter optimization in Python. ...
Data analytics helps basketball teams to create tactics. However, manual data collection and analytics are costly and ineffective. Therefore, we applied a deep bidirectional long short-term memory (BLSTM) and mixture density network (MDN) approach. This model is not only capable of predicting a basketball trajectory based on real data, but it also can generate new trajectory samples. It is an excellent application to help coaches and players decide when and where to shoot. Its structure is particularly suitable for dealing with time series problems. BLSTM receives forward and backward information at the same time, while stacking multiple BLSTMs further increases the learning ability of the model. Combined with BLSTMs, MDN is used to generate a multi-modal distribution of outputs. Thus, the proposed model can, in principle, represent arbitrary conditional probability distributions of output variables. We tested our model with two experiments on three-pointer datasets from NBA SportVu data. In the hit-or-miss classification experiment, the proposed model outperformed other models in terms of the convergence speed and accuracy. In the trajectory generation experiment, eight model-generated trajectories at a given time closely matched real trajectories.
... Using LightGBM to analyze non-indigenous populations or datasets from other regions is a possible direction for future research. Additionally, several automated hyperparameter optimization frameworks have been proposed, including SMAC [57], Hyperopt [58,59], Spearmint [60], Google Vizier [61], AutoTune [62], Ray Tune [63], and Optuna. Among these, Optuna is one of the more recent approaches, which is why this study adopts it. ...
Effectively and equitably allocating medical resources, particularly for minority groups, is a critical issue that warrants further investigation in rural hospitals. Machine learning techniques have gained significant traction and demonstrated strong performance across various fields in recent years. The determination of hyperparameters significantly influences the performance of machine learning models. Thus, this study employs Optuna, a framework specifically designed for optimizing the hyperparameters of machine learning models. Building on prior research, machine learning models with Optuna (MLOPTA) are introduced to forecast diseases of indigenous patients. The numerical results reveal that the designed MLOPTA system can accurately capture the occurrences of specified diseases. Therefore, the MLOPTA system offers a promising approach for disease forecasting. The disease forecasting results can serve as crucial references for allocating hospital resources.
... smaller sets of hyper-parameters over large ranges). In such situations it is advisable to instead use a method such as outlined in , or consider a commercial hyperparameter optimizer such as Optuna (Akiba et al., 2019), Ray Tune (Lai et al., 2018) or HyperOpt (Bergstra et al., 2013). ...
Purpose
Increasing reliance on autonomous systems requires confidence in the accuracies produced from computer vision classification algorithms. Computer vision (CV) for video classification provides phenomenal abilities, but it often suffers from “flickering” of results. Flickering occurs when the CV algorithm switches between declared classes over successive frames. Such behavior causes a loss of trust and confidence in their operations.
Design/methodology/approach
This “flickering” behavior often results from CV algorithms treating successive observations as independent, which ignores the dependence inherent in most videos. Bayesian neural networks are a potential remedy to this issue using Bayesian priors. This research compares a traditional video classification neural network to its Bayesian equivalent based on performance and capabilities. Additionally, this work introduces the concept of smoothing to reduce the opportunities for “flickering.”
Findings
The augmentation of Bayesian layers to CNNs matched with an exponentially decaying weighted average for classifications demonstrates promising benefits in reducing flickering. In the best case the proposed Bayesian CNN model reduces flickering by 67% while maintaining both overall accuracy and class level accuracy.
Research limitations/implications
The training of the Bayesian CNN is more computationally demanding and the requirement to classify frames multiple times reduces resulting framerate. However, for some high surety mission applications this is a tradeoff the decision analyst may be willing to make.
Originality/value
Our research expands on previous efforts by first using a variable number of frames to produce the moving average as well as by using an exponentially decaying moving average in conjunction with Bayesian augmentation.
... Hyperopt [45] with a random grid search algorithm. ...
In this study, we introduce Modular State-based Stackelberg Games (Mod-SbSG), a novel game structure developed for distributed self-learning in modular manufacturing systems. Mod-SbSG enhances cooperative decision-making among self-learning agents within production systems by integrating State-based Potential Games (SbPG) with Stackelberg games. This hierarchical structure assigns more important modules of the manufacturing system a first-mover advantage, while less important modules respond optimally to the leaders' decisions. This decision-making process differs from typical multi-agent learning algorithms in manufacturing systems, where decisions are made simultaneously. We provide convergence guarantees for the novel game structure and design learning algorithms to account for the hierarchical game structure. We further analyse the effects of single-leader/multiple-follower and multiple-leader/multiple-follower scenarios within a Mod-SbSG. To assess its effectiveness, we implement and test Mod-SbSG in an industrial control setting using two laboratory-scale testbeds featuring sequential and serial-parallel processes. The proposed approach delivers promising results compared to the vanilla SbPG, which reduces overflow by 97.1%, and in some cases, prevents overflow entirely. Additionally, it decreases power consumption by 5-13% while satisfying the production demand, which significantly improves potential (global objective) values.
... Figure 2 illustrates the process of nCV and Bayesian optimization. The hyperparameters are determined prior to the training process and are primarily employed to regulate the performance of the selected models [69]. Table 2 presents a comprehensive overview of the hyperparameters used in all chosen algorithms, as well as the enhanced hyperparameters. ...
The interfacial bond strength between the normal strength concrete (NSC) substrate and ultra-high-performance concrete (UHPC) overlays exhibits crucial significance for the longevity and structural integrity of existing or damaged structures. The effectiveness of better repair and retrofitting of NSC is contingent upon the ability of the UHPC-NSC interface to establish a resilient bond with each other under diverse conditions. So, continuous monitoring and assessing the interfacial bond strength with higher prediction accuracy becomes essential for preserving the integrity of NSC structures. Despite numerous empirical formulations, accurately capturing the factors affecting bond strength has remained elusive. In this study, split tensile and slant shear strength of the UHPC-NSC interface are predicted using five data-driven ensemble machine learning algorithms: gradient boosting regressor, adaboost, LightGBM, Xgboost, and Catboost. Notably, gradient boosting regression and Catboost consistently outperformed other models, demonstrating high R2 as 0.963 and 0.827 and low RMSE as 0.351 and 3.137 in testing sets. Nested cross-validation and Bayesian optimization techniques are incorporated for hyperparameter tuning to enhance model robustness. Additionally, the study incorporated Shapley additive explanations plots to reveal the complex relationships between the variables across both local and global scopes, consequently enhancing their viability and interpretability. The results obtained from the SHAP plots unveiled the substantial influence of surface treatment and other factors on bond strength. Further, the study introduced a reverse design approach to elucidate the factors influencing bond strength, guiding future concrete rehabilitation design schemes with a comprehensive understanding of the relationship between input and target features.
... This iterative process continued until the predefined stopping criterion was reached. For the TPE method, we relied on the stochasticity inherent in draws from the models, ensuring diverse candidate suggestions from one iteration to the next while incorporating new recommendations from BO [54]. To obtain a balance between time consumption and precision of the performance metric results, we set the BO stopping criterion to 1e-5. ...
Purpose
More accurate prediction of phenotype traits can increase the success of genomic selection in both plant and animal breeding studies and provide more reliable disease risk prediction in humans. Traditional approaches typically use regression models based on linear assumptions between the genetic markers and the traits of interest. Non-linear models have been considered as an alternative tool for modeling genomic interactions (i.e. non-additive effects) and other subtle non-linear patterns between markers and phenotype. Deep learning has become a state-of-the-art non-linear prediction method for sound, image and language data. However, genomic data is better represented in a tabular format. The existing literature on deep learning for tabular data proposes a wide range of novel architectures and reports successful results on various datasets. Tabular deep learning applications in genome-wide prediction (GWP) are still rare. In this work, we perform an overview of the main families of recent deep learning architectures for tabular data and apply them to multi-trait regression and multi-class classification for GWP on real gene datasets.
Methods
The study involves an extensive overview of recent deep learning architectures for tabular data learning: NODE, TabNet, TabR, TabTransformer, FT-Transformer, AutoInt, GANDALF, SAINT and LassoNet. These architectures are applied to multi-trait GWP. Comprehensive benchmarks of various tabular deep learning methods are conducted to identify best practices and determine their effectiveness compared to traditional methods.
Results
Extensive experimental results on several genomic datasets (three for multi-trait regression and two for multi-class classification) highlight LassoNet as a standout performer, surpassing both other tabular deep learning models and the highly efficient tree based LightGBM method in terms of both best prediction accuracy and computing efficiency.
Conclusion
Through series of evaluations on real-world genomic datasets, the study identifies LassoNet as a standout performer, surpassing decision tree methods like LightGBM and other tabular deep learning architectures in terms of both predictive accuracy and computing efficiency. Moreover, the inherent variable selection property of LassoNet provides a systematic way to find important genetic markers that contribute to phenotype expression.
... We partitioned the data into a training set of roughly 20% of observations and a test set of the remaining 80%. We used the hyperopt Python package [41] for identifying optimal hyperparameters subsequently used for fitting the final model. We evaluate our model on a set of four Period II-III test sets, each consisting of samples taken from each month in the April 2021 -July 2021 time frame. ...
We develop a simulation framework for studying misinformation spread within online social networks that blends agent-based modeling and natural language processing techniques. While many other agent-based simulations exist in this space, questions over their fidelity and generalization to existing networks in part hinder their ability to drive policy-relevant decision making. To partially address these concerns, we create a ’digital clone’ of a known misinformation sharing network by downloading social media histories for over ten thousand of its users. We parse these histories to both extract the structure of the network and model the nuanced ways in which information is shared and spread among its members. Unlike many other agent-based methods in this space, information sharing between users in our framework is sensitive to topic of discussion, user preferences, and online community dynamics. To evaluate the fidelity of our method, we seed our cloned network with a set of posts recorded in the base network and compare propagation dynamics between the two, observing reasonable agreement across the twin networks over a variety of metrics. Lastly, we explore how the cloned network may serve as a flexible, low-cost testbed for misinformation countermeasure evaluation and red teaming analysis. We hope the tools explored here augment existing efforts in the space and unlock new opportunities for misinformation countermeasure evaluation, a field that may become increasingly important to consider with the anticipated rise of misinformation campaigns fueled by generative artificial intelligence.
... Additionally, we discretize the state space into 40. Furthermore, all parameters related to the algorithms and learning approaches pass automated tuning using Hyperopt [41]. Then, the simulation model of the BGLP is publicly accessible through both the MLPro [42] and MLPro-MPPS [43] frameworks. ...
In this paper, we introduce novel gradient-based optimization methods for state-based potential games (SbPGs) within self-learning distributed production systems. SbPGs are recognised for their efficacy in enabling self-optimizing distributed multi-agent systems and offer a proven convergence guarantee, which facilitates collaborative player efforts towards global objectives. Our study strives to replace conventional ad-hoc random exploration-based learning in SbPGs with contemporary gradient-based approaches, which aim for faster convergence and smoother exploration dynamics, thereby shortening training duration while upholding the efficacy of SbPGs. Moreover, we propose three distinct variants for estimating the objective function of gradient-based learning, each developed to suit the unique characteristics of the systems under consideration. To validate our methodology, we apply it to a laboratory testbed, namely Bulk Good Laboratory Plant, which represents a smart and flexible distributed multi-agent production system. The incorporation of gradient-based learning in SbPGs reduces training times and achieves more optimal policies than its baseline.
... Additionally, there is little knowledge regarding how cleaning one type of error impacts how effectively other data errors can be cleaned example, Krishnan and Wu (2019)'s approach can minimize the number of outliers in a dataset or the number of integrity constraints violated. Technically, ML hyperparameter optimizers, such as Python Hyperopt (Bergstra et al. 2013), could be used to configure the data pipelines. However, as Krishnan and Wu (2019) pointed out, these optimizers do not leverage the incremental nature of data cleaning. ...
Machine Learning (ML) is integrated into a growing number of systems for various applications. Because the performance of an ML model is highly dependent on the quality of the data it has been trained on, there is a growing interest in approaches to detect and repair data errors (i.e., data cleaning). Researchers are also exploring how ML can be used for data cleaning; hence creating a dual relationship between ML and data cleaning. To the best of our knowledge, there is no study that comprehensively reviews this relationship. This paper’s objectives are twofold. First, it aims to summarize the latest approaches for data cleaning for ML and ML for data cleaning. Second, it provides future work recommendations. We conduct a systematic literature review of the papers published between 2016 and 2022 inclusively. We identify different types of data cleaning activities with and for ML: feature cleaning, label cleaning, entity matching, outlier detection, imputation, and holistic data cleaning. We summarize the content of 101 papers covering various data cleaning activities and provide 24 future work recommendations. Our review highlights many promising data cleaning techniques that can be further extended. We believe that our review of the literature will help the community develop better approaches to clean data.
... We take one configuration performing quite well for each model on each physical system and compare their performance with and without the NBgE. For each case, the hyperparameters of the encoder, the epochs, the learning rate and the linear decreasing rate 2 are tuned with the help of Tune and HyperOpt [3,19]. In each case, we take 80% of the samples for training (with 10% for validation) and 20% for testing. ...
In the trend of hybrid Artificial Intelligence (AI) techniques, Physic Informed Machine Learning has seen a growing interest. It operates mainly by imposing a data, learning or inductive bias with simulation data, Partial Differential Equations or equivariance and invariance properties. While these models have shown great success on tasks involving one physical domain such as fluid dynamics, existing methods still struggle on tasks with complex multi-physical and multi-domain phenomena. To address this challenge, we propose to leverage Bond Graphs, a multi-physics modeling approach together with Graph Neural Network. We thus propose Neural Bond Graph Encoder (NBgE), a model agnostic physical-informed encoder tailored for multi-physics systems. It provides an unified framework for any multi-physics informed AI with a graph encoder readable for any deep learning model. Our experiments on two challenging multi-domain physical systems - a Direct Current Motor and the Respiratory system - demonstrate the effectiveness of our approach on a multi-variate time series forecasting task.
... [43] making use of the deep learning frameworks, Keras to build the neural network architecture and Optuna to tune the architecture [44,45]. Hyperopt [46] was used to optimize training steps and select the optimal parameters for the best fit of given data without over/under fitting. The program was executed on UAB high-performance computing cluster (Cheaha) to enable efficient training and evaluation of the neural network. ...
Accurate diagnosis for the 400 million people with rare diseases is critical for healthcare decisions, prognosis, understanding disease mechanisms, and identification of treatments. Despite advances in genome sequencing, barriers such as high interpretation costs, diagnostic expertise, throughput associated delays, and uncertain variant classifications persist, with demand exceeding capacity. Many variant classification methods focus narrowly on specific consequences, leading to the use of complex integrative pipelines that often overlook transcript variability and lack prediction transparency. To overcome these limitations, we introduce DITTO. This transparent, transcript-aware machine-learning method demonstrates superior overall performance in accuracy, recall, and precision when compared to existing tools. DITTO is publicly available at https://github.com/uab-cgds-worthey/DITTO
... Therefore, in this paper, the Bayesian optimization (BO) method is used to tune the ML model's hyperparameters (Frazier, 2018). The BO method is a global algorithm based on probability distribution, requiring fewer resources and boasting faster execution speeds (Bergstra et al., 2013). ...
Geotechnical engineering data are usually small-sample and high-dimensional, which brings a lot of challenges in predictive modeling. This paper uses a typical high-dimensional and small-sample swell pressure (Ps) dataset to explore the possibility of using multi-algorithm hybrid ensemble and dimensionality reduction methods to mitigate the uncertainty of soil parameter prediction. Based on six machine learning (ML) algorithms, the base learner pool is constructed, and four ensemble methods, Stacking (SG), Blending (BG), Voting regression (VR), and Feature weight linear stacking (FWL), are used for the multi-algorithm ensemble. Furthermore, the importance of permutation is used for feature dimensionality reduction to mitigate the impact of weakly correlated variables on predictive modeling. The results show that the proposed methods are superior to traditional prediction models and base ML models, where FWL is more suitable for modeling with small-sample datasets, and dimensionality reduction can simplify the data structure and reduce the adverse impact of the small-sample effect, which points the way to feature selection for predictive modeling. Based on the ensemble methods, the feature importance of the five primary factors affecting Ps is the maximum dry density (31.145%), clay fraction (15.876%), swell percent (15.289%), plasticity index (14%), and optimum moisture content (13.69%), the influence of input parameters on Ps is also investigated, in line with the findings of the existing literature.
We report here studies on the use of hydrogen and methane concentrations in exhaled air to predict the concentration of bacteria producing these gases in the large intestine of laboratory rats. Conventional machine learning models were shown to be able to determine exhaled gas concentrations and the concentrations of bacteria producing them in the intestine.
We use a machine learning approach based on the Extreme Gradient Boosting algorithm to analyze the controlling factors of injection induced earthquakes (IIE) in northeast British Columbia (NE BC), particularly hydraulic fracturing induced seismicity within the Montney unconventional resource play. We compile comprehensive datasets incorporating seismological, operational, and stratigraphic features corresponding to various spatiotemporal grids to rank the controlling factors in terms of their importance. Our results show that, in general, the number of hydraulic fracturing stage is the most important factor controlling IIE, followed by cumulative volume of injected fluid and two depth-related features: depth difference between hydraulic fracturing injection and Montney formation and depth difference between Precambrian basement and Montney formation. Out of 168 datasets with varying spatiotemporal grids, the best model performance is achieved for the dataset with spatial grid of 0.1 degree and temporal grid of 10 days. Based on the Shapley Additive exPlanations values, we observe that a threshold of ~ 2200 in the number of hydraulic fracturing stages significantly encourages the occurrence of IIE while the threshold for cumulative volume of injected fluid is ~ 500,000 m³. Moreover, the occurrence of IIE is mostly encouraged when injections occur within ~ 300 m below the top of Montney formation (associated with the maximum thickness of this formation in NE BC) and at ~ 1700 m above the top of Precambrian basement.
The concept of the mechanophore ¹ was only established in the past two decades, during which time their discovery and study have led to new insights and opportunities in fundamental chemical reactivity ² , imaging ³ , drug delivery ⁴ , and molecularly guided material properties 5,6 . Despite many successes in mechanophore design, the plausible design space in the field still far exceeds what has been realized synthetically. For example, ferrocenes are particularly attractive targets as mechanophores due to their combination of high thermal stability and mechanochemical lability. 7,8 However, the handful of demonstrated ferrocene mechanophores is sparse 7–9 in comparison to several thousands of unique ferrocene complexes that have been synthesized. Herein, we report computational, machine learning guided discovery of synthesizable ferrocene mechanophores. We identify over one hundred potential target ferrocene mechanophores with wide-ranging mechanochemical activity and use data-driven computational screening to identify a select number of promising complexes. We highlight design principles to alter mechanochemical activation of ferrocenes, including regio-controlled transition state stabilization through sterically bulky groups and a change in mechanism through non-covalent ligand–ligand interactions. The computational screening is validated experimentally both at the polymer strand level through sonication experiments and at the network level by mechanical testing. These experiments indicate that a computationally discovered ferrocene mechanophore cross-linker leads to greater than 4-fold enhancement in material tearing energy. We expect the computational approach to serve as a blueprint for high-throughput discovery in other families of mechanophores by providing fundamental insights into mechanically coupled reactivity, supporting the elucidation of mechanophore-to-material structure–activity relationships, and leading to polymer networks with new combinations of desired material properties.
Many important managerial decisions are made based on censored observations. Making decisions without adequately handling the censoring leads to inferior outcomes. We investigate the data-driven decision-making problem with an offline dataset containing the feature data and the censored historical data of the variable of interest without the censoring indicators. Without assuming the underlying distribution, we design and leverage {\epsilon}-insensitive operational costs to deal with the unobserved censoring in an offline data-driven fashion. We demonstrate the customization of the {\epsilon}-insensitive operational costs for a newsvendor problem and use such costs to train two representative ML models, including linear regression (LR) models and neural networks (NNs). We derive tight generalization bounds for the custom LR model without regularization (LR-{\epsilon}NVC) and with regularization (LR-{\epsilon}NVC-R), and a high-probability generalization bound for the custom NN (NN-{\epsilon}NVC) trained by stochastic gradient descent. The theoretical results reveal the stability and learnability of LR-{\epsilon}NVC, LR-{\epsilon}NVC-R and NN-{\epsilon}NVC. We conduct extensive numerical experiments to compare LR-{\epsilon}NVC-R and NN-{\epsilon}NVC with two existing approaches, estimate-as-solution (EAS) and integrated estimation and optimization (IEO). The results show that LR-{\epsilon}NVC-R and NN-{\epsilon}NVC outperform both EAS and IEO, with maximum cost savings up to 14.40% and 12.21% compared to the lowest cost generated by the two existing approaches. In addition, LR-{\epsilon}NVC-R's and NN-{\epsilon}NVC's order quantities are statistically significantly closer to the optimal solutions should the underlying distribution be known.
In this paper, we introduce a Physics-Informed Neural Networks (PINNs)-based Topology optimization method that is free from the usual finite element analysis and is applicable for both self-adjoint and non-self-adjoint problems. This approach leverages the continuous formulation of TO along with the continuous adjoint method to obtain sensitivity. Within this approach, the Deep Energy Method (DEM)—a variant of PINN-completely supersedes traditional PDE solution procedures such as a finite-element method (FEM) based solution process. We demonstrate the efficacy of the DEM-based TO framework through three benchmark TO problems: the design of a conduction-based heat sink, a compliant displacement inverter, and a compliant gripper. The results indicate that the DEM-based TO can generate optimal designs comparable to those produced by traditional FEM-based TO methods. Notably, our DEM-based TO process does not rely on FEM discretization for either state solution or sensitivity analysis. During DEM training, we obtain spatial derivatives based on Automatic Differentiation (AD) and dynamic sampling of collocation points, as opposed to the interpolated spatial derivatives from finite element shape functions or a static collocation point set. We demonstrate that, for the DEM method, when using AD to obtain spatial derivatives, an integration point set of fixed positions causes the energy loss function to be not lower-bounded. However, using a dynamically changing integration point set can resolve this issue. Additionally, we explore the impact of incorporating Fourier Feature input embedding to enhance the accuracy of DEM-based state analysis within the TO context. The source codes related to this study are available in the GitHub repository: https://github.com/xzhao399/DEM_TO.git.
This paper investigates the integration of Dance Movement Therapy (DMT) within extended reality (XR) environments, exploring its potential as a non-pharmacological intervention for mental health conditions. The study employed a blend of qualitative evidence synthesis and analyses of primary quantitative data, focusing on the therapeutic implications of Dance in virtual, augmented and mixed realities. Data from wearables and sensors on movements, physiological responses and emotional feedback are analysed using AI/ML algorithms, including Random Forest, SVM, CNNs and RNNs. The research highlighted the importance of data privacy and ethical considerations, emphasising the need for securely storing metadata to ensure user trust and legal compliance. Findings underscored the potential of XR environments like the Metaverse in transforming mental health practices, offering efficient, engaging and effective therapeutic interventions. The study also introduced the novel concept of Physical Intensity Matching and the significance of personalised exercise selection. Despite its ground-breaking insights, the research acknowledged potential biases introduced by wearables and the challenges of ensuring data accuracy. This paper is a foundational exploration into the convergence of DMT, XR and AI, paving the way for future interdisciplinary research in mental health and technology.
In this study, we propose an effective system called RG-Guard that detects potential risks and threats in the use of cryptocurrencies in the metaverse ecosystem. In order for the RG-Guard engine to detect suspicious transactions, Ethereum network transaction information and phishing wallet addresses were collected, and a unique dataset was created after the data preprocessing process. During the data preprocessing process, we manually distinguished the features within the original dataset that contained potential risk indicators. The learning process of the RG-Guard engine in risk classification was achieved by developing a deep learning model based on LSTM + Softmax. In the training process of the model, RG-Guard was optimised for maximum accuracy, and optimum hyperparameters were obtained. The reliability and dataset performance of the preferred LSTM + Softmax model were verified by comparing it with algorithms used in risk classification and detection applications in the literature (Decision tree, XG boost, Random forest and light gradient boosting machine). Accordingly, among the trained models, LSTM + Softmax has the highest accuracy with an F1-score of 0.9950. When a cryptocurrency transaction occurs, RG-Guard extracts the feature vectors of the transaction and assigns a risk level between 1 and 5 to the parameter named βrisk. Since transactions with βrisk > = 3 are labelled as suspicious transactions, RG-Guard blocks this transaction. Thus, thanks to the use of the RG-Guard engine in metaverse applications, it is aimed to easily distinguish potential suspicious transactions from instant transactions. As a result, it is aimed to detect and prevent instant potential suspicious transactions with the RG-Guard engine in money transfers, which have the greatest risk in cryptocurrency transactions and are the target of fraud. The original dataset prepared in the proposed study and the hybrid LSTM + Softmax model developed specifically for the model are expected to contribute to the development of such studies.
Methods for evaluating the fluctuation of texture patterns that are essentially regular have been proposed in the past, but the best method has not been determined. Here, as an attempt at this, we propose a method that applies AI technology (learning EfficientNet, which is widely used as a classification problem solving method) to determine when the fluctuation exceeds the tolerable limit and what the acceptable range is. We also apply this to clarify the tolerable limit of fluctuation in the “Kurume Kasuri” pattern, which is unique to the Chikugo region of Japan, and devise a method to evaluate the fluctuation in real time when weaving the Kasuri and keep it within the acceptable range. This study proposes a method for maintaining a unique faded pattern of woven textiles by utilizing EfficientNet for classification, fine-tuned with Optuna, and LightGBM for predicting subtle misalignments. Our experiments show that EfficientNet achieves high performance in classifying the quality of unique faded patterns in woven textiles. Additionally, LightGBM demonstrates near-perfect accuracy in predicting subtle misalignments within the acceptable range for high-quality faded patterns by controlling the weaving thread tension. Consequently, this method effectively maintains the quality of Kurume Kasuri patterns within the desired criteria.
In an effort to extend the service life of structures under harsh exposure conditions and reduce the carbon dioxide emissions from cement production, researchers are examining the performance of cement concretes with different mineral admixtures. This study focuses on self-compacting concrete (SCC) incorporating high-volume fly ash (HVFA) and silica fume (SF) as supplementary cementitious materials (SCMs). The aim is to investigate the fresh and hardened properties, particularly the compressive strength, of these combinations. SCMs were used to replace varying amounts of cement, with SF showing the greatest improvement in mechanical characteristics across all age groups. Advanced modelling techniques, including Gradient Boosting (GB), Gradient Boosting-Particle Swarm Optimization, Gradient Boosting-Bayesian Optimization, and Gradient Boosting-Differential Evolution (GB-DE), were employed to analyse the strength characteristics of HVFA-SCC. The GB-DE model emerged as the most accurate, with an R² value of 0.995029 and an RMSE of 0.023862. Additionally, an open-source GUI based on GB-DE is introduced to provide engineers with a transparent tool for optimizing concrete mix designs, addressing the opacity of traditional machine learning models.
Waiting Time (WT) stands as a pivotal indicator of the accessibility and equality of ride-hailing service. WT is broken down into two parts: the time taken to match passengers with drivers (matching time), and the time for drivers to pick up passengers (pickup time). Prior research merged the two components, leading to biased results. We aim to individually examine the factors influencing each component, considering demographic attributes of drivers and users, trip characteristics, traffic conditions, and neighborhood built environment (BEs). Using two-week ride-hailing orders collected in Shenzhen, our study reveals that: 1) Trip originating from or arriving at tourist attractions and airports exhibit a shorter matching time but an extended pickup time. 2) Female passengers face bias during the matching process, while female drivers tend to experience prolonged durations in both the matching and pickup phases. 3) Matching time is predominantly determined by trip characteristics, whereas pickup time is more influenced by the neighborhood BEs. 4) The impact of matching time on pickup duration varies across trips, influenced by factors such as passenger gender, idle distance, applied discounts, and additional dispatch fees.
The uncertainty inherent in three-dimensional (3D) mineral prospectivity mapping (MPM) encompasses (a) mineral system conceptual model uncertainty stemming from geological conceptual frameworks, (b) aleatoric uncertainty, attributable to the variability and noise due to multi-source geoscience datasets collection and processing, as well as 3D geological modeling process, and (c) epistemic uncertainty due to predictive algorithm modeling. Quantifying the uncertainty of 3D MPM is a prerequisite for accepting predictive models in exploration. Previous MPM studies were centered on addressing the mineral system conceptual model uncertainty. To the best of our knowledge, few studies quantified the aleatoric and epistemic uncertainties of 3D MPM. This study proposes a novel uncertainty-quantifi-cation machine learning framework to qualify aleatoric and epistemic uncertainties in 3D MPM by the uncertainty-quantification random forest. Another innovation of this framework is utility of the accuracy-rejection curve to provide a quantitative uncertainty threshold for exploration target delineation. The Bayesian hyperparameter optimization tunes the hyperparameters of the uncertainty-quantification random forest automatically. The case study of 3D MPM for exploration target delineation in the Wulong gold district of China demonstrated the practicality of our framework. The aleatoric uncertainty of the 3D MPM indicates that the 3D Early Cretaceous dyke model is the main source of this uncertainty. The 3D exploration targets delineated by the uncertainty-quantification machine learning framework can benefit subsurface gold exploration in the study area.
Many platforms on the web present ranked lists of content to users, typically optimized for engagement-, satisfaction- or retention- driven metrics. Advances in the Learning-to-Rank (LTR) research literature have enabled rapid growth in this application area. Several popular interfaces now include nested lists, where users can enter a 2nd-level feed via any given 1st-level item. Naturally, this has implications for evaluation metrics, objective functions, and the ranking policies we wish to learn. We propose a theoretically grounded method to incorporate 2nd-level feedback into any 1st-level ranking model. Online experiments on a large-scale recommendation system confirm our theoretical findings.
Emotion recognition is vital for augmenting human–computer interactions by integrating emotional contextual information for enhanced communication. Hence, the study presents an intelligent emotion detection system developed utilizing hybrid stacked gated recurrent units (GRU)-recurrent neural network (RNN) deep learning architecture. Integration of GRU with RNN allows the system to make use of both models’ capabilities, making it better at capturing complex emotional patterns and temporal correlations. The EEG signals are investigated in time, frequency, and time–frequency domains, meticulously curated to capture intricate multi-domain patterns. Then, the SMOTE-Tomek method ensures a uniform class distribution, while the PCA technique optimizes features by minimizing data redundancy. A comprehensive experimentation including the well-established emotion datasets: DEAP and AMIGOS, assesses the efficacy of the hybrid stacked GRU and RNN architecture in contrast to 1D convolution neural network, RNN and GRU models. Moreover, the “Hyperopt” technique fine-tunes the model’s hyperparameter, improving the average accuracy by about 3.73%. Hence, results revealed that the hybrid GRU-RNN model demonstrates the most optimal performance with the highest classification accuracies of 99.77% ± 0.13, 99.54% ± 0.16, 99.82% ± 0.14, and 99.68% ± 0.13 for the 3D VAD and liking parameter, respectively. Furthermore, the model’s generalizability is examined using the cross-subject and database analysis on the DEAP and AMIGOS datasets, exhibiting a classification with an average accuracy of about 99.75% ± 0.10 and 99.97% ± 0.03. Obtained results when compared with the existing methods in literature demonstrate superior performance, highlighting potential in emotion recognition.
We conduct an empirical study of machine learning functionalities provided by major cloud service providers, which we call machine learning clouds. Machine learning clouds hold the promise of hiding all the sophistication of running large-scale machine learning: Instead of specifying how to run a machine learning task, users only specify what machine learning task to run and the cloud figures out the rest. Raising the level of abstraction, however, rarely comes free—a performance penalty is possible. How good, then, are current machine learning clouds on real-world machine learning workloads? We study this question by conducting benchmark on the mainstream machine learning clouds. Since these platforms continue to innovate, our benchmark tries to reflect their evolvement. Concretely, this paper consists of two sub-benchmarks—mlbench and automlbench. When we first started this work in 2016, only two cloud platforms provide machine learning services and limited themselves to model training and simple hyper-parameter tuning. We then focus on binary classification problems and present mlbench, a novel benchmark constructed by harvesting datasets from Kaggle competitions. We then compare the performance of the top winning code available from Kaggle with that of running machine learning clouds from both Azure and Amazon on mlbench. In the recent few years, more cloud providers support machine learning and include automatic machine learning (AutoML) techniques in their machine learning clouds. Their AutoML services can ease manual tuning on the whole machine learning pipeline, including but not limited to data preprocessing, feature selection, model selection, hyper-parameter, and model ensemble. To reflect these advancements, we design automlbench to assess the AutoML performance of four machine learning clouds using different kinds of workloads. Our comparative study reveals the strength and weakness of existing machine learning clouds and points out potential future directions for improvement.
Slope failures lead to catastrophic consequences in numerous countries, so accurate slope stability evaluation is critical in geological disaster prevention and control. In this study, the type and characteristics of slope protection structure disease were determined through the field investigation of an expansive soil area, and this information is incorporated into the numerical simulations and works to develop prediction models of slope stability. Four base machine learning (ML) methods are used to capture the relationship between protection structure diseases and factor of safety (FOS). Further, with the help of stacked generalization (SG), four ML models are combined, and the final SG model is used to predict the FOS. The results show that ML methods can effectively utilize this information and achieve excellent prediction results. The proposed SG model exhibits superior accuracy and robustness in predicting FOS compared to other ML methods. With FOS as the regression variable, the main feature contributions are slope height (37.05%) > slip distance of retaining wall (25.43%) > expansive force (18.03%) > slope gradient (12.00%); the coupling relationship among features is also captured by the proposed model. It is concluded that the SG method is particularly suitable for slope stability modeling under small sample conditions. Besides, the SG-based model effectively captures the impact of protection structure diseases on slope stability, enhances the interpretability of the ML model, and provides a reference for the maintenance and repair of the protection structure.
Developing a spiking neural network architecture that could prospectively be trained on energy-efficient neuromorphic hardware to solve various data analysis tasks requires satisfying the limitations of prospective analog or digital hardware, i.e., local learning and limited numbers of connections, respectively. In this work, we compare two methods of connectivity reduction that are applicable to spiking networks with local plasticity; instead of a large fully-connected network (which is used as the baseline for comparison), we employ either an ensemble of independent small networks or a network with probabilistic sparse connectivity. We evaluate both of these methods with a three-layer spiking neural network, which are applied to handwritten and spoken digit classification tasks using two memristive plasticity models and the classical spike time-dependent plasticity (STDP) rule. Both methods achieve an F1-score of 0.93–0.95 on the handwritten digits recognition task and 0.85–0.93 on the spoken digits recognition task. Applying a combination of both methods made it possible to obtain highly accurate models while reducing the number of connections by more than three times compared to the basic model.
Neural networks (NNs) have been used extensively in surface water prediction tasks due to computing algorithm improvements and data accumulation. An essential step in developing an NN is the hyperparameter selection. In practice, it is common to manually determine hyperparameters in the studies of NNs in water resources tasks. This may result in considerable randomness and require significant computation time; therefore, hyperparameter optimization (HPO) is essential. This study adopted five representatives of the HPO techniques in the surface water quality prediction tasks, including the grid sampling (GS), random search (RS), genetic algorithm (GA), Bayesian optimization (BO) based on the Gaussian process (GP), and the tree Parzen estimator (TPE). For the evaluation of these techniques, this study proposed a method: first, the optimal hyperparameter value sets achieved by GS were regarded as the benchmark; then, the other HPO techniques were evaluated and compared with the benchmark in convergence, optimization orientation, and consistency of the optimized values. The results indicated that the TPE-based BO algorithm was recommended because it yielded stable convergence, reasonable optimization orientation, and the highest consistency rates with the benchmark values. The optimization consistency rates via TPE for the hyperparameters hidden layers, hidden dimension, learning rate, and batch size were 86.7%, 73.3%, 73.3%, and 80.0%, respectively. Unlike the evaluation of HPO techniques directly based on the prediction performance of the optimized NN in a single HPO test, the proposed benchmark-based HPO evaluation approach is feasible and robust.
ResearchGate has not been able to resolve any references for this publication.