No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Reducing Return Volatility in Neural Network-Based Asset Allocation via Formal Verification and Certified Training
ResearchGate has not been able to resolve any citations for this publication.
Neural networks are very successful at detecting patterns in noisy data, and have become the technology of choice in many fields. However, their usefulness is hampered by their susceptibility to adversarial attacks . Recently, many methods for measuring and improving a network’s robustness to adversarial perturbations have been proposed, and this growing body of research has given rise to numerous explicit or implicit notions of robustness. Connections between these notions are often subtle, and a systematic comparison between them is missing in the literature. In this paper we begin addressing this gap, by setting up general principles for the empirical analysis and evaluation of a network’s robustness as a mathematical property—during the network’s training phase, its verification, and after its deployment. We then apply these principles and conduct a case study that showcases the practical benefits of our general approach.
We introduce an efficient method for the complete verification of ReLU-based feed-forward neural networks. The method implements branching on the ReLU states on the basis of a notion of dependency between the nodes. This results in dividing the original verification problem into a set of sub-problems whose MILP formulations require fewer integrality constraints. We evaluate the method on all of the ReLU-based fully connected networks from the first competition for neural network verification. The experimental results obtained show 145% performance gains over the present state-of-the-art in complete verification.
Despite having high accuracy, neural nets have been shown to be susceptible to adversarial examples, where a small perturbation to an input can cause it to become mislabeled. We propose metrics for measuring the robustness of a neural net and devise a novel algorithm for approximating these metrics based on an encoding of robustness as a linear program. We show how our metrics can be used to evaluate the robustness of deep neural nets with experiments on the MNIST and CIFAR-10 datasets. Our algorithm generates more informative estimates of robustness metrics compared to estimates based on existing algorithms. Furthermore, we show how existing approaches to improving robustness "overfit" to adversarial examples generated using a specific algorithm. Finally, we show that our techniques can be used to additionally improve neural net robustness both according to the metrics that we propose, but also according to previously proposed metrics.
Several machine learning models, including neural networks, consistently
misclassify adversarial examples---inputs formed by applying small but
intentionally worst-case perturbations to examples from the dataset, such that
the perturbed input results in the model outputting an incorrect answer with
high confidence. Early attempts at explaining this phenomenon focused on
nonlinearity and overfitting. We argue instead that the primary cause of neural
networks' vulnerability to adversarial perturbation is their linear nature.
This explanation is supported by new quantitative results while giving the
first explanation of the most intriguing fact about them: their generalization
across architectures and training sets. Moreover, this view yields a simple and
fast method of generating adversarial examples. Using this approach to provide
examples for adversarial training, we reduce the test set error of a maxout
network on the MNIST dataset.
Deep neural networks are highly expressive models that have recently achieved
state of the art performance on speech and visual recognition tasks. While
their expressiveness is the reason they succeed, it also causes them to learn
uninterpretable solutions that could have counter-intuitive properties. In this
paper we report two such properties.
First, we find that there is no distinction between individual high level
units and random linear combinations of high level units, according to various
methods of unit analysis. It suggests that it is the space, rather than the
individual units, that contains of the semantic information in the high layers
of neural networks.
Second, we find that deep neural networks learn input-output mappings that
are fairly discontinuous to a significant extend. Specifically, we find that we
can cause the network to misclassify an image by applying a certain
imperceptible perturbation, which is found by maximizing the network's
prediction error. In addition, the specific nature of these perturbations is
not a random artifact of learning: the same perturbation can cause a different
network, that was trained on a different subset of the dataset, to misclassify
the same input.
Transformers based on attention mechanisms exhibit vulnerability to adversarial examples, posing a substantial threat to the security of their applications. Aiming to solve this problem, the concept of robustness certification is introduced to formally ascertain the presence of any adversarial example within a specified region surrounding a given sample. However, prior works have neglected the dependencies among inputs of softmax (the most complex function in attention mechanisms) during linear relaxations. This oversight has consequently led to imprecise certification results. In this work, we introduce GaLileo, a general linear relaxation framework designed to certify the robustness of Transformers. GaLileo effectively surmounts the trade-off between precision and efficiency in robustness certification through our innovative n-dimensional relaxation approach. Notably, our relaxation technique represents a pioneering effort as the first linear relaxation for n-dimensional functions such as softmax. Our novel approach successfully transcends the challenges posed by the curse of dimensionality inherent in linear relaxations, thereby enhancing linear bounds by incorporating input dependencies. Our evaluations encompassed a thorough analysis utilizing the SST and Yelp datasets along with diverse Transformers of different depths and widths. The experimental results demonstrate that, as compared to the baseline method CROWN-BaF, GaLileo achieves up to 3.24 times larger certified radii while requiring similar running times. Additionally, GaLileo successfully attains certification for Transformers' robustness against multi-word lp perturbations, marking a notable accomplishment in this field.
We introduce the problem of training neural networks such that they are robust against a class of smooth intensity perturbations modelled by bias fields. We first develop an approach towards this goal based on a state-of-the-art robust training method utilising Interval Bound Propagation (IBP). We analyse the resulting algorithm and observe that IBP often produces very loose bounds for bias field perturbations, which may be detrimental to training. We then propose an alternative approach based on Symbolic Interval Propagation (SIP), which usually results in significantly tighter bounds than IBP. We present ROBNET, a tool implementing these approaches for bias field robust training. In experiments networks trained with the SIP-based approach achieved up to 31% higher certified robustness while also maintaining a better accuracy than networks trained with the IBP approach.
Time-series data arises in many real-world applications (e.g., mobile health) and deep neural networks (DNNs) have shown great success in solving them. Despite their success, little is known about their robustness to adversarial attacks. In this paper, we propose a novel adversarial framework referred to as Time-Series Attacks via STATistical Features (TSA-STAT). To address the unique challenges of time-series domain, TSA-STAT employs constraints on statistical features of the time-series data to construct adversarial examples. Optimized polynomial transformations are used to create attacks that are more effective (in terms of successfully fooling DNNs) than those based on additive perturbations. We also provide certified bounds on the norm of the statistical features for constructing adversarial examples. Our experiments on diverse real-world benchmark datasets show the effectiveness of TSA-STAT in fooling DNNs for time-series domain and in improving their robustness.
The COVID-19 pandemic was a global event that disrupted social structures, reshaped interpersonal behaviors, and challenged institutional trust on an unprecedented scale. This study explores how the pandemic has altered social norms, communication patterns, and relationships, as well as the public's confidence in institutions such as governments, healthcare systems, and media. Drawing on qualitative and quantitative methods, including surveys, interviews, and case studies, the research examines key behavioral changes, such as the rise of virtual interactions, increased individualism, and shifts in community engagement.
Additionally, the study investigates how these changes intersect with perceptions of institutional transparency, effectiveness, and equity during and after the crisis. By analyzing diverse demographic and geographic groups, the research seeks to identify variations in experiences and attitudes, offering a nuanced understanding of the post-pandemic social landscape. Findings aim to inform strategies for rebuilding trust, fostering resilience, and adapting to new societal norms in a rapidly evolving world.
This paper contributes a new machine learning solution for stock movement prediction, which aims to predict whether the price of a stock will be up or down in the near future. The key novelty is that we propose to employ adversarial training to improve the generalization of a neural network prediction model. The rationality of adversarial training here is that the input features to stock prediction are typically based on stock price, which is essentially a stochastic variable and continuously changed with time by nature. As such, normal training with static price-based features (e.g. the close price) can easily overfit the data, being insufficient to obtain reliable models. To address this problem, we propose to add perturbations to simulate the stochasticity of price variable, and train the model to work well under small yet intentional perturbations. Extensive experiments on two real-world stock data show that our method outperforms the state-of-the-art solution [Xu and Cohen, 2018] with 3.11% relative improvements on average w.r.t. accuracy, validating the usefulness of adversarial training for stock prediction task.
Identification of favorable trading opportunities is crucial in financial markets as it could bring additional increment in profits. Backtesting is one of the process which consists of analyzing the past price movements and predict the best possible trading strategy. Current approaches focus only on exact matching of market values. In order to overcome deficiencies in current approaches, this research proposes a fuzzy logic based approximate matching approach by using the technical indicators and trading rules. The evaluation results demonstrate that finding approximate matching places for a particular trading strategy has a positive contribution to successful trading and an average trader can be successful in trading buy following those fuzzy logic based trading strategies.
For most deep learning practitioners, sequence modeling is synonymous with recurrent networks. Yet recent results indicate that convolutional architectures can outperform recurrent networks on tasks such as audio synthesis and machine translation. Given a new sequence modeling task or dataset, which architecture should one use? We conduct a systematic evaluation of generic convolutional and recurrent architectures for sequence modeling. The models are evaluated across a broad range of standard tasks that are commonly used to benchmark recurrent networks. Our results indicate that a simple convolutional architecture outperforms canonical recurrent networks such as LSTMs across a diverse range of tasks and datasets, while demonstrating longer effective memory. We conclude that the common association between sequence modeling and recurrent networks should be reconsidered, and convolutional networks should be regarded as a natural starting point for sequence modeling tasks.
Several machine learning models, including neural networks, consistently mis- classify adversarial examples—inputs formed by applying small but intentionally worst-case perturbations to examples from the dataset, such that the perturbed in- put results in the model outputting an incorrect answer with high confidence. Early attempts at explaining this phenomenon focused on nonlinearity and overfitting. We argue instead that the primary cause of neural networks' vulnerability to ad- versarial perturbation is their linear nature. This explanation is supported by new quantitative results while giving the first explanation of the most intriguing fact about them: their generalization across architectures and training sets. Moreover, this view yields a simple and fast method of generating adversarial examples. Us- ing this approach to provide examples for adversarial training, we reduce the test set error of a maxout network on the MNIST dataset.
Many investment firms and portfolio managers rely on backtests (ie, simulations of performance based on historical market data) to select investment strategies and allocate capital. Standard statistical techniques designed to prevent regression overfitting, such as hold-out, tend to be unreliable and inaccurate in the context of investment backtests. We propose a general framework to assess the probability of backtest overfitting (PBO). We illustrate this framework with specific generic, model-free and non-parametric implementations in the context of investment simulations; we call these implementations combinatorially symmetric cross-validation (CSCV). We show that CSCV produces reasonable estimates of PBO for several useful examples.
Exchange-traded funds are the most popular examples of a category of financial instrument that might be characterized as a “portfolio-in-a-single-share”. In addition to open-end exchange-traded funds based on the SPDR structure, closed-end funds, HOLDRs, exchange-traded notes and even FOLIOs sometimes compete in the portfolio-as-a-share market. While the products all feature multiple instruments in a single transaction, these products and structures have distinct differences in tax treatment, trading costs and convenience. The open-end exchange-traded fund structure offers unique opportunities for increased shareholder efficiency and the delivery of actively managed portfolios in a tax-efficient format. The genesis of exchange-traded funds was in portfolio or program trading and its cousin, index arbitrage.
We evaluate the out-of-sample performance of the sample-based mean-variance model, and its extensions designed to reduce estimation
error, relative to the naive 1/N portfolio. Of the 14 models we evaluate across seven empirical datasets, none is consistently better than the 1/N rule in terms of Sharpe ratio, certainty-equivalent return, or turnover, which indicates that, out of sample, the gain from
optimal diversification is more than offset by estimation error. Based on parameters calibrated to the US equity market, our
analytical results and simulations show that the estimation window needed for the sample-based mean-variance strategy and
its extensions to outperform the 1/N benchmark is around 3000 months for a portfolio with 25 assets and about 6000 months for a portfolio with 50 assets. This
suggests that there are still many “miles to go” before the gains promised by optimal portfolio choice can actually be realized
out of sample.
Attacks Which Do Not Kill Training Make Adversarial Learning Stronger
- J Zhang
- X Xu
- B Han
- G Niu
- L Cui
- M Sugiyama
- M Kankanhalli
- Zhang J.
RobustBench: a standardized adversarial robustness benchmark
- F M Andriushchenko
- V Sehwag
- E Debenedetti
- N Flammarion
- M Chiang
- P Mittal
- M Hein
- Andriushchenko M.
Certified Adversarial Robustness via Randomized Smoothing
- J Cohen
- E Rosenfeld
- J Kolter
- Cohen J.
On the Effectiveness of Interval Bound Propagation for Training Verifiably Robust Models
- Gowal K Dvijotham
- R Stanforth
- R Bunel
- C Qin
- J Uesato R. Arandjelovic
- T Mann
- P Kohli
Efficient Formal Safety Analysis of Neural Networks
- Pei J Whitehouse
- J Yang
- S Jana
Certifying Geometric Robustness of Neural Networks
- M M Balunovic
- G Baader
- T Singh
- M Gehr
- Vechev
OODRobustBench: a Benchmark and Large-Scale Analysis of Adversarial Robustness under Distribution Shift
- L Li
- Y Wang
- C Sitawarin
- M Spratling
- Li L.
Bias Field Robustness Verification of Large Neural Image Classifiers
- P Henriksen
- K Hammernik
- D Rueckert
- A Lomuscio
Improved Branch and Bound for Neural Network Verification via Lagrangian Decomposition
- A. De Palma
- R Bunel
- A Desmaison
- K Dvijotham
- P Kohli
- P Torr
- P Kumar
Robust Multivariate Time-Series Forecasting: Adversarial Attacks and Defense Mechanisms
- L Liu
- Y Park
- T Hoang
- H Hasson
- L Huan
- Liu L.
Design Testing and Optimization of Trading Systems
- R Pardo
Learning Fast and Slow for Online Time Series Forecasting
- Q Pham
- C Liu
- D Sahoo
- S Hoi
Robustness Verification for Transformers
- Z Shi
- H Zhang
- K-W Chang
- M Huang
- C-J Hsieh
Beta-crown: Efficient bound propagation with per-neuron split constraints for complete and incomplete neural network verification
- S Wang
- H Zhang
- K Xu
- X Lin
- S Jana
- C Hsieh
- J Kolter
- Wang S.
Convex Bounds on the Softmax Function with Applications to Robustness Verification
- D Wei
- H Wu
- M Wu
- P-Y. Chen
- C Barrett
- E Farchi
- Wei D.
Beyond the Single Neuron Convex Barrier for Neural Network Certification
- G Singh
- R Ganvir
- M Püschel
- M Vechev
- Singh G.
Deep Reinforcement Learning for Portfolio Optimization using Latent Feature State Space (LFSS) Module
- K Yashaswi
Robustness May Be at Odds with Accuracy
- D Tsipras
- S Santurkar
- L Engstrom
- A Turner
- A Madry
- Tsipras D.