No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
NeuralFactors: A Novel Factor Learning Approach to Generative Modeling of Equities
ResearchGate has not been able to resolve any citations for this publication.
In defined contribution (DC) pension schemes, the regulator usually imposes asset allocation constraints (minimum and maximum limits by asset class) in order to create funds with different risk–return profiles. In this article, we challenge this approach and show that such funds can exhibit erratic risk–return profiles that deviate significantly from the intended design. We propose to replace all minimum and maximum asset allocation constraints by a single risk metric (or measure) that controls risk directly. Thus, funds with different risk–return profiles can be immediately created by adjusting the risk tolerance parameter accordingly. Using data from the Chilean DC pension system, we show that our approach generates funds whose risk–return profiles are consistently ordered according to the intended design, and outperforms funds created by means of asset allocation limits.
Understanding why a model makes a certain prediction can be as crucial as the prediction's accuracy in many applications. However, the highest accuracy for large modern datasets is often achieved by complex models that even experts struggle to interpret, such as ensemble or deep learning models, creating a tension between accuracy and interpretability. In response, various methods have recently been proposed to help users interpret the predictions of complex models, but it is often unclear how these methods are related and when one method is preferable over another. To address this problem, we present a unified framework for interpreting predictions, SHAP (SHapley Additive exPlanations). SHAP assigns each feature an importance value for a particular prediction. Its novel components include: (1) the identification of a new class of additive feature importance measures, and (2) theoretical results showing there is a unique solution in this class with a set of desirable properties. The new class unifies six existing methods, notable because several recent methods in the class lack the proposed desirable properties. Based on insights from this unification, we present new methods that show improved computational performance and/or better consistency with human intuition than previous approaches.
A new recursive algorithm of stochastic approximation type with the averaging of trajectories is investigated. Convergence with probability one is proved for a variety of classical optimization and identification problems. It is also demonstrated for these problems that the proposed algorithm achieves the highest possible rate of convergence.
The goal of portfolio selection is the construction of portfolios that maximize expected returns consistent with individually acceptable levels of risk. Using both historical data and investor expectations of future returns, portfolio selection uses modeling techniques to quantify “expected portfolio returns” and “acceptable levels of portfolio risk,” and provides methods to select an optimal portfolio. It would not be an overstatement to say that modern portfolio theory has revolutionized the world of investment management. Allowing managers to quantify the investment risk and expected return of a portfolio has provided the scientific and objective complement to the subjective art of investment management. More importantly, whereas at one time the focus of portfolio management used to be the risk of individual assets, the theory of portfolio selection has shifted the focus to the risk of the entire portfolio. This theory shows that it is possible to combine risky assets and produce a portfolio whose expected return reflects its components, but with considerably lower risk. In other words, it is possible to construct a portfolio whose risk is smaller than the sum of all its individual parts.
Learning to store information over extended time intervals by recurrent backpropagation takes a very long time, mostly because of insufficient, decaying error backflow. We briefly review Hochreiter's (1991) analysis of this problem, then address it by introducing a novel, efficient, gradient based method called long short-term memory (LSTM). Truncating the gradient where this does not do harm, LSTM can learn to bridge minimal time lags in excess of 1000 discrete-time steps by enforcing constant error flow through constant error carousels within special units. Multiplicative gate units learn to open and close access to the constant error flow. LSTM is local in space and time; its computational complexity per time step and weight is O. 1. Our experiments with artificial data involve local, distributed, real-valued, and noisy pattern representations. In comparisons with real-time recurrent learning, back propagation through time, recurrent cascade correlation, Elman nets, and neural sequence chunking, LSTM leads to many more successful runs, and learns much faster. LSTM also solves complex, artificial long-time-lag tasks that have never been solved by previous recurrent network algorithms.
We propose a new latent factor conditional asset pricing model. Like Kelly, Pruitt, and Su (KPS, 2019), our model allows for latent factors and factor exposures that depend on covariates such as asset characteristics. But, unlike the linearity assumption of KPS, we model factor exposures as a flexible nonlinear function of covariates. Our model retrofits the workhorse unsupervised dimension reduction device from the machine learning literature – autoencoder neural networks – to incorporate information from covariates along with returns themselves. This delivers estimates of nonlinear conditional exposures and the associated latent factors. Furthermore, our machine learning framework imposes the economic restriction of no-arbitrage. Our autoencoder asset pricing model delivers out-of-sample pricing errors that are far smaller (and generally insignificant) compared to other leading factor models.
How can we perform efficient inference and learning in directed probabilistic models, in the presence of continuous latent variables with intractable posterior distributions, and large datasets? We introduce a stochastic variational inference and learning algorithm that scales to large datasets and, under some mild differentiability conditions, even works in the intractable case. Our contributions is two-fold. First, we show that a reparameterization of the variational lower bound yields a lower bound estimator that can be straightforwardly optimized using standard stochastic gradient methods. Second, we show that for i.i.d. datasets with continuous latent variables per datapoint, posterior inference can be made especially efficient by fitting an approximate inference model (also called a recognition model) to the intractable posterior using the proposed lower bound estimator. Theoretical advantages are reflected in experimental results.
The dominant sequence transduction models are based on complex recurrent or convolutional neural networks in an encoder-decoder configuration. The best performing models also connect the encoder and decoder through an attention mechanism. We propose a new simple network architecture, the Transformer, based solely on attention mechanisms, dispensing with recurrence and convolutions entirely. Experiments on two machine translation tasks show these models to be superior in quality while being more parallelizable and requiring significantly less time to train. Our model achieves 28.4 BLEU on the WMT 2014 English-to-German translation task, improving over the existing best results, including ensembles by over 2 BLEU. On the WMT 2014 English-to-French translation task, our model establishes a new single-model state-of-the-art BLEU score of 41.0 after training for 3.5 days on eight GPUs, a small fraction of the training costs of the best models from the literature. We show that the Transformer generalizes well to other tasks by applying it successfully to English constituency parsing both with large and limited training data.
The variational autoencoder (VAE; Kingma, Welling (2014)) is a recently
proposed generative model pairing a top-down generative network with a
bottom-up recognition network which approximates posterior inference. It makes
two strong assumptions about posterior inference: that the posterior
distribution is approximately factorial, and that its parameters can be
approximated with nonlinear regression from the observations. As we show
empirically, the VAE objective can lead to overly simplified representations
which fail to use the network's entire modeling capacity. We present the
importance weighted autoencoder (IWAE), a generative model with the same
architecture as the VAE, but which uses a strictly tighter log-likelihood lower
bound derived from importance weighting. In the IWAE, the recognition network
uses multiple samples to approximate the posterior, giving it increased
flexibility to model complex posteriors which do not fit the VAE modeling
assumptions. We show empirically that IWAEs learn richer latent space
representations than VAEs, leading to improved test log-likelihood on density
estimation benchmarks.
We introduce Adam, an algorithm for first-order gradient-based optimization
of stochastic objective functions. The method is straightforward to implement
and is based an adaptive estimates of lower-order moments of the gradients. The
method is computationally efficient, has little memory requirements and is well
suited for problems that are large in terms of data and/or parameters. The
method is also ap- propriate for non-stationary objectives and problems with
very noisy and/or sparse gradients. The method exhibits invariance to diagonal
rescaling of the gradients by adapting to the geometry of the objective
function. The hyper-parameters have intuitive interpretations and typically
require little tuning. Some connections to related algorithms, on which Adam
was inspired, are discussed. We also analyze the theoretical convergence
properties of the algorithm and provide a regret bound on the convergence rate
that is comparable to the best known results under the online convex
optimization framework. We demonstrate that Adam works well in practice when
experimentally compared to other stochastic optimization methods.
A five-factor model directed at capturing the size, value, profitability, and investment patterns in average stock returns performs better than the three-factor model of Fama and French (FF, 1993). The five-factor model's main problem is its failure to capture the low average returns on small stocks whose returns behave like those of firms that invest a lot despite low profitability. The model's performance is not sensitive to the way its factors are defined. With the addition of profitability and investment factors, the value factor of the FF three-factor model becomes redundant for describing average returns in the sample we examine.
Can we efficiently learn the parameters of directed probabilistic models, in
the presence of continuous latent variables with intractable posterior
distributions, and in case of large datasets? We introduce a novel learning and
approximate inference method that works efficiently, under some mild
conditions, even in the on-line and intractable case. The method involves
optimization of a stochastic objective function that can be straightforwardly
optimized w.r.t. all parameters, using standard gradient-based optimization
methods.
The method does not require the typically expensive sampling loops per
datapoint required for Monte Carlo EM, and all parameter updates correspond to
optimization of the variational lower bound of the marginal likelihood, unlike
the wake-sleep algorithm. These theoretical advantages are reflected in
experimental results.
Traditional econometric models assume a constant one-period forecast variance. To generalize this implausible assumption, a new class of stochastic processes called autoregressive conditional heteroscedastic (ARCH) processes are introduced in this paper. These are mean zero, serially uncorrelated processes with nonconstant variances conditional on the past, but constant unconditional variances. For such processes, the recent past gives information about the one-period forecast variance. A regression model is then introduced with disturbances following an ARCH process. Maximum likelihood estimators are described and a simple scoring iteration formulated. Ordinary least squares maintains its optimality properties in this set-up, but maximum likelihood is more efficient. The relative efficiency is calculated and can be infinite. To test whether the disturbances follow an ARCH process, the Lagrange multiplier procedure is employed. The test is based simply on the autocorrelation of the squared OLS residuals. This model is used to estimate the means and variances of inflation in the U.K. The ARCH effect is found to be significant and the estimated variances increase substantially during the chaotic seventies.
William Falcon and The PyTorch Lightning team
Efficiently Modeling Long Sequences with Structured State Spaces
- Albert Gu
- Karan Goel
- Christopher Re
- Gu Albert
Market Impact Model Handbook
- Barra
- BARRA
Imanol Perez Arribas, and Ben Wood. 2020. A Data-driven Market Simulator for Small Data Environments
- Hans Buehler
- Blanka Horvath
- Terry Lyons
- Buehler Hans
Factor analysis, probabilistic principal component analysis, variational inference, and variational autoencoder: Tutorial and survey
- Benyamin Ghojogh
- Ali Ghodsi
- Fakhri Karray
- Mark Crowley
- Ghojogh Benyamin
ESG Imputation Using DLVMs
- Achintya Gopal
PyTorch: An Imperative Style, High-Performance Deep Learning Library
- Adam Paszke
- Sam Gross
- Francisco Massa
- Adam Lerer
- James Bradbury
- Gregory Chanan
- Trevor Killeen
- Zeming Lin
- Natalia Gimelshein
- Luca Antiga
- Alban Desmaison
- Andreas Kopf
- Edward Yang
- Zachary Devito
- Martin Raison
- Alykhan Tejani
- Sasank Chilamkurthy
- Benoit Steiner
- Lu Fang
- Junjie Bai
- Soumith Chintala
- Paszke Adam
Visualizing Data using t-SNE
- Laurens Van Der Maaten
- Geoffrey Hinton
- van der Maaten Laurens
Accurate Uncertainties for Deep Learning Using Calibrated Regression
- Volodymyr Kuleshov
- Nathan Fenner
- Stefano Ermon
- Kuleshov Volodymyr
MIWAE: Deep Generative Modelling and Imputation of Incomplete Data Sets
- Pierre-Alexandre Mattei
- Jes Frellsen
- Mattei Pierre-Alexandre