NONLINEAR FORECASTING WITH MANY PREDICTORS
BY NEURAL NETWORK FACTOR MODELS
DEPARTMENT OF STATISTICS,LONDON SCHOOL OF ECONOMICS
•This study proposes a nonlinear forecasting tech-
nique based on an improved factor model with
two neural network extensions. This model
would be able to capture both non-linearity and
non-normality of a high-dimensional dataset.
•Speciﬁcation (architecture) of the neural network
Factor model is determined on the basis of sta-
tistical inference and special emphasis is given to
•Linear factor models can be represented as a spe-
cial case of this neural network factor model.It
means that, if there is no non-linearity between
variables, it will work like a linear model.
Forecasting with factor models are a two-step process:
•Factor Estimation, which summarizes the infor-
mation contained in a large data set in a small
number of factors.
•Forecasting Equation, which is the prediction of
the variable of interest by using common factors.
one of the X
=λ ft+1|t+εt+1 (2)
Common factors and the idiosyncratic component can
be forecast simultaneously or separately.
CONTRIBUTION & FORMULATION
Figure 1: The standard auto-associative neural network archi-
tecture for nonlinear PCA (combination of two feed-forward NNs)
The ﬁrst extension proposes a NLPCA (neural network
principal component analysis) as an alternative for fac-
tor estimation, which allows the factors to have a non-
linear relationship to the input variables. NLPCA non-
linearly generalizes the classical PCA method by a non-
linear mapping from data to factors. Both neural net-
work parameters and unobservable factors (f) can be op-
timised simultaneously to minimise the reconstruction
X−X, MSE =E(|| ˆ
•Second extension is a nonlinear factor augmented
forecasting equation based on a single hidden
layer feed-forward neural network model which
can be built in a similar fashion as a statistical
•A neural network model can be deﬁned as:
yt=G(xt;ψ) + εt=α0˜xt+
ixt−βi) + εt
•The function F(˜ω0
ixt−βi), often called the activa-
tion function, is a logistic function.
Figure 2: Artiﬁcial Neuron conﬁguration
 J. H. Stock and M. W. Watson. Forecasting using principal components from a large number of predictors. American Statistical Association,
 M. Forni et al. The generalized dynamic factor model: Estimation and forecasting. American Statistical Association, 100:830–840, 2005.
 M. C. Medeiros and T. Terasvirta. Building neural network models for time series: A statistical approach. J of Forecasting, 25:49–75, 2006.
 C. M. Kuan and H. White. Artiﬁcial neural networks: An econometric perspectiv. Econometric Reviews, 13:1–91, 1994.
 M. Deistler and E. Hamann. Identiﬁcation of factor models for forecasting returns. Financial Econometrics, 3(2):256–281, 2005.
 A. N. Gorban and B. M. Kegl(Eds.). Principal Manifolds for Data Visualization and Dimension Reduction. Springer, 2008.
 M. A. Kramer. Nonlinear principal component analysis using autoassociative neural networks. AIChE, 37:233–243, 1991.
Out-of-sample forecast evaluation results based on dif-
ferent criteria (RMSE, Hit-Rate and Theil) showed that
the proposed neural network factor model (NNFM)
signiﬁcantly outperformed linear factor model and
Phone +44 (0)7737842985
NLPCA ON FINANCIAL RETURNS
Figure 3: Nonlinear PCA can describe the inherent structure of the data by a curved subspace.
CONSTRUCTION OF FNN
Three stages of model building:
by linearizing the model (approximate NN model
by a polynomial of sufﬁciently high order) and
applying well-known techniques of linear vari-
able selection to this approximation.
Estimate the parameters by maximum likelihood,
making use of the normality assumptions made
•Determining the number of hidden units (neu-
Applying Lagrange multiplier type tests. One
possibility is to begin with a small model and se-
quentially add hidden units to the model.
Financial returns present special features and share the
following stylised facts: comovements, non-linearity,
non-gausianity (skewness and heavy tails) and lever-
age effect, which makes the modelling of this variable
Figure 4: monthly return observations of the 52 companies in