ArticlePDF Available

Mixed Frequency Data Sampling Regression Models: The R Package midasr

Authors:

Abstract and Figures

When modeling economic relationships it is increasingly common to encounter data sampled at different frequencies. We introduce the R package midasr which enables estimating regression models with variables sampled at different frequencies within a MIDAS regression framework put forward in work by Ghysels, Santa-Clara, and Valkanov (2002). In this article we define a general autoregressive MIDAS regression model with multiple variables of different frequencies and show how it can be specified using the familiar R formula interface and estimated using various optimization methods chosen by the researcher. We discuss how to check the validity of the estimated model both in terms of numerical convergence and statistical adequacy of a chosen regression specification, how to perform model selection based on a information criterion, how to assess forecasting accuracy of the MIDAS regression model and how to obtain a forecast aggregation of different MIDAS regression models. We illustrate the capabilities of the package with a simulated MIDAS regression model and give two empirical examples of application of MIDAS regression.
Content may be subject to copyright.
Mixed Frequency Data Sampling Regression
Models: the RPackage midasr
Eric Ghysels
University of North Carolina
Virmantas Kvedaras
Vilnius University
Vaidotas Zemlys
Vilnius University
Abstract
Regression models involving data sampled at different frequencies are of general in-
terest. In this document the Rpackage midasr is described within a MIDAS regression
framework with functional constraints on parameters put forward in work by Ghysels,
Santa-Clara, and Valkanov (2002), Ghysels, Santa-Clara, and Valkanov (2006a) and An-
dreou, Ghysels, and Kourtellos (2010).
Keywords: MIDAS, specification test.
1. Introduction
Regression models involving data sampled at different frequencies are of general interest. In
this document we introduce a Rpackage midasr for the regression modeling with the mixed
frequency data based on a framework put forward in recent work by Ghysels et al. (2002),
Ghysels et al. (2006a) and Andreou et al. (2010) using so called MIDAS, meaning Mi(xed)
Da(ta) S(ampling), regressions.1
In a general framework of regressions with functional constraints on parameters, the midasr
package not only provides similar functionality within a standard Rframework of the model
specification comparable to that available in the usual functions lm or nls, but also deals
with an extended model specification analysis for MIDAS regressions.
Several recent surveys on the topic of MIDAS are worth mentioning at the outset. They
are: Andreou, Ghysels, and Kourtellos (2011) who review more extensively some of the ma-
terial summarized in this document, Armesto, Engemann, and Owyang (2010) who provide a
very simple introduction to MIDAS regressions and finally Ghysels and Valkanov (2012) who
discuss volatility models and mixed data sampling.
Econometric analysis of MIDAS regressions appears in Ghysels, Sinko, and Valkanov (2006b),
Andreou et al. (2010), Bai, Ghysels, and Wright (2012), Kvedaras and Raˇckauskas (2010),
Rodriguez and Puggioni (2010), Wohlrabe (2009), among others.
MIDAS regression can also be viewed as a reduced form representation of the linear projection
which emerges from a state space model approach - by reduced form we mean that the MIDAS
regression does not require the specification of a full state space system of equations. Bai et al.
(2012) show that in some cases the MIDAS regression is an exact representation of the Kalman
1Ghysels (2013) also developed a package for MATLAB which deals with the estimation and information
criteria-based specification of MIDAS regressions as well as forecasting and nowcasting of slow frequency series.
2R package midasr
filter, in other cases it involves approximation errors which are typically small. The Kalman
filter, while clearly optimal as far as linear projections goes, has several disadvantages (1) it
is more prone to specification errors as a full system of measurement and state equations is
required and as a consequence (2) requires a lot more parameters, which in turn results in (3)
computational complexities which often limit the scope of applications. In contrast, MIDAS
regressions - combined with forecast combination schemes if large data sets are involved (see
Andreou, Ghysels, and Kourtellos (2013)) are computationally easy to implement and more
prone to specification errors.
The key feature of the package is its flexibility in terms of the model formulation and estima-
tion, which allows for the:
estimation of regression models with its parameters defined (restricted) by a certain
functional constraint, which can be selected from a standard list or can be customized
using a standard Rfunction;
parsimonious aggregation-linked restrictions (as e.g. in Ghysels 2013) as a special case;
estimation of MIDAS models with many variables and (numerous) different frequencies;
constrained, partially constrained, or unconstrained estimation of the model;
various mixtures of restrictions/weighting schemes and also lag orders as they can be
specific to each series;
statistical testing for the adequacy of the model specification and the imposed functional
constraint;
information criteria and testing-based selection of models;
forecasting and nowcasting functionality, including various forecast combinations.
Suppose {yt, t Z}is a univariate process observed at low frequency. Lags of the process
are denoted by Byt=yt1,where Bis the low frequency lag operator. A MIDAS regression
involves linear projections using stochastic processes {x(i)
τ, τ Z}, i = 0, . . . , h, observed at
a higher frequency, i.e. for each low frequency period t=t0we observe the process x(i)
τat
miNhigh frequency periods τ= (t01)mi+ 1, . . . , t0mi. Throughout the article we
represent i-th high frequency period τin terms of low frequency period tas τ= (t1)mi+j,
j= 1, . . . , mi. Note that this notation does not exclude the case mi= 1. In that case the
high frequency process x(i)
τis observed at the same frequency as the low frequency process yt.
However we require that mi1,such that the process ytis observed at the lowest frequency.
Lags of the processes x(i)
τare denoted by Lx(i)xτ=xτ1,where Lis the high frequency lag
operator, which operates on the lag irrespective of the frequency of the process.
The package deals with any specification of mixed-frequency regression model which can be
represented as
ytα1yt1 · · · αpytp=
h
X
i=0
ki
X
j=0
β(i)
jx(i)
tmij+εt,(1)
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 3
where we require
E(εt|yt1, . . . , ytp, x(0)
tm0, . . . , x(0)
tm0ki, . . . , x(h)
tmh, . . . , x(h)
tmhkh)=0,
so that the equation (1) is identified as a projection equation.
The model stated in the equation (1) can be estimated in the usual time series regression
fashion or using a Bayesian approach. However, the number of parameters in this model d=
p+Ph
i=0 kican be very large in terms of the number nof available observations of yt2. Since
the estimation of the model can easily become infeasible, whenever either larger differences in
frequencies or more variables and/or higher lag orders prevail, Ghysels et al. (2002) introduced
a sufficiently flexible parametric restriction to be imposed on the original parameters,
β(i)
j=fi(γi, j), j = 0, . . . , ki,γi= (γ(i)
1, . . . , γ(i)
qi), qiN.(2)
This approach greatly reduces the number of parameters to be estimated, from dto q=
Phi
i=0 qi,which is assumed to be always considerably less than the number of observations
available at the lowest frequency. This gain is offset by the fact that (1) becomes non-linear
model, however if the parameters of an underlying data generating process did follow a cer-
tain functional constraint which is perfectly or well approximated by a constraint function
chosen by a researcher, then significant efficiency gains could be achieved from the imposed
constraints. Figure 1plots the out-of-sample prediction precision (left figure) and the pa-
rameter estimation precision (right figure) in an unconstrained and constrained simple model
with correct and approximate restrictions (see Appendix A for details).
Mean squared error Distance from true values
1.0
1.2
1.4
1.6
0.2
0.4
0.6
0.8
250 500 750 1000 250 500 750 1000
Sample size
Constraint
Unrestricted
Correct
Incorrect
Figure 1: Mean squared error and distance to true coefficient values
As can be seen, even an incorrect constraint might be useful whenever the number of de-
grees of freedom in an unconstrained model is low and, consequently, one cannot rely on the
2In the MIDAS literature it is common to have kimiand mican be large, for example monthly versus
daily data
4R package midasr
large sample properties of unconstrained estimators. Furthermore, this approach seems to be
necessary whenever estimation is simply infeasible because of the lack of degrees of freedom.
2. Theory
The model (1) can be rewritten in a more compact form:
α(B)yt=β(L)0xt,0+εt,(3)
where α(z)=1Pp
j=1 αjzjand
xt,0: = x(0)
tm0, . . . , x(i)
tmi, . . . , x(h)
tmh0,
β(z) =
k
X
j=0
βjzj,βj=β(0)
j, . . . , β(i)
j, . . . , β(h)
j0,
Ljxt,0: = xt,j =Ljx(0)
tm0, . . . , Ljx(i)
tmi, . . . , Ljx(h)
tmh0.
In order to simplify notation, without loss of generality, a single order of the lag polynomials
is used with kbeing the maximum lag order. If the orders of some components of β(z) are
smaller, it is easy to set some coefficients of the polynomial equal to zero.
We require the existence of the continuous second derivative of functional constraint with
respect to its hyper-parameters i.e. 2fi
γiγ0
i,but the function can be even discontinuous with
respect to the lag index jNwhich allows for various non-smooth functions in terms of
the lag index. The functional constraints can vary with each variable and/or frequency, and
therefore we use γto represent a vector of all the parameters of a restricted model (unique
hyper-parameters and parameters in α(z)), with q= dim(γ) their total number.
As will be shown in the next section, all variants of the usual linear (in terms of variables)
MIDAS regression model are covered by regression (3) via the specification of functional
constraints. When each restriction function is an identity mapping, one obtain an unrestricted
MIDAS regression model.3
2.1. Frequency alignment
It is instructive to rewrite the model (1) in a matrix notation. We start with a few examples.
Suppose ytis observed quarterly and we want to explain its variation with the variable xτ,
which is observed monthly. Since each quarter has three months, the frequency mis 3 in
this example. Suppose we assume that the monthly data in the current and the previous
quarter has explanatory power. This means that for each quarter twe want to model ytas
a linear combination of variables x3t, x3t1, x3t2observed in the quarter tand variables yt1
and x3(t1), x3(t1)1, x3(t1)2observed in the previous quarter t1. In matrix notation the
MIDAS model (1) for this example is:
y2
.
.
.
yn
=
y1
.
.
.
yn1
α1+
x6. . . x1
.
.
..
.
..
.
.
x3n. . . x3n5
β0
.
.
.
β5
+
ε2
.
.
.
εn
3See Foroni, Marcellino, and Schumacher (2014).
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 5
By writing the model in the matrix notation we have transformed high-frequency variable
xτinto a low-frequency vector (x3t, . . . , x3t5)0. We call this transformation the frequency
alignment. Note that we require that the number of observations of xτis exactly 3n.
Let us examine another example. Suppose we have another variable ztobserved weekly which
we want to add to the model. The model (1) does not allow varying frequency ratios, so we
need to assume that each month has exactly 4 weeks. If months do not always have four
weeks, as they do in practice, one can simply think of this model as taking a fixed set of
weekly lags. The frequency mfor the variable zτis then 12. We use again the current and
previous quarter data for explaining variation in yt. This means that for quarter twe model
ytas a linear combination of variables x3t, x3t1, x3t2and z12t, z12t1, . . . , z12t11 observed in
the quarter t, and variables yt1,x3(t1), . . . , x3(t1)2and z12(t1), z12(t1)1, . . . , z12(t1)11
observed in the quarter t1. The model in the matrix form is then:
y2
.
.
.
yn
=
y1
.
.
.
yn1
α1+
x6. . . x1
.
.
..
.
..
.
.
x3n. . . x3n5
β0
.
.
.
β5
+
z24 . . . z1
.
.
..
.
..
.
.
z12n. . . z12n23
γ0
.
.
.
γ23
+
ε2
.
.
.
εn
In this example we have aligned xτinto a vector (x3t, .., x3t5)0and zτinto a vector (z12t, . . . , z12t23)0.
Again we require that the number of observations of high frequency variables are multiple of
n, with multiplication factor being the corresponding frequencies. This is not a restrictive
assumption in practical applications as will be further explained in the section 3.
Let us return to the general case of the model (1). We align the frequency of high-frequency
variable xτby transforming it to the low-frequency vector (x(i)
tmi, x(i)
tmi1, . . . , x(i)
tmik)0. The
model (1) is then expressed in the matrix notation as follows:
yl
.
.
.
yn
=
yl1. . . ylp
.
.
..
.
..
.
.
yn1. . . ynp
α1
.
.
.
αp
+
h
X
i=0
X(i)
β(i)
0
.
.
.
β(i)
k
+
εl
.
.
.
εn
,
where
X(i):=
x(i)
lmix(i)
lmi1. . . x(i)
lmik
x(i)
(l+1)mix(i)
(l+1)mi1. . . x(i)
(l+1)mik
.
.
..
.
.. . . .
.
.
x(i)
tmix(i)
tmi1. . . x(i)
tmik
.
.
..
.
.. . . .
.
.
x(i)
(n1)mix(i)
(n1)mi1. . . x(i)
(n1)mik
x(i)
nmix(i)
nmi1. . . x(i)
nmik
,(4)
and lis the smallest integer such that lmik > 0 and l > p.
The purpose of this subsection was to show how the frequency alignment procedure turns a
MIDAS regression into a classical time series regression where all the variables are observed
at the same frequency.
2.2. Estimation
Equation (3) can be estimated directly via ordinary least squares (OLS), without restrictions
6R package midasr
on the parameters. This is a so called U-MIDAS regression model, see Foroni et al. (2014).
Furthermore, a consistent non-parametric approach could be used to estimate the underlying
parameters of a function as e.g. in Breitung, Roling, and Elengikal (2013). Since, none of
these approaches use a parametric functional constraint, they can be estimated using already
available Rpackages. The midasr package aims at the estimation of mixed frequency models
with some parametric functional constraints.
While model (3) is a linear model in terms of variables, any non-linear functional constraints
will result in non-linearities with respect to the hyper-parameters γ.Therefore, in the gen-
eral case, we use in the function midas_r the non-linear least squares (NLS) estimator of
parameters γof a restricted model (3) as defined by
b
γ= argmin
γRq
n
X
d(k+1)/meα(B)ytfγ(L)0xt,02
,(5)
where the lag polynomial of constrained parameters is defined by
fγ(z) =
k
X
j=0
fγ,jzj
with
fγ,j =f0(γ0;j), . . . , fi(γi;j), . . . , fh(γh;j)0
for each (i, j) {0,1, . . . , h} × {0,1, . . . , k}.A number of numerical algorithms are readily
available in R. By default, the optim optimization function is used with optional choices of
optimization algorithms in it. However, a user can also choose within the function midas_r
other procedures available in Rsuch as nls, customizing the desired algorithm which is
suitable for the problem at hand.
The efficiency of the estimator and consistency of the standard errors depend on whether
the errors of the model are spherical. We leave the aspect of efficiency of estimation to be
considered by a user, however the implementation of heteroscedasticity and autocorrelation
(HAC) robust standard errors is an option available in the package sandwich (see Zeileis 2004).
If all the functional relations fi(·) were non-constraining identity mappings, then the NLS
estimator would be equivalent to the ordinary least squares (OLS) problem in terms of the
original parameters. For convenience, such a U-MIDAS version can be dealt with directly
using a different function midas_u of the package (see an illustration in section 3) or a standard
lm function, provided the alignment of data frequencies is performed as discussed in the
previous section.
2.3. Taxonomy of aggregates-based MIDAS regression models
Based on the parsimony of representation argument, the higher-frequency part of conditional
expectation of MIDAS regressions is often formulated in terms of aggregates as follows
β(L)0xt,0=
h
X
i=0
k
X
j=0
β(i)
jx(i)
tmij
=
h
X
i=0
p
X
r=0
λ(i)
r˜x(i)
tr,
(6)
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 7
with some low-frequency number of lags pNand parameter-driven low-frequency aggregates
˜x(i)
tr:= x(i)
tr(δi,r) =
mi
X
s=1
w(i)
r(δi,r;s)x(i)
(t1r)mi+s
which depend on a weighting (aggregating within a low-frequency period) function wr(δi,r;s)
with its hyper-parameter vector δi,r,which, in the general case, can vary with each vari-
able/frequency and/or the low-frequency lag order rN.Here the aggregation weights are
usually non-negative and, for identification of parameters {λ(i)
r}h,p
i=0,r=0,satisfy the normaliza-
tion constraint such as Pmi1
s=0 wr(δi,r;s)=1.To have the weights add to one, it is convenient
to define a weighting function in the following form
i, r w(i)
r(δi,r;s) = ψ(i)
r(δi,r;s)
Pmi
j=1 ψ(i)
r(δi,r;j), s = 1, . . . , mi,(7)
given some underlying function ψ(i)
r(·).Provided that the latter function is non-negatively-
valued (and the denominator is positive), the resulting weights in eq. (7) are also non-negative.
Table 1provides a list of some underlying functions producing, within the context of equation
(7), the usual weighting schemes with non-negative weights (whenever the parameter space
of underlying functions is appropriately bounded, which in some cases is also needed for
identification of hyper-parameters). In order to avoid heavy notation, indices iand r—which
are connected respectively with frequency/variable and the lag order—are dropped in the
table.
Resulting (normal-
ized) weighting scheme ψ(δ;s) := ψ(i)
r(δi,r;s)Related midasr function
Exponential Almon lag
polynomial
ψ(δ;s) = exp Pp
j=1 δjsj, p N,
where δ= (δ1, . . . , δj, . . . , δp)0Rp.
nealmon
Beta (analogue of proba-
bility density function)
ψ(δ;s) = xδ11
s(1 xs)δ21,where
xs:= ξ+ (1 ξ)h(s), h(s) := (s1)/(m1),
with some marginally small quantity ξ > 0,
and δ= (δ1, δ2)0R2
+.
nbeta
Gompertz (analogue of
probability density func-
tion)
ψ(δ;s) = z(s)eδ1z(s),where
z(s) = exp δ2s,and δ= (δ1, δ2)0R2
+.gompertzp
Log-Cauchy (analogue of
probability density func-
tion)
ψ(δ;s) = s1δ2
2+ (ln sδ1)21,where
δ= (δ1, δ2)0R×R+.lcauchyp
Nakagami (analogue of
probability density func-
tion)
ψ(δ;s) = s2δ11exp(δ12s2),where
δ= (δ1, δ2)0, δ10.5, δ2R+.nakagamip
Table 1: Some usual weighting schemes in aggregation-based MIDAS specifications.
Some other weighting functions which do not have a representation as in eq. (7) are also avail-
able in the package such as (non-normalized) almonp and the polynomial specification with
step functions polystep (see Ghysels et al. (2006b) for further discussion of step functions).
8R package midasr
However, the choice of a particular weighting function in the MIDAS regression with aggre-
gates represents only one restriction imposed on β(L) out of many other choices to be made.
To see this, let us note that aggregates-based MIDAS regressions can be connected with the
following restrictions on the conditional expectation of model (3):
Eα(B)yt|yt,1,{x(i)
t,0}k
j=0=β(L)0xt,0
=
h
X
i=0
p
X
r=0
λ(i)
r˜x(i)
tr,
=
h
X
i=0
p
X
r=0
λ(i)
r
mi
X
s=1
w(i)
r(δi,r;s)x(i)
(t1r)mi+s,
w(i)
r(·)=wr(·)
=
h
X
i=0
p
X
r=0
λ(i)
r
mi
X
s=1
wr(δi,r;s)x(i)
(t1r)mi+s,
wr(·)=w(·)
=
h
X
i=0
p
X
r=0
λ(i)
r
mi
X
s=1
w(δi,r;s)x(i)
(t1r)mi+s,
δi,r=δi
=
h
X
i=0
p
X
r=0
λ(i)
r
mi
X
s=1
w(δi;s)x(i)
(t1r)mi+s,
λ(i)
r=λ(i)
=
h
X
i=0
λ(i)
p
X
r=0
mi
X
s=1
w(δi;s)x(i)
(t1r)mi+s,
(8)
where yt,1= (yt1, ..., ytp)0.
As can be seen—and leaving aside other less intuitive restrictions—depending on the choice of
a particular MIDAS specification with aggregates, it can impose restrictions on the equality
of
the applied weighting scheme/function across variables and/or frequencies (i, w(i)
r(·) =
wr(·));
the applied weighting scheme/function across all low-frequency lags r= 0,1, . . . , p of
aggregates (r, wr(·) = w(·));
parameters of the weighting functions in each lag (r, δi,r =δi);
impact of contemporaneous and lagged aggregates for all lags (r, λ(i)
r=λ(i)).
Furthermore, let sistand for an enumerator of ith higher-frequency periods within a low-
frequency period. Then, noting that, given a frequency ratio mi,there is a one-to-one mapping
between higher-frequency index jNand a pair (r, si)N× {1,2, . . . , mi}
j=rmi+si,
it holds
fi(γi;rmi+si) = λ(i)
rw(i)
r(δi,r;s).(9)
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 9
Hence, it is easy to see that the aggregates-based MIDAS induces a certain periodicity of the
functional constraint fiin eq. (3) as illustrated bellow using a stylized case where all the
restrictions are imposed in eq. (8):
fi(·,0), fi(·,1), . . . fi(·, m 1) fi(·, m), fi(·, m + 1), . . . fi(·,2m1) . . .
λ(i)w(·,1), λ(i)w(·,2),... λ(i)w(·, m)λ(i)w(·,1), λ(i)w(·,2),... λ(i)w(·, m) . . . ,
for any i {0,1, . . . , h}.From eq. (9) it is clear that any specification of MIDAS regression
models which relies on aggregates is a special case of representation (3) with just a specific
functional constraint on parameters. On the other hand, not every general constraint β(L)
can be represented using periodic aggregates. For instance, in the above characterized exam-
ple the correspondence necessarily breaches whenever there exists at least one frequency i,
for which none of pNsatisfies k=pmi1.
2.4. Specification selection and adequacy testing
Besides the usual considerations about the properties of the error term, there are two main
questions about the specification of the MIDAS regression models. First, suitable functional
constraints need to be selected, since their choice will affect the precision of the model. Second,
the appropriate maximum lag orders need to be chosen.
One way to address both issues together is to use some information criterion to select the
best model in terms of the parameter restriction and the lag orders using either in- or out-of-
sample precision measures. Functions midas_r_ic_table and amidas_table of the package
allow the user to make an in-sample choice using some usual information criteria, such as AIC
and BIC, and a user-specified list of functional constraints.4
Another way is to test the adequacy of the chosen functional constraints. For instance, when-
ever the autoregressive terms in model (3) are present (p > 0), it was pointed out by Ghysels
et al. (2006b) that, in the general case, φ(L) = β(L)(B) will have seasonal pattern thus
corresponding to some seasonal impact of explanatory variables on the dependent one in a
pure distributed lag model (i.e. without autoregressive terms). To avoid such an effect when-
ever it is not (or is believed to be not) relevant, Clements and Galv˜ao (2008) proposed to
us a common factor restriction which can be formulated as a common polynomial restriction
with a constraint on the polynomial β(L) to satisfy a factorization β(L) = α(B)φ(L),so that
inverting equation (3) in terms of the polynomial α(B) leaves φ(L) unaffected i.e. without
creating/destroying any (possibly absent) seasonal pattern of the impact of explanatory vari-
ables. However, there is little if any knowledge a priori whether the impact in the distributed
lag model should be seasonal or not. Hence, an explicit testing of adequacy of the model and,
in particular, of the imposed functional constraint is obviously useful.
Let βdenote a vector of all coefficients of polynomial β(z) defined in eq. (3), while fγstand
for the corresponding vector of coefficients restricted by a (possibly incorrect) functional
constraint in fγ(z).Let b
βdenote the respective OLS estimates of unconstrained model i.e.
where functional restrictions of parameters are not taken into account. Let ˆ
fγ:= fγγ=b
γ
denote a vector of the corresponding quantities obtained from the restricted model relying
on the NLS estimates b
γas defined in eq. (5). Denote by α,b
α,and b
αγthe corresponding
vectors of coefficients of polynomial α(z),its OLS estimates in an unrestricted model, and its
4Although aimed at forecasting, the function select_forecasts can also be used to perform the selection
of models relying on their out-of-sample performance.
10 R package midasr
NLS estimates in a restricted model.5Let θ:= (α0,β0)0,b
θ:= (b
α0,b
β0)0,and e
θ:= (b
α0
γ,ˆ
f0
γ)0
signify the corresponding vectors of all coefficients in eq. (3). Then, under the null hypothesis
of γRqsuch that fγ=β,it holds
(b
θe
θ)0A(b
θe
θ)χ2dq,
where Ais a suitable normalization matrix (see Kvedaras and Zemlys 2012 for a standard
and Kvedaras and Zemlys 2013 for a HAC-robust versions of the test), and q= dim(γ) and
d= dim(θ) stand for the number of parameters in a restricted and unrestricted models,
respectively. Functions hAh.test and hAhr.test of the package implement the described
testing as will be illustrated later.
2.5. Forecasting
Let us write model (3) for period t+ 1 as
yt+1 =α0yt,0+β(L)0xt+1,0+εt+1,(10)
where yt,0= (yt, . . . , ytp+1)0and α= (α1, α2, . . . , αp)0is a vector of parameters of the
autoregressive terms. This representation is well suited for (one step ahead) conditional
forecasting of yt+1,provided that the information on the explanatory variables is available.
If it were absent, forecasts of xt+1,0would be also necessary from a joint process of {yt,xt,0}
which might be difficult to specify and estimate correctly, especially, bearing in mind the
presence of data with mixed frequencies. Instead, a direct approach to forecasting is often
applied in the MIDAS framework. Namely, given an information set available up to a moment
tdefined by It,0={yt,j,xt,j }
j=0,where
yt,j = (ytj, ..., ytjp+1)0
xt,j = (x(0)
tm0, ..., x(i)
tmi, ..., x(h)
tmh)0,
an `-step ahead direct forecast
˜yt+`=Eyt+`It,0=α0
`yt,0+β`(L)0xt,0, ` N,(11)
can be formed leaning on a model linked to a corresponding conditional expectation
yt+`=α0
`yt,0+β`(L)0xt,0+ε`,t, E ε`,tIt,0,
where α`and β`(L) are the respective horizon `-specific parameters. Note that, in principle,
these conditional expectations have a form of representation (3) with certain restrictions on
the original lag polynomials of coefficients. Hence, in the general case, the suitable restrictions
for each `will have a different form.
Given periods `= 1,2,..., and a selected model or a list of specifications to be considered,
package midasr provides the point forecasts corresponding to the estimated analogue of eq.
(11) evaluates the precision of different specifications, and performs weighted forecasting using
the framework defined in Ghysels (2013).
5Recall that unconstrained αelements make a subset of parameter vector γof a constrained model.
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 11
3. Implementation in midasr package
3.1. Data handling
From a data handling point of view, the key specificity of the MIDAS regression model is
that the length of observations of variables observed at various frequencies differs and needs
to be aligned as described in section 2. There is no existing Rfunction which performs such
a transformation and the package midasr provides a solution to these challenges. The basic
functionality of data handling is summarized in Table 2.
Function Description Example Notes
mls(x,k,m)
Stacks a HF data vector x
into a corresponding ma-
trix of observations at LF
of size dim x
m×dim k: from
the first to the last HF lag
defined by vector k.
mls(x,2:3,3)
dim x
mmust be an integer (NA are al-
lowed).
For m= 1,the function produces lags
of xthat are defined by vector argument
k, e.g., mls(x,2:3,1) yields a dataset
containing the lags xt2and xt3of xt.
fmls(x,k,m)
Same as mls, except that
kis a scalar and the k+ 1
lags are produced starting
from 0 up to k.
fmls(x,2,3) fmls(x,2,3) is equivalent to
mls(x,0:2,3).
dmls(x,k,m)
Same as fmls, apart that
the resulting matrix con-
tains k+ 1 first-order HF
differences of x.
dmls(x,2,3)
mls(x,1,1) can be used in dmls to
get stacking of lagged differences, e.g.,
dmls(mls(x,1,1),2,3).
Table 2: A summary of basic data handling functionality in package midasr.
Function fmls(x,k,m) performs exactly the transformation defined in equation (4), convert-
ing an observation vector xof a given (potentially) higher-frequency series into the corre-
sponding stacked matrix of observations of (k+ 1) low-frequency series (contemporaneous
with klags) as defined by the maximum lag order kand the frequency ratio m. For instance,
given a series of twelve observations
x<- 1:12
we get the following result
fmls(x, k=2,m=3)
## X.0/m X.1/m X.2/m
## [1,] 3 2 1
## [2,] 6 5 4
## [3,] 9 8 7
## [4,] 12 11 10
12 R package midasr
i.e. three variables (a contemporaneous and two lags) with four low-frequency observations
(n= 12/m).
Function mls is slightly more flexible as the lags included can start from a given order rather
than from zero, whereas the function fmls uses a full lag structure. dmls performs in addition
a first-order differencing of the data which is convenient when working with integrated series.
A couple of issues should be taken into account when working with series of different frequen-
cies.
It is assumed that the numbers of observations of different frequencies match exactly
through the frequency ratio (ni=nmi), and the first and last observations of each
series of different frequency are correspondingly aligned (possibly using NA to account
for some missing observations for series of higher frequency).
Because of different lengths of series of various frequencies, the data in the model cannot
be kept in one data.frame. An Robject list needs to bee used if one intends to keep
the data in a single object, but it is not required for the further modeling.
3.2. An example of simulated MIDAS regression
Using the above data handling functions, it is straightforward to simulate a response series
from the MIDAS regression as a data generating process (DGP). For instance, suppose one
is willing to generate a low-frequency response variable yin the MIDAS with two higher-
frequency series xand zwhere the impact parameters satisfy the exponential Almon lag
polynomials of different orders as follows:
yt= 2 + 0.1t+
7
X
j=0
β(1)
jx4tj+
16
X
j=0
β(2)
jz12tj+εt,
xτ1n.i.d.(0,1), zτ2n.i.d.(0,1), εtn.i.d.(0,1),
(12)
where (xτ1, zτ2, εt) are independent for any (τ1, τ2, t)Z3,and
β(i)
j=γ(i)
0
exp Pqi1
s=1 γ(i)
sjs
Pdi1
j=0 exp Pqi1
s=1 γ(i)
sjs, i = 1,2,
where d1=k1+ 1 = 8 is a multiple of the frequency ratio m1= 4,whereas d2=k2+ 1 = 17
is not a multiple of m2= 12.Here q1= 2, q2= 3 with parameterizations
γ1= (1,0.5)0,
γ2= (2,0.5,0.1)0,
which yield the shapes of impact as plotted in Figure 2.
The following Rcode produces a series according to the DGP characterized above:
set.seed(1001)
## Number of low-frequency observations
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 13
0 5 10 15
0.0 0.2 0.4
High frequency lag
Weights
Figure 2: Shapes of impact, blue for xτ, red for zτ.
n<- 250
## Linear trend and higher-frequency explanatory variables
## (e.g. quarterly and monthly)
trend <- c(1:n)
x<- rnorm(4*n)
z<- rnorm(12 *n)
## Exponential Almon polynomial constraint-consistent
## coefficients
fn.x <- nealmon(p=c(1,-0.5), d=8)
fn.z <- nealmon(p=c(2,0.5,-0.1), d=17)
## Simulated low-frequency series (e.g. yearly)
y<- 2+0.1 *trend +mls(x, 0:7,4)%*% fn.x +mls(z, 0:16,
12)%*% fn.z +rnorm(n)
It is of interest to note that the impact of variable xcan be represented using aggregates-based
MIDAS, whereas the impact of zcannot.
3.3. Some specification examples of MIDAS regressions
Suppose now that we have (only) observations of y, x, and zwhich are stored as vectors,
matrices, time series, or list objects in R, and our intention is to estimate a MIDAS regression
model as in equation (12):
a) without restricting the parameters (as in U-MIDAS) and using the OLS;
b) with the exponential Almon lag polynomial constraint on parameters (as in the function
nealmon) and using the NLS.
14 R package midasr
The OLS estimation as in case a) is straightforwardly performed using
eq.u <- lm(y ~trend +mls(x, k=0:7,m=4)+mls(z, k=0:16,
m=12))
or, equivalently
eq.u <- midas_r(y ~trend +mls(x, 0:7,4)+mls(z, 0:16,12),
start =NULL)
The following Rcode estimates the constrained case b) using the function midas_r and reports
the NLS estimates b
γof hyper-parameters with the related summary statistics.
eq.r <- midas_r(y ~trend +mls(x, 0:7,4, nealmon) +mls(z,
0:16,12, nealmon), start =list(x=c(1,-0.5), z=c(2,
0.5,-0.1)))
summary(eq.r)
##
## Formula y ~ trend + mls(x, 0:7, 4, nealmon) + mls(z, 0:16, 12, nealmon)
##
## Parameters:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 1.988196 0.115299 17.24 < 2e-16 ***
## trend 0.099883 0.000777 128.57 < 2e-16 ***
## x1 1.353343 0.151220 8.95 < 2e-16 ***
## x2 -0.507566 0.096670 -5.25 3.3e-07 ***
## z1 2.263473 0.172815 13.10 < 2e-16 ***
## z2 0.409653 0.155685 2.63 0.00905 **
## z3 -0.072979 0.020392 -3.58 0.00042 ***
## ---
## Signif. codes: 0 '***'0.001 '**'0.01 '*'0.05 '.'0.1 ' ' 1
##
## Residual standard error: 0.932 on 242 degrees of freedom
As you can see the syntax of the function midas_r is similar to the standard Rfunction
nls. The model is specified via familiar formula interface. The lags included and functional
restriction used can be individual to each variable and are specified within the respective
mls,fmls, or dmls function used with midas_r. It is necessary to provide a list of starting
values for each variable with restricted coefficients, since it implicitly defines the number of
hyper-parameters of the constraint functions to be used for each series.
The main difference with the function nls is that there is a greater choice of numerical
optimization algorithms. The function midas_r is written in a way that in theory it can use
any Roptimization function. The choice is controlled via Ofunction argument. Currently
it is possible to use functions optim and nls which are present in a standard Rinstallation
and function spg from the package BB. The additional arguments to the aforementioned
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 15
functions can be specified directly in the call to midas_r. So for example if we want to use
the optimization algorithm of Nelder and Mead, which is the default option in the function
optim we use the following code
eq.r2<-midas_r(y~trend+mls(x,0:7,4,nealmon)+mls(z,0:16,12,nealmon),
start=list(x=c(1,-0.5),z=c(2,0.5,-0.1)),
Ofunction="optim",method="Nelder-Mead")
If we want to use Golub-Pereyra algorithm for partially linear least-squares models imple-
mented in the function nls we use the following code
eq.r2<-midas_r(y~trend+mls(x,0:7,4,nealmon)+mls(z,0:16,12,nealmon),
start=list(x=c(1,-0.5), z=c(2,0.5,-0.1)),
Ofunction="nls",method="plinear")
It is possible to re-estimate the NLS problem with the different algorithm using as starting
values the final solution of the previous algorithm. For example it is known, that the default
algorithm in nls is sensitive to starting values. So first we can use the standard Nelder-Mead
algorithm to find “more feasible” starting values and then use the nls to get the final result:
eq.r2<-midas_r(y~trend+mls(x,0:7,4,nealmon)+mls(z,0:16,12,nealmon),
start=list(x=c(1,-0.5),z=c(2,0.5,-0.1)),
Ofunction="optim",method="Nelder-Mead")
eq.r2<-midas_r(eq.r2,Ofunction="nls")
The output of the optimization function used can be found by inspecting the element opt of
midas_r output.
eq.r2<-midas_r(y~trend+mls(x,0:7,4,nealmon)+mls(z,0:16,12,nealmon),
start=list(x=c(1,-0.5),z=c(2,0.5,-0.1)),
Ofunction="optim",method="Nelder-Mead")
eq.r2$opt
## $par
## (Intercept) trend x1 x2 z1 z2
## 1.98565 0.09990 1.35252 -0.50816 2.26318 0.40886
## z3
## -0.07284
##
## $value
## [1] 210
##
## $counts
## function gradient
## 502 NA
16 R package midasr
##
## $convergence
## [1] 1
##
## $message
## NULL
Here we observe that the Nelder-Mead algorithm evaluated the cost function 502 times.
The optimization functions in Rreport the status of the convergence of optimization algorithm
by the numeric constant, 0 indicating succesful convergence. This code is reported as the
element convergence of the midas_r output.
eq.r2$convergence
## [1] 1
In this case the convergence was not successfull. The help page of the function optim indicates
that convergence code 1 means that the iteration limit was reached.
In order to improve the convergence it is possible to use user defined gradient functions. To
use them it is necessary to define gradient function of the restriction. For example for the
nealmon restriction the gradient function is defined in the following way:
nealmon.gradient <- function(p,d,m) {
i<- 1:d
pl <- poly(i,degree=length(p)-1,raw=TRUE)
eplc <- exp(pl%*%p[-1])[,,drop=TRUE]
ds <- apply(pl*eplc,2,sum)
s<- sum(eplc)
cbind(eplc/s,p[1]*(pl*eplc/s-eplc%*%t(ds)/s^2))
}
The naming convention restriction_name.gradient is important, currently this is the way
the midas_r “knows” about the gradient functions. To use this function for the optimization
it is necessary to set user.gradient=TRUE in call to midas_r:
eq.r2<-midas_r(y~trend+mls(x,0:7,4,nealmon)+mls(z,0:16,12,nealmon),
start=list(x=c(1,-0.5),z=c(2,0.5,-0.1)),
user.gradient=TRUE)
This way midas_r calculates the exact gradient of the NLS problem (5) using the specified
gradient function of the restriction. For all the types of the restrictions referenced in table 3
the gradient functions are specified in the package midasr.
The gradient and the hessian of the NLS problem are supplied as the output of midas_r. The
numerical approximation of the hessian is calculated using the package numDeriv, the exact
gradient is calculated if user.gradient=TRUE and the numerical approximation otherwise.
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 17
Having the gradient and hessian calculated allows to check whether the necessary and sufficient
conditions for the convergence are satisfied. This is performed by the function deriv_test
which calculates the Euclidean norm of the gradient and the eigenvalues of the hessian. It
then tests whether the norm of gradient is close to zero and whether the eigenvalues are
positive.
deriv_tests(eq.r, tol =1e-06)
## $first
## [1] FALSE
##
## $second
## [1] TRUE
##
## $gradient
## [1] 0.0042441 0.0503346 -0.0008513 0.0021252 0.0004805 -0.0004505
## [7] -0.3851794
##
## $eigenval
## [1] 1.048e+07 5.888e+04 3.664e+02 1.221e+02 8.117e+01 5.148e+01 4.596e+01
To retrieve a vector of constrained estimates ˜
θ(and, hence, also ˆ
f=fγγ=b
γ) which.min
corresponds to the vector θ(β, respectively), the function midas_coef can be used as follows
midas_coef(eq.r)
## (Intercept) trend x1 x2 x3 x4
## 1.988e+00 9.988e-02 5.481e-01 3.300e-01 1.986e-01 1.196e-01
## x5 x6 x7 x8 z1 z2
## 7.197e-02 4.332e-02 2.608e-02 1.570e-02 3.347e-01 4.050e-01
## z3 z4 z5 z6 z7 z8
## 4.235e-01 3.827e-01 2.989e-01 2.018e-01 1.177e-01 5.932e-02
## z9 z10 z11 z12 z13 z14
## 2.584e-02 9.728e-03 3.165e-03 8.898e-04 2.162e-04 4.539e-05
## z15 z16 z17
## 8.237e-06 1.292e-06 1.750e-07
In the example provided above, a functional constraint was imposed directly on β(L) terms
corresponding to each series without the usage of aggregates. Relying on the relationship
(9), it is always possible to write such an explicit general constraint from an aggregates-
based one. For convenience of a user, the function amweights can be used to form several
standard periodic functional constraints with ’typical’ restrictions explicated in equation (7).
For instance,
18 R package midasr
amweights(p=c(1,-0.5), d=8,m=4,weight = nealmon, type ="C")
## [1] 0.4551 0.2760 0.1674 0.1015 0.4551 0.2760 0.1674 0.1015
with type="C" corresponds to a fully restricted version of aggregates-based expression (7)
apart the cross-restriction on the equality of weighting schemes between different variables/frequencies.
Note that the code above repeats the result of
nealmon(p=c(1,-0.5), d=4)
## [1] 0.4551 0.2760 0.1674 0.1015
twice (d/m = 2), as implied by the number of periods at higher-frequency (d=8) and the
frequency ratio (m=4). In this way, the function amweights can be used to define explicitly
a new functional constraint relying on the relationship (9). Alternatively, one can indicate
directly within the function midas_r that the aggregates-based restriction must be used as
follows
eq.r2 <- midas_r(y ~trend +mls(x, 0:7,4, amweights, nealmon,
"C")+mls(z, 0:16,12, nealmon), start =list(x=c(1,-0.5),
z=c(2,0.5,-0.1)))
where the first variable follows and aggregates-based MIDAS restriction scheme. Note that
the selection of alternative types A”and B” are connected with specifications having a larger
number of parameters (see Table 3), hence the list of starting values needs to be adjusted to
account for an increase in the number of (potentially unequal) impact parameters.
It should be also noted that, whenever the aggregates-connected restrictions are used, the
number of periods must be a multiple of the frequency ratio. For instance, the current lag
specification for variable zis not consistent with this requirement and cannot be represented
through the (periodic) aggregates, but either mls(z,0:11,12,amweights,nealmon,"C") or
mls(z,0:23,12,amweights,nealmon,"C") would be valid expressions from the code imple-
mentation point of view.
Table 3 summarizes and provides various other examples of correspondence between midas_r
coding and the analytical specifications of MIDAS regressions.
3.4. Adequacy testing of restrictions
Given a MIDAS regression model estimated with midas_r, the empirical adequacy of the
functional restrictions can be tested under quite standard assumptions (see Kvedaras and
Zemlys 2012 and Kvedaras and Zemlys 2013) using functions hAh.test and hAhr.test of
the package. In the case of a stationary series {yt}they can be applied directly, whereas
whenever {yt}is cointegrated with explanatory variables, a special transformation needs to
be applied before the testing (see e.g. Bilinskas and Zemlys 2013). The hAh.test can be used
whenever errors of the process are independently and identically distributed, whereas the
hAhr.test uses a HAC-robust version of the test. We should just point out that, whenever
no significant HAC in the residuals are observed, we would suggest using hAh.test which
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 19
Description Code example Analytical expression Notes
Different constraint
functions
midas_r(ymls(x,0:7,4,nealmon)+
mls(z,0:16,12,gompertzp),
start=list(x=c(1,-0.5),z=c(1,0.5,0.1)))
yt=c+P7
j=0 β(1)
jx4tj+
P16
j=0 β(2)
jz12tj+εt
Constraints on β(i)
j, i = 1,2
are given by different func-
tions.
Partial constraint
(only on z)
midas_r(ymls(x,0:7,4)+mls(z,0:16,12,
nealmon),start=list(z=c(1,-0.5)))
yt=c+P7
j=0 β(1)
jx4tj+
P16
j=0 β(2)
jz12tj+εt
xenters linearly with uncon-
strained β(1)
j.
With unrestricted
autoregressive
terms
midas_r(ymls(y,1:2,1)+mls(x,0:7,4,
nealmon),start=list(x=c(1,-0.5)))
yt=c+P2
j=1 αjytj+
P7
j=0 βjx4tj+εt
Autoregressive terms enter
linearly with unconstrained
coefficients.
With a common
factor restriction
midas_r(ymls(y,1:2,1,"*")+mls(x,0:7,4,
nealmon),start=list(x=c(1,-0.5))) α(B)yt=c+α(B)λ(L)x4t+εt,
Here coefficients of λ(z) are
assumed to satisfy nealmon
restriction.
With autoregr. pa-
rameters restricted
by a function
midas_r(ymls(y,1:6,1,nealmon)
+mls(x,0:7,4,nealmon),
start=list(y=c(1,-0.5),x=c(1,-0.5)))
yt=c+P6
j=1 αjytj+
P7
j=0 βjx4tj+εt
Autoregressive parameters
αj, j = 1,...,6 are con-
strained to satisfy nealmon
restriction.
Aggregates-based
(Case A)
midas_r(ymls(x,0:7,4,amweights,
nealmon,"A"),start=list(x=c(1,1,1,-0.5)))
yt=c+
P1
r=0 λrP4
s=1 w(δr;s)x4(t1r)+s+εt
The same weighting scheme
(not parameters) is used in
aggregation.
Aggregates-based
(Case B)
midas_r(ymls(x,0:7,4,amweights,
nealmon,"B"),start=list(x=c(1,1,-0.5)))
yt=c+
P1
r=0 λrP4
s=1 w(δ;s)x4(t1r)+s+εt
The same weights are used in
aggregation.
Aggregates-based
(Case C)
midas_r(ymls(x,0:7,4,amweights,
nealmon,"C"),start=list(x=c(1,-0.5)))
yt=c+
λP1
r=0 P4
s=1 w(δ;s)x4(t1r)+s+εt
A common impact parameter
of lags and the same weights
are used in aggregation.
With a user-defined
constraint
midas_r(ymls(x,0:101,4,fn),
start= list(x=c(0,0)))
yt=c+P101
j=0 βjx4tj+εt,
βj=γ1(j+ 1)γ2, j = 0,1,...,101.
fn in the midas_r code is as
e.g. provided in the note of
the table.
Table 3: Some functionality of specification of MIDAS regressions in midasr package.
Note: A the function fn in the last example of the table is defined e.g. by fn<-function(p,d){p[1]*c(1:d)^p[2]}
20 R package midasr
would then have more precise test sizes in small samples. In the case of integrated series {yt}
which is co-integrated with explanatory variables, some other alternatives are available (see
Bilinskas, Kvedaras, and Zemlys 2013).
For illustration, let us use the name eq.r of an estimated model as in the previous subsections.
Then the functions produce, respectively,
hAh.test(eq.r)
##
## hAh restriction test
##
## data:
## hAh = 16.55, df = 20, p-value = 0.6818
hAhr.test(eq.r)
##
## hAh restriction test (robust version)
##
## data:
## hAhr = 14.85, df = 20, p-value = 0.7847
Here the value of a test statistic, the degrees of freedom (the number of binding constraints
on parameters in equation (3)), and the empirical significance of the null hypothesis that a
functional constraint is adequate are reported.
As can be seen, such a specification, which in fact corresponds to the underlying DGP, can-
not be rejected at the usual significance levels, whereas e.g. reducing the number of hyper-
parameters of functional constraint of variable zto only two instead of three is quite strongly
rejected using either version of the test:
eq.rb<-midas_r(y~trend+mls(x,0:7,4,nealmon)+mls(z,0:12,12,nealmon),
start=list(x=c(1,-0.5),z=c(2,-0.1)))
hAh.test(eq.rb)
##
## hAh restriction test
##
## data:
## hAh = 36.89, df = 17, p-value = 0.00348
hAhr.test(eq.rb)
##
## hAh restriction test (robust version)
##
## data:
## hAhr = 32.88, df = 17, p-value = 0.01168
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 21
Whenever the empirical adequacy cannot be reject at some appropriate level of significance
for a couple of models, we could further rely on information criteria to make the selection of
the best candidate(s).
3.5. Model selection
Suppose that we want to investigate which out of several functional constraints—for instance,
the normalized (”nealmon”) or non-normalized (”almonp”) exponential Almon lag polynomials,
or with polynomial of order 2 or 3, and so on—are better suited in a MIDAS regression model
of yon xand z(possibly different for each variable). Since the best maximum number of lags
can differ with a functional constraint and/or variable/frequency, let us first define using the
midasr function expand_weights_lags the sets of potential models corresponding to each
explanatory variable as follows
set.x<-expand_weights_lags(weights=c("nealmon","almonp"),
from=0,to=c(5,10),m=1,
start=list(nealmon=c(1,-1),almonp=c(1,0,0)))
set.z <- expand_weights_lags(c("nealmon","nealmon"),
0,c(10,20),1,
start=list(nealmon=c(1,-1),nealmon=c(1,-1,0)))
Here, for each variable, vector (or list) weights defines the potential restrictions to be con-
sidered and a list start gives the appropriate starting values defining implicitly the number
of hyper-parameters per a function.
The potential lag structures are given by the following ranges of high-frequency lags: from
[from;mmin(to)] to [from;mmax(to)]. When aggregates-based modeling is involved using
amweights in midas_r,mcan be set to the frequency ratio which ensures that the considered
models (lag structures) are multiples of it. Otherwise, we would recommend to operate
with high-frequency lag structures without changing the default value m= 1.Then, the
set of potential models is defined as all possible different combinations of functions and lag
structures with a corresponding set of starting values. A simple example bellow illustrates
the result in order to reveal the underlying structure, which, besides the understanding of it,
is otherwise not needed for a user.
expand_weights_lags(weights=c("nealmon","nbeta"),
from=1,to=c(2,3),m=1,
start=list(nealmon=c(1,-1),nbeta=rep(0.5,3)))
## weights lags starts
## 1 nealmon 1:2 c(1, -1)
## 2 nealmon 1:3 c(1, -1)
## 3 nbeta 1:2 c(0.5, 0.5, 0.5)
## 4 nbeta 1:3 c(0.5, 0.5, 0.5)
Given the sets of potential specifications for each variable as defined above, the estimation of
all the models is performed by
22 R package midasr
eqs.ic <- midas_r_ic_table(y ~trend +mls(x, 0,m=4)+fmls(z,
0,m=12), table =list(z= set.z, x= set.x))
The function midas_r_ic_table returns a summary table of all models together with the cor-
responding values of the usual information criteria and the empirical sizes of adequacy testing
of functional restrictions of parameters. The result of derivative tests and the convergence
status of the optimization function is also returned.
The summary table is a data.frame where each row corresponds to candidate model, so this
table can be manipulated in the usual Rway. The table can be accessed as table element of
the list returned by midas_r_ic_table. The list of fitted midas_r objects of all candidate
models can be accessed as candlist element. It is possible to inspect each candidate model
and fine-tune its convergence if necessary.
eqs.ic$candlist[[5]] <- midas_r(eqs.ic$candlist[[5]], Ofunction ="nls")
The summary table can be recalculated by simply passing the fine-tuned list in to the function
midas_r_ic_table again.
midas_r_ic_table(eqs.ic)
It should be pointed out that there is no need to provide the weighting function nor a specific
lag order in the mls functions in such a case, since they are defined by the respective potential
sets of models under option table in the function midas_r_ic_table. Any provided values
with mls (or other similar functions) are over-written by those defined in table.
Finally, the best model in terms of a selected information criterion in a restricted or unre-
stricted model then is simply obtained by using
modsel(eqs.ic, IC ="AIC",type ="restricted")
which also prints the usual summary statistics as well as the testing of adequacy of the
applied functional restriction using, by default, the hAh.test. A word of caution is needed
here to remind that, as is typical, the empirical size of a test corresponding to a complex
model-selection procedure might not correspond directly to a nominal one of a single-step
estimation.
3.6. Forecasting
Conditional forecasting (with confidence intervals, etc.) using unrestricted U-MIDAS regres-
sion models which are estimated using lm can be performed using standard Rfunctions e.g.
predict.lm. Conditional point prediction given a specific model is also possible relying on a
standard predict function.
The function predict works similarly to predict.lm. It takes the new data, transforms it into
an appropriate matrix and multiplies it with the coefficients. Suppose we want to produce the
forecast ˆyT+1|Tfor the model (12). To produce this forecast we need the data x4(T+1), ..., x4T3
and z12(T+1), ..., z12T4.It would be tedious to calculate precisely the required data each time
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 23
we want to perform a forecasting exercise. To alleviate this problem the package midasr
provides the function forecast. This function assumes that the model was estimated with
the data up to low frequency index T. It is then assumed that the new data is the data after
the low frequency Tand then calculates the appropriate forecast. For example suppose that
we have new data for one low frequency period for the model (12). Here is how the forecast
for one period would look like:
newx <- rnorm(4)
newz <- rnorm(12)
forecast(eq.rb, newdata =list(x= newx, z= newz, trend =251))
## [1] 28.29
It is also common to estimate models which do not require new data for forecasting
yt+`= 2 + 0.1t+
7
X
j=0
β(1)
jx4tj+
16
X
j=0
β(2)
jz12tj+εt+`,
where `is the desired forecasting horizon. This model can be rewritten as
yt= 2 + 0.1(t`) +
7+4`
X
j=4`
β(1)
jx4tj+
16+12`
X
j=12`
β(2)
jz12tj+εt,
and can be estimated using midas_r. For such a model we can get forecasts ˆyT+`|T, ..., ˆyT+1|T
using the explanatory variable data up to low frequency index T. To obtain these forecasts
using the function forecast we need to supply NA values for explanatory variables. An
example for `= 1 is as follows:
eq.f <- midas_r(y ~trend +mls(x, 4+0:7,4, nealmon) +mls(z,
12 +0:16,12, nealmon), start =list(x=c(1,-0.5), z=c(2,
0.5,-0.1)))
forecast(eq.f, newdata =list(x=rep(NA,4), z=rep(NA,12),
trend =251))
## [1] 27.2
Note that we still need to specify a value for the trend.
In addition, the package midasr provides a general flexible environment for out-of-sample
prediction, forecast combination, and evaluation of restricted MIDAS regression models us-
ing the function select_and_forecast. If exact models were known for different forecasting
horizons, it can also be used just to report various in- and out-of-sample prediction charac-
teristics of the models. In the general case, it also performs an automatic selection of the
best models for each forecasting horizon from a set of potential specifications defined by all
combinations of functional restrictions and lag orders to be considered, and produces forecast
combinations according to a specified forecast weighting scheme.
In general, the definition of potential models in the function select_and_forecast is similar
to that one uses in the model selection analysis described in the previous section. However,
24 R package midasr
different best performing specifications are most likely related with each low-frequency fore-
casting horizon `= 0,1,2,.... Therefore the set of potential models (parameter restriction
functions and lag orders) to be considered for each horizon needs to be defined.
Suppose that, as in the previous examples, we have variables xand zwith frequency ratios
m1= 4 and m2= 12,respectively. Suppose that we intend to consider forecasting of yup
to three low-frequency periods ` {1,2,3}ahead. It should be noted that, in terms of high-
frequency periods, they correspond to `m1 {4,8,12}for variable x, and `m2 {12,24,36}
for variable z. Thus these variable-specific vectors define the lowest lags6of high-frequency
period to be considered for each variable in the respective forecasting model (option from
in the function select_and_forecast). Suppose further that in all the models we want
to consider specifications having not less than 10 high-frequency lags and not more than
15 for each variable. This defines the maximum high-frequency lag of all potential models
considered for each low-frequency horizon period ` {1,2,3}.Hence, for each variable, three
corresponding pairs (`m1+ 10, `m1+ 15), ` {1,2,3}will define the upper bounds of ranges
to be considered (option to in the function select_and_forecast). For instance, for variable
x, three pairs (14,19),(18,23),and (22,27) correspond to `= 1,2,and 3 and together with
that defined in option from (see x=(4,8,12)) imply that the following ranges of potential
models will be under consideration for variable x:
`= 1 : from [4 14] to [4 19],
`= 2 : from [8 18] to [8 23],
`= 3 : from [12 22] to [12 27].
The other options of the function select_and_forecast do rather not require further expla-
nation
cbfc<-select_and_forecast(y~trend+mls(x,0,4)+mls(z,0,12),
from=list(x=c(4,8,12),z=c(12,24,36)),
to=list(x=rbind(c(14,19),c(18,23),c(22,27)),
z=rbind(c(22,27),c(34,39),c(46,51))),
insample=1:200,outsample=201:250,
weights=list(x=c("nealmon","almonp"),
z=c("nealmon","almonp")),
wstart=list(nealmon=rep(1,3),almonp=rep(1,3)),
IC="AIC",
seltype="restricted",
ftype="fixed",
measures=c("MSE","MAPE","MASE"),
fweights=c("EW","BICW","MSFE","DMSFE")
)
The names of weighting schemes are taken from MIDAS Matlab toolbox Ghysels (2013).
Similarly forecasting using rolling and recursive model estimation samples defined therein
Ghysels (2013) is supported by setting option seltype="rolling" or seltype="recursive".
6Including lags smaller than that would imply that more information on explanatory variables is available
and, in fact, `1 forecasting horizon is actually under consideration.
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 25
Then, among others,
cbfc$accuracy$individual
cbfc$accuracy$average
report, respectively:
the best forecasting equations (in terms of a specified criterion out of the above-defined
potential specifications), and their in- and out-of-sample forecasting precision measures
for each forecasting horizon;
the out-of-sample precision of forecast combinations for each forecasting horizon.
The above example illustrated a general usage of the function select_and_forecast including
selection of best models. Now suppose that a user is only interested in evaluating a one step
ahead forecasting performance of a given model. Suppose further that he/she a priori knows
that the best specifications to be used for this forecasting horizon `= 1 is with
mls(x,4:12,4,nealmon) with hyper-parameters x=c(2,10,1,-0.1) (the first one rep-
resenting an impact parameter and the last three being the hyper-parameters of the
normalized weighting function), and
mls(z,12:20,12,nealmon) with hyper-parameters z=c(-1,2,-0.1) i.e. with one hyper-
parameter less in the weighting function.
Given already preselected and evaluated models, user can use the function average_forecast
to evaluate the forecasting performance. To use this function at first it is necessary to fit the
model and then pass it to function average_forecast specifying the in-sample and out-of-
sample data, accuracy measures and weighting scheme in a similar manner to select_and_forecast
mod1 <- midas_r(y ~trend +mls(x, 4:14,4, nealmon) +mls(z, 12:22,12, nealmon),
start=list(x=c(10,1,-0.1),z=c(2,-0.1)))
avgf <- average_forecast(list(mod1),
data=list(y=y,x=x,z=z,trend=trend),
insample=1:200,outsample=201:250,
type="fixed",
measures=c("MSE","MAPE","MASE"),
fweights=c("EW","BICW","MSFE","DMSFE"))
It should also be pointed out that the forecast combinations in the function select_and_forecast
are obtained only from the forecasts linked to different restriction functions on parameters.
The forecasts related to different lag specifications are not combined, but the best lag or-
der is chosen in terms of a given information criterion. If there is a need to get forecast
combinations for a group of models which the user selected using other criteria, the function
average_forecast should be used in a manner outlined in the previous example.
26 R package midasr
4. Empirical illustrations
4.1. Forecasting GDP growth
We replicate the example provided in Ghysels (2013). In particular we run MIDAS regression
to forecast quarterly GDP growth with monthly non-farms payroll employment growth. The
forecasting equation is the following
yt+1 =α+ρyt+
8
X
j=0
θjx3tj+εt,
where ytis the log difference of quarterly seasonally adjusted real US GDP and x3tis the log
difference of monthly total employment non-farms payroll. The data is taken from St. Louis
FRED website.
First we load the data and perform necessary transformations.
library(quantmod)
gdp <- getSymbols("GDP",src ="FRED",auto.assign =FALSE)
payems <- getSymbols("PAYEMS",src ="FRED",auto.assign =FALSE)
y<- window(ts(gdp, start =c(1947,1), frequency =4), end =c(2011,
2))
x<- window(ts(payems, start =c(1939,1), frequency =12), end =c(2011,
7))
yg <- log(y/lag(y, -1)) *100
xg <- log(x/lag(x, -1)) *100
nx <- ts(c(NA, xg, NA,NA), start =start(x), frequency =12)
ny <- ts(c(rep(NA,33), yg, NA), start =start(x), frequency =4)
The last two lines are needed to equalize the sample sizes, which are different in the original
data. We simply add additional NA values at the beginning and the end of the data. The
graphical representation of the data is shown in Figure 4.1.
To specify the model for the midas_r function we rewrite it in the following equivalent form:
yt=α+ρyt1+
11
X
j=3
θjx3tj+εt,
As in Ghysels (2013) we restrict the estimation sample from the first quarter of 1985 to the
first quarter of 2009. We evaluate the models with the Beta polynomial, Beta with non-zero
and U-MIDAS weight specifications.
xx <- window(nx, start =c(1985,1), end =c(2009,3))
yy <- window(ny, start =c(1985,1), end =c(2009,1))
beta0 <- midas_r(yy ~mls(yy, 1,1)+mls(xx, 3:11,3, nbeta),
start =list(xx =c(1.7,1,5)))
coef(beta0)
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 27
Time
Percentages
1940 1960 1980 2000
−4 −2 0 2 4 6
Figure 3: Quaterly GDP and Monthly Non-Farm Payroll Employment Growth Rate
## (Intercept) yy xx1 xx2 xx3
## 0.8348 0.1054 2.5795 1.0188 13.1287
betan <- midas_r(yy ~mls(yy, 1,1)+mls(xx, 3:11,3, nbetaMT),
start =list(xx =c(2,1,5,0)))
coef(betan)
## (Intercept) yy xx1 xx2 xx3 xx4
## 0.93868 0.06977 2.24646 0.98693 1.46542 -0.09264
um <- midas_r(yy ~mls(yy, 1,1)+mls(xx, 3:11,3), start =NULL)
coef(um)
## (Intercept) yy xx1 xx2 xx3 xx4
## 0.93459 0.08329 1.97711 0.92310 0.44524 -0.21101
## xx5 xx6 xx7 xx8 xx9
## 0.20813 1.20043 -0.50631 -0.67365 -1.24858
We can evaluate the forecasting performance of these three models on the out of sample data,
containing 9 quarters, from 2009Q2 to 2011Q2
fulldata <- list(xx=window(nx,start=c(1985,1),end=c(2011,6)),
yy=window(ny,start=c(1985,1),end=c(2011,2)))
insample <- 1:length(yy)
28 R package midasr
outsample <- (1:length(fulldata$yy))[-insample]
avgf<-average_forecast(list(beta0,betan,um),
data=fulldata,
insample=insample,
outsample=outsample)
sqrt(avgf$accuracy$individual$MSE.out.of.sample)
## [1] 0.5412 0.5002 0.4623
We see that the unrestricted MIDAS regression model gives the best out-of-sample RMSE.
4.2. Forecasting realized volatility
As another demonstration we use the package midasr to forecast the daily realized volatility.
A simple model for forecasting the daily realized volatility was proposed by Corsi (2009). The
heterogeneous autoregressive model of realized volatility (HAR-RV) is defined as
RV (d)
t+1 =c+β(d)RV (d)
t+β(w)RV (w)
t+β(m)RV (m)
t+wt+1,
where RVtis the daily realized volatility and RV (w)
tand RV (m)
tare weekly and monthly
averages:
RV (w)
t=1
5RV (d)
t+RV (d)
t1+... +RV (d)
t4
RV (m)
t=1
20 RV (d)
t+RV (d)
t1+... +RV (d)
t19,
where we assume that a week has 5 days, and a month has 4 weeks. This model is a special
case of a MIDAS regression:
RV (d)
t+1 =c+
19
X
j=0
βjRV (d)
tj+wt+1,
where
βj=
β(d)+1
5β(w)+1
20 β(m),for j= 0,
1
5β(w)+1
20 β(m),for j= 1, .., 4,
1
20 β(m),for j= 5, ..., 19.
The corresponding R code is the following
harstep <- function(p,d,m) {
if (d != 20)
stop("HAR(3)-RV process requires 20 lags")
out <- rep(0,20)
out[1]<- p[1]+p[2]/5+p[3]/20
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 29
out[2:5]<- p[2]/5+p[3]/20
out[6:20]<- p[3]/20
out
}
For empirical demonstration we use the realized variance data on stock indices provided by
Heber, Lunde, Shephard, and Sheppard (2009).
We estimate the model for the annualized realized volatility of the S&P500 index, which is
based on 5-minute returns data.
load("data/spx2.RData")
spx2.rvol <- na.omit(100 *sqrt(252 *SPX2))
mh <- midas_r(rv ~mls(rv, 1:20,1, harstep), data =list(rv = spx2.rvol),
start =list(rv =c(1,1,1)))
summary(mh)
##
## Formula rv ~ mls(rv, 1:20, 1, harstep)
##
## Parameters:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.8300 0.3645 2.28 0.02283 *
## rv1 0.3407 0.0446 7.63 3.0e-14 ***
## rv2 0.4113 0.0693 5.93 3.3e-09 ***
## rv3 0.1932 0.0508 3.80 0.00015 ***
## ---
## Signif. codes: 0 '***'0.001 '**'0.01 '*'0.05 '.'0.1 ' ' 1
##
## Residual standard error: 5.56 on 3434 degrees of freedom
For comparison we also estimate the model with the normalized exponential Almon weights
mr <- midas_r(rv ~mls(rv, 1:20,1, nealmon), data =list(rv = spx2.rvol),
start =list(rv =c(0,0,0)), user.gradient =TRUE)
summary(mr)
##
## Formula rv ~ mls(rv, 1:20, 1, nealmon)
##
## Parameters:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.8376 0.3776 2.22 0.027 *
## rv1 0.9447 0.0278 34.04 < 2e-16 ***
## rv2 -0.7683 0.0961 -7.99 1.8e-15 ***
## rv3 0.0291 0.0056 5.19 2.2e-07 ***
## ---
30 R package midasr
## Signif. codes: 0 '***'0.001 '**'0.01 '*'0.05 '.'0.1 ' ' 1
##
## Residual standard error: 5.54 on 3434 degrees of freedom
We can test which of these restrictions is compatible with the data using the heteroscedasticity
and autocorrelation robust weight specification test hAhr.test.
hAhr.test(mh)
##
## hAh restriction test (robust version)
##
## data:
## hAhr = 28.07, df = 17, p-value = 0.04413
hAhr.test(mr)
##
## hAh restriction test (robust version)
##
## data:
## hAhr = 19.27, df = 17, p-value = 0.3132
We can see that the null hypothesis pertaining to the HAR-RV-implied constraints in the
MIDAS regression model is rejected at the 0.05 significance level, whereas the null hypothesis
that the exponential Almon lag restriction is adequate, cannot be rejected.
Figure 4.2 illustrates the coefficients of the fitted MIDAS regressions and the coefficients of
U-MIDAS regression with their corresponding 95% confidence bounds.
For the exponential Almon lag specification we can choose the number of lags via AIC or
BIC.
tb <- expand_weights_lags("nealmon",from=1,to=c(5,15),
start=list(nealmon=c(0,0,0)))
mtb <- midas_r_ic_table(rv~mls(rv,1:20,1,nealmon),
data=list(rv=spx2.rvol),table=list(rv=tb),
test="hAh.test",
user.gradient=TRUE,showprogress=FALSE)
mtb$candlist <- lapply(mtb$candlist,midas_r,Ofunction="nls")
mtb$test <- "hAhr.test"
mtb <- midas_r_ic_table(mtb)
Here we used two optimization methods to improve the convergence. The function midas_r_ic_table
applies the test function for each candidate model. The function hAhr.test takes a lot of
computing time, especialy for models with larger number of lags, so we calculate it only for
the second final step, and we restrict the number of lags choose from. The AIC selects the
model with 9 lags:
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 31
0 5 10 15
0.0 0.2 0.4
h
Beta coefficients
Figure 4: Comparison of HAR-RV (blue), Nealmon (green) and U-MIDAS (black) models.
bm <- modsel(mtb)
##
## Selected model with AIC = 21547
## Based on restricted MIDAS regression model
## The p-value for the null hypothesis of the test hAhr.test is 0.5532
##
## Formula rv ~ mls(rv, 1:9, 1, nealmon)
##
## Parameters:
## Estimate Std. Error t value Pr(>|t|)
## (Intercept) 0.9610 0.3695 2.60 0.0093 **
## rv1 0.9371 0.0273 34.33 < 2e-16 ***
## rv2 -1.1923 0.1929 -6.18 7.1e-10 ***
## rv3 0.0966 0.0219 4.41 1.1e-05 ***
## ---
## Signif. codes: 0 '***'0.001 '**'0.01 '*'0.05 '.'0.1 ' ' 1
##
## Residual standard error: 5.52 on 3439 degrees of freedom
The HAC robust version of hAh.test again does not reject the null hypothesis of the expo-
nential Almon lag specifications.
We can look into the forecast performance of both models, using a rolling forecast with 1000
observation window. For comparison we also calculate the forecasts of an unrestricted AR(20)
model.
32 R package midasr
ar20 <- midas_r(rv~mls(rv,1:20,1),data=list(rv=spx2.rvol),start=NULL)
forc <- average_forecast(list(ar20,mh,bm),
data=list(rv=spx2.rvol),
insample=1:1000,outsample=1001:1100,
type="rolling",showprogress=FALSE)
forc$accuracy$individual
## Model MSE.out.of.sample MAPE.out.of.sample
## 1 rv ~ mls(rv, 1:20, 1) 10.83 26.60
## 2 rv ~ mls(rv, 1:20, 1, harstep) 10.46 25.93
## 3 rv ~ mls(rv, 1:9, 1, nealmon) 10.34 25.90
## MASE.out.of.sample MSE.in.sample MAPE.in.sample MASE.in.sample
## 1 0.8200 28.62 21.57 0.8334
## 2 0.8020 29.25 21.59 0.8367
## 3 0.7944 29.08 21.81 0.8401
We see that exponential Almon lag model slightly outperforms the HAR-RV model and both
models outperform the AR(20) model.
5. Final remarks
Only a part of the available functionality of the discussed functions of the package midasr was
discussed. As it is usual in R, much more information on the resulting objects and all the infor-
mation on the package-specific functions can be retrieved the using generic functions objects
and ?, respectively. Furthermore, in order to save space, the coding examples provided were
almost always presented with minimal accompanying output obtained after running the code.
The package page contains all the codes and complete output together with some additional
illustration of the functionality of the package. Other information with a list of the functions
and a number of demonstration codes is accessible using the usual ??midasr.
6. Appendix
The figure 1was created using Monte-Carlo simulation. The following DGP was used
yt= 2 + 0.1t+
16
X
j=0
βjz12tj+ut, zτN(0, σ2), utN(0, σ2),
where zτand utare independent.
The coefficients βjwere chosen to come from the normalized exponential Almon polynomial
restriction:
nealmon(p=c(2,0.5,-0.1), d=17)
The data for this DGP was generated for low frequency sample sizes 50, 100, 200, 300, 500,
750 and 1000. For each sample size an additional out-of-sample data set was generated using
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 33
a quarter of the size of an in-sample data set. Three MIDAS regression models were estimated
using in-sample data set: an unrestricted MIDAS, a restricted one using the correct constraint
from the DGP, and the one with an incorrect restriction (non-exponential Almon polynomial).
The forecast was calculated using the out-of-sample data-set. The Euclidean distance between
the model coefficients and the coefficients of the DGP was recorded together with the mean
squared error of the forecast.
This process was repeated 1000 times. The points in the figure are the averages of the
replications. The full code can be found on the website of package midasr.
References
Andreou E, Ghysels E, Kourtellos A (2010). “Regression Models With Mixed Sampling Fre-
quencies.” Journal of Econometrics,158, 246–261.
Andreou E, Ghysels E, Kourtellos A (2011). “Forecasting with mixed-frequency data.” In
C M, D Hendry (eds.), “Oxford Handbook of Economic Forecasting,” pp. 225–245.
Andreou E, Ghysels E, Kourtellos A (2013). “Should macroeconomic forecasters look at daily
financial data?” Journal of Business and Economic Statistics,31, 240–251.
Armesto M, Engemann K, Owyang M (2010). “Forecasting with mixed frequencies.” Federal
Reserve Bank of St. Louis Review,92, 521–536.
Bai J, Ghysels E, Wright J (2012). “State Space Models and MIDAS Regressions.” Econo-
metric Reviews (forthcoming).
Bilinskas B, Kvedaras V, Zemlys V (2013). “Testing the functional constraints on parameters
in cointegrated MIDAS regressions. Working paper, available upon a request.
Bilinskas B, Zemlys V (2013). “Testing the Functional Constraints on Parameters in Re-
gression Models with Cointegrated Variables of Different Frequency.” Submitted, available
upon a request.
Breitung J, Roling C, Elengikal S (2013). “The statistical content and empirical testing of the
MIDAS restrictions.” Working paper, URL http://www.ect.uni-bonn.de/mitarbeiter/
joerg-breitung/npmidas.
Clements M, Galv˜ao A (2008). “Macroeconomic Forecasting with Mixed Frequency Data:
Forecasting US output growth.” Journal of Business and Economic Statistics,26, 546–554.
Corsi F (2009). “A simple approximate long-memory model of realized volatility.” Journal of
Financial Econometrics,7, 174–196.
Foroni C, Marcellino M, Schumacher C (2014). “U-MIDAS: MIDAS regressions with unre-
stricted lag polynomials.” Journal of the Royal Statistical Society ˆa ˘
S Series A (forthcom-
ing).
Ghysels E (2013). “Matlab Toolbox for Mixed Sampling Frequency Data Analysis using
MIDAS Regression Models.” Available on Matlab Central at http://www.mathworks.
com/matlabcentral/fileexchange/45150-midas-regression.
34 R package midasr
Ghysels E, Santa-Clara P, Valkanov R (2002). “The MIDAS touch: Mixed data sampling
regression models.” Working paper, UNC and UCLA.
Ghysels E, Santa-Clara P, Valkanov R (2006a). “Predicting volatility: getting the most out
of return data sampled at different frequencies.” Journal of Econometrics,131, 59–95.
Ghysels E, Sinko A, Valkanov R (2006b). “MIDAS regressions: Further results and new
directions.” Econometric Reviews,26, 53–90.
Ghysels E, Valkanov R (2012). “Forecasting volatility with MIDAS.” In L Bauwens, C Hafner,
S Laurent (eds.), “Handbook of Volatility Models and Their Applications,” pp. 383–401.
Wiley.
Heber G, Lunde A, Shephard N, Sheppard K (2009). “Oxford-Man Institute’s realized library.”
Oxford-Man Institute, University of Oxford. Library Version 0.2.
Kvedaras V, Raˇckauskas A (2010). “Regression Models with Variables of Different Frequencies:
The Case of a Fixed Frequency Ratio.” Technical report,Oxford Bulletin of Economics and
Statistics, forthcoming.
Kvedaras V, Zemlys V (2012). “Testing the functional constraints on parameters in regressions
with variables of different frequency.” Economics Letters,116(2), 250–254.
Kvedaras V, Zemlys V (2013). “The statistical content and empirical testing of the MIDAS
restrictions.” Submitted, available upon a request.
Rodriguez A, Puggioni G (2010). “Mixed frequency models: Bayesian approaches to estima-
tion and prediction.” International Journal of Forecasting,26, 293–311.
Wohlrabe K (2009). “Forecasting with mixed-frequency time series models.” Technical report,
Ph. D. Dissertation Ludwig-Maximilians-Universitat Munchen.
Zeileis A (2004). “Econometric Computing with HC and HAC Covariance Matrix Esti-
mators.” Journal of Statistical Software,11(10), 1–17. ISSN 1548-7660. URL http:
//www.jstatsoft.org/v11/i10.
Affiliation:
Eric Ghysels
Department of Economics
University of North Carolina - Chapel Hill
Gardner Hall, CB 3305 Chapel Hill, NC 27599-3305
E-mail: eghysels@unc.edu
URL: http://www.unc.edu/~eghysels/
Virmantas Kvedaras
Department of Econometric Analysis
Faculty of Mathematics and Informatics
Vilnius University
Eric Ghysels, Virmantas Kvedaras, Vaidotas Zemlys 35
Naugarduko g. 24, Vilnius, Lithuania
E-mail: virmantas.kvedaras@mif.vu.lt
Vaidotas Zemlys
Department of Econometric Analysis
Faculty of Mathematics and Informatics
Vilnius University
Naugarduko g. 24, Vilnius, Lithuania
E-mail: vaidotas.zemlys@mif.vu.lt
URL:http://mif.vu.lt/~zemlys/
... The most typical machine learning techniques for building nowcast models are the Kalman filter [16], [17], regression on Mixed Data Sampling (MIDAS) datasets [20], Bridge Equation (BE) regression [18], Economic Random Forest Regression [13], and deep neural network learning [12]. Among deep-learning techniques, the LSTM networks are superior [6], [12]. ...
... The factor MIDAS model under consideration is [20], [33]: ...
... There are many ways to constrain the θ τ coefficients; therefore, many restricted MIDAS models exist. The following sections introduce some of the most commonly used restricted MIDAS models for practical applications [20]. ...
Article
Full-text available
Nowadays nowcasts are closely related to big data. Currently, the most popular nowcast model-building approach is to use the factor bridge equation (BE) model or factor mixed data sampling (MIDAS) model, where the factors are extracted from large datasets using the principal component analysis (PCA) method. However, the PCA method only suits datasets approximating a hyperplane and real-world datasets are sometimes unsuitable. This study proposes a kernel-based dimension-reduction method. It is a natural extension of the PCA method and is called KTPCA. The KTPCA can reduce the dimensionality of datasets without approximating a hyperplane. The dimension reduction performance of the iterative KTPCA method was superior to that of the PCA, Sparse PCA (SPCA), Randomized SPCA (RSPCA), and Robust SPCA (ROBSPCA) methods. In addition, PCA and SPCA methods are competitive in terms of dimension reduction performance. This study also proposed a nowcasting procedure using the iterative KTPCA method. This procedure can be applied to build nowcast models and update forecasts of a target variable at a low frequency under real-time data flows of original variables at a higher frequency.
... 3.2, three modified penalized least squares methods (Lasso-RAR, Ridge-RAR and EN-RAR) and JMA-RAR in Sect. 3.3, Dynamic Factor Model (DFM, De Valk et al. (2019)) and Mixed Data Sampling (MIDAS, Ghysels et al. (2016)). ...
Article
Full-text available
Real-time nowcasting plays a crucial role in the fields of economics and finance by providing timely information, improving decision-making, supporting policy formulation, and assisting financial transactions. It holds significant importance in understanding and adapting to economic changes. The official release of chinese GDP data typically experiences a certain time delay and requires some time to acquire and publish. Nowcasting GDP can provide more timely economic indicator forecasts, bridging the information gap caused by the lag in data release. Conventional penalized least squares methods yield satisfactory nowcasting results when fitting GDP data, but they neglect the influence of residual autocorrelation. In this paper, we propose several nowcasting methods for monthly chinese GDP with mixed frequency data. We demonstrate that our proposed method outperform the conventional penalized methods in nowcasting chinese GDP. Furthermore, after removing residual autocorrelation, the JMA method has the smallest RMSE and achieves the best nowcasting performance.
... It is worth noting that, since the ADL model can not include the lead information of high-frequency monthly data, we only examined the ADL model under four forecast horizons without leads. Fig 3 shows the empirical design process of the PSO-MIDAS flowchart, the algorithm programming is carried out via The MIDAS toolbox for R (midasr package) [26]. Regarding the PSO algorithm's specific hyperparameter setting, we refer to the standard PSO algorithm's settings of related hyperparameters in the Standard Particle Swarm Optimisation 2011 (SPSO 2011) proposed by Clerc et al. [27]. ...
Article
Full-text available
In this study, the traditional lag structure selection method in the Mixed Data Sampling (MIDAS) regression model for forecasting GDP was replaced with a machine learning approach using the particle swarm optimization algorithm (PSO). The introduction of PSO aimed to automatically optimize the MIDAS model’s mixed-frequency lag structures, improving forecast accuracy and resolving the "forecast accuracy" and "forecast cost" weighting problem. The Diebold–Mariano test results based on U.S. macroeconomic data show that when the forecast horizon is large, the forecast accuracy of the PSO-MIDAS model is significantly better than other benchmark models. Empirical results show that, compared to the benchmark MIDAS model, the forecast accuracy of both univariate and multivariate PSO-MIDAS models improves by an average of 10% when the forecast horizon exceeds 2 quarters, and the optimization effect is greater compared to other benchmark models. The innovative use of the PSO algorithm addresses the limitations of traditional lag structure selection methods and enhances the predictive potential of the MIDAS model.
... This can lead to nonlinearity in terms of parameters. As a result, in the general situation of estimating the MIDAS model, the non-linear least squares (NLS) method can be used to estimate the parameters. is used as follows (Eric et al., 2016): ...
Article
Full-text available
Objective: This Article aims to investigate the impact of oil price fluctuations on economic growth in Algeria using the Mixed Data Sampling (MIDAS) model during the period 1990-2022 Theoretical Framework: The theoretical aspect addressed the different time-frequency models by defining the Midas models and addressing the weighting functions used in estimating these models. Method: Estimating the relationship between a high-frequency variable (oil price) and a low-frequency variable (economic growth rate GDP) using the MIDAS model using Almon's PDL Weighting Function. Results and Discussion: The results of the study concluded economic growth in Algeria is directly affected by oil prices, The study also found that MIDAS models have a great predictive ability, allowing researchers to utilize all available information for high-frequency variables, instead of using annual average values that lose a lot of information. avoid converting low-frequency variables to match high-frequency variables, reducing econometric issues in model diagnostics. Research Implications: Research Implications: It is suggested that rentier countries such as Algeria work to diversify their economy away from oil, and follow up the process of economic diversification and accelerate it during periods of high fuel prices by working to develop non-strategic sectors as agriculture, tourism and manufacturing industries. Originality/Value: These results are expected to benefit policymakers in government, by urging them to invest in renewable energies as a crucial strategy for achieving sustainable development and a key factor in mitigating oil price shocks, particularly during recent recurring global crises. Future research directions include exploring existing gaps and promoting standardized studies.
Article
Purpose This study delves into Bitcoin’s return dynamics to address its pronounced volatility, particularly in extreme market conditions. We analyze a broad range of explanatory variables, including traditional financial indicators, innovative cryptocurrency-specific metrics and market sentiment gauges. We uniquely introduce the Conference Board Leading Economic Indicator (LEI) to the cryptocurrency research landscape. Design/methodology/approach We employ quantile regression to examine Bitcoin’s daily and monthly returns. This approach captures timescale dependencies and evaluates the consistency of our findings across different market conditions. By conducting a thorough analysis of the entire return distribution, we aim to reveal how various factors influence Bitcoin’s behavior at different risk levels. The research incorporates a comprehensive set of explanatory variables to provide a holistic view of Bitcoin’s market dynamics. Additionally, by segmenting the study period, we assess the consistency of the results across diverse market regimes. Findings Our results reveal that factors driving Bitcoin returns vary significantly across market conditions. For instance, during downturns, an increase in transaction volume is linked to lower Bitcoin returns, potentially indicating panic selling. When the market stabilizes, a positive correlation emerges, suggesting healthier ecosystem activity. Active addresses emerge as a key predictor of returns, especially during bearish phases, and sentiment indicators such as Wikipedia views reveal shifting investor optimism, depending on market trends. Monthly return analysis suggests Bitcoin might act as a hedge against traditional markets due to its negative correlation with the S&P 500 during normal conditions. Practical implications The study’s findings have significant implications for investors and policymakers. Understanding how different factors influence Bitcoin returns in varying market conditions can guide investment strategies and regulatory approaches. Originality/value A novel contribution of this study is the identification of Bitcoin’s sensitivity to broader economic downturns as demonstrated by the negative correlation between LEI and returns. These insights not only deepen our understanding of Bitcoin market behaviour but also offer practical implications for investors, risk managers and policymakers navigating the evolving cryptocurrency landscape.
Article
Full-text available
Mixed-data sampling (MIDAS) regressions allow to estimate dynamic equations that explain a low-frequency variable by high-frequency variables and their lags. When the difference in sampling frequencies between the regressand and the regressors is large, distributed lag functions are typically employed to model dynamics avoiding parameter proliferation. In macroeconomic applications, however, differences in sampling frequencies are often small. In such a case, it might not be necessary to employ distributed lag functions. In this paper, we discuss the pros and cons of unrestricted lag polynomials in MIDAS regressions. We derive unrestricted MIDAS regressions (U-MIDAS) from linear high-frequency models, discuss identification issues, and show that their parameters can be estimated by OLS. In Monte Carlo experiments, we compare U-MIDAS to MIDAS with functional distributed lags estimated by NLS. We show that U-MIDAS generally performs better than MIDAS when mixing quarterly and monthly data. On the other hand, with larger differences in sampling frequencies, distributed lag-functions outperform unrestricted polynomials. In an empirical application on out-of-sample nowcasting GDP in the US and the Euro area using monthly predictors, we find a good performance of U-MIDAS for a number of indicators, albeit the results depend on the evaluation sample. We suggest to consider U-MIDAS as a potential alternative to the existing MIDAS approach in particular for mixing monthly and quarterly variables. In practice, the choice between the two approaches should be made on a case-by-case basis, depending on their relative performance. --
Article
Full-text available
The paper proposes an additive cascade model of volatility components defined over different time periods. This volatility cascade leads to a simple AR-type model in the realized volatility with the feature of considering different volatility components realized over different time horizons and thus termed Heterogeneous Autoregressive model of Realized Volatility (HAR-RV). In spite of the simplicity of its structure and the absence of true long-memory properties, simulation results show that the HAR-RV model successfully achieves the purpose of reproducing the main empirical features of financial returns (long memory, fat tails, and self-similarity) in a very tractable and parsimonious way. Moreover, empirical results show remarkably good forecasting performance.
Article
We propose a test for the evaluation of statistical acceptability of a functional constraint which is imposed on parameters in the mixed data sampling regressions. The asymptotic behavior of the test statistic is characterized and a few other extensions are discussed.
Article
We consider various MIDAS (Mixed Data Sampling) regression models to predict volatility. The models differ in the specification of regressors (squared returns, absolute returns, realized volatility, realized power, and return ranges), in the use of daily or intra-daily (5-minute) data, and in the length of the past history included in the forecasts. The MIDAS framework allows us to compare models across all these dimensions in a very tightly parameterized fashion. Using equity return data, we find that daily realized power (involving 5-minute absolute returns) is the best predictor of future volatility (measured by increments in quadratic variation) and outperforms model based on realized volatility (i.e. past increments in quadratic variation). Surprisingly, the direct use of high-frequency (5-minute) data does not improve volatility predictions. Finally, daily lags of one to two months are sucient to capture the persistence in volatility. These findings hold both in- and out-of-sample.
Article
In the present work we propose a new realized volatility model to directly model and forecast the time series behavior of volatility. The purpose is to obtain a conditional volatility model based on realized volatility which is able to reproduce the memory persistence observed in the data but, at the same time, remains parsimonious and easy to estimate. Inspired by the Heterogeneous Market Hypothesis and the asymmetric propagation of volatility between long and short time horizons, we propose an additive cascade of different volatility components generated by the actions of different types of market participants. This additive volatility cascade leads to a simple AR-type model in the realized volatility with the feature of considering volatilities realized over different time horizons. We term this model, Heterogeneous Autoregressive model of the Realized Volatility (HAR-RV). In spite of the simplicity of its structure, simulation results seem to confirm that the HAR-RV model successfully achieves the purpose of reproducing the main empirical features of financial data (long memory, fat tail, self-similarity) in a very simple and parsimonious way. Preliminary results on the estimation and forecast of the HAR-RV model on USD/CHF data, show remarkably good out of sample forecasting performance which steadily and substantially outperforms those of standard models.
Article
We examine the relationship between MIDAS regressions and Kalman filter state space models applied to mixed frequency data. In general, the latter involves a system of equations, whereas in contrast MIDAS regressions involve a (reduced form) single equation. As a consequence, MIDAS regressions might be less efficient, but also less prone to specification errors. First we examine how MIDAS regressions and Kalman filters match up under ideal circumstances, that is in population, and in cases where all the stochastic processes -low and high frequency -are correctly specified by a linear state space model. We characterize cases where the MIDAS regression exactly replicates the steady state Kalman filter weights. In cases where the MIDAS regression is only an approximation, we compute the approximation error and find it to be small (using two different metrics). Both in population and in small samples, we find that forecasts from MIDAS regressions are generally quite similar to those from the Kalman filter. Kalman filter forecasts are typically a little better, but MIDAS regressions can be more accurate if the state-space model is mis-specified or over-parameterized. The paper concludes with an empirical application comparing MIDAS and Kalman filtering to predict future GDP growth, using monthly macroeconomic series.
Article
We introduce easy to implement regression-based methods for predicting quarterly real economic activity that use daily financial data. Our analysis is designed to elucidate the value of daily information and provide real-time forecast updates of the current (nowcasting) and future quarters. Our findings show that while on average the predictive ability of all models worsens substantially following the financial crisis, the models we propose suffer relatively less losses. Moreover, these predictive gains are primarily driven by the asset classes of government securities, equities, and especially corporate risk.