ArticlePDF Available

What do we know about G-7 macro forecasts

Authors:

Abstract

Fildes and Stekler’s (2002) survey of the state of knowledge about the quality of economic forecasts focused primarily on US and UK data. This paper will draw on some of their findings but it will not examine any additional US forecasts. The purpose is to determine whether their results are robust by examining the predictions of other countries. The focus will be on (1) directional errors, (2) the magnitude of the errors made in estimating growth and inflation, (3) whether there were biases and systematic errors, (4) the sources of the errors and (5) whether there has been an improvement in forecasting ability.
Research Program on Forecasting (RPF) Working Papers represent preliminary work circulated for
comment and discussion. Please contact the author(s) before citing this paper in any publications. The
views expressed in RPF Working Papers are solely those of the author(s) and do not necessarily represent
the views of RPF or George Washington University.
What Do We Know About G-7 Macro Forecasts?
Herman O. Stekler
Department of Economics
The George Washington University
Washington, DC 20052
Tel: 202-994-6150
Fax: 202-994-6147
hstekler@gwu.edu
RPF Working Paper No. 2008-009
http://www.gwu.edu/~forcpgm/2008-009.pdf
August 18, 2008
RESEARCH PROGRAM ON FORECASTING
Center of Economic Research
Department of Economics
The George Washington University
Washington, DC 20052
http://www.gwu.edu/~forcpgm
  1
Was wissen wir über die makroökonomischen
Vorhersagen für die Länder der G 7?
What Do We Know About G-7 Macro Forecasts?
Von Herman O. Stekler1
Fildes & Stekler’s (2002) survey of the state of knowledge about the quality of economic
forecasts focused primarily on US and UK data. This paper will draw on some of their
findings but it will not examine any additional US forecasts. The purpose is to determine
whether their results are robust by examining the predictions of other countries. The focus
will be on (1) directional errors, (2) the magnitude of the errors made in estimating growth
and inflation, (3) whether there were biases and systematic errors, (4) the sources of the
errors and (5) whether there has been an improvement in forecasting ability. It appears
that few of the papers that have analyzed any of the G7 forecasts have examined all of
these issues. A notable exception is a paper coauthored by Ullrich Heilemann, whom
we are honoring.2 However, a number of the papers that are cited here provided data
that give us a greater understanding of the forecasting process. Some the forecasts were
issued by international organizations such as the OECD and IMF. The others come from
institutions or from private forecasters as reported in Consensus Forecasts.
The next section examines directional errors followed by a discussion of the magni-
tude of the quantitative discrepancies. The third section questions whether the accuracy
of forecasts has improved. The subsequent sections discuss the existence of bias in the
various G7 countries and the sources of the errors. The final section summarizes what
we have learned about forecasting in the G7 countries from the evaluation studies that
have been summarized here.
I Directional Errors
There is only limited information about forecasters’ ability to predict turning points because
most evaluations focused on the magnitude of the errors. Fildes & Stekler indicated that
forecasters were not able to forecast the cyclical turning points in the UK economy and
were only partially successful in identifying a turn when it occurred. The inflation surges
of 1979-80 and 1989-90 were also not predicted until a number of months after prices
had started rising rapidly; a similar result was observed when inflation tapered off.
As for other countries, the turning points in German GDP were not predicted. However,
the accelerations and decelerations of GDP but not of inflation were forecast accurately.
1 Prof. Herman O. Stekler, Research Professor of Economics, George Washington University, 2201 G
Street, NW, Washington, DC 20052 USA, hstekler@gwu.edu. I would like to thank Ulrich Fritsche
for his comments on an earlier draft.
2 Heilemann & Stekler (2003).
2  Herman O. Stekler
(Heilemann & Stekler, 2003; Dopke & Fritsche, 2006a). Loungani (2001) indicates that
there were 60 cyclical turns in his sample but private forecasters forecast only three of
them a year in advance.3 The evidence is similar for the forecasts of research institutes and
international organizations.4 (Ash et al.,1998; Öller & Barot, 2000; Blix et al., 2001; Vogel,
2007). In fact, Ashiya (2003) shows that the IMF’ forecasts of the direction of change
GDP made 15 months in advance are not significantly related to the actual changes. Going
even further, Vuchelen & Guitierrez (2005b) found that the 24 month OECD forecasts
for the G7 countries were worthless.5 The preponderance of the evidence is that most
forecasters are not able to predict turning points in advance and even a suggestion that
they are not detected quickly.
II Magnitude of the Errors
Because most evaluations focus on the magnitude of the errors, there is considerable in-
formation that permits us to synthesize the results. Fildes & Stekler (2002) reported that
the Mean Absolute Error (MAE) of the UK annual GDP forecasts was around 1%, but
Öller & Barot (2000) found that the errors were larger for some of the other European
countries. Newer studies found similar results: the MAE of the GDP forecasts for the G7
countries was around 1% and the MAE of the one year-ahead inflation predictions was
as low as 0.33% (Mestre, 2007), but others found it to be between 0.5% and 0.75% (
Loungani, 2001; Heilemann & Stekler, 2003; Ashiya, 2006; Dopke & Fritsche, 2006a;
Isiklar & Lahiri, 2007; Bowles et al., 2007; Timmermann, 2007).6 The universality of
these results suggests, that given present knowledge and procedures, we cannot expect
to achieve higher levels of accuracy.7
III Have the Forecasts Improved?
Given the number of papers that have evaluated forecasts, it is surprising how few have
asked whether the quality of the forecasts has improved over time. There are sufficient
data because we have 40 years worth of forecasts for some countries. The evidence from
a limited number of analyses is not definitive. Heilemann & Stekler (2003) examined Ger-
man forecasts and adjusted the errors for the difficulty in forecasting the relevant periods.
3 Loungani did not distinguish between the cyclical turns of developed and underdeveloped coun-
tries.
4 Batchelor (2001) found that the private sector forecasts were superior to those of the international
organizations, but Boweles et al (2007) indicated that the ECB SPF forecasts were comparable to
those of the private Consensus Forecasts.
5 But same authors (Vuchelen and Guitierrez, 2005a) and other studies had found that the one year
forecasts were useful.
6 Unfortunately, Campbell & Murphy (2006) showed that the range of the GDP forecasts made by
Canadian economists who provide information to the government encompassed the actual GDP
numbers only 10-35% of the time depending upon the sample period. The inflation rate forecasts
were better: the actual value was within the range of the forecasts 60% of the time.
7 An appropriate procedure would be to compare these errors to the mean absolute change or the
volatility of the series.
Was wissen wir über die makroökonomischen Vorhersagen für die G7-Länder?  3
The results were mixed. Dopke & Fritsche (2006a) surveyed previous studies of German
forecasts which had differing results but they suggest that accuracy may have improved.
Vogel (2007) shows that the RMSE of the OECD forecasts had declined over time, but
these statistics had not been adjusted for the changes in the volatility of the economies.
On the other hand, Timmerman (2007) indicates that the quality of the IMF forecasts had
declined since they had been last evaluated in the 1990s. Earlier Fildes & Stekler (2002)
had summarized the conclusions about US forecasts and found mixed results. Despite all
the efforts that have been devoted to forecasting, there is no clear evidence that accuracy
has improved. We need to ask why? Stock and Watson (2007) provided one explanation.
The decline in volatility has made it harder to forecast inflation.8
IV Bias, Rationality, and Systematic Errors
There are a number of possible explanations for the existence of these errors. While they
might be the lowest achievable errors, there are at least two alternative explanations. If
there were symmetric loss functions, the errors might be biased and thus irrational. Al-
ternatively, individuals might have asymmetric loss functions that explain the existence
of these errors. (Elliot & Timmermann, 2008). Most evaluations have focused on the
possibility that the forecasts were biased.
Many studies have indicated that forecasters make systematic errors and are biased.
(Fildes & Stekler, 2002). These biases consist of overestimates of growth rates during
slowdowns and recessions and underestimates during recoveries and booms. Similar
systematic errors are observed with regard to inflation forecasts: overpredictions occur
when inflation is low and underpredictions are observed when inflation is high.
Among non-US and UK studies cited by Fildes & Stekler, the IMF end of year forecasts
were rational for all countries Pons (2001) and the six month-ahead predictions of Ger-
man research institutes were rational while the year-ahead were not Kirchgassner (1993).
The OECD predictions did not include all available information (Ash et al. 1990) and
tended to underestimate changes in the trends of both real output growth and inflation.
(Anderson, 1997).9 As shown below, the newer studies do not reach a single definitive
conclusion.
Consensus Forecasts: Loungani (2001) found that the year-ahead Consensus Forecasts
of GDP had an optimistic bias that had diminished by the end of the year for which the
predictions were made. More recently and examining a longer time period, Batchelor
(2007) and Ager et al. (2007) reach different conclusions regarding the predictions of
GDP for the G7 countries. Batchelor finds that there is a difference between the US and
other G7 forecasters. The latter have large biases that diminish only gradually. Ager et
al. used the pooled method of Clements et al. (2007) and found that over all only the
Italian growth forecasts were biased. For other countries there were biases at some hori-
8 On the other hand, if individuals have expectations that are anchored on an inflation target, that
forecast may be sufficiently accurate and no further improvement in accuracy should be expected.
Secondly, the Great Moderation may mean that small forecast errors, while statistically significant
are not economically meaningful. However, the economic events of 2007-2008 may signify the end
of the Great Moderation.
9 However, in periods when inflation was decelerating, the rate of inflation was overpredicted.
4  Herman O. Stekler
zons and some of the revisions were predictable. The two studies agree that the inflation
predictions were not biased.
Canadian Forecasts: Some Canadian forecasters are systematically optimistic or pes-
simistic. (Campbell & Murphy, 2006).
Eurozone Forecasts: The European Central Bank has selected a group of individuals
to prepare forecasts for the ECB. The evidence indicates that inflation forecasts made one
and two years in advance are both biased: the rate of inflation is underpredicted. Only the
two-year-ahead GDP estimates are biased. (Bowles et al., 2007; Garcia & Manzanares,
2007).
German Forecasts: Heilemann & Stekler (2003) indicated that there were systematic
errors in German predictions of inflation but that these errors were not observed in the
GDP forecasts. Dopke & Fritsche (2006a) agreed that the German GDP forecasts were
unbiased, but argued that the inflation forecasts were unbiased but inefficient.
Japanese Forecasts: Ashiya (2006) found that some Japanese institutions were always
either optimistic or pessimistic. Moreover, they frequently overreact to new information,
make excessive revisions and thus are not rational. (Ashiya, 2003).
UK Forecasts: In order to provide more information about the probabilities of specific
rates of inflation, the Bank of England issues both a point forecast and also provides a fan
chart. The NIESR now also publishes a fan chart. The evidence indicates that the Bank’s
point forecasts are unbiased. (Wallis, 2003; Clements, 2004; Mitchell & Hall, 2005).
However, the Bank’s probability forecasts are biased because of the exaggerated concern
with upside risks but better than the appropriate benchmark. The Bank of England also
conducts a survey of professional forecasters. The evidence from that survey indicates
that the professional forecasters’ uncertainty about the rate of inflation has declined but
they also may not have noticed that the persistence of UK inflation had declined. (Boero
et al, 2008).
International Agency Forecasts: Timmermann (2007) showed that the IMF forecasts
of both growth and inflation for the G7 countries were significantly biased. Economic
growth was overpredicted for all countries except the US while inflation was also over
predicted for four of the G7 countries. These forecasts are similar in quality to those made
by Consensus Forecasts, but the current year IMF forecasts are slightly less biased than
the others. Vogel’s (2007) analysis of the OECD found that the current year forecasts were
not biased, but that the year-ahead predictions were biased. They were overly optimistic
when growth was slowing down and were more biased than the Consensus estimates.
These findings are diverse making it hard to generalize, but there are two common
themes: (1) forecaster beliefs are persistent and make consistent over or under estimates
(Isiklar and Lahiri( 2008), and (2) they use information inefficiently Kirchgassner (1993)
and Dopke & Fritsche (2006a). Furthermore, there is a large variation among the forecast-
ers, with some perennially optimistic or pessimistic. In fact, their biases exist even at short
horizons. Ashiya (2006) reported this result for Japanese forecasters and Batchelor (2007)
noted this among the Consensus forecasters. This finding can be explained in a number
of ways: (1) the forecasters are irrational and do not learn from their perennial mistakes,
(2) they have asymmetric loss functions or (3) they exhibit this characteristic for strategic
reasons. Additional research is required to test these alternative explanations.10
10 However, it is important to note that weather forecasters have had biases but these systematic errors
were reduced when they had quick feedback on the quality of their predictions.
Was wissen wir über die makroökonomischen Vorhersagen für die G7-Länder?  5
V Sources of Errors
Our discussion has indicated that there are analytic difficulties associated with deter-
mining why the errors occurred. Nevertheless these newer studies have provided some
evidence about the sources of the errors. Heilemann & Stekler (2003) indicated that a
large portion of the German forecast errors were attributable to the failure to predict
recessions. Similarly, Dopke & Fritsche (2006b) found that the variability (dispersion) of
the German GDP forecasts was higher before and during recessions. They surmised that
forecasters must disagree about the importance of shocks and the economy’s response to
these shocks. Bowles et al. (2007) argue that the European SPF underestimates of inflation
were due to asymmetric shocks. All the shocks tended to raise the rate of inflation, and
when the forecasts were adjusted for the impact of these shocks, there was less evidence
of a bias.
VI What Have We Learned?
The evidence about the G7 macro forecasts that has been presented here is very similar to
the findings of Fildes & Stekler that primarily related only to US and UK forecasters. Both
studies found that recessions are not predicted in advance and account for a significant
portion of the quantitative errors. Neither study was able to show that forecast accuracy
has improved and both found that there were systematic errors.
However, the studies that are summarized here provide us with some new insights.
There may be a limit beyond which forecast accuracy cannot be improved. (Heilemann
& Stekler, 2003; Isiklar & Lahiri, 2007). A second important insight is that forecasts
longer than 12-18 months might not be valuable. (Vuchelen & Guitierrez, 2005b; Isiklar
& Lahiri, 2007). While studies found systematic errors, they are now not necessarily
considered biases; they may be attributable to asymmetric loss functions. Finally, we now
have a somewhat better understanding of the forecasting process. Batchelor (2007) showed
how the systematic errors or “bias” was related to the forecasters’ optimism (pessimism)
and conservatism in revising their predictions. He notes that standard rationality tests are
not appropriate if there has been a structural break. The pattern of the errors can then
provide a way of understanding the forecasters’ learning process about the impact of this
structural break. Isiklar & Lahiri (2007) use forecast revisions to explain the behavioral
characteristics of forecasters, i.e. how they react to news and when is news important.11
We have also learned that there is much more work to be done in determining the impor-
tance of asymmetric losses, the sources of biases, the limits of accuracy, etc. Much can
be learned if we undertake more studies, about the sources of error, similar to the paper
published by our honoree. (Heilemann, 2002).
11 In the forecasts made for year t, the most important revisions occur at the end of t-1.
6  Herman O. Stekler
References
Ager. P., Kappler M., and Osterloh, S., 2007. The accuracy and efficiency of the Consensus Forecasts: A
further application and extension of the pooled approach, Centre for European Economic Research,
Discussion Paper No. 07–058.
Anderson, P. S., 1997, Forecast errors and financial developments, Working Paper no 51, Bank for Inter-
national Settlements, Basle, Switzerland.
Ash, J.C.K., Smyth, D.J. and Heravi, S.M., 1990, The accuracy of OECD forecasts of the international
economy, International Journal of Forecasting, 6, 379–392.
Ash, J.C.K., Smyth, D.J. and Heravi, S.M., 1998, Are OECD forecasts useful? A directional analysis,
International Journal of Forecasting, 14, 381–391.
Ashiya, M., 2003, Testing the rationality of Japanese GDP forecasts: the sign of the forecast revision
matters, Journal of Economic Behavior and Organization, 50, 263–269.
Ashiya, M., 2003b, The directional accuracy of 15-months-ahead forecasts made by the IMF, Applied
Economics Letters, 10, 331–333.
Ashiya, M., 2006, Forecast accuracy and product differentiation of Japanese institutional forecasters,
International Journal of Forecasting, 22, 395–401.
Batchelor, R. 2001, How useful are the forecasts of international agencies? The IMF versus the consensus,
Applied Economics, 33, 225–235.
Batchelor, R. 2007, Bias in macroeconomic forecasts, International Journal of Forecasting, 23, 189–
203.
Boero, G., Smith J. and Wallis K. F., 2008, Uncertainty and disagreement: the Bank of England Survey of
External Forecasters, Economic Journal, forthcoming.
Bowles, C., Friz, R., Genre, V., Keany, G., Meylen, A. and Rautanen, T., 2007, The ECB survey of professi-
onal forecasters (SPF), A review after eight years’ experience, ECB Occasional Paper Series No 59.
Campbell, B. and Murphy S., (2006), The recent performance of the Canadian forecasting industry,
Canadian Public Policy, 32, 23–40.
Clements, M. P., 2004, Evaluating the Bank of England density forecasts of inflation, The Economic
Journal, 114, 844–866.
Dopke, J. & Fritsche U., 2006a, Growth and inflation forecasts for Germany, Empirical Economics, 31,
777–798.
Dopke, J. and Fritsche U., 2006b, When do forecasters disagree? An assessment of German growth and
inflation forecast dispersion, International Journal of Forecasting, 22, 125–135.
Elliott, G. and Timmermann, A., 2008, Economic forecasting, Journal of Economic Literature, 46,
3–56.
Fildes, R. and Stekler, H., 2002, The state of macroeconomic forecasting, Journal of Macroeconomics,
24, 435–468.
Garcia, J. A. and Manzanares, A., 2007, Reporting biases and survey results, evidence from European
professional forecasters, European Central Bank Working Paper 836.
Heilemann, U., 2002, Increasing the transparency of macroeconometric forecasts: A report from the
trenches, International Journal of Forecasting, 18, 85–105.
Heilemann, U. and Stekler, H.O. (2003), Has the accuracy of German macroeconomic forecasts improved?,
Discussion paper of the German Research Council’s research centre, 475, 31/03.
Isiklar, G. and Lahiri, K., 2007, How far ahead can we forecast? Evidence from cross-country surveys,
International Journal of Forecasting, 23, 167–187.
Kirchgassner, G., 1993, Testing weak rationality of forecasts with different time horizons, Journal of
Forecasting, 12, 541–558.
Loungani, P., 2001, How accurate are private sector forecasts? Cross-country evidence from Consensus
Forecasts of output growth, International Journal of Forecasting, 17, 419–432.
Mestre, R., 2007, Are survey-based inflation expectations in the Euro area informative? European Central
Bank Working Paper 721.
Mitchell, J. and Hall S. G., 2005, Evaluating, comparing, and combining density forecasts using the KLIC
with an application to the Bank of England and NIESR ‘fan’ charts of inflation, Oxford Bulletin of
Economics and Statistics, 67, Supplement, 995–1033.
Öller, L-E. and Barot, B., 2000, Comparing the accuracy of European GDP forecasts, International Journal
of Forecasting, 16, 293–315.
Was wissen wir über die makroökonomischen Vorhersagen für die G7-Länder?  7
Pons, J., 2001, The rationality of price forecasts: a directional analysis, Applied Financial Economics,
11, 287–290.
Stock, J. H. and Watson, Mark W., 2007, Why has U.S. inflation become harder to forecast? Journal of
Money, Credit and Banking, Supplement 39, 3–33.
Timmerman, A., 2007, An evaluation of the World Economic Outlook, IMF Staff Papers, 54, 1–33.
Vogel, L., 2007. How do the OECD growth projections for the G7 economies perform? OECD Economics
Department Working Papers No. 573.
Vuchelen, J. and Hutierrez, M-I, 2005a, A direct test of the information content of the OECD growth
forecasts, International Journal of Forecasting, 21, 103–117.
Vuchelen, J. and Hutierrez, M-I, 2005b, Do the OECD 24 month horizon growth forecasts for the G7-
countries contain information?, Applied Economics, 37, 855–862.
Wallis, K. F., 2003, Chi-squared tests of interval and density forecasts and the Bank of England’s fan charts,
International Journal of Forecasting, 19, 165–175.
... The other does a similar analysis of the G7 (excluding the US) predictions. (Stekler, 2008). By comparing the forecasts of various countries, we can determine whether the findings are robust. ...
... The turning points in German GDP were not predicted, but the accelerations and decelerations of the growth rate were forecast accurately. (Stekler, 2008, summarizes the literature relating to the forecasts of the G7 countries and indicates that the results apply equally to private forecasters, research institutes and international organizations). The evidence suggests that forecasters are not able to predict turning points in advance and may even have difficulty in detecting them quickly once they have occurred. ...
... Similarly, inflation was underpredicted when it was rising and overpredicted when it was declining. (See the surveys of Fildes and Stekler (2002) and Stekler (2008) for the specific studies from which these results were obtained). Fildes and Stekler concluded: "these errors occurred when the economy was subject to major perturbations, just the time when accurate forecasts were most needed." ...
Article
Full-text available
Over the past 50 or so years, I have been concerned with the quality of economic forecasts and have written both about the procedures for evaluating these predictions and the results that were obtained from these evaluations. In this paper I provide some perspectives on the issues involved in judging the quality of these forecasts. These include the reasons for evaluating forecasts, the questions that have been asked in these evaluations, the statistical tools that have been used, and the generally accepted results. (I do also present some new material that has not yet been published.) I do this in two parts: first focusing on short-run GDP and inflation predictions and then turning to labor market forecasts.
Article
Historically, forecasters have failed to predict cyclical turning points and the forecasting record in this regard has not improved. This suggests that we should focus on what should be an easier task, recognizing recessions as they occur. We present a new approach that will enable us to determine in real-time when there is a significant deviation from an economy’s dynamic growth path. This approach uses a CUSUM-like methodology and requires us to construct an index,that we call the Economic News Index, from real-time data that shows how the economy is functioning. We apply this approach to German data to nowcast the recessions that began in 2008 and 2012.
Article
The paper evaluates the quality of the German national accounting data (GDP and its useside components) as measured by the magnitude and dispersion of the forecast / revision errors. It is demonstrated that government consumption series are the least reliable, whereas real GDP and real private consumption data are the most reliable. In addition, early forecasts of GDP, private consumption, and investment growth rates are shown to be systematically upward biased. Finally, early forecasts of all the variables seem to be no more accurate than naïve forecasts based on the historical mean of the final data.
Article
This paper asks whether the accuracy of German macroeconomic forecasts has improved over time. We examine one-year-ahead forecasts of rates of real GDP growth and inflation for the years 1967 to 2010 by three major German forecasters and the OECD. We find that overall error levels are high but not much different from those for the U.S. and U.K. In the 1980s and 1990s accuracy improved somewhat, but has now returned to its 1970s level, indicating that it reflects the variance of growth and inflation. Benchmark comparisons of these predictions with ex post forecasts of a macroecono-metric model indicate that accuracy can be improved but it will be difficult to achieve.
Article
Full-text available
Macroeconomic forecasts are used extensively in industry and government. The historical accuracy of US and UK forecasts are examined in the light of different approaches to evaluating macroforecasts. Issues discussed include the comparative accuracy of macroeconometric models compared to their time series alternatives, whether the forecasting record has improved over time, the rationality of macroeconomic forecasts and how a forecasting service should be chosen. The role of judgement in producing the forecasts is also considered where the evidence unequivocally favors such interventions. Finally the use of macroeconomic forecasts and their effectiveness is discussed. The conclusion drawn is that researchers have paid too little attention to the issue of improving the forecasting accuracy record. Finally, areas where improvements are most likely to be found are discussed.
Article
Starting with conventional tests for weak rationality, we show how additional tests can be performed if predictions with different time horizons are commonly used in them. Next, we show that most of these tests can still be applied if the structure of the economic system changes but only some of them if the true system is unknown. Finally, these tests are applied to the semi-annual one-and two-step predictions of the group of five leading economic research institutes in the Federal Republic of Germany. For the two-step predictions we find more evidence against the rational expectations hypothesis than for the one-step predictions.
Article
We develop a regression test to evaluate forecasts relative to the naı̈ve ‘last observation’ benchmark forecast. To this end, the current year forecast is decomposed into the last observation and the information content that forecasters add. Similarly, the 1-year-ahead forecast can be viewed as the sum of the last available realization, the information content of the current year forecast and the information content added for next year. Evaluation regressions provide a test of the statistical significance of the components of forecasts. The test is applied to the current and 1-year-ahead growth forecast produced by the OECD for 21 countries. The results show that when the evaluation statistics indicate a poor quality of forecasts, occasionally they do contain information. The information content of the 18-month horizon forecasts is low. As the horizon shortens, the quality of the forecasts increases. Alternative naı̈ve forecasts such as the last realization or the current year forecast as a 1-year-ahead forecast do not, generally, perform better.
Article
This paper examines the accuracy of forecasts of the international economy made by the OECD. Our large data set, comprising some 5500 pairs of forecasts and outcomes, includes one-, two-, and three-step ahead semi-annual forecasts of eight components of the balance of payments for the G7 economies over a 20-year period. There is considerable variation in the accuracy of these forecasts. Although they are generally superior to naive and time-series predictions, there are some marked exceptions particularly as the forecast horizon lengthens. Forecasting error is overwhelmingly non-systematic. However, our study reveals numerous instances of forecasts which could be improved by a simple linear correction, or by incorporating information contained in known, recent forecast errors. The OECD's forecasts of services and private transfers, and official transfers are cause for particular concern: the accuracy of these forecasts is low, often below that of corresponding time-series forecasts, and rationality tests indicate that they are most prone to inefficiency and inconsistency.
Article
This article reviews recently proposed likelihood ratio tests of goodness-of-fit and independence of interval forecasts. It recasts them in the framework of Pearson chi-squared statistics, and considers their extension to density forecasts. The use of the familiar framework of contingency tables increases the accessibility of these methods to users, and allows the incorporation of two recent developments, namely a more informative decomposition of the chi-squared goodness-of-fit statistic and the calculation of exact small-sample distributions. The tests are applied to two series of density forecasts of inflation, namely the US Survey of Professional Forecasters and the Bank of England fan charts. This first evaluation of the fan chart forecasts finds that, whereas the current-quarter forecasts are well-calibrated, this is less true of the one-year-ahead forecasts. The fan charts fan out too quickly and the excessive concern with the upside risks was not justified over the period considered.
Article
The acceptance of macroeconometric model results suffers from the procedures of model builders/users that are often opaque. This paper describes in detail the production of a macroeconomic forecast with a macroeconometric model for the German economy. The model used is the RWI-business cycle model, a medium sized macroeconometric model (41 stochastic equations, 86 identities). It has been used since the late 1970s in the RWI and in the biannual ‘Joint Diagnosis’ of German economic research institutes. The paper starts with a presentation of the analytical foundations of econometric model forecasting in general and of various ways and forms of incorporating outside-model information in particular. The presentation of the main features of the model is followed by the detailed description of the various stages of forecast production. The paper ends with suggesting a more intensive use of the analytical possibilities of the models and their forecast.
Article
Using monthly GDP forecasts from Consensus Economics, Inc. for 18 developed countries, reported over 24 different forecast horizons during the period 1989–2004, we find that the survey forecasts do not have much value when the horizon goes beyond 18 months. Using two alternative approaches to measure the flow of new information in fixed-target survey forecasts, we find that the biggest improvement in forecasting performance comes when the forecast horizon is around 14 months. The dynamics of information accumulation over forecast horizons can provide both the forecasters and their clients with an important clue in their selection of the timing and frequency in the use of forecasting services. The limits to forecasting that these private market forecasters exhibit are indicative of the current state of macroeconomic foresight.
Article
This paper documents the presence of systematic bias in the real GDP and inflation forecasts of private sector forecasters in the G7 economies in the years 1990–2005. The data come from the monthly Consensus Economics forecasting service, and bias is measured and tested for significance using parametric fixed effect panel regressions and nonparametric tests on accuracy ranks. We examine patterns across countries and forecasters to establish whether the bias reflects the inefficient use of information, or whether it reflects a rational response to financial, reputational and other incentives operating for forecasters. In several G7 countries – Japan, Italy, Germany and France – there is evidence of a change in the trend growth rate. In these circumstances, standard tests for rationality are inappropriate, and a bias towards optimism in the consensus forecast is inevitable as rational forecasters learn about the new trend. In all countries there is evidence that individual forecasters converge on the consensus forecast too slowly. However, the persistent optimism of some forecasters, and the persistent pessimism of others, is not consistent with the predictions of models of “rational bias” that have become popular in the finance and economics literature.
Article
This paper examines the accuracy of forecasts of the international economy made by the OECD. Our large data set, comprising over 7,000 pairs of forecasts and outcomes, includes one-, two-, and three-step ahead semi-annual forecasts of the main components of demand, output and prices for Canada, France, Germany, Italy, Japan, the U.K. and the U.S.A. over the twenty-year period 1968–1987. Various measures of accuracy are computed; also a comparison is made with competing naive and time-series predictions. The analysis includes a full range of diagnostic checks on forecast performance, including rationality tests for unbiasedness, efficiency and consistency. Although there is considerable variation in the accuracy of these forecasts, they are generally superior to the naive and time-series predictions. Error is predominantly non-systematic. However, our analysis exposes exceptions, particularly forecasts of government consumption, and in some of the forecasts of fixed and inventory investment, the foreign balance and inflation. Accuracy in these cases could be improved by a simple linear correction, or by incorporating information contained in recent, known forecast errors. At least half the OECD forecasts fail one or more of the rationality tests.
Article
The OECD produces two-year-ahead growth forecasts for the G7-countries since 1987; these forecasts have never been evaluated. A regression is developed that tests for the information content of the forecasts. The idea is that this content is the added value forecasters incorporate in their forecasts. The information content is defined relative to the forecast for the previous year. In the end, the added value contained in the current year forecast is calculated relative to the last observation. The test consists in checking whether the information content reduces the forecasts error. The study begins with a calculation of the usual accuracy statistics. These indicate an extreme low quality for the forecasts. The regression tests support this conclusion although the forecasts for Japan do possess some information. Alarming for users of forecasts is that there are no obvious alternatives.