Are 'unbiased' forecasts really unbiased? Another look at the Fed forecasts

Abstract

This paper reconciles contradictory findings obtained from forecast evaluations: the existence of systematic errors and the failure to reject rationality in the presence of such errors. Systematic errors in one economic state may offset the opposite types of errors in the other state such that the null of rationality is not rejected. A modified test applied to the Fed forecasts shows that the forecasts were ex post biased.

Full text

Page 1

Research Program on Forecasting (RPF) Working Papers represent preliminary work circulated for
comment and discussion. Please contact the author(s) before citing this paper in any publications. The
views expressed in RPF Working Papers are solely those of the author(s) and do not necessarily represent
the views of RPF or George Washington University.


Are 'unbiased' forecasts really unbiased?
Another look at the Fed forecasts1


Tara M. Sinclair
Department of Economics
George Washington University
Washington DC 20052
tsinc@gwu.edu

Fred Joutz
Department of Economics
George Washington University
Washington DC 20052
bmark@gwu.edu

H. O. Stekler
Department of Economics
George Washington University
Washington DC 20052
hstekler@gwu.edu


RPF Working Paper No. 2008-010
http://www.gwu.edu/~forcpgm/2008-010.pdf


August 19, 2008


RESEARCH PROGRAM ON FORECASTING
Center of Economic Research
Department of Economics
The George Washington University
Washington, DC 20052
http://www.gwu.edu/~forcpgm

1 We would like to thank Peter Tinsley for helpful discussion and Lauren Taylor for excellent research
assistance. M. Al Sadek’s undergraduate thesis on this subject provided the stimulus for this paper.

Page 2

Are 'unbiased' forecasts really unbiased?
Another look at the Fed forecasts


Abstract


This paper reconciles contradictory findings obtained from forecast evaluations: the existence of
systematic errors and the failure to reject rationality in the presence of such errors. Systematic
errors in one economic state may offset the opposite types of errors in the other state such that
the null of rationality is not rejected. A modified test applied to the Fed forecasts shows that the
forecasts were ex post biased.

Page 3

1
Are 'unbiased' forecasts really unbiased?
Another look at the Fed forecasts


Monetary policy decisions are based on forecasts of future economic activity. It is for this
reason that economists have evaluated the forecasts made by the staff of the Board of Governors
of the Federal Reserve System in preparation for the Open Market Committee meetings. The
forecasts are quarterly predictions made several times each quarter with horizons of one to eight
periods. The staff predicts GDP, its components, various price indices, unemployment, etc.
Previous evaluations have primarily focused on the predictions of real and nominal GDP
growth rates, inflation rates and the unemployment rate. The studies of Karamouzis and Lombra
(1989), Scotese (1994), and Jansen and Kishan (1996, 1999) yielded mixed results about the
rationality of these forecasts. Romer and Romer (2000) and Sims (2002) found that the Fed’s
forecasts did not reject rationality and that they were as good as the predictions made by the
private sector. On the other hand, while Joutz and Stekler (2000) found that the forecasts were
unbiased, there was evidence of inefficiency, and, more importantly, the Fed staff made
systematic errors.1
This finding that the forecasts displayed systematic errors, such as reported by Joutz and
Stekler (2000), has also been observed in the results of other forecast evaluations (See Zarnowitz
and Braun, 1992; Fildes and Stekler, 2002). Forecasters overestimated the rate of growth during
slowdowns and recessions and underestimated it during recoveries and booms. Similarly,
inflation was underpredicted when it was rising and overpredicted when it was declining.

1 In addition, Clements et al. (2007) used a non-traditional approach to test for bias and found that the forecasts were
biased, whereas the traditional approach had not rejected the null of no bias.

Page 4

Particularly large errors occurred during the periods when prices were rising rapidly during the

2
1970s and early 1980s. Hanson and Whitehorn (2006) also observed these systematic errors but
associated them with particular time periods rather than with stages of the business cycle.
Despite these systematic errors, the null of unbiasedness or rationality is frequently not
rejected when the forecasts are tested. In particular, Joutz and Stekler (2000, pp. 30-31) note that,
even though there were systematic errors, the null of unbiasedness was not rejected for any of the
sets of forecasts of nominal or real GNP (GDP) or of the deflator made by the Federal Reserve.
This result held for all forecast horizons. These two findings are mutually inconsistent and create
a puzzle that we investigate.
The inconsistency is that forecasts that are considered unbiased should not display
systematic errors. Since the systematic errors are observations that are clearly present and
observable, the inconsistency must arise from the statistical methodology that is used to test for
unbiasedness.
This paper will, therefore, reexamine the Fed forecasts to determine whether they are
biased or unbiased and whether or not they display systematic errors. We will also attempt to
reconcile any discrepancies in the results that we discover. In the next section, we present the
data. This is followed by an analysis of the errors that are observed when applying the standard
statistical methodology. Given the inconsistency in the results, we modify the statistical
procedure and show that the Fed forecasts are ex post biased.
I. Data
We examine the Fed Greenbook forecasts that are made in preparation for the FOMC
meetings. We analyze forecasts made between 1965IV and 2001IV for the current quarter and

Page 5

one quarter ahead for three variables: the real output growth rate (GNP from 1965IV to

3
1991QIII and GDP from 1991IV on), the GNP/GDP deflator inflation rate, and the
unemployment rate. During the entire sample period, there is not a continuous series of
forecasts for the longer horizons. Whenever there are multiple forecasts in each quarter, we use
the last one.2 The actual data are the NIPA estimates that are released approximately 45 days
after the quarter to which they refer.3 All data, with the exception of the unemployment rate, are
converted into annualized growth rates.
II. Characteristics of the Forecast Errors
Figures (1a-1d,.., 3a-3d) present data for each variable: the annualized growth rate of real
GNP/GDP, the annualized inflation rate as measured by the GNP/GDP deflator, and the
unemployment rate. The upper left hand graphs display the historical data used in the analysis.
The other quadrants of these figures display, for each series: the current quarter and one quarter
ahead forecast errors and the forecast revisions between the current and one quarter ahead
forecasts.
There were six recessions in this period. The shaded areas of Figures 1 and 3 represent
the dates of these recessions, as defined by the NBER. In Figure 2 the shaded areas represent
periods when the inflation rate was increasing (moving from trough to peak). The dates for the
periods of the inflation “cycle” were obtained from Dr. Anirvan Banerji of the Economic Cycle

2 We use the last forecast for each quarter because those are the ones for which the Fed had the maximum amount of
information on which to base their current and next quarter forecasts. If forecasts were made within the first 10 days
of the next quarter, they are considered made in the previous quarter because there would be no new information for
the subsequent quarter. We also analyzed forecasts made in the middle of the quarter. The results were similar.
3 Use of the real time data avoids definitional and classification changes and is the most consistently available data
set for our sample. The terminology for these data releases varied over the sample: Before 1974, the “final” data
were released 45 days after the end of the quarter. Starting in 1974, releases at approximately the same time became
known as “first revision” data (with a second revision about 75 days after the quarter). Since 1988, the “preliminary”
data are released approximately two months after the quarter. We obtained similar results using the 90 day releases.