Page 1

Kobe University Repository : Kernel

TitleTesting homogeneity of Japanese CPI forecasters

Author(s)Ashiya, Masahiro

CitationJournal of Forecasting, 29(5): 435-441

Issue date2010-08

Resource TypeJournal Article / 学術雑誌論文

Resource Versionauthor

URLhttp://www.lib.kobe-u.ac.jp/handle_kernel/90001225

Create Date: 2011-12-31

Page 2

Testing homogeneity of Japanese CPI forecasters*

Masahiro ASHIYA +

February 2009

JEL Classification Codes: E37; C53; E17.

Keywords: Macroeconomic Forecast; Forecast evaluation; Analysis of variance.

* I gratefully acknowledge financial supports from Grant-in-Aid for Encouragement of Young

Scientists from the Japanese Ministry of Education, Science and Culture.

+ Faculty of Economics, Kobe University, 2-1 Rokko-dai, Nada, Kobe, 657-8501, Japan;

E-mail: ashiya@econ.kobe-u.ac.jp

Page 3

1

Testing homogeneity of Japanese CPI forecasters

February 2009

This paper investigates whether some forecasters consistently outperform others using

Japanese CPI forecast data of 42 forecasters over the past 18 quarters. It finds that the

accuracy rankings of zero, one, two, and five-month forecasts are significantly different from

those that might be expected when all forecasters had equal forecasting ability. Moreover,

their rankings of the relative forecast levels are also significantly different from a random one.

JEL Classification Codes: E37; C53; E17.

Keywords: Macroeconomic Forecast; Forecast evaluation; Analysis of variance.

Page 4

2

1. Introduction

The world economy has gone through tumultuous changes over the past few years; the rise

and fall of the commodity prices, and the boom and bust of the financial markets. These

unprecedented shocks have made the task of macroeconomic forecasters formidable. As a

result, many of them have failed to foresee the volatile fluctuation of output and prices. This

experience leads us to the following question: Were all forecasters equally successful (or

unsuccessful) in this period? Was there any significant difference in their forecast accuracy?

We answer this question using the monthly data of 42 Japanese consumer price index (CPI)

forecasters from April 2004 through August 2008.

Section 2 explains the data. Section 3 evaluates the homogeneity of forecast accuracy. It

finds that forecasting ability is unequal among the forecasters. More precisely, the accuracy

rankings of zero, one, two, and five-month forecasts are significantly different from those that

might be expected when all forecasters had equal forecasting ability. This result is contrary to

the findings of Batchelor (1990), Batchelor and Dua (1990a, b), Kolb and Stekler (1996), and

Ashiya (2006).

Section 4 investigates the biases of the relative forecast level of the individual forecasters.

It finds that the rankings of the relative forecast levels are significantly different from a

random one for three, four, and five-month forecasts. Namely, forecasters differ

systematically in their forecast levels. This result is consistent with the findings of Batchelor

and Dua (1990b) and Ashiya (2006). Section 5 concludes the paper.

2. Data

The Economic Planning Association has conducted a monthly survey of professional

forecasters, “ESP Forecast Survey,” since April 2004. We use the forecast data of the

consumer price index (CPI) through August 2008. We select the data of 42 forecasters (out of

44 forecasters), who participated in 18 surveys or more (the excluded forecasters participated

in five surveys).

Let

t

CPI be the CPI of month t . Then the rate of change over the year,

tp , is

computed by the following equation:

100

12

12×

−

≡

−

−

t

tt

t

CPI

CPICPI

p

.

The quarterly average change over the year is calculated as the simple arithmetic mean of

tp .

More specifically, the quarterly average change over the year from month

2

−

t

to month t,

Page 5

3

tq , is defined as

() 3

12

tttt

pppq

++≡

−−

.

Let

i

−

tktf

,

be the k-month-ahead forecast of forecaster i with respect to

tq , which is

released in month

kt − . The forecast error is defined as

t

i

−

tkt

i

tkt

qfFE

−≡

−

,,

. Its absolute

value is

t

i

−

tkt

i

tkt

qfAFE

−≡

−

,,

.

We analyze zero through five-month-ahead forecasts in this paper. Zero-month forecasts

and three-month forecasts are released in March, June, September, and December. One-month

forecasts and four-month forecasts are released in February, May, August, and November.

Two-month forecasts and five-month forecasts are released in January, April, July, and

October. The sample period of each forecast series is as follows: from the first quarter of 2004

through the second quarter of 2008 (18 quarters) for zero-month forecast; from the second

quarter of 2004 through the second quarter of 2008 (17 quarters) for one-month forecast; from

the second quarter of 2004 through the third quarter of 2008 (18 quarters) for two-month

forecast and three-month forecast; from the third quarter of 2004 through the third quarter of

2008 (17 quarters) for four-month forecast and five-month forecast.

Table 1 presents the values of several traditional measures of forecast accuracy for the

individual forecasters. The first row in Table 1 shows the summary statistics (average,

standard deviation, minimum, and maximum) of the mean absolute error (MAE). As for

zero-month forecast, the average of the MAE among forecasters is 0.072 percentage points,

and the MAE of the best forecaster is zero. The second row shows the summary statistics of

the root mean square error (RMSE).

The third row of Table 1 shows the summary statistics of modified Theil’s U, constructed

as the ratio of the RMSE of each forecaster to the RMSE of the “same-as-the-last-month”

forecast. More specifically, define

i

−

tktT

,

as the set of quarters in which forecaster i released

i

−

tktf

,

. Let

i

tkt

U

,

−

be the Theil’s U of forecaster i for

i

−

tktf

,

. Then

i

tkt

U

,

−

is defined as

=

−

i

tkt

U

,

()

()

∑∑

−−

∈

−−

∈

−

−−

i

tkt

i

tkt

Tt

tkt

Tt

t

i

tkt

qpqf

,,

2

1

2

,

for

5 ,, 1 , 0 L

=

k

.

If

1

,>

t

−

i

kt

U

, then forecaster i is inferior to the “same-as-the-last-month” forecast. Table 1

shows that

i

tkt

U

,

−

is on average 2.269 for zero-month forecast and 1.104 for one-month

forecast.

These descriptive statistics seem to indicate that there are some differences in forecasting