Article

Combined Forecasts of the 2012 Election: The PollyVote

Abstract

The present study reviews the accuracy of four methods for forecasting the 2013 German election: polls, prediction markets, expert judgment, and quantitative models. On average, across the two months prior to the election, polls were most accurate, with a mean absolute error of 1.4 percentage points, followed by quantitative models (1.6), expert judgment (2.1), and prediction markets (2.3). In addition, the study provides new evidence for the benefits of combining forecasts. Averaging all available forecasts within and across the four methods provided more accurate predictions than the typical component forecast. The error reductions achieved through combining forecasts ranged from 5% (compared to polls) to 41% (compared to prediction markets). The results conform to prior research on US presidential elections, which showed that combining is one of the most effective methods to generating accurate election forecasts.
FORESIGHT Winter 2013
50
INTRODUCTION
For the third time since its appearance in
2004, the PollyVote (www.pollyvote.com)
has demonstrated the value of combining
forecasts to predict the two-party popular
vote in U.S. presidential elections. e nal
forecast, calculated on election eve, gave
President Obama 51% of the popular vote,
which missed the actual election result by
only 0.7 percentage points.
e 2012 PollyVote was comprised of fore-
casts from ve component methods:
1. Trial-heat polls, using averages calculated
by ve polling aggregators: RealClear-
Politics, Princeton Election Consortium,
TPM PollTracker, Pollster at Hungton
Post, and Election Projection
2. Vote-share contract prices on the Iowa
Electronic Market, using seven-day rolling
averages
3. e PollyPanel of sixteen experts on Amer-
ican politics, responding to monthly sur-
veys, with variable participation rates
4. Fourteen econometric models
5. Eight index models
To generate the PollyVote predictions, we
rst calculated unweighted averages of fore-
casts within the same component predic-
tion method. en we averaged these across
components (also unweighted) to create the
PollyVote forecasts.
POLLY’S PERFORMANCE
In its rst forecast released on January
1st of 2011, almost two years prior to the
election, the PollyVote predicted that Presi-
dent Obama would win the popular vote.
ough the predicted margins for Obama
varied over the succeeding months, the fore-
cast of an Obama victory remained constant
through Election Day. e nal forecast, re-
leased on the eve of the election, predicted
Obama to receive 51.0% of the two-party
vote, a miss of 0.7% percentage points (as of
end of November).
Corresponding PollyVote errors in 2004
and 2008 were 0.3 and 0.7 percentage points
respectively. us, the mean absolute error
of the PollyVote’s nal forecasts for the past
three elections is 0.6 percentage points. By
comparison, the error of the Gallup pre-
election poll for all three elections averaged
1.8 percentage points, three times higher.
While the 2012 PollyVote consistently and
accurately forecast the reelection of the
president, the major competing predic-
tors, prediction markets and polls at times
pointed to an opposition Republican vic-
tory. Moreover, PollyVote forecasts were less
volatile than predictions from markets and
polls. e constancy of the predicted 2012
election outcome is similar to the PollyVote
performance in the two preceding elections.
For 8 months prior to the 2004 election the
PollyVote consistently forecast that George
W. Bush would win. In 2008 the PollyVote
predicted an Obama victory throughout the
14 months prior to the election. Looking at
the complete day-by-day results for the three
elections, the PollyVote now has a track
Combined Forecasts of the 2012 Election:
The PollyVote
Andreas Graefe, J. Scott Armstrong,
Randall J. Jones, Alfred G. Cuzán
Election Postmortem
www.forecasters.org/foresight FORESIGHT 51
record of more than 44 months
of correct daily forecasts of the
election winner.
POLLY VS.
FIVETHIRTYEIGHT
We compared the PollyVote
2012 performance with that of
Nate Silver’s popular New York
Times blog, FiveirtyEight.
We found that FiveirtyEight
provided accurate forecasts,
though PollyVote was more ac-
curate than FiveirtyEight for
most of the ve-month horizon.
Figure 1 reports the error of
both approaches from May 31,
when Silver released his rst
forecast, until the election.
Any point on the PollyVote or
FiveirtyEight lines in the
chart shows the average error across all daily
forecasts from that time until the election.
For example, on the le side of the chart the
value is 0.26 for the PollyVote and 0.43 for
FiveirtyEight, meaning that the mean ab-
solute error for the PollyVote’s daily forecasts
across the entire 159-day horizon is 0.26 per-
centage points, and for FiveirtyEight it is
0.43. However, as the graph shows, Fiveir-
tyEight was more accurate across the nal
two weeks prior to Election Day.
CONCLUSION
Previous studies have shown that combin-
ing is particularly appropriate in forecast-
ing environments having high uncertainty,
which is usually the case with long horizons
(Graefe and colleagues, 2012). e PollyVote
was designed to provide accurate long-term
election forecasts. For short-term predic-
tions, individual methods, such as polls and
prediction markets, can also be quite accu-
rate as they contain the most recent infor-
mation.
REFERENCE
Graefe, A., Armstrong, J.S., Jones, R.J. Jr., & Cuz-
án, A.G. (2012). Combining forecasts: An appli-
cation to elections. 2011 APSA Annual Meeting
Paper, ssrn.com/abstract=1902850.
Andreas Graefe is LMU Research Fel-
low associated with the Department of
Communication Science and Media Re-
search at the Ludwig-Maximilian Univer-
sität LMU Munich and Foresight Editor for
Prediction Markets.
graefe.andreas@gmail.com
Alfred Cuzán is Distinguished Profes-
sor of Political Science at the University of
West Florida and creator of a scal model
for forecasting presidential elections.
Randy Jones is Professor of Political Sci-
ence at the University of Central Oklahoma
and author of Who Will Be in the White House:
Predicting Presidential Elections (2002).
Figure 1. PollyVote vs. FiveirtyEight
Scott Armstrong is Professor of Market-
ing at the Wharton School of the University
of Pennsylvania and one of the founders of
the International Institute of Forecasters.
... International Journal of Applied Forecasting (Cuzán et al., 2005;Graefe et al., 2009Graefe et al., , 2013 A more sophisticated approach to increasing poll accuracy is to calculate "poll projections", as we term them. Poll projections take into account the historical record of the polls when making predictions of the election outcome. ...
Article
Full-text available
We summarize the literature on the effectiveness of combining forecasts by assessing the conditions under which combining is most valuable. Using data on the six US presidential elections from 1992 to 2012, we report the reductions in error obtained by averaging forecasts within and across four election forecasting methods: poll projections, expert judgment, quantitative models, and the Iowa Electronic Markets. Across the six elections, the resulting combined forecasts were more accurate than any individual component method, on average. The gains in accuracy from combining increased with the numbers of forecasts used, especially when these forecasts were based on different methods and different data, and in situations involving high levels of uncertainty. Such combining yielded error reductions of between 16% and 59%, compared to the average errors of the individual forecasts. This improvement is substantially greater than the 12% reduction in error that had been reported previously for combining forecasts.
Combining forecasts: An application to elections
  • A Graefe
  • J S Armstrong
  • R J Jones
  • A G Cuzán
Graefe, A., Armstrong, J.S., Jones, R.J. Jr., & Cuzán, A.G. (2012). Combining forecasts: An application to elections. 2011 APSA Annual Meeting Paper, ssrn.com/abstract=1902850.