ArticlePDF Available

Accuracy gains of adding vote expectation surveys to a combined forecast of US presidential election outcomes

Authors:

Abstract and Figures

In averaging forecasts within and across four component methods (i.e., polls, prediction markets, expert judgment, and quantitative models), the combined PollyVote provided highly accurate predictions for the US presidential elections from 1992 to 2012. This research note shows that the PollyVote would have also outperformed vote expectation surveys, which prior research identified as the most accurate individual forecasting method during that time period. In addition, adding vote expectations to the PollyVote would have further increased the accuracy of the combined forecast. Across the last 90 days prior to the six elections, a five-component PollyVote (i.e., including vote expectations) would have yielded a mean absolute error of 1.08 percentage points, which is 7% lower than the corresponding error of the original four-component PollyVote. This study thus provides empirical evidence in support of two major findings from forecasting research. First, combining forecasts provides highly accurate predictions, which are difficult to beat by even the most accurate individual forecasting method available. Second, the accuracy of a combined forecast can be improved by adding component forecasts that rely on a different method and different data than the forecasts already included in the combination.
Content may be subject to copyright.
Research and Politics
January-March 2015: 1 –5
© The Author(s) 2015
DOI: 10.1177/2053168015570416
rap.sagepub.com
Creative Commons NonCommercial-NoDerivs CC-BY-NC-ND: This article is distributed under the terms of the Creative
Commons Attribution-NonCommercial-NoDerivs 3.0 License (http://www.creativecommons.org/licenses/by-nc-nd/3.0/) which
permits non-commercial use, reproduction and distribution of the work as published without adaptation or alteration, without further permission
provided the original work is attributed as specified on the SAGE and Open Access pages (http://www.uk.sagepub.com/aboutus/openaccess.htm).
Introduction
Combining forecasts is a well-established and powerful
method to increase forecast accuracy (Armstrong, 2001;
Clemen, 1989). The reason is that a combined forecast
includes more information than forecasts from any single
component method. In addition, the systematic and random
errors associated with individual component forecasts are
likely to cancel out in the combined forecast.
As has been demonstrated with the PollyVote for pre-
dicting US presidential elections, combining forecasts is
particularly beneficial if one can draw on component fore-
casts that use different methods and data. The PollyVote
averages forecasts within and across four different compo-
nent methods: polls, prediction markets, quantitative mod-
els, and expert judgment. Across the six elections from
1992 to 2012, the resulting combined forecast reduced the
error of a typical poll, model, and expert judgment by more
than half. Compared with prediction markets, the most
accurate component method, error was reduced by 16%
(Graefe et al., 2014b). Forecasts made on Election Eve
prior to the three elections from 2004 to 2012 missed the
final vote share on average by 0.6 percentage points. To put
this in perspective, the average error of the final Gallup poll
was more than three times higher (Graefe et al., 2014a).
This performance was achieved even though the
PollyVote did not include forecasts from vote expectation
surveys, also known as ‘citizen forecasts’, which were
recently shown to provide highly accurate forecasts of US
presidential election outcomes (Graefe, 2014). These sur-
veys simply ask respondents whom they expect to win.1
Accuracy gains of adding vote expectation
surveys to a combined forecast of US
presidential election outcomes
Andreas Graefe
Abstract
In averaging forecasts within and across four-component methods (i.e. polls, prediction markets, expert judgment and
quantitative models), the combined PollyVote provided highly accurate predictions for the US presidential elections from
1992 to 2012. This research note shows that the PollyVote would have also outperformed vote expectation surveys,
which prior research identified as the most accurate individual forecasting method during that time period. Adding vote
expectations to the PollyVote would have further increased the accuracy of the combined forecast. Across the last
90 days prior to the six elections, a five-component PollyVote (i.e. including vote expectations) would have yielded a
mean absolute error of 1.08 percentage points, which is 7% lower than the corresponding error of the original four-
component PollyVote. This study thus provides empirical evidence in support of two major findings from forecasting
research. First, combining forecasts provides highly accurate predictions, which are difficult to beat for even the most
accurate individual forecasting method available. Second, the accuracy of a combined forecast can be improved by
adding component forecasts that rely on different data and different methods than the forecasts already included in the
combination.
Keywords
Combining forecasts, election forecasting, vote expectations, citizen forecasts, presidential research
LMU Research Fellow, Department of Communication Studies and
Media Research, LMU Munich, Germany
Corresponding author:
Andreas Graefe, LMU Research Fellow, Department of Communication
Studies and Media Research, Oettingenstr. 67, LMU Munich 80538,
Germany.
Email: a.graefe@lmu.de
570416RAP0010.1177/2053168015570416Research & PoliticsGraefe
research-article2015
Research Note
2 Research and Politics
The aggregate responses are then used as a forecast of who
will win the election. If data on historical elections are
available, the aggregate responses can also be translated to
popular vote-share forecasts using simple linear regression
(Lewis-Beck and Stegmaier, 2011; Lewis-Beck and Tien,
1999).
Vote expectation surveys have been around at least as
long as scientific polling (Hayes, 1936), but have long been
overlooked as a method for forecasting election outcomes.
Although early work pointed to the accuracy of vote expec-
tations, these studies focused on identifying factors that
explain why most citizens are able to accurately predict
election outcomes (e.g. Lewis-Beck and Skalaban, 1989;
Lewis-Beck and Tien, 1999). Only recently have research-
ers begun to specifically study vote expectation surveys as
a method for forecasting elections (Lewis-Beck and
Stegmaier, 2011; Murr, 2011, 2014).
In a previous study, I compared the accuracy of vote
expectations to forecasts from polls, prediction markets,
quantitative models, and expert judgment (Graefe, 2014).
Across the last 100 days prior to the seven US presidential
elections from 1988 to 2012, vote expectations provided
more accurate forecasts of election winners and vote shares
than each of the four established methods. Compared with
polls, vote expectations reduced the error of vote-share pre-
dictions by 51%. Compared with prediction markets, error
was reduced by 6%. In other words, vote expectation sur-
veys appear to be the most accurate individual method for
forecasting US presidential elections available to date.
The present research note builds on this work and con-
tributes to knowledge on combining forecasts by analysing
(1) the relative accuracy of vote expectations and the
PollyVote and (2) the accuracy gains from adding vote
expectations to the PollyVote.
Method and data
Accuracy is analysed for forecasts of the national two-party
popular vote in the six US presidential elections from 1992
to 2012, the time period for which forecast data on both the
PollyVote and vote expectation surveys are available. The
absolute error, calculated as the absolute difference of the
predicted and actual national two-party popular vote of the
incumbent party’s candidate, was used as the measure of
accuracy.
Forecasts from the original four-component PollyVote
and vote expectations were obtained from publicly availa-
ble datasets at the Harvard Dataverse Network. These data-
sets provide daily forecasts of the national two-party
popular vote for each of the six US presidential elections
from 1992 to 2012.2 From these data, a new set of daily
forecasts was calculated by adding vote expectations as a
fifth component method to the original PollyVote. That is,
this new (five-component) PollyVote was computed by
averaging forecasts across five (instead of four) component
methods: (1) polls, (2) prediction markets, (3) quantitative
models, (4) expert judgment, and (5) vote expectations. For
more information on the calculation of the original
PollyVote see Graefe et al. (2014b). All data and calcula-
tions are available at the Harvard Dataverse Network.3
Results
Figure 1 shows the mean absolute errors (MAE) of fore-
casts from vote expectations, the original PollyVote (with-
out vote expectations), and the new PollyVote (including
vote expectations) across the six elections from 1992 to
2012. Vote expectations were less accurate than the original
PollyVote for both long-term (90–70 days prior to Election
0.00
0.50
1.00
1.50
2.00
2.50
11121314151617181
Mean absolute error (1992-2012)
Days to Election Day
New PollyVote (including vote expectations)
Original PollyVote (without vote expectations)
Vote expectations
Figure 1. Mean absolute errors of forecasts from vote expectations, the original PollyVote (without vote expectations) and the
new PollyVote (with vote expectations), 1992–2012.
Graefe 3
Day) and short-term forecasts (from 20 days prior to
Election Day). For medium-term forecasts (60–20 days),
however, vote expectations performed similar to – and
sometimes better than – the original PollyVote.
Figure 1 further shows that adding vote expectations to
the original PollyVote increases accuracy. Except for long-
term forecasts, the new five-component PollyVote provides
at least as – and usually more – accurate forecasts as the
original four-component PollyVote.
Figure 2 presents the same data in a different way by
showing the MAE of vote expectations and both PollyVote
versions across the remaining days in the forecast horizon.
That is, at any given day, the chart depicts the average error
that one would have achieved by picking one of the three
methods and relying on its forecast until Election Day. For
example, if one had relied on the vote expectation forecasts
starting 90 days before the election, an average error of
1.32 percentage points would have resulted. In comparison,
the corresponding error of the original PollyVote would
have been 12% lower (1.16 percentage points). In general,
the gains in accuracy by relying on the PollyVote rather
than vote expectations tend to increase as the election
comes closer. Furthermore, Figure 2 demonstrates the ben-
efit of adding vote expectations, as the error of the new
(five-component) PollyVote was consistently lower than
the error of the original (four-component) PollyVote. For
example, starting 90 days prior to Election Day, the MAE
of the new PollyVote was 1.08, which is 7% lower than the
corresponding error of the original PollyVote.
Discussion
This research note provides empirical evidence in support
of two major findings from the forecasting literature. First,
combining forecasts from different methods that use differ-
ent data provides highly accurate forecasts, which are dif-
ficult to beat by even the most accurate individual method
available. Across the past 90 days prior to each of the six
elections from 1992 to 2012, the original PollyVote – which
averages forecasts within and across polls, prediction mar-
kets, quantitative models, and expert judgment, but does
not include vote expectations – missed the incumbent par-
ty’s final vote share on average by 1.16 percentage points.
This error is 12% lower than the corresponding error of
vote expectation surveys, which prior research found to the
most accurate method for the examined time period (Graefe,
2014).
Second, and more importantly, the accuracy of a com-
bined forecast can be further improved by adding compo-
nent forecasts that rely on a different method and different
data than the forecasts already included in the combination.
After adding vote expectations as a fifth component
method, the new PollyVote reduced the error of the original
four-component version by 7%, a substantial improvement
given the already very low forecast error. On average across
the 90 days prior to the six elections, the new five-
component PollyVote missed the final election result by lit-
tle more than one (i.e. 1.08) percentage point.
This performance was achieved by calculating simple
unweighted averages within and across forecasts of five
component methods. Calculating unweighted averages
may appear as a naïve approach to combining forecasts, as
it does not account for the component methods’ relative
accuracy. However, an early review of more than 200
papers showed that the simple average provides a good
starting point for combining forecasts, and is difficult to
beat by more complex approaches (Clemen, 1989). These
results still hold today, despite many efforts in search of
0.50
0.75
1.00
1.25
11121314151617181
Mean absolute error across remaining
days to election (1992-2012)
Days to Election Day
New PollyVote (including vote expectations)
Original PollyVote (without vote expectations)
Vote expectations
Figure 2. Mean absolute error of forecasts from vote expectations, the original PollyVote (without vote expectations) and the
new PollyVote (including vote expectations), calculated across the remaining day to election, 1992–2012.
4 Research and Politics
sophisticated methods for combining. The problem with
complex statistical procedures that aim to estimate compo-
nent weights from historical data is that they tend to per-
form poorly in situations with limited and messy data,
which are common in the social sciences. A recent example
is Ensemble Bayesian Model Averaging (EBMA), a method
that has been shown to perform well for combining fore-
casts in the data-heavy domain of weather forecasting.
However, when applied to problems with scarce and noisy
data, such as in economic and election forecasting, EBMA
provided less accurate forecasts than the simple equal-
weights average (Graefe et al., 2015).
When pre-specifying equal weights to component fore-
casts, analysts ignore the components’ relative accuracy.
Instead, they deliberately introduce a bias that reduces vari-
ance and thereby limits a model’s ability to explain given
data. At the same time, however, lower variance avoids the
danger of overfitting a model to historical data. Thus, low
variance can be beneficial when predicting new data, in
particular, in situations that involve much uncertainty. In
statistical theory, this relationship is known as the bias–
variance tradeoff (Hastie et al., 2001).
When forecasting presidential elections, for example,
uncertainty occurs due to ambiguity about the component
methods’ relative accuracy, external shocks (e.g. campaign
events), or the existence of noisy data. Calculating
unweighted averages across forecasts is a simple way to
account for such uncertainties or, in other words, to incor-
porate prior knowledge that prediction in the situation at
hand is difficult.
Differential weights can be useful if there is strong prior
knowledge about the methods’ relative accuracy. For exam-
ple, polls are well known to have little predictive value until
shortly before the election (Erikson and Wlezien, 2012).
Thus, it might be useful to assign lower weights to polls
early in the campaign and then gradually increase their
weight as the election comes closer, an approach that is
becoming standard practice in models that combine struc-
tural (fundamental) data and updated polls over time. For a
review of existing models see Lewis-Beck and Dassonneville
(2015), who also incorporate this prior knowledge about the
relative accuracy of polls over time to develop forecasting
models for French, German, and UK elections.
While combining is most powerful when aggregating
many forecasts that use a different method and different
data, the approach can also be used if fewer methods are
available. In a recent study, combining forecasts from three
methods (a quantitative model, prediction markets, and
polls) yielded accurate predictions of the 2012 US Electoral
College and senatorial elections, a situation in which data
are scarce (Rothschild, 2014).
Finally, the benefits of combining are of course not lim-
ited to US elections. In a recent validation test, the PollyVote
was used to predict vote shares of seven parties in the 2013
German federal election by averaging forecasts within and
across the four-component methods that were used in the
original US PollyVote. Across the 58 days for which fore-
casts from all four components were available, the com-
bined PollyVote forecast was more accurate than each
component’s typical forecast. Error reductions ranged from
5%, compared with a typical poll, to 41%, compared with a
typical prediction market (Graefe, 2015).
Conclusion
Since 2004, the PollyVote has demonstrated the benefits of
combining for forecasting the national vote in US presiden-
tial elections by averaging forecasts from four-component
methods. Combining is a simple and powerful strategy to
generate accurate forecasts. Combining forecasts from dif-
ferent component methods typically yields more accurate
predictions than the average (i.e. randomly selected) com-
ponent, and often outperforms even the best component.
Adding forecasts that use a different method and different
data to the combination can be expected to further improve
accuracy. Given the results of the present research note, the
PollyVote will add vote expectations as a fifth component
for forecasting the 2016 US presidential election.
Declaration of conflicting interest
The authors declare that there is no conflict of interest.
Funding
This research received no specific grant from any funding agency
in the public, commercial, or not-for-profit sectors.
Notes
1. For example, “Regardless of whom you support, and trying
to be objective as possible, who do you think will win the
presidential election in November (2008)–Barack Obama or
John McCain?” Gallup Poll, October 23–26, 2008.
2. The PollyVote data are available at http://dx.doi.org/10.7910/
DVN/23184. The vote expectations data are available at http://
dx.doi.org/10.7910/DVN/VOTEEXPECTATIONSURVEYS.
3. The data and calculations for the present research note are
available at: http://dx.doi.org/10.7910/DVN/27967.
Supplementary material
The replication files are available at: http://thedata.harvard.edu/dvn/
dv/agraefe/faces/study/StudyPage.xhtml?globalId=doi:10.7910/
DVN/27967
References
Armstrong JS (2001) Combining forecasts. In: Armstrong JS (ed)
Principles of Forecasting: A Handbook for Researchers and
Practitioners. New York: Springer, pp.417–439.
Clemen RT (1989) Combining forecasts: A review and anno-
tated bibliography. International Journal of Forecasting 5:
559–583.
Graefe 5
Erikson RS and Wlezien C (2012) The Timeline of Presidential
Elections: How Campaigns Do (And Do Not) Matter.
Chicago: University of Chicago Press.
Graefe A (2014) Accuracy of vote expectation surveys in fore-
casting elections. Public Opinion Quarterly 78: 204–232.
Graefe A (2015) German election forecasting: Comparing and
combining methods for 2013. German Politics (forthcoming)
http://ssrn.com/abstract=2540845.
Graefe A, Armstrong JS, Jones RJJ, et al. (2014a) Accuracy
of combined forecasts for the 2012 Presidential Election:
The PollyVote. PS: Political Science & Politics 47: 427–
431.
Graefe A, Armstrong JS, Jones RJJ, et al. (2014b) Combining
forecasts: An application to elections. International Journal
of Forecasting 30: 43–54.
Graefe A, Küchenhoff H, Stierle V, et al. (2015) Limitations of
Ensemble Bayesian Model Averaging for forecasting social
science problems. International Journal of Forecasting DOI:
10.2139/ssrn.2266307 (forthcoming).
Hastie T, Tibshirani R and Friedman J (2001) The Elements of
Statistical Learning. Data Mining, Inference, and Prediction.
New York: Springer.
Hayes SP (1936) The predictive ability of voters. The Journal of
Social Psychology 7: 183–191.
Lewis-Beck MS and Dassonneville R (2015) Forecasting elections in
Europe: Synthetic models. Research & Politics Epub ahead of
print January 2015. DOI: 10.1177/2053168014565128.
Lewis-Beck MS and Skalaban A (1989) Citizen forecasting:
can voters see into the future? British Journal of Political
Science 19: 146–153.
Lewis-Beck MS and Stegmaier M (2011) Citizen forecasting: Can
UK voters see the future? Electoral Studies 30: 264–268.
Lewis-Beck MS and Tien C (1999) Voters as forecasters: A
micromodel of election prediction. International Journal of
Forecasting 15: 175–184.
Murr AE (2011) “Wisdom of crowds”? A decentralised elec-
tion forecasting model that uses citizens’ local expectations.
Electoral Studies 30: 771–783.
Murr AE (2014) The wisdom of crowds: Applying Condorcet’s
jury theorem to forecasting U.S. Presidential Elections.
International Journal of Forecasting (forthcoming).
Rothschild D (2014) Combining forecasts for elections: Accurate,
relevant, and timely. International Journal of Forecasting
http://dx.doi.org/10.1016/j.ijforecast.2014.1008.1006.
... Economics and managerial articles focused on targets like: the number of tourist arrivals, defects in programming code, and monthly demand of products (Song et al., 2013;Kabak andÜlengin, 2008;Huang et al., 2016;Failing et al., 2004;Shin et al., 2013). Political articles focused on predicting presidential outcomes, a categorical target (Hurley and Lior, 2002;Graefe et al., 2014a;Morgan, 2014;Graefe, 2015Graefe, , 2018Graefe et al., 2014b). Risk-related targets were continuous and categorical: the probability of structural damage, nuclear fallout, occupational hazards, and balancing power load Zio and Apostolakis, 1997;Cabello et al., 2012;Adams et al., 2009;Neves and Frangopol, 2008;Jana et al., 2019;Wang et al., 2008;Ren-jun and Xian-zhong, 2002;Zio, 1996b;Baecke et al., 2017;Brito and Griffiths, 2016b;Craig et al., 2001;Mu and Xianming, 1999;. ...
... Articles using neither Bayesian or Frequentist models (Johnson et al., 2018;Li et al., 2012;Petrovic et al., 2006;Song et al., 2013;Graefe et al., 2014a;Morgan, 2014;Cai et al., 2016;Kabak andÜlengin, 2008;Graefe, 2015Graefe, , 2018Failing et al., 2004;Ren-jun and Xian-zhong, 2002;Hora et al., 2013;Baron et al., 2014) resorted to: dynamical systems, simple averages of point estimates and quantiles from experts, and tree-based regression models. ...
... Articles applied to politics stress that experts have poor accuracy when forecasting complex (and rapidly changing) systems unless they receive feedback about their forecast accuracy and have contextual information about the forecasting task (Graefe et al., 2014a;Graefe, 2015Graefe, , 2018Satopää et al., 2014). Political experts, it is argued, receive feedback by observing the outcome of elections and often have strong contextual knowledge about both candidates. ...
Preprint
Forecasts support decision making in a variety of applications. Statistical models can produce accurate forecasts given abundant training data, but when data is sparse, rapidly changing, or unavailable, statistical models may not be able to make accurate predictions. Expert judgmental forecasts---models that combine expert-generated predictions into a single forecast---can make predictions when training data is limited by relying on expert intuition to take the place of concrete training data. Researchers have proposed a wide array of algorithms to combine expert predictions into a single forecast, but there is no consensus on an optimal aggregation model. This scoping review surveyed recent literature on aggregating expert-elicited predictions. We gathered common terminology, aggregation methods, and forecasting performance metrics, and offer guidance to strengthen future work that is growing at an accelerated pace.
... where, MSE denotes mean square error defined in (8). ...
... The friendship paradox states, "on average, the number of friends of a random friend is always greater than the number of friends of a random individual". Formally:3 Intent polling and expectation polling have been considered intensively in literature, mostly in the context of forecasting elections and, it is generally accepted that expectation polling is more efficient compared to intent polling[8],[9],[10],[11],[12].4 [13],[14] discuss how expectation polling can give rise to misinformation propagation in social learning and, propose Bayesian filtering methods to eliminate the misinformation propagation. ...
Article
This paper deals with randomized polling of a social network. In the case of forecasting the outcome of an election between two candidates A and B, classical intent polling asks randomly sampled individuals: who will you vote for? Expectation polling asks: who do you think will win? In this paper, we propose a novel neighborhood expectation polling (NEP) strategy that asks randomly sampled individuals: what is your estimate of the fraction of votes for A? Therefore, in NEP, sampled individuals will naturally look at their neighbors (defined by the underlying social network graph) when answering this question. Hence, the mean squared error (MSE) of NEP methods rely on selecting the optimal set of samples from the network. To this end, we propose three NEP algorithms for the following cases: (i) the social network graph is not known but, random walks (sequential exploration) can be performed on the graph (ii) the social network graph is unknown. For case (i) and (ii), two algorithms based on a graph theoretic consequence called friendship paradox are proposed. Theoretical results on the dependence of the MSE of the algorithms on the properties of the network are established. Numerical results on real and synthetic data sets are provided to illustrate the performance of the algorithms.
... Why call it NEP? The term neighborhood expectation polling is derived, from the fact that the response q(v) of each sampled individual v ∈ S is the expected label 9 Intent polling and expectation polling have been considered intensively in literature, mostly in the context of forecasting elections and, it is generally accepted that expectation polling is more efficient compared to intent polling [27,26,65,66,61]. 10 [47,45] discuss how expectation polling can give rise to misinformation propagation in social learning and, propose Bayesian filtering methods to eliminate the misinformation propagation. value among her neighbors i.e. q(v) = E{s (U) } where, U is a random neighbor of the sampled individual v ∈ S. ...
Preprint
Dynamic models and statistical inference for the diffusion of information in social networks is an area which has witnessed remarkable progress in the last decade due to the proliferation of social networks. Modeling and inference of diffusion of information has applications in targeted advertising and marketing, forecasting elections, predicting investor sentiment and identifying epidemic outbreaks. This chapter discusses three important aspects related to information diffusion in social networks: (i) How does observation bias named friendship paradox (a graph theoretic consequence) and monophilic contagion (influence of friends of friends) affect information diffusion dynamics. (ii) How can social networks adapt their structural connectivity depending on the state of information diffusion. (iii) How one can estimate the state of the network induced by information diffusion. The motivation for all three topics considered in this chapter stems from recent findings in network science and social sensing. Further, several directions for future research that arise from these topics are also discussed.
... Compared to a typical poll, for example, vote expectations reduced the forecast error by about 50% on average. Furthermore, an ex post analysis for the elections from 1992 to 2012 found that adding citizen forecasts to the PollyVote would have reduced forecast error by another 7% (Graefe 2015a). ...
Chapter
Full-text available
The PollyVote uses evidence-based techniques for forecasting the popular vote in presidential elections. The forecasts are derived by averaging existing forecasts generated by six different forecasting methods. In 2016, the PollyVote correctly predicted that Hillary Clinton would win the popular vote. The 1.9 percentage-point error across the last 100 days before the election was lower than the average error for the six component forecasts from which it was calculated (2.3 percentage points). The gains in forecast accuracy from combining are best demonstrated by comparing the error of PollyVote forecasts with the average error of the component methods across the seven elections from 1992 to 2016. The average errors for last 100 days prior to the election were: public opinion polls (2.6 percentage points), econometric models (2.4), betting markets (1.8), and citizens’ expectations (1.2); for expert opinions (1.6) and index models (1.8), data were only available since 2004 and 2008, respectively. The average error for PollyVote forecasts was 1.1 percentage points, lower than the error for even the most accurate component method.
... Although the use of the expectation question in pre-election surveys goes back before the emergence of intention polling (Hayes, 1936), scholars have only recently begun to study its value for predicting election outcomes in plurality elections in the UK and the US (Graefe, 2014(Graefe, , 2015aLewis-Beck & Stegmaier, 2011;Murr, 2011Murr, , 2015aMurr, , 2015bRothschild & Wolfers, 2012). For example, one study compared the accuracy of the expectation question to polls, prediction markets, quantitative models, and expert judgment for predicting election winners and vote shares in the seven US presidential elections from 1988 to 2012. ...
Article
Full-text available
This study tests non-representative expectation surveys as a method for forecasting elections. For dichotomous forecasts of the 2013 German election (e.g., who will be chancellor, which parties will enter parliament), two non-representative citizen samples performed equally well than a benchmark group of experts. For vote-share forecasts, the sample of more knowledgeable and interested citizens performed similar to experts and quantitative models, and outperformed the less informed citizens. Furthermore, both citizen samples outperformed prediction markets but provided less accurate forecasts than representative polls. The results suggest that non-representative surveys can provide a useful low-cost forecasting method, in particular for small-scale elections, where it may not be feasible or cost-effective to use established methods such as representative polls or prediction markets.
Article
Forecasts support decision making in a variety of applications. Statistical models can produce accurate forecasts given abundant training data, but when data is sparse or rapidly changing, statistical models may not be able to make accurate predictions. Expert judgmental forecasts—models that combine expert‐generated predictions into a single forecast—can make predictions when training data is limited by relying on human intuition. Researchers have proposed a wide array of algorithms to combine expert predictions into a single forecast, but there is no consensus on an optimal aggregation model. This review surveyed recent literature on aggregating expert‐elicited predictions. We gathered common terminology, aggregation methods, and forecasting performance metrics, and offer guidance to strengthen future work that is growing at an accelerated pace. This article is categorized under: • Statistical Learning and Exploratory Methods of the Data Sciences > Clustering and Classification • Statistical Learning and Exploratory Methods of the Data Sciences > Exploratory Data Analysis • Statistical Learning and Exploratory Methods of the Data Sciences > Modeling Methods • Statistical and Graphical Methods of Data Analysis > Multivariate Analysis
Article
Research on election forecasting suggests there are benefits from combining different sources of information. This paper discusses the experience of a combined forecasting method that was developed for the UK’s EU referendum in 2016. The sources included opinion polls, vote expectation surveys, prediction and betting markets, expert and volunteer forecasts, and various forecasting models based on polling and other kinds of data. Averages of sources within each of these categories all, in our final forecast, suggested Remain was more likely to win but with varying degrees of certainty. Combining them produced a forecast that beat some but not others. Opinion polls and citizen forecasts came closest to the true outcome. Betting and prediction market participants and volunteer forecasters were the most overconfident that the UK would vote Remain. This may have been because they were distrustful of the polls following the 2015 general election miss and had too strong an expectation of a late swing towards the status quo similar to those in Scotland in 2014 and Quebec in 1995.
Article
Full-text available
The PollyVote Forecast for the 2016 American Presidential Election - Volume 49 Issue 4 - Andreas Graefe, Randall J. Jones, J. Scott Armstrong, Alfred G. Cuzán
Article
Full-text available
Scientific work on national election forecasting has become most developed for the United States case, where three dominant approaches can be identified: Structuralists, Aggregators, and Synthesizers. For European cases, election forecasting models remain almost exclusively Structuralist. Here we join together structural modeling and aggregate polling results, to form a hybrid, which we label a Synthetic Model. This model contains a political economy core, to which poll numbers are added (to tap omitted variables). We apply this model to a sample of three Western European countries: Germany, Ireland, and the United Kingdom. This combinatory strategy appears to offer clear forecasting gains, in terms of lead and accuracy.
Article
Full-text available
The present study reviews the accuracy of four methods for forecasting the 2013 German election: polls, prediction markets, expert judgment, and quantitative models. On average, across the two months prior to the election, polls were most accurate, with a mean absolute error of 1.4 percentage points, followed by quantitative models (1.6), expert judgment (2.1), and prediction markets (2.3). In addition, the study provides new evidence for the benefits of combining forecasts. Averaging all available forecasts within and across the four methods provided more accurate predictions than the typical component forecast. The error reductions achieved through combining forecasts ranged from 5% (compared to polls) to 41% (compared to prediction markets). The results conform to prior research on US presidential elections, which showed that combining is one of the most effective methods to generating accurate election forecasts.
Article
Full-text available
We compare the accuracy of simple unweighted averages and Ensemble Bayesian Model Averaging (EBMA) to combining forecasts in the social sciences. A review of prior studies from the domain of economic forecasting finds that the simple average was more accurate than EBMA in four out of five studies. On average, the error of EBMA was 5% higher than the error of the simple average. A reanalysis and extension of a published study provides further evidence for US presidential election forecasting. The error of EBMA was 33% higher than the corresponding error of the simple average. Simple averages are easy to describe, easy to understand and thus easy to use. In addition, simple averages provide accurate forecasts in many settings. Researchers who develop new approaches to combining forecasts need to compare the accuracy of their method to this widely established benchmark. Forecasting practitioners should favor simple averages over more complex methods unless there is strong evidence in support of differential weights.
Article
Full-text available
Simple surveys that ask people who they expect to win are among the most accurate methods for forecasting U.S. presidential elections. The majority of respondents correctly predicted the election winner in 193 (89%) of 217 surveys conducted from 1932 to 2012. Across the last 100 days prior to the seven elections from 1988 to 2012, vote expectation surveys provided more accurate forecasts of election winners and vote shares than four established methods (vote intention polls, prediction markets, econometric models, and expert judgment). Gains in accuracy were particularly large compared to polls. On average, the error of expectation-based vote-share forecasts was 51% lower than the error of polls published the same day. Compared to prediction markets, vote expectation forecasts reduced the error on average by 6%. Vote expectation surveys are inexpensive, easy to conduct, and the results are easy to understand. They provide accurate and stable forecasts and thus make it difficult to frame elections as horse races. Vote expectation surveys should be more strongly utilized in the coverage of election campaigns.
Book
Principles of Forecasting: A Handbook for Researchers and Practitioners summarizes knowledge from experts and from empirical studies. It provides guidelines that can be applied in fields such as economics, sociology, and psychology. It applies to problems such as those in finance (How much is this company worth?), marketing (Will a new product be successful?), personnel (How can we identify the best job candidates?), and production (What level of inventories should be kept?). The book is edited by Professor J. Scott Armstrong of the Wharton School, University of Pennsylvania. Contributions were written by 40 leading experts in forecasting, and the 30 chapters cover all types of forecasting methods. There are judgmental methods such as Delphi, role-playing, and intentions studies. Quantitative methods include econometric methods, expert systems, and extrapolation. Some methods, such as conjoint analysis, analogies, and rule-based forecasting, integrate quantitative and judgmental procedures. In each area, the authors identify what is known in the form of `if-then principles', and they summarize evidence on these principles. The project, developed over a four-year period, represents the first book to summarize all that is known about forecasting and to present it so that it can be used by researchers and practitioners. To ensure that the principles are correct, the authors reviewed one another's papers. In addition, external reviews were provided by more than 120 experts, some of whom reviewed many of the papers. The book includes the first comprehensive forecasting dictionary.
Article
A reanalysis and extension of Montgomery, Hollenbach, and Ward (2012) shows that the predictive performance of Ensemble Bayesian Model Averaging (EBMA) strongly depends on the conditions of the forecasting problem. EBMA is of limited value in situations with small samples and many component forecasts, a situation that is common for social science prediction problems. These results conform to a large body of research, which has determined that simple approaches to combining (such as equal weights) often perform as well as sophisticated approaches when combining forecasts. Simple averages are easy to describe, easy to understand, and easy to use. They should be favored over more complex methods unless one has strong evidence that differential weights will improve accuracy.
Article
Increasingly, professional forecasters rely on citizen forecasts when predicting election results. Following this approach, forecasters predict the winning party to be the one which most citizens have said will win. This approach predicts winners and vote shares well, but related research has shown that some citizens forecast better than others. Extensions of Condorcet’s jury theorem suggest that naïve citizen forecasting can be improved by delegating the forecasting to the most competent citizens and by weighting their forecasts by their level of competence. Indeed, doing so increases both the accuracy of vote share predictions and the number of states forecast correctly. Allocating the state’s electoral votes to the candidate who the most weighted delegates say will win yields a simple but successful forecasting model of the US Presidency. The ‘wisdom of crowds’ model predicts eight presidential elections out of nine correctly. The results suggest that delegating and weighting provide easy ways to improve citizen forecasting.
Article
This paper increases the efficiency and understanding of forecasts for Electoral College and senatorial elections by generating forecasts based on voter intention polling, fundamental data, and prediction markets, then combining these forecasts. The paper addresses the most relevant outcome variable, the probability of victory in state-by-state elections, while also solving for the traditional outcomes, and ensuring that the forecasts are easy to update continuously over the course of the main election cycle. In an attempt to maximize both these attributes and the accuracy, I create efficient forecasts for each of these three types of raw data, with innovations in aggregating the data, then correlate the aggregated data with the outcomes. This paper demonstrates that all three data types make significant and meaningful contributions to election forecasting. Various groups of stakeholders, including researchers, election investors, and election workers, can benefit from the efficient combined forecasts defined in this paper. Finally, the forecast is tested on the 2012 elections and excels out-of-sample.
Article
The leading approaches to election forecasting have been statistical models, prediction markets, or vote intention polls. This paper explores a different, little used approach - citizen forecasting - a method never yet tried for the case of the United Kingdom. Citizen forecasting simply asks voters in a scientific pre-election survey who they think will win. This aggregated (averaged) result is then assessed for its accuracy in forecasting the overall election outcome. This assessment is carried out on UK general elections from 1951 to 2005, and then applied to 2010. We find, among other things, that generally speaking the majority of the electorate can foretell the winner in advance.
Book
ABOUT THIS BOOK In presidential elections, do voters cast their ballots for the candidates whose platform and positions best match their own? Or is the race for president of the United States come down largely to who runs the most effective campaign? It’s a question those who study elections have been considering for years with no clear resolution. In The Timeline of Presidential Elections, Robert S. Erikson and Christopher Wlezien reveal for the first time how both factors come into play. Erikson and Wlezien have amassed data from close to two thousand national polls covering every presidential election from 1952 to 2008, allowing them to see how outcomes take shape over the course of an election year. Polls from the beginning of the year, they show, have virtually no predictive power. By mid-April, when the candidates have been identified and matched in pollsters’ trial heats, preferences have come into focus—and predicted the winner in eleven of the fifteen elections. But a similar process of forming favorites takes place in the last six months, during which voters’ intentions change only gradually, with particular events—including presidential debates—rarely resulting in dramatic change. Ultimately, Erikson and Wlezien show that it is through campaigns that voters are made aware of—or not made aware of—fundamental factors like candidates’ policy positions that determine which ticket will get their votes. In other words, fundamentals matter, but only because of campaigns. Timely and compelling, this book will force us to rethink our assumptions about presidential elections.