Content uploaded by Andreas Murr
Author content
All content in this area was uploaded by Andreas Murr on Dec 10, 2019
Content may be subject to copyright.
The Party Leadership Model predicts a
Conservative outright majority
Andreas E. Murr
University of Warwick
a.murr@warwick.ac.uk
December 9, 2019
Abstract: Party leadership elections predict British general elections. Be-
cause Members of Parliament usually want to get re-elected and know their
colleagues, they are motivated and able to vote in leadership contests for
the colleague who is most likely to deliver electoral victory. Therefore, the
party with the more popular leader among MPs typically wins most votes
among citizens. This prediction rule, the so-called Party Leadership Model,
correctly predicted the electoral victories of David Cameron in 2015 and
Theresa May in 2017. Because Boris Johnson is more popular among Con-
servative MPs than Jeremy Corbyn is among Labour’s, the model predicts
an electoral victory for Boris Johnson in 2019, with an outright majority
of 342 seats for the Conservatives as the most likely outcome. The model
also implies that unless Labour selects a more popular leader, or the Conser-
vatives a less popular one, Labour’s chances of winning any future election
remain low.
Keywords: Election forecasting; British general elections; party leadership contests
1
Members of Parliament usually want to get re-elected. They typically also find it easy
to identify one of their colleagues that would have the best chance to lead their party
to win the election—MPs have experience of fighting elections and of working with
their colleagues. If MPs vote for party leadership candidates based on their perceived
chances to deliver electoral victory, then party leadership elections should predict general
elections. Is that so?
The Party Leadership Model, as its name suggests, uses party leadership elections to
predict general elections (Murr 2015). For each party leadership election, it calculates
the difference in vote share between elected leader and main contender in the final ballot
or nomination process among MPs (see Table 1). The model then picks the leader with
the larger difference as the predicted winner of the general election (see Table 2). Since
the model was introduced it correctly predicted the re-elections of David Cameron in
2015 and of Theresa May in 2017. In addition, when applied in retrospect, it correctly
predicted 8 out of 11 general elections between 1966 and 2010. This is a remarkable
accuracy given that the model can forecast as soon as we know who leads the two main
parties into a general election. In the past this has been more than four years in advance
on average (see Table 3).
The prediction for the December 2019 election is clear. In July 2019, the Conservatives
elected Boris Johnson as their party leader. In the 5th and final ballot among MPs, he re-
ceived 51.1 per cent of the votes, whereas his main contender, Jeremy Hunt, received 24.6
per cent. Later on, party members selected Boris Johnson as their leader. Accordingly,
Johnson’s popularity among MPs was 51.1−24.6 = 26.5 per cent. In September 2015,
Labour elected Jeremy Corbyn as their party leader. In the nomination process among
MPs, he received 15.5 per cent of the votes, whereas his main contender, Andy Burnham
received 29.3 per cent. Later on, party members and registered or affiliated supporters
selected Jeremy Corbyn as leader of the Labour Party. Accordingly, Corbyn’s popularity
among MPs was 15.5−29.3 = −13.8 per cent. Because the performance among MPs
of Boris Johnson (26.5 per cent) was better than of Jeremy Corbyn (−13.8 per cent),
the Party Leadership Model predicts a re-election of Boris Johnson. To foreshadow the
result of a Bayesian analysis reported below, the certainty of this forecast is 72.7 per
cent. By using leadership contests, the Party Leadership Model makes this forecast with
data from more than four months before the event.1
1As a more recent indicator of Jeremy Corbyn’s popularity, we could consider the 2016 party leadership
election, triggered by MPs passing a motion of no-confidence in Jeremy Corbyn. We know that there
were 40 Corbyn votes and 172 no-confidence votes. Hence, we can conclude that Corbyn was less
popular among MPs than the main contender in the leadership election, Owen Smith. We don’t
2
Table 1: Performance of Conservative and Labour party leaders in their party leadership
election among MPs in per cent.
General election Party Ballot Name Vote share Performance
leadership Elected Main Elected Main (vl−vc)
election leader contender leader (vl) contender (vc)
Conservative Party
1966–1974 1965 1 Edward Heath Reginald 50.3 44.6 5.7
Maudling
1979–1987 1975 2 Margaret William 52.9 28.6 24.3
Thatcher Whitelaw
1992 1990 2 John Major Michael 49.7 35.2 14.5
Heseltine
1997 1995 1 John Major John Redwood 66.3 27.1 39.2
2001 1997 3 William Hague Kenneth Clarke 56.8 43.2 13.6
2005 2003 — Michael Howard — — — —
2010–2015 2005 2 David Cameron David Davis 45.5 28.8 16.7
2017 2016 2 Theresa May Andrea Leadsom 60.5 25.5 35.0
2019 2019 5 Boris Johnson Jeremy Hunt 51.1 24.6 26.5
Labour Party
1966–1974 1963 2 Harold Wilson George Brown 58.3 41.7 16.6
1979 1976 3 James Callaghan Michael Foot 56.2 43.8 12.4
1983 1980 2 Michael Foot Denis Healey 51.9 48.1 3.8
1987 1983 1 Neil Kinnock Roy Hattersley 49.3 26.1 23.2
1992 1988 1 Neil Kinnock Tony Benn 82.8 17.2 65.6
1997–2005 1994 1 Tony Blair John Prescott 60.5 19.9 40.6
2010 2007 — Gordon Brown John McDonnell 88.2 8.2 80.0
2015 2010 4 Ed Miliband David Miliband 46.6 53.4 −6.8
2017–2019 2015 — Jeremy Corbyn Andy Burnham 15.5 29.3 −13.8
Note: This table only includes party leadership elections that selected a party leader who stood in a General Election. Michael
Howard (Conservative Party in 2003) faced no contender in his party leadership election. For him a performance measure is
unavailable. Gordon Brown (Labour Party in 2007) faced a contender (John McDonnell) in the nomination rounds, but McDonnald
did not get the necessary number of nominations to secure a place on the ballot, so Brown was the only successfully nominated
candidate. Source: Quinn (2012) and own calculations.
3
Table 2: The Party Leadership Model correctly predicts 10 out of 13 past elections when
data is sufficiently available to make a forecast.
General election Incumbent (yt) Performance of party leaders Prediction ( ˜yt) Winner (yt+1)
(vl−vc)
Conservative Labour
1966 Labour 5.7 16.6 Labour Labour
1970 Labour 5.7 16.6 Labour Conservative
1974 (Feb) Conservative 5.7 16.6 Labour Labour
1974 (Oct) Labour 5.7 16.6 Labour Labour
1979 Labour 24.3 12.4 Conservative Conservative
1983 Conservative 24.3 3.8 Conservative Conservative
1987 Conservative 24.3 23.2 Conservative Conservative
1992 Conservative 14.5 65.6 Labour Conservative
1997 Conservative 39.2 40.6 Labour Labour
2001 Labour 13.6 40.6 Labour Labour
2005 Labour — 40.6 — Labour
2010 Labour 16.7 80.0 Labour Conservative
2015 Conservative 16.7 −6.8 Conservative Conservative
2017 Conservative 35.0 −13.8 Conservative Conservative
2019 Conservative 26.5 −13.8 Conservative ?
Note: The leaders of the Conservatives faced no contender in the party leadership elections relevant
for the 2005 General Election. For this General Election a forecast is unavailable.
4
Table 3: Forecasting lead of the Party Leadership Model. The lead equals the number of
days between the dates of the last party leadership election and of the General
Election.
Conservatives Labour Forecast Election Lead
28/07/65 14/02/63 28/07/65 31/03/66 246 days
28/07/65 14/02/63 28/07/65 18/06/70 1786 days
28/07/65 14/02/63 28/07/65 28/02/74 3137 days
28/07/65 14/02/63 28/07/65 10/10/74 3361 days
11/02/75 03/04/76 03/04/76 03/05/79 1125 days
11/02/75 10/11/80 10/11/80 09/06/83 941 days
11/02/75 02/10/83 02/10/83 11/06/87 1348 days
27/11/90 02/10/88 27/11/90 09/04/92 499 days
04/07/95 21/07/94 04/07/95 01/05/97 667 days
19/06/97 21/07/94 19/06/97 07/06/01 1449 days
06/11/03 21/07/94 — 05/05/05 —
06/12/05 24/06/07 24/06/07 06/05/10 1047 days
06/12/05 25/09/10 25/09/10 07/05/15 1685 days
11/07/16 12/09/15 11/07/16 08/06/17 332 days
22/07/19 12/09/15 22/07/19 12/12/19 143 days
Mean = 1269 days
Note: Michael Howard (Conservative Party in 2003) faced
no contender in his party leadership elections. For the corre-
sponding General Election a forecast is unavailable. Source:
Quinn (2012) and own calculations.
5
What about the possibility of a hung parliament? And what about seat shares? So far
the Party Leadership Model hasn’t made predictions about hung parliaments or seats
shares. In the previous version, only two outcomes could happen: the incumbent party
was re-elected or not. So we extend the model below to four possible outcomes: incum-
bent outright majority, incumbent-led hung parliament, opposition-led hung parliament,
or opposition outright majority. In addition, the previous version didn’t make predic-
tions about seat shares. So we extend the model below to also predict the seat shares of
incumbent and opposition parties. We also show that the model predicts these outcomes
better than models based on the historical average and the previous seat shares.
Forecasting who wins and whether winner has an
outright majority
As indicated above, the Party Leadership Model in its initial form forecasts whether
the incumbent party will be re-elected or not. Here the incumbent party is the party
with the most seats. Murr (2015) found that the Party Leadership Model predicted
the winning party more accurately than than a “poll of polls” of vote intentions of any
lead time. With hung parliaments occurring more regularly in recent times, and having
consequences for stability of governments and their ability to govern, it is of course
important to also be able to forecast whether hung parliaments occur, or at least how
likely it is that they do. In this section, we extend the Party Leadership Model to do
just that, and to test for an association between whether the incumbent-party has the
more popular party leader and whether a hung parliament will occur.
We create a 4×2 cross-tabulation of outcome in the rows and predictor in the columns.
The outcome has four possible values (1 = outright majority for incumbent party, 2 =
hung parliament led by incumbent party, 3 = hung parliament led by opposition party,
4 = outright majority for opposition party ). The predictor has two possible values
(0 = opposition-party leader more popular, 1 = incumbent-party leader more popular).
The cells display the number of observations in parentheses as well as the posterior
know for certain, however, how popular Owen Smith was compared to Jeremy Corbyn among MPs.
Because Jeremy Corbyn was allowed to be on the ballot among party members and registered
affiliates or affiliated without having to be nominated by MPs and MEPs, we cannot compute a
popularity measure as before. This said, Jeremy Corbyn was less popular than his main contender
both in 2015 (Andy Burnham) and in 2016 (Owen Smith). Therefore, the prediction of the Party
Leadership Model remains the same in 2017 and 2019: a Conservative victory. For consistent
reporting of a popularity measure, Table 1 reports the 2015 party leadership election as relevent for
the 2017 and 2019 general elections.
6
Table 4: Posterior probabilities of general election outcomes by relative leader popularity
among MPs. Number of observations in parentheses.
More popular leader
Opposition party Incumbent party
Outright majority for incumbent 25.0 (1) 59.1 (6)
Incumbent-led hung parliament 8.3 (0) 13.6 (1)
Opposition-led hung parliament 25.0 (1) 13.6 (1)
Outright majority for opposition 41.7 (2) 13.6 (1)
100.0 (4) 100.0 (9)
probabilities of the outcome conditional on the predictor value.
An initial glance at Table 4 suggest that having the more popular leader is associated
with the chances of re-election. For instance, when the incumbent-party leader was
more popular, the incumbent was re-elected 7 out of 9 times, but when its leader was
less popular, the incumbent party was re-elected only 1 out of 4 times. Indeed, if we deem
each outcome conditional on the predictor equally likely a priori and our prior sample
size equals 2, then a Bayesian analysis puts the probability of a positive association at
95 per cent.2That is, we have strong evidence that having the more popular leader
increases the chances of re-election of the incumbent party (see also Murr 2015).
The same Bayesian analysis also returns posterior probabilities of outcomes condi-
tional on the predictor value. Here, each cell has half of a prior observation added to it
to indicate our ignorance before conducting the analysis, making each outcome equally
likely. The posterior probability is then the number of observations in each cell plus half
a prior observation divided by the total number of observations in the column, prior or
otherwise. This means that because the Conservatives have the more popular leader
in the 2019, the chances of re-election are 59.1 + 13.6 = 72.7 per cent. If Labour had
the more popular leader, the chances of re-election for the Conservatives would have
dropped to 25.0+8.3 = 33.3 per cent. In other words, the Party Leadership Model
2More formally, we have an outcome ywhich takes on value k∈ {1,2,3,4}. Our predictor xcan take
on two possible values, 0 and 1. We denote the probability that y=kconditional on xas θk(x).
The prior distribution for θk(x) conditional on xis Dirichlet with prior sample size of .5 for each
category. We denote the frequency of outcome y=kconditional on xas nk(x). The posterior
distribution for θk(x) is then also Dirichlet, with posterior sample size .5 + nk(x). The posterior
mean of θk(x) is then [.5 + nk(x)]/P4
k=1[.5 + nk(x)]. We use MCMC simulation to compute the
probability that θ1(1) + θ2(1) > θ1(0) + θ2(0) and that θ2(1) + θ3(1) < θ2(0) + θ3(0). See Murr
(2015) for a similar analysis where K= 2 and for Gelman et al. (2014, 69f, 578f) more details on
Bayesian analysis with the Dirichlet distribution.
7
forecasts for 2019 the re-election of the Conservative government.
How are the chances of an outright majority? Table 4 puts the chances of an outright
majority for the incumbent party if its leader is more popular at 59.1 per cent, and
for the opposition party at 13.6 per cent, totalling 59.1 + 13.6 = 72.7 per cent. In
other words, there is a chance that there will be a hung parliament. The corresponding
probability amounts to 27.3 per cent. When the opposition has the more popular leader,
this probability increases to 8.3 + 25.0 = 33.3 per cent. A Bayesian analysis puts the
probability of a negative association at 61 per cent. That is, we have only weak evidence
that having an incumbent-party leader that is more popular decreases the chances of a
hung-parliament. Hence the evidence of an association is weaker here than in the case
of re-election. Nevertheless, it allows us to forecast cautiously that Boris Johnson will
win an outright majority with 59.1 per cent certainty.
Forecasting seat shares
Next, we take on the task of forecasting seat shares with the Party Leadership Model. To
do so, we build a simple regression model of a party’s seat share using as the only predic-
tor whether it had the more popular party leader. We evaluate its absolute forecasting
accuracy on the 13 election for which party leadership election results were available.
We then evaluate its relative accuracy: does the Party Leadership Model predict more
accurately than benchmark models? We consider two such benchmark models, inspired
by the literature on weather forecasting. Finally, we use the models to predict seat
shares in the upcoming 2019 British general election.
Meteorologists use two benchmarks that are easily available to evaluate their forecast-
ing models of weather characteristics such as temperature (e.g., Murphy 1992). These
benchmarks set a minimum standard that every weather forecasting model should pass.
The two benchmarks are the historical average (‘history’) and the previous value (‘per-
sistence’). Everyone can use the historical average to predict the current value or use the
previous value to predict the current one. When forecasting general elections, these two
benchmarks have long lead times: they allow to make predictions of the next election
right after the current election results are known. We think that these two benchmarks
are useful standards of reference to adopt also in election forecasting. Any forecasting
model worth its salt should forecast more accurately than these two benchmarks.
Hence we compare the Party Leadership Model with these two benchmarks. Our
target is to forecast the seat share of the incumbent party and of the opposition party.
8
We do so first using the two benchmarks: one forecast is the historical seat share average
of incumbent and opposition parties (the History Model); the other forecast comes from a
regression model of current seat shares that uses previous seat shares as its only predictor
(the Persistence Model). This regression model raises the bar compared to simply using
the previous seat share as the predicted current seat share—it makes fewer assumptions
and uses data to approximate the optimal combination of the historical average and
the previous seat share. Finally, we forecast using information from party leadership
elections. We regress the seat share of a party on whether its leader was more popular
than the other (the Party Leadership Model). In terms of lead time, the History and
Persistence Model have more, though the Party Leadership Model doesn’t have much
less.
Table 5 shows the estimates for each regression model based on the 13 elections de-
scribed above. Table 5 displays six regression models in total, one for the incumbent
party and one for the opposition party for each of the three forecasting models. All
three models show an incumbency advantage: the intercepts for the incumbent party
are higher than for the opposition party. The History Model shows that incumbent
parties win a higher seat share than the opposition party on average. The Persistence
Model demonstrates that this holds true even if both parties had only slightly different
previous seat shares. Finally, the Party Leadership Model confirms that this also holds
true even if the incumbent-party leader is less popular than the opposition-party leader.
This said, the party with the more popular leader is predicted to gain a higher seat
share than the party with the less popular leader. If the incumbent party has the more
popular leader, its expected seat share is 11.3 points higher than if it hasn’t. For the
opposition party this expected difference is 12.4, and so is comparable in size.
Table 5 also shows how well the models fit the data. As indicated by the two in-sample
fit statistics R2and the standard error of the regression (SE), the Party Leadership Model
fits the data best. Its R2is about twice as large as the one of the Persistence Model.
This is remarkable because the Party Leadership Model with a binary predictor accounts
for more of the variation than the Persistence Model with a continuous predictor. (By
definition, the R2of an intercept-only model such as the History Model is zero.) In
addition, the Party Leadership Model has the lowest standard error of the regression
(SE) for both parties. Its SEs are 8.9 and 8.5 for incumbent and opposition party,
respectively. The next best model is again the Persistence Model with SEs of 9.6 for
both parties, followed by the History Model with SEs of 10.1 for both parties. In other
words, when evaluated on in-sample fit statistics, the Party Leadership Model is best.
9
Table 5: Regression models of seat share (%) for the incumbent and opposition parties between 1966
and 2017, together with a forecast for 2019. Standard errors in parentheses. SE = standard
error of the regression. RMSE = root mean squared error. n= number of observations.
Incumbent party Opposition party
M1M2M3M1M2M3
Intercept 49.2 69.9 41.4 42.9 61.9 39.1
(2.8) (14.6) (4.4) (2.8) (13.1) (2.8)
Previous seat share −0.4 −0.4
(0.3) (0.3)
Leader performed better 11.3 12.4
(5.3) (5.1)
R20.00 0.16 0.29 0.00 0.17 0.35
SE (in-sample) 10.1 9.6 8.9 10.1 9.6 8.5
RMSE (out-of-sample) 10.5 10.3 10.0 10.5 10.6 9.3
n13
2019 forecast 49.2 49.1 52.7 42.9 44.5 39.1
[95 per cent interval] [26.5,71.9] [27.1,71.1] [32.1,73.2] [20.1,65.6] [22.5,66.6] [19.4,58.7]
M1: The History Model.
M2: The Persistence Model.
M3: The Party Leadership Model.
10
What about out-of-sample fit statistics? That is, how do they compare in terms of
forecasting accuracy?
We evaluate the forecasting accuracy of the models using leave-one-out cross-validation.
We leave out one election at a time from the data set used to fit the regression models,
but then predict the left-out election based on the fitted models, and compare the pre-
dicted with the actual result. Consequently, we get 13 out-of-sample prediction errors
which we square, sum, and then take the root mean of. In short, our comparison uses
the root mean squared error as computed by leave-one-out cross-validation as a measure
of predictive accuracy.
Table 5 shows the root mean squared error (RMSE) in the leave-out-out cross-validation
exercise for each regression model. The most accurate forecasting model for both in-
cumbent and opposition parties is the Party Leadership Model. For the incumbent
party the most accurate model is the Party Leadership Model (10.0), followed by the
Persistence Model (10.3), and the History Model (10.5). For the opposition party the
most accurate model is again the Party Leadership Model (9.3), followed this time by
the History Model (10.5), and the Persistence Model (10.6). In other words, relative to
the best benchmark, the Party Leadership model increases the forecasting accuracy by
about 1 −(10.0/10.3) = 3 per cent for the incumbent party and 1 −(9.3/10.5) = 11 per
cent for the opposition party.
What seat shares do the models forecast for the 2019 election? The last two rows of
Table 5 display the point and 95 per cent interval estimates for the incumbent (Conser-
vative) and main opposition (Labour) parties. For instance, because the Conservatives
are the incumbent party with the more popular leader, the Party Leadership Model
predicts them to gain 52.7 per cent of seats, while it predicts Labour to gain 39.1 per
cent of seats. This said, and as indicated also by the average out-of-sample error, the
prediction intervals for each forecast are wide. For instance, the Party Leadership Model
has the shortest ones with [32.1,73.2] for Conservatives and [19.4,58.7] for Labour.
Forecast of the 2019 British General Election
To sum up, here is the forecast of the Party Leadership Model for the 2019 December
election. It predicts that Boris Johnson will win an outright majority with 59.1 per cent
certainty. More specifically, it forecasts that the Conservatives will win 52.7 per cent
of the seats, that is 342 seats in total, and that Labour will win 39.1 per cent of seats,
that is 254 seats in total. This forecast is made with data from more than four months
11
before the date of this election.
Acknowledgments
I am very grateful to Ericka Rasc´on Ramirez and Steve Fisher for useful discussions,
comments, and encouragement. All remaining errors are my own. A replication archive
will be made available at Harvard Dataverse.
References
Gelman, A., Carlin, J. B., Stern, H. S., Dunson, D. B., Vehtari, A. & Rubin, D. B.
(2014), Bayesian Data Analysis, third edn, Boca Raton, Florida: CRC Press.
Murphy, A. H. (1992), ‘Climatology, persistence, and their linear combination as stan-
dards of reference in skill scores’, Weather and Forecasting 7(4), 692–698.
Murr, A. E. (2015), ‘The Party Leadership Model: An early forecast of the 2015 British
General Election’, Research & Politics 2(2), 1–9.
Quinn, T. (2012), Electing and Ejecting Party Leaders in Britain, Basingstoke: Palgrave
Macmillan.
12
