Content uploaded by Federico Fioravanti
Author content
All content in this area was uploaded by Federico Fioravanti on Dec 20, 2019
Content may be subject to copyright.
Effort of rugby teams according to the bonus
point system: a theoretical and empirical
analysis∗
Federico Fioravanti †1, Alejandro Neme‡2, Fernando Tohm´e §1,
and Fernando Delbianco ¶1
1INMABB, Universidad Nacional del Sur, Bah´ıa Blanca,
Argentina
2IMASL, Universidad Nacional de San Luis, San Luis,
Argentina
Abstract
Using a simple game-theoretical model of contests, we compare
the effort exerted by rugby teams under three different point systems
used in tournaments around the world. The scoring systems under
consideration are NB,+4 and 3+. We state models of the games
under the three point systems, both static and dynamic. In all those
models we find that the 3+ system ranks first, +4 second and N B
third. We run empirical analyses using data from matches under the
three scoring systems. The results of those statistical analyses confirm
our theoretical conclusions.
∗We thank the support of the Agencia Nacional de Promoci´on Cient´ıfica y Tecnol´ogica
(Argentina) through grant PICT-2017-2355. Alejandro Neme also acknowledges the fi-
nancial support received from the UNSL (grant PROICO-32016) and from the Consejo
Nacional de Investigaciones Cient´ıficas y T´ecnicas (Argentina), through grant PIP 112-
200801-00655.
†federico.fioravanti9@gmail.com (Corresponding author)
‡aneme@unsl.edu.ar
§ftohme@criba.edu.ar
¶fernando.delbianco@uns.edu.ar
1
1 Motivation
Rugby is a sport in constant evolution. This can be easily seen in the fact
that its rules are being continuously reevaluated. This leads to experiments in
which alternative rules are tested and, if the results are satisfactory, become
part of the rulebook.
These continuous revisions and modifications are intended to increase both
the safety of the players and the pleasure of watching the game. That is, some
rules are changed to make the game safer, reducing the number of injuries
suffered by the players, while others are modified to make the game more
entertaining for players, coaches and spectators alike.
Some of the latter kind of modifications involve the points awarded to teams,
depending on, among other factors, the number of tries scored. Fortunately
for the exploration of alternative rules, the organizers of tournaments are
given a free hand to choose the point system to be applied and thus to
experiment with those variations. For example, in the World Cup (or in
the Rugby Championship, a tournament in which the only participants are
the best four national teams of the Southern Hemisphere), four points are
awarded to the winning team, two to each team in case of a tie and no points
to the losing team. Besides the points awarded for winning, tying or losing
a game, an extra point (usually called bonus point) is awarded to the team
that scores four or more tries, and an extra point to the losing team, if the
difference in the score is seven points or less.
Another point system used around the world, for example in the Super Rugby
or French Top 14, two of the most important club tournaments in the world,
consists in giving an extra point to the winning team if it scores three more
tries than its rival, and an extra point is awarded to the losing team if the
difference in the score is seven or less.
In this work we compare the effort of the teams under these two point systems
as well as under the point system in which no bonus points are awarded.
Clearly, there is no consensus on which one is the best, reflected in the fact
that different important tournaments around the world use different point
systems. But there exists a consensus on that games are more entertaining
when teams fight to the end to win a match. This behavior can be induced
by point rules that give incentives to teams to exert more effort in order to
succeed.
To start analyzing the effort aspects of point systems, notice that there are
different ways in which a team can score in rugby . One is by grounding the
2
ball in the other team’s ingoal.1This is called scoring a try and the team
that does it, earns five points. When a team scores a try, it can opt to kick
to the posts, getting two more points. On the other hand, a kick from the
ground (called a penalty kick) or a drop kick (a kick after the ball bounced)
that goes through the posts, awards the team three points.
The aforementioned scoring methods are under evaluation. World Rugby
(the international governing body of rugby union) is interested in knowing
whether giving more points for a try or giving less for a place kick induces
teams to score more tries.
A common understanding of a team devoting more effort in a game is that
it plays more offensively, defends with more attitude and forces its players
to play as hard as they can. We want to see if the effort spent in each bonus
point system correlates in some way with the number of tries. We intend
to determine, resorting to a game-theoretical analysis, which point system
induces teams to devote more effort. Furthermore, using real-world data
we check the validity of our theoretical conclusions, which can be useful for
sports planners who intend to design tournaments on sound theoretical and
empirical grounds.
Although a sport like rugby can be really difficult to model, due to the
presence of variables with an uncountable number of possible values, there
are effective ways of simplifying the analysis. The use of a simple game-
theoretical model of contests allows to predict the behavior of the teams and
to find the most adequate strategies for each instance of a match.
Some of the authors that have modeled different aspects of sports using
game-theoretical tools are Walker and Woodens (2001) for tennis, Chiappori,
Levitt and Groseclose (2002), Palomino, Rigotti and Rustichini (1999) in the
case of soccer and Petr´oczi and Haugen (2012) to evaluate the effectiveness
of anti-doping policies. An analysis particularly relevant for our purposes
compares the strategies of two soccer teams under the two and the three
points scoring systems (Brocas and Carrillo, 2004). The authors conclude,
rather unsurprisingly, that teams become more offensive if they are awarded
three points when they win. But interestingly, they also find that by giving
more than three points to a winner makes the teams more defensive in the
first half of the game and so, in average, higher offensiveness is not induced
by this point system.
In our case, we start modeling a rugby game statically, comparing the effort
devoted by the teams under different point systems. We seek to find out
which one pushes teams to the limit, making the game more entertaining.
1An ingoal is a rectangular area at the end of the field, which has two of them, each
one corresponding to one of the teams.
3
We then try to answer the same question, this time in the framework of a
dynamic model, using the results of Mass´o - Neme (1996), by analyzing the
feasible and equilibrium payoffs. We consider the average of joint efforts used
to obtain those equilibrum payoffs in each point system, to find out which one
induces the teams to play more agressively. Finally, we check the real-world
validity of our conclusions using data from different tournaments around the
world.
The plan of the paper is as follows. In Section 2 we present the static model
and examine the degrees of offensiveness associated to the different point
systems. In Section 3 we do the same, but in the context of a dynamic model.
We find that the order of offensiveness is the same in both cases, being 3+
the system that ranks on the top. In Section 4 we run empirical analyses
in order to corroborate the validity of those results, which are confirmed by
the data of various tournaments under the three systems. Finally, Section 5
concludes.
2 The Static Model
We intend here to model, in a simple way, the effects of point systems on the
choice of the levels of effort of teams. We consider two teams, Aand B. The
possible events in a match are denoted (a, b)∈N0×N0, where N0represents
the natural numbers plus 0. Letters aand bstand for the tries scored by
teams Aand B, respectively.
To simplify the analysis, we disregard the precise differences between goals,
penalty kicks or drop kicks, and just focus on the tries scored and the joint
efforts of the teams. In each event, we consider a contest in which two risk -
neutral contestants are competing to score a try, and win the points awarded
by the points system.2The contestants differ in their valuation of the prize.
Each contestant i∈ {A, B}independently exerts an irreversible and costly
effort ei≥0, which will determine, through a contest success function (CSF),
which team wins the points. Formally, the CSF maps the profile of efforts
(eA, eB) into probabilities of scoring a try. We adopt the logit formulation,
since it is the most widely used in the analysis of sporting contests (Dietl et
al., 2011). Its general form was introduced by Tullock (1980), although we
use it here with a slight modification:3
2We consider that teams totally discount the future, and assume that the game can
end after they score a try.
3When teams exert no effort, the probability of scoring a try is 0. This allows to get a
tie as a result.
4
pi(eA, eB) =
eα
i
eα
A+eα
B
if max{eA, eB}>0
0otherwise
The parameter α > 0 is called the “discriminatory power” of the CSF,
measuring the sensitivity of success to the level of effort exerted.4We nor-
malize it and set α= 1. Associated to effort there is a cost function ci(ei),
often assumed linear in the literature,
ci(ei) = cei
where c > 0 is the (constant) marginal cost of effort.
The utility or payoff function when the profile of efforts is (eA, eB) and
the score is (a, b), has the following form (we omit the effort argument for
simplicity):
UA((eA, eB),(a, b)) =
pA(fA(a+ 1, b) + kB1) + (1 −pA−pB)(fA(a, b) + kB2)) + pBfA(a, b + 1) −ceA
for team A, and
UB((eA, eB),(a, b)) =
pAfB(a+1, b)+(1−pA−pB)(fB(a, b)+kA2))+ pB(fB(a, b +1)+kA1))−ceB
for team B, where
kB1=fB(a, b + 1) −fB(a+ 1, b)
kB2=fB(a, b + 1) −fB(a, b)
kA1=fA(a+ 1, b)−fA(a, b + 1)
kA2=fA(a+ 1, b)−fA(a, b)
and fi:N0×N0→ {0,1,2,3,4,5}depends on the point system we are work-
ing with. It is defined on the final scores and yields the points earned by
team i. Each point system is characterized by a different function.
In the case where no bonus point (NB system) is awarded, we have:
4This CSF satisfies homogeneity. That is, when teams exert the same level of effort,
they have the same probabilities of winning the contest. This is a plausible hypothesis
when teams have the same level of play.
5
fNB
A(a, b) =
4if a > b
2if a =b
0if a < b
fNB
B(a, b) =
0if a > b
2if a =b
4if a < b
When a bonus point is given for scoring 4 or more tries, and for losing by
one try (+4 system) the functions are:
f+4
A(a, b) =
4if a > b and a < 4
5if a > b and a ≥4
2if a =b or [a > 4and b −a= 1]
3if a =b and a ≥4
0if a < 4and b −a > 1
1if [a+ 1 < b and a ≥4] or b −a= 1
f+4
B(a, b) =
4if b > a and b < 4
5if b > a and b ≥4
2if a =b or [b > 4and a −b= 1]
3if a =b and b ≥4
0if b < 4and a −b > 1
1if [b+ 1 < a and b ≥4] or a −b= 1
Finally, when a difference of 3 tries gives the winning team a bonus point
and losing by one try gives the bonus point to the loser (3+ system) we have:
f3+
A(a, b) =
4if 0< a −b < 3
5if a −b≥3
2if a =b
0if b −a > 1
1if b −a= 1
f3+
B(a, b) =
4if 0< b −a < 3
5if b −a≥3
2if a =b
0if a −b > 1
1if a −b= 1
In all three cases the utility functions represent the weighted sum of three
probabilities, namely that of team Ascoring, that of none of the teams scor-
ing and that of team Bscoring. The corresponding weights are the points
earned in each case plus the gain of blocking the other team, precluding it of
winning points. This gain is defined as the difference between the points that
the other team can earn if it scores and the points they get times , where
6
0< << 1 is not very large. This intends to measure the importance
of blocking the other team and not letting it score and earn more points.
Teams are playing a tournament, so making it hard for the other team to
earn points is an incentive (although not a great one) in a match. The way
we define the utility function rests on the simple idea that to score four tries,
one has to be scored first. This captures the assumption that teams care
only about the immediate result of scoring, and not about what can happen
later.
Under these assumptions, we seek to find the equilibria corresponding to the
three point systems. The appropriate notion of equilibrium here is in terms
of strict dominant strategies since the chances of each team are independent
of what the other does. Notice that, trivially, each dominant strategies equi-
librium is (the unique) Nash equilibrium in the game.5Once obtained these
equilibria, the next step of the analysis is to compare them, to determine how
the degree of offensiveness changes with the change of rules. This comparison
is defined in terms of the following relation:
(eA, eB)(eA0, eB0) if eA+eB≥eA0+eB0
while
(eA, eB)∼(eA0, eB0) in any other case.
where (eA, eB)(eA0, eB0) is understood as “with (eA, eB) both teams exert
more effort than with (eA0, eB0)”.
We look for the maximum number of tries that can be scored by a team,
in order to limit the number of cases to analyze. We use the statistics of
games played in different tournaments around the world, which show that,
in average, teams can get at most 7 tries ([12]-[23])(see Section 4). This, in
turn leads us to 64 possible instances (events).
At each event we compare the equilibrium strategies. We thus obtain a
ranking of the point systems, based on the relation. The reaction function
of team i, describing the best response to any possible effort choice of the
other team, can be computed from the following first order conditions:
eB
(eA+eB)2kA=c
for team A, and
5Nash equilibria exist since the game trivially satisfies the condition of having compact
and convex spaces of strategies while the utility functions have the expected probability
form, which ensures that the best response correspondence has a fixed point.
7
eA
(eA+eB)2kB=c
for team B, where kAand kBobtain by rearranging the constants of the
corresponding utility function. The equilibrium (e∗
A, e∗
B) in pure strategies is
characterized by the intersection of the two reaction functions and is given
by:
(e∗
A, e∗
B)=( k2
AkB
c(kA+kB)2,kAk2
B
c(kA+kB)2)
As an example, consider, without loss of generality, a particular instance
(for simplicity we omit the arguments):
•Event (2,0)
NB system
UA((eA, eB),(2,0)) = pA(4 + 0) + (1 −pA−pB)(4 + 0)) + pB4−ceA
UB((eA, eB),(2,0)) = pA0 + (1 −pA−pB)(0 + 0)) + pB(0 + 0)−ceB
The Nash (dominant strategies) equilibrium is given by (e∗
A, e∗
B) = (0,0)
3+ system
UA((eA, eB),(2,0)) = pA(5 + ) + (1 −pA−pB)(4 + )) + pB4−ceA
UB((eA, eB),(2,0)) = pA0 + (1 −pA−pB)(0 + )) + pB(1 + )−ceB
The equilibrium is (e∗
A, e∗
B) = ( (1 + )2(1 + )
c(1 + +1+)2,(1 + )(1 + )2
c(1 + +1+))2).
+4 system
UA((eA, eB),(2,0)) = pA(4 + ) + (1 −pA−pB)(4 + )) + pB4−ceA
UB((eA, eB),(2,0)) = pA0 + (1 −pA−pB)(0 + 0)) + pB(1 + 0)−ceB
The equilibrium is (e∗
A, e∗
B) = ( 21
c(1 + )2,12
c(1 + )2).
Since we assume that is sufficiently small, we can infer that teams will
exert more effort under the 3+ system, then in the +4 and finally in the NB.
The comparison of all the possible events yields:
8
Proposition 1. The 3+ system is the Condorcet winner in the comparison
among the point systems. By the same token, teams exert more effort under
the +4 system than in the NB one.
Proof. We analyze the 64 possible events. Table 1 shows the results favoring
team A. By symmetry, analogous results can be found for team B.
Consider the following pairwise comparisons:
•NB vs. 3+: 22 events rank higher under 3+, while 7 under NB.
•+4 vs. N B: 22 events for the former against 7 with the latter.
•3+ vs. +4: 16 with the former against 6 with +4.
This indicates that 3+ is the Condorcet winner, while NB is the Condorcet
loser.
9
Events NB Equilibrium 3+ Equilibrium +4 Equilibrium Ranking
(0,0),(1,1),(2,2) ( (4 + 4)3
4c(4 + 4)2,(4 + 4)3
4c(4 + 4)2) ( (3 + 3)3
4c(3 + 3)2,(3 + 3)3
4c(3 + 3)2) ( (3 + 3)3
4c(3 + 3)2,(3 + 3)3
4c(3 + 3)2)NB 3+ ∼+4
(3,3) ( (4 + 4)3
4c(4 + 4)2,(4 + 4)3
4c(4 + 4)2) ( (3 + 3)3
4c(3 + 3)2,(3 + 3)3
4c(3 + 3)2) ( (4 + 4)3
4c(4 + 4)2,(4 + 4)3
4c(4 + 4)2)NB ∼+4 3+
(4,4),(5,5),(6,6), (7,7) ( (4 + 4)3
4c(4 + 4)2,(4 + 4)3
4c(4 + 4)2) ( (3 + 3)3
4c(3 + 3)2,(3 + 3)3
4c(3 + 3)2) ( (3 + 3)3
4c(3 + 3)2,(3 + 3)3
4c(3 + 3)2)NB 3+ ∼+4
(1,0),(2,1) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 2)2) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 3)2) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 2)2)NB ∼3+ ∼+4
(3,2) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 2)2) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 3)2) ((3 + 2)2(2 + 3)
c(5 + 5)2,(3 + 2)(2 + 3)3
c(5 + 5)2) +4 3+ ∼NB
(4,3) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 2)2) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 3)2) ((2 + 3)2(3 + 2)
c(5 + 5)2,(2 + 3)(3 + 2)3
c(5 + 5)2) +4 3+ ∼NB
(5,4), (6,5), (7,6) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 2)2) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 3)2) ( (2 + 2)3
4c(2 + 2)2,(2 + 2)3
4c(2 + 2)2)NB ∼3+ ∼+4
(2,0) (0,0) ( (1 + )3
4c(1 + 1)2,(1 + )3
4c(1 + )2) ( 2
c(1 + )2,
c(1 + )2) 3+ +4 NB
(3,1) (0,0) ( (1 + )3
4c(1 + 1)2,(1 + )3
4c(1 + )2) ( (1 + )3
4c(1 + 1)2,(1 + )3
4c(1 + )2) 3+ ∼+4 NB
(4,2) (0,0) ( (1 + )3
4c(1 + 1)2,(1 + )3
4c(1 + )2) ( 2
c(1 + )2,
c(1 + )2) 3+ +4 NB
(5,3) (0,0) ( (1 + )3
4c(1 + 1)2,(1 + )3
4c(1 + )2) ( 82
4c(1 + )2,8
4c(1 + )2) 3+ +4 NB
(6,4), (7,5) (0,0) ( (1 + )3
4c(1 + 1)2,(1 + )3
4c(1 + )2) ( 2
c(1 + )2,
c(1 + )2) 3+ +4 NB
(3,0) (0,0) (
c(1 + )2,2
c(1 + )2) (
c(1 + )2,2
c(1 + )2) 3+ ∼+4 NB
(4,1), (5,2) (0,0) (
c(1 + )2,2
c(1 + )2) (0,0) 3+ +4 ∼NB
(6,3) (0,0) (
c(1 + )2,2
c(1 + )2) (
c(1 + )2,2
c(1 + )2) 3+ ∼+4 NB
(7,4) (0,0) (
c(1 + )2,2
c(1 + )2) (0,0) 3+ +4 ∼NB
(4,0), (5,1), (6,2) (0,0) (0,0) (0,0) 3+ ∼+4 ∼NB
(7,3) (0,0) (0,0) (
c(1 + )2,2
c(1 + )2) +4 3+ ∼NB
(5,0), (6,1), (7,2) (0,0) (0,0) (0,0) 3+ ∼+4 ∼NB
(6,0), (7,1) (0,0) (0,0) (0,0) 3+ ∼+4 ∼NB
(7,0) (0,0) (0,0) (0,0) 3+ ∼+4 ∼NB
Table 1: Comparison of point systems
10
3 The Dynamic Model
In this section we model a rugby game following the argumentation line in
Mass´o - Neme (1996). We conceive it as a dynamic game in which the feasible
and equilibrium payoffs of the teams under the three point systems can be
compared. In this setting, we first find the minimax feasible payoffs in every
point system.6This minimax payoff defines a region of equilibrium payoffs.
We consider the Nash equilibriums used to reach this minimax payoffs in
and take the average joint efforts in each system. Again, we want to find
which point system makes the teams spend more effort in order to attain the
equilibirum payoffs.
Formally, let us define a dynamic game as G= ({A, B},(W, (0,0)), E∗, T ),
where:
1. There are again two teams, Aand B. A generic team will be denoted
by i.
2. We restrict the choices of actions to a finite set of joint actions E∗where
E∗={(eA, eB)∈R2
+}where each eiwas used in a Nash equilibrium of
the static game.
3. A finite set of events W, each of which represents a class of equivalent
pairs of scores of the two teams.
•(a, b)∼(7,1) if a > 7 and b= 1
(a, b)∼(7,2) if a > 7 and b= 2
(a, b)∼(7,3) if a > 7 and b= 3
(a, b)∼(7,4) if a−b≥3 and b≥4
(a, b)∼(7,5) if a−b≥2 and b≥5
(a, b)∼(7,6) if a−b≥1 and b≥6
(a, b)∼(7,7) if a=band b≥7
In each case, we say that two scores belong to the same event if the
two teams get the same payoffs in both cases in a finite instantaneous
game in normal form defined as
(a, b)=({A, B }, E∗,(((a,b)
US
i)i∈{A,B})
where S=N B, +4 or 3+ and (a,b)
US
irepresents the utility function of
team iused in the static model in the instantaneous game in the event
(a, b), with the point system S.
6The smallest payoff which the other team can force a team to receive. Formally:
¯vi= mins−imaxsiU(si, s−i).
11
4. All the point systems have the same initial event, namely (0,0).
5. A transition function T, which specifies the new event as a function of
the current event and the joint actions taken by both teams. Therefore
T:W×E∗−→ W.
The transition function has only three possible outcomes (we use a
representative element, i.e. a pair of scores, for any event in W):
(a) T((a, b),(eA, eB)) = (a, b)
(b) T((a, b),(eA, eB)) = (a+ 1, b)
(c) T((a, b),(eA, eB)) = (a, b + 1)
These outcomes represent the fact that, upon a choice of joint efforts,
either no team scores, Ascores or Bscores, respectively.
Some further definitions will be useful in the rest of this work:
Definition 1. For every t∈N, define Htas
ttimes
z }| {
E∗×. . . ×E∗i.e. an element
h∈Htis a history of joint efforts of length t. We denote by H0={e}
the set of histories of length 0, with estanding for the empty history. Let
H=∪∞
t=0Htbe the set of all possible histories in G.
We define recursively a sequence of t+1 steps of events starting with (a, b),
namely {(a, b)j}t
j=0, where (a, b)0= (a, b), . . ., (a, b)t−1=T((a, b), ht−1\ht−2),
(a, b)t=T((a, b)t−1, ht\ht−1), where (h0, . . . , ht)∈His such that for each
j= 0, . . . , t,hj−1is the initial segment of hjand hj\hj−1is the event exerted
at the j-th step.
Definition 2. A strategy of team i∈ {A, B}in the game Gis a func-
tion fi:H→E∗
isuch that for each ht−1, the ensuing htis the sequence
(ht−1,(fA(ht−1), fB(ht−1)).7We will denote by Fithe set of all these func-
tions for team iand, by extension, we define F=FA×FB.
Thus, any f∈Fdefines recursively a sequence of consecutive histories.
We also have that fdefines a sequence of instantaneous games for each
scoring system Sdefined as (a, b)S(f) = {(a, b)S
t(f)}∞
t=0, where each game
corresponds to an event (a, b)S
j: (a, b)S
0= (a, b) and for every t≥1, (a, b)S
t=
T((a, b)S
t−1, f (ht)).
7This assumes perfect monitoring. That is, that teams decide their actions knowing all
the previous actions in the play of the game.
12
Definition 3. A joint strategy f= (fA, fB)∈Fis stationary if for every
h, h0∈Hsuch h=htand h0=ht0and the event generated by both is the
same, namely (¯a, ¯
b), we have that f(h) = f(h0).
That is, a stationary strategy only depends on the event at which it is
applied, and not on how the event was reached.
We note the set of stationary strategies as S ⊆ Fand in what follows we
only consider strategies drawn from this set. In other words, we assume that
teams act disregarding how a stage of the game was reached and play only
according to the current state of affairs. For example, if the match at tis tied
(3,3), teams Aand Bwill play in the same way, irrespectively of whether
the score before was (3,0) or (0,3).
It can be argued that the assumption of stationarity does not seem to hold
in some real-world cases since the way a given score is reached may take an
emotional toll on teams. If, say, Ais winning at (4,0), and suddenly the score
becomes (4,4), the evidence shows that A’s players will feel disappointed and
anxious, changing the incentives under which they act (Cresswell & Eklund,
2006).
But the theoretical assumption of stationarity is clearly applicable to the
case of matches between high performance teams. For instance, consider the
first round of the Rugby World Cup 2015, when All Blacks (New Zealand’s
national team), the best team of the world, played against Los Pumas (Ar-
gentina’s team), an irregular team. At the start of the second half Los Pumas
were 4 points ahead. All Blacks, arguably the best rugby team of the world
(and one of the best in any sport (Conkey, 2017)), instead of losing temper
kept playing in a “relaxed” mode. This ensured that they ended winning the
game by 26 to 16 (Cleary, 2015). So, the assumption of stationarity seems
acceptable for high performance teams, reflecting their mental strength.
We have that,
Lemma 1 (Mass´o-Neme (1996)).Let s∈ S. There exist two natural numbers
M, R ∈Nsuch that (a, b)t+R(s)=(a, b)t+R+M(s)for every t≥1. That is, a
stationary strategy (every strategy in our framework) produces a finite cycle
of instantaneous games of length Mafter Rperiods.
For every s∈ S and a scoring system Swe define bS
(a,b)(s) = {(a, b)S
1(s),...,
. . . (a, b)S
R(s)}and cS
(a,b)(s) = {(a, b)S
R+1(s),...,(a, b)S
R+M(s)}as the initial
path and the cycle of instantaneous games generated by s, where Rand M
are the smallest numbers of Lemma 1. There are many ways in which the
games in a cycle can be reached from the outcomes of another one:
Definition 4. Consider sl, sl0∈ S and (a, b)an initial event under a point
system S.
13
1. We say that sland sl0are directly connected, denoted sl∼sl0, if
cS
(a,b)(sl)∩cS
(a,b)(sl0)6=∅.
2. We say that sland sl0are connected, denoted sl≈sl0, if there exist
s1, . . . , sm∈ S such that sl∼s1∼. . . ∼sm∼sl0.8
Then, we have (for simplicity we assume an initial event (a, b) and a
scoring system S):
Definition 5. For every i∈ {A, B},Ui(s) = ( 1
|s|)ΣM
r=1Uj(r)
i((eA, eB)R+r(s)),
where Uj(r)
iis i’s payoff function in the instantaneous game (a, b)R+r(s)and
(eA, eB)R+ris the profile of choices in that game.
This means that the payoff of a stationary strategy is obtained as the
average of the payoffs of the cycle. To apply this result in our setting, we
have to characterize the set of feasible payoffs of G:
Definition 6. A vector v∈R2is feasible if there exists a strategy s∈ S
such that v= (UA(s),UB(s)).
We have that:
Theorem 1 (Mass´o-Neme (1996)).A vector v∈R2is feasible if and only
if there exists S(v) = {s1, . . . , sk} ⊆ S such that for every sr, sr0∈ S(v),
sr≈sr0and there exists (α1, . . . , αk)∈∆(the k-dimensional unit simplex)
such that
v= Σk
k=1αk(UA(sk),UB(sk)).
The definition of Nash equilibria in this game is the usual one:
Definition 7. A strategy s∗∈ S is a Nash equilibrium of game Gif for all
i∈ {A, B},Ui(s∗)≥Ui(s0)for all s0∈ S, with s∗
i6=s0
iwhile s∗
−i=s0
−i. A
vector v∈R2is an equilibrium payoff of Gif there exists a Nash equilibrium
of G,s∈ S, such that (UA(s),UB(s)) = v.
The following result characterizes the equilibrium payoffs of G:
8In words, two strategies are directly connected if from the cycle of instaneous games
corresponding to one of them, teams have direct access to the cycle of instantaneous games
of the other and vice versa. If instead, they are (not directly) connected, teams can access
from one of the cycles to the instantaneous games of the other one through a sequence of
stationary strategies.
14
Theorem 2 (Mass´o-Neme (1996)).Let vbe a feasible payoff of G. Then vis
an equilibrium payoff if and only if there exist s1, s2, s3∈ S, and (α1, α2, α3)∈
∆3such that v= Σ3
k=1αk(UA(sk),UB(sk)) and the payoff viis better or equal
than the higher payoff that team ican guarantee by itself through a deviation
of the cycles of s1, s2, s3, the connected cycles and the initial path from those
strategies.
In words: vis an equilibrium payoff if each team gets a better pay than
the pay they can get if they deviate.
Finally, with all these definitions and theorems at hand, we can analyze the
game where two teams that are far away in the position table face each
other, so we set = 0 in the utility functions, disregarding the importance
of blocking the attacks of the other team and focusing on scoring tries.
The set of feasible payoffs of this game is given, as said, by a convex
combination of payoffs of stationary strategies. The feasible payoffs region
corresponding to each point system are represented in Figures 2 to 4. In
each figure we can also see the minimax payoffs for the cycles favoring team
A.9Every feasible payoff above and to the right of the minimax payoff is an
equilibrium payoff.
Figure 2 shows the results for the NB system and Table 5 shows the cycles
that receive that minimax payoffs.
Figure 1: Feasible and Minimax payoffs in N B
9Where c(a, b) is the cycle {(a, b)}and c(a, b)(c, d) is the cycle {(a, b),(c, d)}.
15
Minimax payoff Cycle
(2.16,2.16) c(0,0), c(1,1), c(2,2), c(3,3), c(4,4), c(5,5), c(6,6), c(7,7)
(4,1) c(1,0), c(2,1), c(3,2), c(4,3), c(5,4), c(5,5), c(7,6), c(7,5)(7,6)
(4,0.16) c(2,0), c(3,1), c(4,2), c(5,3), c(6,4), c(7,5), c(7,4)(7,5)
(4,0) c(3,0), c(4,1), c(5,2), c(6,3), c(7,4)
c(4,0), c(5,1), c(6,2), c(7,3), c(5,0), c(6,1), c(7,2)c(6,0), c(7,1), c(7,0)
(4,2.16) c(7,6)(7,7)
Table 2: NB System
Figures 3 and Table 6 show the results for the 3+ system.
Figure 2: Feasible payoffs and Minimax payoffs in 3+
Minimax payoff Cycle
(2.16,2.16) c(0,0), c(1,1), c(2,2), c(3,3), c(4,4), c(5,5), c(6,6), c(7,7)
(4,1.53) c(1,0), c(2,1), c(3,2), c(4,3), c(5,4), c(5,5), c(7,6), c(7,5)(7,6)
(4,0.16) c(2,0), c(3,1), c(4,2), c(5,3), c(6,4), c(7,5)
(5,0) c(3,0), c(4,1), c(5,2), c(6,3), c(7,4)
c(4,0), c(5,1), c(6,2), c(7,3), c(5,0), c(6,1), c(7,2)c(6,0), c(7,1), c(7,0)
(4,2.16) c(7,6)(7,7)
(5,0.16) c(7,4)(7,5)
Table 3: 3+ System
16
Finally, Figure 4 and Table 7 do the same for the +4 system.
Figure 3: Feasible payoffs and Minimax payoffs in +4
Minimax payoff Cycle
(2.16,2.16) c(0,0), c(1,1)
(2.53,2.53) c(2,2)
(3.168,3.168) c(3,3), c(4,4), c(5,5), c(6,6), c(7,7)
(4,1.53) c(1,0), c(2,1)
(5,2) c(3,2)
(5,2.53) c(4,3), c(5,4), c(6,5), c(7,6), c(7,5)(7,6)
(4,0.16) c(2,0)
(5,0.168) c(3,1), c(5,2)
(5,0.535) c(4,2)
(5,1.16) c(5,3), c(6,4), c(7,5), c(7,4)(7,5)
(5,0) c(3,0), c(4,1), c(4,0), c(5,1), c(6,2), c(5,0)
c(6,1), c(7,2), c(6,0), c(7,1), c(7,0)
(5,1) c(6,3), c(7,4), c(7,3)
(5,3.168) c(7,6)(7,7)
Table 4: +4 System
17
The fact that some minimax payoffs are outside the feasible region indi-
cates that some cycles do not have equilibrium payoffs, so one or both of the
teams have incentives to change strategies and get a better payoff. When
we consider the joint efforts that yield the minimax payoffs we obtain an
average joint effort of (0,0) in the N B system, (0.18,0.18) in the 3+ system
and (0.1776,0.1776) in the +4 system.
18
4 Empirical Evidence
In order to check the empirical soundness of our theoretical analyses we will
use a database of 473 rugby matches. They were played from 1987 to 2015 in
different competitions, including the Rugby Word Cup, the Six Nations and
club tournaments. We compiled this database drawing data from different
sources ([12]-[23]). Each match is represented by a vector with four compo-
nents, namely the number of tries of the local team, the number of tries of
the visiting team, as well as the scores of the winning and the losing team,
respectively.
We perform a Least Squares analysis to explain the number of tries of each
team and the differences in scores in terms of some explanatory variables. We
consider as such the scoring system used in each match (our key variable),
the nature of each team (a club team or a national team), a time trend and
a constant. The selection of this kind of analysis is justified by, on one hand,
its simplicity, but on the other because we lack a panel or temporal structure
which could provide a richer information. Notice also that it is natural to
posit a linear model in the presence of categorical variables (e.g. the scoring
system in a tournament) (Wooldridge, 2020).
We run OLS regressions on different variants of the aforementioned gen-
eral model, changing the way in which explanatory variables are included
of changing the sample of matches to be analyzed. In the latter case we
divided the entire sample in terms of the homogeneity or heterogeneity of
teams playing in each match. In all cases we had to use robust errors esti-
mators to handle the heteroskedasticity of the models. Also, assuming that
each tournament is idiosyncratic, we controlled for clustered errors.
The general functional form of the model can be stated as:
Ti=β0+β1C2i+β2C3i+γXi+i(1)
There are many alternative ways of characterizing the dependent variable,
which represents the number of tries in a match i, i.e. Ti. The first and
obvious choice is to define it as the total number of tries in a match. But
we also analyze variants in which we allow Tito represent either the number
of tries of the local team, of the visitor team, the difference between them,
those of the winning team (be it as a local or visiting team) and those of the
losing team.
With respect of our variable of interest, i.e. the system of bonus points, we
specify +4 as the categorical base, to compare it to the 3+ and no bonus
systems, represented by means of dummy variables, denoted C2 and C3 for
NB and 3+ respectively. Both γand Xare vectors, containing the control
19
variables and their parameters. We will vary the composition of Xin order
to check the robustness of the effects of the scoring systems. Finally, β0is the
constant, while is the error term (specified to account for heterokedasticity
or clustered errors)
We will first present the descriptive statistics of the database. Then we give
the results of the regressions on the different models built by varying both the
definition of the dependent and the explanatory variables. Finally, we divide
the sample in the classes of matches played by homogeneous or heterogeneous
rivals, to compare their results for the same model.
4.1 Descriptive Statistics
Figure 5 illustrates different aspects of the distribution of the number of tries
in the database of matches. Notice that the number of matches is not the
same under the three scoring methods: for +4 we have 260, 93 under NB
and 120 under 3+. Nevertheless the evidence indicates that the 3+ scoring
method yields the highest scores, hinting that it is the one that induces a
more aggressive play.
4.2 Samples and Exploratory Regressions
We run regressions on different specifications of the general model represented
by expression (1) in order to make inferences beyond the casual evidence. We
use the variable code, to represent the scoring system, with the base value
1 for +4, 2 for NB and 3 for 3+, as we expressed above with the variables
C2 and C3 in (1). For Tiwe use different specifications, namely T riesT otal,
T riesLocal and T riesV is, representing the number of total tries, tries by
the local team and tries of the visiting team, respectively. With respect to
the control variables Xwe use different selections from a set that includes
SR is a dummy variable indicating that a match correspond to a Super
Rugby tournament ( because Super Rugby is clearly different from the other
tournaments analyzed here); Club, which indicates whether a match is played
by club teams or not; previous, a dichotomous variable taking value 1 on the
older matches in our database, namely those played between 1987 and 1991.
Of particular interest are two variables that can be included in X. One is
diff =|T riesLocal −T riesV is|representing the difference in absolute value
between the tries of the local and the visiting team. The other, related,
control variable is dif f2 = T riesLocal −T riesV is, capturing the possible
advantage of being the local team. Finally, we include year as to capture the
possible existence of a temporal trend.
20
(a) Differences of tries
(b) Differences for club teams
Figure 4: Histograms of distributions of differences of tries in each match.
The results can be seen in Table 5.10 It can be seen that 3+ is indeed the
scoring method that achieves the highest number of tries, namely between 1
and over 2 more than +4 (which is our benchmark). N B induces, in general,
less tries than +4, except in the case of number of tries of the visiting teams.
With respect to the control variables, we can see that SR has a negative
impact while Club and previous have a positive influence. The time trend is
10All tables of this section can be found at the end of the article.
21
(a) Total tries
(b) Total tries in homogeneous matches
Figure 5: Histograms of distributions of tries.
not significant in any of the regressions.
22
4.3 The Homogeneous Case
Table 6 presents the results of running the aforementioned regressions but
only on the class of matches between homogeneous teams.11 The dependent
variables of the regressions are on the first rows, where the first four columns
indicate robust errors while the other four give the errors clustered by tour-
naments.
The transition from +4 to NB does not make a difference in robust errors
but it does so for tournament errors, adding a little more than half a try (not
for the losing team, for which it does not make any difference). The effect
of changing from +4 to 3+ is stronger, adding more than 2 total tries and
more than 1 for the winning team.
On the other hand, any of the scoring systems induces almost 2 more total
tries in club tournaments than with national teams. Finally, nor year or the
constant are significant.
4.4 The Non-Homogeneous Case
This analysis, represented in Table 7 is performed on the same variables and
with the same interpretation of errors as the previous case, but including all
the matches.
We do not find differences between NB and +4. 3+, instead, makes a differ-
ence, although with a lower impact than in the homogeneous case. Another
relevant difference is that in this case the effect of Club gets reversed. That
is, winning teams score less while losing ones more, reducing in almost 2 tries
the differences with national teams.
Another interesting feature is that year becomes significant. That is, there
exists a trend towards increasing the differences in time.
4.5 Final Remarks
All the results obtained, both in the general case and distinguishing between
homogeneous and heterogeneous teams, indicate that the results of our the-
oretical models seem to hold in the real world.
11National teams are considered homogeneous if they are in the same Tier ([24]), and
clubs are considered homogeneous if they belong to the same country.
23
5 Conclusions
The results of analyzing rugby games in theoretical and empirical terms are
consistent. The 3+ system induces teams to exert more effort both in the
static and empirical models. Moreover, in the particular instance analyzed
of the dynamic model, with = 0, the result is the same. In all the models,
we find that the 3+ system ranks first, +4 second and NB third.
While choosing different values of in the dynamical model may change a
bit the results, it seems that a sports planner should use the 3+ bonus point
system if the goal is to make the game more entertaining.
Some possible extensions seem appropiate topics for future research. If we
consider as a measure of the “distance” between teams playing in a league,
the choice of the appropriate bonus point system may depend on the teams
and the moment of the tournament they are playing. Incentives at the beg-
gining are not the same at the end of the tournament. The idea of condi-
tionalizing the design of a tournament taking into account this in an optimal
way, can be of high interest.
References
[1] Brocas, I., Carrillo, J.: “Do the ‘Three-Point Victory’ and the ‘Golden
Goal’ Rule Make Soccer More Exciting?”, Journal of Sports Economics,
5.2: 169–185, 2004.
[2] Chiappori, P. A., Levitt, S., Groseclose, T.: “Testing Mixed Strategy
Equilibria when Players are Heterogeneous: the Case of Penalty Kicks in
Soccer”, American Economic Review, 92: 1138–1151, 2002.
[3] Cleary, M., 2015, “New Zealand 26 Argentina 16, Rugby World Cup,
match report: Richie McCaw sin-binned as NZ labour to victory”. Re-
trieved September 27, 2019, from https://www.telegraph.co.uk.
[4] Conkey, B., 2017, “The All Blacks are so Good They Made a
Greatest Teams List Twice”. Retrieved September 5, 2019, from
https://www.theroar.com.au.
[5] Cresswell, S.L., Eklund, R.C.:“Changes in athlete burnout over a thirty-
week ‘rugby year’. Journal of Science and Medicine in Sport 9.1, 125-134,
2006.
[6] Dietl, H., Franck, E., Grossmann, M. and Lang, M.: “Contest Theory and
its Applications in Sports”, The Oxford Handbook of Sports Economics,
24
edited by S. Shmanske and L. Kahane. New York, USA: Oxford University
Press, 2011.
[7] Masso, J., Neme, A.: “Equilibrium Payoffs of Dynamic Games”, Inter-
national Journal of Game Theory, 25.4: 437–453, 1996.
[8] Palomino, F., Rigotti, L., Rustichini, A.: “Skill, Strategy and Passion:
An Empirical Analysis of Soccer”, [Mimeo], Tilburg University, 1999.
[9] Petr´oczi, A., Haugen, K.: “The Doping Self-Reporting Game: The Para-
dox of a ‘False-Telling Mechanism’ and its Potential Research and Policy
Implications”, Sport Management Review, 15.4: 513–517, 2012.
[10] Tullock, G.: “Efficient rent-seeking”, in J. Buchanan, R. Tollison and G.
Tullock, eds, Toward a Theory of Rent Seeking Society. Texas: University
Press: 97?112, 1980.
[11] Walker, M., Wooders, J.: “Minimax Play at Wimbledon”, American
Economic Review, 91: 1521-1538, 2001.
[12] Wikipedia contributors, 2019, “1987 Rugby World Cup”. In Wikipedia,
The Free Encyclopedia. Retrieved 21:31, September 29, 2019, from
https://en.wikipedia.org.
[13] Wikipedia contributors, 2018, “1991 Rugby World Cup”. In Wikipedia,
The Free Encyclopedia. Retrieved 21:32, September 29, 2019, from
https://en.wikipedia.org.
[14] Wikipedia contributors, 2019, “2011 Rugby World Cup”. In Wikipedia,
The Free Encyclopedia. Retrieved 19:56, September 29, 2019, from
https://en.wikipedia.org.
[15] Wikipedia contributors, 2019, “2015 Rugby World Cup”. In Wikipedia,
The Free Encyclopedia. Retrieved 21:31, September 29, 2019, from
https://en.wikipedia.org.
[16] Wikipedia contributors, 2019, “2014 Six Nations Championship”. In
Wikipedia, The Free Encyclopedia. Retrieved 21:47, September 29, 2019,
from https://en.wikipedia.org.
[17] Wikipedia contributors, 2019, “2015 Six Nations Championship”. In
Wikipedia, The Free Encyclopedia. Retrieved 21:47, September 29, 2019,
from https://en.wikipedia.org.
25
[18] Wikipedia contributors, 2019, “2016 Six Nations Championship”. In
Wikipedia, The Free Encyclopedia. Retrieved 21:48, September 29, 2019,
from https://en.wikipedia.org.
[19] Wikipedia contributors, 2019, “2016 European Rugby Champions Cup
season”. In Wikipedia, The Free Encyclopedia. Retrieved 21:33, September
29, 2019, from https://en.wikipedia.org.
[20] Wikipedia contributors, 2019, “2017 European Rugby Champions Cup
season”. In Wikipedia, The Free Encyclopedia. Retrieved 21:24, September
29, 2019, from https://en.wikipedia.org.
[21] Wikipedia contributors, 2019, “2018 European Rugby Champions Cup
season”. In Wikipedia, The Free Encyclopedia. Retrieved 21:22, September
29, 2019, from https://en.wikipedia.org.
[22] Wikipedia contributors, 2019, “2015 Super Rugby season”. In
Wikipedia, The Free Encyclopedia. Retrieved 21:35, September 29, 2019,
from https://en.wikipedia.org.
[23] Wikipedia contributors, 2019, “2018 Super Rugby season”. In
Wikipedia, The Free Encyclopedia. Retrieved 21:36, September 29, 2019,
from https://en.wikipedia.org.
[24] Wikipedia contributors, 2019, “List of international rugby union teams”.
In Wikipedia, The Free Encyclopedia. Retrieved 21:36, September 29,
2019, from https://en.wikipedia.org.
[25] Wooldridge, J. M. 2020. Introductory Econometrics: A Modern Ap-
proach. 7th ed. Boston, MA: Cengage-Learning.
26
Table 5: General Case
(1) (2) (3) (4) (5) (6) (7) (8) (9) (10) (11) (12)
VARIABLES TriesLocal TriesVis TriesTotal diff diff2 TriesLocal TriesVis TriesLocal TriesVis TriesLocal TriesVis TriesTotal
TriesVis -0.146**
(0.067)
2.code -2.339*** 0.760*** -1.579*** -1.576*** -3.099*** -2.228*** 0.624** -1.514*** 0.760*** 0.799
(0.522) (0.284) (0.585) (0.488) (0.603) (0.521) (0.290) (0.395) (0.284) (0.590)
3.code 1.475*** 0.975*** 2.450*** 0.339* 0.006 1.617*** 1.061*** 1.475*** 0.975*** 2.159***
(0.292) (0.250) (0.369) (0.197) (0.303) (0.298) (0.255) (0.291) (0.250) (0.466)
Club -1.375*** 1.088*** -0.287 -1.843*** -2.710*** -1.216*** 1.008*** 1.088***
(0.425) (0.180) (0.461) (0.398) (0.449) (0.428) (0.184) (0.180)
previous 2.022*** 0.103 2.125*** 2.056*** 1.919** 2.037*** 0.220 2.022*** 0.103
(0.632) (0.401) (0.654) (0.575) (0.832) (0.621) (0.410) (0.631) (0.401)
SR -0.775*** -0.033 -0.808** -0.780*** -0.078
(0.278) (0.246) (0.367) (0.280) (0.247)
TriesLocal -0.058**
(0.027)
Club 1.883***
(0.523)
year -0.004
(0.031)
Constant 4.650*** 1.262*** 5.912*** 3.688*** 3.388*** 4.834*** 1.533*** 2.500*** 2.317*** 3.825*** 1.262*** 10.969
(0.391) (0.111) (0.398) (0.380) (0.414) (0.407) (0.169) (0.222) (0.200) (0.192) (0.111) (63.039)
Observations 473 473 473 473 473 473 473 180 180 293 293 245
R-squared 0.090 0.152 0.102 0.123 0.110 0.098 0.159 0.113 0.077 0.045 0.076 0.203
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
27
Table 6: Homogeneus case
(1) (2) (3) (4) (5) (6) (7) (8) (9)
VARIABLES TriesTotal TriesWin TriesLoss Diff TriesTotal TriesWin TriesLoss Diff TriesTotal
2.code 0.799 0.731* 0.067 0.664* 0.799*** 0.731*** 0.067 0.664*** 0.799
(0.590) (0.438) (0.224) (0.369) (0.203) (0.159) (0.125) (0.201) (0.590)
3.code 2.159*** 1.364*** 0.873*** 0.491* 2.159*** 1.364*** 0.873*** 0.491** 2.159***
(0.466) (0.319) (0.211) (0.289) (0.538) (0.304) (0.213) (0.180) (0.466)
Club 1.883*** 1.171*** 0.635*** 0.536* 1.883*** 1.171*** 0.635** 0.536** 1.883***
(0.523) (0.367) (0.213) (0.306) (0.542) (0.308) (0.223) (0.215) (0.523)
year -0.004 -0.013 0.009 -0.023 -0.004 -0.013 0.009 -0.023 -0.004
(0.031) (0.029) (0.011) (0.031) (0.031) (0.023) (0.009) (0.016) (0.031)
Constant 10.969 29.152 -17.653 46.805 10.969 29.152 -17.653 46.805 10.969
(63.039) (58.695) (21.625) (62.070) (61.675) (45.887) (17.956) (32.588) (63.039)
Observations 245 245 245 245 245 245 245 245 245
R-squared 0.203 0.138 0.213 0.030 0.203 0.138 0.213 0.030 0.203
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
28
Table 7: Non Homogeneus case
(1) (2) (3) (4) (5) (6) (7) (8) (9)
VARIABLES TriesTotal TriesWin TriesLoss Diff TriesTotal TriesWin TriesLoss Diff TriesTotal
2.code -0.440 -0.463 0.023 -0.486 -0.440 -0.463 0.023 -0.486 0.799
(0.526) (0.487) (0.147) (0.490) (0.712) (0.683) (0.080) (0.663) (0.590)
3.code 1.907*** 1.122*** 0.785*** 0.336* 1.907*** 1.122*** 0.785*** 0.336*** 2.159***
(0.297) (0.205) (0.146) (0.197) (0.229) (0.118) (0.122) (0.071) (0.466)
Club -0.559 -1.200*** 0.641*** -1.841*** -0.559* -1.200*** 0.641*** -1.841*** 1.883***
(0.438) (0.395) (0.138) (0.398) (0.253) (0.193) (0.131) (0.211) (0.523)
year 0.003*** 0.002*** 0.001*** 0.002*** 0.003*** 0.002*** 0.001*** 0.002*** -0.004
(0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.000) (0.031)
Constant 10.969
(63.039)
Observations 473 473 473 473 473 473 473 473 245
R-squared 0.814 0.766 0.719 0.557 0.814 0.766 0.719 0.557 0.203
Robust standard errors in parentheses
*** p<0.01, ** p<0.05, * p<0.1
29
