ArticlePDF Available

A Lexical Index of Electoral Democracy

Authors:

Abstract and Figures

Recent years have seen an efflorescence of work focused on the definition and operationalization of democracy. One debate concerns the scale, i.e., whether democracy is best measured by binary or graded scales. Critics of binary indices point out at that they are overly reductionist; all features of a regime must be reduced to a single coding decision, producing binary sets that lack discriminating power. Defenders counter that the different levels of graded measures are not associated with a specific set of conditions, meaning that they are difficult to interpret. Against this backdrop, we propose to operationalize electoral democracy as a series of necessary-andsufficient conditions arrayed in an ordinal scale. The resulting "lexical" index of electoral democracy, based partly on new data collected by the authors, covers all independent countries of the world from 1800 to 2008. It incorporates binary coding of its sub-components based on factual characteristics of regimes and in this way reduces the problem of subjective judgments by coders for non-binary democracy indices. Binary codings are aggregated into an ordinal scale using a cumulative logic. In this fashion, we arrive at an index that performs a classificatory function - each level identifies a unique and theoretically meaningful regime type - as well as a discriminating function.
Content may be subject to copyright.
Title
A Lexical Index of Electoral Democracy
Manuscript accepted for publication in Comparative Political Studies. The final, definitive version of this
paper has been published in Comparative Political Studies 48(12), Oct/2015 by SAGE Publications Ltd,
All rights reserved. © Svend-Erik Skaaning, John Gerring, Henrikas Bartusevičius. Link to the
published version: doi: 10.1177/0738894215570423
Abstract
Recent years have seen an efflorescence of work focused on the definition and operationalization of
democracy. One debate concerns whether democracy is best measured by binary or graded scales.
Critics of binary indices point out at that they are overly reductionist, while defenders counter that
the different levels of graded measures are not associated with a specific set of conditions. Against
this backdrop, we propose to operationalize electoral democracy as a series of necessary and
sufficient conditions arrayed in an ordinal scale. The resulting “lexical” index of electoral democracy,
based partly on new data, covers all independent countries of the world from 1800 to 2013. It
incorporates binary coding of its subcomponents, which are aggregated into an ordinal scale using a
cumulative logic. In this fashion, we arrive at an index that performs a classificatory functioneach
level identifies a unique and theoretically meaningful regime-typeas well as a discriminating
function.
Keywords: political regimes, democratization, regime change, democracy index, measurement
Corresponding Author: Svend-Erik Skaaning, Department of Political Science, Aarhus University,
Bartholins Allé 7, 8000 Aarhus C, Denmark. Email: skaaning@ps.au.dk.
2
Recent years have seen an efflorescence of work focused on the definition and operationalization of
democracy. The concept serves as an ongoing touchstone in methodological discussions of concept
formation (e.g., Goertz, 2006; Schedler, 2012; Seawright & Collier, 2014) and new democracy indices
continually appear, which are periodically reviewed and critiqued (e.g., Armstrong, 2011; Coppedge
& Gerring et al., 2011; Gleditsch & Ward, 1997; Knutsen, 2010; Munck, 2009; Vermillion, 2006).
One way to categorize this growing corpus of indicators is by the type of scale employed to
measure the key concept (democracy)—binary, ordinal, or interval. Binary indices include the
Democracy-Dictatorship (DD”) index produced by Przeworski and collaborators (Alvarez et al.,
1996; Cheibub et al., 2010) and an index produced by Boix, Miller, and Rosato (2013, hereafter
“BMR”). Ordinal measures include the Political Rights (“PR”) index and the Civil Liberty (“CL”)
index, both produced by Freedom House (2013), along with the Polity2 index drawn from the Polity
IV database (Marshall et al., 2013). Interval measures include the Index of Democracy produced by
Vanhanen (2000), the Contestation and Inclusiveness indices produced by Coppedge, Alvarez, and
Maldonado (2008), and the Unified Democracy Scores (“UDS) produced by Pemstein, Meserve,
and Melton (2010).
There is much more to a democracy index than its choice of scale. Even so, scaling is a
critical issue in measurement and one that has garnered considerable controversy, especially as
concerns the virtues and vices of binary measures (contrast Elkins, 2000 and Cheibub et al., 2010).
Critics of binary indices point out their reductionist elements: all features of a regime must be
reduced to a single coding decision, producing binary sets that are highly heterogeneous and
borderline cases that may not fit neatly into either category. Binary indices, by construction, lack
discriminating power. Defenders counter that if the definition of these binary sets is properly
grounded in theory, the two-part typology may succeed in identifyingfrom the multifarious
elements of democracy—that condition, or set of conditions, that serves a crucial role in political life
(see Collier & Adcock, 1999). However, this is not an easy claim to sustainwitness the
proliferation of binary indices that identify different defining conditions of democracy.1
We take for granted that different sorts of scales are useful for different purposes. Our aim is
thus not to subsume or replace extant measures of democracy. The discipline is well served by a
variety of measures for this central concept. Instead, we propose a new method of scale
1 A short list would include DD and BMRalready discussedas well as Bernhard, Nordstrom, and Reenock (2001
and the overview of electoral democracies by Freedom House (2013).
3
construction that combines the differentiation of an ordinal scale with the distinct categories of a
typology.
Specifically, we propose to operationalize electoral democracy as a series of necessary-and-
sufficient conditions arrayed in an ordinal scale. We refer to this scaling procedure as lexical.” The
resulting lexical index of electoral democracy, partly based on novel data construction, covers all
independent countries of the world from 1800 to 2013 and is thus the most comprehensive measure
of democracy currently available.2 It incorporates binary coding based on factual characteristics of
regimes and in this way avoids the problem of subjective judgments by coders and the “mashup”
quality of non-binary indices (Ravallion, 2011). However, each binary coding is aggregated together
using the cumulative logic of a lexical scale with seven levels. In this fashion, we arrive at an index
that performs a classificatory functioneach level identifies a unique regime typeas well as a
discriminating function. This approach to measurement offers theoretical and empirical advantages
over other methods of representing the complex concept of electoral democracy that may be useful
in certain settings.
The first section of the paper shows that extant data sets of democracy fall short in
simultaneously providing fine-grained discriminatory power and meaningful categories. The second
section develops conditions that define our lexical index of electoral democracy. The third section
discusses how this index is coded through history and across the universe of independent states. The
fourth section deals with the anticipated validity of the coding. The fifth section explores features of
the lexical index, which is compared with extant indices in the sixth section. The seventh section
applies the new measure to the question of state repression, showing how its fixed meanings to the
different levels inform the interpretation of statistical relationships in a way that is not accessible
through conventional democracy indices. The eight section offers additional thoughts on the
application of the lexical index to causal questions pertaining to democracy. We conclude with a
summary of the value that a lexical approach to measurement may add to our understanding of
electoral democracy.
1. Discrimination vs. Meaningful Categories
2 The dataset (and future editions) can be downloaded at www.ps.au.dk/dedere and
http://thedata.harvard.edu/dvn/dv/skaaning.
4
The Freedom House indices recognize seven categories each, and the Polity2 index twenty-one
categories.3 In contrast to binary indices, the levels in these ordinal indices are not qualitatively
different from each other. A “3” on the PR, CL, or Polity2 scale signifies that a polity is more
democratic than a country coded as “2” but it does not identify specific traits that distinguish polities
falling into each of these categories. Extant ordinal indices identify thus countries with more or less
democracy but not different kinds of democracy. In this respect, they resemble interval scales.
Interval indices of democracy are generally second-order indices. That is, they are
constructed by aggregating together information provided by other democratic indices—through
factor analysis (Contestation and Inclusiveness) or Bayesian latent variable models (UDS). The
exception is Vanhanen’s Index of Democracy. However, the distribution of data on this index is so
highly skewed and so evidently censorednearly 50 percent of the observations are at the zero
point of a 100-point scale—that it loses discriminatory power.4 Thus, our discussion of interval
measures focuses on the Contestation, Inclusiveness, and UDS indices.
The purpose of a well-constructed interval index is to identify fine distinctions among
entities. The Contestation, Inclusiveness, and UDS indices achieve this goal as well as can be
expected. However, the goal of reducing the plenitude of characteristics associated with
“democracy” to a single unidimensional index is elusive. It is elusive because the concept itself is
multidimensional and because extant indicators are limited in their purview (Coppedge & Gerring et
al., 2011). An appropriate response is to define the resulting index in a carefully delineated wayas
representing only one dimension of a multifaceted concept. Thus, Coppedge, Alvarez, and
Maldonado (2008) describe one component from their principal components analysis as
Contestation and the other as Inclusiveness. The UDS is simply described as a measure of
democracy. However, these are ex post descriptions resulting from a rather ad hoc process, putting
together myriad indices—whose definition and construction is often ambiguous and lacks
justificationwith a statistical model and labeling the central tendencies resulting from that model
as “X.” It is unclear, for example, whether an index labeled “Inclusiveness” includes all measures
relevant to that concept and no measures irrelevant to that concept and whether the included
elements are aggregated in an appropriate way. Aggregation techniques are virtually limitless, given
3 The Polity2 index is less discriminating than it appears; countries tend to bunch in two areastoward the bottom of
the index and at the very topproducing a strongly bimodal distribution
4 In addition, Vanhanen’s index has been criticized for low construct validity (Munck, 2009).
5
that researchers must make many choices in the construction of a factor analysis or Bayesian latent-
variable model.5
If all indices are in some sense arbitrary, they are arbitrary in strikingly different ways. The
arbitrariness of a binary scale lies in the choice of necessary condition(s) that define the two
categories. The arbitrariness of an ordinal or interval scale lies in the choice of indicators to include
as elements of the index and the choice of aggregation method to combine those indicators into a
single index.
If all indices are informative, they are informative in strikingly different ways. The
information contained in a binary index is classificatory, that is, it groups polities in a fashion that is
(arguably) theoretically and empirically fecund. The information contained in an interval index is
discriminating, that is, it identifies small differences between polities that allow us to distinguish the
degree to which they possess the core attribute of interest. Ordinal scales occupy a middle position
in this respect. However, extant ordinal indices of democracy perform neither task very well, for
reasons explained above.
2. Developing a Lexical Index
The core meaning of democracy is rule by the people; on this there is little dispute. One theory of
democracy, which can be traced back to E.E. Schattschneider (1942) and Joseph Schumpeter (1950),
among others, proposes that the mechanism by which people exert control over political decisions is
electoral. Citizens are empowered to rule through competitive elections, which allow them to select
leaders and discipline those leaders, establishing relationships of responsiveness and accountability.
By electoral democracy, therefore, we mean a regime where leaders are selected through contested
elections held periodically before a broad electorate.
Our proposed index of democracy focuses explicitly on this electoral model of democracy,
sometimes referred to as a competitive, elite, minimalist, procedural, realist, ‘thin’, or Schumpeterian
conception of democracy (Møller & Skaaning, 2013; Przeworski et al., 2000; Schumpeter, 1950). We
are not concerned with other aspects of democracy such as civil liberties, rule of law, constraints on
executive power, deliberation, or non-electoral mechanisms of participation. Electoral refers to
elections, tout court.
5 Here, the term “factor analysis” is used in a general fashion to refer to a large class of models including principal
components analysis.
6
As such, our definition of the topic is somewhat narrower than definitions of democracy
adopted by most extant indices. This is especially true for indices that assume an ordinal and interval
scale (e.g., PR, CL, Polity2, Contestation, Inclusiveness, UDS), which tend to range widely, including
a broad range of features associated with the concept of democracy. This important definitional
contrast should be highlighted from the outset, as it affects everything that follows.
A lexical approach to measurement is concept-driven (Gerring et al., 2014). Thus, we begin
with a survey of attributes associated with the key concept, electoral democracy, as defined. In
identifying attributes for possible inclusion in our index we are mindful of the vast literature on this
topic, with special attention to linguistic studies of the concept (e.g., Held, 2006; Lively, 1975; Naess
et al., 1956) and foundational works in the electoral tradition (listed above).
To form a lexical scale one must arrange attributes so that each serves as a necessary-and-
sufficient condition within an ordered scale. That is, each successive level is comprised of an
additional condition, which defines the scale in a cumulative fashion. Condition A is necessary and
sufficient for L1; conditions A&B are necessary and sufficient for L2; and so forth. In achieving
these desiderata four criteria must be satisfied: (1) binary values for each condition, (2)
unidimensionality, (3) qualitative differences, and (4) centrality or dependence (see Gerring,
Skaaning, and Pemstein 2014).
First, each level in the scale must be measurable in a binary fashion without recourse to
arbitrary distinctions. It is either satisfied or it is not. To be sure, the construction of a binary
condition may be the product of a set of necessary and/or sufficient conditions. Collectively,
however, these conditions must be regarded as necessary and sufficient.
Second, levels in a lexical scale must be understood as elements of a single latent
(unobserved) concept. Conceptual multidimensionality must be eliminated, either by dropping the
offending attribute and/or by refining the concept in a clearer and perhaps more restrictive fashion,
as we have in moving from “democracy” to “electoral democracy.”
Third, each level must demarcate a distinct step or threshold in a concept, not simply a
matter of degrees. Levels in a lexical scale are intended to identify qualitative differences. A “3” on a
lexical scale is not simply a midway station between “2” and “4.” Indeed, each level may be viewed
as a subtype of the larger concept. In this respect, the lexical index is reminiscent of “diminished
subtypes” of democracy (Collier & Levitsky, 1997; Merkel, 2004). However, while subtypes revolve
in a radial fashion around a central concept – possessing all the attributes of the ideal-type except
7
one – the lexical index is more akin to a classical concept, where new concepts are created by
cumulative combinations of attributesA, A&B, A&B&C, and so forth.
The most challenging aspect of lexical scale construction is the ordering of attributes, which
follows a conceptual, rather than empirical, logic. One attribute may be considered prior to another
if it is more central to the concept of theoretical interest (from some theoretical vantage point). This
follows a constitutive approach to measurement, where attributes are the defining elements of a
concept (Goertz, 2006). Alternatively, one attribute may be considered prior if it is a logical,
functional, or causal prerequisite of another. The dependence of B on A is what mandates that A
assume a lower level on a scale. Whether responding to considerations of centrality or dependence,
the levels of a lexical scale bear an asymmetric relationship to each other; some are more
fundamental than others. This is the most distinctive feature of lexical scaling.6
Based on these considerations, we arrive at a lexical index of electoral democracy with six
conditions and seven levels, as follows:
L0: No elections.
L1: No-party or one-party elections.
L2: Multiparty elections for legislature.
L3: Multiparty elections for legislature and executive.
L4: Minimally competitive, multiparty elections for legislature and executive.
L5: Minimally competitive, multiparty elections with full male or female suffrage for legislature and
executive.
L6: Minimally competitive, multiparty elections with universal suffrage for legislature and executive.
Further elaboration of this minimalist approach to electoral democracy can easily be
envisioned. For example, one might try to measure aspects of electoral integrity such as high respect
for political liberties (see Howard & Roessler, 2006; Møller & Skaaning, 2013). For present
purposes, we restrict ourselves to what might be considered the most basic properties of electoral
6 Where lexical ordering is unclear a priori (according to considerations of centrality and dependence), one is well advised
to consider the shape of the empirical universe. Specifically, if A is always (or almost always) present where B is present,
there may be grounds for considering A as more central or more fundamental than B. However, any conclusions
reached on the basis of an exploration of empirical properties must be justified as a matter of centrality or dependence.
Thus, we regard the relative prevalence of attributes as a clue to asymmetric relationships among the properties of a
concept, not as a desideratum. In constructing a lexical scale, deductive considerations trump data distributions.
8
democracy. Happily, these properties are also the most easily measured, allowing for an index that
stretches back in time and across all independent states. The point is that the index proposed here is
not the only index of electoral democracy that might be constructed. We trust that other approaches
either more detailed or more concise – would nonetheless be consistent with the judgments
incorporated into this index, as discussed below.
Importantly, to qualify as an election (condition #1) the electorate may be quite small but
must be separable from, and much larger than, the group of officials it is charged with selecting.
Examples include South Africa under Apartheid and virtually all national elections in Europe and
the Americas during the nineteenth century. However, the selection of a king by a legislature or
estates general, typical of the Standestaat (Poggi, 1978), would not qualify, as the electorate is
infinitesimal as a share of the citizenry (whom for present purposes we shall understand as
permanent residents in whatever territory is claimed as sovereign), and difficult to distinguish from
the chosen monarch since they both share royal blood and may all claim the title. “Indirect”
elections count as elections unless there are multiple steps in between the electorate and the chosen
representative(s), as in China today and Uganda in the 1970s. It follows that leadership positions
filled through a one-stage electoral college (e.g., US presidents, chosen by an electoral college, or
prime ministers chosen by an elected parliament, who serve as an electoral college) are considered
elective offices.
Having laid out the index, we now explore its rationale in relation to the four criteria
presented above. The first criterion is that each condition be coded in a binary fashion (0/1). This
criterion does little violence to reality as most of the conditions are naturally dichotomous. The
exception is suffrage, a continuous variable. Note, however, that our understanding of an election
presumes an electorate that is considerably larger than the body it selects and separable from that
body. An “election” where 0.0001% of citizens qualify for the vote would not qualify as an election
under our definition. In the event, one does not find any modern examples of multiparty elections
for national offices in independent countries where less than 5% of the electorate can vote. After the
Reform Act of 1832, demarcating the introduction of significant contestation in England, more than
650,000 males – approximately nine percent of the adult population – had the right to vote (Phillips
& Wetherell, 1995: 414). In the United States, around 60-70% of adult white men could vote by
1790 (Keyssar, 2009: 21). Arguably, this feature of the historical record reflects a functional
relationship. If the electorate is miniscule there is less need for an electoral process by which to
choose leaders and establish a relationship of accountability, and even if there is a perceived need it
9
will be difficult to establish and maintain multiparty elections with a miniscule electorate (Gerring et
al. forthcoming). At the other end of the spectrum, nearly universal suffrage elections (where just a
few, small categories of voters are excluded) are understood as universal, and are not, in any case, a
stable category. Once suffrage has been granted to nearly all men or nearly all women, it becomes
very difficult – and also rather pointless – to maintain the barrier. Again, there seems to be a
functional logic at work. Thus, we find that in polities with competitive elections but without
universal male or female suffrage, female suffrage is usually 0 and male suffrage is generally between
20 and 60% of the adult male population. By setting the bar for L5 at 100% we are thus comparing
full (or nearly full) male suffrage with partial male suffrage.
The second criterion concerns unidimensionality, a feature that informs any index. The main
challenge to this objective lies in the twin principles of inclusion and contestation, often regarded as
constituting separate dimensions of electoral democracy (Dahl, 1971). Empirically, there is no
question that these elements are distinct (Coppedge et al., 2008). Countries with high inclusion (as
measured, e.g., by suffrage rights) may have very low contestation, or none at all. However, the
lexical index is theoretically driven rather than empirically driven. Our claim is that once a minimal
level of inclusion has been attained – sufficient to constitute an electorate and hence the pre-
condition for an election, as discussed – further increases in suffrage are irrelevant unless and until
elections are competitive. This argument will be taken up below. For the moment, we note that the
claim to unidimensionality is deductive rather than inductive.
The third criterion concerns qualitative differences across the identified levels. We want to
claim that there is a degree of coherence to each category such that they can be considered as
meaningful regime-types. That is to say, members of each category constitute a set that shares
additional (unmeasured) characteristics. This claim is addressed in §4, where we connect lexical types
with research drawn from the literature on democratization. Relatedly, we suppose that each step in
the index is consequential, at least for some outcomes. This claim is taken up (for one particular
outcome) in §7.
The final criterion concerns the ordering of attributes into a lexical scale according to
centrality or dependence. Recall that this is the most important and controversial aspect of lexical
scaling, and its application to the concept of electoral democracy is by no means self-evident. We
need to carefully explain and justify our choices.
The existence of elections is judged fundamental (conditio sine qua non), as other attributes
associated with electoral democracy make no sense outside of an electoral context (Collier &
10
Adcock, 1999, p. 559; Merkel, 2004, pp. 3638). Country A is not more of an electoral democracy
than Country B if neither polity holds elections, regardless of what other characteristics7 those
polities might possess. Likewise, some attributes depend upon other attributes in a logical manner.
Specifically, an electoral regime is a necessary condition of multiparty elections and multiparty
elections are necessary conditions of competitive elections. Moreover, a regime in which both
legislature and executive are elective is arguably more democratic than a regime in which only one of
these offices is elective. These features of the lexical index may be regarded as self-evident.
Some of the attributes of democracy depend for their meaning on other attributes in a
functional manner. The most important of these involve the relationship of inclusion and
contestation, referenced above. So long as the size of an electorate is non-trivial, we regard the
extent of suffrage as irrelevant to electoral democracy unless and until elections count for
something. The reasoning behind this assessment returns us to the electoral theory of democracy,
according to which citizens are empowered through an electoral connection. In order to establish
relationships of responsiveness and accountability between officials and the citizenry, the electoral
theory suggests that it is essential for political offices to be elective, for citizens do the selecting, for
there to be more than one choice, and for choices occur at regular intervals (introducing the threat
of electoral punishment). If these elements are not present the right of suffrage is meaningless, and
apt to serve as a tool of elite control rather than one of democratic accountability. This logic is
apparent in classic theoretical work in the electoral tradition (e.g., Dahl, 1971; Przeworski et al.,
2000; Schattschneider, 1942; Schumpeter, 1950) and is ratified by recent empirical work (reviewed in
Gandhi & Lust-Okar, 2009).
To gain an intuitive sense for our prioritization of competitiveness over inclusion let us
consider several examples. We begin with electoral authoritarian regimes, where universal suffrage
exists but elections lack multiparty competition or the most important policymaking offices are
nonelective (L1-3 in the lexical index). In our view, nothing of consequence distinguishes electoral
authoritarian regimes that impose limits on suffrage from those that allow universal suffrage. Soviet-
era Russia is not more democratic than pre-revolutionary Russia, despite the inauguration of
universal male suffrage in 1918. Likewise, if an electoral authoritarian regime like North Korea
decided to restrict access to the ballot to certain classes of citizens it would hardly be any less
democratic. Similarly, we regard regimes with minimal competition but restricted suffrage such as
Britain during the nineteenth century as more democratic than, say, present-day Rwanda, which is
7 Such as the non-electoral powerbase or the level of civil liberties, the rule of law, or socioeconomic equality.
11
characterized by universal suffrage but not electoral competition. All of these examples seem to
reinforce the notion that competitiveness stands prior to inclusion in the attainment of electoral
democracy; the latter is functionally dependent upon the former.
3. Coding
To code the lexical index we make use of five variables developed initially in the Political Institutions
and Events (PIPE) dataset (Przeworski et al., 2013): LEGSELEC, EXSELEC, OPPOSITION,
MALE SUFFRAGE, and FEMALE SUFFRAGE. Since PIPE does not attempt to measure the
quality of elections, we generate a sixth variable: COMPETITION. All variables are binary, coded 1
if the following circumstances obtain, and 0 otherwise.
LEGSELEC: A legislative body issues at least some laws and does not perform
executive functions. The lower house (or unicameral chamber) of the legislature is at
least partly elected. The legislature has not been closed.
EXSELEC: The chief executive is either directly or indirectly elected (i.e., chosen
by people who have been elected).
OPPOSITION: The lower house (or unicameral chamber) of the legislature is (at
least in part) elected by voters facing more than one choice. Specifically, parties are
not banned and (a) more than one party is allowed to compete or (b) elections are
nonpartisan (i.e., all candidates run without party labels).
MALE SUFFRAGE: Virtually all male citizens are allowed to vote in national
elections. Legal restrictions pertaining to age, criminal conviction, incompetence, and
local residency are not considered. Informal restrictions such as those obtaining in
the American South prior to 1965 are also not considered.8
FEMALE SUFFRAGE: Virtually all female citizens are allowed to vote in national
elections. Similar coding rules apply.
COMPETITION: The chief executive offices and seats in the effective legislative
body are filled by elections characterized by uncertainty (see Przeworski 2000: 16-
17), meaning that the elections are, in principle, sufficiently free to enable the
8 This is consistent with usage of the suffrage concept by Schumpeter and Przeworski and also with many extant indices
such as BMR.
12
opposition to gain power if they were to attract sufficient support from the
electorate. This presumes that control over key executive and legislative offices is
determined by elections, the executive and members of the legislature have not been
unconstitutionally removed, and the legislature has not been dissolved. With respect
to the electoral process, this presumes that the constitutional timing of elections has
not been violated (in a more than marginal fashion), non-extremist parties are not
banned, opposition candidates are generally free to participate, voters experience
little systematic coercion in exercising their electoral choice, and electoral fraud does
not determine who wins. With respect to the outcome, this presumes that the
declared winner of executive and legislative elections reflects the votes cast by the
electorate, as near as can be determined from extant sources. Incumbent turnover (as
a result of multi-party elections) is regarded as a strong indicator of competition, but
is neither necessary nor sufficient.9 In addition, we rely on reports from outside
observers (as reported in books, articles, and country reports) about whether the
foregoing conditions have been met in a given election (see Svolik 2012: 24). Coding
for this variable does not take into account whether there is a level playing field,
whether all contestants gain access to funding and media, whether media coverage is
unbiased, whether civil liberties are respected, or other features associated with fully
free and fair elections. COMPETITION thus sets a modest threshold.
Although we employ PIPE as an initial source for coding LEGSELEC, EXSELEC,
OPPOSITION, MALE SUFFRAGE, and FEMALE SUFFRAGE, we deviate from PIPE—based
on our reading of country-specific sourcesin several ways. First, with respect to executive
elections, in the PIPE dataset “Prime ministers are always coded as elected if the legislature is open.”
However, for our purposes we need an indicator that also takes into account whether the
government is responsible to an elected parliament if the executive is not directly electeda
situation generated by a number of European monarchies prior to World War I, by episodes of
9 It is not necessary since an incumbent party can be sufficiently popular to win a long sequence of genuinely contested
elections, as happened for decades in, e.g., Botswana, Japan, and Sweden. It is not sufficient because the opposition can
gain power through a flawed election if the incumbents have only weak control on power or have stepped down.
Moreover, the fact that the incumbents step down after a particular election, does not necessarily mean that previous
elections under their leadership were competitiveas it is assumed by the DD if the previous elections took place under
the same electoral rules. That said, in all but a few cases executive turnover in conjunction with elections is associated
with a coding of 1 for COMPETITION.
13
international supervision such as Bosnia-Herzegovina in the first years following the civil war, and
by some monarchies in the Middle East and elsewhere (e.g., Liechtenstein, Monaco, and Tonga) in
the contemporary era. To illustrate, PIPE codes Denmark as having executive elections from 1849
to 1900 although the parliamentary principle was not established until 1901. Before then, the
government was accountable to the king. Among the current cases with elected multiparty
legislatures not fulfilling this condition, we find Jordan and Morocco. In order to achieve a higher
level of concept-measure consistency, we have thus recoded all country-years (based on country-
specific accounts) for this variable where our sources suggested doing so.
We also conduct original coding for countries whose coding is incomplete in PIPE and for
additional countries such as the German principalities that are not covered in PIPE. In this fashion,
we generate a complete dataset for all six variables covering all independent countries of the world
in the period under study (18002013). Whereas the numbers of observations for the PIPE variables
range between 14,465 and 15,302, our dataset provides 18,142 observations for all variables. Except
for minor adjustments regarding executive elections (mentioned above), this additional coding
follows the rules laid out in the PIPE codebook. Coding decisions are based on country-specific
sources that are too numerous to specify. In rare instances we stumbled upon information that
required a re-coding of PIPE variables, so the two datasets do not correspond exactly.
To generate the lexical index from these six binary variables, a country-year is assigned the
highest score (L0–6) for which it fulfills all requisite criteria, as follows:
L0: LEGSELEC=0 & EXSELEC=0.
L1: LEGSELEC=1 or EXSELEC=1.
L2: LEGSELEC=1 & OPPOSITION=1.
L3: LEGSELEC=1 & OPPOSITION=1 & EXSELEC=1.
L4: LEGSELEC=1 & OPPOSITION=1 & EXSELEC=1 &
COMPETITION=1.
L5: LEGSELEC=1 & OPPOSITION=1 & EXSELEC=1 &
COMPETITION=1 & (MALE SUFFRAGE=1 or FEMALE SUFFRAGE=1).10
10 In no extant cases was universal female suffrage introduced before universal male suffrage, so in practice this level is
reserved for countries with male (only) suffrage.
14
L6: LEGSELEC=1 & OPPOSITION=1 & EXSELEC=1 &
COMPETITION=1 & MALE SUFFRAGE=1 & FEMALE SUFFRAGE=1.
Countries are coded across these conditions for the length of their sovereign existence
within the 18002013 timespan, generating a dataset with 221 countries. To identify independent
countries we rely on Gleditsch (2013) and Correlates of War (2011), supplemented from 1800 to
1815 by various country-specific sources. Importantly, electoral democracy does not presume
complete sovereignty. A polity may be constrained in its actions by other states, by imperial control
(as over a colony), by international treaties, or by world markets. Thus, to say that a polity is an
electoral democracy is to say that it functions as such for policies over which it enjoys decision-
making power. Scores for each indicator reflect the status of a country on the last day of the
calendar year (31 December) and are not intended to reflect the mean value of an indicator across
the previous 364 days.
Evidently, a lexical index reduces the potential property space of the component conditions.
Exactly how this works can be seen in Table 1. The first column lists all six conditions, while the
second column shows the number (N) and share (percent) of total observations in our dataset that
meet that criterion. Thus, the first (positive) condition the existence of elections for either the
legislature or executive – is satisfied in 13,584 election-years, constituting 75% of the observations in
our dataset (N=18,142). The second condition multiparty elections for the legislature – is satisfied
in 10,583 election-years, constituting 58% of our total observations. And so forth.
[Table 1 about here]
Coding for the lexical index derives from these six conditions, as indicated in the second
section of Table 1. A polity receives a score of 0 if the first condition is not met, i.e., there are no
elections for either the legislature or the executive. All other conditions are irrelevant. This situation
obtains in 4,569 country-years, constituting 25% of our dataset, as shown in the bottom row of
Table 1. A polity receives a score of 1 if the first condition is met, i.e., there are national elections,
but the second condition (multiparty elections for the legislature) is not satisfied. This situation
obtains in 2,964 country-years, 16% of the country-years recorded in our dataset. The highest (most
demanding) score of 6 is accorded to a polity that satisfies all conditions, as shown in the final
column of Table 1. This situation obtains in 4,870 country years, 27% of the total observations in
our dataset.
In this fashion, any circumstance can be coded unambiguously into the typology. Of course,
many attributes are irrelevant for this coding, as noted in Table 1. Specifically, as soon as a condition
15
is not satisfied all higher conditions become irrelevant. If a polity does not allow for multiparty
elections the extent of suffrage is irrelevant, for example. This “deductive” quality is what
distinguishes a lexical scale from a Guttman or Mokken scale.
4. Validity
When contrasted with most continuous measures of democracy the lexical index is relatively simple,
enhancing its transparency and reproducibility. Coding decisions are generally factual in nature,
resting on institutional features that require historical knowledge but not subjective judgments on
the part of the coder. To be sure, uncertainties are introduced when source material for a country is
weak. But we assume that this sort of bias is random rather than systematic (as it might be if coder
judgments involved questions of meaning and interpretation). In this respect, the lexical index
echoes a feature of most binary indices (e.g., DD and BMR). Indeed, it is quite similar to these
indices insofar as it relies on binary codings, which are combined to form a cumulative index.
Another important feature of the coding procedure is its separability from other factors that
sometimes confound our ability to measure political institutions. When coding democracy and
governance indices—particularly those that assume a continuous distribution—there is a strong
possibility that coders may view the state of democracy or governance in Country X as inseparable
from the general state of affairs in that country, including its economic performance. When things
are going well, X may receive a higher score. When things are going poorly, it may receive a lower
score, even if its political institutions are substantially unchanged (Kurtz & Schrank, 2007). The
coding of the lexical index offers little opportunity for this species of measurement error because
coding decisions rest on clear-cut thresholds and because the features that are being coded are not
amenable to “state of affairs” confounders.
To provide an empirical check on reproducibility we conducted an inter-coder reliability test.
By design, one of the authors (HB) was not involved in the construction of the index or the original
coding of the dataset and was not informed of codings arrived at by the other authors or by the
PIPE dataset. He was then assigned the task of re-coding twenty-two countries (10 percent of the
sample), chosen at random, based on the coding rules presented above and using only country-
specific sources (which he chose based upon his review of the extant literature).
Three standard statistical measures of inter-coder reliability are presented in Table 2: percent
agreement, Cohen’s kappa, and Krippendorff’s alpha. These are calculated at the variable level (for
LEGSELEC, EXSELEC, OPPOSITION, MALE SUFFRAGE, FEMALE SUFFRAGE, and
16
COMPETITION) and at the composite level (for the lexical index). All measures report high levels
of inter-coder reliability, suggesting that the index is readily reproducible. It is worth noting that this
conclusion applies no less to the new competition indicator, although some might consider it to be
less reliable because it is less directly observable.11
[Table 2 about here]
5. Distribution of Regime-types Over Time
A frequency distribution of scores across the entire 1800–2013 period is provided in the bottom row
of Table 1. It will be seen that the most populated categories are L0, L1, L3, and L6, while others
(notably L5) have fewer occupants. A fairly high proportion of cases stack up at the two ends of the
index, in common with many ordinal and interval indices (Cheibub et al., 2010, p. 77; Treier &
Jackman, 2008).
The distribution of cases changes over time, as one might expect. In order to get a feel for
the application of the lexical index, we provide country scores for the median year in our sample,
1904, as shown in Table 3. At that time, there were fifty-three independent countries in the world.
These were distributed fairly evenly across the seven categories of the lexical index, with the
exception of the most democratic category (L6), which has only one occupant. Only Australia
granted universal suffrage to both men and women, while satisfying the other criteria stipulated in
the index. (New Zealand—often considered as the first country to introduce universal suffrage—did
not become independent before 1907 according to our criteria.)
[Table 3 about here]
A comprehensive picture of change over time is portrayed in a stacked graph of the regime-
types across each year, shown in Figure 1. Note that our sample grows over time—from 27 in 1800
to 195 in 2013due to the appearance of newly sovereign states (e.g., in Africa) and the break-up of
sovereign states (e.g., the Soviet Union).
[Figure 1 about here]
At an aggregate level, Figure 1 highlights those periods in which electoral democracy
advanced throughout the world – notably, at the end of World War I, World War II, and the Cold
Waras well as those periods in which it declinednotably, the 1930s. The more important feature
of this diagram, however, is the disaggregated picture of regime evolution it presents. By
11 In case of disagreements, we searched for additional sources and revised the coding if additional information
suggested doing so.
17
decomposing the concept of electoral democracy into constituent parts we can view changes in
membership across regime-types over time.
In 1800 polities were predominantly of type 0 (no elections), which we call non-electoral regimes.
Later in the nineteenth century we see the rise of types 15 and the concomitant decline of non-
electoral regimes. This is the most diverse period, when no single type is dominant, as illustrated by
our snapshot of the world in 1904 (see Table 3).
Over the course of the twentieth century we can see the extraordinary rise of type 1
(elections without multiparty competition), often referred to as one- and no-party regimes (Hadenius &
Teorell, 2007). A steep decline for these regime-types begins in the 1980s, coincident with the Third
Wave of democratization (Huntington 1991).
Apart from some transitional regimes, Type 2 regimes (multi-party elections for legislature
but not the executive) corresponds for the most part to what Therborn (1977: 9) has called non-
parliamentary constitutional monarchies. This regime-type, widespread in nineteenth-century Europe, falls
into desuetude in the contemporary era, describing just a few polities at the present time.
Type 3 regimes (multiparty executive and legislative elections without real competition), a
modestly sized category a century ago, began to grow in the late twentieth century to the point
where it constitutes today the second-most dominant regime-type. This regime-type is similar to
polities described as electoral, competitive, or limited multiparty authoritarian regimes (Schedler, 2002;
Levitsky & Way, 2002; Hadenius & Teorell, 2007). We prefer the latter label since it captures the
intension of the concept.
Exclusive democracies and male democracies, respectively, have been suggested as plausible labels
for Types 4 and 5 (see Collier & Levitsky, 1997; Merkel, 2004). With the growing illegitimacy of
suffrage restrictions, these regime-types have become virtually extinct in the 21st century, though
they constituted a significant share of all polities prior to World War II.
A final and equally striking pattern in evidence over the past century is the rise of type 6
regimes, the highest level of our lexical index, corresponding to polities that satisfy all assessed
criteria for electoral democracy. This category, largely capturing what democratization scholars have
referred to as electoral democracies (Diamond, 2002; Møller & Skaaning, 2013), now comprises over half
of all polities in the world.
6. Contrasts with Extant Indices
18
Table 4 summarizes salient features of the lexical index alongside the nine extant democracy indices
introduced at the outset. It will be seen that the lexical index has much broader historical coverage
than DD, PR, CL, Contestation, Inclusiveness, and UDS—all of which are focused on the
contemporary eraand slightly better coverage than Polity2, BMR, and Vanhanen’s Democracy
index.
[Table 4 about here]
As is to be expected, the lexical index generally co-varies with other indices. For example, it
correlates with Polity2 at 0.80 and with the Political Rights index at 0.85 (Spearman’s rho). However,
when the highest scoring cases (lexical=6) are dropped from the sample these correlations drop to
0.59 and 0.42, respectively. If we split the sample, distinguishing between years before and after
1900, inter-correlations between the full lexical index and the BMR are 0.46 for the nineteenth
century and 0.83 for the twentieth century, while inter-correlations with Polity2 are 0.65 and 0.82.
Thus, while the lexical index overlaps with other indices of democracy it is by no means redundant.
A more detailed look at the relationships between extant binary (DD, BMR) and ordinal (PR,
Polity2) indices and the lexical index is portrayed in cross-tabulations in Table 5. This confirms that
while various measures of electoral democracy are related, they are not very highly correlated.
[Table 5 about here]
One might infer that the lexical index is an outlier among democracy indices. However, a
principal components analysis, shown in Table 6, reveals that this is not the case. Again, we find a
striking contrast between full sample and partial sample results. In the full sample, 83 percent of the
variance across these ten indices is explained by the first component. In the partial sample
(Lexical<6), only 52 percent of the variance can be explained by the first component. However, in
neither analysis is the lexical index an outlier, as shown in the eigenvalues.
[Table 6 about here]
In elucidating the distinctive features of our lexical index a useful point of comparison is
provided by binary indices. The latter generally combine several of the features identified in our
ordinal scale. For example, DD may be said to combine L1–4 while BMR combines L1–5, with
suffrage understood as a majority of men rather than all men. In doing so, the authors suggest that a
polity cannot be called an electoral democracy until it has satisfied a number of conditions—though
these conditions do not exactly map onto the condition utilized to score the lexical index, as shown
in Table 5.
19
Our index does not take issue with this determination. However, a lexical approach to
scaling suggests that polities that fail to pass all four or five of these conditions may nonetheless be
regarded as partial members of the class “electoral democracy.” For example, a polity with elections
is closer to the electoral ideal than a polity without elections. And it suggests that this distinction
along with others identified along the seven-level index—has consequences, consequences that can
be understood as greater/lesser possession of various traits associated with electoral democracy.
Thus, rather than insisting that a number of necessary conditions be met, we regard each condition
as providing a threshold on a single ordinal scale.
Clearly, the lexical index allows one to represent more information than is possible in a
binary scale. At the same time, the sensitivity of a seven-level ordinal scale is lower than that
provided by a longer ordinal scale (e.g., Polity) and much lower than an interval scale (e.g., UDS). In
terms of discriminatory ability, the lexical index occupies a midway point.
The advantage of lexical scaling relative to more differentiated ordinal scales or interval
scales is in clarity. While the latter are derived from complex models (e.g., UDS) or less formulaic
but often opaque weightings across dimensions (e.g., Freedom House and Polity), the lexical value
affixed to a country in a particular year is immediately interpretable. We know what a “5” means
because there is only one combination of attributes that will yield a score of 5 on a lexical scale.
Likewise, we can understand the categories of the scale as indicating discrete regime-types,
which can be tracked through time, as in Figure 1. By way of contrast, longer ordinal scales (e.g.,
Polity) and interval indices (e.g., UDS) allow one to track the overall trends—more or less
democracy through time—by examining changes in the mean over time. But they cannot indicate
anything about the specific content (quality) of regimes or about which regime-types expanded or
contracted at different points in time. The latter information is both substantively important as well
as useful for tracing causal mechanisms, as discussed below.
7. The Lexical Index at Work: Democracy and State Repression
One purpose of the lexical index of electoral democracy is descriptive: to differentiate regime-types
in the world (Table 2) and to portray changes over time (Figure 1). Another use is to probe causal
relationships between regime-type and other factors. As an example of this sort of work we shall
explore the relationship between regime-type and state repression of personal integrity rights
(Davenport & Armstrong, 2004).
20
Democracies are expected to be less repressive than autocracies for a variety of reasons.
First, a democratic framework is thought to promote tolerance. Second, low respect for human
rights may be punished by the electorate at the ballot box. Finally, political participation and
contestation provide an outlet for protests and secure legitimacy in the broader population,
alleviating the extra-constitutional challenges that often spur violent government repression. Extant
theory thus presents a strong prima facie case for political regime-type as an influence on state
repression.
However, it is not clear what the precise empirical relationship might be. Extant work on the
subject suggests three possible patterns. As summarized by Davenport and Armstrong (2004: 538
39): (1) “with every step toward democracy, the likelihood of state-related civil peace is enhanced”;
(2) “human rights conditions are not only improved when full democracy exists but also when full
autocracy is present”; or (3) “there may…be some threshold of domestic democratic peace, below
which there is no effect of democracy on repression, but above which a negative influence can be
found.”
Our interest in this question is heuristic. We probe the empirical relationship between
electoral democracy and state repression in order to demonstrate how the lexical index may be
brought to bear on a causal hypothesis where countries are the relevant units of analysis. Specifically,
we wish to utilize the special qualities of the lexical index in order to gain insight into the
mechanisms at work in this (putatively) causal relationship.
To simplify things, we adopt the empirical format employed by Davenport and Armstrong
(2004), with some minor modifications to update the analysis through 2004.12 We readily grant that
there are other approaches to causal modeling that might be adopted in this instance. However,
since our purpose is to compare extant indices—rather than to make causal claimsdifficult choices
among estimators, specifications, and samples may be put aside.
Following Davenport and Armstrong, state repression is measured by the Political Terror
Scale (PTS), in turn based on the State Department human rights country reports (Wood and
Gibney 2010). We enlist OLS regression to assess the model and employ a battery of covariates
including interstate armed conflict (UCDP/PRIO), internal armed conflict (UCDP/PRIO), military
dictatorship (Cheibub et al., 2010), population (ln) (PWT), GDP/cap (ln) (PWT), and a one-period
lag of the outcome. Democracy is measured in the first instance by the 10-point Polity Democracy
index (scaled from 0 to 10) drawn from the Polity IV dataset (Marshall et al., 2013).
12 Apart from the Lexical index, all data used are taken from the QoG standard dataset (Teorell et al., 2013).
21
Our second measure of democracy is the lexical index, with one notable coding change. Data
on state repression are available only from 1976, meaning that there is little variation in suffrage laws
during the observed period. Distinctions across L4L6 of the lexical index are therefore rendered
moot, prompting us to collapse them into a single category (L4). The resulting index has five
levelsL0L4, with roughly equal membership—and is otherwise identical to the index described
above.
To probe possible links between regime-type and state repression, we adopt a series of
approaches, summarized in Table 7. First, we test the possibility of a linear relationship. Polity
(Model 1) and Lexical (Model 2) both indicate a negative relationship: more democracy is correlated
with less repression, corroborating the general theory but leaving the problem of causal explanation
opaque. Next, we test the possibility of a curvilinear relationship by introducing a multiplicative
term. The coefficients for Polity (Model 3) and Lexical (Model 4) are similar, though only Polity
offers support for the notion that democracy’s impact on repression is nonlinear.
[Table 7 about here]
Finally, we attempt to explore each category of these indices separately through the use of
dummy variables representing each level (with the first level omitted as a reference category).
Results, shown in Models 5 and 6, are again broadly similar across the two indices, though there are
some important differences. The coefficient for L1 in the Polity index is significantly more
repressive than the reference category, L2–6 do not show results are statistically distinguishable
from the null, and L7–10 show negative, and statistically significant coefficients. By contrast, L1, L2,
and L4 in the lexical index are statistically significant from the reference category, but not L3.
Additional tests (not reported) show that the differences between L4 on the one hand and L1, L2,
and L3 on the other are significant. The only additional significant difference is that between L1 and
L3. We have re-run all the analyses holding sample constant across the parallel models based on
Polity and Lexical, respectively. The results (not reported) are virtually identical, meaning that
varying coverages do not account for the differences.
Leaving aside for a moment the question of which index offers a truer representation of the
relationship between democracy and repression, let us consider what might be learned from Models
5 and 6. Davenport and Armstrong (2004: 548) conclude that “there are important differences
between the political systems associated with the highest levels of the Polity measure …” This is a
reasonable conjecture. But they cannot follow this statement up with any speculation about what is
distinctive about the higher levels of the Polity index or what might be driving the apparently
22
curvilinear relationship between democracy and repression. This is because the levels of the Polity
index are not individually interpretable. In this respect, ordinal indices of democracy such as Polity,
PR, and CL function very much like interval indices. They inform us about quantities (more or less
of some latent trait) but not about qualities (categorical differences across levels).
By contrast, the lexical index provides ample fodder for theorizing because each level defines
a discrete category and each category is plausibly approached as a regime-type. Let us begin by
reviewing the information contained in Model 6. No level in the lexical index reveals higher levels of
state repression than level L0 (no elections). While it is unsurprising to discover that a non-electoral
state has high levels of repression (for all the reasons set forth in our initial theory), it is somewhat
surprising to find that there is no (statistically significant) difference in levels of repression across L0
and L3. If the model is correct, repression decreases significantly when a polity moves
(hypothetically) from no national (L0) elections to a situation of national elections (L1), national
multiparty elections for the legislature (L2), or—most effectivelyminimally competitive elections
for the legislature and the executive, while the degree of state repression in a situation of multiparty
elections for legislative and executive offices that are not minimally competitive (L3) is not
significantly different from a situation without national elections.
An explanation may be found in the hybrid nature of the L3 regimes, which are
characterized by many of the constitutional features of democracy without the crucial missing step in
which elections are allowed to become competitive. That is, L3 polities look as if they are
democraticand undoubtedly are portrayed by their leaders as democratic. But even though
opposition groups are free to organize and to participate in the political system, they are not allowed
to win government power (Schedler, 2002). Some of the hybrid features of this setting are likely to
engender more repression than in the other settings characterized by national elections. Because the
opposition is free to organize, it is likely to pose a significant challenge to the government. And
because the elections are not free, the opposition is likely to pursue extra-constitutional measures,
which in turn are likely to provoke government repression. In short, both government and
opposition have means and motive to engage in a cycle of protest and retaliation, a setting that is
likely to feed levels of state repression that are indistinguishable from settings without national
elections.
8. Extensions
23
In this section, we discuss possible applications of the lexical index for understanding causal
questions about democracy. To begin, let us reemphasize that the short explanatory sketch offered
in the previous section is not intended to convince. In order to be fully convincing, a causal
argument would need to be accompanied by a much longer theoretical discussion intended to make
sense of case-based evidence and extant theorizing on this well-trodden subject—not to mention a
battery of robustness tests. Our purpose is illustrative. We hope to have shown that a lexical
approach to measurement provides a useful tool for gaining insight into causal relationships and
specifically into the causal mechanisms that may be at work. This feature derives from the fact that
the levels of a lexical scale are individually meaningful.
By contrast, binary scales are generally too diffuse to be useful in this context. Country-years
scored as 0 are different from country-years scored as 1 in many ways. It is not clear which of these
differentiating features might be responsible for a causal effect or whether their impact is
combinatorial (a compound treatment). Extant ordinal scales can, in principle, be disaggregated into
their component parts, as we have done with the Polity2 index. However, since these components
are not uniquely defined, they are not very informative. We know that L3 is higher than L2, but this
is about all we know. Interval scales may also be disaggregated. However, establishing the break-
points is a highly arbitrary affair, and the resulting categories contain no useful information.
We suspect that the same aspects that render the lexical index useful in the context of state
repression might also be useful in the context of other research questions where regime-type lies on
the right side of a causal model. Consider the vaunted democratic peace hypothesis (Brown et al.,
1996). While a new scale of democracy will assuredly not solve this obdurate research question by
itself, it does allow a more nuanced test of the thesis (at least as pertains to the electoral components
of democracy). Specifically, we can explore whether there is a specific level in the lexical index
beyond which conflict between nations ceases to occur and whether one or both members of the
dyad must surpass that threshold. This is arguably more informative than a binary or
ordinal/interval analysis of the problem.
As a second example, one might consider the contested relationship between development
and democracy (Przeworski et al., 2000). With democracy on the left side of the model, one may
investigate whether the empirical relationship of socioeconomic development to electoral democracy
is different at various points in the lexical index. Do increases in per capita GDP have a greater
impact on electoral democracy at certain thresholds? With democracy on the right side of the model,
one might investigate whether different thresholds of electoral democracy have varying relationships
24
to economic growth. For example, does the initial transition to multiparty elections have a different
impact on growth performance than the transition to competitive elections?13
9. Conclusions
The lexical approach to index construction is unique in that each level in the ordinal scale is defined
by an additional attribute of the core concept (electoral democracy). These cumulative attributes are
assigned according to theoretical expectations rather than empirical distributions, as would be the
case for Guttman and Mokken scales. This produces levels that correspond to distinct types. These
types are of great value in understanding the progress/regress of democracy around the world,
grouping regimes into similar categories, and explaining various outcomes of interest. Note that
while any ordinal scale can be disaggregated into categories corresponding to each level, this does
not normally reveal groupings that have much in common with each other. (There are many ways to
receive a ”-6” on the Polity2 index, for example.) Additionally, the lexical index has greater country
and historical coverage than any extant index of democracy. It may also claim greater precision than
most indices by virtue of the largely factual coding criteria (demonstrated by high inter-coder
reliability) and simple aggregation procedures.
It should be clear that in launching a new index of electoral democracy we are not proposing
that the lexical index has any claim to ontological priority over other sorts of indices, each of which
represent certain aspects of reality (while occluding others) and each of which has its uses.
Sometimes, relationships are continuous and hence are best measured with an interval scale.
Sometimes, they have only one threshold and hence are best measured with a binary scale. Our
claim is that, sometimes, descriptive and causal relationships are ordinal in character or require an
ordinal scale to test various threshold possibilities. In these settings, which may apply to many
theories about democratic development (as cause or effect), a lexical scale may be appropriate. Here,
ordinal levels are constructed in order to represent qualitatively different categories. These categories
are informative insofar as they are fecund, attaining the desiderata of any classificatory scheme, that
is, to group phenomena in categories that are mutually exclusive and exhaustive.
Note that the utility of a lexical definition of democracy (like that of all others) rests
ultimately on how well it explains the world around us. The electoral interpretation of democracy
13 In placing the lexical index on the left side of a causal model one would of course want to employ an appropriate
estimator. Traditionally, ordinal outcomes are tested with ordered logit or ordered probit models. One might also
construct binary variables using different cut-points on the lexical index, which could then be analyzed with logit or
survival models.
25
presumes that one dimension of democracy—grounded in elections—has the greatest impact on
governance, wellbeing, and perhaps also on other aspects of democracy (liberal, deliberative, et al.).
It treats the electoral component as causally exogenous. Likewise, our lexical index is premised on a
notion of which features of electoral democracy are likely to be most fundamental. On this basis, we
included some attributes and excluded others and arrived at a lexical ordering of those that were
included. Whether this construction of the world is fruitful rests on empirical investigations that
unfold over time. Our attempt in this study is to demonstrate that this approach to
conceptualization and measurement bears further exploration.
26
Acknowledgements
Previous versions of this article were presented at the annual meetings of the American Political
Science Association, the University of Amsterdam, Aarhus University, and the University of
Heidelberg. We thank participants at these meetings for their comments. We also acknowledge
valuable feedback received from Michael Coppedge, Adam Glynn, Gary Goertz, Staffan Lindberg,
James Mahoney, Jørgen Møller, Gerardo Munck, Alexander Schmotz, Andries van der Ark, and the
anonymous reviewers for the journal.
Funding
The authors received financial support for the research from the Danish Council for Independent
Research [11932].
27
References
Alvarez, R. M., Cheibub, J.A., Limongi, F., & Przeworski, A. (1996). Classifying political regimes.
Studies in Comparative International Development, 31(2), 3–36.
Armstrong II, D.A. (2011). Stability and change in the Freedom House political rights and civil
liberties measures. Journal of Peace Research 48:5, 65362.
Bernhard, M., Nordstrom, T., & Reenock, C. (2001). Economic performance, institutional
intermediation, and democratic survival. Journal of Politics, 63(3), 775803.
Boix, C., Miller, M. & Rosato, S. (2013). A Complete Dataset of Political Regimes, 18002007.
Comparative Political Studies, 46(12), 152354.
Brown, M.E., Lynn-Jones, S.M., & Miller, S.E. (eds.) (1996). Debating the democratic peace. Cambridge:
MIT Press.
Cheibub, J.A., Gandhi, J. & Vreeland, J.R.. (2010). Democracy and dictatorship revisited. Public
Choice, 143(1–2), 67101.
Collier, D. & Levitsky, S. (1997). Democracy with adjectives: Conceptual innovation in comparative
research. World Politics, 49(3), 430-51.
Collier, D. & Adcock, R. (1999). Democracy and dichotomies: A pragmatic approach to choices
about measures. Annual Review of Political Science, 2, 53765.
Coppedge, M., Alvarez, A., & Maldonado, C. (2008). Two Persistent Dimensions of Democracy:
Contestation and Inclusiveness. Journal of Politics, 70(3), 33550.
Coppedge, M. & Gerring, J. with Altman, D., Bernhard, M., Fish, S., Hicken, A., Kroenig, M.,
Lindberg, S.I., McMann, K., Paxton, P., Semetko, H.A., Skaaning, S.-E., Staton, J., & Teorell, J.
(2011). “Conceptualizing and measuring democracy: A new approach. Perspectives on Politics, 9(1),
24767.
Correlates of War. 2011. State system membership (v2011).
http://www.correlatesofwar.org/COW2%20Data/SystemMembership/2011/System2011.html
Dahl, R.A. (1971). Polyarchy: Participation and opposition. New Haven: Yale University Press.
Davenport, C. & Armstrong, D. (2004). Democracy and the Violation of Human Rights: A
Statistical Analysis from 19761996. American Journal of Political Science, 48(3), 53854.
Diamond, L. (2002). Thinking about hybrid regimes. Journal of Democracy, 13(2), 21-35.
Elkins, Z. (2000). Gradations of democracy? Empirical tests of alternative conceptualizations.
American Journal of Political Science, 44(2), 28794.
28
Freedom House (2013). Freedom in the world 2013. http://www.freedomhouse.org/report/freedom-
world-2013/methodology, accessed December 16, 2013.
Gandhi, J. & Lust-Okar, E. (2009). Elections Under Authoritarianism. Annual Review of Political
Science, 12, 40322.
Gerring, J., Palmer, M., Teorell, J., Zarecki, D. (forthcoming). Demography and democracy: A
global, district-level analysis of electoral contestation. American Political Science Review.
Gerring, J., Skaaning, S.-E., & Pemstein, D. (2014). A concept-driven approach to measurement: The lexical
scale. Unpublished manuscript. Boston: Boston University.
Gleditsch, K.S. (2013). List of Independent States. http://privatewww.essex.ac.uk/~ksg/statelist.html.
Gleditsch, K.S. & Ward, M.D. (1997). Double take: A re-examination of democracy and autocracy in
modern polities. Journal of Conflict Resolution, 41(3), 36183.
Goertz, G. (2006). Social science concepts: A users guide. Princeton: Princeton University Press.
Hadenius, A. & Teorell, J. (2007). Pathways from authoritarianism. Journal of Democracy, 18(1), 143
56.
Held, D. (2006). Models of democracy. Stanford: Stanford University Press.
Howard, M. & Roessler, P. (2006). Liberalizing electoral outcomes in competitive authoritarian
regimes. American journal of Political Science, 50:2, 36581.
Huntington, S.P. (1991). The third wave: Democratization in the late twentieth century. Norman, OK:
University of Oklahoma Press.
Keyssar, A. (2009). The right to vote: The contested history of democracy in the United States. New York: Basic
Books.
Knutsen, C.H. (2010). Measuring effective democracy. International Political Science Review, 31(2), 109
28.
Kurtz, M. & Schrank, A. (2007). Growth and governance: Models, measures, and mechanisms.
Journal of Politics, 69(2), 53854.
Levitsky, S. & Way, L. (2002). The Rise of Competitive Authoritarianism.” Journal of Democracy, 13(2),
5155.
Lively, J. (1975). Democracy. New York: St. Martin’s.
Marshall, M.G., Gurr, T. & Jaggers, K. (2013). Polity IV project: Dataset users’ manual.
http://www.systemicpeace.org/inscr/p4manualv2012.pdf, accessed December 16, 2013.
Merkel, W. (2004). Embedded and defective democracies. Democratization, 11(1), 3358.
29
Møller, J., Skaaning, S.-E. (2013). Regime types and democratic sequencing. Journal of Democracy,
24(1), 14256.
Munck, G.L. (2009). Measuring democracy: A bridge between scholarship and politics. Baltimore: John
Hopkins University Press.
Naess, A., Christophersen, J., & Kvalo, S. (1956). Democracy, ideology, and objectivity. Oslo:
Universitetsforlaget.
Pemstein, D., Meserve, S., & Melton, J. (2010). Democratic compromise: A latent variable analysis
of ten measures of regime type. Political Analysis, 18(4), 42649.
Phillips, J. & Wetherell, C. (1995). The Great Reform Act of 1832 and the political modernization of
England. American Historical Review, 100(2), 411-36.
Poggi, G. (1978). The development of the modern state: A sociological introduction. Stanford, CA: Stanford
University Press.
Przeworski, A., Alvarez, M., Cheibub, J.A., & Limongi, F. (2000). Democracy and development: Political
institutions and material well-being in the world, 19501990. New York: Cambridge University Press.
Przeworski, A. (2013). Political Institutions and Political Events (PIPE) Data Set.
https://sites.google.com/a/nyu.edu/adam-przeworski/home/data
Ravallion, M. (2011). “Mashup indices of development. World Bank Research Observer, 27, 1–32.
Schattschneider, E. E. 1942. Party Government. New York: Rinehart.
Schedler, A. (2002). The menu of manipulation. Journal of Democracy, 13(2), 3650.
Schedler, A. (2012). Judgment and measurement in political science. Perspectives on Politics, 10(1), 21
36.
Schumpeter, J.A. (1950). Capitalism, socialism and democracy. New York: Harper & Bros.
Seawright, J. & Collier, D. (2014). Rival strategies of validation: Tools for evaluating measures of
democracy. Comparative Political Studies, 47(1), 11138.
Svolik, M. (2012). The politics of authoritarian rule. New York: Cambridge University Press.
Teorell, J., Charron, N., Dahlberg, S. Holmberg, S., Rothstein, B., Sundin, P., & Svensson, R. (2013).
The quality of government dataset, version 20Dec13. University of Gothenburg: The Quality of
Government Institute. http://www.qog.pol.gu.se/, accessed February 1, 2014.
Treier, S. & Jackman, S. (2008). Democracy as a Latent Variable. American Journal of Political Science,
52(1), 20117.
Vanhanen, T. (2000). A new dataset for measuring democracy, 18101998. Journal of Peace Research,
37(2), 25165.
30
Vermillion, J. (2006). Problems in the measurement of democracy. Democracy at Large, 3(1), 2630.
Wood, R. & Gibney, M. (2010). The political terror scale (PTS): A re-introduction and a comparison
to CIRI. Human Rights Quarterly, 32(2), 367400.
31
Figure 1:
Absolute Distribution of Political Regimes, 18002013
L0=non-electoral regimes. L1=one- and no-party regimes. L2=non-parliamentary constitutional
monarchies. L3=limited multiparty authoritarian regimes. L4=exclusive democracies. L5=male
democracies. L6=electoral democracies.
0
50
100
150
200
6543210
32
Table 1:
Frequency Distribution, 18002013
_________CONDITIONS__________ _____________LEXICAL INDEX_____________
Sa tisfied
L0
L2
L3
L4
L5
L6
A.
Elections
13,584
(75%)
N Y Y Y Y Y Y
B.
Multiparty elections for legislature
10,583
(58%)
I N Y Y Y Y Y
C.
Multiparty elections for executive
9,080 (50%)
I
N
Y
Y
Y
Y
D.
Minimally competitive elections
6,287 (35%)
I
I
N
Y
Y
Y
E.
Full male or female suffrage
11,335
(62%)
I I I I N Y Y
F.
Universal suffrage
9,553 (53%)
I
I
I
I
N
Y
4,569
(25%)
1,522
(8%)
2,808
(15%)
929
(5%)
480
(3%)
4,870
(27%)
Numbers represent observations (N) and share (%) of total sample (18,142 country-years) satisfying
the specified condition(s), rounded to the nearest integer. N=No, Y=Yes, I=Irrelevant.
33
Table 2:
Inter-coder Reliability
Agreement
(%)
Cohen’s
Kappa
Krippendorff’s
alpha
LEGSELEC
93.72 .831 .831
EXSELEC
95.63
.903
.903
OPPOSITION
94.54
.888
.888
MALE SUFFRAGE 95.49 .898 .898
FEMALE SUFFRAGE
96.38
.926
.926
COMPETITION
96.66 .920 .920
LEXICAL INDEX
87.37
.840
.943
Countries (randomly) selected into the inter-coder reliability test: Equatorial Guinea, Cameroon, Zambia, Iraq, South
Korea, Lebanon, Korea (pre-1910), Hesse Darmstadt (Grand Duchy of Hesse), Solomon Islands, St. Lucia, Malta,
Kyrgyzstan, Peru, Trinidad and Tobago, Romania, Israel, Parma, West Germany, Burundi, Marshall Islands, Argentina,
and Syria.
34
Table 3:
Frequency Distribution, 1904
0 1 2 3 4 5 6
-
National
elections
+
National
elections
+
Multiparty
elections for
legislature
+
Multiparty
elections for
executive
+
Competitive
elections
+
Male or
female suffrage
+
Universal
suffrage
Afghanistan
China
Ethiopia
Iran
Korea
Montenegro
Nepal
Oman
Ottoman Emp.
Russia
Thailand
Colombia
Cuba
Haiti
Honduras
Liberia
Mexico
Nicaragua
Paraguay
Venezuela
Austria-
Hungary
Bulgaria
Germany
Japan
Portugal
Romania
Spain
Sweden
Argentina
Bolivia
Brazil
Dominican Rep.
Ecuador
El Salvador
Guatemala
Italy
Peru
Serbia
Chile
Costa Rica
Denmark
Luxembourg
Netherlands
United
Kingdom
Uruguay
United States
Belgium
Canada
France
Greece
Panama
Switzerland
Australia
N=53.
35
Table 4:
Indices Compared
INDEX SCALE COVERAGE CORRELATION
(with Lexical)
Type
Range
Countries
Years
Obs
All obs
Lexical<6
Lexical (authors)
Lexical
0–6
221
18002013
18,142
DD (Cheibub et al.)
Binary
0–1
197
19462008
9,115
.84 (S)
.37 (S)
BMR (Boix et al.)
Binary
0–1
210
18002007
16,308
.78 (S)
.45 (S)
Polity2 (Marshall et al.)
Ordinal
-1010
185
18002012
16,327
.80 (S)
.59 (S)
PR (Freedom House)
Ordinal
1–7
200
19722012
7,040
.85 (S)
.42 (S)
CL (Freedom House)
Ordinal
1–7
200
19722012
7,040
.79 (S)
.31 (S)
Democracy Index (Vanhanen)
Interval
0–100*
192
18102010
15,149
.78 (P)
.48 (P)
Contestation (Coppedge et al.)
Interval
-1.841.96
197
19502000
7,534
.91 (P)
.65 (P)
Inclusiveness (Coppedge et al.)
Interval
-3.041.91
197
19502000
7,534
.59 (P)
.54 (P)
UDS (Pemstein et al.)
Interval
-2.112.26
201
19462012
9,850
.87 (P)
.62 (P)
*=theoretical range. S = Spearman’s rho. P = Pearson’s correlation coefficient. Country counts are based on COW
country codes (extended with additional, unique country codes for Orange Free State, Transvaal, Tibet, and United
Provinces of Central America, as suggested by Gleditsch), whereas years and observations are taken from the original
datasets.
36
Table 5:
Cross-tabulations
Lexical
L0
L1
L2
L3
L4
L5
L6
Sum
DD
0
1,378
1,949
290
1,181
50
2
259
5,109
1
0
10
26
170
101
49
3,587
3,943
Sum
1,378
1,959
316
1,351
151
51
3,846
9,052
BMR
0
3,796
2,878
1,242
2,410
403
90
183
11,002
1
8
10
43
92
480
390
3,906
4,929
Sum
3,804
2,888
1,285
2,502
883
480
4,089
15,931
PR
0
421
502
18
113
0
0
0
1,054
1
373
483
49
350
1
0
2
1,258
2
150
202
78
275
15
0
30
750
3
22
54
51
257
4
0
206
594
4
3
16
12
67
0
0
499
597
5
2
0
21
26
0
0
959
1,008
6
0
0
21
1
0
0
1,734
1,756
Sum
971
1,257
250
1,089
20
0
3,430
7,017
Polity2
0
1,126
123
40
0
0
0
0
1,289
1
256
624
114
126
0
0
4
1,124
2
131
216
77
75
0
1
0
500
3
658
779
212
109
0
0
0
1,758
4
677
222
160
202
0
0
1
1,262
5
120
186
41
205
20
0
9
581
6
58
78
258
197
24
3
1
619
7
245
263
88
426
45
52
15
1,131
8
21
24
34
166
39
1
3
288
9
64
125
51
253
6
4
25
528
10
119
44
11
143
8
9
22
356
11
146
34
76
100
8
7
3
374
12
15
41
44
186
66
22
39
413
13
9
24
8
81
76
14
45
257
14
87
10
28
131
161
8
81
506
15
6
23
11
103
50
19
179
391
16
7
3
26
31
53
22
315
457
17
8
13
1
53
52
37
340
504
18
5
12
8
33
36
46
571
711
19
0
1
1
6
50
56
437
551
20
3
2
0
10
187
176
1,896
2,274
Sum
3,761 2,847 1,289 2,633 881 477 3,986 15,874
PR and Polity2 rescaled so that 0 = lowest value.
37
Table 6:
Principal Components Analysis
Full sample
Partial sample
Component
Eigenvalue
Difference
Proportion
Cumulative
Eigenvalue
Difference
Proportion
Cumulative
1
8.32
7.59
0.83
0.83
5.18
3.85
0.52
0.52
2
0.73
0.40
0.07
0.90
1.33
0.11
0.13
0.65
3
0.33
0.11
0.03
0.94
1.22
0.55
0.12
0.77
4
0.22
0.09
0.02
0.96
0.67
0.24
0.07
0.84
5
0.13
0.04
0.01
0.97
0.43
0.03
0.04
0.88
6
0.10
0.03
0.01
0.98
0.40
0.11
0.04
0.92
7
0.07
0.01
0.01
0.99
0.29
0.04
0.03
0.95
8
0.06
0.03
0.01
1.00
0.25
0.11
0.03
0.98
9
0.03
0.01
0.00
1.00
0.14
0.07
0.01
0.99
10
0.02
.
0.00
1.0000
0.08
.
0.01
1.00
EIGENVECTORS
Full sample
Partial sample
Variable
Comp1
Unexplained
Comp1
Unexplained
Lexical
0.33
0.10
0.32
0.46
DD
0.31
0.18
0.21
0.78
BMR
0.32
0.14
0.16
0.87
Polity2
0.33
0.09
0.35
0.35
PR
0.33
0.08
0.36
0.34
CL
0.32
0.17
0.32
0.49
Democracy index
0.32
0.17
0.33
0.43
Contestation
0.34
0.03
0.41
0.14
Inclusiveness
0.20
0.66
0.18
0.83
UDS
0.34
0.06
0.41
0.13
Principal factor analysis of democracy indices (unrotated). Number of observations = 4028 (full
sample) and 2431 (partial sample, where Lexical <L6). Factors retained: 1.
38
Table 7:
Electoral Democracy as a Predictor of State Repression: Lexical and Polity Compared
Linear Curvilinear Disaggregated
1
2
3
4
5
6
Polity
-.025 (.003) ***
.051 (.011) ***
Polity2
-.009 (.001) ***
Polity,
L1
.120 (.045) **
Polity,
L2
-.067 (.050)
Polity,
L3
-.010 (.067)
Polity,
L4
.073 (.067)
Polity,
L5
-.033 (.058)
Polity,
L6
-.011 (.040)
Polity,
L7
-.064 (.045)
Polity,
L8
-.119 (.037) ***
Polity,
L9
-.170 (.041) ***
Polity,
L 10
-.400 (.037) ***
Lexical
-.037 (.007) ***
-.002 (.029)
Lexical2
-.008 (.007)
Lexical,
L1
-.132 (.031) ***
Lexical,
L2
-.128 (.060) *
Lexical,
L3
-.044 (.033)
Lexical,
L4
-.223 (.031) ***
PTSsdt–1
.691 (.011) ***
.715 (.010) ***
.669 (.011) ***
.714 (.010) ***
.659(.012) ***
.702 (.010) ***
Interstate conflict
.057 (.054)
.019 (.053)
.086 (.053)
.021 (.053)
.088(.053) *
.031 (.053)
Internal conflict
.371 (.026) ***
.375 (.025) ***
.387 (.026) ***
.377 (.025) ***
.388(.026) ***
.381 (.025) ***
Military dictator
.042 (.027)
.057 (.026) *
.096 (.028) ***
.056 (.026) *
.086(.028) **
.041 (.026)
Population (ln)
.051 (.007) ***
.042 (.006) ***
.055 (.007) ***
.042 (.006) ***
.056(.007) ***
.044 (.006) ***
GDP/cap. (ln)
-.051 (.009) ***
-.073 (.008) ***
-.025 (.010) **
-.070 (.008)***
-.022(.010) *
-.071 (.008) ***
Constant
.727 (.095) ***
.929 (.090) ***
.497 (.100) ***
.896(.094) ***
.499(.101) ***
.975 (.095) ***
R
2
.748
.770
.752
.770
.754
.772
N
3553
3924
3553
3924
3553
3924
Sample period: 19762004. Countries: 155/165. Estimator: OLS. Standard errors in parentheses. *<.1,
**<.01, ***<.001 (two-tailed test). L=levels on an ordinal scale (not lags).
39
Table A1:
Variables, Definitions, Sources
DEMOCRACY INDICES
BMR. To qualify as democratic a country must satisfy the following conditions: “(1) The executive is directly or
indirectly elected in popular elections and is responsible either directly to voters or to a legislature, (2) the legislature
(or the executive if elected directly) is chosen in free and fair elections, (3) a majority of adult men has the right to
vote” (Boix, Miller, and Rosato 2013: 1531).
CL. Civil Liberties, an index measuring the respect for civil liberties (Freedom House 2013), reversed scale.
Contestation. An index derived from the first component of a principal components analysis including a large number
of democracy indicators (Coppedge, Alvarez, and Maldonado 2008).
DD. Democracy-dictatorship index. To qualify as democratic a country must satisfy the following conditions: “(1) The
chief executive must be chosen by popular election or by a body that was itself popularly elected, (2) The legislature
must be popularly elected, (3) There must be more than one party competing in the elections, (4) An alternation in
power under electoral rules identical to the ones that brought the incumbent to office must have taken place”
(Cheibub, Gandhi, and Vreeland 2010: 69).
Democracy index (Polity). The upper half of the Polity2 index, stretching from 0 to 10 (Marshall, Gurr, and Jaggers
2013).
Democracy index (Vanhanen). The product of (1) the vote-share or seat-share of all but the largest party and (2) the
share of adult population that voted (Vanhanen 2000).
Inclusiveness. An index derived from the second component of a principal components analysis including a large
number of democracy indices (Coppedge et al. 2008).
Lexical. Lexical index of electoral democracy, as described in text.
PR. Political Rights, an index measuring the extent of political rights (Freedom House 2013), reversed scale.
Polity2. Polity2 index, combining Autocracy and Democracy variables, from the Polity IV dataset (Marshall, Gurr, and
Jaggers 2013).
UDS. Unified Democracy Score, derived from an IRT model including a large number of democracy indicators
(Pemstein, Meserve, and Melton 2010).
OTHER VARIABLES
PTSsd. Political Terror Scale (US State Department), an index measuring levels of political violence that a country
experiences in a particular year based on a “terror scale” developed by Freedom House. The scale ranges from 1
(“Countries are under secure rule of law, people are not imprisoned for their views, and torture is rare or exceptional.
Political murders are extremely rare”) to 5 (“Terror has expanded to the whole population. The leaders of these
societies place no limits on the means or thoroughness with which they pursue personal or ideological goals”).
Interstate conflict. An armed conflictas defined by the UCDP/PRIO Armed Conflict Dataset (i.e., a contested
incompatibility that concerns government or territory where the use of armed force between two parties, of which at
least one is the government of a state, results in at least 25 battle-related deaths)that occurs between two or more
states.
Internal conflict. An armed conflictas defined by the UCDP/PRIO Armed Conflict Datasetthat occurs between
the government of a state and one or more internal opposition group(s). This category also includes “internationalized
internal conflicts,” incompatibilities with intervention from other states on one or both sides.
Military dictatorship. A binary variable indicating whether a country is a military dictatorship, defined as a regime in
which the executive relies on the armed forces to come to and stay in power (Cheibub, Gandhi, and Vreeland 2010).
Population size. Population size in thousands, from the Penn World Tables.
GDP/cap. Real GDP per capita (chain series) in constant prices, from the Penn World Tables.
40
Table A2:
Descriptive Statistics for Variables used in Table 7
Obs
Mean
SD
Min
Max
Democracy index (Polity)
4176
4.097
4.169
0
10
Lexical
5101
2.585
1.577
0
4
PTSsd
4504
2.331
1.152
1
5
Interstate conflict
4606
0.030
0.172
0
1
Internal conflict
4606
0.220
0.414
0
1
Military dictatorship
5037
0.184
0.388
0
1
Population size (ln)
4790
8.509
2.006
2.851
14.071
GDP/cap. (ln)
4728
8.327
1.295
5.081
11.158
Sample period: 1976–2004 (corresponding to analysis in Table 7).
41
Table A3:
Country-year coverage of the dataset
Afghanistan
1800-2013
Albania
1913-1938
1944-2013
Algeria
1816-1829
1962-2013
Andorra
1994-2013
Angola
1975-2013
Antigua and Barbuda
1981-2013
Argentina
1816-2013
Armenia
1991-2013
Australia
1901-2013
Austria
1918-1937
1945-2013
Austria-Hungary
1800-1917
Azerbaijan
1991-2013
Baden
1806-1870
Bahamas
1973-2013
Bahrain
1971-2013
Bangladesh
1971-2013
Barbados
1966-2013
Bavaria
1806-1870
Belarus
1991-2013
Belgium
1830-1939
1945-2013
Belize
1981-2013
Benin
1960-2013
Bhutan
1949-2013
Bolivia
1825-2013
Bosnia and Herzegovina
1992-2013
Botswana
1966-2013
Brazil
1822-2013
Brunei
1984-2013
Bulgaria
1878-2013
Burkina Faso
1960-2013
Burundi
1962-2013
Cambodia
1953-2013
Cameroon
1960-2013
Canada
1867-2013
Cape Verde
1975-2013
Central African Republic
1960-2013
Chad
1960-2013
Chile
1818-2013
China
1800-2013
42
Colombia
1830-2013
Comoros
1975-2013
Congo Brazzaville
1960-2013
Congo, Democratic Republic
1960-2013
Costa Rica
1840-2013
Cote d'Ivoire
1960-2013
Croatia
1991-2013
Cuba
1902-1905
1909-2013
Cyprus
1960-2013
Czech Republic
1993-2013
Czechoslovakia
1919-1937
1945-1992
Denmark
1800-1939
1945-2013
Djibouti
1977-2013
Dominica
1978-2013
Dominican Republic
1844-1915
1924-2013
East Timor
2002-2013
Ecuador
1830-2013
Egypt
1827-1854
1922-2013
El Salvador
1840-2013
Equatorial Guinea
1968-2013
Eritrea
1993-2013
Estonia
1918-1939
1991-2013
Ethiopia
1855-1935
1941-2013
Fiji
1970-2013
Finland
1917-2013
France
1800-1940
1941-2013
Gabon
1960-2013
Gambia
1965-2013
Georgia
1991-2013
Germany
1867-1944
1990-2013
Germany, East
1949-1989
Germany, West
1949-1989
Ghana
1957-2013
Gran Colombia
1821-1829
Greece
1828-1940
1944-2013
43
Grenada
1974-2013
Guatemala
1840-2013
Guinea
1958-2013
Guinea-Bissau
1974-2013
Guyana
1966-2013
Haiti
1804-1914
1934-2013
Hanover
1816-1870
Hesse-Darmstadt
1815-1870
Hesse-Kassel
1813-1870
Honduras
1840-2013
Hungary
1918-2013
Iceland
1944-2013
India
1947-2013
Indonesia
1945-2013
Iran
1800-2013
Iraq
1932-2013
Ireland
1921-2013
Israel
1948-2013
Italy
1861-2013
Jamaica
1962-2013
Japan
1800-1944
1952-2013
Jordan
1946-2013
Kazakhstan
1991-2013
Kenya
1963-2013
Kiribati
1979-2013
Korea
1800-1909
Korea, North
1948-2013
Korea, South
1948-2013
Kosovo
2008-2013
Kuwait
1961-1989
1991-2013
Kyrgyzstan
1991-2013
Laos
1954-2013
Latvia
1918-1939
1991-2013
Lebanon
1944-2013
Lesotho
1966-2013
Liberia
1847-2013
Libya
1800-1833
1951-2013
Liechtenstein
1990-2013
Lithuania
1918-1939
1991-2013
44
Luxembourg
1867-1939
1945-2013
Macedonia
1991-2013
Madagascar
1800-1895
1960-2013
Malawi
1964-2013
Malaysia
1963-2013
Maldives
1965-2013
Mali
1960-2013
Malta
1964-2013
Marshall Islands
1986-2013
Mauritania
1960-2013
Mauritius
1968-2013
Mecklenburg-Schwerin
1815-1870
Mexico
1821-2013
Micronesia
1986-2013
Modena
1815-1860
Moldova
1991-2013
Monaco
1993-2013
Mongolia
1921-2013
Montenegro
1878-1914
2006-2013
Morocco
1800-1903
1956-2013
Mozambique
1975-2013
Myanmar
1800-1884
1948-2013
Namibia
1990-2013
Nauru
1968-2013
Nepal
1800-2013
Netherlands
1800-1939
1945-2013
New Zealand
1907-2013
Nicaragua
1840-2013
Niger
1960-2013
Nigeria
1960-2013
Norway
1905-1939
1945-2013
Oman
1800-2013
Orange Free State
1854-1901
Ottoman Empire
1800-1920
Pakistan
1947-2013
Palau
1994-2013
Panama
1903-2013
Papal states, the
1815-1869
45
Papua New Guinea
1975-2013
Paraguay
1811-1869
1876-2013
Parma
1815-1860
Peru
1824-2013
Philippines
1946-2013
Poland
1918-1938
1945-2013
Portugal
1800-2013
Prussia
1800-1866
Qatar
1971-2013
Romania
1878-2013
Russia
1800-1921
1991-2013
Rwanda
1962-2013
Samoa
1962-2013
San Marino
1992-2013
Sao Tome and Principe
1975-2013
Sardinia
1815-1860
Saudi Arabia
1927-2013
Saxony
1806-1870
Senegal
1960-2013
Serbia
1878-1914
2006-2013
Serbia-Montenegro
1992-2005
Seychelles
1976-2013
Sicily
1800-1860
Sierra Leone
1961-2013
Singapore
1965-2013
Slovakia
1993-2013
Slovenia
1992-2013
Solomon Islands
1978-2013
Somalia
1960-2013
South Africa
1910-2013
South Sudan
2011-2013
Spain
1800-2013
Sri Lanka
1948-2013
St. Kitts and Nevis
1983-2013
St. Lucia
1979-2013
St. Vincent and the Grenadines
1979-2013
Sudan
1956-2013
Suriname
1975-2013
Swaziland
1968-2013
Sweden
1800-2013
Switzerland
1815-2013
46
Syria
1946-2013
Taiwan
1949-2013
Tajikistan
1991-2013
Tanzania
1961-2013
Thailand
1800-2013
Tibet
1913-1949
Togo
1960-2013
Tonga
1970-2013
Transvaal
1852-1876
1881-1901
Trinidad and Tobago
1962-2013
Tunisia
1800-1880
1956-2013
Turkey
1921-2013
Turkmenistan
1991-2013
Tuscany
1815-1860
Tuvalu
1978-2013
USSR
1922-1990
Uganda
1962-2013
Ukraine
1991-2013
United Arab Emirates
1971-2013
United Kingdom
1800-2013
United Provinces of Central America
1823-1839
United States
1800-2013
Uruguay
1830-2013
Uzbekistan
1991-2013
Vanuatu
1980-2013
Venezuela
1829-2013
Vietnam
1800-1892
1975-2013
Vietnam, North
1954-1974
Vietnam, South
1954-1974
Wuerttemburg
1806-1870
Yemen
1990-2013
Yemen, North
1918-1989
Yemen, South
1967-1989
Yugoslavia
1918-1941
1944-1991
Zambia
1964-2013
Zanzibar
1963
Zimbabwe
1965-2013
... This result implies that in classifying a country as a democracy, the conventional cut-offs of FH, Polity and RoW require a more stringent set of conditions than those prescribed in the dichotomous measures. We further conduct analogous analyses using different benchmark variables: BMR's democracy coding including female suffrage and the eight-fold regime classification from Skaaning et al. (2015)'s Lexical Index of Electoral Democracy (LIED). 4 Our results confirm that the conventional cut-off levels of FH, Polity and EDI represent a higher standard than the procedural minimum notion of democracy. ...
... Third, using these measures, we empirically derive the cut-off point for each continuous measure that best approximates the dichotomous measures' regime classification. Fourth, we conduct similar optimisation analyses with two alternative sets of benchmark measures: BMR's extension variable that includes female suffrage condition, and Skaaning et al. (2015)'s LIED. In the conclusion section, we summarise the implications of this study. ...
... Alternatively, Bogaards suggested choosing the Polity components that presumably measure the phenomenon of interest and using them either as-is or combining them (Bogaards, 2012). Also, Skaaning et al. (2015) developed an eight-point electoral democracy scale that allows users to interpret the meaning of each category. In contrast, we tried to preserve the idea of using cut-off points for continuous democracy measures by providing reference points for conceptual evaluation. ...
... Of the five attributes of democracy, arguably the most essential and least contested is representation (Beetham 1999: 155, 162-63). It emphasizes contested and inclusive popular elections for legislative and directly or indirectly elected executive office (Dahl 1971; see also Alvarez et al. 1996;Boix, Miller and Rosato 2014;Skaaning, Gerring and Bartusevičius 2015;Møller and Skaaning 2011;Munck 2009). Most of the features associated with representative government are covered by the concepts of electoral integrity (see Norris 2014), free and fair elections (see Elklit and Svensson 1997) and electoral democracy (see Diamond 1999). ...
... Against this backdrop, the aim of the subdivision is to establish more conceptual coherence and to bring together features that are frequently combined in the academic literature. Finally, the inclusiveness of elections is now represented by a separate category as recommended in the literature (see Paxton 2000;Coppedge, Alvarez and Maldonado 2008;Skaaning, Gerring and Bartusevičius 2015;Munck 2016). ...
Book
The Global State of Democracy is a biennial report that aims to provide policymakers with an evidence-based analysis of the state of global democracy, supported by the Global State of Democracy (GSoD) Indices, in order to inform policy interventions and identify problem-solving approaches to trends affecting the quality of democracy around the world. This document revises and updates the conceptual and measurement framework that guided the construction of Version 5 of the GSoD Indices, which depicts democratic trends at the country, regional and global levels across a broad range of different attributes of democracy in the period 1975–2020. The data underlying the GSoD Indices is based on a total of 116 indicators developed by various scholars and organizations using different types of source, including expert surveys, standards-based coding by research groups and analysts, observational data and composite measures.
... 9 I define autocracy using the democracy measure from the updated version of Boix, Miller, and Rosato (2013), which defines a country as an autocracy if it does not regularly hold free and fair elections. To test my theoretical prediction, I disaggregate electoral authoritarian regimes from other autocracies using the classification in Skaaning, Gerring, and Bartusevičius (2015). This data contains information about the following circumstances: (1) legislative elections, (2) executive elections, (3) opposition parties in legislatures, (4) male suffrage, (5) female suffrage, and (6) free and fair electoral competition. ...
... * p < 0.1, ** p < 0.05, *** p < 0.01. , 2015Madagascar 2013, 2015Mali 2012Mozambique 2008, 2015Namibia 2008Nigeria 2008Sudan 2013Tanzania 2008Togo 2012Tunisia 2013Uganda 2008, 2011, 2015Zimbabwe 2009 Note: I only include countries that regularly hold multiparty elections (Skaaning et al. 2015). Note: Multilevel mixed-effects models estimate the impact of the electoral cycle on the perception of government redistribution. ...
Article
Full-text available
While political budgetary cycles in democracies have been rigorously studied for the past several decades, surprisingly little is known about electorally motivated policy manipulation in authoritarian regimes. This study analyzes how dictators strategically change the priorities of autocratic policies to cultivate electoral dominance even when election results are predetermined. I argue that dictators spend more money on redistributive policies in election periods. Using budgetary spending data from 63 autocratic countries between 1972 and 2015, this paper presents cross-national evidence of the existence of an electoral cycle in autocratic redistribution. Analyzing Afrobarometer survey data from 18 African autocracies between 2008 and 2015, this study also finds that citizens’ evaluations of redistributive policy fluctuate according to the electoral calendar. These findings contribute to the literature on authoritarian politics by exploring macro- and micro-level mechanisms through which authoritarian rulers improvise policy manipulation to cultivate electoral dominance.
... In the early 2000s, Asian democracies began to modify the inherent majoritarianism of their systems, embarking on new reforms of a more consensual nature. Table 1 overviews the outcomes of these institutional reforms in countries with multiparty elections in East, Southeast, and South Asia. 2 The cells in bold letters indicate institutional changes that have taken place 2 As well as the Freedom House threshold noted earlier, for case selection we consulted the Lexical Index of Electoral Democracy (LIED) database created by Skaaning et al. (2015). The countries under study were coded as three and higher for at least 10 years between 1990 and 2020 in LIED, denoting that the given country-year had multiparty elections for both legislature and executive branches (although such elections did not necessarily fulfil the competitive election condition, that is, elections were not characterised by uncertain outcomes.) ...
Article
We investigate whether and how women’s political empowerment relates to technological change, the main driver of long-term economic growth. We argue that three aspects of empowerment – descriptive representation, civil liberties protection, and civil society participation – advance technological change and thereby economic growth through (a) increasing the number and variability of new ideas introduced in the economy and (b) improving the selection of more efficient ideas. Drawing on data from 182 countries and 221 years, we test various implications from our argument. Women’s political empowerment is positively related to subsequent economic growth. This relationship persists across various model specifications and when accounting for different potential confounders. The three sub-components of empowerment are also, individually, related to growth, although not as strongly as the aggregated concept. The relationship is retained across different contexts, but is clearer for “Non-Western” countries and in earlier time periods. We also find evidence that women’s political empowerment enhances technological change.
Article
Aggregation tools transform multidimensional data into indices. To investigate how the design of an aggregation process affects regression results, we build democracy indices that differ regarding their scale and aggregation function. Using the democracy-growth nexus as a testing ground, we illustrate that the choice of the aggregation procedure significantly affects OLS and 2SLS estimates since different methods produce systematically different index values for observations at the lower and upper end of the autocracy-democracy spectrum. We also illustrate that dichotomous measures produce significantly smaller OLS estimates than continuous measures due to lower discriminating power. Whether continuous and dichotomous indicators create different 2SLS estimates depends on their design. Because of the methodological similarities of democracy indicators and other social science indicators, we expect similar consequences for other empirical analyses.
Article
A large literature addresses the impact of regimes on domestic policies and outcomes, e.g., education, health, inequality, redistribution, public spending, wages, infrastructure, volatility, productivity, and economic growth. This study focuses on another vital outcome – industrialization – that has yet to be systematically explored using cross-country data. We argue that autocratic leaders are more likely to adopt an economic model of development centered on heavy industry because of three factors that distinguish democratic and autocratic regimes: different social bases, different security concerns, and different policy tools. Accordingly, autocracies have stronger incentives and better capabilities to pursue a rapid and comprehensive course of industrialization. We test this hypothesis using different measures of industrialization in a dataset spanning 200 years and most countries of the world. After a comprehensive series of tests, we conclude that industrialization stands out as one of the few areas where autocracies may enjoy a significant advantage over democracies.
Book
Full-text available
The Global State of Democracy is a biennial report that aims to provide policymakers with an evidence-based analysis of the state of global democracy, supported by the Global State of Democracy Indices (GSoD Indices), in order to inform policy interventions and identify problem-solving approaches to trends affecting the quality of democracy around the world. This document presents revised and updated information about all the variables included in the GSoD indices data set that enabled the construction of Version 5 of the GSoD Indices, which depicts democratic trends at the country, regional and global levels across a broad range of different attributes of democracy in the period 1975–2020. The data underlying the GSoD Indices is based on a total of 116 indicators developed by various scholars and organizations using different types of source, including expert surveys, standards-based coding by research groups and analysts, observational data and composite measures.
Article
Scholars frequently dichotomize continuous measures of democracy by setting a regime cut-off. However, such cut-offs often lack theoretical or empirical justifications, making the resulting classifications difficult to interpret conceptually. We investigate this challenge involving three major continuous democracy measures: the Freedom House score (FH), the Polity score, and the Regime of the World (RoW) that is based on the V-Dem's Electoral Democracy Index (EDI). We develop a framework to empirically derive thresholds using categorical democracy measures as benchmarks. Our analyses find that the cut-offs that yield the highest consistency with the classifications of BMR, CGV, and GWF are 3.5 for FH, 5 for Polity and 0.39 for EDI/RoW. These levels are lower than the conventional cut-offs, implying less demanding democratic standards. Consequently, the conventional cut-offs (2.5 for FH, 6 for Polity and 0.5 for EDI/RoW) endeavour to reflect more stringent standards of democracy than what these dichotomous measures employ.
Article
A key obstacle to measurement is the aggregation problem. Where indicators tap into common latent traits in theoretically meaningful ways, the problem may be solved by applying a data-informed (“inductive”) measurement model, for example, factor analysis, structural equation models, or item response theory. Where they do not, researchers solve the aggregation problem by appeal to concept-driven (“deductive”) criteria, that is, aggregation schemes that do not presume patterns of covariance across observable indicators. This article introduces a novel approach to scale construction that builds on the properties of concepts to solve the aggregation problem. This is accomplished by regarding conceptual attributes as necessary-and-sufficient conditions arrayed in an ordinal scale. While different sorts of scales are useful for different purposes, we argue that “lexical” scales are in many cases superior for research questions where it is relevant to combine the differentiation of an ordinal scale with the distinct, meaningful categories of a typology.
Article
O desafio de encontrar as ferramentas adequadas para validar o processo de mensurar, tem sido objeto de preocupação permanente, na Ciência Política. Este artigo procura analisar quatro diferentes modelos de validação, e trás exemplos da pesquisa transnacional sobre democracia: a abordagem os níveis-de-medição, o modelo de equação estrutural com variáveis ocultas (latentes), a tradição pragmática e o método baseado em estudo de casos. Os metodologistas tem disputado de forma bastante contundente os méritos de modelos alternativos. Nós incentivamos os pesquisadores- e também os analistas da democracia- a prestar mais atenção nessas disputas e com isso reavaliar os pontos fortes e fracos das ferramentas de validação que eles tem usado. Um apêndice online trás um resumo da avaliação de seis grupos de dados sobre democracia sob a perspectiva de abordagens alternativas de validação. O objetivo geral é abrir um novo espaço de discussão sobre estratégias alternativas de validação.
Article
What drives politics in dictatorships? Milan W. Svolik argues that all authoritarian regimes must resolve two fundamental conflicts. First, dictators face threats from the masses over which they rule – this is the problem of authoritarian control. A second, separate challenge arises from the elites with whom dictators rule – this is the problem of authoritarian power-sharing. Crucially, whether and how dictators resolve these two problems is shaped by the dismal environment in which authoritarian politics takes place: in a dictatorship, no independent authority has the power to enforce agreements among key actors and violence is the ultimate arbiter of conflict. Using the tools of game theory, Svolik explains why some dictators, such as Saddam Hussein, establish personal autocracy and stay in power for decades; why leadership changes elsewhere are regular and institutionalized, as in contemporary China; why some dictatorships are ruled by soldiers, as Uganda was under Idi Amin; why many authoritarian regimes, such as PRI-era Mexico, maintain regime-sanctioned political parties; and why a country's authoritarian past casts a long shadow over its prospects for democracy, as the unfolding events of the Arab Spring reveal. When assessing his arguments, Svolik complements these and other historical case studies with the statistical analysis of comprehensive, original data on institutions, leaders, and ruling coalitions across all dictatorships from 1946 to 2008.
Article
Although democracy is a widely held value, concrete measurement of it is elusive. Gerardo L. Munck's constructive assessment of the methods used to measure democracies promises to bring order to the debate in academia and in practice. Drawing on his years of academic research on democracy and measurement and his practical experience evaluating democratic practices for the United Nations and the Organization of American States, Munck's discussion bridges the theories of academia with practical applications. In proposing a more open and collaborative relationship between theory and action, he makes the case for reassessing how democracy is measured and encourages fundamental changes in methodology. Munck's field-tested framework for quantifying and qualifying democracy is built around two instruments he developed: the UN Development Programme's Electoral Democracy Index and a case-by-case election monitoring tool used by the OAS. Measuring Democracy offers specific, real-world lessons that scholars and practitioners can use to improve the quality and utility of data about democracy.
Article
According to the classical perspective, polity size and democracy are inversely related. In this article, we argue that there is an important exception that manifests itself at the district level in settings where multiparty competition is allowed. Specifically, we find that larger districts encourage greater contestation. This results from a little-noticed mechanical effect as well as from several features of constituencies that are affected by size and have direct repercussions for contestation. To demonstrate this thesis we assembled a unique dataset, the Multi-level Election Archive (MLEA), which unites electoral contests across a variety of districts (national, regional, and local) and elective offices from the eighteenth century to the present, including a total of 88 countries, 2,344 elections, 79,658 districts, and more than 400,000 contests. With this evidence we were able to conduct a broad array of statistical tests, some global and others focused on particular countries or election types, all of which support our general argument.