No full-text available
To read the full-text of this research,
you can request a copy directly from the authors.
Lost on the Way: Nonresponse and its Influence on Turnout Bias in Postelection Surveys
Abstract
Earlier studies on turnout bias in postelection surveys have focused on vote overreporting (measurement bias) and have disregarded
nonresponse bias associated with voter overrepresentation. Based on a unique data set of validated votes collected for two
different ballots, we offer an in-depth analysis of the size and sources of nonresponse bias. On the aggregate level, we find
that voter overrepresentation accounts for a larger share of total turnout bias than misreporting. On the individual level,
past voting record, ballot vote, and age appear as strong predictors of survey participation. Breaking down further the analysis,
we show that these factors contribute to nonresponse bias through both contact and cooperation. By contrast, the effects of
some other factors work at cross-purpose in the contact and cooperation stages.
... Since we are lacking variables from electoral registers to validate reported participation or votes, it is impossible to disentangle these two effects. To be able to do so would require surveys with validated votes (e.g., Sciarini and Goldberg 2015). Election surveys are unique in this respect because in no other domain are substantive (dependent) variables in principle available. ...
... and turnout(Sciarini and Goldberg 2015, Selb andMunzert 2013), and underestimate right-wing populist party voting(Hooghe and Reeskens 2007, Lubbers et al. 2002). In Switzerland, the Swiss People's Party has been underrepresented since the start of the Selects surveys in 1995. ...
In this paper, we analyze to what extent a sequential mixed mode survey, consisting of push-to-web, telephone and mail-paper modes, is able to improve representativeness in terms of socio-demographic variables and reduce bias in terms of voting behavior compared to a single mode survey. In addition, we study whether changes in mode lead to measurement error issues by focusing on income. We find that adding the telephone mode improves sample representation in terms of socio-demographic variables. Meanwhile, adding the paper mode does not show further improvements in this respect. However, adding the telephone, and in particular the paper mode, turn out to reduce bias in voting behavior when compared to official figures. As for measuring income, the web and the telephone mode perform similarly well. Finally, we find little evidence of measurement differences of income when the same respondents are interviewed first by web or telephone and subsequently by paper questionnaire.
... This has led Belli, Traugott and Beckmann (2001, p. 496) to comment on their own research that 'the most serious limitation of this study is that it is based only upon analysis of data from the United States'. Studies have examined other countries, notably Denmark (Dahlgaard et al. 2019), Israel (Waismel-Manor and Sarid 2011), Norway (Waldahl and Aardal 2000), Sweden (Granberg and Holmberg 1991;Gorecki 2011) and Switzerland (Sciarini and Goldberg 2017). While many of the recent single country studies have advanced our knowledge of overreporting by matching survey estimates with administrative data (see, for example, Dahlgaard et al. 2019), the findings cannot be extrapolated to other countries. ...
Election turnout is central to the health of a democracy, yet measuring it accurately in national election surveys presents major problems. Surveys show that respondents consistently overreport their turnout, usually by between 10 and 20 percentage points. To date, most research has been conducted on the United States, and what little comparative research exists has typically covered a small number of countries. In this paper, we present the largest comparative study yet undertaken on turnout overreporting. By utilizing 20 years of data from the Comparative Study of Electoral Systems (CSES), we analyse 184 elections across 55 different states. We examine the importance of the survey method and the election context as explanations for overreporting. The results suggest that aspects of both may shape the degree of error. These exploratory results provide a corrective to current studies by showing that institutional and contextual factors may be at least as important as survey method in shaping overreporting.
... However, the overestimation may be explained by the low turnout context of EP elections, which favours turnout overestimation particularly due to an overrepresentation of voters in the survey (e.g. Sciarini and Goldberg, 2017;Selb and Munzert, 2013). Figure 1 shows considerable differences across countries, with citizens in Denmark being twice as likely to participate in the EP elections compared to Czechs. ...
Research has investigated numerous factors influencing turnout for European Parliament elections but paid insufficient attention to the role of campaign influences. Using survey data collected in the context of the 2019 European Parliament elections, we assess citizens’ passive exposure to media coverage and political advertisements, active forms of engagement such as visiting a party’s website and interpersonal communication on- and offline. We test to which extent these activities contribute to the likelihood that citizens vote. Our study highlights the importance of information factors beyond well-established turnout determinants. The results confirm the mobilizing influence of a number of variables, but we also find consistent negative effects of online forms of communication and engagement. We discuss these findings with regard to a potentially ‘toxic’ online information environment.
... 10. Theoretically, turnout bias consists of over-representation of voters (selection bias) and over-reporting by respondents (misreporting bias) (e.g., Sciarini & Goldberg 2017;Goldberg & Sciarini 2019). Whereas misreporting should be a rather stable phenomenon, the selection bias is more important for a longitudinal analysis. ...
Since the heyday of cleavage voting in the 1960s and 1970s, the majority of studies presents evidence of a decline in cleavage voting – caused by either structural or behavioural dealignment. Structural dealignment denotes changes in group size responsible for a decrease in cleavage voting, whereas behavioural dealignment concerns weakening party–voter links over time. A third phenomenon posited in this article is the collective voting abstention of certain (social) groups, here referred to as ‘political dealignment’, which results in a new type of division of voting versus abstention. The purpose of this article is to examine the three underlying mechanisms for the decline in social class and religious cleavage voting across four Western countries (Great Britain, the Netherlands, Switzerland and the United States) over the last 40–60 years using longitudinal post‐election data. The results prove a strong presence of political dealignment and increasing turnout gaps regarding both the class and religious cleavage. Furthermore, whenever a decline in cleavage voting is present, it is mainly caused by changes in the social groups’ behaviour and less by changing social structures in a country.
... In contrast to legislative studies, the validation of survey data is well established in other research areas, e.g. postelection studies (Comşa and Postelnicu, 2013;Selb and Munzert, 2013;Sciarini and Goldberg, 2015;Sciarini and Goldberg, 2016). These studies show that survey data tends to be biased. ...
This article discusses the methodological challenges of legislative surveys. Following an overview of different types of survey biases, the article argues that self-selection and misreporting are the most critical problems for legislative surveys. In order to identify the self-selection and misreporting biases, we compare the answers with a survey from Swiss members of parliament with their observed behavior in the parliament. The empirical analysis shows that the survey sample has a substantial misreporting bias. We conclude that the parliamentarian’s attitude toward the survey object is strongly linked to their response. These issues should be addressed to improve the design and quality of legislative surveys.
... In contrast to legislative studies, the validation of survey data is well established in other research areas, e.g. postelection studies (Comşa and Postelnicu, 2013;Selb and Munzert, 2013;Sciarini and Goldberg, 2015;Sciarini and Goldberg, 2016). These studies show that survey data tends to be biased. ...
This article discusses the methodological challenges of legislative surveys. Following an overview of different types of survey biases, the article argues that self-selection and misreporting are the most critical problems for legislative surveys. In order to identify the self-selection and misreporting biases, we compare the answers to a survey from Swiss members of parliament with their observed behavior in the parliament. The empirical analysis shows that the survey sample has a substantial misreporting bias. We conclude that the parliamentarian’s attitude towards the survey object is strongly linked to their response. These issues should be addressed to improve the design and quality of legislative surveys.
Autrefois réputé pour sa grande stabilité, le système politique suisse a connu de profondes transformations au cours des trois dernières décennies. Ce livre se fait l’écho de ces changements, au travers d’un état des lieux complet et transdisciplinaire des connaissances sur la politique suisse en ce début de 21e siècle. Il traite tour à tour des institutions fondamentales (fédéralisme, neutralité, démocratie directe), des acteurs politiques étatiques (gouvernement, Parlement) et non étatiques (partis politiques, groupes d’intérêt), des processus de prise de décision, ainsi que des arcanes et des détenteurs du pouvoir. L’auteur adopte une approche résolument comparative et situe la politique suisse dans son environnement international et européen. D’une grande clarté, cet ouvrage vient pallier le manque d’un ouvrage de synthèse sur la politique suisse en langue française. Il s’adresse aux chercheuses et chercheurs et aux étudiantes et étudiants en sciences sociales et politiques, et plus généralement au public averti intéressé par la politique suisse.
Electoral participation research points to political interest as a strong and consistent predictor of voting, but overlooks the propensity of interested people to overreport in surveys. Vote validation studies demonstrate that vote overreporting exaggerates the association between political interest and turnout, but fails to highlight the pathways through which this bias occurs. To fill this gap and link these two strands of literature, we take a closer look at the relationships between political interest, likelihood of voting (as defined by one's actual voting record), and participation in given vote, while taking care of overreporting. Applying causal mediation models to a unique set of validated vote data, we show that the modus operandi of overreporting is both direct and indirect: A portion of politically interested people overreport their vote regardless of whether they usually vote or not. Yet, the indirect path also matters: Another portion of interested people do vote frequently, which prompts them to overreport if they for once abstain. This is consequential for the commonly found association between political interest and turnout.
In the context of an ever growing importance and usage of referendums around the globe, this article provides a comprehensive approach to analyse the determinants of participation in direct democratic votes. In the absence of conclusive empirical evidence about which factors drive direct democratic participation, studies tend to adopt election‐specific findings and assume the determinants of electoral turnout to equally apply for referendums. Yet, a strict empirical test of these numerous determinants in a referendum context is still missing. By examining aspects stemming from both election‐specific and referendum‐specific contexts, this article aims to first test the applicability of common electoral theories of turnout for direct democratic participation, and second to analyse the relevance of each factor when simultaneously examined with other contextual and individual factors. This holistic approach represents reality as adequate as possible, that is, to consider various factors that may simultaneously influence the individual decision to vote. Next to individual variables, the analysis particularly focuses on two contextual levels, the community a person lives in and factors linked to a given referendum. The discussion and joint analysis of competing factors addresses the problem of underspecified turnout models, which commonly prevents a detailed assessment of the relative importance of the determinants of turnout. The study uses registered data from the canton of Geneva, Switzerland, which provides official information about individual participation across 43 referendums in 45 communities. We match this individual data with referendum‐related factors, such as campaign intensity and importance of the issues at stake, and community‐level variables, such as wealth and urbanisation. The results of our multilevel, cross‐classified models show significant context‐related effects, stemming mainly from the referendum and less from the community level. Still, the main driver of direct democratic participation is individual determinants, in particular citizens' past participation record. This article is protected by copyright. All rights reserved
Empirical evidence suggests that e-voting has no measurable effects on turnout. However, existing studies did (or could) not look at e-voting effects on the individual level. We innovate by analyzing whether and to what extent the availability of e-voting fosters turnout among specific groups of citizens, and how this influences the equality of participation. To that end, we estimate Bayesian multi-level models on a unique set of official data on citizens’ participation covering 30 ballots between 2008 and 2016 in Geneva, Switzerland, which has the most far-reaching experience with e-voting worldwide. Despite the fact that e-voting was added to an easy-to-use form of postal voting, we find that offering e-voting has increased turnout among abstainers and occasional voters. By contrast, the effects of e-voting availability on the equality of participation are mixed with respect to the age cohorts and gender.
This article assesses whether—and to what extent—turnout bias in postelection surveys is reduced by adding a short nonresponse follow-up (NRFU) survey to a mixed-mode survey. Specifically, we examine how the NRFU survey influences response propensities across demographic groups and political factors and whether this affects data quality. We use a rich dataset on validated voter turnout data, collected across two different ballots. In addition to the main survey that comprises computer-assisted telephone interviews (CATI) and web respondents, both studies include a short follow-up mail survey for nonrespondents. The results demonstrate that collecting extra information from additional respondents on so-called “central” questions is worth the effort. In both studies, the NRFU survey substantially increases representativeness with respect to sociodemographic and participation variables. In particular, voters and politically active citizens are more accurately represented in the NRFU survey. This tends to result in better estimates of turnout determinants in the final (combined) sample than is seen from CATI/web respondents only. Moreover, the increase in response rate and the decrease in nonresponse bias comes at almost no price in terms of measurement errors. Vote overreporting is only slightly higher in the mail follow-up survey than in the main CATI/web survey.
The share of voters participating in national elections from abroad is evergrowing. Despite this, expatriates constitute one of the most understudied groups of electors. Some existing analyses have shown that expatriates support different parties than voters residing at home. However, the reasons for this effect remain in the dark. In this article, we test common electoral theories—socio-structural, socio-psychological, and issue voting—and their relevance for voters at home and abroad. Additionally, we test if differences in voting behaviour are due to compositional or behavioural reasons. In line with previous studies, we show that expatriates support other parties, in the Swiss case left parties, than voters at home. We further show that this gap cannot be explained by the different composition of the expatriate community but rather by their different behavioural motivations. Expatriates more often base their vote choice on their social class and religious beliefs. Partisanship voting and, to some extent, issue voting are less important in the expatriate community. The findings are based on the Swiss National Election Study 2011 and additional interviews conducted among Swiss residing abroad.
Introduction
Postelection surveys consistently overestimate turnout because of the overrepresentation of voters and the overreporting of nonvoters. The problem of overrepresentation is primarily a consequence of nonresponse, which results from the higher propensity of voters to participate in surveys (Sciarini & Goldberg, 2017; Voogt & Saris, 2003). Overreporting is a measurement bias caused by the tendency of nonvoters to claim that they have voted. This research attempts to resolve a long-standing contradiction in the literature on turnout overreporters.
Survey respondents who voted at least intermittently if not habitually have been considered prime suspects for overreporting (Belli, Traugott, & Beckmann, 2001; Tittle & Hill, 1967). These respondents are under suspicion because their voting experiences can be misleading when they are recalling an election from which they happened to abstain (Abelson, Loftus, & Greenwald, 1992), and their voting motivations (e.g., to conform to the norms of voting, or to avoid guilt feelings about abstention) can lead them to overreport (Bernstein, Chadha, & Montjoy, 2001; Brenner, 2012). This argument has been supported by numerous studies, which reported that overreporters more closely resemble voters than nonvoters on various characteristics (Ansolabehere & Hersh, 2011, 2012; Belli et al., 2001; Hill & Hurley, 1984; Kitt & Gleicher, 1950; Sigelman, 1982; Silver, Anderson, & Abramson, 1986; Tittle & Hill, 1967; Tourangeau, Groves, & Redline, 2010; Traugott & Presser, 1992; Weiss, 1968). However, evidence has not been conclusive because of lack of direct observations on the turnout histories of overreporters.
This book analyzes the link between economic and political inequalities and investigates the mechanisms that lead to economically rooted inequalities in the political representation of citizens’ policy preferences. Focusing on the case of Switzerland and evaluating data from the post-electoral survey, Selects 2007, the author demonstrates that the policy preferences of members of the Federal Assembly best reflect those of rich citizens. This pattern is explained by differential levels of political participation and knowledge across income groups, party finance, the fact that representatives tend to come from higher economic strata, and the failure of the party-system structure to reflect the complexity of policy preferences among citizens.
Economically rooted inequalities in representation in Switzerland are restricted to redistributive and welfare states issues. In these domains, however, the preferences of higher income strata appear to be much better reflected in parliament than the preferences of citizens with lower levels of resources. This phenomenon can be to some extent explained by differences in political participation and information across income groups; by the fact that representatives tend to belong to higher income groups themselves; by the absence of party finance regulations; and by the structure of the party system which does not allow many voters to translate their complex preferences into a single vote choice. Future research should pay a closer attention to the role of interest groups in the decision making process and monitor how responsive political institutions are to the demands of (different groups of) the citizenry.
Different methods have been developed to study substantive representation. Research has notably focused on the correlation or on the proximity between preferences of citizens’ and political actors’ positions or public policy. The choice of focusing on one or the other method, as well as the definition of political actors to be taken into account, depends on the researchers’ emphasis on different aspects of representation. It is argued that in the case of Switzerland, where representatives also play a role in shaping citizens preferences in direct democratic votes, an analysis of the congruence in terms of broad policy orientations between citizens and representatives is appropriate to study inequalities in representation. The chapter further elaborates on the definition of groups to be studied and presents the survey data from the Swiss electoral study (Selects) used in the empirical analyses.
Voters hold complex policy preferences whose configuration is not necessarily reflected in the party system. As a result, they need to make an electoral choice according to some issues and not others. This multidimensionality of electoral competition can result in a substantial gap between the preferences of voters and those of elected officials. Based on survey data on the positioning of citizens and parties, the analysis shows that a very large proportion of voters hold preferences that are closer to parties from the left on economic issues but are better represented by conservative parties on cultural issues. Furthermore, the analysis shows that in this particular case, the mismatch between the structure of preferences among citizens and the party system, advantages the election of parties that defend economically liberal policies as compared to the share of the population endorsing such policies.
Survey researchers have long speculated that there may be a link between nonresponse and measurement error—that is, people likely to become nonrespondents to a survey are also likely to make poor reporters if they do take part. Still, there is surprisingly little evidence of such a link. It could be that nonresponse is generally the product of one set of factors and reporting errors, the product of an unrelated set, or both nonresponse and reporting errors may be itemspecific so that no general relationship between the two is likely to emerge. Our study examined a situation in which we thought there would be a link between response propensities and the propensity to give inaccurate answers. We asked samples of voters and nonvoters to take part in a survey that included items about voting. Past research shows that nonvoters misreport that fact and that they are less likely than voters in general to take part in surveys. We thought we could heighten the differences between voters and nonvoters in both response rates and levels of misreporting if we characterized the survey as being about politics. However, only nonresponse biases were larger when the topic of the survey was described as political, and this difference was only marginally significant. These two ways of framing the study had even smaller effects on estimates derived from other items in the questionnaire.The overall biases in estimates derived from the voting items are very substantial, and both nonresponse and measurement error contribute to them.
A common hypothesis about practices to reduce survey nonresponse is that those persons brought into the respondent pool through persuasive efforts may provide data filled with measurement error. Two questions flow from this hypothesis. First, does the mean square error of a statistic increase when sample persons who are less likely to be contacted or cooperate are incorporated into the respondent pool? Second, do nonresponse bias estimates made on the respondents, using survey reports instead of records, provide accurate information about nonresponse bias? Using a unique data set, the Wisconsin Divorce Study, with divorce records as the frame and questions about the frame information included in the questionnaire, this article takes a first look into these two issues. We find that the relationship between nonresponse bias, measurement error bias, and response propensity is statistic- specific and specific to the type of nonresponse. Total bias tends to be lower on estimates calculated using all respondents, compared with those with only the highest contact and cooperation propensities, and nonresponse bias analyses based on respondents yield conclusions simi- lar to those based on records. Finally, we find that error properties of statistics may differ from error properties of the individual variables used to calculate the statistics.
Examines a fundamental problem for opinion polls and those who use them.
Misreporting is a problem that plagues researchers who use survey data. In this article, we develop a parametric model that
corrects for misclassified binary responses using information on the misreporting patterns obtained from auxiliary data
sources. The model is implemented within the Bayesian framework via Markov Chain Monte Carlo (MCMC) methods
and can be easily extended to address other problems exhibited by survey data, such as missing response and/or covariate
values. While the model is fully general, we illustrate its application in the context of estimating models of turnout using
data from the American National Elections Studies.
Methods for estimating nonresponse bias are reviewed and several methods are tried on the 1980 GSS. The results indicate that
various estimating procedures are inappropriate and that even the more promising techniques can provide faulty estimates of
nonresponse bias. By its nature, nonresponse bias is very difficult to assess accurately and no simple, certain method exists.
One interpretation for the common survey finding that the background characteristics of vote overreporters resemble those
of actual voters is that misreporters usually vote. This hypothesis—that misreporters regularly voted in earlier elections—is
tested with data from the 1972–74–76 Michigan Election Panel. It receives no support: the 1972 and 1974 validated turnout
of the 1976 misreporters was very low. Moreover, misreporting was a fairly stable respondent characteristic: misreporting
about an election in one interview was correlated with misreporting about the remaining elections in each of the other two
interviews. A comparison of regressions predicting turnout using the validated reports versus the self-reports shows that
the respondent errors can distort conclusions about the correlates of voting. For example, controlling for three other variables,
education was related to self-reported voting but not to validated voting. Here, as well as in surveys of other socially desirable
or undesirable issues, respondent self-reports may bias survey data in favor of commonsense models of the world.