Thomas J. Leeper’s research while affiliated with European School Of Economics and other places

Scholars have made considerable strides in evaluating and improving the external validity of experimental research. However, little attention has been paid to a crucial aspect of external validity – the topic of study. Researchers frequently develop a general theory and hypotheses (e.g., about policy attitudes), then conduct a study on a specific topic (e.g., environmental attitudes). Yet, the results may vary depending on the topic chosen. In this paper, we develop the idea of topic sampling – rather than studying a single topic, we randomly sample many topics from a defined population. As an application, we combine topic sampling with a classic survey experiment design on partisan cues. Using a hierarchical model, we efficiently estimate the effect of partisan cues for each policy, showing that the size of the effect varies considerably, and predictably, across policies. We conclude with advice on implementing our approach and using it to improve theory testing.

The emergent literature on citizen forecasting suggests that the public, in the aggregate, can often accurately predict the outcomes of elections. However, it is not clear how citizens form judgments about election results or what factors influence individual predictions. Drawing on an original survey experiment conducted during the campaign for the United Kingdom’s Brexit referendum, we provide novel evidence of what influences citizen forecasts in a so-far unexplored context of direct democracy. Specifically, we investigate the effect of voting preferences and political sophistication, in addition to three “exogenous factors” that we manipulate experimentally—i.e., social cues, elite cues and campaign arguments. Our findings indicate that citizens are reasonably accurate in their predictions, with the average forecast being close to the actual result of the referendum. However, important individual heterogeneity exists, with politically sophisticated voters being more accurate in their predictions and less prone to wishful thinking than non-sophisticated voters. Experimental findings show that partisan voters adjust their predictions in response to cues provided by their favorite party’s elites and partly in response to campaign arguments, and the effects are larger for low-sophisticated voters. We discuss the mechanisms accounting for the experimental effects, in addition to the implications of our findings for public opinion research and the literature on citizen forecasting.

How do votes in direct democratic ballots translate into policy preferences about future outcomes and affect the perceived legitimacy of those outcomes? This article examines these questions in the context of sovereignty referendums: specifically, the 2016 referendum on British membership of the European Union (EU). While the referendum result gave the British government a mandate for Britain leaving the EU, it did not provide any firm guidance as to the kind of Brexit that voters would prefer and consider legitimate. To examine the perceived desirability and legitimacy of different Brexit outcomes, we conducted a nationally representative conjoint experiment measuring attitudes towards different possible negotiation outcomes. Our findings show that ‘Leave’ and ‘Remain’ voters were highly divided over what they wanted from Brexit on salient negotiation issues, but also that most voters did not regard any possible outcome as legitimate.

Mass media are frequently cited as having the potential to inform the public, raising knowledge levels and reducing political knowledge gaps between citizens. But media are also seen as a force for segmentation, disengagement, and widening differences between citizens. If media have no effect on political knowledge, gaps between the engaged and disengaged persist regardless of who is exposed to news because no one learns. But gaps can also persist even if everyone learns from the news, particularly if learning effects are heterogeneous across those inclined and disinclined to seek out news and/or across environments that consist of different media alternatives. Yet past research on political communication has not sufficiently linked media choice to debates about possibly heterogeneous effects of media exposure on political knowledge levels. The present study contributes a novel and large-scale choice-based experiment on knowledge of the ongoing crisis in Syria that finds media effects are relatively homogeneous across those with different media preferences and across different media environments. This suggests that under most conditions – even when everyone learns from the news – knowledge gaps between the politically engaged and disengaged are widened or at least sustained after incidental exposure to politics. While closing such gaps may be impossible, the results have important implications for understanding how citizens learn about politics and how to study learning from self-selected media experiences.

The nature of democratic governance is intimately connected with how citizens respond to candidate position taking. But when will a generally uninformed public base its vote choices on candidate positions? Since Converse scholars have argued that citizens should place greater weight on candidate positions on issues they consider personally important. However, this claim has received mixed empirical support. We revisit this question with compelling new evidence. First, we expand the limited temporal focus of existing work in our first study where we analyze all available ANES data on importance and issue voting between 1980 and 2008. We then overcome endogeneity concerns through a nationally representative conjoint experiment in which we randomize two candidate’s positions on five issues. Results from both studies demonstrate that there is scant evidence that subjective issue importance consistently moderates the relationship between candidate positions and vote choices. We discuss the implications of these results for “issue public” theories of political engagement, for research on voting behavior, and for political representation.

Studying Identities with Experiments: Weighing the Risk of Posttreatment Bias Against Priming Effects – Corrigendum - Samara Klar, Thomas Leeper, Joshua Robison

A well-functioning democracy requires a degree of mutual respect and a willingness to talk across political divides. Yet numerous studies have shown that many electorates are polarized along partisan lines, with animosity towards the partisan out-group. In this article, we further develop the idea of affective polarization, not by partisanship, but instead by identification with opinion-based groups. Examining social identities formed during Britain’s 2016 referendum on European Union membership, we use surveys and experiments to measure the intensity of partisan and Brexit-related affective polarization. The results show that Brexit identities are prevalent, felt to be personally important, and cut across traditional party lines. These identities generate affective polarization as intense as that of partisanship in terms of stereotyping, prejudice, and various evaluative biases, convincingly demonstrating that affective polarization can emerge from identities beyond partisanship.

This chapter begins by reviewing the strengths and weaknesses of various approaches for obtaining samples of intersectional identity groups, and then provides some empirical evidence for the viability of general population samples for providing large numbers of respondents from intersectional identity groups. Purposive sampling can be thought of as a subset of convenience sampling, in that respondents are chosen subjectively. While convenience sampling necessarily relies upon untestable assumptions rather than probability‐based sampling methods, the chapter argues that such methods may be more appropriate for survey‐experimental research than for observational research. For the purposes of experimental research on small intersectional identity groups, many purposive samples may be fit for use because they trade off design‐based representativeness against obtaining a sample size sufficiently large to powerfully estimate an experimental effect size.

Political scientists rely heavily on survey research to gain insights into public attitudes and behaviors. Over the past decade, survey data collection has moved away from personal face-to-face and telephone interviewing towards a model of computer-assisted self-interviewing. A hallmark of many online surveys is the prominent display of the survey’s sponsor, most often an academic institution, in the initial consent form and/or on the survey website itself. It is an open question whether these displays of academic survey sponsorship could increase total survey error. We measure the extent to which sponsorship (by a university or marketing firm) affects data quality, including satisficing behavior, demand characteristics, and socially desirable responding. In addition, we examine whether sponsor effects vary depending on the participant’s experience with online surveys. Overall, we find no evidence that response quality is affected by survey sponsor or by past survey experience.

Scholars from across the social sciences argue that identities – such as race, ethnicity, and gender – are highly influential over individuals’ attitudes, actions, and evaluations. Experiments are becoming particularly integral for allowing identity scholars to explain how these social attachments shape our political behavior. In this letter, we draw attention to how identity scholars should approach the common practice of assessing moderators, measuring control variables, and detecting effect heterogeneity using covariates. Special care must be taken when deciding where to place measures of demographic covariates in identity-related experiments, as these cases pose unique challenges from how scholars traditionally approach experimental design. We argue in this letter that identity scholars, particularly those whose subjects identify as women or minorities, are often right to measure covariates of interest posttreatment.