No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Reducing Political Bias in Political Science Estimates
Abstract
Political science researchers have flexibility in how to analyze data, how to report data, and whether to report on data. A review of examples of reporting flexibility from the race and sex discrimination literature illustrates how research design choices can influence estimates and inferences. This reporting flexibility—coupled with the political imbalance among political scientists—creates the potential for political bias in reported political science estimates. These biases can be reduced or eliminated through preregistration and preacceptance, with researchers committing to a research design before completing data collection. Removing the potential for reporting flexibility can raise the credibility of political science research.
... The focus of this article has been on retractions and related editorial decisions to intervene in published work. It goes without saying that the problem of ideological bias is not limited to the treatment of published articles but affects every aspect of the field, including: which topics scholars are willing to study (Duarte et al., 2015), how/whether "controversial" findings are reported (Zigerell, 2017), and who gets selected as journal editors and in other leadership positions in professional associations (Forgas, 2022). The prevailing atmosphere threatens the credibility of a truly scientific study of social facts. ...
Disciplines in social and behavioral science have become increasingly committed to promoting social justice activism at the expense of viewpoint diversity. The present study documents the impact of this ideological project on the literature on racial disparities in criminal justice outcomes. Evidence from a recent high-profile retraction suggests that elite gatekeepers are willing to disregard ethical guidelines to censor an article embraced by conservatives. Evidence from additional case studies illustrates selective application of methodological standards based on the political implications of the findings. Serious errors are less likely to be treated as fatal if the contribution supports the activist agenda. By contrast, methodologically sound contributions are labeled as flawed if they challenge the dominant narrative. Although this manner of conduct is consistent with the evolved morality of human tribes, it violates the established norms of science. Observations from recent literature suggest that the integrity of social science research is compromised by our collective failure to value and protect contrarian perspectives. To remedy the situation, I propose that the open science movement – which has been successful at reducing questionable research practices – expand its scope from the craft of doing science to the realm of ideas, assumptions, and viewpoints. In short, I advocate for an “open minds” initiative for the improvement of social science research.
... Whereas journalists write news articles about current events, academics write reports about current research. Over the last few years, there has been a surge of interest in the political leanings of academics [24,[35][36][37][38] . The results from this literature mirror those seen for journalists, but generally reveal even larger skews towards left-wing parties and political attitudes. ...
We gathered survey data on journalists' political views in 17 Western countries. We then matched these data to outcomes from national elections, and constructed metrics of journalists' relative preference for different political parties. Compared to the general population of voters, journalists prefer parties that have more left-wing positions overall (r's-.47 to-.53, depending on the metric used), and that are associated with certain ideologies, namely environmentalism, feminism, social liberalism, socialism, and support for the European Union. We used Bayesian model averaging to assess the validity of the predictors in multivariate models. We found that, of the ideology tags in our dataset, 'conservative' (negative), 'nationalist' (negative) and 'green' (positive) were the most consistent predictors with nontrivial effect sizes. We also computed estimates of the skew of journalists' political views in different countries. Overall, our results indicate that the Western media has a left-liberal skew.
... Ironically, by moving closer to an experimental ideal, statistical flukes-that is, variance falsepositives-become a greater concern (Young 2019). Moreover, whereas taste for novelty is arguably less of an issue than in psychology, political desirability can have similar influence (Zigerell 2017). Furthermore, certain problems that have been identified in psychology also have been pointed out in political science, including low computational reproducibility (Stockemer, Koehler, and Lentz 2018;cf. ...
... They find that academic 8 We are estimating the effect of perceived author gender in the minds of other authors making citation decisions. 9 The finding is critiqued in Zigerell (2017) and defended in Maliniak, Powers, and Walter (2017). 10 Scholarship in international relations is sometimes organized into "paradigms," or schools of thought about which factors are most crucial for explaining international relations. ...
We identify situations in which conditioning on text can address confounding in observational studies. We argue that a matching approach is particularly well‐suited to this task, but existing matching methods are ill‐equipped to handle high‐dimensional text data. Our proposed solution is to estimate a low‐dimensional summary of the text and condition on this summary via matching. We propose a method of text matching, topical inverse regression matching, that allows the analyst to match both on the topical content of confounding documents and the probability that each of these documents is treated. We validate our approach and illustrate the importance of conditioning on text to address confounding with two applications: the effect of perceptions of author gender on citation counts in the international relations literature and the effects of censorship on Chinese social media users.
... A Mitchell and Martin (2018) addressing these comments might be less likely to be accepted as evidence that "student course evaluations...have been shown to be biased against female instructors" (Sen 2018, 336). In addition to opening peer review to volunteers, journal editors can reduce the potential for political imbalance in research reporting by adopting policies to further reduce reporting flexibility, given that reported estimates and inferences can depend on how data are analyzed and on whether data are omitted from reported analyses (Simmons, Nelson, and Simonsohn 2011;Zigerell 2017). Journal policies to further reduce reporting flexibility can, for instance, require that articles reporting on nonpublic surveys include full questionnaires in the article or appendices, with reasonable exceptions. ...
Left Unchecked: Political Hegemony in Political Science and the Flaws It Can Cause - L.J Zigerell
... but only a small proportion also use results-blind reviews. There are also some examples of journals using the results-blind review process permanently or on a trial basis in disciplines such as medicine (Button et al., 2016;Sridharan & Greenland, 2009), neuroscience (Chambers, Feredoes, Muthukumaraswamy, & Etchells, 2014), and political science (Findley et al., 2016;Nyhan, 2015;Zigerell, 2017), among others (Armstrong, 1997). However, the vast majority of journals use neither registered reports nor results-blind reviews. ...
The organizational sciences are entering a confidence crisis where a growing number of scholars recognize our science is imperfect and our knowledge base is potentially flawed. One contributing factor that scholars have identified is the peer-review process; issues such as QRPs, positive outcome biases and publication bias are, in part, a product of the traditional peer-review process and contribute to the questionable credibility of our science. The current paper focuses on the results-blind review as a solution to improve the existing system because it has received little attention yet fits well with the nature of our discipline (e.g., not entirely grant-funded, field-research oriented). Using a mixed-methods approach, we surveyed 203 editorial board members to understand the scientific community’s reactions to the results-blind review initiative and readiness to implement change. Our results suggest that there are noted advantages of the results-blind review process and the scientific community is open to the initiative in our field. However, our data also suggest that there may be some hesitations about the initiative, particularly with implementing a new review process. Based on the results, we offer actionable recommendations for authors, reviewers, and editors regarding the results-blind review.
... but only a small proportion also use results-blind reviews. There are also some examples of journals using the results-blind review process permanently or on a trial basis in disciplines such as medicine (Button et al., 2016;Sridharan & Greenland, 2009), neuroscience (Chambers, Feredoes, Muthukumaraswamy, & Etchells, 2014), and political science (Findley et al., 2016;Nyhan, 2015;Zigerell, 2017), among others (Armstrong, 1997). However, the vast majority of journals use neither registered reports nor results-blind reviews. ...
... This problem is further aggravated by the highly contentious political nature of the relationship between ethnic diversity and trust. As pinpointed by Zigerell (2017), the choice of research design may be based on the results they produce through political biases of the authors. This makes it all the more important to provide a strong rationale for including a given set of control variables and presenting different models. ...
Due to its wide-ranging implications for social cohesion in diversifying Western countries, the question of the potential negative consequences of ethnic diversity for social trust is arguably the most contentious question in the literature on social trust. In this chapter we critically review the empirical evidence for a negative relationship between contextual ethnic diversity (measured locally within countries) and social trust. We cautiously conclude that there are indications of a negative relationship, although with important variations across study characteristics including national setting, context unit analyzed, and conditioning on moderating influences. Building on the review, we highlight a number of paths for theoretical and methodological advances, which we argue would push the literature on the relationship between ethnic diversity and social trust forward.
Experiments are a central methodology in the social sciences. Scholars from every discipline regularly turn to experiments. Practitioners rely on experimental evidence in evaluating social programs, policies, and institutions. This book is about how to “think” about experiments. It argues that designing a good experiment is a slow moving process (given the host of considerations) which is counter to the current fast moving temptations available in the social sciences. The book includes discussion of the place of experiments in the social science process, the assumptions underlying different types of experiments, the validity of experiments, the application of different designs, how to arrive at experimental questions, the role of replications in experimental research, and the steps involved in designing and conducting “good” experiments. The goal is to ensure social science research remains driven by important substantive questions and fully exploits the potential of experiments in a thoughtful manner.
Witnessing the ongoing "credibility revolutions" in other disciplines, also political science should engage in meta-scientific introspection. Theoretically, this commentary describes why scientists in academia's current incentive system work against their self-interest if they prioritize research credibility. Empirically, a comprehensive review of meta-scientific research with a focus on quantitative political science demonstrates that threats to the credibility of political science findings are systematic and real. Yet, the review also shows the discipline's recent progress toward more credible research. The commentary proposes specific institutional changes to better align individual researcher rationality with the collective good of verifiable, robust, and valid scientific results.
Forschung als soziales Dilemma: Eine meta-wissenschaftliche Bestandsaufnahme zur Glaubwürdigkeit politikwissenschaftlicher Befunde und ein Appell zur Veränderung akademischer Anreizstrukturen Angesichts der "Glaubwürdigkeitsrevolution" in anderen Sozialwissenschaften liegen Fragen nach der Verlässlichkeit institutioneller Wissensproduktion auch in der Politikwissenschaft nahe. Dieser Kommentar beschreibt warum Wissenschaftler entgegen ihrem Eigeninteresse handeln, wenn sie Forschungsvalidität priorisieren. Ein umfassender Überblick der meta-wissenschaftlichen Literatur mit Fokus auf die quantitative weist einerseits auf jüngst eingeleitete Reformen zur Sicherung reliabler Forschung hin. Andererseits offenbar dieser Überblicksartikel systematische Probleme in der Glaubwürdigkeit veröffentlichter Forschungsbefunde. Dieser Kommentar schlägt konkrete Maßnahmen vor individuelle Forscheranreize in Einklang zu bringen mit dem gemeinschaftlichen Ziel verlässlicher Forschung.
This study reports results from a new analysis of 17 survey experiment studies that permitted assessment of racial discrimination, drawn from the archives of the Time-sharing Experiments for the Social Sciences. For White participants (n=10 435), pooled results did not detect a net discrimination for or against White targets, but, for Black participants (n=2781), pooled results indicated the presence of a small-to-moderate net discrimination in favor of Black targets; inferences were the same for the subset of studies that had a political candidate target and the subset of studies that had a worker or job applicant target. These results have implications for understanding racial discrimination in the United States, and, given that some of the studies have never been fully reported on in a journal or academic book, the results also suggest the need for preregistration to reduce or eliminate publication bias in racial discrimination studies.
The data includes measures collected for the two experiments reported in “False-Positive Psychology” [1] where listening to a randomly assigned song made people feel younger (Study 1) or actually be younger (Study 2). These data are useful because they illustrate inflations of false positive rates due to flexibility in data collection, analysis, and reporting of results. Data are useful for educational purposes.
In a well-publicized study, Gilens and Page argue that economic elites and business interest groups exert strong influence on US government policy while average citizens have virtually no influence at all. Their conclusions are drawn from a model which is said to reveal the causal impact of each group’s preferences. It is shown here that the test on which the original study is based is prone to underestimating the impact of citizens at the 50th income percentile by a wide margin. In addition, descriptive analysis of the authors’ dataset reveals that average Americans have received their preferred policy outcome roughly as often as elites have when the two groups have disagreed with each other. Evidence that average citizens are effectively ignored by the policy process may not be as strong as is suggested by the authors.
Empirically analyzing empirical evidence
One of the central goals in any scientific endeavor is to understand causality. Experiments that seek to demonstrate a cause/effect relation most often manipulate the postulated causal factor. Aarts et al. describe the replication of 100 experiments reported in papers published in 2008 in three high-ranking psychology journals. Assessing whether the replication and the original experiment yielded the same result according to several criteria, they find that about one-third to one-half of the original findings were also observed in the replication study.
Science , this issue 10.1126/science.aac4716
Background: We explore whether the number of null results in large National Heart Lung, and Blood Institute (NHLBI) funded trials has increased over time.
Methods: We identified all large NHLBI supported RCTs between 1970 and 2012 evaluating drugs or dietary supplements for the treatment or prevention of cardiovascular disease. Trials were included if direct costs >$500,000/year, participants were adult humans, and the primary outcome was cardiovascular risk, disease or death. The 55 trials meeting these criteria were coded for whether they were published prior to or after the year 2000, whether they registered in clinicaltrials.gov prior to publication, used active or placebo comparator, and whether or not the trial had industry co-sponsorship. We tabulated whether the study reported a positive, negative, or null result on the primary outcome variable and for total mortality.
Results: 17 of 30 studies (57%) published prior to 2000 showed a significant benefit of intervention on the primary outcome in comparison to only 2 among the 25 (8%) trials published after 2000 (χ2=12.2,df= 1, p=0.0005). There has been no change in the proportion of trials that compared treatment to placebo versus active comparator. Industry co-sponsorship was unrelated to the probability of reporting a significant benefit. Pre-registration in clinical trials.gov was strongly associated with the trend toward null findings.
Conclusions: The number NHLBI trials reporting positive results declined after the year 2000. Prospective declaration of outcomes in RCTs, and the adoption of transparent reporting standards, as required by clinicaltrials.gov, may have contributed to the trend toward null findings.
This study revisits an important recent article about racial bias and finds that many of its inferences are weakened when we analyze the data more completely. DeSante in 2013 reported evidence from a survey experiment indicating that Americans reward Whites more than Blacks for hard work but penalize Blacks more than Whites for laziness. However, the present study demonstrates that these inferences were based on an unrepresentative selection of possible analyses: the original article does not include all possible equivalent or relevant analyses, and when results from these additional analyses are combined with the results reported in the original article, the strength of inferences is weakened. Moreover, newly-reported evidence reveals heterogeneity in racial bias: respondents given a direct choice between equivalent targets of different races favored the Black target over the White target. These results illustrate how the presence of researcher degrees of freedom can foster production of inferences that are not representative of all inferences that a set of data could produce. This study thus highlights the value of preregistering research design protocols and required public posting of data.
This paper reanalyses data from a recent widely-discussed study reporting that female-authored papers published in top international relations journals received fewer citations than equivalent male-authored papers. The reanalysis indicated that the gender citation gap is largely limited to elite papers, defined either as papers in the right tail of the citation distribution or as papers published in the most familiar journals. Results suggest that the original study’s recommendation to consider the gender citation gap in promotion and review requires more data and a better understanding of the factors that influence whether a paper enters the discipline’s elite.
Virtually all previous studies of domestic economic redistribution find white Americans to be less enthusiastic about welfare for black recipients than for white recipients. When it comes to foreign aid and international redistribution across racial lines, I argue that prejudice manifests not in an uncharitable, resentful way but in a paternalistic way because intergroup contact is minimal and because of how the media portray black foreigners. Using two survey experiments, I show that white Americans are more favorable toward aid when cued to think of foreign poor of African descent than when cued to think of those of East European descent. This relationship is due not to the greater perceived need of black foreigners but to an underlying racial paternalism that sees them as lacking in human agency. The findings confirm accusations of aid skeptics and hold implications for understanding the roots of paternalistic practices in the foreign aid regime.
Psychologists have demonstrated the value of diversity—particularly diversity of viewpoints—for enhancing creativity, discovery, and problem solving. But one key type of viewpoint diversity is lacking in academic psychology in general and social psychology in particular: political diversity. This article reviews the available evidence and finds support for four claims: 1) Academic psychology once had considerable political diversity, but has lost nearly all of it in the last 50 years; 2) This lack of political diversity can undermine the validity of social psychological science via mechanisms such as the embedding of liberal values into research questions and methods, steering researchers away from important but politically unpalatable research topics, and producing conclusions that mischaracterize liberals and conservatives alike; 3) Increased political diversity would improve social psychological science by reducing the impact of bias mechanisms such as confirmation bias, and by empowering dissenting minorities to improve the quality of the majority’s thinking; and 4) The underrepresentation of non-liberals in social psychology is most likely due to a combination of self-selection, hostile climate, and discrimination. We close with recommendations for increasing political diversity in social psychology.
Academic social scientists overwhelmingly vote Democratic, and the Democratic hegemony has increased significantly since 1970. Moreover, the policy preferences of a large sample of the members of the scholarly associations in anthropology, economics, history, legal and political philosophy, political science, and sociology generally bear out conjectures about the correspondence of partisan identification with left/right ideal types; although across the board, both Democratic and Republican academics favor government action more than the ideal types might suggest. Variations in policy views among Democrats is smaller than among Republicans. Ideological diversity (as judged not only by voting behavior, but by policy views) is by far the greatest within economics. Social scientists who deviate from left‐wing views are as likely to be libertarian as conservative.
Legal precedent establishes juvenile offenders as inherently less culpable than adult offenders and thus protects juveniles from the most severe of punishments. But how fragile might these protections be? In the present study, simply bringing to mind a Black (vs. White) juvenile offender led participants to view juveniles in general as significantly more similar to adults in their inherent culpability and to express more support for severe sentencing. Indeed, these differences in participants' perceptions of this foundational legal precedent distinguishing between juveniles and adults accounted for their greater support for severe punishment. These results highlight the fragility of protections for juveniles when race is in play. Furthermore, we suggest that this fragility may have broad implications for how juveniles are seen and treated in the criminal justice system.
In this article, we accomplish two things. First, we show that despite empirical psychologists' nominal endorsement of a low rate of false-positive findings (≤ .05), flexibility in data collection, analysis, and reporting dramatically increases actual false-positive rates. In many cases, a researcher is more likely to falsely find evidence that an effect exists than to correctly find evidence that it does not. We present computer simulations and a pair of actual experiments that demonstrate how unacceptably easy it is to accumulate (and report) statistically significant evidence for a false hypothesis. Second, we suggest a simple, low-cost, and straightforwardly effective disclosure-based solution to this problem. The solution involves six concrete requirements for authors and four guidelines for reviewers, all of which impose a minimal burden on the publication process.
This article first examines the ideological composition of American university faculty and then tests whether ideological homogeneity has become self-reinforcing. A randomly based national survey of 1643 faculty members from 183 four-year colleges and universities finds that liberals and Democrats outnumber conservatives and Republicans by large margins, and the differences are not limited to elite universities or to the social sciences and humanities. A multivariate analysis finds that, even after taking into account the effects of professional accomplishment, along with many other individual characteristics, conservatives and Republicans teach at lower quality schools than do liberals and Democrats. This suggests that complaints of ideologically-based discrimination in academic advancement deserve serious consideration and further study. The analysis finds similar effects based on gender and religiosity, i.e., women and practicing Christians teach at lower quality schools than their professional accomplishments would predict.
We studied publication bias in the social sciences by analyzing a known population of conducted studies—221 in total—in which there is a full accounting of what is published and unpublished. We leveraged Time-sharing Experiments in the Social Sciences (TESS), a National Science Foundation–sponsored program in which researchers propose survey-based experiments to be run on representative samples of American adults. Because TESS proposals undergo rigorous peer review, the studies in the sample all exceed a substantial quality threshold. Strong results are 40 percentage points more likely to be published than are null results and 60 percentage points more likely to be written up. We provide direct evidence of publication bias and identify the stage of research production at which publication bias occurs: Authors do not write up and submit null findings.
In 2015, Comparative Political Studies embarked on a landmark pilot study in research transparency in the social sciences. The editors issued an open call for submissions of manuscripts that contained no mention of their actual results, incentivizing reviewers to evaluate manuscripts based on their theoretical contributions, research designs, and analysis plans. The three papers in this special issue are the result of this process that began with 19 submissions. In this article, we describe the rationale for this pilot, expressly articulating the practices of preregistration and results-free review. We document the process of carrying out the special issue with a discussion of the three accepted papers, and critically evaluate the role of both preregistration and results-free review. Our main conclusions are that results-free review encourages much greater attention to theory and research design, but that it raises thorny problems about how to anticipate and interpret null findings. We also observe that as currently practiced, results-free review has a particular affinity with experimental and cross-case methodologies. Our lack of submissions from scholars using qualitative or interpretivist research suggests limitations to the widespread use of results-free review.
In this white paper, I propose a series of reforms intended to improve the rigor, timeliness, and replicability of published political science research. These changes could be proposed as part of the next solicitation for editors of the American Political Science Review or other disciplinary journals. If implemented, they would make our discipline a leader in publishing practices intended to minimize publication bias, encourage rapid and high-quality reviews, and maximize replicability and fairness to authors. In this way, they could help to enhance the quality and pace of the profession’s scholarly output and improve perceptions of its rigor both within and outside of academia over time, increasing trust in political science findings and building the credibility that is necessary for successful public engagement.
Many scholars have raised concerns about the credibility of empirical findings in psychology, arguing that the proportion of false positives reported in the published literature dramatically exceeds the rate implied by standard significance levels. A major contributor of false positives is the practice of reporting a subset of the potentially relevant statistical analyses pertaining to a research project. This study is the first to provide direct evidence of selective underreporting in psychology experiments. To overcome the problem that the complete experimental design and full set of measured variables are not accessible for most published research, we identify a population of published psychology experiments from a competitive grant program for which questionnaires and data are made publicly available because of an institutional rule. We find that about 40% of studies fail to fully report all experimental conditions and about 70% of studies do not report all outcome variables included in the questionnaire. Reported effect sizes are about twice as large as unreported effect sizes and are about 3 times more likely to be statistically significant.
This article describes the current debate on the practice of preregistration in political science—that is, publicly releasing a research design before observing outcome data. The case in favor of preregistration maintains that it can restrain four potential causes of publication bias, clearly distinguish deductive and inductive studies, add transparency regarding a researcher’s motivation, and liberate researchers who may be pressured to find specific results. Concerns about preregistration maintain that it is less suitable for the study of historical data, could reduce data exploration, may not allow for contextual problems that emerge in field research, and may increase the difficulty of finding true positive results. This article makes the case that these concerns can be addressed in preregistered studies, and it offers advice to those who would like to pursue study registration in their own work.
The accuracy of published findings is compromised when researchers fail to report and adjust for multiple testing. Preregistration
of studies and the requirement of preanalysis plans for publication are two proposed solutions to combat this problem. Some
have raised concerns that such changes in research practice may hinder inductive learning. However, without knowing the extent
of underreporting, it is difficult to assess the costs and benefits of institutional reforms. This paper examines published
survey experiments conducted as part of the Time-sharing Experiments in the Social Sciences program, where the questionnaires
are made publicly available, allowing us to compare planned design features against what is reported in published research.
We find that: (1) 30% of papers report fewer experimental conditions in the published paper than in the questionnaire; (2)
roughly 60% of papers report fewer outcome variables than what are listed in the questionnaire; and (3) about 80% of papers
fail to report all experimental conditions and outcomes. These findings suggest that published statistical tests understate
the probability of type I errors.
This article examines the impact of candidate race and campaign negativity on candidate evaluations and turnout. Unlike previous research, we argue that candidate race and campaign negativity should be considered simultaneously. In order to test this argument, we conduct a survey experiment of a nationally representative sample of White adults and a replication study. While we find, consistent with previous research, that respondents unfavorably evaluate candidates who decide to sponsor a negative ad, there are two important exceptions to this pattern: When the ad sponsor is Black, among White respondents who view Blacks negatively, the penalty for going negative is disproportionately large, while among White respondents who view Blacks positively, the penalty for going negative is disproportionately small. More generally, our findings suggest that the effects of candidate attributes and campaign strategy on voter behavior should not be considered in isolation, as they are mutually reinforcing.
Pundits and politicians debated whether race was implicated in the rancorous public forums and demonstrations over health care reform. Research suggests that for many white Americans, racial predispositions play a greater role in their opinions on health care than non-racial predispositions. Building on this work, I examine the extent to which anger uniquely activates white racial attitudes and increases their effect on preferences for health care reform. My theory suggests this effect occurs because anger and thoughts about race are tightly linked in memory. Using a nationally representative experiment over two waves, I induced several emotions to elicit anger, fear, enthusiasm, or relaxation. The results show that anger uniquely pushes racial conservatives to be more opposing of health care reform while it triggers more support among racial liberals. On the other hand, anger does not enhance the effect of race-neutral principles on health care reform.
We replicate Reinhart and Rogoff (2010A and 2010B) and find that selective exclusion of available data, coding errors and inappropriate weighting of summary statistics lead
to serious miscalculations that inaccurately represent the relationship between public debt and GDP growth among 20 advanced
economies. Over 1946–2009, countries with public debt/GDP ratios above 90% averaged 2.2% real annual GDP growth, not −0.1%
as published. The published results for (i) median GDP growth rates for the 1946–2009 period and (ii) mean and median GDP
growth figures over 1790–2009 are all distorted by similar methodological errors, although the magnitudes of the distortions
are somewhat smaller than with the mean figures for 1946–2009. Contrary to Reinhart and Rogoff’s broader contentions, both
mean and median GDP growth when public debt levels exceed 90% of GDP are not dramatically different from when the public debt/GDP
ratios are lower. The relationship between public debt and GDP growth varies significantly by period and country. Our overall
evidence refutes RR’s claim that public debt/GDP ratios above 90% consistently reduce a country’s GDP growth.
This article makes the case for the systematic registration of political studies. By proposing a research design before an outcome variable is observed, a researcher commits him- or herself to a theoretically motivated method for studying the object of interest. Further, study registration prompts peers of the discipline to evaluate a study's quality on its own merits, reducing norms to accept significant results and reject null findings. To advance this idea, the Political Science Registered Studies Dataverse (http://dvn.iq.harvard.edu/dvn/dv/registration) has been created, in which scholars may create a permanent record of a research design before completing a study. This article also illustrates the method of registration through a study of the impact of the immigration issue in the 2010 election for the U.S. House of Representatives. Prior to the election, a design for this study was posted on the Society for Political Methodology website (http://polmeth.wustl.edu/mediaDetail.php?docId=1258). After the votes were counted, the study was completed in accord with the design. The treatment effect in this theoretically specified design was indiscernible, but a specification search could yield a significant result. Hence, this article illustrates the argument for study registration through a case in which the result could easily be manipulated. © The Author 2013. Published by Oxford University Press on behalf of the Society for Political Methodology. All rights reserved.
Can a single conversation change minds on divisive social issues, such as same-sex marriage? A randomized placebo-controlled trial assessed whether gay (n = 22) or straight (n = 19) messengers were effective at encouraging voters (n = 972) to support same-sex marriage and whether attitude change persisted and spread to others in voters' social networks. The results, measured by an unrelated panel survey, show that both gay and straight canvassers produced large effects initially, but only gay canvassers' effects persisted in 3-week, 6-week, and 9-month follow-ups. We also find strong evidence of within-household transmission of opinion change, but only in the wake of conversations with gay canvassers. Contact with gay canvassers further caused substantial change in the ratings of gay men and lesbians more generally. These large, persistent, and contagious effects were confirmed by a follow-up experiment. Contact with minorities coupled with discussion of issues pertinent to them is capable of producing a cascade of opinion change.
Copyright © 2014, American Association for the Advancement of Science.
We studied publication bias in the social sciences by analyzing a known population of conducted studies—221 in total—in which
there is a full accounting of what is published and unpublished. We leveraged Time-sharing Experiments in the Social Sciences
(TESS), a National Science Foundation–sponsored program in which researchers propose survey-based experiments to be run on
representative samples of American adults. Because TESS proposals undergo rigorous peer review, the studies in the sample
all exceed a substantial quality threshold. Strong results are 40 percentage points more likely to be published than are null
results and 60 percentage points more likely to be written up. We provide direct evidence of publication bias and identify
the stage of research production at which publication bias occurs: Authors do not write up and submit null findings.
This article investigates the extent to which citation and publication patterns differ between men and women in the international relations (IR) literature. Using data from the Teaching, Research, and International Policy project on peer-reviewed publications between 1980 and 2006, we show that women are systematically cited less than men after controlling for a large number of variables including year of publication, venue of publication, substantive focus, theoretical perspective, methodology, tenure status, and institutional affiliation. These results are robust to a variety of modeling choices. We then turn to network analysis to investigate the extent to which the gender of an article's author affects that article's relative centrality in the network of citations between papers in our sample. Articles authored by women are systematically less central than articles authored by men, all else equal. This is likely because (1) women tend to cite themselves less than men, and (2) men (who make up a disproportionate share of IR scholars) tend to cite men more than women. This is the first study in political science to reveal significant gender differences in citation patterns and is especially meaningful because citation counts are increasingly used as a key measure of research's quality and impact.Daniel Maliniak is a Ph.D. candidate in the Department of Political Science at the University of California, San Diego. E-mail: dmaliniak@ucsd.edu
Ryan Powers is a graduate student in the Department of Political Science at the University of Wisconsin—Madison. E-mail: rpowers@wisc.edu
Barbara F. Walter is Professor of Political Science and Associate Dean at the Graduate School of International Relations and Pacific Studies, University of California, San Diego. E-mail: bfwalter@ucsd.edu
Researchers have argued that explicit racial appeals are rejected in contemporary American politics because they are perceived as violating the norm of racial equality. We test this claim with an experimental design, embedded in a representative survey of Georgia where, until recently, the state flag featured the Confederate battle emblem. In our experiment, we manipulate the salience of racial cues in news accounts of the state flag controversy in Georgia. We hypothesize that women are more likely than men to reject explicit racial appeals. We focus on the effects of explicit messages in two areas: support for Confederate symbols and identification with the Democratic Party. As hypothesized, when the racial significance of this debate is made explicit support for the Confederate flag declines, but only among women. Similarly, explicit appeals lead to lower levels of Democratic identification among men, but among women the effects are weaker and less consistent.
Does High Public Debt Consistently Stifle Economic Growth? A Critique of Reinhart and Rogoff
- Michaelash Thomasherndon
The Immigration Issue and the 2010 House Elections: A Research Design.” Political Science Registered Studies Dataverse at the University of Georgia
- James E Monogan
- Iii
The Impact of Anti-Black Racism on Approval of Barack Obama’s Job Performance and on Voting in the 2012 Presidential Election
- Jon A Joshpasek
- Trevortompson Krosnick
AP Poll: Majority Harbor Prejudice against Blacks
- Sonyaross Jenniferagiesta
Reproduction data for: “Statistical Discrimination, Stereotyping, and Evaluations of Worker Productivity
- Stephen W Benard
Benard, Stephen W. 2005. Reproduction data for: "Statistical Discrimination,
Stereotyping, and Evaluations of Worker Productivity." http://www.
tessexperiments.org/data/benard241.html.
Irregularities in LaCour
- Joshuakalla Davidbroockman
What the AP Poll on Racial Attitudes Really Tells Us
- David Wmoore
The Case of the Amazing Gay-Marriage Data: How a Graduate Student Reluctantly Uncovered a Huge Scientific Fraud
- Jesse Singal
Singal, Jesse. 2015. "The Case of the Amazing Gay-Marriage Data: How a Graduate
Student Reluctantly Uncovered a Huge Scientific Fraud." New York Magazine,
May 29. http://nymag.com/scienceofus/2015/05/how-a-grad-student-uncovereda-huge-fraud.html.
Underreporting in Psychology Experiments: Evidence from a Study Registry
- Neilmalhotra Anniefranco
The Impact of Explicit Racial Cues on Gender Differences in Support for Confederate Symbols and Partisanship
- L Vincent
- Hutchings
- Andreabenjamin Haneswalton
Why Study Social Science
- John Sides
Sides, John. 2013. "Why Study Social Science." The Monkey Cage, February 5.
http://themonkeycage.org/2013/02/why-study-social-science.
Evidence from US Public Opinion.”; citation_author=Baker, Andy; citation_publication_date=2015; citation_journal_title=American
- Race
- Foreign Paternalism
- Aid
The Impact of Anti-Black Racism on Approval of Barack Obama’s Job Performance and on Voting in the 2012 Presidential Election
- Josh Pasek
- Jon A Krosnick
- Trevor Tompson
Pasek, Josh, Jon A. Krosnick, and Trevor Tompson. 2012. "The Impact of Anti-Black
Racism on Approval of Barack Obama's Job Performance and on Voting in the
2012 Presidential Election." Unpublished report. http://joshpasek.com/
wp-content/uploads/2012/10/2012-Voting-and-Racism.pdf.
What the AP Poll on Racial Attitudes Really Tells Us, Part 1
- David W Moore
Moore, David W. 2012. "What the AP Poll on Racial Attitudes Really Tells Us, Part 1."
iMediaEthics, Nov. 2. http://www.imediaethics.org/News/3555/What_the_ap_
poll_on_racial_attitudes_really_tells_us__part_1_.php.
The Public’s Fear: Revisiting the Emotional Influences on White Racial Attitudes about Health Care Reform
- L J Zigerell
---. 2015b. "Is the Gender Citation Gap in International Relations Driven by
Elite Papers?" Research & Politics 2 (2). doi: 10.1177/2053168015585192.
---. 2016. "The Public's Fear: Revisiting the Emotional Influences on White
Racial Attitudes about Health Care Reform." Unpublished manuscript.
AP Poll: Majority Harbor Prejudice against Blacks
- Jennifer Agiesta
- Sonya Ross
Agiesta, Jennifer and Sonya Ross. 2012. "AP Poll: Majority Harbor Prejudice
against Blacks." Associated Press, October 27. http://news.yahoo.com/
ap-poll-majority-harbor-prejudice-against-blacks-073551680-election.html.
- David Broockman
- Joshua Kalla
- Peter Aronow
Broockman, David, Joshua Kalla, and Peter Aronow. 2015. "Irregularities in LaCour
(2014)." http://web.stanford.edu/∼dbroock/broockman_kalla_aronow_lg_
irregularities.pdf.
“False-Positive Psychology Undisclosed Flexibility in Data Collection and Analysis Allows Presenting Anything as Significant.”
- Simmons