Jeffrey A. Friedman’s research while affiliated with Dartmouth College and other places

Foreign policymakers must grapple with complexity, uncertainty, and subjectivity. These challenges raise the possibility that “strategy is an illusion”: that there is no reliable method for assessing skill at managing international politics. By contrast, we show that researchers can objectively evaluate a critical component of foreign policy competence using a standard we call “relative foresight,” defined as decision-makers’ ability to anticipate consequences of their choices as compared to alternative views based on similar information. Relative foresight can be measured without relying on value judgments or subjective probabilities. By contrast, other common frameworks for gauging foreign policy competence, such as comparing leaders’ behavior to the rational actor model or assessing procedural rationality, almost always leave room for reasonable disagreement. We demonstrate that relative foresight provides a useful tool for evaluating major foreign policy choices through case studies of Barack Obama’s decisions regarding the Afghan Surge and the raid on Osama bin Laden’s compound. Our framework has broad implications for research on normative, prescriptive, and descriptive dimensions of foreign policy analysis.

We interviewed national security professionals to understand why the U.S. Intelligence Community has not systematically incorporated prediction markets or prediction polls into its intelligence reporting. This behavior is surprising since crowdsourcing platforms often generate more accurate predictions than traditional forms of intelligence analysis. Our interviews suggest that three principal barriers to adopting these platforms involved (i) bureaucratic politics, (ii) decision-makers lacking interest in probability estimates, and (iii) lack of knowledge about these platforms’ capabilities. Interviewees offered many actionable suggestions for addressing these challenges in future efforts to incorporate crowdsourcing platforms or other algorithmic tools into intelligence tradecraft.

Why do Americans’ priorities for combating risks like terrorism, climate change, and violent crime often seem so uncorrelated with the dangers that those risks objectively present? Many scholars believe the answer to this question is that heuristics, biases, and ignorance cause voters to misperceive risk magnitudes. By contrast, this article argues that Americans’ risk priorities primarily reflect judgments about the extent to which some victims deserve more protection than others and the degree to which it is appropriate for government to intervene in different areas of social life. The article supports this argument with evidence drawn from a survey with 3,000 respondents, using pairwise comparisons to elicit novel measures of how respondents perceive nine dimensions of 100 life‐threatening risks. Respondents were well informed about these risks’ relative magnitudes—the correlation between perceived and actual mortality was .82—but those perceptions explained relatively little variation in policy preferences relative to judgments about the status of victims and the appropriate role of government. These findings hold regardless of political party, education, and other demographics. The article thus argues that the key to understanding Americans’ divergent reactions to risk lies more with their values than with their grasp of factual information.

Scholars, practitioners, and pundits often leave their assessments of uncertainty vague when debating foreign policy, arguing that clearer probability estimates would provide arbitrary detail instead of useful insight. We provide the first systematic test of this claim using a data set containing 888,328 geopolitical forecasts. We find that coarsening numeric probability assessments in a manner consistent with common qualitative expressions—including expressions currently recommended for use by intelligence analysts—consistently sacrifices predictive accuracy. This finding does not depend on extreme probability estimates, short time horizons, particular scoring rules, or individual attributes that are difficult to cultivate. At a practical level, our analysis indicates that it would be possible to make foreign policy discourse more informative by supplementing natural language-based descriptions of uncertainty with quantitative probability estimates. More broadly, our findings advance long-standing debates over the nature and limits of subjective judgment when assessing social phenomena, showing how explicit probability assessments are empirically justifiable even in domains as complex as world politics.

When making decisions under uncertainty, it is important to distinguish between the probability that a judgment is true and the confidence analysts possess in drawing their conclusions. Yet analysts and decision-makers often struggle to define “confidence” in this context, and many ways that scholars use this term do not necessarily facilitate decision-making under uncertainty. To help resolve this confusion, we argue for disaggregating analytic confidence along three dimensions: reliability of available evidence, range of reasonable opinion, and responsiveness to new information. After explaining how these attributes hold different implications for decision-making in principle, we present survey experiments examining how analysts and decision-makers employ these ideas in practice. Our first experiment found that each conception of confidence distinctively influenced national security professionals' evaluations of high-stakes decisions. Our second experiment showed that inexperienced assessors of uncertainty could consistently discriminate among our conceptions of confidence when making political forecasts. We focus on national security, where debates about defining “confidence levels” have clear practical implications. But our theoretical framework generalizes to nearly any area of political decision-making, and our empirical results provide encouraging evidence that analysts and decision-makers can grasp these abstract elements of uncertainty.

This book is the first work to build a conceptual framework describing how the pursuit of military effectiveness can present military and political tradeoffs, such as undermining political support for the war, creating new security threats, and that seeking to improve effectiveness in one aspect can reduce effectiveness in other aspects. Here are new ideas about military effectiveness, covering topics such as military robotics, nuclear weapons, insurgency, war finance, public opinion, and others. The study applies these ideas to World War II, the Korean War, the Vietnam War, and the 1973 October War, as well as ongoing conflicts and public policy debates, such as the War on Terror, drone strikes, ISIS, Russian aggression against Ukraine, US-Chinese-Russian nuclear competitions, and the Philippines insurgency, among others. Both scholarly and policy-oriented readers will gather new insights into the political dimensions of military power, and the complexities of trying to grow military power. Develops new ideas about important elements of military effectiveness. Mathematical treatments are avoided. Addresses contemporary policy topics such as drone strikes, military robotics, counterinsurgency, and others.

National security is one of many fields where experts make vague probability assessments when evaluating high-stakes decisions. This practice has always been controversial, and it is often justified on the grounds that making probability assessments too precise could bias analysts or decision makers. Yet these claims have rarely been submitted to rigorous testing. In this paper, we specify behavioral concerns about probabilistic precision into falsifiable hypotheses which we evaluate through survey experiments involving national security professionals. Contrary to conventional wisdom, we find that decision makers responding to quantitative probability assessments are less willing to support risky actions and more receptive to gathering additional information. Yet we also find that when respondents estimate probabilities themselves, quantification magnifies overconfidence, particularly among low-performing assessors. These results hone wide-ranging concerns about probabilistic precision into a specific and previously undocumented bias that training may be able to correct.