No full-text available
To read the full-text of this research,
you can request a copy directly from the author.
Expert Political Judgment: How Good is It? How can We Know?
Abstract
The intelligence failures surrounding the invasion of Iraq dramatically illustrate the necessity of developing standards for evaluating expert opinion. This book fills that need. Here, Philip E. Tetlock explores what constitutes good judgment in predicting future events, and looks at why experts are often wrong in their forecasts. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. He evaluates predictions from experts in different fields, comparing them to predictions by well-informed laity or those based on simple extrapolation from current trends. He goes on to analyze which styles of thinking are more successful in forecasting. Classifying thinking styles using Isaiah Berlin's prototypes of the fox and the hedgehog, Tetlock contends that the fox--the thinker who knows many little things, draws from an eclectic array of traditions, and is better able to improvise in response to changing events--is more successful in predicting the future than the hedgehog, who knows one big thing, toils devotedly within one tradition, and imposes formulaic solutions on ill-defined problems. He notes a perversely inverse relationship between the best scientific indicators of good judgement and the qualities that the media most prizes in pundits--the single-minded determination required to prevail in ideological combat. Clearly written and impeccably researched, the book fills a huge void in the literature on evaluating expert opinion. It will appeal across many academic disciplines as well as to corporations seeking to develop standards for judging expert decision-making.
... Participants in high-stakes policy debates frequently make claims about long-run futures that may not be resolvable for decades, or may never be resolvable because of the impossibility of rerunning history and quantifying groundtruth probability distributions of possible outcomes (Tetlock, 2005). For instance, will we ever know how fragile nuclear deterrence was in the Cold War? Perhaps there was an 80% probability of a nuclear war between 1948 and 1991, but humanity fortunately fell into the 20% bin of relatively peaceful possible worlds. ...
... But they are fallible. Experts working in domains without regular accuracy feedback have more difficulty translating their causal knowledge into probabilistic predictions and often fail to outperform educated generalists in forecasting tournaments (Bolger & Wright, 2017;Kahneman & Klein, 2009;Tetlock, 2005). ...
... These scoring rules treat errors of underand overestimation as equally serious (Brier, 1950;Good, 1952;Jose & Winkler, 2009). These rules should help check the temptation in polarized debates to pump up or tamp down probabilities to assist the rhetorical mission of mobilizing or depressing public support for particular policy proposals (Grundmann, 2022;Jervis, 1976;Mercier & Sperber, 2011;Tetlock, 2005). ...
We organized adversarial collaborations between subject‐matter experts and expert forecasters with opposing views on whether recent advances in Artificial Intelligence (AI) pose an existential threat to humanity in the 21st century. Two studies incentivized participants to engage in respectful perspective‐taking, to share their strongest arguments, and to propose early‐warning indicator questions (cruxes) for the probability of an AI‐related catastrophe by 2100. AI experts saw greater threats from AI than did expert forecasters, and neither group changed its long‐term risk estimates, but they did preregister cruxes whose resolution by 2030 would sway their views on long‐term risk. These persistent differences shrank as questioning moved across centuries, from 2100 to 2500 and beyond, by which time both groups put the risk of extreme negative outcomes from AI at 30%–40%. Future research should address the generalizability of these results beyond our sample to alternative samples of experts, and beyond the topic area of AI to other questions and time frames.
... At the core of experts' responsibilities lies the imperative to make non-trivial decisions, such as diagnoses and selecting interventions (Tetlock 2017). The prevailing assumption is that most experts often make correct decisions. ...
... Decision accuracy is integral to an expert's judgment quality and expertise (Tetlock 2017) and, thus, consequential to managers and consumers of experts' services. However, no scalable, inexpensive, and reliable means to infer experts' decision accuracies is available to inform managers and consumers. ...
... Decision accuracy is a fundamental aspect of experts' judgment quality (Tetlock 2017). When ground truth labels are scarce, poor transparency of experts' decision accuracies undermines consumers' choices and the effective management of these experts. ...
Expert workers make non-trivial decisions with significant implications. Experts’ decision accuracy is, thus, a fundamental aspect of their judgment quality, key to both management and consumers of experts’ services. Yet, in many important settings, transparency in experts’ decision quality is rarely possible because ground truth data for evaluating the experts’ decisions is costly and available only for a limited set of decisions. Furthermore, different experts typically handle exclusive sets of decisions, and thus, prior solutions that rely on the aggregation of multiple experts’ decisions for the same instance are inapplicable. We first formulate the problem of estimating experts’ decision accuracy in this setting and then develop a machine–learning–based framework to address it. Our method effectively leverages both abundant historical data on workers’ past decisions and scarce decision instances with ground truth labels. Using both semi-synthetic data based on publicly available data sets and purposefully compiled data sets on real workers’ decisions, we conduct extensive empirical evaluations of our method’s performance relative to alternatives. The results show that our approach is superior to existing alternatives across diverse settings, including settings that involve different data domains, experts’ qualities, and amounts of ground truth data. To our knowledge, this paper is the first to posit and address the problem of estimating experts’ decision accuracies from historical data with scarce ground truth, and it is the first to offer comprehensive results for this problem setting, establishing the performances that can be achieved across settings as well as the state-of-the-art performance on which future work can build.
This paper was accepted by Anindya Ghose, information systems.
Funding: T. Geva acknowledges research grants from the Jeremy Coller Foundation and from the Henry Crown Institute for Business Research.
Supplemental Material: The online appendix and data files are available at https://doi.org/10.1287/mnsc.2021.03357 .
... In this way, overconfidence in general is when our probabilities for a set of propositions exceed the proportion of those propositions that are true. 11 Numerous studies have reported that humans are often overconfident in this and similar ways (Barnsley et al. 2004;Berner and Graber 2008;Callender et al. 2016;Ehrlinger et al. 2008;Hall et al. 2016;Kruger and Dunning 1999;Lechuga and Wiebe 2011;Lundeberg et al. 2000;Meyer et al. 2013;Perel et al. 2016;Podbregar et al. 2001;Tetlock 2005;Whitcomb et al. 1995;Wright et al. 1978;Yates et al. 1989Yates et al. , 1996Yates et al. , 1997Yates et al. , 1998Yates et al. , 2002. For example, one study found that one group of students collectively assigned a probability of 95% or more to a set of propositions, but only 73% of those propositions were correct (Lechuga and Wiebe 2011). ...
... Another study found that a group of "political experts" were certain-with a probability of 100%-that particular events would not happen in the long-term. But it turns out 19% of those things actually did happen (Tetlock 2005). According to that study, having a PhD or many years of professional experience did not substantially improve calibration either. ...
... But we cannot trust that such a feature bestows calibration unless we know the feature is of a kind that has a good track record of past calibration. (And in this case, Tetlock's (2005) study found that having a PhD in, say, political topics was not significantly correlated with superior calibration when forecasting about political topics. The Forecasting Collaborative (2023) found that holding a PhD had no statistically significant influence on forecasting accuracy about social scientific topics too.) ...
All of us make judgments of probability, and we rely on them for our decision-making. This paper argues that such judgments are trustworthy only to the extent that one has good reasons to think that they are produced by maximally inclusive, well calibrated cognitive processes. A cognitive process is maximally inclusive when it takes into account all the evidence which one regards as relevant, and it is well calibrated when anything it would assign, say, an 80% probability to would be true 80% of the time. We further have good reasons to think these judgments are trustworthy when, inter alia, they are produced by processes that have good track records of calibration. Call this inclusive calibrationism—or just “calibrationism” for short. In arguing for calibrationism, I also appeal to various empirical results, including research into probabilistic reasoning funded by the US intelligence community. Together, these ideas and results have implications for some important philosophical problems: the problem of the priors, the problem of unique events and the use of intuition in probabilistic reasoning. These theses and results also imply that our judgments are often less trustworthy than we might hope for potentially many domains, including law, medicine and others—barring good track records, that is.
... In some cases, predictions appear as if they perfectly anticipate the future, but in many other cases, the predictions of even our best experts and models appear to be no better than random. 9,10,11 While predictions vary in many ways, they all share one characteristic -they are the product of at least one of three sources: (i) a single human, typically an expert, (ii) a group of humans, commonly referred to as a crowd, or (iii) a model -statistical, rule-based, or otherwise -i.e., an algorithm. It is also possible to construct a "model of models" or ensemble of all three approaches; in fact, crowds are themselves a "model of models" built on predictions of individuals. ...
... The average number of predictions per participant per outcome is approximately 1.1, and some cast many more predictions; in fact, some of our highestscoring participants act like "foxes," frequently updating their predictions for some cases. 9 While patterns of behavior like "foxes" and "hedgehogs" are worthy of additional research, for the purposes of this paper, we exclude from our analysis all "non-final" predictions. The resulting set of "final" predictions -the last recorded prediction for each participant for each Justice in each case that was cast before midnight on the date of decision of the case -consists of 545,845 remaining predictions. ...
Scholars have increasingly investigated "crowdsourcing" as an alternative to expert-based judgment or purely data-driven approaches to predicting the future. Under certain conditions, scholars have found that crowdsourcing can outperform these other approaches. However, despite interest in the topic and a series of successful use cases, relatively few studies have applied empirical model thinking to evaluate the accuracy and robustness of crowdsourcing in real-world contexts. In this paper, we offer three novel contributions. First, we explore a dataset of over 600,000 predictions from over 7,000 participants in a multi-year tournament to predict the decisions of the Supreme Court of the United States. Second, we develop a comprehensive crowd construction framework that allows for the formal description and application of crowdsourcing to real-world data. Third, we apply this framework to our data to construct more than 275,000 crowd models. We find that in out-of-sample historical simulations, crowdsourcing robustly outperforms the commonly-accepted null model, yielding the highest-known performance for this context at 80.8% case level accuracy. To our knowledge, this dataset and analysis represent one of the largest explorations of recurring human prediction to date, and our results provide additional empirical support for the use of crowdsourcing as a prediction method.
... On the one hand, proponents of various methods have claimed to accurately predict a wide variety of phenomena, ranging from the box-office success of movies [2], to the outcomes of political conflicts [13] and social trends [19], to the spread of epidemics [29,12] and "viral" products [6], to the stock market [9]. On the other hand, critics have contended that claims of success often paper over track records of failure [48], that expert predictions are no better than random [55,20], that most predictions are wrong [47,14,40], and even that predicting social and economic phenomena of any importance is essentially impossible [54]. Repeated attempts to deflate expectations notwithstanding, the steady arrival of new methods-game theory [13], prediction markets [52,1], and machine learning [17]-along with new sources of data-search logs [11], social media [2,9], MRI scans [7]-inevitably restore hope that accurate predictions are just around the corner. ...
... Compounding the lack of clarity in the claims themselves is an absence of a consistent and rigorous evaluation framework. For example, it is well understood in theory that predictive accuracy cannot be reliably estimated from isolated predictions, especially when selected ex-post [55]. Likewise, is uncontroversial to state that model performance can be evaluated only with respect to the most relevant baseline [23], or that incremental performance improvements do not necessarily translate to meaningful improvements in the outcome of interest [24]. ...
How predictable is success in complex social systems? In spite of a recent profusion of prediction studies that exploit online social and information network data, this question remains unanswered, in part because it has not been adequately specified. In this paper we attempt to clarify the question by presenting a simple stylized model of success that attributes prediction error to one of two generic sources: insufficiency of available data and/or models on the one hand; and inherent unpredictability of complex social systems on the other. We then use this model to motivate an illustrative empirical study of information cascade size prediction on Twitter. Despite an unprecedented volume of information about users, content, and past performance, our best performing models can explain less than half of the variance in cascade sizes. In turn, this result suggests that even with unlimited data predictive performance would be bounded well below deterministic accuracy. Finally, we explore this potential bound theoretically using simulations of a diffusion process on a random scale free network similar to Twitter. We show that although higher predictive power is possible in theory, such performance requires a homogeneous system and perfect ex-ante knowledge of it: even a small degree of uncertainty in estimating product quality or slight variation in quality across products leads to substantially more restrictive bounds on predictability. We conclude that realistic bounds on predictive accuracy are not dissimilar from those we have obtained empirically, and that such bounds for other complex social systems for which data is more difficult to obtain are likely even lower.
... The process could be constructed for different teams or organizations with varying time frames separately to compare their relative performance and control for background factors along the lines of e.g. Tetlock (2005), Tetlock and Gardner (2015), Tetlock et al. (2014). ...
... The Brier score, developed by Glenn W. Brier in 1950(Brier, 1950, has emerged as a prominent metric for quantifying the accuracy of probabilistic forecasts, and it has been popularized especially by Tetlock (2005), Tetlock and Gardner (2015). The score is designed for discrete cases and it measures the mean squared difference between predicted probabilities and the actual outcomes. ...
... Past research has found that consumers use heuristics when making purchasing decisions (Bettman et al., 1991). The literature has also established that although organizations employ 'professional experts' to make purchasing decisions, the experts also tend to rely on heuristics (Kahneman et al., 1982;Tetlock, 2005). ...
... Although experts have greater technical knowledge than non-professional consumers, they may still rely on heuristics of names in the ODD market for three key reasons. First, the literature on judgement and decision making has found that experts are subject to cognitive biases, learn poorly from the past experience, and often make decisions that are not superior to those of non-experts (Kahneman et al., 1982;Northcraft and Neale, 1987;Smit, 2023;Tetlock, 2005). Second, experts are likely to make more informed decisions (i.e., invest more time and resources into careful data collection and analysis of a product) and learn from experience, when they buy components of critical importance for a long-term use than when they buy less critical components with a short life span (O'Hara and Payne, 1998). ...
How do customers discover new products? Recent research has found that a firm can facilitate the discovery and subsequent purchase of its product by giving it an advantageous name. However, no product exists in isolation, rather it competes for customer attention with other products both within and across product niches. We theorize that a product may benefit from the names of competitors’ products within its niche because certain product names can trigger a positive spillover effect. Specifically, product viability should increase with the proliferation of products with informative names in a focal niche because informative names attract attention to the niche, and consequently benefit all its products, regardless of whether they have informative names or not. This beneficial influence should be especially strong when a niche is new. Additionally, a product's market fate may depend not only on the prevalent naming practices in its niche, but also on naming practices in competing niches. We find support for our theorizing in event‐history analyses of all CD‐drive products shipped in the worldwide optical disk drive industry, 1983–1999. Ultimately, our findings suggest that in high‐velocity markets, to facilitate product discovery by customers, firms should enter niches populated by products with informative names.
... Examples of environments that are not suitable for intuitive decisions are the prediction of political events with a long time lag, the prediction of the development of stock prices or even the probability that a person will actually commit suicide in the future (Bryan and Rudd, 2006;Kahneman, 2016). While intuitive decisions are also made in such environments, the success and accuracy of such a decision are mostly due to lucky coincidences (Kahneman, 2016;Taleb, 2018;Tetlock, 2017). These lucky predictions can lead to a high degree of conviction in one's own intuition, but this is not an indicator for evaluating its correctness (Kahneman, 2016). ...
... This challenge is a particularly difficult one, as highly experienced professionals in particular tend to trust and systematically overestimate their intuition (Sicora et al., 2021a;Tetlock, 2017), making it challenging to assess the limits of their own expertise (Kahneman and Klein, 2009). Sicora et al. (2021a) report that social workers in Italy and China not only unconsciously use intuition but also view it as helpful and of high quality. ...
Decision-making is an essential part of social work practice. Intuition is one possible basis for these decisions, but relying solely on it might not always be the best choice. No overarching framework for social work exists on how to deal with intuition. By building on the work of Kahneman and Klein, who describe conditions for successful intuitive-reasoning, the aim of the article is thus to offer such a framework for further research and reflection on social work practice. It can be used as a guideline for social work practice and social care policy to improve intuitive-reasoning of social workers.
... Experts, at best, do only slightly better than random chance, and those who are more open-minded and self-critical tend to do a better job making predictions than those who are more arrogant and dogmatic. Thus, one should seek experts willing to revise their opinions when presented with new evidence (Tetlock, 2005). Kahneman (2011, pp. ...
... Kahneman (2011, pp. 218-219) cites research conducted by Tetlock (2005) that demonstrates how poorly experts who make a living "commenting or offering advice on political and economic trends" actually perform. They do not do better than monkeys throwing darts on a board displaying the various possible outcomes (Kahneman 2011, p. 219). ...
... Second, there is what we might call 'the Teltock problem.' Philip Tetlock's seminal work (P. Tetlock, 2005) found that, on a range of topics, predictions of subject-matter experts tend to be less reliable than predictions of nonexperts or simple base-rate extrapolations. The most recent investigation of the data is a bit more friendly to expertise, but it's unclear how much succor it should give the fans of expert rule. ...
This paper argues that if the critics of the currently dominant notice-and-consent model of governing digital data transactions are correct, then they should oppose political reforms of the model. The crux of the argument is as follows: the reasons the critics give for doubting the effectiveness of notice-and-consent in protecting user privacy (namely, ordinary users’ various cognitive deficiencies and the inherent inscrutability of the subject matter) are also reasons for doubting the effectiveness of protecting user privacy through democratic or regulatory means. Furthermore, insofar as attempts to improve the notice-and-consent model through more legislation or regulation would also involve more coercion than the status quo, they should be resisted on normative grounds. So, out of the bad options we have when it comes to protecting digital privacy, it seems – contrary to the majority position advanced in the literature – that we should stick with notice-and-consent.
... There are some occasions upon which we do have independent ways of verifying experts' past performance and hence can at least in principle accumulate track record data. This is most obviously so when experts make straightforwardly verifiable predictions about future events, for example, when economists forecast whether inflation will increase over the next quarter (Tetlock, 2017). Even in these cases, though, any kind of systematic track record data would require a lot of work to gather. ...
... They did not entertain the idea that other views may be correct. When they are wrong, they argue about the event's implications and focus on justifying their decision (37)(38)(39). ...
In the fog of war, failure is not an option. It is uncertainty, not a fog; uncertainty can be engaged. Failure is an option and continues to be an option until the event is resolved. Three discontinuities prevent full discussion of High-Reliability Operations as performed in dangerous contexts: (1) time as a separate dimension, (2) the organization’s orientation toward performance over learning, (3) between training and education for competency versus the mentored experience necessary for proficiency. This article describes the characteristics of these three discontinuities.
... Overconfidence Bias and the Illusion of Expertise: Overconfidence bias can lead experts to overestimate the accuracy of their judgments, particularly when they are highly experienced and respected in their field (Tetlock, 2005). This overreliance on social intuition can be particularly problematic in fields like criminal profiling, where the pressure to conform to established methods and the desire for quick solutions can override a more nuanced and objective assessment of evidence. ...
This paper challenges the common view of intuition as a purely innate and reliable capacity, arguing instead that it is significantly shaped and often distorted by societal forces, particularly the pervasive influence of standardization. Through a multidisciplinary approach encompassing history, sociology, psychology, and neuroscience, the paper examines how standardization has molded social intuition, often leading to systematic biases and flawed judgments. The analysis reveals the historical use of
standardization as a tool for social control, the suppression of individuality, and the perpetuation of inequalities. Focusing on the field of criminal profiling, the paper critiques the limitations of
standardized profiles and the dangers of uncritical reliance on intuition in high-stakes decision-making. Finally, it explores the transformative potential of artificial intelligence (AI) to augment human memory, enhance pattern recognition, and potentially mitigate biases. The paper advocates for a balanced, humancentered approach to AI development in profiling, emphasizing the enduring importance of cultivating
authentic intuition, critical thinking, and ethical considerations alongside technological advancements. The central argument is that achieving genuine objectivity in profiling, and in other fields, requires moving beyond the illusion of intuition as it commonly operates in a standardized world and embracing a more nuanced understanding of the interplay between mind, society, and technology.
... Analysts must synthesise complex, and often incomplete information to support critical judgements. This process is influenced by factors both internal to the individual -such as cognitive biases, emotional responses, and personal values -and external, including organisational structures, policies, and cultural norms (Bazerman & Moore, 2012;Kahneman, 2011;Tetlock, 2005). These factors not only shape how decisions are made but also create the conditions under which dilemmas emerge. ...
... Given the fallibility of human judgment, scholars must approach their work with humility. Countless studies have demonstrated the limitations of expert predictions, which often do not perform better than monkeys throwing darts on a board displaying the various possible outcomes, highlighting the dangers of overconfidence (Huded, Rosno, and Prasad, 2013;Ioannidis, 2005;Kahneman 2011, p. 219;Tetlock, 2005). The prevalence of contradictory research further underscores the need for intellectual modesty. ...
The post-truth era, characterized by information overload, the ascendance of image over substance, and the proliferation of misinformation, has led to a decline in critical thinking and a rise in dangerously simplistic binary thinking. This phenomenon, exacerbated by a mental health crisis among young adults, necessitates the cultivation of intellectual humility. This paper provides a framework for achieving success and happiness. By adopting a multifaceted approach, including intellectual humility and critical thinking, individuals can navigate the challenges of the modern world and enhance their mental well-being. Organizational leaders will also find these concepts useful.
... Research shows that some individuals are particularly good at making predictions and can improve their abilities to prognosticate over time (Tetlock & Gardner, 2015). A groundbreaking study at the dawn of the 21st century demonstrated that the average expert was no better at predicting political outcomes than a dart-throwing chimpanzee (Tetlock, 2005). But later works since then have been able to identify that certain nonexperts who use a three-part forecast-measure-revise technique can significantly outpredict experts who may even have access to better information. ...
Throughout the Covid-19 pandemic, commentators in broadly accessible media have offered a surfeit of predictions about the future of higher education. Due to the absence of accountability mechanisms, however, the accuracy of these claims has been heretofore unknown. Research shows that op-eds and other forms of public scholarship influence public policy, heightening the significance of predictions. This paper asks who makes predictions about higher education, in what venues they issue them, on what topics they make predictions, and how accurate they are. It answers these questions by drawing from an original data set of 91 distinct predictions issued by 22 unique authors in 31 separate texts across a 19-month time span from March 2020 to October 2021. It finds that predictions most often appeared in op-eds written by senior academic white men in higher education trade journals. More than half of predictions could not be evaluated a year or more after they were first issued. Still, predictions with determinable outcomes tended to bear out accurately. Enrollment patterns and teaching modalities were the most common topics. Women and people of color were significantly under-represented among predictors. The paper concludes with suggestions for improving equity and performance.
... 28 When it comes to such systems, it is difficult to predict their evolution, and particularly, how interventions within a particular area might affect behavior in another area. These difficulties are illustrated in some of Philip Tetlock's (2005) work on forecasting. Even experts find it hard to be reliable at predicting the behavior of complex social and political systems. ...
Censorship involves, inter alia, adopting a certain type of epistemic policy. While much has been written on the harms and benefits of free expression and the associated rights thereof, the epistemic preconditions of justified censorship are relatively underexplored. In this paper, I argue that examining intrapersonal norms of how we ought to treat evidence that might come to us over time can shed light on interpersonal norms of evidence generation and sharing that are relevant in the context of censorship. The upshot is that justified censorship requires the censor to meet a very high epistemic burden regarding the target proposition(s)—importantly, one that exceeds knowledge.
... 1. Experience: Political decision-maker experience is argued as being a moderator of the effective use of heuristics (Hafner-Burton et al., 2013). 2. Context: Experience is domain specific (Schreiber, 2007) and so judgements made out of context benefit little from experience (Ericsson, 2005) and can result in poor outcomes (Tetlock, 2005). 3. Complexity: More complex issues are associated with greater use of heuristics in decisionmaking (MacGillivray, 2014). ...
The political elite make policy decisions in noisy environments and under time pressure, and so are prone to using heuristics. There are conflicting schools of thought as to whether it is appropriate for them to do so. Experienced decision‐makers are thought to be more effective at using heuristics, so it is possible that for the political elite with experience in a particular context, heuristic decision‐making is appropriate. Yet, many politicians are asked to make decisions on matters about which they are not experts. To add to the debate, we facilitated a discussion with a highly experienced cohort of 21 current and former senior politicians, former advisers, and current and former senior bureaucrats. When presented with a carefully considered and innovative new transport network pricing policy, we sought to identify whether and, if so, how they used heuristics to make a decision. We found that they used heuristics (1) to decide whether to engage with the issue at all and (2) how to act, having made the decision to engage. We describe how these heuristics were used and discuss the implications for theory and public administration practice.
Points for practitioners
There is a growing body of evidence that the political elite use heuristics for decision‐making and that the use of heuristics is influenced by seven factors.
We gained rare access to the political elite deciding on a politically risky issue and observed not only which heuristics they used, but how they used them.
We observed a three‐step decision tree, incorporating the ‘wait‐and‐see’ heuristic being used to decide whether to act, and political empathy, or intuiting voter heuristics to help decide how to act.
We outline five options for public administrators who think that the political elite are using heuristics inappropriately for decision‐making.
... Kahneman (2011, pp. 218-219) cites research conducted by Tetlock (2005) that demonstrates how poorly experts who make a living "commenting or offering advice on political and economic trends" actually perform. They do not perform better than monkeys throwing darts on a board displaying the various possible outcomes (Kahneman 2011, p. 219). ...
Education's role extends beyond job preparation; it should equally nurture students' moral character, compassion, and civic duty. This essay argues that many higher education institutions fall short of their educational responsibilities. Instead of producing well-rounded individuals with strong ethical principles, critical thinking abilities, and job-ready skills, some schools may inadvertently stifle creativity and discourage open debate. The author suggests that a limited focus on particular ideological views and an overemphasis on identity politics can restrict students' exposure to diverse perspectives. Rather than becoming adept at logical reasoning, students have become proficient in harmful binary thinking and pathological dualism-a mindset that is likely to lead to misery.
... To estimate key parameters for our model and obtain quantitative outcomes, we employed a combination of estimates found in prior literature, close historical proxies to the parameter of interest, and academic surveys sent to domain experts. Given the limited forecasting abilities of experts 30 and considerable uncertainty in the relevant parameters, we aim to bound the potential bene ts from VDS and PVI to inform future public health investments. ...
Preventing and mitigating future zoonotic pandemics are global health priorities, but there are few quantitative estimates of how best to target interventions. Here we construct a mathematical model to evaluate the benefits of 1) virus discovery and sequencing (VDS) in animals and 2) pandemic virus identification (PVI) via laboratory characterization of pandemic potential. Drawing on historical data and expert surveys of One Health and vaccine researchers, we estimate that intensifying virus discovery efforts by three-fold could prevent between 0 and 1.46 million expected deaths per decade by improving non-pharmaceutical interventions and broad-spectrum vaccines. In contrast, because researchers estimate that there are well over a hundred pandemic-capable viruses in nature, identification through laboratory characterization would prevent 48,000 deaths per decade [10,500; 93,600], or just 0.62% of expected pandemic deaths. Further identifications would offer diminishing returns. Given wide-ranging survey responses and limited cost-effectiveness compared to proven global health interventions such as insecticide-treated bed nets, our model suggests that health establishments aiming to mitigate future pandemics should focus on monitoring spillover hotspots and empowering local communities to detect, sequence, and suppress nascent epidemics rather than characterizing pandemic potential in laboratories.
... Berlin categorized various poets and luminaries as either abstract "hedgehogs" (e.g., Plato, Hegel, Dostoyevsky, or Nietzsche) or pluralist, context-aware "foxes" (e.g., Aristotle, Franklin, Pushkin, or Diderot) who draw on both the abstract and the concrete to move on "many levels, seizing upon the essence of a vast variety of experiences." In many areas of life imbued with uncertainty (Keil, 2010) such as geopolitics (Tetlock, 2005), one may take a step back and approach an issue abstractly or consider how to balance abstract analytical principles with concrete features of the situation at hand (Brunswick, 1955;Hammond, 2010). ...
We explored how individuals’ mental representations of complex and uncertain situations impact their ability to reason wisely. To this end, we introduce situated methods to capture abstract and concrete mental representations and the switching between them when reflecting on social challenges. Using these methods, we evaluated the alignment of abstractness and concreteness with four integral facets of wisdom: intellectual humility, open-mindedness, perspective-taking, and compromise-seeking. Data from North American and UK participants (N = 1,151) revealed that both abstract and concrete construals significantly contribute to wise reasoning, even when controlling for a host of relevant covariates and potential response bias. Natural language processing of unstructured texts among high (top 25%) and low (bottom 25%) wisdom participants corroborated these results: semantic networks of the high wisdom group reveal greater use of both abstract and concrete themes compared to the low wisdom group. Finally, employing a repeated strategy-choice method as an additional measure, our findings demonstrated that individuals who showed a greater balance and switching between these construal types exhibited higher wisdom. Our findings advance understanding of individual differences in mental representations and how construals shape reasoning across contexts in everyday life.
... and avoid the distractions of short-term market fluctuations (Locke & Latham, 2002). Conducting thorough research, meanwhile, enables investors to make informed decisions based on objective data rather than emotional responses or cognitive biases (Tetlock, 2005). Seeking diverse perspectives, on the other hand, provides a safeguard against the pitfalls of groupthink and confirmation bias, ensuring that investment decisions are well-rounded and consider multiple viewpoints. ...
This report delves into the fascinating world of psychological biases and their profound impact on individual investment decisions and stock market returns. Drawing insights from the field of behavioral finance, it explores how the human mind's tendencies and cognitive quirks can lead investors astray from rational, profit-maximizing choices. The report begins by setting the stage for understanding the significance of this topic, emphasizing the importance of exploring the interplay between psychology and investing. It then dives into the core concepts, theories, and key factors that shape behavioral finance, providing a solid foundation for the subsequent analysis. A crucial aspect of the report is the in-depth examination of specific psychological biases, such as overconfidence, loss aversion, and confirmation bias, and how they manifest in stock market investing. The report presents empirical evidence, case studies, and real-world examples to illustrate the tangible effects of these biases on investment performance and personal returns. Recognizing the need for practical solutions, the report offers strategies and techniques for mitigating the influence of psychological biases. It equips investors with the tools and knowledge necessary to make more informed, rational decisions, ultimately enhancing their chances of achieving their financial goals. By the end of the report, readers will have a comprehensive understanding of the complex interplay between human psychology and investing, as well as a roadmap for navigating the challenges posed by behavioral biases. The report concludes with a summary of key insights and recommendations for future research directions, solidifying its position as a valuable resource for investors, financial professionals, and scholars alike.
... The details of Gaus's reasons for this position are beyond the scope of this paper. But his account is grounded in part on a reading of Tetlock (2006 ) and Tetlock and Gardner (2015 ). ...
This paper defends traditional political philosophy against the challenges Gaus leverages against it in The Open Society and Its Complexities. Granting Gaus that consensus on the principles of political philosophy is not forthcoming and that complexity undermines many of our most ambitious reform efforts, the paper argues that much work remains for political philosophy as it has been practiced for centuries. This is for three reasons. First, Gaus's own defense of the open society requires resources from the very traditions that he tends to dismiss. Secondly, the kinds of reform Gaus encourages us to focus on in light of complexity presuppose answers to philosophical questions, answers that philosophers have long attempted to provide. Thirdly, not all politically salient philosophical questions are aimed at action guidance. Despite the value in Gaus's analysis, I conclude that political philosophy's future will look, at least in part, like its storied past.
... Most people show a bias towards exaggerated personal qualities and capabilities, an illusion of control over events and invulnerability to risk… [It] amounts to an ʻerror' of judgement or decision-making, because it leads to overestimating one's capabilities and/or underestimating an opponent, the difficulty of a task, or possible risks. (Johnson and Fowler, 2011) Kahneman (2011) (Tetlock, 2005), which analyzed more than 80,000 predictions. ...
Abstract: Hydrologic modeling is an essential tool for analyzing the environmental effects of wildfires.
Simulations of watershed behavior are uniquely suited to emergency assessments in which data are
limited and time is scarce, such as those performed under the Burned Area Emergency Response
(BAER) Program used by Federal Land Management Agencies in the United States. In these
situations—when the values at risk (VARS) include lives and property—it is critical to remember:
“All models are wrong, but some are useful” (Box and Draper, 1987). However, all too often, neither
reports nor results rigorously reflect this imperative. With the wildfire crisis worsening each year,
improving the state of the practice can be a strategic force multiplier for agencies, NGOs, and
researchers alike.
Herein, the twin questions of how wrong and how useful are used as the foundation for an
overview of meaningful modeling within the context of postfire hydrologic assessments. Therefore,
this paper focuses on how to: (1) think about watershed modeling, (2) select a modeling strategy, and
(3) present the simulations in a meaningful way. The beginning and the end—the bread of a modeling
sandwich. Nearly a third of the content is about science communication.
While the focus is on burnt watersheds, BAER, and the US, the basic principles of modeling,
grappling with uncertainty, and science communication are universal—and often not taught in many
academic programs.
[This provisional version has not undergone use testing or formal review by theUS Forest Service and will continue to evolve until the agency officially releases it. However, it was included as chapter 9 of Wheelock S.J. (2024) Marscapes to Terrestrial Moonscapes: A Variety of Water Problems."
... 1954) investigated the extent to which people can make predictions about the future. He organised 'predict the future' competitions (Tetlock et al., 2006) in which participants were asked questions such as "Do you think a member state will leave the EU before [a given date]?" The results show that, when presented with a period longer than a year, people's predictions were hardly better than if they had just guessed. ...
The Consumer Choice Action Plan: background and
context
Financial decisions play a crucial role in our daily lives. In our
modern world, it is essential to understand how consumers behave
when faced with an important financial choice. Consumers must
confront important financial choices even though they may not
have the time, knowledge or motivation to make the right decision,
and people are often influenced by unconscious and irrational
factors.
In practice, for example, when consumers are presented with several options they will often choose the middle one, regardless of the price and features of the other alternatives. They put off creating a savings buffer until the refrigerator and washing machine both
break down at once, and they wait till the end of their current mortgage period before looking up information on interest-only mortgage options. Left to themselves, consumers do not always make the most rational decisions in their own interest. Many people see financial matters as a means, rather than an end in itself: they don’t want a mortgage – they want a house. They don’t want insurance – they just want to enjoy their skiing holiday. The financial side is a tool, a necessary evil, something they just have to do so they can get what they want.
This is not a new insight; as the 2019 report by the Netherlands Scientific Council for Government Policy (WRR) put it, “Knowing what to do is not enough”. The partners in the Consumer Choices Action Plan have always aimed to support consumers to make better
financial decisions. We want to publicise these insights and practices throughout the financial sector so that all stakeholders can take advantage of them, and so we can achieve our ultimate goal of promoting healthy financial decisions through behavioural insights.
This publication arose from the Consumer Choices Action Plan, a collaboration involving more than 50 partners and individuals from diverse backgrounds in the financial sector.
The initiatives in the Consumer Choices Action Plan are divided into five key themes, each dealing with challenges for promoting good consumer choices:
1 Financial literacy: Consumers need to be able to take action when their financial situation calls for it. How can we encourage consumers to be proactive and seek help as soon as they need it?
2 Investing: Investing has evolved. Instead of involving a physical visit to the bank, now
people mostly invest online, and often without the help of advisers. Do consumers really understand the risks?
3 Saving: Ideally, consumers should have a savings buffer to deal with unexpected financial setbacks. How can we encourage consumers to set aside money for a buffer when they have the means to do so?
4 Borrowing and leasing: Another area of finance that’s largely moved online is the world of loans and lease agreements, and again it’s less common for a financial advisor to be involved. When it comes to long-term, important financial commitments, consumers need to make the best choices for their financial situation. How can behavioural insights help them do that?
5 Financial sustainability: Although consumers are keen to use sustainable financial products and energy-saving measures, factors such as inertia, habit and a short-term focus still get in the way. What behavioural interventions will help people actually take the plunge? The structure of this publication
As you read further into this publication, the information becomes more in-depth. That said, each chapter can also be read on its own, so you can feel free to flip through and read only the parts that are relevant to you.
Chapters 2 and 3 focus on practical results from the Consumer Choice Action Plan. In Chapter 2, we take the specific results from the Action Plan’s partner organisations and make them easily accessible to other stakeholders. Chapter 3 combines findings from
Action Plan studies and experiments with the very latest knowledge and insights from financial behavioural science.
Chapter 4 offers background information on various psychological mechanisms and some associated behavioural models. We want to inform and inspire people who work in the financial industry with an overview of a wide range of psychological mechanisms that impact financial behaviour. Psychological mechanisms have a big effect on the way people make financial decisions. The literature uses various different terms to discuss these mechanisms, from ‘interventions’, ‘techniques’ and ‘heuristics’ to ‘biases’, ‘illusions’, ‘nudges’ and ‘boosts’. This chapter explains specific mechanisms and models based on these terms and gives real-world examples.
Chapter 5 provides an overview of some of the theoretical ideas and scientific traditions behind the various ways of influencing people’s behaviour. We use this overview to go into more detail on what we now know about behavioural change. In this chapter, we dive into
the ocean of scientific theories on behavioural change and identify different types of behavioural models. We take you on a journey from Aristotle’s virtue theory to Seligman’s positive psychology; from Smith’s ‘invisible hand’ to Thaler’s nudges; and from Kant’s deontology to Cialdini’s theory of influence. As well as exploring these three main schools of thought, in this chapter we also identify procedural and mixed models.
This publication was produced in association with experts from finance, government and science. Insights gained from the literature reinforce and enhance the results of the Consumer Choices Action Plan. We believe this blend of practical knowledge and scientific
expertise represents a valuable contribution to the work of encouraging the people of the Netherlands to make good financial choices.
... For example, the reader can refer to the work of Tetlock 71-73 or Snyder 6 . Tetlock's work deals with decision-making and the capacity of lay people to sometimes make better decisions than experts in a field 72 . Tetlock argues that people do not only argue or think in terms of a static or personal belief system or framework. ...
Why would anybody be interested in theories of any sort? I hear a saying many times that goes in the following direction: do you think you are going to fix the world by learning theories? This simple question puzzles anybody interested in learning anything at all because it targets and challenges our decision making. I recall I got this question many times when studying human rights law. It took me many years to realize a simple answer. I worked it out by realizing that, under closer inspection, such questions are as ideological as the ideologies they attempt to debunk. By learning “theories” we are incorporating something in, or rather altering, our understanding of the world and the self. This world and the image of the self are what sustains our capacity to make decisions. Decision making is also the main target of this text.
All being considered, the “world” has already “fixed”, if by this we mean manipulated, those asking the above type of question. They have been manipulated because they have “bought” a theory that claims that the only use of theories is changing the world. My simple answer goes as follows: forget about fixing the world and focus on the world not fooling you. While there is some cynicism in this answer, a more general one is discussed in this text. In short, the intention is to discuss discourse theory, ideologies, psychology, and sociology as the foundation to understand manipulation theory.
The main aim of this text is to link 20th and 21st century thinking from psychology, sociology and philosophy to this common but sometimes misused, or overused, terminology. In fact, a main act of social manipulation currently consists in introducing sophisticated terminology into circulating discourses that is amplified and extended by mass media and dominating ideologies. This is the case to the extent that we end up making critical decisions in everyday life by invoking such vague concepts, i.e., I must leave my job because the atmosphere is toxic, or I must doubt my identity because I have been told I have been manipulated.
Rather than trusting the current circulating lexicon, here, theories of discourse, relevant concepts such as ideology, and other terms borrowed from psychology and sociology are presented and discussed as the foundation of manipulation theory.
The interpretations of several authors including Fromm, Derrida, Tetlock, Snyder, Maslow, Nietzsche, Foucault and Chomsky are discussed on the basis of the broader debate involving structuralism, post-structuralism, social constructionism, theories of ideology and current theories in psychology and sociology including the reality negotiation process.
The text is presented in four chapters. The first chapter introduces the topic. The second and third chapters focus on the different theories of discourse. The fourth and last chapter links everyday examples employed by manipulators to the broader problem of individual and social manipulation.
... Scenario planning is therefore seen as a method of corporate planning, or as a policy tool to be used in conjunction with decision impact simulations (Barma et al., 2015). Scenario planning is the art of juxtaposing current trends in unexpected and anticipated combinations to formulate a surprising yet plausible future (Asal, 2005;Barma, 2016;Tetlock, 2005). ...
This article addresses the issue of the role and importance of the young precariat for the functioning of the democratic system. Based on scenario planning, it presents three possible directions for the development of democracy in the context of meeting the needs of the young precariat. The first scenario assumes a continuation of the measures applied so far by democratic governments towards young precarious people related to social policy and the low representation of the young generation in politics. The second involves a move away from democracy towards non-democratic systems, where the needs of precarious people are irrelevant. The third assumes a new approach among democratic governments to the needs of young precarious workers and the shaping of new social policies, as well as the creation of incentives for young precarious workers to be more widely involved in these policies. The empirical context for these considerations is an attempt to determine the possibility of the occurrence of each of these scenarios in Polish conditions, based on the results of qualitative studies conducted via asynchronous interviews with representatives of the young Polish precariat. The research relates to Poland and takes into account the characteristics of the Polish precariat. The article uses a mixed research methodology, combining different methods for solving research problems, including collecting, analyzing, interpreting, and presenting quantitative and qualitative data.
... Other studies suggest analysts may be unique in the sense that they already think relatively well. For example, a plethora of studies suggest humans are often overconfident in their judgments, 87 even in geopolitical domains 88 ; in contrast, studies of Canadian intelligence analysts revealed good calibration, albeit with a significant degree of underconfidence in their judgments. 89 Findings like these may suggest that analysts are relatively good at reasoning, perhaps because they make judgments in teams or under conditions of greater risk, stringency and accountability. ...
... In addition, there are epistemic worries, that is, concerns about whether experts actually contribute to ensuring more rational and informed policy-making. Just like non-experts, experts can make cognitive mistakes (Kahneman 2012;Tetlock 2005), and be biased by ideology or by their disciplinary culture and socialization. Expert involvement in policy-making is thus no guarantee against-and may even increase the likelihood of-skewed and poor decisions (e.g. ...
... One could expect experts, based on their knowledge, to be better at foreseeing the future and to take swift measures. Yet, as famously started by Tetlock (2005), experts are in general bad at making predictions. Evidence-based decision-making, which is most often advocated by experts, comes with some shortcomings in situations characterized by uncertainty and the need for speedy decisions. ...
The academic discussion on the role of experts in crisis decision making subject is scattered and diverged. The debate has foremost been connected to the discussion of centralized versus decentralized responses. Inspired by the notion of contingency and Karl Weick, this article explores the role of the Swedish Public Health Agency in coping with the Covid‐19 pandemic. One of the key findings from the study is that even if experts might have a better technical understanding, at the same time, they risk getting stuck in identity concerns and previous experiences which result in rigid responses. The study also underlines that it is risky for decision‐makers to rely upon one single body of expertise at times when uncertainty is high and there is a lack of solid evidence and knowledge, given the likelihood of fixation and rigidity. Rather, policy makers should encourage a deliberative debate, involving a diversity of experts and expertise.
Dieser Beitrag zeigt, wie soziologische Zeitdiagnosen einen Pessimismus an den Tag legen, der durch empirische Daten nicht gedeckt ist. Historische Veränderungen von Gewalt, Armut, Umweltverschmutzung und Sozialkapital widersprechen den pessimistischen Diagnosen von Marx, Adorno, Bauman, Beck und Rosa, sowie kontemporären Diagnosen einer Abstiegsgesellschaft oder gar eines Untergangs des Kapitalismus. Des Weiteren wird erklärt, wie ein pessimistischer Überbietungswettbewerb, konzeptinduzierter Prävalenzwandel und der Drang, die Gesellschaft zu verbessern, in Verbindung mit radikalkonstruktivistischen Sichtweisen zu einem übermäßig negativen Gesellschaftsbild der Soziologie geführt haben. Der Text schließt mit vier Vorschlägen, wie die Soziologie ein Gesellschaftsbild entwickeln kann, das besser an die Realität angepasst ist.
This article shows how sociological diagnoses of society are beset by a pessimism that is incompatible with empirical data. Historical changes in violence, poverty, pollution and social capital contradict widespread pessimistic diagnoses such as of Marx, Adorno, Bauman, Beck, and Rosa, as well as contemporary views of widespread downward mobility or even an end of capitalism. The paper explains how such unwarranted pessimism can be explained by sociologists competing for the most negative views on society, fueled by a prevalence-induced concept change, as well as an urge to improve society and radical constructivist perspectives, which led sociology towards a view of society that is more negative than society itself. The article concludes with four suggestions on how sociology can develop a more realistic view of society.
Epistemic paternalism involves interfering with the inquiry of others, without their consent, for their own epistemic good. Recently, such paternalism has been discussed as a method of getting the public to have more accurate views on important policy matters. Here, I discuss a novel problem for such paternalism— epistemic spillovers . The problem arises because what matters for rational belief is one’s total evidence, and further, individual pieces of evidence can have complex interactions. Because of this, justified epistemic paternalism requires the would-be paternalist to be in an unusually strong epistemic position, one that most would-be paternalists are unlikely to meet.
This chapter argues that effective leadership in today's complex, rapidly changing world hinges on cultivating a "questioning mind." Moving beyond traditional leadership models, it explores the deep psychological underpinnings of inquiry as a core competency, asserting that the ability to challenge assumptions, question narratives, and rigorously analyze information is a strategic imperative. Drawing on cognitive psychology, evolutionary biology, and historical analysis, the work demonstrates how questioning drives learning, fuels innovation, and enables adaptation, while also examining how cognitive biases can hinder effective inquiry. It highlights how societies that fostered intellectual curiosity and challenged the status quo consistently outperformed those clinging to tradition. The rise of artificial intelligence (AI) and quantum computing is presented as a fundamental reshaping of the questioning landscape, offering unprecedented opportunities for leaders to augment their cognitive capabilities. Howeve
This paper explores the imperative of multi-layered leadership in navigating environments characterized by "hostile complexity" – a confluence of interconnectedness, rapid change, unpredictability, and often, deliberate deception. Drawing upon cognitive science, social psychology, game theory, and strategic studies, it argues that effective leadership in the 21st century demands a sophisticated behavioral architecture, one that integrates the principles of art, particularly narrative and the shaping of perception, alongside strategic design. By cultivating the ability to operate on multiple levels simultaneously, to understand the dynamic interplay between individual, organizational, and societal factors, and to harness the power of narrative, leaders can foster resilience, adaptability, and ethical action in an increasingly complex world. This work illuminates the psychological foundations of multi-layered existence, its practical applications, its strategic deployment, and the inherent ethical dilemmas, ultimately presenting a framework for leadership that transcends traditional models and embraces the multifaceted nature of human experience.
When an organization has a learning orientation, operators are attracted to problems-a new learning experience. When the primary orientation is performance-based, operators become inclined to avoid problems, thus protecting themselves from criticism and failure. The combination of learning and performance orientation drives the organization to increase operators' capabilities. They become better equipped to enter liminal states, if not create structure in such circumstances. The result is a sense of agency in operators and enhanced organization performance. Introduction:
Cognitive biases play a crucial role in shaping corporate financial decisions, particularly in the high-stakes arenas of mergers, acquisitions, and investments. This paper investigates the various cognitive biases that influence the decision-making processes of executives and boards, often leading to suboptimal financial outcomes. By conducting a comprehensive review of existing literature, performing detailed case studies, and employing robust data analysis techniques, this research aims to illuminate the intricate mechanisms through which cognitive biases operate within the realm of corporate finance. The study identifies key biases such as overconfidence, anchoring, confirmation bias, and loss aversion, examining how these psychological phenomena can distort judgment and lead to flawed decision-making. For instance, overconfidence may drive executives to pursue aggressive growth strategies without adequately assessing risks, while confirmation bias can result in the neglect of critical information that contradicts existing beliefs. Furthermore, the paper explores the broader implications of these biases on organizational performance and financial health, highlighting the potential for significant economic repercussions when biases go unchecked. By synthesizing insights from behavioral finance and organizational psychology, this research not only elucidates the impact of cognitive biases but also offers practical recommendations for mitigating their effects. Strategies such as fostering diverse decision-making teams, implementing structured decision-making frameworks, and enhancing awareness through training programs are proposed to help organizations navigate the complexities of corporate finance more effectively. Ultimately, this paper contributes to the understanding of how cognitive biases can shape financial decision-making in corporate settings and underscores the importance of adopting a more analytical and evidence-based approach to enhance decision quality and organizational outcomes.
As psychology evolves, the phenomenon of concept overlap becomes more pronounced, increasing participant burden and complicate data interpretation. This study introduces an Embedding-based Semantic Analysis Approach (ESAA) for detecting redundancy in psychological concepts, which are operationalized through their respective scales, using natural language processing techniques. ESAA utilizes OpenAI’s GPT-3 large model to generate high-dimensional semantic embeddings of scale items and applies hierarchical clustering to group semantically similar items, uncovering potential redundancy. In three preliminary experiments, ESAA was tested on well-known psychological scales, such as Conscientiousness, Gratitude, and Grit. The experiments assessed ESAA’s ability to (1) converge semantically similar items, (2) discriminate semantically distinct items, and (3) identify overlapping scales measuring concepts known for redundancy. Additionally, comparative analyses were conducted to assess ESAA's robustness and incremental validity against the most advanced chat bots based on GPT-4. The results demonstrated that ESAA consistently produced stable outcomes and surpassed all evaluated chatbots in performance. As a novel, objective approach for analyzing relationships between concepts operationalized as scales, ESAA has potential to facilitate future research on theory refinement and scale optimization.
This chapter elaborates on how institutionalized decision-making strategies impede the development of ethical reasoning. Three rationalistic strategies are highlighted: strict rule-following, evidence-based practice, and economic calculations. A fundamental argument is developed against relying exclusively on instrumental rationality, based on insights from incrementalism, institutional theory, and cognitive psychology. These traditions clarify how decision-making largely deviates from instrumental rationality since real-life decisions are largely affected by experiences, heuristics, and social forces. This means good prospects for ethical reasoning, but it is also argued that rationalistic methods should not be dismissed since they can be useful and well-integrated into frameworks of ethical reasoning.
In 2022, the world experienced the deadliest year of armed conflict since the 1994 Rwandan genocide. Much of the intensity and frequency of recent conflicts has drawn more attention to failures in forecasting—that is, a failure to anticipate conflicts. Such capabilities have the potential to greatly reduce the time, motivation, and opportunities peacemakers have to intervene through mediation or peacekeeping operations. In recent years, the growth in the volume of open-source data coupled with the wide-scale advancements in machine learning suggests that it may be possible for computational methods to help the international community forecast intrastate conflict more accurately, and in doing so reduce the rise of conflict. In this commentary, we argue for the promise of conflict forecasting under several technical and policy conditions. From a technical perspective, the success of this work depends on improvements in the quality of conflict-related data and an increased focus on model interpretability. In terms of policy implementation, we suggest that this technology should be used primarily to aid policy analysis heuristically and help identify unexpected conflicts.
This paper argues that the well-established fact of political irrationality imposes substantial constraints on how governments may combat the threat of political misinformation. Though attempts at regulating misinformation are becoming increasingly popular, both among policymakers and theorists, I intend to show that, for a wide range of anti-misinformation interventions (collectively termed “debunking” and “source labeling”), these attempts ought to be abandoned. My argument relies primarily on the fact that most people process politically-relevant information in biased and motivated ways. Since debunking or factual correction of politically relevant misinformation (as well as source labeling) themselves consist of providing politically-relevant information, they are also very likely to be processed in irrational ways. This makes it extremely difficult to effectively correct people’s political beliefs and political information processing. Since governments should not pursue policies likely to be futile, they should refrain from mandating such interventions. My conclusion is of relevance to considerable literature in digital ethics of misinformation. It shows that many celebrated works in the field ignore political irrationality and fail to consider its implications.
The subject of this paper is how the epistemic limitations of individuals and their biases in reasoning affect collective decisions and in particular the functioning of democracies. In fact, while the cognitive sciences have largely shown how the imperfections of human rationality shape individual decisions and behaviors, the implications of these imperfections for collective choice and mass behaviors have not yet been studied in such detail. In particular, the link between these imperfections and the emergence of contemporary populisms has not yet been thoroughly explored. This is done in this paper by considering both fundamental dimensions of the political space: the cultural-identitarian and the socio-economic one. As has been noted, reflections on these points induce to revise the picture of democracy as a regime producing collective decisions that come out from the interaction of independent individuals well aware of their values and interests, and rationally (in the sense of rational choice theory) pursuing them. This leads to a certain skepticism towards the idealization of democracy as human rationality in pursuit of the common good, which serves to provide cover for those who profit from the distortions and biases in the policy-making processes of actual democracies. A natural conclusion of the paper is that contemporary democracies are quite vulnerable in the face of populist leaders and parties, that are systematically trying to exploit to their advantage people’s imperfect rationality (using “easy arguments”, emotions, stereotypes…).
Sporda Liderliğin Çerçevesi Pervin BİLİR Liderlik Nedir? Pervin BİLİR Tolga ÇELİK Liderliğin Özellik Teorileri Pervin BİLİR Seyhan BİLİR GÜLER Liderlik Stilleri Selahattin AKPINAR Öznur AKPINAR Liderlik, Takımlar ve Etkileşim: Ya Grubun İçindesiniz ya da Dışında Melih N. SALMAN Yıkıcı ve Toksik Liderlik Ergun YURDADÖN Spor Yönetiminde Kolaylaştırıcı Liderlik Pervin BİLİR Spor Yönetiminde İletişim Canan SAYIN TEMUR Saadet GÖNEN Stratejik Planlama ile Stratejik Liderlik Pervin BİLİR Nurşen ŞAHİN Kurumsal Değişime Liderlik Etmek Enver DÖŞYILMAZ Spor Liderliğinde Çeşitlilik Özge AYDIN Antrenör Tayfun ŞİRİN Ozan Ç. SARIKAYA Antrenörlükte Liderlik Modelleri Yeliz ERATLI ŞİRİN Antrenörlükte Örgütsel Liderlik Yaklaşımları Nurşen ŞAHİN Antrenörlükte Liderliğin Ölçülmesi Eren ULUÖZ Gamze ÜNALDI Empati ile Sürdürülebilir Performans Zeynep Filiz DİNÇ Cem Yoksuler YILMAZ Gelecek: Spor Liderliğindeki Yönelimler ve Zorluklar Erdem EROĞLU Sultan EROĞLU
Die Szenarioanalyse ist eine Methode zur Reduzierung von Unsicherheiten in Entscheidungsprozessen. Sie zählt zu den Strukturierten Analysetechniken der Intelligence Analysis sowie zu den Methoden der strategischen Vorausschau. Szenarioanalysen stellen eine wichtige Komponente in den Intelligence-Fähigkeiten von Konzernsicherheiten dar. Das vorliegende Dokument führt in diese Analysemethode ein und beschreibt ihre praktische Anwendung, um zu einer Professionalisierung der Intelligence-Tätigkeiten in Unternehmen beizutragen. Es richtet sich an Analystinnen und Analysten, die in entsprechenden Funktionen tätig sind.
Our social theories are often unresponsive to logical and empirical challenges. Earlier research has demonstrated that the process of creating causal explanations or general scenarios to explain observed events contributes to such unwarranted theory perseverance. An analysis of possible cognitive mechanisms underlying theory perseverance suggests that explanation processes might be used in “debiasing” techniques. It is predicted that theory perseverance would be reduced by inducing subjects to create causal explanations for both possible relationships between two social variables. College student subjects were given two case studies suggestive of either a positive or a negative relationship between preference for risk and success as a firefighter. Some subjects had only to consider the relationship suggested by their case history data. Others were induced to consider both possible relationships. Subjects forced to consider both relationships showed significantly less theory perseverance, supporting the analysis of the cognitive mechanisms underlying theory perseverance and suggesting possible applications in real-world contexts. The probable locus of the effects of the debiasing techniques and possible boundary conditions are discussed.
Examined the likelihood that only a portion of the relevant information will be accessed when people consider the contingency between 2 variables, thus resulting in erroneous judgments. Inasmuch as people estimate contingency largely on the basis of frequency rather than probability information, severe imbalance in the marginal frequencies of either variable is a condition likely to lead to such errors. In Exp I, 243 undergraduates were exposed to the evidence in a 2 × 2 contingency table, where the 2 marginal frequencies for each variable differed in a ratio of 2:1. Ss were then cued to recall information corresponding to either a row or column of the table and judged the contingency between the 2 factors. It was found that the particular sample that was recalled significantly influenced the subsequent contingency judgment. In Exp II, 112 undergraduates read a mock news article in which 2 populations of differing frequencies were described. Analogous to Exp I, different forms of the article implicitly cued different samples of information. Later judgments about the intellectual ability of the 2 populations were influenced by the sample of information that was cued. (16 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Although clinicopathologic conferences (CPCs) have been valued for teaching differential diagnosis, their instructional value may be compromised by hindsight bias. This bias occurs when those who know the actual diagnosis overestimate the likelihood that they would have been able to predict the correct diagnosis had they been asked to do so beforehand. Evidence for the presence of the hindsight bias was sought among 160 physicians and trainees attending four CPCs. Before the correct diagnosis was announced, half of the conference audience estimated the probability that each of five possible diagnoses was correct (foresight subjects). After the correct diagnosis was announced the remaining (hindsight) subjects estimated the probability they would have assigned to each of the five possible diagnoses had they been making the initial differential diagnosis. Only 30% of the foresight subjects ranked the correct diagnosis as first, versus 50% of the hindsight subjects (p less than 0.02). Although less experienced physicians consistently demonstrated the hindsight bias, more experienced physicians succumbed only on easier cases.
The United States and China have emerged from the cold war with a reciprocal fear of each other's nuclear intentions. "In this mutual perception is a classic security dilemma: defensive military programs undertaken by both sides are viewed by their counterpart as offensive and threatening. In the United States, the Cox committee's worst-case analysis of China's espionage has enhanced this perception".
China's role as a great power is at best uncertain and confused. . . . It has still not found a way to reassure neighbors or to free itself from dependence on the security and stability provided by the United States - and the resentment that creates.
"China is not scrambling to dismantle socialism; it is scrambling to regulate a market system that it has-for better or worse, intentionally or inadvertently-adopted thoroughly."
Predictions of psychiatric patients' imminent dangerousness by 30 experienced psychologists and psychiatrists were studied. These clinical judges reviewed summary descriptions of 40 male patients newly admitted to an acute-care psychiatric unit and predicted whether each patient would engage in an assault during his first 7 days on the unit. Statistically significant but low levels of reliability were found among individual judges' predictions, but a strong relationship was found between composite judgments by psychologists and psychiatrists. Judges attained a low accuracy rate in predicting patients' violence. Cue-utilization analyses suggested that on the whole both actual violence and judges' forecasts were linearly predictable. Variables in the model of clinical forecasts of violence included patients' assaultiveness prior to admission as well as rated hostility and depressive mood, whereas empirical correlates of violence in this patient sample included the absence of emotional withdrawal and the presence of hallucinatory behavior. The apparent extent to which judges used valid predictors of violence was related to their accuracy. Implications for the prediction of imminent violence in acute psychiatric care settings are discussed.
Many scholars view integratively complex reasoning as either cognitively or morally superior to integratively simple reasoning. This value judgment is, however, too simple to capture the complex, subtle, and even paradoxical linkages between integrative complexity and "good judgment" in historical context. Our case studies add to the growing literature on this topic by assessing the integrative and cognitive complexity of policy statements that Winston Churchill and his political adversaries made during two key foreign policy debates of the 1930s-the appeasement of Nazi Germany (where contemporary opinion overwhelmingly favors Churchill) and the granting of self-government to India (where contemporary opinion overwhelmingly favors Churchill's opponents). In both private and public, Churchill expressed less integratively complex but more cognitively complex opinions than did his opponents on both Nazi Germany and self-government for India. The results illustrate (a) impressive consistency in Churchill's integrative but not cognitive complexity in both private and public communications over time and issue domains, and (b) the dependence of normative judgments of styles of thinking on speculative counterfactual reconstructions of history and on moral-political values. We close by arguing that, although integrative complexity can be maladaptive in specific decision-making settings, it can still be highly adaptive at the meta-decision-making level where leaders "decide how to decide." Good judgment requires the ability to shift from simple to complex modes of processing in timely and appropriate ways.
I will start with a Jewish proverb and then will come to a Jewish story. First the proverb: an insincere peace is better than a sincere war.
We tested the hypothesis that the importance of the topic of research can make people overlook methodological flaws in the research. Two samples of scientists, faculty at a major medical school and research psychologists, evaluated the methodological rigor and publishability of brief descriptions of flawed research studies. Two versions of each study were identical except for the importance of the topic (e.g., heart disease vs. heartburn). When the topic was important, scientists in both samples were significantly more likely to overlook the methodological flaws, and significantly more lenient in their recommendations that the studies be published.
Scalar and vector partitions of the probability score (PS) in the two-state situation are described and compared. These partitions, which are based upon expressions for the PS in which probability forecasts are considered to be scalars and vectors, respectively, provide similar, but not equivalent (i.e., linearly related), measures of the reliability and resolution of the forecasts. Specifically, the reliability (resolution) of the forecasts according to the scalar partition is, in general, greater (less) than their reliability (resolution) according to the vector partition. A sample collection of forecasts is used to illustrate the differences between these partitions.
Several questions related to the use of scalar and vector partitions of the PS in the two-state situation are discussed, including the interpretation of the results of previous forecast evaluation studies and the relative merits of these partitions. The discussions indicate that the partition most often used in such studies has been a special “scalar” partition, a partition which is equivalent to the vector partition in the two-state situation, and that the vector partition is more appropriate than the scalar partition.
Verbal phrases denoting uncertainty are usually held to be more vague than numerical probability statements. They are, however, directionally more precise, in the sense that they are either positive, suggesting the occurrence of a target outcome, or negative, drawing attention to its non-occurrence. A numerical probability will, in contrast, sometimes be perceived as positive and sometimes as negative. When asked to complete sentences such as ‘The operation has a 30% chance of success, because’ some people will give reasons for success (‘the doctors are expert surgeons’), whereas others will give reasons for failure (‘it is a difficult operation’). It is shown in two experiments that positive reasons are given more often than negative ones, even for p values below 0.5, especially when the probability is higher than expected, and the target outcome is non-normal, undesirable, and phrased as a negation. We conclude that the directionality of numerical probabilities (as opposed to verbal phrases) is context-dependent, but biased towards a positive interpretation. Copyright © 2000 John Wiley & Sons, Ltd.
Most of our knowledge in psychology and allied social sciences is based on observing consequences and seeking antecedents. The statistical analysis of such retrospective knowledge thus involves conditioning on consequences. This article demonstrates that given the common conditions of investigating "unusual" consequences, the degree of statistical contingency between a single consequence and a single antecedent is greater when conditioning on the consequence than when conditioning on the antecedent--which is, of course, necessary for prediction. Moreover, this asymmetry is exacerbated when the investigator is free to search for antecedents in a situation involving multiple potential antecedents. This asymmetry is exacerbated to an even greater extent when the investigator relies on memory rather than recorded observations in this search. Thus, if we identify the degree of statistical contingency in prediction with the degree we find in retrospection, we seriously overestimate. Our subsequent disappointment in our deficient predictive abilities can easily lead to rejecting variables and analyses that are in fact predictive in favor of those of unknown validity.
Some results of a nationwide survey of National Weather Service
forecasters with regard to probability forecasting in general and
precipitation probability forecasting in particular are summarized.
Specifically, the questionnaire which was used in the survey, the
participants in the survey (i.e., the forecasters). and the nature of
the results are briefly described, and some recommendations based upon
these results are presented.
When individuals learn the outcome of an event or the correct answer to a question, they overestimate its prior predictability: that is, they tend to believe they “knew it all along.” Cognitive and motivational interpretations of hindsight bias are briefly reviewed and a study designed to test the motivational interpretation is reported. Specifically, it was hypothesized that individual differences in the strengths of two motives, a need for predictability and a self-presentation motive, should be positively related to individual differences in the magnitude of hindsight bias. Sixty-eight subjects completed a Dogmatism Scale and an Intolerance for Ambiguity Scale (the predictability motive) and the Marlowe-Crowne Social Desirability Scale (the self-presentation motive) before participating in a standard hindsight-bias paradigm. Measures of both motives, as well as a self-reported ego-involvement measure, were positively associated with the amount of hindsight bias exhibited. Implications of this result for interpretations of hindsight and other conceptually related phenomena are discussed.
In an attempt to regulate disappointments people may sometimes change their perceptions of the events leading to an undesirable outcome so that in retrospect this outcome seems almost inevitable. This retroactive pessimism effect was demonstrated in three studies. In the first, sports fans rated the likelihood of success for their team and its opponent before and after an important soccer match. Evidence for significant pre- and post-game probability shifts was found for the fans of the defeated team but not for the supporters of the winning opponent. In the second and the third experiments participants responded to a scenario depicting a loss of stipend that was either large or small in value. Participants were expected to show more evidence of retroactive pessimism with greater disappointment. Indeed, estimates of the probability of a more favorable counterfactual outcome were sensitive to the magnitude of the loss with lower estimates of the probability that things could have turned out better in the large stipend condition. The effect was attenuated, however, when the loss was not personal but rather that of a friend (Experiment 2), or when the disappointment was mitigated (Experiment 3). Copyright © 2002 John Wiley & Sons, Ltd.
Proper scoring rules are over evaluation measures that reward accurate probabilities Specific rules encountered in the literature and used in practice are invariably symmetric in the sense that the expected score for a perfectly-calibrated probability assessor (or model generating probabilities) is minimized at a probability of one-half. A family of asymmetric scoring rules that provide better measures of the degree of skill inherent in the probabilities and render scores that are more comparable in different situations is developed here. One member of this family, a quadratic asymmetric rule, is applied to evaluate an extensive set of precipitation probability forecasts from the U.S. National Weather Service. Connections to previous characterizations of proper scoring rules are investigated, and some relevant issues pertaining to the design of specific asymmetric rules for particular inferential and decision-making problems are discussed briefly.
When six equally qualified candidates compete for the same position, p = 1/6 for each. People seem to accept this principle more readily for numerical than for verbal probabilities. Equal chances with three to six alternatives are often verbally described in a positive vein as "entirely possible" or "a good chance" and rarely negatively as "doubtful" or "improbable." This equiprobability effect of verbal probabilities is demonstrated in five studies describing job applicants, lottery players, competing athletes, and examination candidates. The equiprobability effect is consistent with a causal (propensity) view of probabilities, where chances are believed to reflect the relative strength of facilitating and preventive causes. If important conditions in support of the target outcome are present (the candidate is qualified for the position), and there is little to prevent it from occurring (no stronger candidates), chances appear to be good. In the presence of obstacles (one stronger candidate), or in the absence of facilitating conditions (the candidate is poorly qualified), chances appear to be poor, even when numerical p values remain constant. The findings indicate that verbal and numerical probability estimates can reflect different intuitions. Copyright 2001 Academic Press.