Article
To read the full-text of this research, you can request a copy directly from the authors.

Abstract

People often express political opinions in starkly dichotomous terms, such as “Trump will either trigger a ruinous trade war or save U.S. factory workers from disaster.” This mode of communication promotes polarization into ideological in-groups and out-groups. We explore the power of an emerging methodology, forecasting tournaments, to encourage clashing factions to do something odd: to translate their beliefs into nuanced probability judgments and track accuracy over time and questions. In theory, tournaments advance the goals of “deliberative democracy” by incentivizing people to be flexible belief updaters whose views converge in response to facts, thus depolarizing unnecessarily polarized debates. We examine the hypothesis that, in the process of thinking critically about their beliefs, tournament participants become more moderate in their own political attitudes and those they attribute to the other side. We view tournaments as belonging to a broader class of psychological inductions that increase epistemic humility and that include asking people to explore alternative perspectives, probing the depth of their cause-effect understanding and holding them accountable to audiences with difficult-to-guess views.

No full-text available

Request Full-text Paper PDF

To read the full-text of this research,
you can request a copy directly from the authors.

... While uncertainty and ambiguity are generally unfavourable in decision making especially if stakes are high, actively embracing the limits of one's knowledge and forecasting tools is an important factor in making rational decisions [5]. An obvious advantage of such "intellectual" [5] or "epistemic" [6] humility is risk and contingency management: being aware of perhaps being wrong helps mitigate the effects of actual errors (by being attentive and prepared). A less well-recognized benefit is increased freedom in strategy selection and combination: being aware that "all models are wrong" (even the sophisticated ones currently considered as reference-standard) opens the competition for a greater variety of epistemic approaches [6]. ...
... An obvious advantage of such "intellectual" [5] or "epistemic" [6] humility is risk and contingency management: being aware of perhaps being wrong helps mitigate the effects of actual errors (by being attentive and prepared). A less well-recognized benefit is increased freedom in strategy selection and combination: being aware that "all models are wrong" (even the sophisticated ones currently considered as reference-standard) opens the competition for a greater variety of epistemic approaches [6]. ...
Article
Full-text available
Background The main task of applied sport science is to inform decision-making in sports practice, that is, enabling practitioners to compare the expectable outcomes of different options (e.g. training programs). Main Body The “evidence” provided may range from group averages to multivariable prediction models. By contrast, many decisions are still largely based on the subjective, experience-based judgement of athletes and coaches. While for the research scientist this may seem “unscientific” and even “irrational”, it is important to realize the different perspectives: science values novelty, universal validity, methodological rigor, and contributions towards long-term advancement. Practitioners are judged by the performance outcomes of contemporary, specific athletes. This makes out-of-sample predictive accuracy and robustness decisive requirements for useful decision support. At this point, researchers must concede that under the framework conditions of sport (small samples, multifactorial outcomes etc.) near certainty is unattainable, even with cutting-edge methods that might theoretically enable near-perfect accuracy. Rather, the sport ecosystem favors simpler rules, learning by experience, human judgement, and integration across different sources of knowledge. In other words, the focus of practitioners on experience and human judgement, complemented—but not superseded—by scientific evidence is probably street-smart after all. A major downside of this human-driven approach is the lack of science-grade evaluation and transparency. However, methods are available to merge the assets of data- and human-driven strategies and mitigate biases. Short Conclusion This work presents the challenges of learning, forecasting and decision-making in sport as well as specific opportunities for turning the prevailing “evidence vs. eminence” contrast into a synergy.
... While uncertainty and ambiguity are generally unfavourable in decision making especially if stakes are high, actively embracing the limits of one's knowledge and forecasting tools is an important factor in making rational decisions [5]. An obvious advantage of such "intellectual" [5] or "epistemic" [6] humility is risk and contingency management: Being aware of perhaps being wrong helps mitigate the effects of actual errors (by being attentive and prepared). A less wellrecognized benefit is increased freedom in strategy selection and combination: Being aware that "all models are wrong" (even the sophisticated ones currently considered as reference-standard) opens the competition for a greater variety of epistemic approaches [6]. ...
... An obvious advantage of such "intellectual" [5] or "epistemic" [6] humility is risk and contingency management: Being aware of perhaps being wrong helps mitigate the effects of actual errors (by being attentive and prepared). A less wellrecognized benefit is increased freedom in strategy selection and combination: Being aware that "all models are wrong" (even the sophisticated ones currently considered as reference-standard) opens the competition for a greater variety of epistemic approaches [6]. ...
... Consequently, interest in cultivating intellectual humility has come from multiple research areas and subfields in psychology, including social-personality, cognitive, clinical, educational, and leadership and organizational behaviour [7][8][9][10][11] . Cumulatively, research suggests that intellectual humility can decrease polarization, extremism and susceptibility to conspiracy beliefs, increase learning and discovery, and foster scientific credibility [12][13][14][15] . ...
... Preference for these competing accounts of intellectual humility varies across subfields of psychology, linked to methodological preferences and historical emphasis on social and contextual factors. Cognitive psychologists tend to favour metacognitive accounts that emphasize how people think about evidence, knowledge and beliefs, without much attention to social contexts 13 . Conversely, developmental, educational and clinical psychologists tend to favour a multidimensional account that considers how real world, cognitive, behavioural and interpersonal factors come together to form intellectual humility [38][39][40] . ...
Article
Full-text available
In a time of societal acrimony, psychological scientists have turned to a possible antidote — intellectual humility. Interest in intellectual humility comes from diverse research areas, including researchers studying leadership and organizational behaviour, personality science, positive psychology, judgement and decision-making, education, culture, and intergroup and interpersonal relationships. In this Review, we synthesize empirical approaches to the study of intellectual humility. We critically examine diverse approaches to defining and measuring intellectual humility and identify the common element: a meta-cognitive ability to recognize the limitations of one’s beliefs and knowledge. After reviewing the validity of different measurement approaches, we highlight factors that influence intellectual humility, from relationship security to social coordination. Furthermore, we review empirical evidence concerning the benefits and drawbacks of intellectual humility for personal decision-making, interpersonal relationships, scientific enterprise and society writ large. We conclude by outlining initial attempts to boost intellectual humility, foreshadowing possible scalable interventions that can turn intellectual humility into a core interpersonal, institutional and cultural value. Intellectual humility involves acknowledging the limitations of one’s knowledge and that one’s beliefs might be incorrect. In this Review, Porter and colleagues synthesize concepts of intellectual humility across fields and describe the complex interplay between intellectual humility and related individual and societal factors.
... An openness to changing one's mind has been linked to superior performance by forecasting groups 153 . For instance, Mellers et al. 154 showed that structured critical thinking in a forecasting tournament -translating beliefs into nuanced proba bility judgements and tracking accuracy over time and questions -can lead to depolarization of opinions by increasing flexible belief updating and probabilistic ver sus dichotomous framing of issues. By increasing crit ical thinking and openness to alternative perspectives, tournament group members became more moderate and less polarized in their opinions. ...
... Research to date has largely validated the notion of a collective intelligence beyond the wisdom of crowds, by showing its robustness across species and demonstrating numerous instances where collectives are capable of more accuracy than individuals 3,6,24,29,31,38,104,132,161,[167][168][169][170] . Recent research on group forecasting has also demonstrated ways to improve collective intelligence through train ing and expertisebased group composition 48,72,73,153,154 . As one underlying principle, we have argued that optin or optout behavioural mechanisms can promote col lective intelligence further in both consensus 72,73,[83][84][85] and combined 96,97,99,101,108 decisionmaking through capitalizing on individual heterogeneity in knowledge, skills and ability. ...
Article
Full-text available
In humans and other gregarious animals, collective decision-making is a robust behavioural feature of groups. Pooling individual information is also fundamental for modern societies, in which digital technologies have exponentially increased the interdependence of individual group members. In this Review, we selectively discuss the recent human and animal literature, focusing on cognitive and behavioural mechanisms that can yield collective intelligence beyond the wisdom of crowds. We distinguish between two group decision-making situations: consensus decision-making, in which a group consensus is required, and combined decision-making, in which a group consensus is not required. We show that in both group decision-making situations, cognitive and behavioural algorithms that capitalize on individual heterogeneity are the key for collective intelligence to emerge. These algorithms include accuracy or expertise-weighted aggregation of individual inputs and implicit or explicit coordination of cognition and behaviour towards division of labour. These mechanisms can be implemented either as ‘cognitive algebra’, executed mainly within the mind of an individual or by some arbitrating system, or as a dynamic behavioural aggregation through social interaction of individual group members. Finally, we discuss implications for collective decision-making in modern societies characterized by a fluid but auto-correlated flow of information and outline some future directions. Collective intelligence emerges in group decision-making, whether it requires a consensus or not. In this Review, Kameda et al. describe cognitive and behavioural algorithms that capitalize on individual heterogeneity to yield gains in decision-making accuracy beyond the wisdom of crowds. View-only file is available. https://rdcu.be/cL3QB
... extant psychological work stresses that intuitive predictions are usually worse than statistical predictions and, sometimes, worse than chance (Dawes, Faust, and Meehl 1989). People tend to be terrible forecasters notably because they largely "misuse" probabilistic rules when updating their beliefs and test their hypotheses in suboptimal ways (kahneman, Slovic, and Tversky 1982;Mellers et al. 2019 ). Moreover, probability estimates proved to be highly susceptible to base rate neglect, hindsight bias, and overconfidence (see Gilovich, Griffin, and kahneman [2002] for a literature review). ...
... Based on this source of data, Tetlock and his team established that the accuracy of previsions correlates with individual cognitive abilities. Their work stresses, in particular, that the most accurate predictions were made by individuals who scored higher on fluid intelligence and cognitive flexibility, that is, the ability to update one's beliefs and take into considerations various points of views (see Mellers et al. 2015Mellers et al. , 2019. They also stress the importance of benefiting from "an enriched environment" (Ungar et al. 2012;Mellers et al. 2015), highlighting that working in groups (to combine predictions from multiple sources) proved to be systematically associated with more accurate forecasts than working alone. ...
Article
Full-text available
Future migration is central to contemporary politics, but we know little of how citizens and policy-makers perceive and predict migratory trends. I analyze migration forecasting in a representative sample of the population of France, using survey data and administrative records to document differences in the accuracy of forecasting among groups of individuals. The article takes an interdisciplinary approach to future-oriented thinking, conceiving it as a distributed cognitive process, and showing that educational attainment and migratory background shape one’s ability to predict short-term trends. My analysis stresses the importance of accounting for sociodemographic characteristics and social networks in forecasting: I show that social diversity can improve predictions and extend studies based on the Delphi methodology by discussing the relevant expertise to forecast in different realms.
... By tracking performance over time, assigning participants to different conditions, and collecting data about their cognitive profiles, the GJP has produced valuable insights about the sources and limits of human foresight. Among them is the discovery that participation in forecasting tournaments can induce intellectual humility (Mellers et al. 2019). Mellers, Tetlock and Arkes note that "Tournaments put participants in an unusual social world…", a world in which debiasing strategies are naturally implemented and reinforced. ...
Article
Full-text available
While there are many competing accounts and scales of intellectual humility, philosophers and psychologists are generally united in treating it as an epistemically beneficial disposition of individual agents. I call the research guided by this supposition the traditional approach to studying intellectual humility. The traditional approach is entirely understandable in light of recent findings that individual differences in intellectual humility are associated with various deleterious epistemic tendencies. Nonetheless, I argue that its near monopoly has resulted in an underestimation of important limitations of human cognition. In particular, it neglects the fact that intellectual arrogance can be both deeply recalcitrant and significantly beneficial for bounded cognitive agents whose reliance on one another is profound. I propose to integrate these insights into the study of intellectual humility by treating it as a collective virtue that gets manifested in the structure of epistemic environments. More specifically, it is an interactive virtue that harnesses and constrains intellectual arrogance to yield benefits for both individuals and collectives. I claim that this can happen only within humbling environments, such as forecasting tournaments and open institutional science.
... Although this may be true for social work, in fields such as business, economics and politics considerable efforts have been made to find and develop ways of supporting human judgement and decisionmaking. In particular, the American political scientist Philip Tetlock and colleagues have shown that it is possible with a combination of targeted selection, training, group work and aggregation to significantly improve judgemental accuracy (Atanasov et al., 2020;Chang et al., 2016;Friedman et al., 2018;Karger et al., 2022;Mellers et al., 2017Mellers et al., , 2019Moore et al., 2017;Tetlock et al., 2014;Tetlock & Gardner, 2016). Despite this, it is important to acknowledge from the outset the vast differences that exist between business, economic and political judgements, and those made by social workers (Taylor & White, 2006). ...
Article
Full-text available
Child and family social workers routinely make professional judgements involving significant legal and moral questions (e.g. whether a child has been abused) and more 'everyday' issues (e.g. will the child be re-referred again if we close the case now?) Yet the world is capricious, and we rarely know with certainty what is going to happen in future or the likely impact of our different choices. Given the consequences of their judgements and decisions, it is imperative that social workers are provided with the best possible support. This paper reports a proof-of-concept study of a set of interventions to improve the judgemental accuracy of social workers: (i) a survey to identify respondents with above-average existing abilities, (ii) training sessions on cognitive debiasing and (iii) structured group working and (iv) three methods for aggregating individual judgements. Findings indicate that it is possible to measure the accuracy of social work judgements in relation to case-study materials and retrospective questions, while the feedback about the training was largely positive. Any future studies should aim to recruit a more diverse set of respondents, test judgemental accuracy in relation to prospective judgements and explore what types of questions would be most helpful for real-world decision-making.
... The results of the research program show that those participants who base their judgments on statistics, psychology, and their forecasting training along with other levels of interaction between individual forecasters, consistently produced the best forecasts for policymakers. Moreover, such flexible forecasting behavior tends to increase accuracy on the predictions as it reduces polarization among participants (Mellers, Tetlock, and Arkes, 2019). However, collective judgments may be threatened by "groupthink," information cascades, and herding effects (Sunstein, 2006). ...
... [11]) introduces inequality by definition which goes against the democratic potential of forecasting tournaments (cf. [12] and discussed in [13]). Weighing by competence privileges people who were right in the past. ...
Article
Full-text available
Predicting the future can bring enormous advantages. Across the ages, reliance on supernatural foreseeing was substituted by the opinion of expert forecasters, and now by collective intelligence approaches which draw on many non-expert forecasters. Yet all of these approaches continue to see individual forecasts as the key unit on which accuracy is determined. Here, we hypothesize that compromise forecasts, defined as the average prediction in a group, represent a better way to harness collective predictive intelligence. We test this by analysing 5 years of data from the Good Judgement Project and comparing the accuracy of individual versus compromise forecasts. Furthermore, given that an accurate forecast is only useful if timely, we analyze how the accuracy changes through time as the events approach. We found that compromise forecasts are more accurate, and that this advantage persists through time, though accuracy varies. Contrary to what was expected (i.e. a monotonous increase in forecasting accuracy as time passes), forecasting error for individuals and for team compromise starts its decline around two months prior to the event. Overall, we offer a method of aggregating forecasts to improve accuracy, which can be straightforwardly applied in noisy real-world settings.
... To the extent that psychologists want to make predictions for such events -which our work shows they seem willing to do and are expected to do so well by the public -then it may be advantageous for psychological scientists to learn strategies that improve forecasting accuracy at both the group (Morgan, 2014) and individual level (Grossmann et al., 2021;Mellers et al., 2019). Note. ...
Article
Full-text available
At the onset of the COVID-19 pandemic, psychological scientists frequently made on-the-record predictions in public media about how individuals and society would change. Such predictions were often made outside these scientists’ areas of expertise, with justifications based on intuition, heuristics, and analogical reasoning (Study 1; N = 719 statements). How accurate are these kinds of judgments regarding societal change? In Study 2, we obtained predictions from scientists (N = 717) and lay Americans (N = 394) in Spring 2020 regarding the direction of change for a range of social and psychological phenomena. We compared them to objective data obtained at 6 months and 1 year. To further probe how experience impacts such judgments, 6 months later (Study 3), we obtained retrospective judgments of societal change for the same domains (Nscientists = 270; Nlaypeople = 411). Bayesian analysis suggested greater credibility of the null hypothesis that scientists’ judgments were at chance on average for both prospective and retrospective judgments. Moreover, neither domain-general expertise (i.e., judgmental accuracy of scientists compared to laypeople) nor self-identified domain-specific expertise improved accuracy. In a follow-up study on meta-accuracy (Study 4), we show that the public nevertheless expects psychological scientists to make more accurate predictions about individual and societal change compared to most other scientific disciplines, politicians, and nonscientists, and they prefer to follow their recommendations. These findings raise questions about the role psychological scientists could and should play in helping the public and policymakers plan for future events.
... It is possible that subjective confidence in domain expertise conflates expertise and overconfidence 26,27,28 (versus intellectual humility). There is some evidence that overconfident forecasters are less accurate 29,30 . ...
Article
Full-text available
How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender–career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.
... It is possible that subjective confidence in domain expertise conflates expertise and overconfidence 26,27,28 (versus intellectual humility). There is some evidence that overconfident forecasters are less accurate 29,30 . ...
Article
Full-text available
How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing the accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender–career and racial bias. After we provided them with historical trend data on the relevant domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N = 86 teams and 359 forecasts), with an opportunity to update forecasts on the basis of new data six months later (Tournament 2; N = 120 teams and 546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than those of simple statistical models (historical means, random walks or linear regressions) or the aggregate forecasts of a sample from the general public (N = 802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models and based predictions on prior data.
... It is possible that subjective confidence in domain expertise conflates expertise and overconfidence 28 (versus intellectual humility). There is some evidence that overconfident forecasters are less accurate 29 . These findings, along with a lack of domain-general effect of social science expertise on performance compared to the general public, invite consideration of whether what usually counts as expertise in social sciences translates into greater ability to predict future real-world trends. ...
Article
How well can social scientists predict societal change, and what processes underlie their predictions? To answer these questions, we ran two forecasting tournaments testing accuracy of predictions of societal change in domains commonly studied in the social sciences: ideological preferences, political polarization, life satisfaction, sentiment on social media, and gender-career and racial bias. Following provision of historical trend data on the domain, social scientists submitted pre-registered monthly forecasts for a year (Tournament 1; N=86 teams/359 forecasts), with an opportunity to update forecasts based on new data six months later (Tournament 2; N=120 teams/546 forecasts). Benchmarking forecasting accuracy revealed that social scientists’ forecasts were on average no more accurate than simple statistical models (historical means, random walk, or linear regressions) or the aggregate forecasts of a sample from the general public (N=802). However, scientists were more accurate if they had scientific expertise in a prediction domain, were interdisciplinary, used simpler models, and based predictions on prior data.
... However, the latter run the risk of provoking additional efforts at ego justification if they are not implemented carefully and subtly 133,134 . Finally, cultivating a spirit of intellectual humility (responding generously and humbly towards others) could help to reduce political polarization by reducing ego defensiveness [135][136][137][138] . However, these studies focused on relatively short term changes; more research is needed to develop interventions that promote lasting forms of change. ...
Article
Healthy democratic polities feature competing visions of a good society but also require some level of cooperation and institutional trust. Democracy is at risk when citizens become so polarized that an ‘us versus them’ mentality dominates. Despite a vast multidisciplinary literature, no coherent conceptual framework of the microlevel dynamics that increase or decrease polarization has been presented. In this Review, we provide a conceptual framework to integrate scientific knowledge about cognitive–motivational mechanisms that influence political polarization and the social-communicative contexts in which they are enacted. Ego-justifying and group-justifying motives lead individuals to defend their own pre-existing beliefs and those of their in-group, respectively. However, a distinct class of system-justifying motives contributes to asymmetric forms of polarization. Whereas conservative-rightist ideology is associated with valuing tradition, social order and maintenance of the status quo, liberal-leftist ideology is associated with a push for egalitarian social change. These cognitive–motivational mechanisms interact with social influence processes linked to communication source, message and channel factors, all of which might contribute to increased or decreased polarization. We conclude with a discussion of unanswered questions and ways in which our framework can be extended to the study of culture and institutions. Democracy is at risk when citizens become so polarized that an ‘us versus them’ mentality dominates. In this Review, Jost et al. provide a conceptual framework that integrates scientific knowledge about cognitive–motivational mechanisms that influence political polarization and the social-communicative contexts in which they are enacted.
... The "time capsule" approach of the World after COVID project will allow scholars to see in a broad sense the extent to which these visions do or do not come to pass. We hope among other things that this rich dataset will provide a useful tool for fostering improved predictions for key societal and psychological trends and for nurturing intellectual humility (Mellers et al., 2019). The data is publicly available (https://github.com/grossmania/wac) and we invite other scholars interested in research synthesis, post-COVID reflections, and science communication to peruse it for their needs. ...
Article
Full-text available
How do experts in human behavior think the world might change after the coronavirus (COVID-19) pandemic? What advice do they have for the postpandemic world? Is there a consensus on the most significant psychological and societal changes ahead? To answer these questions, we analyzed interviews from the World After COVID Project—reflections of more than 50 of the world’s top behavioral and social science experts, including fellows of National Academies and presidents of major scientific societies. These experts independently shared their thoughts on possible psychological changes in society in the aftermath of the COVID-19 pandemic and provided recommendations how to respond to the new challenges and opportunities these shifts may bring. We distilled these predictions and suggestions via human-coded analyses and natural language processing techniques. In general, experts showed little overlap in their predictions, except for convergence on a set of social/societal themes (e.g., greater appreciation for social connection, increasing political conflict). Half of the experts approached their post-COVID predictions dialectically, highlighting both positive and negative features of the same domain of change, and many expressed uncertainty in their predictions. The project offers a time capsule of experts’ predictions for the effects of the pandemic on a wide range of outcomes. We discuss the implications of heterogeneity in these predictions, the value of uncertainty and dialecticism in forecasting, and the value of balancing explanation with predictions in expert psychological judgment.
... Showing that their longheld beliefs are not necessarily correct could make people want to continue use of software that uses MABs. Indeed, political beliefs are perhaps some of the most strongly held beliefs people have, but even political beliefs can change after people make forecasts that subsequently turn out to be wrong (Mellers, Tetlock, & Arkes, 2018). ...
Book
Full-text available
Design Recommendations for Intelligent Tutoring Systems (ITSs) explores the impact of intelligent tutoring system design on education and training. Specifically, this volume examines "Self-Improving Systems". The "Design Recommendations" book series examines tools and methods to reduce the time and skill required to develop Intelligent Tutoring Systems with the goal of improving the Generalized Intelligent Framework for Tutoring (GIFT). GIFT is a modular, service-oriented architecture developed to capture simplified authoring techniques, promote reuse and standardization of ITSs along with automated instructional techniques and effectiveness evaluation capabilities for adaptive tutoring tools and methods.
Article
L’attention médiatique est très accaparée par une crainte de la montée des mouvements qualifiés de complotistes. Plusieurs critères de l’écosystème informationnel actuel favorisent effectivement une production et une mise en visibilité de contenus différents de ceux attendus dans l’espace public. Pour autant, la crise que traversent les autorités informationnelles institutionnelles ne saurait se résumer à ce rapport bien spécifique à la réalité qu’est la logique complotiste. Bien au contraire, les formes de contestation, de recherche de solutions alternatives ou d’indifférence aux médias pour dire le monde s’avèrent bien souvent plus variées et reposent sur un ensemble de déterminants complexes. À partir d’une enquête approfondie des conversations en ligne à propos de l’actualité au Québec, cet article propose une grille de lecture des critiques adressées aux autorités informationnelles institutionnelles qui rendent davantage compte de cette complexité.
Article
Social workers routinely make decisions and formulate care plans in the course of their work with children and families. These decisions and care plans are based at least in part on the professional judgement of the worker. An important component of judgemental quality is the extent to which explicit or implicit forecasts about the likelihood of different future actions and events are made with accuracy. In this article, we report an analysis of 21,193 forecasts made by 581 child and family social workers in England about the likelihood of different future actions and events following referrals to children’s services. We found that the more likely social workers said an action or event was to happen (as the forecast likelihood increased towards 100 per cent), the more often it occurred. However, we also found that social workers tend to over-estimate the likelihood of almost all specified future actions and events. Social workers were most accurate when forecasting something to be very unlikely, and less accurate when they forecast something to be likely or very likely. We consider these findings in relation to false positive and negative errors in child protection, and the theory of judgemental rationalism.
Article
I argue that Hegel uses two distinct approaches to hermeneutics in his works, viz. the assimilative and the reconstructive approach. I characterize them, explain why Hegel uses both, and focus on their presuppositions. In light of these reflections, I address the alleged contradiction between Hegel’s reflections on interpretation and their application. Contrary to some literature, I argue that Hegel’s hermeneutical practice does not deviate from his theoretical approach to interpretation. Then I focus on the issue of whether and to what extent these two approaches are intellectually humble practices. I argue that only the reconstructive approach advanced in the Phenomenology is intellectually humble in contrast to the assimilative approach advanced in Hegel’s Lectures .
Article
Across the UK, child protection social workers are routinely called upon to assess the likelihood of future significant harm to children. Yet making consistently accurate judgements about what may or may not happen in future can be a difficult task. In a previous study, we tested social workers’ abilities (n = 283) to forecast the likelihood of different actions, events and outcomes following real-life referrals to social services. Aggerate group performance was only 6% better than you would expect by chance. As a result, we wondered whether social workers could make more accurate forecasts in relation to families they know well. In this paper, we report the results of an in-depth case study, involving two social workers and five families. For eight-months, the social workers generated their own forecasting questions and provided estimates about the likelihood of different actions, events, and outcomes. One of the social workers, with more experience, made forecasts that were on average 12% more accurate than you would expect by chance. The other social worker, with less experience, made forecasts that were 6% less accurate than chance. These findings suggest that simply having more information about the family may not make a consistent difference to the accuracy of social work judgements. What we do not know is the extent to which these findings might be replicated with a larger sample, or the nature of any potential relationship between more accurate judgements and better decision-making within the complex ecology of social work.
Article
Full-text available
During the last decade, intellectual humility has gone from a topic of philosophical inquiry to one of serious scientific investigation. It has been variously described as a remedy for political polarization, a tool for advancing scientific credibility, and a disposition that promotes learning. However, less attention has been paid to how intellectual humility has been defined and measured or how well psychologists’ definitions and measures align with one another or with philosophers’ accounts. Through a systematic review of empirical intellectual humility research, we identified 18 separate definitions and 20 measures including16 unique questionnaires. We then synthesized this research to advance a new framework of intellectual humility. Implications of this framework for measurement and future research on intellectual humility are discussed.
Article
In two large-scale longitudinal datasets (combined N = 5761), we investigated ability-related political polarization in responses to the COVID-19 pandemic. We observed more polarization with greater ability in emotional responses, risk perceptions, and product-purchase intentions across five waves of data collection with a diverse, convenience sample from February 2020 through July 2020 (Study 1, N = 1267). Specifically, more liberal participants had more negative emotional responses and greater risk perceptions of COVID-19 than conservative participants. Compared to conservatives, liberal participants also interpreted quantitative information as indicating higher COVID-19 risk and sought COVID-related news more from liberal than conservative news media. Of key importance, we also compared verbal and numeric cognitive abilities for their independent capacity to predict greater polarization. Although measures of numeric ability, such as objective numeracy, are often used to index ability-related polarization, ideological differences were more pronounced among those higher in verbal ability specifically. Similar results emerged in secondary analysis of risk perceptions in a nationally representative longitudinal dataset (Study 2, N = 4494; emotions and purchase intentions were not included in this dataset). We further confirmed verbal-ability-related polarization findings on non-COVID policy attitudes (i.e., weapons bans and Medicare-for-all) measured cross-sectionally. The present Study 2 documented ability-related polarization emerging over time for the first time (rather than simply measuring polarization in existing beliefs). Both studies demonstrated verbal ability measures as the most robust predictors of ability-related polarization. Together, these results suggest that polarization may be a function of the amount and/or application of verbal knowledge rather than selective application of quantitative reasoning skills.
Article
Full-text available
Research on moral judgment has been dominated by rationalist models, in which moral judgment is thought to be caused by moral reasoning. The author gives 4 reasons for considering the hypothesis that moral reasoning does not cause moral judgment; rather, moral reasoning is usually a post hoc construction, generated after a judgment has been reached. The social intuitionist model is presented as an alternative to rationalist models. The model is a social model in that it deemphasizes the private reasoning done by individuals and emphasizes instead the importance of social and cultural influences. The model is an intuitionist model in that it states that moral judgment is generally the result of quick, automatic evaluations (intuitions). The model is more consistent than rationalist models with recent findings in social, cultural, evolutionary, and biological psychology, as well as in anthropology and primatology.
Article
Full-text available
This research examines the development of confidence and accuracy over time in the context of forecasting. Although overconfidence has been studied in many contexts, little research examines its progression over long periods of time or in consequential policy domains. This study employs a unique data set from a geopolitical forecasting tournament spanning three years in which thousands of forecasters predicted the outcomes of hundreds of events. We sought to apply insights from research to structure the questions, interactions, and elicitations to improve forecasts. Indeed, forecasters’ confidence roughly matched their accuracy. As information came in, accuracy increased. Confidence increased at approximately the same rate as accuracy, and good calibration persisted. Nevertheless, there was evidence of a small amount of overconfidence (3%), especially on the most confident forecasts. Training helped reduce overconfidence, and team collaboration improved forecast accuracy. Together, teams and training reduced overconfidence to 1%. Our results provide reason for tempered optimism regarding confidence calibration and its development over time in consequential field contexts. This paper was accepted by Yuval Rottenstreich, judgment and decision making.
Article
Full-text available
Understanding how individuals revise their political beliefs has important implications for society. In a preregistered study (N = 900), we experimentally separated the predictions of 2 leading theories of human belief revision—desirability bias and confirmation bias—in the context of the 2016 U.S. presidential election. Participants indicated who they desired to win, and who they believed would win, the election. Following confrontation with evidence that was either consistent or inconsistent with their desires or beliefs, they again indicated who they believed would win. We observed a robust desirability bias—individuals updated their beliefs more if the evidence was consistent (vs. inconsistent) with their desired outcome. This bias was independent of whether the evidence was consistent or inconsistent with their prior beliefs. In contrast, we found limited evidence of an independent confirmation bias in belief updating. These results have implications for the relevant psychological theories and for political belief revision in practice.
Article
Full-text available
The heuristics-and-biases research program highlights reasons for expecting people to be poor intuitive forecasters. This article tests the power of a cognitive-debiasing training module (“CHAMPS KNOW”) to improve probability judgments in a four-year series of geopolitical forecasting tournaments sponsored by the U.S. intelligence community. Although the training lasted less than one hour, it consistently improved accuracy (Brier scores) by 6 to 11% over the control condition. Cognitive ability and practice also made largely independent contributions to predictive accuracy. Given the brevity of the training tutorials and the heterogeneity of the problems posed, the observed effects are likely to be lower-bound estimates of what could be achieved by more intensive interventions. Future work should isolate which prongs of the multipronged CHAMPS KNOW training were most effective in improving judgment on which categories of problems.
Chapter
Full-text available
The chapter proposes a model to integrate the cognitive and motivational perspectives on social inference. The model specifies (1) the conditions under which affective and motivational factors do and do not influence inferential processes and (2) the mechanisms through which affective and motivational processes influence inferential processes to produce biased conclusions. The chapter focuses on the role of a self-esteem motive in producing the self-serving attribution bias. This particular motive is chosen because a wide variety of theorists throughout the history of psychology have suggested that the need for self-esteem exerts a powerful influence on people's cognitions and behavior. It should be pointed out; however, the model is quite general and applicable to the mechanisms through which other motives influence inferences as well. Influenced by recent developments in cognitive psychology and information processing, the theorists focus on the way people encode and organize—the retrieve information and on the knowledge structures—transformation rules and heuristics that are used to make inferences of various kinds. The chapter briefly discusses some of the major influences on various steps in the sequence when the only goal of the process is to arrive at an accurate attribution for the observed event.
Article
Full-text available
Across a wide range of tasks, research has shown that people make poor probabilistic predictions of future events. Recently, the U.S. Intelligence Community sponsored a series of forecasting tournaments designed to explore the best strategies for generating accurate subjective probability estimates of geopolitical events. In this article, we describe the winning strategy: culling off top performers each year and assigning them into elite teams of superforecasters. Defying expectations of regression toward the mean 2 years in a row, superforecasters maintained high accuracy across hundreds of questions and a wide array of topics. We find support for four mutually reinforcing explanations of superforecaster performance: (a) cognitive abilities and styles, (b) task-specific skills, (c) motivation and commitment, and (d) enriched environments. These findings suggest that superforecasters are partly discovered and partly created-and that the high-performance incentives of tournaments highlight aspects of human judgment that would not come to light in laboratory paradigms focused on typical performance. © The Author(s) 2015.
Article
Full-text available
This article extends psychological methods and concepts into a domain that is as profoundly consequential as it is poorly understood: intelligence analysis. We report findings from a geopolitical forecasting tournament that assessed the accuracy of more than 150,000 forecasts of 743 participants on 199 events occurring over 2 years. Participants were above average in intelligence and political knowledge relative to the general population. Individual differences in performance emerged, and forecasting skills were surprisingly consistent over time. Key predictors were (a) dispositional variables of cognitive ability, political knowledge, and open-mindedness; (b) situational variables of training in probabilistic reasoning and participation in collaborative teams that shared information and discussed rationales (Mellers, Ungar, et al., 2014); and (c) behavioral variables of deliberation time and frequency of belief updating. We developed a profile of the best forecasters; they were better at inductive reasoning, pattern detection, cognitive flexibility, and open-mindedness. They had greater understanding of geopolitics, training in probabilistic reasoning, and opportunities to succeed in cognitively enriched team environments. Last but not least, they viewed forecasting as a skill that required deliberate practice, sustained effort, and constant monitoring of current affairs.
Article
Full-text available
An important component of political polarization in the United States is the degree to which ordinary people perceive political polarization. We used over 30 years of national survey data from the American National Election Study to examine how the public perceives political polarization between the Democratic and Republican parties and between Democratic and Republican presidential candidates. People in the United States consistently overestimate polarization between the attitudes of Democrats and Republicans. People who perceive the greatest political polarization are most likely to report having been politically active, including voting, trying to sway others' political beliefs, and making campaign contributions. We present a 3-factor framework to understand ordinary people's perceptions of political polarization. We suggest that people perceive greater political polarization when they (a) estimate the attitudes of those categorized as being in the "opposing group"; (b) identify strongly as either Democrat or Republican; and (c) hold relatively extreme partisan attitudes-particularly when those partisan attitudes align with their own partisan political identity. These patterns of polarization perception occur among both Democrats and Republicans. © The Author(s) 2015.
Article
Full-text available
Most subjective probability aggregation procedures use a single probability judgment from each expert, even though it is common for experts studying real problems to update their probability estimates over time. This paper advances into unexplored areas of probability aggregation by considering a dynamic context in which experts can update their beliefs at random intervals. The updates occur very infrequently, resulting in a sparse data set that cannot be modeled by standard time-series procedures. In response to the lack of appropriate methodology, this paper presents a hierarchical model that takes into account the expert's level of self-reported expertise and produces aggregate probabilities that are sharp and well calibrated both in- and out-of-sample. The model is demonstrated on a real-world data set that includes over 2300 experts making multiple probability forecasts over two years on different subsets of 166 international political events.
Article
Full-text available
Five university-based research groups competed to recruit forecasters, elicit their predictions, and aggregate those predictions to assign the most accurate probabilities to events in a 2-year geopolitical forecasting tournament. Our group tested and found support for three psychological drivers of accuracy: training, teaming, and tracking. Probability training corrected cognitive biases, encouraged forecasters to use reference classes, and provided forecasters with heuristics, such as averaging when multiple estimates were available. Teaming allowed forecasters to share information and discuss the rationales behind their beliefs. Tracking placed the highest performers (top 2% from Year 1) in elite teams that worked together. Results showed that probability training, team collaboration, and tracking improved both calibration and resolution. Forecasting is often viewed as a statistical problem, but forecasts can be improved with behavioral interventions. Training, teaming, and tracking are psychological interventions that dramatically increased the accuracy of forecasts. Statistical algorithms (reported elsewhere) improved the accuracy of the aggregation. Putting both statistics and psychology to work produced the best forecasts 2 years in a row.
Article
Full-text available
This paper begins by presenting a simple model of the way in which experts estimate probabilities. The model is then used to construct a likelihood-based aggregation formula for combining multiple probability forecasts. The resulting aggregator has a simple analytical form that depends on a single, easily-interpretable parameter. This makes it computationally simple, attractive for further development, and robust against overfitting. Based on a large-scale dataset in which over 1300 experts tried to predict 69 geopolitical events, our aggregator is found to be superior to several widely-used aggregation algorithms.
Article
Full-text available
Experiments exploring the effects of group discussion on attitudes, jury decisions, ethical decisions, judgments, person perceptions, negotiations, and risk taking (other than the choice-dilemmas task) are generally consistent with a "group polarization" hypothesis, derived from the risky-shift literature. Recent attempts to explain the phenomenon fall mostly into 1 of 3 theoretical approaches: (a) group decision rules, especially majority rule (which is contradicted by available data); (b) interpersonal comparisons (for which there is mixed support); and (c) informational influence (for which there is strong support). A conceptual scheme is presented which integrates the latter 2 viewpoints and suggests how attitudes develop in a social context. (41/2 p ref)
Article
Full-text available
People often hold extreme political attitudes about complex policies. We hypothesized that people typically know less about such policies than they think they do (the illusion of explanatory depth) and that polarized attitudes are enabled by simplistic causal models. Asking people to explain policies in detail both undermined the illusion of explanatory depth and led to attitudes that were more moderate (Experiments 1 and 2). Although these effects occurred when people were asked to generate a mechanistic explanation, they did not occur when people were instead asked to enumerate reasons for their policy preferences (Experiment 2). Finally, generating mechanistic explanations reduced donations to relevant political advocacy groups (Experiment 3). The evidence suggests that people's mistaken sense that they understand the causal processes underlying policies contributes to political polarization.
Article
Full-text available
Previous research has suggested that an effective strategy for debiasing judgments is to have participants "consider the opposite." The present research proposes that considering any plausible alternative outcome for an event, not just the opposite outcome, leads participants to simulate multiple alternatives, resulting in debiased judgments. Three experiments tested this hypothesis using an explanation task paradigm. Participants in all studies were asked to explain either 1 hypothetical outcome (single explanation conditions) or 2 hypothetical outcomes (multiple explanation conditions) to an event; after the explanation task, participants made likelihood judgments. The results of Studies 1 and 2 indicated that debiasing occurred in all multiple explanation conditions, including those that did not involve the opposite outcome. Furthermore, the findings indicated that debiased judgments resulted from participants' spontaneous consideration of additional alternatives in making their likelihood judgments. The results of Study 3 also identified the perceived plausibility of the explained alternative as an important moderating variable in debiasing. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Explores the relations among value conflict, cognitive style, and policy preferences in pre-Civil War America. Drawing on major historical works, prominent political figures were classified into 1 of 4 political positions: abolitionists, free-soil Republicans who would tolerate slavery in the South but prevent further spread, Buchanan Democrats who would permit slavery in new territories, and advocates of slavery. Results revealed (1) greatest integrative complexity among free-soil Republicans and Buchanan Democrats, with declines in complexity moving either leftward toward abolitionists or rightward toward slavery supporters; (2) integrative complexity was a positive function of endorsing values widely regarded as in conflict in that historical period (property rights, states' rights, and domestic peace vs the threat of "Southern slave power" to free labor and democracy). The results are consistent with the value pluralism model and raise warnings against the tendency to view integratively simple reasoning as both cognitively and morally inferior to complex reasoning. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Explored the impact of accountability (the need to justify one's views to others) on the complexity of people's thinking on controversial social issues. 48 undergraduates reported their thoughts on 3 issues and then responded to a series of attitude scales relevant to each topic. Ss provided this information under 1 of 4 conditions: expecting their attitudes to be anonymous or expecting to justify their attitudes to an individual with liberal, conservative, or unknown views. Consistent with previous work on strategic attitude shifts, Ss reported more liberal attitudes when they expected to justify their views to a conservative. Accountability also increased the integrative complexity and evaluative inconsistency of the thoughts reported on each issue but only when Ss expected to justify their attitudes to an individual with unknown views. Findings suggest that accountability leads to more complex information processing only when people do not have the cognitively lazy option of simply expressing views similar to those of the individual to whom they feel accountable. (39 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
People who hold strong opinions on complex social issues are likely to examine relevant empirical evidence in a biased manner. They are apt to accept "confirming" evidence at face value while subjecting "disconfirming" evidence to critical evaluation, and, as a result, draw undue support for their initial positions from mixed or random empirical findings. Thus, the result of exposing contending factions in a social dispute to an identical body of relevant empirical evidence may be not a narrowing of disagreement but rather an increase in polarization. To test these assumptions, 48 undergraduates supporting and opposing capital punishment were exposed to 2 purported studies, one seemingly confirming and one seemingly disconfirming their existing beliefs about the deterrent efficacy of the death penalty. As predicted, both proponents and opponents of capital punishment rated those results and procedures that confirmed their own beliefs to be the more convincing and probative ones, and they reported corresponding shifts in their beliefs as the various results and procedures were presented. The net effect of such evaluations and opinion shifts was the postulated increase in attitude polarization. (28 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Questioned the ecological validity of judgmental biases demonstrated in the laboratory. One objection to these demonstrations is that evolutionary pressures would have rendered such maladaptive behaviors extinct if they had any impact in the "real world." The author attempts to show that even beneficial adaptations may have costs. This argument is extended to propose 3 types of judgment errors (strategy-based errors, association-based errors, and psychophysical based errors), each of which is a cost of a highly adaptive system. This taxonomy of judgment behaviors is used to advance hypotheses as to which debiasing techniques are likely to succeed in each category. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Results of a study of senatorial speeches given during the 82nd Congress indicate that isolationists made significantly less complex policy statements and expressed significantly more positive in-group and negative out-group attitudes than did nonisolationists. Ambivalent isolationists fell between these 2 groups. Results illustrate how content analysis methods can be used to test the generality of psychological hypotheses in high-level political settings in which more traditional measurement approaches are not feasible. (23 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Examined the effects of differential outcome contingencies on the operation of anticipatory position shifts in 2 studies with 81 female high school seniors and 125 undergraduates. Ss were found to shift their positions on an issue while they were expecting to engage an opponent in a discussion of that issue. As predicted, it was possible to influence the size and direction of such anticipatory shifts by manipulating the personal relevance of the discussion topic and the timing of the discussion onset. Shifts could also be nullified by canceling the expectation of discussion. Results are taken to support a general formulation of anticipatory shifts as strategic responses to immediate situational pressures rather than genuine changes in attitude. It was also found that the durability of anticipatory change was associated with the tendency to engage in cognitive activity supportive of the change. The possibility is discussed that most attitude-change studies have not involved attitude shifts but rather the elastic shifts obtained in the present experiments. (20 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Two experiments with 268 paid volunteers investigated the possibility that assessment of confidence is biased by attempts to justify one's chosen answer. These attempts include selectively focusing on evidence supporting the chosen answer and disregarding evidence contradicting it. Exp I presented Ss with 2-alternative questions and required them to list reasons for and against each of the alternatives prior to choosing an answer and assessing the probability of its being correct. This procedure produced a marked improvement in the appropriateness of confidence judgments. Exp II simplified the manipulation by asking Ss first to choose an answer and then to list (a) 1 reason supporting that choice, (b) 1 reason contradicting it, or (c) 1 reason supporting and 1 reason contradicting. Only the listing of contradicting reasons improved the appropriateness of confidence. Correlational analyses of the data of Exp I strongly suggested that the confidence depends on the amount and strength of the evidence supporting the answer chosen. (21 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
Full-text available
Those who consider the likelihood of an event after it has occurred exaggerate their likelihood of having been able to predict that event in advance. We attempted to eliminate this hindsight bias among 194 neuropsychologists. Foresight subjects read a case history and were asked to estimate the probability of three different diagnoses. Subjects in each of the three hindsight groups were told that one of the three diagnoses was correct and were asked to state what probability they would have assigned to each diagnosis if they were making the original diagnosis. Foresight-reasons and hindsight-reasons subjects performed the same task as their foresight and hindsight counterparts, except they had to list one reason why each of the possible diagnoses might be correct. The frequency of subjects succumbing to the hindsight bias was lower in the hindsight-reasons groups than in the hindsight groups not asked to list reasons χ–2(1, N = 140) = 4.12, p
Article
Full-text available
Tested the cognitive vs rhetorical style hypothesis (conservatives have more simplistic rhetorical, not cognitive styles than liberals or moderates) by assessing the integrative complexity of 10 paragraph-sized statements of 81 senators in 5 US Congresses: 3 dominated by liberals and moderates (the 82nd, 94th, and 96th Congresses) and 2 dominated by conservatives (the 83rd and 97th Congresses). Results show that liberals and moderates were more complex than conservatives in the 82nd, 94th, and 96th Congresses but that these differences among ideological groups were much less pronounced in the 83rd and 97th Congresses. The change in pattern was due to sharp declines in the complexity of liberals and, to a lesser extent, moderates in conservative-dominated sessions, not to an increase in the complexity in conservatives. Conservatives displayed more traitlike stability in integrative complexity both within and across Congressional sessions. It is suggested that the integrative complexity of senatorial debate may be a joint product of relatively context-specific styles of political impression management and relatively stable cognitive styles of organizing the political world. (41 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
This essay traces continuities and changes in focus of research and theory in my career. I describe early work on insensitivity to role-conferred advantages in self-presentation (and the personal experiences that prompted that work) and the subsequent identification and naming of the “fundamental attribution error.” I next describe my work on the role that construal processes play in determining responses to various decision-making and attributional contexts. That work, in turn, culminated in identification and exploration of what I now deem the truly “fundamental attribution error”: the illusion of superior personal objectivity and its various consequences for interpersonal and intergroup interactions. I conclude with the lessons I have drawn from my applied work on conflict resolution.
Book
Democracy requires a connection to the “will of the people.” What does that mean in a world of “fake news,” relentless advocacy, dialogue mostly among the like-minded, and massive spending to manipulate public opinion? What kind of opinion can the public have under such conditions? What would democracy be like if the people were really thinking in depth about the policies they must live with? This book argues that “deliberative democracy” is not utopian. It is a practical solution to many of democracy’s ills. It can supplement existing institutions with practical reforms. It can apply at all levels of government and for many different kinds of policy choices. This book speaks to a recurring dilemma: listen to the people and get the angry voices of populism or rely on widely distrusted elites and get policies that seem out of touch with the public’s concerns. Instead, there are methods for getting a representative and thoughtful public voice that is really worth listening to. Democracy is under siege in most countries. Democratic institutions have low approval and face a resurgent threat from authoritarian regimes. Deliberative democracy can provide an antidote. It can reinvigorate our democratic politics. This book draws on the author’s research with many collaborators on “Deliberative Polling”-a process he has conducted in twenty-seven countries on six continents. It contributes both to political theory and to the empirical study of public opinion and participation, and should interest anyone concerned about the future of democracy and how it can be revitalized. © James S. Fishkin 2018 and Part III, Section 2: James S. Fishkin, Thad Kousser, Robert C. Luskin, and Alice Siu and Part III, Section 4: James S. Fishkin, Roy William Mayega, Lynn Atuyambe, Nathan Tumuhamye, Julius Ssentongo, Alice Siu, and William Bazeyo and Part III, Section 5: James S. Fishkin, Robert C. Luskin, and Alice Siu.
Article
Scholars, practitioners, and pundits often leave their assessments of uncertainty vague when debating foreign policy, arguing that clearer probability estimates would provide arbitrary detail instead of useful insight. We provide the first systematic test of this claim using a data set containing 888,328 geopolitical forecasts. We find that coarsening numeric probability assessments in a manner consistent with common qualitative expressions—including expressions currently recommended for use by intelligence analysts—consistently sacrifices predictive accuracy. This finding does not depend on extreme probability estimates, short time horizons, particular scoring rules, or individual attributes that are difficult to cultivate. At a practical level, our analysis indicates that it would be possible to make foreign policy discourse more informative by supplementing natural language-based descriptions of uncertainty with quantitative probability estimates. More broadly, our findings advance long-standing debates over the nature and limits of subjective judgment when assessing social phenomena, showing how explicit probability assessments are empirically justifiable even in domains as complex as world politics.
Article
• Tested the cognitive vs rhetorical style hypothesis (conservatives have more simplistic rhetorical, not cognitive styles than liberals or moderates) by assessing the integrative complexity of 10 paragraph-sized statements of 81 senators in 5 US Congresses: 3 dominated by liberals and moderates (the 82nd, 94th, and 96th Congresses) and 2 dominated by conservatives (the 83rd and 97th Congresses). Results show that liberals and moderates were more complex than conservatives in the 82nd, 94th, and 96th Congresses but that these differences among ideological groups were much less pronounced in the 83rd and 97th Congresses. The change in pattern was due to sharp declines in the complexity of liberals and, to a lesser extent, moderates in conservative-dominated sessions, not to an increase in the complexity in conservatives. Conservatives displayed more traitlike stability in integrative complexity both within and across Congressional sessions. It is suggested that the integrative complexity of senatorial debate may be a joint product of relatively context-specific styles of political impression management and relatively stable cognitive styles of organizing the political world. (41 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved) • Tested the cognitive vs rhetorical style hypothesis (conservatives have more simplistic rhetorical, not cognitive styles than liberals or moderates) by assessing the integrative complexity of 10 paragraph-sized statements of 81 senators in 5 US Congresses: 3 dominated by liberals and moderates (the 82nd, 94th, and 96th Congresses) and 2 dominated by conservatives (the 83rd and 97th Congresses). Results show that liberals and moderates were more complex than conservatives in the 82nd, 94th, and 96th Congresses but that these differences among ideological groups were much less pronounced in the 83rd and 97th Congresses. The change in pattern was due to sharp declines in the complexity of liberals and, to a lesser extent, moderates in conservative-dominated sessions, not to an increase in the complexity in conservatives. Conservatives displayed more traitlike stability in integrative complexity both within and across Congressional sessions. It is suggested that the integrative complexity of senatorial debate may be a joint product of relatively context-specific styles of political impression management and relatively stable cognitive styles of organizing the political world. (41 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
The most widely debated conception of democracy in recent years is deliberative democracy--the idea that citizens or their representatives owe each other mutually acceptable reasons for the laws they enact. Two prominent voices in the ongoing discussion are Amy Gutmann and Dennis Thompson. In Why Deliberative Democracy?, they move the debate forward beyond their influential book, Democracy and Disagreement. What exactly is deliberative democracy? Why is it more defensible than its rivals? By offering clear answers to these timely questions, Gutmann and Thompson illuminate the theory and practice of justifying public policies in contemporary democracies. They not only develop their theory of deliberative democracy in new directions but also apply it to new practical problems. They discuss bioethics, health care, truth commissions, educational policy, and decisions to declare war. In "What Deliberative Democracy Means," which opens this collection of essays, they provide the most accessible exposition of deliberative democracy to date. They show how deliberative democracy should play an important role even in the debates about military intervention abroad. Why Deliberative Democracy? contributes to our understanding of how democratic citizens and their representatives can make justifiable decisions for their society in the face of the fundamental disagreements that are inevitable in diverse societies. Gutmann and Thompson provide a balanced and fair-minded approach that will benefit anyone intent on giving reason and reciprocity a more prominent place in politics than power and special interests.
Article
One of the most salient attributes of information is valence: whether a piece of news is good or bad. Contrary to classic learning theories, which implicitly assume beliefs are adjusted similarly regardless of valence, we review evidence suggesting that different rules and mechanisms underlie learning from desirable and undesirable information. For self-relevant beliefs this asymmetry generates a positive bias, with significant implications for individuals and society. We discuss the boundaries of this asymmetry, characterize the neural system supporting it, and describe how changes in this circuit are related to individual differences in behavior.
Article
The intelligence failures surrounding the invasion of Iraq dramatically illustrate the necessity of developing standards for evaluating expert opinion. This book fills that need. Here, Philip E. Tetlock explores what constitutes good judgment in predicting future events, and looks at why experts are often wrong in their forecasts. Tetlock first discusses arguments about whether the world is too complex for people to find the tools to understand political phenomena, let alone predict the future. He evaluates predictions from experts in different fields, comparing them to predictions by well-informed laity or those based on simple extrapolation from current trends. He goes on to analyze which styles of thinking are more successful in forecasting. Classifying thinking styles using Isaiah Berlin's prototypes of the fox and the hedgehog, Tetlock contends that the fox--the thinker who knows many little things, draws from an eclectic array of traditions, and is better able to improvise in response to changing events--is more successful in predicting the future than the hedgehog, who knows one big thing, toils devotedly within one tradition, and imposes formulaic solutions on ill-defined problems. He notes a perversely inverse relationship between the best scientific indicators of good judgement and the qualities that the media most prizes in pundits--the single-minded determination required to prevail in ideological combat. Clearly written and impeccably researched, the book fills a huge void in the literature on evaluating expert opinion. It will appeal across many academic disciplines as well as to corporations seeking to develop standards for judging expert decision-making.
Article
The past decade has witnessed an explosion of interest in the partisan polarization of the American electorate. Scholarly investigation of this topic has coincided with the media’s portrayal of a polity deeply divided along partisan lines. Yet little research so far has considered the consequences of the media’s coverage of political polarization. We show that media coverage of polarization increases citizens’ beliefs that the electorate is polarized. Furthermore, the media’s depiction of a polarized electorate causes voters to moderate their own issue positions but increases their dislike of the opposing party. These empirical patterns are consistent with our theoretical argument that polarized exemplars in journalistic coverage serve as anti-cues to media consumers. Our findings have important implications for understanding current and future trends in political polarization.
Article
Theorists argue that deliberation promotes enlightenment and consensus, but scholars do not know I how deliberation affects policy opinions. Using the deliberative democracy and public opinion JL literatures as a guide, I develop a theory of opinion updating where citizens who deliberate revise their prior beliefs, particularly when they encounter consensual messages. A key aspect of this model is that opinion strength moderates the deliberative opinion change process. In two separate propensity score analyses using panel survey data from a deliberative forum and cross-sectional surveys, I show how deliberation and discussion both affect opinions toward Social Security reform. However, deliberation differs from ordinary discussion in that participants soften strongly held views, encounter different perspectives, and learn readily. Thus, deliberation increases knowledge and alters opinions, but it does so selectively based on the quality and diversity of the messages as well as the willingness of participants to keep an open mind.
Article
When aggregating the probability estimates of many individuals to form a consensus probability estimate of an uncertain future event, it is common to combine them using a simple weighted average. Such aggregated probabilities correspond more closely to the real world if they are transformed by pushing them closer to 0 or 1. We explain the need for such transformations in terms of two distorting factors: The first factor is the compression of the probability scale at the two ends, so that random error tends to push the average probability toward 0.5. This effect does not occur for the median forecast, or, arguably, for the mean of the log odds of individual forecasts. The second factor-which affects mean, median, and mean of log odds-is the result of forecasters taking into account their individual ignorance of the total body of information available. Individual confidence in the direction of a probability judgment (high/low) thus fails to take into account the wisdom of crowds that results from combining different evidence available to different judges. We show that the same transformation function can approximately eliminate both distorting effects with different parameters for the mean and the median. And we show how, in principle, use of the median can help distinguish the two effects.
Article
Many scholars view integratively complex reasoning as either cognitively or morally superior to integratively simple reasoning. This value judgment is, however, too simple to capture the complex, subtle, and even paradoxical linkages between integrative complexity and "good judgment" in historical context. Our case studies add to the growing literature on this topic by assessing the integrative and cognitive complexity of policy statements that Winston Churchill and his political adversaries made during two key foreign policy debates of the 1930s-the appeasement of Nazi Germany (where contemporary opinion overwhelmingly favors Churchill) and the granting of self-government to India (where contemporary opinion overwhelmingly favors Churchill's opponents). In both private and public, Churchill expressed less integratively complex but more cognitively complex opinions than did his opponents on both Nazi Germany and self-government for India. The results illustrate (a) impressive consistency in Churchill's integrative but not cognitive complexity in both private and public communications over time and issue domains, and (b) the dependence of normative judgments of styles of thinking on speculative counterfactual reconstructions of history and on moral-political values. We close by arguing that, although integrative complexity can be maladaptive in specific decision-making settings, it can still be highly adaptive at the meta-decision-making level where leaders "decide how to decide." Good judgment requires the ability to shift from simple to complex modes of processing in timely and appropriate ways.
Article
Can partisan media (in particular, partisan TV news) polarize viewers? I outline a set of hypotheses to explain the conditions under which partisan media will increase attitudinal polarization. I use original experiments to test this theory, and find that like-minded messages do have a strong polarizing effect on viewers' attitudes. I also show that cross-cutting messages have, on average, little effect on attitudes, but that they can have strongly polarizing or moderating effects for voters with particular traits. I provide evidence supporting one of the primary hypothesized mechanisms, and also show their duration outside of the lab. I draw on experimental techniques from biomedical studies to show how viewer's preferences for watching partisan media shape these effects. I conclude by discussing the implications of these findings for both theories of media effects and political behavior more broadly. + The author thanks Daniella Lejitneker and the staff of the Wharton Behavioral Lab for help implementing experiment 1 in the paper. Thanks also to Pope, and participants at the MIT American Politics Conference for comments, and to the School of Arts and Sciences and the Vice-Provost for Research at the University of Pennsylvania for funding these experiments. Any remaining errors are my own. The supplemental data with details on the experiments is available upon request from the author.
Chapter
Attribution theory is concerned with the attempts of ordinary people to understand the causes and implications of the events they witness. It deals with the “naive psychology” of the “man in the street” as he interprets his own behaviors and the actions of others. For man—in the perspective of attribution theory—is an intuitive psychologist who seeks to explain behavior and draw inferences about actors and their environments. To better understand the perceptions and actions of this intuitive scientist, his methods must be explored. The sources of oversight, error, or bias in his assumptions and procedures may have serious consequences, both for the lay psychologist himself and for the society that he builds and perpetuates. These shortcomings, explored from the vantage point of contemporary attribution theory, are the focus of the chapter. The logical or rational schemata employed by intuitive psychologists and the sources of bias in their attempts at understanding, predicting, and controlling the events that unfold around them are considered. Attributional biases in the psychology of prediction, perseverance of social inferences and social theories, and the intuitive psychologist's illusions and insights are described.
Article
Five studies tested when and why individuals engage in confirmatory information searches (selective exposure) following predictions. Participants engaged in selective exposure following their own predictions, even when their predictions were completely arbitrary (Studies 1 and 3). The selective exposure was not simply the result of a cognitive bias tied to the salience of a prediction option (Study 2). Instead, it appears that making a prediction—regardless of how ill-informed a person is while making the prediction—can cause the person to anticipate enjoyment from being right (Studies 4 and 5) and to select new information consistent with that outcome. The results establish a desirability account that can explain post-prediction selective exposure effects even in cases when defense motivations, pre-existing differences, or positive-test strategies can be ruled out as explanations.
Article
Evidence from 4 studies with 584 undergraduates demonstrates that social observers tend to perceive a "false consensus" with respect to the relative commonness of their own responses. A related bias was shown to exist in the observers' social inferences. Thus, raters estimated particular responses to be relatively common and relatively unrevealing concerning the actors' distinguishing personal dispositions when the responses in question were similar to the raters' own responses; responses differing from those of the rater, by contrast, were perceived to be relatively uncommon and revealing of the actor. These results were obtained both in questionnaire studies presenting Ss with hypothetical situations and choices and in authentic conflict situations. The implications of these findings for the understanding of social perception phenomena and for the analysis of the divergent perceptions of actors and observers are discussed. Cognitive and perceptual mechanisms are proposed which might account for distortions in perceived consensus and for corresponding biases in social inference and attributional processes. (33 ref) (PsycINFO Database Record (c) 2012 APA, all rights reserved)
Article
People must often engage in sequential sampling in order to make predictions about the relative quantities of two options. We investigated how directional motives influence sampling selections and resulting predictions in such cases. We used a paradigm in which participants had limited time to sample items and make predictions about which side of the screen contained more of a critical item. Sampling selections were biased by monetary desirability manipulations, and participants exhibited a desirability bias for both dichotomous and continuous predictions.
Article
Deliberating groups, including juries, typically end up in a more extreme position in line with their predeliberation tendencies. A jury whose members are inclined, before deliberation, to find a defendant not guilty will likely render a verdict of not guilty; a jury whose members want to award punitive damages will likely produce an award higher than that of the median juror. The phenomenon of group polarization, found in many domains, stems from a combination of information pooling and peer pressure. The events portrayed in the film 12 Angry Men seem to defy the logic of group polarization, but the film nonetheless shows an acute psychological sense.