ArticlePDF Available

Costs and Benefits of Judgment Errors: Implications for Debiasing

Authors:

Abstract

Questioned the ecological validity of judgmental biases demonstrated in the laboratory. One objection to these demonstrations is that evolutionary pressures would have rendered such maladaptive behaviors extinct if they had any impact in the "real world." The author attempts to show that even beneficial adaptations may have costs. This argument is extended to propose 3 types of judgment errors (strategy-based errors, association-based errors, and psychophysical based errors), each of which is a cost of a highly adaptive system. This taxonomy of judgment behaviors is used to advance hypotheses as to which debiasing techniques are likely to succeed in each category. (PsycINFO Database Record (c) 2012 APA, all rights reserved)
... All people, including experts, are susceptible to biased decisions [1]. People are largely unaware of how or when bias affects their judgments [2][3][4][5], hence, competent and wellintentioned professionals can be biased without any awareness of it. ...
... Once biased it is very difficult, if not impossible, to debias one's perspective. Recall that bias largely resides outside of awareness [2]. Decision makers who adopt the view that they are biased, without substantial evidence of its presence, are reliant on theories of how they are biased and the magnitude of that bias [113]. ...
Preprint
Full-text available
Psychological research demonstrates how our perceptions and cognitions are affected by context, motivation, expectation, and experience. A mounting body of research has revealed the many sources of bias that affect the judgments of experts as they execute their work. Professionals in such fields as forensic science, intelligence analysis, criminal investigation, medical and judicial decision-making find themselves at an inflection point where past professional practices are being questioned and new approaches developed. Workplace investigation is a professional domain that is in many ways analogous to the aforementioned decision-making environments. Yet, workplace investigation is also unique, as the sources, magnitude, and direction of bias are specific to workplace environments. The workplace investigation literature does not comprehensively address the many ways that the workings of honest investigators' minds may be biased when collecting evidence and/or rendering judgments; nor does the literature offer a set of strategies to address such happenings. The current paper is the first to offer a comprehensive overview of the important issue of cognitive bias in workplace investigation. In it I discuss the abilities and limitations of human cognition, provide a framework of sources of bias, as well as, offer suggestions for bias mitigation in the investigation process.
... associative assumptions about what they might have seen before (Morewedge & Kahneman, 2010). Detailed in the social psychology domain, such intuitive impressions may be wildly inaccurate when assessed from the basis of objective fact (Arkes, 1991;Busemeyer et al., 2011). ...
Article
Full-text available
According to the knowledge transfer literature, members of a project workgroup who share collaborative history possess a “shared understanding” thought to increase psychological safety and efficiency, thereby improving current project performance. But knowledge misfit can occur depending on where they have previously collaborated (internal or external to the focal firm) and the type of project currently pursued. We contend that misfit engenders “shared misunderstanding” where prior collaborators impose irrelevant knowledge onto a project, undermining performance. We extend research on knowledge transfer and human capital mobility by predicting that internal collaborative history helps exploitative projects but undermines imitative projects, while external collaborative history aids imitative projects but undercuts exploitative projects. We test these predictions in the context of the video game industry, with a sample of 1,000 video games released by 53 game publishers from 1996-2013, spanning 47,437 person-game observations, and find results largely consistent with our hypotheses.
... The combination of a performance-based financial incentive with a loss framing was not sufficient to eliminate the reminder bias. This is consistent to the view that some decisions are rather difficult to debias, especially if relying on a nonlinear relationship between the decision prospects (gain/benefit vs. loss/cost) and the subjective values assigned to those prospects (Arkes, 1991). Future research should thus continue investigating the joint--and hopefully debiasing--role of framing, metacognition, and cognitive effort on cognitive offloading, with a specific focus on how to best achieve more optimal decisions in offloading situations. ...
Article
Full-text available
Nowadays individuals can readily set reminders to offload intentions onto external resources, such as smartphone alerts, rather than using internal memory. Individuals tend to be biased, setting more reminders than would be optimal. We address the question whether the reminder bias depends on offloading scenarios being framed as either gains or losses, both between-participants (Experiment 1) and within-participants (Experiment 2). In both experiments, framing of reminders in terms of gains resulted in participants employing a risk-averse strategy and using more reminders than would be optimal. Importantly, however, participants used reminders more optimally and were more willing to choose the risk-seeking option of remembering internally when reminders implied a loss. Based on metacognitive measures in Experiment 2, the reminder bias increased the more underconfident participants were about their memory abilities in both framing scenarios. Framing did not alter this relationship between erroneous metacognitive underconfidence and reminder bias but provides an additional influence. We conclude that emphasizing the losses (costs) associated with external reminders helps in achieving more optimal decisions in offloading situations, and that in addition to cognitive effort and metacognitive judgments, framing needs to be considered in improving individuals’ offloading behavior.
... Similar instructions have been effective in eliminating biases related to overconfidence (Koriat et al., 1980), social judgments (Lord et al., 1984), hindsight bias (Arkes et al., 1988), the framing effect (Cheng et al., 2014) and the anchoring effect (Adame, 2016). There are several other debiasing strategies (e.g., Arkes, 1991), but we focused on elaborative feedback and consider-the-opposite because we believed that they were strong candidates to improve CRT performance. ...
Article
Cognitive reflection, or the ability to inhibit intuitive and incorrect responses in favour of correct responses, predicts performance on a variety of cognitive tasks. The present study examined interventions to improve cognitive reflection. In two experiments, college students (N = 491) were assigned to one of three conditions, completed two versions of a cognitive reflection test (CRT), and then completed transfer tasks. Between the two CRTs, some participants were provided with elaborative feedback, others were instructed to consider additional responses for their initial responses and the final group was a control. In both experiments, CRT performance increased between the first and second CRT in the feedback and instruction groups, but not in the control group. There was little evidence, however, for transfer to other tasks. These results suggest that cognitive reflection performance can be improved with brief interventions but that this improvement may not transfer to related tasks.
Article
Psychological research demonstrates how our perceptions and cognitions are affected by context, motivation, expectation, and experience. A mounting body of research has revealed the many sources of bias that affect the judgments of experts as they execute their work. Professionals in such fields as forensic science, intelligence analysis, criminal investigation, medical and judicial decision-making find themselves at an inflection point where past professional practices are being questioned and new approaches developed. Workplace investigation is a professional domain that is in many ways analogous to the aforementioned decision-making environments. Yet, workplace investigation is also unique, as the sources, magnitude, and direction of bias are specific to workplace environments. The workplace investigation literature does not comprehensively address the many ways that the workings of honest investigators' minds may be biased when collecting evidence and/or rendering judgments; nor does the literature offer a set of strategies to address such happenings. The current paper is the first to offer a comprehensive overview of the important issue of cognitive bias in workplace investigation. In it I discuss the abilities and limitations of human cognition, provide a framework of sources of bias, as well as, offer suggestions for bias mitigation in the investigation process.
Chapter
There are many challenges and complexities involved when making health security decisions. We can better cope with these challenges and complexities if we implement a conceptually robust framework for making reasoned health security decisions, which is introduced in this chapter. We also position this decision making framework within Decision Sciences. This scientific basis is important, as it provides an adequate conceptual background for understanding how decisions can be represented, analyzed, and improved. We also highlight the potential benefits of using such a rigorous decision framework and the important distinction between decision processes and decision outcomes.
Article
The effect of performance‐based incentives on judgment biases is a controversial issue. A recent extensive meta‐analysis of Brañas‐Garza et al. (2019, https://doi.org/10.1016/j.socec.2019.101455) found no effect of incentives on performance in the cognitive reflection test (CRT), a test commonly used for assessing heuristic versus deliberative judgments. We conducted a meta‐analysis of studies that directly compared an incentive versus no‐incentive conditions (n = 2836) and found a small positive effect of incentivization (Cohen's d = .21) on CRT performance. In addition, we re‐examined the Brañas‐Garza et al. (2021) dataset (n = 39,385), taking into consideration not only whether incentivization was used but also the nominal and fiscal size of the incentive. The results show that surprisingly when including only studies that used monetary incentives, the effect of incentivization and incentive size are significant, even when taking into consideration other aspects of the studies. The results thus suggest a small but robust effect of incentives on judgment biases.
Thesis
İktisadın bir bilim olarak kabulünden önce de bir bilim dalı olarak görüldükten sonra da öznesi ve nesnesi insan olmuştur. Bilim Devrimiyle birlikte doğa bilimleri dışındaki bilimlerin tam nesnel olamamaları nedeniyle gerçek bilim olarak kabul edilmemesi iktisat biliminin iktisadi insanı tanımlamasına neden olmuştur. İktisadi insan ile iktisat bilimi konu edindiği insanı nesnelleştirerek iktisadi modelleri matematikselleştirmiştir. İktisattaki insanın gerçek insan olma vasıflarını yitirmesinden rahatsızlık duyan iktisatçılar, davranışsal iktisat çalışmalarıyla iktisattaki insana insanlığını geri kazandırmayı amaçlamışlardır. 1979'da Kahneman ve Tversky tarafından geliştirilen Beklenti Teorisi bu amaçla davranışsal iktisat altında üretilen kuramlardan biridir. Beklenti Teorisine göre bireyler risk altında karar ve seçim problemlerinde kazanç durumunda riskten kaçınan davranış sergilerlerken, kayıp durumunda risk alan yani kayıptan kaçınan davranış sergilemektedirler. Ayrıca bireylerin, kâr amacı gütmeseler de bir mal için vermiş oldukları minimum satma fiyatları maksimum satın alma fiyatlarından büyüktür yani bireylerde sahiplenme etkisi mevcuttur. Bu çalışmanın ana amacı Beklenti Teorisinin tutarlılığı ve geçerliğinin test edilmesidir. Bu doğrultuda çalışmanın ana hipotezi Beklenti Teorisinin geçerli olduğu yönünde belirlenmiştir. Verilerin toplanması amacıyla grup içi tasarımla bir deney prosedürü hazırlanmış ve Aydın Adnan Menderes Üniversitesi Nazilli İktisadi ve İdari Bilimler Fakültesi lisans öğrencilerine uygulanmıştır. Toplanan veriler betimleyici ve non-parametrik istatistik veri analiz yöntemleriyle incelenmiştir. Elde edilen bulgular Beklenti Teorisinin geçerli olduğu yönündedir. Deney katılımcılarında çoğunlukla kazançlarda riskten kaçınma, kayıplarda risk alma davranışları gözlenmiştir ve deney katılımcılarında çoğunlukla sahiplenme etkisinin varlığı görülmüştür. Beklenti Teorisi konusunda Türkiye'de yapılmış alan çalışması ve Türkçe literatür çok kısıtlıdır. Bu çalışma literatürdeki bu boşluğu dolduracak öncü bir çalışma olması nedeniyle önem arz etmektedir. Before and after economics is considered a science discipline, the subject, and object of economics have been the human. With the Scientific Revolution, the fact that sciences other than natural sciences were not accepted as real science due to their inability to be fully objective caused the science of economics to define the economic man. With economic man, the science of economics has mathematized the economic models by objectifying the human subject. The economists, who are disturbed by the fact that the human being in economics has lost their qualifications to be a real human, aimed to restore the humanity of the human in economics through behavioral economics studies. Prospect Theory, developed by Kahneman and Tversky in 1979, is one of the theories produced under behavioral economics for this purpose. According to the Prospect Theory, in decision and choice problems under risk individuals exhibit risk-averse behavior in case of gains, while they display risk-taking, that is, loss-averse behavior in case of losses. In addition, even if individuals do not seek profit, the minimum selling prices for a good are higher than the maximum purchase prices, that is, the individuals have an endowment effect. The main purpose of this study is to test the consistency and validity of the Prospect Theory. In this direction, the main hypothesis of the study was determined as the Prospect Theory is valid. In order to collect the data, an experimental procedure was prepared with a within-group design and applied to the undergraduate students of Aydın Adnan Menderes University Nazilli Faculty of Economics and Administrative Sciences. The collected data were analyzed with descriptive and non-parametric statistical data analysis methods. The findings show that the Prospect Theory is valid. It is mostly observed that experiment participants have risk avoidance in gains and risk-taking in losses, and it was observed that the experimental participants mostly had the endowment effect. Field studies and Turkish literature on Prospect Theory in Turkey are very limited. This study is important because it is a pioneering study that will fill this gap in the literature. Tez No/Thesis Number: 720927 Tezin tam metnine aşağıdaki adreste "tarama terimi giriniz" kısmına tez numarasını (720927) yazıp "aranacak alan" kısmında "Tez No" seçeneğini seçip aratarak ulaşabilirsiniz./You can reach the full text of the thesis from the web adress below by writing the thesis number (720927) in the "Enter your search terms" section and choosing the "Thesis No" in the "Search in" section and clicking on the "Search" button . https://tez.yok.gov.tr/UlusalTezMerkezi/
Article
Diagnostic processes are difficult to manage because they require the decision maker (DM) to dynamically balance the benefit of acquiring more diagnostic information against the cost of doing so. When additional and unattended diagnostic tasks build up over time, making this tradeoff becomes especially challenging. In their study “Mismanaging Diagnostic Accuracy Under Congestion,” Kremer and de Véricourt uncover different biases to which DMs are subject when making diagnostic decisions while unattended diagnostic tasks accumulate over time. The authors find that, in their experiments, DMs are overall insufficiently sensitive to congestion. As a result, DMs acquire too little information at low congestion levels, but too much at high levels, compared with an optimal normative benchmark. This in fact increases both the diagnostic errors and congestion levels in the system. The authors disentangle the underlying mechanisms for these effects and suggests different approaches to debias the DMs.
Article
The studies described in this paper investigated the use of a decisional heuristic—anchoring-and-adjustment—in an information-rich, real world setting. In order to assess the generalizability of laboratory research on this decision heuristic, students and real estate agents toured and made pricing decisions about real estate properties. It was hypothesized that manipulated listing prices would anchor values assigned to the properties. Results were consistent with the use of an anchoring-and-adjustment value estimation strategy in information-rich, real world settings. Implications for the understanding of judgmental expertise are discussed.
Article
This article offers a critique of Littlefield and Rushton's (1986) application of sociobiological principles to bereavement following the death of a child. The following general issues are considered: (a) whether behavior is always adaptive and (b) the distinction between proximate and ultimate explanations. It is argued that grief is a maladaptive by-product of another, adaptive feature and that hypotheses about the severity of grief are best derived from proximate considerations rather than genetic relatedness. The use of a single-item rating scale to measure grief is questioned, and it is noted that interspouse reliabilities reported in the article were low, a problem not solved (as claimed) by aggregation. Criticisms are made of the specific hypotheses, notably in terms of their origins in sociobiological theory. It is argued that functional hypotheses are not alternatives to proximate mechanisms, but enable some proximate mechanisms to be viewed from the perspective of evolutionary biology.