Project

Mass pre-registered replications of classic findings in judgment and decision making

Goal: See http://mgto.org/pre-registered-replications/
Projects: https://osf.io/5z4a8/

In 2016, following recent developments in psychological science (the so-called “replication crisis”) and gaining my academic independence, I decided to make serious changes to my research agenda to prioritize pre-registered replications and focus on the realm of judgment and decision making. The aim was to revisit research findings I once took for granted and re-establish the foundations on which I hope to build my research. I, therefore, decided that all my teaching and mentoring work with guided thesis students will involve pre-registered replications or pre-registered meta-analyses, to examine the classics in the field.

In 2017 I guided 3 masters students at Maastricht University to pre-test this realignment. It far exceeded my expectations. We completed 3 pre-registered replication, 3 pre-registered meta-analyses, and one review paper summarizing the insights gained. Once joining HKU, in 2018, I decided to scale up and mass-mobilize HKU’s undergraduate students and lead a mass pre-registered replication effort. In the first year, two semesters, of running this project, we’ve successfully completed 45 replication projects, making this one of the largest replication efforts in social-psychology. For each of the replication projects, we have full pre-registrations, data/code, and all written up in APA style submission ready student reports. In the second semester, most of the replications also include extensions with interesting insights.

I will continue running this in academic year 2019-2020 with 20+ new replications+extensions. If any of this is of interest to you – lots of ways to join in. I am looking for interested early career researchers to join us.

Updates
0 new
14
Recommendations
0 new
17
Followers
0 new
61
Reads
6 new
979

Project log

Gilad Feldman
added a research item
The temporal pattern of regret is the phenomenon that people perceive or experience stronger regret over action compared to inaction in the short-term, yet stronger regret over inaction compared to action in the long term. Following mixed and null findings in the literature, we conducted replications and extension of Studies 1, 3, 4, and 5 in the classic Gilovich and Medvec (1994) which first demonstrated this phenomenon, with a single combined data collection in randomized display order with an online sample of Americans on MTurk (N = 988). We found support for the original findings using different designs in Studies 1, 3, and 4, yet with weaker effects. We failed to find support for such a pattern in Study 5. We discuss possible interpretations for these differences: our replication adjustments, the change in the meaning of action and inaction, or change in hypothetical versus real-life personal experiences. Extending the replications, we found support for stronger responsibility for action compared to inaction both in the short-term and the long-term. We conclude overall support for the effects, yet with follow-up work necessary to resolve the inconsistencies in the findings of the Study 5 replication. Pre-registration, materials, data, and code were made available on: https://osf.io/7m3q2/
Gilad Feldman
added a research item
Malle and Knobe's (1997) Study 1 found that people exhibit a shared understanding of intentionality and apply it consistently in their judgments. The study found that different people tend to judge intentionality similarly, and that intentionality ratings were consistent across a set of behaviors from an actor's or an observer's perspective. Additionally, the presence or absence of a definition of intentionality as part of the study instruction did not seem to affect the intentionality judgments. We conducted two pre-registered replications of Study 1 (N=46; N=817). The replication results provide support for the findings of the original study. Consistent with the original study, we found high inter-rater agreement across perspective (actor vs. observer) experimental conditions; and no evidence for differences in intentionality ratings depending on whether a definition of intentionality was provided. We observed that actor perspective led to a higher average rating of intentionality than the observer perspective. Materials, data, and code are available on: https://osf.io/4q5ce/
Gilad Feldman
added a research item
[IMPORTANT: Method and results were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection. For the purpose of the simulation, we wrote things in past tense, but no pre-registration or data collection took place yet.] Bastian et al. (2012) argued that the ‘meat paradox’–caring for animals yet eating them–is maintained by motivated moral disengagement driven by a psychologically aversive tension between people’s moral standards (caring for animals) and their behavior (eating them). One disengagement mechanism that is thought to play a central role is the denial of food animal minds, and therefore their status as moral patients. This idea has garnered substantial interest and has framed much of the psychological approach to meat consumption. We propose to subject Studies 1 and 2 of Bastian et al. (2012) to high-powered direct replications. For Study 1, our replication [failed to find/found] support for the original findings: perceptions of animals’ minds were negatively related to their perceived edibility, and positively related to moral concern for them and negative affect related to eating them, [summary effect sizes + CIs will be added here]. For Study 2, our replication [failed to find/found] an effect of learning that animals will be used for food on the tendency to deny them mental capabilities. Overall, our findings [matched/did not match] with the original’s, and we [found/failed to find] support for the relationship between animal mind denial and perceptions of their status as sources of food. Materials, data, and code are available on the OSF: https://osf.io/h2pqu/.
Gilad Feldman
added a research item
[IMPORTANT: This is a Registered Report Stage 1 before data collection. Written in past tense as a template to simulate what the final manuscript will look like. No pre-registration or data collection have been conducted.] The sunk cost effect is the tendency for an individual's decision-making to be biased based on previous investments of resources. Soman (2001) found that sunk cost effect is weaker for time than for money (Studies 1 and 2), and that the facilitation of money-like accounting strengthens the sunk cost effect for time (Study 5). We conducted a close, high-powered, pre-registered replication of Soman (2001) with an online sample of US American Amazon Mechanical Turk (N = 540). We also conducted additional exploratory analyses, testing robustness of the original’s hypotheses. We found support/failed to find support for [effect sizes and confidence intervals of original versus replication, split per hypothesis/effect]. Materials, data, and code are available on: https://osf.io/pm264/
Gilad Feldman
added a research item
[IMPORTANT: This is a Registered Report Stage 1 before data collection. Written in past tense as a template to simulate what the final manuscript will look like. No pre-registration or data collection have been conducted.] Shame and guilt are unpleasant self-conscious emotions associated with negative evaluations of oneself or one’s behavior. Smith et al. (2002) demonstrated that shame and guilt are distinct and are impacted differently by public exposure, that is, the (potential) exposure to disapproving appraisals of one’s misdeeds by others. The impact of public exposure (compared to no exposure) was greater for feelings of shame than for feelings of guilt. We conducted a direct replication of Smith et al. (2002)’s Study 1 and found that exposure […] feelings of shame and guilt (effect sizes and confidence intervals). Our results suggest that …
Gilad Feldman
added a research item
[IMPORTANT: This is currently a Registered Report at PCI-RR. This is the thesis submitted after PCIRR Stage 1 in-principle acceptance and data collection, and is pending verifications and Stage 2 peer review] People tend to view their own “true self” as generally positive, and as guiding inner moral values. Newman et al. (2014) demonstrated that the true-self link to morality extends also to attributions towards others’ behaviors and changes. We conducted a pre-registered replication and extensions project of Newman et al. (2014)’s Studies 1 and 2, with a US American online Amazon Mechanical Turk sample (N = 803). We found support for Study 1’s findings that morally positive changes in others are perceived as more reflective of true-self than morally negative changes [i) forced-choice measure: original: η² p=.39, 95%CI[.25, .51]; replication: η² p= .20; 95% CI [.16, .23]; ii) true self rating: original: η² p=.33, 95%CI[.19, .45]; replication:η² p=.22, 95%CI[.15, .25]. We found support for Study 2’s findings that changes more aligned with observers’ political moral views are perceived as more reflective of true-self [original:η² p=.04, 95%CI[.00, .11] ; replication: .35, 95%CI[.29, .41]. Extending the replication, we examined associations between true-self attributions and perceived social norms and found that social norms was positively correlated with true self attribution [Study1: most of the rs ranged from .07 to .21; Study 2: all rs ranged from .10 to .30]. Supplementary, materials, raw data and analysis files/code are available here: https://osf.io/9fvtq/ .
Gilad Feldman
added 2 research items
[IMPORTANT: Method and results were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection. For the purpose of the simulation, we wrote things in past tense, but no pre-registration or data collection took place yet.] Weiner et al. (1988) found that compared to mental-behavioral stigmas, physically based stigmas were perceived as less controllable, more stable (irreversible) and were therefore associated with more pity, less anger, and more willingness to help. We conducted a pre-registered replication and extension of Experiment 2 by Weiner et al. (1988) with an American online Amazon Mechanical Turk sample (N = 1000). Our replication [failed to find/found] support for the original findings on the associations between stigma source, stability, emotional reactions, and willingness to help [summary effect sizes + CIs will be added here]. Extending the replication, we tested the model for four new stigmas prevalent in the last decade and reassessed the original’s categorizations of stigmas sources. Overall, participants’ categorizations of stigmas by source [matched/did not match] with the original’s, and we [found/failed to find] support for the relationship between stigma source and the attribution-affect-help judgment model. Materials, data, and code are available on the OSF: https://osf.io/gwcbt/.
People aim to diversify choices evenly resulting in a phenomenon coined “partition dependence” - partitioning options in a choice-set leads people to diversify allocations across and within partitions. We conducted a pre-registered replication and extensions of Experiments 1, 2, and 5 from the seminal paper on partition dependence by Fox et al. (2005) with an American online Amazon Mechanical Turk sample (N = 607). We found support for partition dependence in replication of Study 1 (original: d = 3.54 [3.10, 3.98]; replication: d = 2.12 [1.92, 2.32]), Study 2 (original: d = 1.34 [0.56, 2.12]; replication: d = .43 [.27, .59]), and Study 5 (original: Wald χ²=23.57; replication: Wald χ²= 26.6), yet no support for expertise as a moderator (original expertise: Wald χ²= 7.62; replication: Wald χ²= 0.04). Extending the replication, we found no support for trait desire for choice diversity as a predictor of partition dependence. Materials, data, and code are available on the OSF: https://osf.io/fujsv/ .
Gilad Feldman
added a research item
[IMPORTANT: Results were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection.] Ambiguity avoidance is a phenomenon that refers to people’s tendency to prefer choice options with lower ambiguity. Curley et al. (1986) attempted to uncover the psychological mechanisms underlying ambiguity aversion. We conducted a pre-registered very close replication of Curley et al. (1986)’s Study 1 and extensions of conceptual replications of their Studies 2 and 4, with an online US American Amazon Mechanical Turk sample (N =1000). In our replication of Study 1, we found support for ambiguity avoidance (V ̂_(Cramer ) = 0.34, 95% CI [.25, 1.00]) and the associations between risk aversion and ambiguity aversion (r(247) = .89, 95% CI [0.86, 0.91]). In our extensions, we found weak to no support for ambiguity aversion being impacted by hostility bias (VCramer = 0.40, 95% CI[0.31, 1.00]), anticipated future regret (VCramer = 0.17, 95%CI[0.00, 0.23]), or being evaluated by others (d = -0.09, 95% CI[-0.27, 0.08]). Supplementary materials, data, and code are available on the OSF: https://osf.io/ycxh3/.
Gilad Feldman
added a research item
[IMPORTANT: Method and results sections were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection. This is written in past tense yet no pre-registration or data collection have been conducted.] The relationship between risk and benefit is complex and has been studied in a number of different ways. Fischhoff et al. (1978) approached the relationship in a novel way, using psychometric analysis to measure public attitudes regarding the perceived risks and perceived benefits of certain technologies and activities, finding a negative correlation between perceived risk and perceived benefit. In a well-powered, pre-registered study we set out to conduct a replication of Fischhoff et al. (1978) with several adjustments and extensions. Using a simplified survey design with dummy data generated in Qualtrics (N = 1000) and improved statistical testing, we found [weak to no] empirical support for the negative correlation between perceived risk and perceived benefit. Extending the study, we added (1) items related to the ongoing Covid-19 pandemic to a modified list of technologies and activities from the original for which, using dummy data generated in Qualtrics (N = 1000), we found [weak to no] empirical support for the negative relationship between perceived risk and perceived benefit and (2) a within subjects condition to the study, which we will analyze for empirical support for the negative correlation between perceived risk and perceived benefit. Supplementary, materials, data, and analysis files/code are available here: https://osf.io/hcvmz/.
Gilad Feldman
added 2 research items
[IMPORTANT: Results were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection.] The appraisal-tendency framework proposed that specific emotions predispose individuals to appraise future events corresponding to the core appraisal themes of the emotions. In a pre-registered experiment with an American online Amazon Mechanical Turk sample (N = 700), we conducted an independent close replication of Experiments 1, 2, and 3 in Lerner and Keltner (2001). Our replication [failed to find/found] support for the original findings regarding associations between dispositional emotions and two risk-relevant measures: risk preference and risk optimism [summary effects sizes and CIs]. Extending the replication, we added hope as one dispositional emotion and [failed to find/found] support for the assumptions of the appraisal-tendency framework [effects sizes and CIs]. Materials, data, and code were made available on: https://osf.io/t5kz9/ .
The phenomenon that contemplating future events elicits stronger emotions than contemplating past events has been coined “temporal value asymmetry” (TVA) (Caruso et al. 2008). We conducted very close replications of three experiments derived from two influential TVA papers: Studies 1 and 4 in Caruso et al. (2008), demonstrating TVA in monetary valuation, and Study 1 in Caruso (2010), demonstrating TVA in moral judgment. We also attempted to conceptually replicate whether TVA in monetary valuation would extend to moral judgments. We failed to find support for TVA in monetary valuation (Caruso et al., 2008). We also failed to find support for TVA in moral judgments (Caruso, 2010) and in our conceptual extension. Exploratory analyses excluding potential outliers and z-transforming the dependent variable were consistent with our preregistered analyses. We discuss potential explanations for our results and future directions for research about the effects of time on judgments of value and morality.
Gilad Feldman
added a research item
[IMPORTANT: Results were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection.] Double distancing is the phenomenon that an inconsistency between behavior and moral values results in a state of an ethical dissonance, leading to harsher judgments of ethically questionable behavior and stronger tendency for virtuous self-presentation. In a pre-registered experiment with an American online Amazon Mechanical Turk sample (N = 800), we conducted a replication of Studies 1, 2 and 3 from Barkan et al. (2012). Our replication [failed to find/found] support for the original findings regarding associations between the state of ethical dissonance and harsher moral judgement towards others [summary effect sizes+CIs will be added here]. Thus, we found [weak to no / weak / medium / strong] empirical support for the hypothesis that people engage in double distancing in response to ethical dissonance. Supplementary, materials, raw data and analysis files/code are available here: https://osf.io/xj5pd/.
Gilad Feldman
added 2 research items
[IMPORTANT: Results were written in past tense using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection.] Numeracy is individuals’ capacity to understand and process basic probability and numerical information required to make decisions. We conducted a pre-registered replication and extension of Peters et al. (2006) examining associations between numeracy and positive-negative framing (Experiment 1), frequency-percentage framing (Experiment 2), ratio effect (Experiment 3), and loss vs. no-loss (Experiment 4). We collected data with an online US American Amazon Mechanical Turk sample (N =850). Our replication [failed to find/found] support for the original findings regarding associations between numeracy and four decision-making effects: [summary effect sizes+CIs will be added here]. Extending the replication, we [found/failed to find] support for an association between numeracy and confidence [summary effect sizes+CIs will be added here]. Materials, data, and code are available on: https://osf.io/4hjck/.
[IMPORTANT: Method and results sections were written using a randomized dataset produced by Qualtrics to simulate what these sections will look like after data collection. These will be updated following the data collection. This is written in past tense yet no pre-registration or data collection have been conducted.] Mental accounting, the internal categorization system individuals adopt to manage their financial activities, makes people prone to irrational decision-making. In a pre-registered study with an American online Amazon Mechanical Turk sample (N = 1000), we replicated 17 mental accounting problems reviewed in Thaler (1999). Out of the 17 problems, we found empirical support for X with effect sizes ranging from X.XX [X.XX, X.XX] to X.XX [X.XX, X.XX], and no empirical support for Y with effect sizes ranging from X.XX [X.XX, X.XX] to X.XX [X.XX, X.XX]. Extending the replication, we provided an initial test of four predictions not previously empirically tested that were described in Thaler’s (1999) paper as predictions. The replications and extensions examined different parts of the mental accounting theory and the results were interpreted separately. Materials, dataset, and analysis code were made available on the OSF: https://osf.io/pwa68/.
Gilad Feldman
added a research item
People tend to stick with a default option instead of switching to another option. For instance, Johnson and Goldstein (2003) found a default effect in an organ donation scenario: if organ donation is the default option, people are more inclined to consent to it. Johnson et al. (2002) found a similar default effect in a health-survey scenarios: if receiving more information about your health is the default, people are more inclined to consent to it. Much of the highly cited, impactful work on these default effects, however, has not been replicated in well-powered samples. In two well-powered samples (N = 1920), we conducted a close replication of the default effect in Johnson and Goldstein (2003) and in Johnson, Bellman, and Lohse (2002). We successfully replicated Johnson and Goldstein (2003). In an extension of the original findings, we also show that default effects are unaffected by the permanence of these selections. We, however, failed to replicate the findings of Johnson, Bellman, and Lohse’s (2002) study; we did not find evidence for a default effect. We did, however, find a framing effect: participants who read a positively-framed scenario consented to receive health-related information at a higher rate than participants who read a negatively framed scenario. We also conducted a conceptual replication of Johnson et al. (2002) that was based on an organ-donation scenario, but this attempt failed to find a default effect. Our results suggest that default effects depend on framing and context. Materials, data, and code are available on: https://osf.io/8wd2b/.
Gilad Feldman
added a research item
Heyman and Ariely (2004) demonstrated that the expected effectiveness of soliciting help varied depending on the "market", a money market represented by cash rewards versus a social market represented by goods as rewards. They showed that, as cash rewards increase, individuals expected others to be more willing to help, yet, when offering social goods as rewards such as candy, expected willingness to help was insensitive to rewards’ monetary worth. We conducted two pre-registered replication studies (total: N = 3302, MTurk/Prolific) of Study 1 in Heyman and Ariely (2004) and found support for one of their main claims that people are more sensitive to worth when the reward is cash than goods. However, the rewards' monetary worth impacted expected willingness to help even in social markets, deviating from the original findings. Extensions further compared between-subject and within-subject designs, examined perceived affect (joy and regret), and added a new control condition. We concluded that higher compensation is generally perceived as better when soliciting help, yet more so for the money market cash rewards than for the social market goods rewards. All materials, data, and code are provided on https://osf.io/y9p7u/
Gilad Feldman
added a research item
Above-and-below-average effects are well-known phenomena that arise when comparing oneself to others. Kruger (1999) found that people rate themselves as above average for easy abilities and below average for difficult abilities. We conducted a successful pre-registered replication of Kruger’s (1999) Study 1, the first demonstration of the core phenomenon (N = 756, US MTurk workers). Extending the replication to also include a between-subject design, we added two conditions manipulating easy and difficult interpretations of the original ability domains, and with an additional dependent variable measuring perceived difficulty. We observed an above-average-effect in the easy extension and below-average-effect in the difficult extension, compared to the neutral replication condition. Both extension conditions were perceived as less ambiguous than the original neutral condition. Overall, we conclude strong empirical support for Kruger’s above-and-below-average effects, with boundary conditions laid out in the extensions expanding both generalizability and robustness of the phenomenon. All materials, data, and code are available on: https://osf.io/7yfkc/ .
Gilad Feldman
added a research item
The temporal pattern of regret is the phenomenon that people perceive or experience stronger regret over action compared to inaction in the short-term, yet stronger regret over inaction compared to action in the long term. Following mixed and null findings in the literature, we conducted replications and extension of Studies 1, 3, 4, and 5 in the classic Gilovich and Medvec (1994) which first demonstrated this phenomenon, with a single combined data collection in randomized display order with an online sample of Americans on MTurk (N = 988). We found support for the original findings using different designs in Studies 1, 3, and 4, yet with weaker effects. We failed to find support for such a pattern in Study 5. We discuss possible interpretations for these differences: the change in the meaning of action and inaction, or change in hypothetical versus real-life personal experiences. Extending the replications, we found support for stronger responsibility for action compared to inaction both in the short-term and the long-term. We conclude overall support for the effects, yet with follow-up work necessary to resolve the inconsistencies in findings. Pre-registration, materials, data, and code were made available on: https://osf.io/7m3q2/
Gilad Feldman
added a research item
[IMPORTANT: This is a Registered Report Stage 1 before data collection. Written in past tense as a template to simulate what the final manuscript will look like. No pre-registration or data collection have been conducted.] Human beings have a fundamental need to connect with others. Epley, Akalis, et al. (2008) found that participants higher in chronic loneliness had a stronger tendency to anthropomorphize non-human objects, presumably for fulfilling unmet needs for social connection. We conducted a close replication of Epley, Akalis, et al. (2008): Based on the setup of their Study 1, we examined the correlations between loneliness and anthropomorphism of technological gadgets (original Study 1) and pets (original Study 3) and belief in supernatural beings (original Study 2), with a large U.S. sample recruited from MTurk (n = [XX; target n is 1,000]). Meanwhile, we extended the replication by examining the association between belief in free will and anthropomorphism. We found [weak-to-no / weak / medium / strong] empirical support for the original conclusion / our extensions. […]. All materials, data, and code are available on https://osf.io/2sb7x/.
Gilad Feldman
added a research item
Risks and benefits are negatively related in people’s minds. Finucane et al. causally demonstrated that increasing risks of a hazard leads people to judge its benefits as lower. Vice versa, increasing benefits leads people to judge its risks as lower (original: r = −.74 [−0.92, −0.30]). This finding is consistent with an affective explanation, and the negative relationship is often presented as evidence for an affect heuristic. In two well-powered studies, using a more stringent analytic strategy, we replicated the original finding. We observed a strong negative relationship between judgments of risks and benefits across three technologies, although we do find that there was no change in risks when highlighting low benefits. We note that risks seem to be more responsive to manipulation (as opposed to benefits) and find evidence that the negative relationship can depend on incidental mood. We provided materials, data sets, and analyses on https://osf.io/sufjn/?view_only=6f8f5dc6ff524149a4ed5c6de9296ae8 .
Gilad Feldman
added a research item
We attempted a pre-registered replication and extension of Studies 1, 2, and 3 from Pronin and Ross ‎‎(2006) regarding the effects of social and temporal distance on trait attributions with an online American ‎Amazon MTurk sample (N = 911). We concluded mixed findings. We found support for the original findings: ‎participants attributed more dispositional traits to others compared to themselves, although with weaker ‎effects (original: f = 0.35, 95% CI [0.09, 0.61]; replication: f = 0.10, 95% CI [0.03, 0.16]). Also, similar to the ‎original, we found that participants tended to attribute a favorable ratio of positive traits when making ‎self-assessments (original: f = 0.77, 95% CI [0.29, 1.25]; replication: f = 0.87, 95% CI [0.50, 1.24]). ‎However, unlike the original, we failed to find support for the core hypothesis that participants would ‎ascribe more dispositional traits to their temporally distant selves compared to their present self (original: f ‎‎= 0.54, 95% CI [0.27, 0.77]; replication: f = 0.01, 95% CI [0.00, 0.05]). Furthermore, in contrast to the ‎original, we found that the positive traits ratio increases with temporal distance (original: f = 0.15, 95% CI ‎‎[0.00, 0.31]; replication: f = 0.32, 95% CI [0.22, 0.41] in opposite direction). Contrary to our hypothesis, in ‎an extension we found that people were more likely to ascribe a greater ratio of positive traits to their ‎friends than to themselves (ξ=0.3, 95% CI [0.21, 0.38]). All materials, data, and code are provided on: ‎https://osf.io/gs2rx/.
Gilad Feldman
added a research item
The better-than-average effect refers to the tendency to rate oneself as better than the average person on desirable traits and skills. In a classic study, Svenson (1981) asked participants to rate their driving safety and skill compared to other participants in the experiment. Results showed that the majority of participants rated themselves as far above the median, despite the statistical impossibility of more than 50% of participants being above the median. We report a preregistered, well-powered (total N = 1,203), very close replication and extension of the Svenson (1981) study. Our results indicate that the majority of participants rated their driving skill and safety as above average. We added different response scales as an extension and findings were stable across all three measures. Thus, our findings are consistent with the original findings by Svenson (1981). Materials, data, and code are available at https://osf.io/fxpwb/.
Gilad Feldman
added a research item
Is it better to save 4500 lives out of 11,000 or 4500 lives out of 250,000? Fetherstonhaugh et al. (1997) showed that people prefer the former: to save lives if they are a higher proportion of the total, a phenomenon they termed “psychophysical numbing”. We attempted to replicate Studies 1 and 2 of Fetherstonhaugh et al. (1997) (5 data collections, total N = 4799, MTurk and Prolific, USA and UK), and added several extensions (e.g., donation amounts, procedural differences, and individual-level ideology and knowledge). We found mixed support, with two successful replications of Study 2 that indeed showed psychophysical numbing (original: η²p = 0.55, 90% CI [0.45, 0.62], Study 2a: η²p = 0.62, 90% CI [0.58, 0.66], Study 2b: η²p = 0.24, 90% CI [0.21, 0.27], all in same direction), yet also three unsuccessful replications of Study 1 showing instead an opposite psychophysical sensitization, a preference for saving a smaller proportion of lives (original effect size: η²p = 0.14, 90% CI [0.02, 0.28], replications: Study 1a: η²p = 0.06, 90% CI [0.02, 0.10], Study 1b: η²p = 0.21, 90% CI [0.17, 0.26]; Study 1c: η²p = 0.13, 90% CI [0.08, 0.17], all in the opposite direction). We discuss theoretical implications and potential drivers of psychophysical numbing and sensitization, including evaluation mode, comparison procedure, ideology, knowledge, and prioritizing of one's own country, and practical implications for research on perceptions of charity, aid effectiveness, and donations. Materials, preregistrations, data, and analyses are available at https://osf.io/786jg/.
Gilad Feldman
added a research item
The moral typecasting hypothesis proposes that the perception of moral agency (i.e., the ‎capacity of doing good or evil) is inversely related to that of moral patiency (i.e., the capacity of ‎being a target of good and evil). In this Registered Report Stage 1, we report our plan to conduct ‎well-powered (total N = []) direct replications of four studies in K. Gray and Wegner (2009), ‎where the typecasting hypothesis was initially put forward and systematically examined. We ‎plan to extend the replications by including additional agency and patiency measures as well as ‎experimental conditions to test the robustness of the original findings. Our results suggest ‎that …‎
Gilad Feldman
added a research item
Above-and-below-average effects are well-known phenomena that arise when comparing ‎oneself to others. Kruger (1999) found that people rate themselves as above average for easy ‎abilities and below average for difficult abilities. We conducted a successful pre-registered ‎replication of Kruger’s (1999) Study 1, the first demonstration of the core phenomenon (N = ‎‎756, US MTurk workers). Extending the replication to also include a between-subject design, ‎we added two conditions manipulating easy and difficult interpretations of the original ability ‎domains, and with an additional dependent variable measuring perceived difficulty. We ‎observed an above-average-effect in the easy extension and below-average-effect in the ‎difficult extension, compared to the neutral replication condition. Both extension conditions ‎were perceived as less ambiguous than the original neutral condition. Overall, we conclude ‎strong empirical support for Kruger’s above-and-below-average effects, with boundary ‎conditions laid out in the extensions expanding both generalizability and robustness of the ‎phenomenon. All materials, data, and code are available on: https://osf.io/7yfkc/
Gilad Feldman
added a research item
Bias Blind Spot (BBS) is the phenomenon that people tend to perceive themselves as less susceptible to biases than others. In three pre-registered experiments (overall N = 969), we replicated two experiments of the first demonstration of the phenomenon by Pronin et al. (2002). We found support of the BBS hypotheses, with effects in line with findings in the original study: Participants rated themselves as less susceptible to biases than others (d = -1.00 [-1.33, -0.67]). Deviating from the original, we found an unexpected effect that participants rated themselves as having fewer shortcomings (d = -0.34 [-0.46, -0.23]), though there was support for the target's main premise that BBS was stronger for biases than for shortcomings (d = -0.43 [-0.56, -0.29]). Extending the replications, we found that beliefs in own free will were positively associated with BBS (r ~ 0.17–0.22) and that beliefs in both own and general free will were positively associated with self-other asymmetry related to personal shortcomings (r ~ 0.16–0.24). Materials, datasets, and code are available on https://osf.io/3df5s/.
Gilad Feldman
added a research item
Rottenstreich and Hsee (2001) proposed an s-shaped weighting function where affect-rich ‎outcomes are more preferential to affect-poor outcomes when the probability of obtaining ‎the outcome is low, whilst affect-poor outcomes are more preferable under certainty. Two ‎pre-registered experiments (N = [X] Americans via MTurk and [X] (UK) via Prolific) each ‎replicated and extended all three original experiments. Signals for the reported effects were ‎‎[un/detected] and [smaller than/similar to/larger than] the effect sizes reported in the ‎original article. Exploring two extensions, [similar/different in/consistencies] were ‎identified in the additional response options and the affect-manipulations were considered ‎to be [un/successful/inconsistent]. The replication efforts provided [in/consistent/no] ‎support the S-shaped weighting function originally proposed and observed. All materials, ‎data, and analysis code are available here: ‎https://osf.io/9vkat/?view_only=4dc8e3bc3f24465eaf312c35efe13e84 .
Gilad Feldman
added a research item
The phenomenon that contemplating future events elicits stronger emotions than contemplating ‎past events has been coined "temporal value asymmetry (TVA)” (Caruso et al. 2008). We ran ‎very close replications of three experiments derived from two influential TVA papers: Studies ‎‎1 and 4 in Caruso et al. (2008), demonstrating TVA in monetary valuation, and Study 1 in ‎Caruso (2010), demonstrating TVA in moral judgment. We also attempted to conceptually ‎replicate whether TVA in monetary valuation would extend to moral judgments. We failed to ‎find support for TVA in monetary valuation (Caruso et al., 2008; Study 1: d = 0.41, 95% CI ‎‎[0.04, 0.76], replication: d = 0.03, 95% CI [-0.24, 0.30]; Study 4: ηp2 = 0.05, 95% CI [0.01, ‎‎0.13], replication: ηp2 = 0.00, 95% CI [0.00, 0.01]). We also failed to find support for TVA in ‎moral judgments (Caruso, 2010; Study 1: d = 0.43, 95% CI [0.06, 0.80], replication: d = 0.13, ‎‎95% CI [-0.06, 0.32]) and in our conceptual extension. Exploratory analyses excluding potential ‎outliers and z-transforming the dependent variable were consistent with our preregistered ‎analyses. We discuss potential explanations for our results and future directions for research ‎about the effects of time on judgments of value and morality. All data, code, and materials are ‎available on: https://osf.io/xcy9f/
Gilad Feldman
added 2 research items
Hindsight bias refers to the tendency to perceive an event outcome as more probable after being ‎informed of that outcome. We conducted very close replications of two classic experiments of ‎hindsight bias and a conceptual replication testing hindsight bias regarding the perceived ‎replicability of hindsight bias. In Study 1 (N = 890), we replicated Experiment 2 in Fischhoff ‎‎(1975), and found support for hindsight bias in retrospective judgments (dmean = 0.60). In Study ‎‎2 (N = 608), we replicated Experiment 1 in Slovic and Fischhoff (1977), and found support for ‎hindsight bias in prospective judgments (dmean = 0.40). In Study 3 (N = 520) we found strong ‎support for hindsight bias regarding perceived likelihood of our replication of hindsight bias (d ‎‎= 0.43 ~ 1.03). We also included extensions examining surprise, confidence, and task difficulty, ‎yet found mixed evidence with weak to no effects. We concluded support for hindsight bias in ‎both retrospective and prospective judgments, and in evaluations of replication findings, and ‎therefore call for establishing measures to address hindsight bias in valuations of replication ‎work and interpreting research outcomes. All materials, data, and code, were shared on: ‎https://osf.io/nrwpv/.
[IMPORTANT: This is a Registered Report Stage 1 before data collection. Written in past tense as a template to simulate what the final manuscript will look like. No pre-registration or data collection have been conducted.] Moral licensing is the phenomenon that moral behavior serves to justify subsequent immoral or problematic behavior. Monin and Miller (2001, Study 2) demonstrated that an expression of intent to hire a candidate from disadvantaged groups in a first task was associated with stronger expressions of prejudice in a second task. We conducted a direct replication of their study with U.S. participants from MTurk (N = ). We found [weak-to-no / weak / medium / strong] empirical support for the original findings (summary of effects and CIs). We also introduced two extensions to the replications, and we found […] (summary of effects and CIs). All materials, data, and analysis scripts are shared at https://osf.io/phym3.
Gilad Feldman
added a research item
Self-interest is a central driver of attitudes and behaviors, but people also act against their ‎immediate self-interest through prosocial behaviors, voting incongruously with their finances, ‎or punishing others at personal cost. How much people believe that self-interest causes ‎attitudes and behaviors is important, because this belief may shape regulation, shared ‎narratives, and institutional structures. An influential paper claimed that people overestimate ‎the power of self-interest on others' attitudes and behavioral intentions (Miller & Ratner, ‎‎1998). We present two registered, close, and successful replications (U.S. MTurk, N = 800; ‎U.K. Prolific, N = 799) that compared actual to estimated intentions, with open data and code. ‎Consistent with the original article, participants overestimated the impact of payment on blood ‎donation in Study 1, ds = 0.59 [0.51, 0.66], 0.57 [0.49, 0.64], and overestimated the importance ‎of smoking status for smoking policy preferences in Study 4, ds = 0.75 [0.59, 0.90], 0.84 [0.73, ‎‎0.96]. These replications included two extensions: 1) communal orientation as a moderator of ‎overestimation and 2) a more detailed measure of self-interest in Study 4 (ordinal smoking ‎status). Communal orientation did not predict overestimation, and the ordinal smoking measure ‎yielded similar results to the main study. Verifying the overestimation error informs behavioral ‎theories across several fields and has practical implications for institutions that require trust ‎and cooperation. All materials, data, and code are available at https://osf.io/57mdc/
Gilad Feldman
added a research item
Status quo bias refers to people’s general preference to stick to, or continue with, a previously ‎chosen option. In two pre-registered experiments with U.S. participants recruited on the ‎Amazon Mechanical Turk (n1 = 311, n2 = 316), we attempted to replicate four decision ‎scenarios (Question 1, 2, 4, and 6) from Samuelson and Zeckhauser (1988), the seminal article ‎that provided the first experimental demonstration of the status quo bias. We found strong ‎empirical support for the status quo bias in three decision scenarios out of the four, including ‎budget allocation (Scenario 1/Question 1 in the original article), investment portfolios ‎‎(Scenario 3/Question 2), and college job offers (Scenario 4/Question 4). However, we failed to ‎find substantial support for the status quo bias in the wagon color choice scenario (Scenario ‎‎2/Question 6). We discuss the implications of our results and possible explanations using ‎multiple accounts put forward in the status quo bias literature.‎
Gilad Feldman
added a research item
Kruger, Wirtz, van Boven, and Altermatt (2004) described the effort heuristic as the tendency to evaluate the quality and the monetary value of an object as higher if the production of that object was perceived as involving more effort. We attempted two preregistered replications (total N = 1405; U.S. American participants from MTurk and Prolific) of their Experiments 1 and 2. Our first replication using an MTurk sample found support for the original’s findings regarding Experiment 2, yet failed to find support for the original’s findings in Experiment 1. Our second revised attempt of Experiment 1 on Prolific was mixed with more nuanced findings, showing support for an effort heuristic effect for liking/quality and no support for monetary value.
Gilad Feldman
added a research item
We attempted a pre-registered replication and extension of Studies 1, 2, and 3 from Pronin and Ross ‎‎(2006) regarding the effects of social and temporal distance on trait attributions with an online American ‎Amazon MTurk sample (N = 911). We concluded mixed findings, likely failed replication. We found support ‎for the original findings that participants would attribute more dispositional traits to others compared to ‎themselves, although with weaker effects (original: f = 0.35, 95% CI [0.09, 0.61]; replication: f = 0.10, 95% ‎CI [0.03, 0.16]). Also, similar to the original, we found that participants tend to attribute a favorable ratio ‎of positive traits when making assessments of themselves (original: f = 0.77, 95% CI [0.29, 1.25]; ‎replication: f = 0.87, 95% CI [0.50, 1.24]). However, unlike the original, we failed to find support for the ‎core hypothesis that participants would ascribe more dispositional traits to their temporally distant selves ‎compared to their present self (original: f = 0.54, 95% CI [0.27, 0.77]; replication: f = 0.01, 95% CI [0.00, ‎‎0.05]). Furthermore, in contrast to the original, we found that the positive traits ratio increases with ‎temporal distance (original: f = 0.15, 95% CI [0.00, 0.31]; replication: f = 0.32, 95% CI [0.22, 0.41] in ‎opposite direction). Contrary to our hypothesis, in an extension we found that people were more likely to ‎ascribe a greater ratio of positive traits to their friends than to themselves (ξ=0.3, 95% CI [0.21, 0.38]).
Gilad Feldman
added a research item
We conducted a preregistered close replication and extension of Studies 1, 2, and 4 in Hsee ‎‎(1998). Hsee found that when evaluating choices jointly, people compare and judge the option ‎higher on desirable attributes as better ("more is better"). However, when people evaluate ‎options separately, they rely on contextual cues and reference points, sometimes resulting in ‎evaluating the option with less as being better ("less is better"). We found support for "less is ‎better" across all studies (N=403; Study 1 original d=0.70 [0.24,1.15], replication d=0.99 ‎‎[0.72,1.26]; Study 2 original d=0.74 [0.12,1.35], replication d=0.32 [0.07,0.56]; Study 4 ‎original d=0.97 [0.43,1.50], replication d=0.76 [0.50,1.02]), with weaker support for "more is ‎better" (Study 2 original d=0.92 [0.42,1.40], replication d=0.33 [.23,.43]; Study 4 original ‎d=0.37 [0.02,0.72], replication d=0.09 [-0.05,0.23]). Against predictions, an extension to Study ‎‎1 found “less is better” effect in the joint condition—undermining evidence for the evaluability ‎hypothesis. Materials/data/code: ‎https://osf.io/xfysr/?view_only=524c91c1f1be474eabeacfc7aa62abad
Gilad Feldman
added a research item
The decoy effect refers to the phenomenon whereby an inferior, unpreferable option reverses ‎people’s preferences and increases the choice share of a targeted option. In two pre-registered ‎experiments with an Amazon Mechanical Turk (MTurk) sample (N after exclusion = 1,001), we ‎attempted to replicate Experiment 1 from Ariely and Wallsten (1995) (Study 1) and Experiment ‎‎1 from Connolly, Reb, and Kausel (2013) (Study 2). We failed to replicate the original results in ‎Study 1. The observed effects were not in the predicted direction, and their sizes were trivial. ‎We replicated the decoy effect in Study 2, yet with a much smaller effect size than in the ‎original. In addition, we concluded inconclusive evidence for the central hypothesis of the ‎original study that regret salience weakens the decoy effect. We found some indication for a ‎weak reduction, yet our sample size did not provide adequate power to detect this difference. ‎Extending the replication in Study 2, we tested whether making salient the low reversibility of ‎decisions can have a similar impact as regret salience. We again found indication for an effect ‎in the predicted direction, yet the effect was too weak to be detected given our sample size. We ‎discuss potential reasons for the discrepancies between the original and the replication results, ‎as well as the implications. All materials, data, and analysis codes are available at ‎https://osf.io/vsbzk .‎
Gilad Feldman
added 2 research items
Shafir, Diamond, and Tversky (1997) described money illusion as people’s inclination to ‎think of money without taking inflation sufficiently into account, i.e., in nominal terms ‎rather than in real terms. We successfully replicated Problems 1 to 4 of Shafir, Diamond, ‎and Tversky’s study (1997) on money illusion (MTurk; N = 604). We found effect sizes in ‎line with the original ones for assessments of income (Problem 1; original: Cramer’s V = ‎‎0.26, 95% CI [0.17, 0.37]; replication: V = 0.28 [0.21, 0.36]), transactions (Problem 2; ‎original: 48% [42%, 52%]; replication: 70% [66%, 73%]), sales and consumption (Problem ‎‎3; original buy: 38% [33%, 43%]; replication buy: 47% [43%, 51%]; original sell: 43%% ‎‎[38%, 48%]; replication sell: 43% [39%, 47%]), and contracts (Problem 4; original: V = ‎‎0.25 [0.13, 0.42]; replication: V = 0.17 [0.10, 0.25]). With an added extension, we found no ‎support for the notion that knowing about or correctly estimating the inflation rate affected ‎money illusion. We discuss theoretical implications for the study of money illusion and the ‎psychological meaning of inflation, as well as for the understanding of the antecedents of ‎money illusion. All data, code, and materials are available on https://osf.io/rv9mw/‎ . ‎
Gilad Feldman
added a research item
Self-interest is a central driver of attitudes and behaviors, but people also act against their ‎immediate self-interest through prosocial behaviors, voting incongruously with their finances, ‎or punishing others at personal cost. How much people believe that self-interest causes ‎attitudes and behaviors is important, because this belief may shape regulation, shared ‎narratives, and institutional structures. An influential paper claimed that people overestimate ‎the power of self-interest on others' attitudes and behavioral intentions (Miller & Ratner, ‎‎1998). We present two registered, close, and successful replications (U.S. MTurk, N = 800; ‎U.K. Prolific, N = 799) that compared actual to estimated intentions, with open data and code. ‎Consistent with the original article, participants overestimated the impact of payment on blood ‎donation in Study 1, ds = 0.59 [0.51, 0.66], 0.57 [0.49, 0.64], and overestimated the importance ‎of smoking status for smoking policy preferences in Study 4, ds = 0.75 [0.59, 0.90], 0.84 [0.73, ‎‎0.96]. These replications included two extensions: 1) communal orientation as a moderator of ‎overestimation and 2) a more detailed measure of self-interest in Study 4 (ordinal smoking ‎status). Communal orientation did not predict overestimation, and the ordinal smoking measure ‎yielded similar results to the main study. Verifying the overestimation error informs behavioral ‎theories across several fields and has practical implications for institutions that require trust ‎and cooperation. All materials, data, and code are available at osf.io/57mdc/‎
Gilad Feldman
added a research item
We conducted a replication of Shafir (1993) who showed that people are inconsistent in their ‎preferences when faced with choosing versus rejecting decision-making scenarios. The effect ‎was demonstrated using an enrichment paradigm, asking subjects to choose between enriched ‎and impoverished alternatives, with enriched alternatives having more positive and negative ‎features than the impoverished alternative. Using eight different decision scenarios, Shafir ‎found support for a compatibility principle: subjects chose and rejected enriched alternatives in ‎choose and reject decision scenarios (d = 0.32 [0.23, 0.40]), respectively, and indicated greater ‎preference for the enriched alternative in the choice task than in the rejection task (d = 0.38 ‎‎[0.29, 0.46]). In a preregistered very close replication of the original study (N = 1026), we found ‎no consistent support for the hypotheses across the eight problems: two had similar effects, two ‎had opposite effects, and four showed no effects (overall d: -0.01 [-0.06, 0.03]). Seeking ‎alternative explanations, we tested an extension, and found support for the accentuation ‎hypothesis. Materials, data, and code, are available at https://osf.io/ve9bg/ . ‎
Gilad Feldman
added a research item
Is it better to save 4,500 lives out of 11,000 or 4,500 lives out of 250,000? Fetherstonhaugh ‎et al. (1997) showed that people prefer the former: to save lives if they are a higher ‎proportion of the total, a phenomenon they termed “psychophysical numbing”. We ‎attempted to replicate Studies 1 and 2 of Fetherstonhaugh et al. (1997) (5 data collections, ‎total N = 4799, MTurk and Prolific, USA and UK), and added several extensions (e.g., ‎donation amounts, procedural differences, and individual-level ideology and knowledge). ‎We found mixed support, with two successful replications of Study 2 that indeed showed ‎psychophysical numbing (original: η2p = 0.55, 90% CI [0.45, 0.62], Study 2a: η2p = 0.62, 90% ‎CI [0.58, 0.66], Study 2b: η2p = 0.24, 90% CI [0.21, 0.27], all in same direction), yet also ‎three unsuccessful replications of Study 1 showing instead an opposite psychophysical ‎sensitization, a preference for saving a smaller proportion of lives (original effect size: η2p = ‎‎0.14, 90% CI [0.02, 0.28], replications: Study 1a: η2p = 0.06, 90% CI [0.02, 0.10], Study 1b: ‎η2p = 0.21, 90% CI [0.17, 0.26]; Study 1c: η2p = 0.13, 90% CI [0.08, 0.17], all in the opposite ‎direction). We discuss theoretical implications and potential drivers of psychophysical ‎numbing and sensitization, including evaluation mode, comparison procedure, ideology, ‎knowledge, and prioritizing of one’s own country, and practical implications for research on ‎perceptions of charity, aid effectiveness, and donations. Materials, preregistrations, data, ‎and analyses are available at https://osf.io/786jg/. ‎
Gilad Feldman
added a research item
Mellers, Hertwig, and Kahneman (2001) conducted an adversarial collaboration to try and ‎resolve Hertwig’s contested view that frequency formats eliminate conjunction effects, and that ‎conjunction effects were largely due to semantic ambiguity. We conducted a pre-registered ‎well-powered very close replication ‎(N = 1032), ‎testing two personality profiles (Linda and ‎James) in a four between-subject design comparing unlikely and likely items to "and" and "and ‎or" conjunctions. Linda profile findings replicated conjunction effect and were consistent with ‎Tversky and Kahneman’s (1983) arguments for a representative heuristic. We found no support ‎for semantic ambiguity. The results of James profile were a likely failure, with no conjunction ‎effect. We provided additional tests addressing possible reasons, in line with later literature ‎suggesting conjunction effects may be context sensitive. We discuss implications for research ‎on conjunction effects, and call for further well-powered pre-registered replications and ‎extensions of classic findings in judgment and decision-making.‎
Gilad Feldman
added a research item
Hsee and Zhang (2004) argued that when people face a decision and must predict future affective ‎states, they are often in a joint evaluation (JE) mode where direct comparisons between different ‎choice options are relatively easy. When actually experiencing (or predicting affective states for) ‎only one of these options, people are usually in single evaluation (SE) mode where direct ‎comparisons are more difficult. This situational difference in evaluation mode was observed to ‎lead to overpredictions of positive affect (happiness) when people were in JE in comparison to SE ‎for options that were quantitatively different, but there were no overpredictions for options that ‎were qualitatively different. This effect was coined distinction bias. In the present paper, we ‎replicated (and extended on) Studies 1 and 2 from Hsee and Zhang with 824 MTurk participants. ‎In Study 1 we replicated the original findings: Relative to people in SE, people in JE overpredicted ‎the happiness derived from quantitatively different hypothetical scenarios (i.e., selling 80 vs 160 ‎books, or 160 vs 240 books), but did not overpredict the difference between qualitatively different ‎hypothetical scenarios (i.e., selling 0 books vs 80 books; 0 being the implicit reference point that ‎makes the two scenarios qualitatively different). Study 2 failed to find support for the original ‎findings: People in JE did not consistently overpredict happiness for quantitatively different ‎scenarios (i.e., copy-pasting 25 negative words vs 10 negative words, or 10 positive words vs 25 ‎positive words). Taken together, the present paper provides mixed support for distinction bias. ‎
Gilad Feldman
added a research item
Malle and Knobe's (1997) Study 1 found that people exhibit a shared understanding of intentionality and apply it consistently in their judgments. The study found that different people tend to judge intentionality similarly, and that intentionality ratings were consistent across a set of behaviors from an actor's or an observer's perspective. Additionally, the presence or absence of a definition of intentionality as part of the study instruction did not seem to affect the intentionality judgments. We conducted two pre-registered replications of Study 1 (N=46; N=817). The replication results provide support for the findings of the original study. Consistent with the original study, we found high inter-rater agreement across perspective (actor vs. observer) experimental conditions; and no evidence for differences in intentionality ratings depending on whether a definition of intentionality was provided. We observed that actor perspective led to a higher average rating of intentionality than the observer perspective.
Gilad Feldman
added a research item
People tend to regard themselves as better than average. We conducted a replication and ‎extension of Alicke's (1985) classic study on trait dimensions in evaluations of self versus ‎others with U.S. American MTurk workers in two waves (total N = 1573; 149 total traits). ‎We successfully replicated the trait desirability effect, such that participants rated more ‎desirable traits as being more descriptive of themselves than of others (original: ηp2 = .78, ‎‎95% CI [.73, .81]; replication: sr2 = .54, 95% CI [.43, .65]). The effect of desirability was ‎stronger for more controllable traits (effect of desirability X controllability interaction on ‎self-other ratings difference, original: ηp2 = .21, 95% CI [.12, .28]; replication: sr2 = .07, ‎‎95% CI [.02, .12]). In an extension, we found that desirable traits were rated as more ‎common for others, but not for the self. Thirty-five years later, the better-than-average ‎effect appears to remain robust.‎
Gilad Feldman
added a research item
Hindsight bias refers to the tendency to perceive an event outcome as more probable after being ‎informed of that outcome. We conducted very close replications of two classic experiments ‎testing the hindsight bias and conceptual replication to test hindsight bias regarding the ‎replicability of hindsight bias. In Study 1 (N = 890), we replicated Experiment 2 in Fischhoff ‎‎(1975), and found support for hindsight bias in retrospective judgments (dmean = 0.60). In Study ‎‎2 (N = 608), we replicated Experiment 1 in Slovic and Fischhoff (1977), and found support for ‎hindsight bias in prospective judgments (dmean = 0.40). In Study 3 (N = 520) we found strong ‎support for hindsight bias regarding perceived likelihood of our replication of hindsight bias (d ‎‎= 0.43 ~ 1.03). We also included extensions examining surprise, confidence, and task difficulty, ‎yet found mixed evidence with weak to no effects. We concluded support for hindsight bias in ‎both retrospective and prospective judgments, and in evaluations of replication findings, and ‎therefore call for establishing measures to address hindsight bias in valuations of replication ‎work and interpreting research outcomes.‎
Gilad Feldman
added 2 research items
Risks and benefits are negatively related in people’s minds. Finucane et al. (2000) causally ‎demonstrated that increasing risks of a hazard leads people to judge its benefits as lower. Vice ‎versa, increasing benefits leads people to judge its risks as lower (original: r = -0.74[-0.92,-‎‎0.30]). This finding is consistent with an affective explanation and the negative relationship is ‎often presented as evidence for an affect heuristic. In two well-powered studies, using a more ‎stringent analytic strategy, we replicated the original finding. We observed a strong negative ‎relationship between judgments of risks and benefits across three technologies, although we do ‎find that there was no change in risks when highlighting low benefits. We note that risks seem ‎to be more responsive to manipulation (as opposed to benefits) and find evidence that the ‎negative relationship can depend on incidental mood. We provided materials, datasets, and ‎analyses on https://osf.io/sufjn/ .
People tend to regard themselves as better than average. We conducted a replication and ‎extension of Alicke's (1985) classic study on trait dimensions in evaluations of self versus ‎others with U.S. American MTurk workers in two waves (total N = 1573; 149 total traits). ‎We successfully replicated the trait desirability effect, such that participants rated more ‎desirable traits as being more descriptive of themselves than of others (original: ηp2 = .78, ‎‎95% CI [.73, .81]; replication: sr2 = .54, 95% CI [.43, .65]). The effect of desirability was ‎stronger for more controllable traits (effect of desirability X controllability interaction on ‎self-other ratings difference, original: ηp2 = .21, 95% CI [.12, .28]; replication: sr2 = .07, ‎‎95% CI [.02, .12]). In an extension, we found that desirable traits were rated as more ‎common for others, but not for the self. Thirty-five years later, the better-than-average ‎effect appears to remain robust.‎
Ignazio Ziano
added a research item
The ownership effect is the phenomenon that owning an object increases liking and ‎perceived value of that object (Beggan, 1992). We conducted close replications of three ‎ownership effect experiments using different paradigms in two data collections (MTurk, ‎total n = 1312). We successfully replicated Nuttin’s (1987) name-letter effect with ‎participants rating a higher liking for letters of the alphabet included in their first names ‎‎(vs. letters not included) (Study 1: d = 1.08 to 1.42). We found partial support for Mandel ‎‎(2002), with participants indicating higher price for an object when they were the owners ‎‎(vs. non-owners) (original: d = 0.50; Study 3a: d = 0.65; Study 3b: d = 0.49), but mixed ‎findings regarding the hypothesized moderator. Finally, we failed to find support for Irmak, ‎Wakslak, and Trope’s (2013) study that showed differences in prices set by sellers/owners ‎and buyers (original: d = 0.99; Study 2a: d = 0.10; Study 2b: d = 0.01 to 0.06). Our results ‎suggest that ownership effects may depend on the paradigm of choice. We discuss potential ‎moderators of the ownership effect and suggest future research directions. Materials, ‎datasets, and code are available on https://osf.io/2cg3e/ .‎
Gilad Feldman
added a research item
Omission bias is the preference for harm caused through omissions over harm caused ‎through commissions. In a pre-registered experiment (N = 313), we successfully replicated ‎an experiment from Spranca, Minsk, and Baron (1991), considered a classic demonstration ‎of the omission bias, examining generalizability to a between-subject design with ‎extensions examining causality, intent, and regret. Participants in the harm through ‎commission condition(s) rated harm as more immoral and attributed higher responsibility ‎compared to participants in the harm through omission condition (d = 0.45 to 0.47 and d = ‎‎0.40 to 0.53). An omission-commission asymmetry was also found for perceptions of ‎causality and intent, in that commissions were attributed stronger action-outcome links and ‎higher intentionality (d = 0.21 to 0.58). The effect for regret was opposite from the classic ‎findings on the action-effect, with higher regret for inaction over action (d = -0.26 to -0.19). ‎Overall, higher perceived causality and intent were associated with higher attributed ‎immorality and responsibility, and with lower perceived regret. All materials are available ‎on: https://osf.io/9gsqe/ ‎
Gilad Feldman
added a research item
The ownership effect is the phenomenon that owning an object increases liking and ‎perceived value of that object (Beggan, 1992). We conducted close replications of three ‎ownership effect experiments using different paradigms in two data collections (MTurk, ‎total n = 1312). We successfully replicated Nuttin’s (1987) name-letter effect with ‎participants rating a higher liking for letters of the alphabet included in their first names ‎‎(vs. letters not included) (Study 1: d = 1.08 to 1.42). We found partial support for Mandel ‎‎(2002), with participants indicating higher price for an object when they were the owners ‎‎(vs. non-owners) (original: d = 0.50; Study 3a: d = 0.65; Study 3b: d = 0.49), but mixed ‎findings regarding the hypothesized moderator. Finally, we failed to find support for Irmak, ‎Wakslak, and Trope’s (2013) study that showed differences in prices set by sellers/owners ‎and buyers (original: d = 0.99; Study 2a: d = 0.10; Study 2b: d = 0.01 to 0.06). Our results ‎suggest that ownership effects may depend on the paradigm of choice. We discuss potential ‎moderators of the ownership effect and suggest future research directions. Materials, ‎datasets, and code are available on https://osf.io/2cg3e/ .‎
Gilad Feldman
added a research item
Status quo bias refers to people’s general preference to stick to, or continue with, a previously ‎chosen option. In two pre-registered experiments with U.S. participants recruited on the ‎Amazon Mechanical Turk (n1 = 311, n2 = 316), we attempted to replicate four decision ‎scenarios (Question 1, 2, 4, and 6) from Samuelson and Zeckhauser (1988), the seminal article ‎that provided the first experimental demonstration of the status quo bias. We found strong ‎empirical support for the status quo bias in three decision scenarios out of the four, including ‎budget allocation (Scenario 1/Question 1 in the original article), investment portfolios ‎‎(Scenario 3/Question 2), and college job offers (Scenario 4/Question 4). However, we failed to ‎find substantial support for the status quo bias in the wagon color choice scenario (Scenario ‎‎2/Question 6). We discuss the implications of our results and possible explanations using ‎multiple accounts put forward in the status quo bias literature.‎
Gilad Feldman
added a research item
Shafir, Diamond and Tversky (1997) described money illusion as people’ inclination to ‎think of money without taking inflation sufficiently into account, i.e., in nominal terms ‎rather than in real terms. We successfully replicated Problems 1 to 4 of Shafir, Diamond and ‎Tversky’s study (1997) on money illusion (MTurk; N = 604). We found effect sizes in line ‎with the original ones for assessments of income (Problem 1; original: Cramer’s V = 26, ‎‎95% CI [.17, .37]; replication: V = 28 [.21, .36]), transactions (Problem 2; original: 48% ‎‎[42%, 52%]; replication: 70% [66%, 73%]), sales and consumption (Problem 3; original ‎buy: 38% [33%, 43%]; replication buy: 47% [43%, 51%]; original sell: 43%% [38%, 48%]; ‎replication sell: 43% [39%, 47%]), and contracts (Problem 4; original: V = .25 [.13, .42]; ‎replication: V = .17 [.10, .25]). With an added extension, we found no support for the notion ‎that knowing about or correctly estimating the inflation rate affected money illusion. We ‎discuss theoretical implications for the study of money illusion and the psychological ‎meaning of inflation, and for the understanding of the antecedents of money illusion.‎
Gilad Feldman
added a research item
The decoy effect refers to the phenomenon whereby an inferior, unpreferable option reverses ‎people’s preferences and increases the choice share of a targeted option. In two pre-registered ‎experiments with an Amazon Mechanical Turk (MTurk) sample (n = 1053), we attempted to ‎replicate Experiment 1 from Ariely and Wallsten (1995) (Study 1) as well as Experiment 1 from ‎Connolly, Reb, and Kausel (2013) (Study 2). We failed to replicate the original results in Study ‎‎1. The observed effects were not in the predicted direction, and their sizes were trivial. We ‎replicated the decoy effect in Study 2, yet with a much smaller effect than in the original. In ‎addition, we concluded inconclusive evidence for their central hypothesis that regret salience ‎weakens the decoy effect. We found some indication for a weak reduction, yet our sample size ‎did not have adequate power to detect this difference. Extending the replication in Study 2, we ‎tested whether making salient the low reversibility of decisions can have a similar impact as ‎regret salience. We again found indication for an effect in the predicted direction, yet the effect ‎too weak to be detected given our sample size. We discuss potential reasons for the ‎discrepancies between the original results and ours, as well as their implications. All materials, ‎data, and analysis codes are available at https://osf.io/vsbzk .‎
Gilad Feldman
added 2 research items
We conducted a replication of Shafir (1993) who showed that people are inconsistent in their ‎preferences when faced with choosing versus rejecting decision-making scenarios. The effect ‎was demonstrated using an enrichment paradigm, asking subjects to choose between enriched ‎and impoverished alternatives, with enriched alternatives having more positive and negative ‎features than the impoverished alternative. Using eight different decision scenarios, Shafir ‎found support for his compatibility principle: subjects chose and rejected enriched alternatives ‎in choose and reject decision scenarios (d = 0.32 [0.23, 0.4]), respectively, and indicated greater ‎preference for the enriched alternative in the choice task than in the rejection task (d = 0.38 ‎‎[0.29, 0.46]). In a pre-registered very close replication of the original study (N = 1026), we ‎found no consistent support for the hypotheses across the eight problems: two had similar ‎effects, two had opposite effects, and four showed no effects (overall d: -0.01 [-0.06, 0.03]). ‎Seeking alternative explanations, we tested and found support for the accentuation hypothesis (b ‎‎= −0.51(0.044), p < .001). ‎
Royzman and Baron (2002) demonstrated that people prefer indirect harm to direct harm: ‎they judge actions that produce harm as a by-product to be more moral than actions that ‎produce harm directly. In two preregistered studies, we successfully replicated Study 2 of ‎Royzman and Baron (2002) with a Hong Kong student sample (N = 46) and an online ‎American Mechanical Turk sample (N = 314). We found consistent evidential support for ‎the preference for indirect harm phenomenon (d = 0.46 [0.26, 0.65] to 0.47 [0.18, 0.75]), ‎weaker than effects reported in the original findings of the target article (d = 0.70 [0.40, ‎‎0.99]). We also successfully replicated findings regarding reasons underlying a preference ‎for indirect harm (directness, intent, omission, probability of harm, and appearance of ‎harm). All materials, data, and code are available on: https://osf.io/ewq8g/
Gilad Feldman
added a research item
Hsee and Zhang (2004) argued that when people face a decision and must predict future ‎affective states, they are often in a joint evaluation (JE) mode where direct comparisons between ‎different choice options is relatively easy. When actually experiencing (or predicting affective ‎states for) only one of these options, people are usually in single evaluation (SE) mode where ‎direct comparisons are more difficult. This situational difference in evaluation mode was ‎observed to lead to overpredictions of positive affect (happiness) when people were in JE in ‎comparison to SE for options that were quantitatively different, but there were no overpredictions ‎for options that were qualitatively different. This effect was coined distinction bias. In the present ‎paper, we replicated (and extended on) Studies 1 and 2 from Hsee and Zhang with 824 MTurk ‎participants. In Study 1 we replicated the original findings: Relative to people in SE, people in JE ‎overpredicted the happiness derived from quantitatively different hypothetical scenarios (i.e., ‎selling 80 vs 160 books, or 160 vs 240 books), but did not overpredict the difference between ‎qualitatively different hypothetical scenarios (i.e., selling 0 books vs 80 books; 0 being the ‎implicit reference point that makes the two scenarios qualitatively different). Study 2 failed to ‎find support for the original findings: People in JE did not consistently overpredict happiness for ‎quantitatively different scenarios (i.e., copy-pasting 25 negative words vs 10 negative words, or 10 ‎positive words vs 25 positive words). Taken together, the present paper provides mixed support ‎for distinction bias. ‎
Gilad Feldman
added a research item