Content uploaded by Gilad Feldman
Author content
All content in this area was uploaded by Gilad Feldman on Jul 08, 2022
Content may be subject to copyright.
Revisiting the temporal pattern of regret in action versus inaction:
Replication of Gilovich and Medvec (1994) with extensions examining
responsibility
Siu Kit Yeung
University of Hong Kong, Hong Kong SAR, China
u3517520@connect.hku.hk / yskjdmmh@gmail.com
^Gilad Feldman
ORCID: 0000-0003-2812-6599
University of Hong Kong, Hong Kong SAR, China
gfeldman@hku.hk / giladfel@gmail.com
In press at Collabra: Psychology
Accepted for publication on July 8, 2022
^Corresponding author
Word: abstract – 193, manuscript – 9862 (excluding Tables and Figures, including
References and Abstract) / 11399 (including Tables, Figures, References, and Abstract)
Preregistration, data, code, and materials are publicly available: https://osf.io/7m3q2/
Author bios:
Siu Kit Yeung is an MPhil graduate of the University of Hong Kong psychology department,
and an incoming PhD student at the Chinese University of Hong Kong psychology
department. His research focuses on emotions, judgment and decision-making.
Gilad Feldman is an assistant professor with the University of Hong Kong psychology
department. His research focuses on judgment and decision-making.
Authorship declaration:
Kit wrote the pre-registrations, conducted data analyses, and wrote the manuscript, as part of
his mPhil thesis. Gilad Feldman supervised Siu Kit Yeung throughout, conducted the pre-
registrations, and ran data collection. Gilad and Kit jointly finalized the manuscript for
journal submission.
Corresponding author
Gilad Feldman, Department of Psychology, University of Hong Kong, Hong Kong SAR,
China; gfeldman@hku.hk
Contributor Roles Taxonomy
In the table below, we employ CRediT (Contributor Roles Taxonomy) to identify the roles of
the contributors. Check https://www.casrai.org/credit.html for more information about the
roles.
Role
Siu Kit Yeung
Gilad Feldman
Conceptualization
V
Pre-registration
V
Data curation
V
Formal analysis
V
Funding acquisition
V
Investigation
V
Pre-registration peer review / verification
V
Data analysis peer review / verification
Methodology
V
Project administration
V
Resources
Software
V
Supervision
V
Validation
Visualization
V
Writing-original draft
V
Writing-review and editing
V
Abstract
The temporal pattern of regret is the phenomenon that people perceive or experience stronger
regret over action compared to inaction in the short-term, yet stronger regret over inaction
compared to action in the long term. Following mixed and null findings in the literature, we
conducted replications and extension of Studies 1, 3, 4, and 5 in the classic Gilovich and
Medvec (1994) which first demonstrated this phenomenon, with a single combined data
collection in randomized display order with an online sample of Americans on MTurk (N =
988). We found support for the original findings using different designs in Studies 1, 3, and 4,
yet with weaker effects. We failed to find support for such a pattern in Study 5. We discuss
possible interpretations for these differences: our replication adjustments, the change in the
meaning of action and inaction, or change in hypothetical versus real-life personal
experiences. Extending the replications, we found support for stronger responsibility for
action compared to inaction both in the short-term and the long-term. We conclude overall
support for the effects, yet with follow-up work necessary to resolve the inconsistencies in
the findings of the Study 5 replication. Pre-registration, materials, data, and code were made
available on: https://osf.io/7m3q2/
Keywords: temporal pattern, judgment and decision-making, pre-registered replication,
regret, action-inaction
Gilovich and Medvec (1994) Replication and Extension 2
Revisiting the Temporal Pattern of Regret:
Replication of Gilovich and Medvec (1994) with extensions examining responsibility
Background
The temporal pattern of regret regarding action and inaction was first demonstrated by
Gilovich and Medvec (1994) who showed that whereas people tend to experience stronger
regret for actions over inactions in the short term, they tend to experience stronger regret for
things they did not do over things they did when reflecting back on their lives. A large body
of literature has found consistent support for an action-effect, the phenomenon that people
associate stronger regret with action compared to inaction (e.g. Gleicher et al., 1990;
Kahneman & Tversky, 1982). However, Gilovich and Medvec (1994) suggested that this
classic effect is moderated by temporal distance, such that when retrospectively recalling
their lifetime and long-term regrets, people tend to associate stronger regret with inaction
than with action.
Over the years the literature has seen many mixed findings on temporal patterns in
regret (e.g. Bonnefon & Zhang, 2008; Byrne & McEleney, 2000; Feldman et al., 1999;
Towers et al., 2016), possibly due to differences in methods and scenarios. This suggests the
need for revisiting these classic effects with pre-registered replications.
We conducted direct replications and extensions of Studies 1, 3, 4, and 5 in Gilovich
and Medvec (1994). Our first goal was to conduct independent direct pre-registered well-
powered replications of the temporal pattern of regret. Our second goal was to use the same
base methods to extend these findings and examine whether a similar action-inaction pattern
of asymmetry would also be found regarding perceptions and experiences of personal
responsibility.
Gilovich and Medvec (1994) Replication and Extension 3
We begin by introducing the literature on the action-effect and the temporal action-
inaction effect. We then discuss the motivations for the current replication and outline
replication hypotheses and designs, with an introduction of our extension to attributions of
responsibility.
Temporal Pattern of Regret
Kahneman and Tversky (1982) were the first to demonstrate the action-effect, the
stronger regret associated with action over inaction, with many successful follow-up
demonstrations (e.g. Feeney & Handley, 2006; Gleicher et al., 1990; Landman, 1987). The
action-effect has been previously explained using several paradigms, such as the higher
perceived causality and responsibility associated with action (Kordes-de Vaal, 1996), and
using norm theory suggesting that actions are perceived as an exception to the norm of not
acting in such situations, and exceptions are more cognitively mutable than routines and
therefore associated with higher regret (Kahneman & Miller, 1986).
The classic experiments by Kahneman and Tversky (1982) demonstrated the action
effect by presenting participants with hypothetical short-term decision-making situations.
Gilovich and Medvec (1994) tested whether these results would extend to evaluations of real-
life long-term experiences. In their Studies 1 and 5, they found that for the retrospective and
lifetime reflections the action-effect reversed into an inaction-effect, in that participants
tended to report stronger regret for their inaction compared to action. In their Studies 3 and 4
they also demonstrated their findings using scenarios that were very similar to those of
Kahneman and Tversky (1982) when manipulating short-term versus long-term reflections.
In follow-up work, Gilovich and Medvec (1995) proposed, investigated, and discussed
several possible mechanisms for these effects, including mechanisms related to decrease in
intensity of action regrets and increase in intensity of inaction regrets overtime For example,
Gilovich and Medvec (1994) Replication and Extension 4
they suggested that people engage in more compensatory behavior for action regrets
compared to inaction regrets. Meaning, that people tend to do more to try and rectify their
action mistakes compared to inaction mistakes, explaining why inaction regrets may be
stronger in the long-run compared to action regrets which may be weakened over time.
Another possible explanation is that over time people seem to become more confident that
they would have succeeded if they had taken actions (Gilovich et al., 1993), as their memory
and/or concerns regarding outcome uncertainties or risks of action diminished over time
(Gilovich & Medvec, 1995). Furthermore, Gilovich and Medvec (1995) proposed that
intensity of inaction regrets may increase overtime as people perceive inaction mistakes seem
to result in more negative consequences, perhaps in a wider range of areas overtime
(Rajagopal et al., 2006). Moreover, the debate with Kahneman led to a coauthored adversarial
collaboration with three studies, in which Gilovich et al. (1998) concluded that action regrets
tend to primarily elicit hot emotions (e.g., anger) whereas inaction regrets tend to elicit
feelings of wistfulness (e.g., nostalgia) and despair (e.g., misery), which may be the cause of
the temporal differences. A follow-up conceptual replication and extension by Leach and
Plaks (2009) found support for the temporal pattern of regret, mediated by the higher level of
abstraction of distant inaction regret. We note that we did not set out to investigate the
mechanisms of the proposed temporal pattern of regret and to first focus on revisiting and
reassessing the core phenomenon.
Choice of article for replication: Gilovich and Medvec (1994)
We chose the Gilovich and Medvec (1994) article based on several factors: the
absence of direct replications, its impact, lack of statistical power in empirical evidence, and
mixed or null findings.
Gilovich and Medvec (1994) Replication and Extension 5
We chose to replicate Studies 1a, 3, 4, and 5 as these studies focused on the intensity
of regret rather than the number of action-inaction regrets, and were a better fit for our target
sample, as Study 2 involved face-to-face interviews with several groups of participants. Study
1b examining greatest lifetime regrets overlapped with the more comprehensive Study 5,
which manipulated temporal distance, and included
greatest regrets. Overall, the target studies for replication covered both scenario experiments
(Studies 3 and 4) and surveys regarding real-life experiences (Studies 1 and 5).
To the best of our knowledge, there have been no published direct replications of these
target studies. The article has been influential on research in social-cognitive psychology,
emotions, and decision-making. At the time of writing (December 2021), there were 549
Google Scholar citations of the article and many important follow-up theoretical and
empirical articles (e.g., Bonnefon & Zhang, 2008; Feldman et al., 1999; Gilovich & Medvec,
1995; Towers et al., 2016).
The original studies had small sample sizes (under 100 participants for 2-4
conditions), with similar samples in conceptual replications, and revisiting underpowered
classics is valuable in addressing possible concerns over false-positive rates (Christley,
2010). In addition, findings were not always consistent with some of the original hypotheses.
For example, Study 5 failed to find support for action-effect in the short-run, whereas Study 4
found support for action-effect in the short-run but failed to find support for a meaningful
reversal to inaction-effect in the long run.
We aimed to revisit the original findings to try and address mixed findings in follow-
up literature identifying boundary conditions, and findings that were not in support of the
temporal pattern. Leach and Plaks (2009) conducted a successful conceptual replication of
Gilovich and Medvec (1994) using scenario experiments, and found that the level of
Gilovich and Medvec (1994) Replication and Extension 6
abstraction mediated the temporal pattern of regret. Furthermore, Zeelenberg et al. (1998)
found support for temporal pattern of regret with a series of studies that coded interpersonal
regrets in the and real life regrets. Also, Bonnefon and Zhang (2008)
asked participants to one single regret only,
did not specify whether the event is the most regretful or not) and found that the difference
for short-term regrets was minimal (48% inaction) whereas long-term regrets were more
likely to be inactions. In contrast, several follow-up studies examining temporal patterns of
the action-effect indicated limited generalizability and identified various possible boundary
conditions. Byrne and McEleney (2000) failed to conceptually replicate scenario experiments
Studies 3 and 4 in Gilovich and Medvec (1994) adapting Kahneman and Tversky's (1982)
investor scenario. A plausible explanation is that in Byrne and McEleney (2000) scenario
experiments, the factual and counterfactual consequences were matched for the actor whereas
the counterfactual consequences might be perceived to possibly be better than factual
consequences for the non-actor. Byrne and McEleney (2000) argued that the temporal pattern
of action-
mental
(p. 1330). Moreover, Towers et al. (2016) asked participants about their single greatest regret
in life and found action regrets were more intense than inaction regrets, contradicting
Gilovich and Medvec (1994) findings. Towers et al. (2016) did not directly contrast actions
versus inactions, but rather compared intensities of regret coded as action or inaction in
reports of regretful events and measured temporal distance continuously but not categorically
(lifetime vs recent). Another highly cited article by Feldman et al. (1999) asked participants
about personal experiences of regrets and found that participants reported higher numbers of
long-term inaction regrets compared to action regrets but failed to find support for difference
Gilovich and Medvec (1994) Replication and Extension 7
in intensity of action-inaction regret, which is the focus of our replication (Studies 1, 3, 4 and
5 of Gilovich and Medvec, 1994).
The above studies differ from Gilovich and Medvec (1994) in methods or scenarios,
reaching different conclusions. It is unclear if the failure to support Gilovich and Medvec
(1994) is due to original results being unreplicable or the differences in methods or scenarios.
Gilovich and Medvec (1994) inspired later work with important possible implications
on regret. Later work proposed that inaction regrets may be more distressing and depressing
over a longer period (Broomhall et al., 2017; Gilovich & Medvec, 1995), perhaps because
negative feelings and senses of disquiet are stronger when one cannot fulfill the need for
action (Roese et al., 1999). There appears to be evidence that in the long run, people tend to
ruminate over inactions more compared to actions (Gilovich et al., 1995; Savitsky et al.,
1997), possibly because they perceive or imagine more possibilities of counterfactual
outcomes from inactions compared to actions with clearer links to outcomes (Leach & Plaks,
2009; Rajagopal et al., 2006). However, before getting into mechanisms and practical
implications to sort out this literature, we believe it is essential to revisit the classic effects
and assess their reliability and replicability (e.g., Brandt et al., 2014; Zwaan et al., 2018),
with preregistered high-powered direct replications and extensions.
Methods, hypotheses, and findings of the target article
The original Study 1 was conducted with adult participants on the telephone, asking
participants to compare intensity of their action and inaction regrets in general (Study 1A),
and compare intensity of their greatest action regret and greatest inaction regret (Study 1B).
The original Studies 3 (within-subject) and 4 (temporal distance as the between-subject
factor) were conducted with undergraduates and using scenario experiments that asked
participants to compare the intensity of regret of the decision-makers in the short run and the
Gilovich and Medvec (1994) Replication and Extension 8
long run. The original Study 5 was conducted with participants in public areas, asking
participants to compare the intensity of regret of their greatest action regret and greatest
inaction regret in the past week and in their lifetime. We did not include Study 1B in our
replication, as it consisted of questions on greatest lifetime regret, which overlapped with
those of Study 5.
We summarized the hypotheses in Table 1. The original authors hypothesized that
there would be stronger regret for inaction in the long run and stronger regret for action in the
short run. We provide more details regarding the original article in the supplementary. We
calculated Cramer V based on the information provided, reported in Supplementary Table 4.
Table 1
Gilovich and Medvec (1994): Summary of hypotheses
Study
Hypothesis
Study 1 real-life
Regret over past action-
inactions
Participants are more likely to report having
experienced regret for life's inactions compared to
life's actions.
Study 3 (within-subject)
Study 4 (between-subject)
Hypothetical scenarios
Participants are more likely to associate stronger
regret with recent actions than with recent
inactions.
Participants are more likely to associate stronger
regret with distant past inactions than with distant
past actions.
Study 5 real-life
Regret over recent versus
distant past action-inactions
Participants are more likely to experience stronger
regret over their most regrettable recent action than
over their most regrettable recent inaction.
Participants are more likely to experience stronger
regret over their most regrettable distant past
inaction than over their most regrettable distant past
action.
Gilovich and Medvec (1994) Replication and Extension 9
Extension: Responsibility
We aimed to extend the replication to investigate the generalizability of the temporal
action-inaction effects in regret findings to responsibility. Regret is associated with
evaluations of self-agency and self-blame, key components of responsibility (Frijda et al.,
1989; Zeelenberg et al., 2002). Most of the evidence on the regret-responsibility link in the
context of action-inaction is based on hypothetical scenario experiments. There are only few
real-life experience surveys on regret-responsibility in the action-inaction literature, but there
have been some real-life experience successful demonstrations on regret-responsibility link
outside the action-inaction literature (e.g. Breugelmans et al., 2014).
We note that regret and responsibility are positively correlated yet separate constructs.
Ordónez and Connolly (2000) argued that some people experience some levels of regret over
outcomes that they have no agency over (e.g. the outcome was reached by computer
reassignment). There are situations in which the decision-maker experiences limited
responsibility but stronger regret, such as choosing a lesser-known product brand (Simonson,
1992). Another plausible key difference is that regret tends to be associated with
counterfactual thoughts (Huang & Zeelenberg, 2012) compared to responsibility, which is
more strongly associated with agency, causality, and morality (Connolly et al., 1997; Kordes-
de Vaal, 1996). There may be discrepancies in action-inaction regret and responsibility
findings, yet there are several studies reporting a positive regret-responsibility link (e.g.
Ordónez & Connolly, 2000; Zeelenberg et al., 2000, 2002). To the best of our knowledge,
there are no studies that compared long-term feelings of responsibility regarding action vs
inaction. We expected findings for regret to extend similarly to responsibility. See Table 2 for
the extension hypotheses.
Gilovich and Medvec (1994) Replication and Extension 10
Table 2
Summary of extension hypotheses
Study
Hypothesis
Study 1 real-life
Responsibility for past action-
inactions
Participants are more likely to report feeling more
responsible for life's inactions compared to life's
actions.
Study 3 (within-subject)
Study 4 (between-subject)
Hypothetical scenarios
Participants are more likely to associate stronger
responsibility with recent actions than with recent
inactions.
Participants are more likely to associate stronger
responsibility with distant past inactions than with
distant past actions.
Study 5 real-life
Responsibility for recent versus
distant past action-inactions
Participants are more likely to experience stronger
feelings of responsibility for their most responsible
recent past action compared to their recent most
responsible inaction.
Participants are more likely to experience stronger
feelings of responsibility for their most responsible
distant past inaction compared to their most
responsible distant past action.
Gilovich and Medvec (1994) Replication and Extension 11
Method
Transparency and Openness
We report the determination of sample size, all data exclusions, all manipulations, and all
measures in our studies (Simmons et al., 2012). This manuscript is in line with Appelbaum et
al. (2018) Journal Article Reporting Standards (JARS) and Transparency and Openness
Promotion (TOP) Guidelines (Nosek et al., 2015). We preregistered designs and analysis
plans of all studies before data collection. Pre-registration, all data, code, and materials are
available on the Open Science Framework (OSF): https://osf.io/342td/ and
https://osf.io/7m3q2/. Open-science details, disclosures, original effects calculations, power
analyses, and pre-exclusion results are provided in the supplementary. We analyzed data
using RMarkdown (Xie et al., 2018, see RMarkdown output in site for analyses with the list
of packages) with RStudio version 1.3.1073 (RStudio Team, 2021) and produced plots with
the package ggplot2 version 3.3.3 (Wickham, 2016).
Participants
We recruited US-American participants from Amazon Mechanical Turk (MTurk) with
TurkPrime.com/CloudResearch (Litman et al., 2017). Based on our extensive experience of
running similar replications on MTurk, to ensure high-quality data collection, we employed
the following CloudResearch options: Duplicate IP Block. Duplicate Geocode Block,
Suspicious Geocode Block, Verify Worker Country Location, Enhanced Privacy,
CloudResearch Approved Participants, Block Low Quality Participants, etc. We also
employed the Qualtrics fraud and spam prevention measures: reCAPTCHA, prevent multiple
submission, prevent ballot stuffing, bot detection, security scan monitor, and relevant ID.
MTurk has been shown to be a reliable platform for conducting studies in social psychology,
judgment, and decision-making (Anderson et al., 2019; Buhrmester et al., 2011; Thomas &
Gilovich and Medvec (1994) Replication and Extension 12
Clifford, 2017). Several recent studies on the action-effect (e.g. Feldman, 2020; Feldman &
Albarracín, 2017) and a recent large-scale collaborative project with over 80 replications of
judgment and decision-making phenomena has shown MTurk, with
TurkPrime.com/CloudResearch to be a highly suitable platform for this research design
(Collaborative Open-science Research, 2022). Recently, Eyal et al. (2021) compared levels of
attention, comprehension, and dishonesty of participants between several platforms and
panels and found that CloudResearch and Prolific provided higher quality compared to other
methods (Qualtrics, MTurk without CloudResearch, Dynata).
A total of 1017
1
participants completed the study. We excluded 29 participants based
on our pre-registered exclusion criteria (see supplementary for details), resulting in a total
sample of 988 participants (Mage = 43.94, SD = 13.62; 566 females, 408 males, 8 others, 6
prefer not disclosing their gender). We report full results comparing pre-exclusions versus
post-exclusions in the supplementary. We provide a comparison of the target article samples
and the replication samples in Table 3.
1
567 out of 1584 participants decided to drop out during the survey, likely because of the writing description
task warnings.
Gilovich and Medvec (1994) Replication and Extension 13
Table 3
Differences and similarities between Gilovich and Medvec (1994) and our replication
Gilovich and Medvec
(1994)
Replication and
extension
Sample size
Study 1A: 60, Study 1B:
30, Study 3: 80, Study 4:
76, Study 5: 32
Combined sample: 988
after exclusion
Geographic origin
United States
United States
Gender
Not reported
566 females, 408 males,
8 others, 6 prefer not
disclosing their gender
Median age (years)
Not reported
42
Average age (years)
Study 1A: 40.3, Study
1B: 40.1. Not reported
for other studies
43.94
Standard deviation age
(years)
Not reported
13.62
Age range (years)
Not reported
18-89
Medium (location)
Telephone (Study 1), on
the streets (Study 5), and
lab (Study 3 and Study
4)
Computer (online),
Amazon Mechanical
Turk
Compensation
Not reported
Nominal payment: $0.8
USD/participant
Year
1994 or before
2021
To estimate the required sample size, we used pwr package version 1.3 (Champely et
al., 2018) and conducted an a-priori power analysis for chi-square goodness of fit 50-50 tests,
comparing the proportion of action with stronger regret versus inaction with stronger regret,
and chi-square tests of association, testing the association between temporal distance and
action-inaction regret. We calculated and reported the original effect sizes in Supplementary
Table 4. Aiming for a statistical power of 95% with an alpha of .05, and based on the weakest
meaningful effect (V = 0.24) that the original authors hypothesized and claimed to find
Gilovich and Medvec (1994) Replication and Extension 14
support for, with more participants perceiving stronger regret for inaction than for action in
the long-term in Study 4 (but p > .05), the required sample was 920 participants. As we
expected some participants to be excluded, we aimed for 1000 participants. We provide more
details in the supplementary.
Design and procedure
We made adjustments to the design of the original studies. Extending the original
studies and deviating from their procedures, we combined the replications of Studies 1A, 3, 4,
and 5 into a singular design in one data collection, with added extensions examining
responsibility. First, participants read the consent form. We first presented Study 5, followed
by Study 1. Both were personal experience studies. We then randomized participants into
either Study 3, a within-subject design, or Study 4, a between-subject design, as Studies 3 and
4 consisted of the same hypothetical scenario and questions. We placed Study 3/Study 4 at
the end to prevent the stimuli in the scenario from affecting personal responses. We then
presented participants with funneling and demographic questions, followed by a debriefing
statement. See below sections for more specific and detailed information about all studies.
We note that the study numbers below are based on the study numbers of the original article,
but not the order of our replications.
We decided on this design in order to address possible concerns regarding the sample.
Despite our ample experience and accumulated evidence in support of validity of our chosen
MTurk/CloudResearch sample for replications of classics in judgment and decision-making,
reviewers often expressed concerns about online samples regarding inattentiveness,
suitability to context (time, setting, etc.), and overall data quality. When some of the findings
replicate and others do not, combining the studies allows ruling out inattentiveness as a
concern, adequacy of the target sample for these replications, or the adjustments to updated
Gilovich and Medvec (1994) Replication and Extension 15
context (time, setting, etc.), so that we can instead focus on the implications regarding
specific designs and found effects. This design has been tested and shown to be successful in
several recent replications (Adelina & Feldman, 2021; Chandrashekar et al., 2021; Chen et
al., 2021; Ziano et al., 2021).
Deviations from original studies
We provided detailed information of designs (type of study, sample, variables, exact
wordings) of the original studies in the Methods and Analyses of the original article section
of the Supplementary. We note several deviations from the original, summarized in Table 3
and Table 4. We combined studies into a single survey, and we added responsibility questions
as extensions. Finally, we recruited participants through MTurk online, instead of participants
from New York and Chicago, or Cornell University students.
Gilovich and Medvec (1994) Replication and Extension 16
Table 4
Classification of the replication, based on LeBel et al. (2018)
Design facet
Replication
Details of deviation
Effect/hypothesis
Same
IV construct
Same
DV construct
Same
IV
operationalization
Same
DV
operationalization
Same
Population (e.g.
age)
Similar
Both with American participants. However, our replications consist
of participants from Amazon Mechanical Turk, instead of adults
from New York and Chicago, or Cornell University undergraduate
students in the original
IV stimuli
Different for Study
5; Same for Studies
1, 3, and 4
Study 5: Minor changes to ensure the wordings across conditions and
action vs inaction are consistent
DV stimuli
Different for Study
5; Same for Studies
1, 3, and 4
Study 5: Minor wording change to Study 5 regret question. We
participants to describe their regrets and responsibilities2 in Study 5
(which was not required in the original), as this lowers the chance of
quick irrelevant or random responses, ensuring participants are
thinking about the task and responding seriously. We asked
participants for brief descriptions and reminded them that they did
not have to disclose information they did not feel comfortable with.
Procedural details
Different
We combined Studies 1, 3, 4, and 5 into a single Qualtrics survey.
The original article used separate samples.
Physical settings
Different
Online data collection in our replication vs real-life and telephone
data collection in the original
Contextual
variables
Different
The original authors conducted their studies in the early 1990s
whereas we conducted our replications in 2021.
Replication
classification
Studies 1, 3, 4:
Very close
replication;
Study 5: Close
replication
For our Study 5, the IV stimuli and the DV stimuli are different from
that of the original study.
2
Additionally, we checked descriptions of participants and conducted exploratory analyses excluding incorrect
and irrelevant descriptions of action-inaction regrets or responsibilities. The findings are reported in the
Supplementary Exploratory Analyses of Study 5 section. Such findings are very similar to findings reported in
the main manuscript.
Gilovich and Medvec (1994) Replication and Extension 17
Replications classification
We summarized Studies 1, 3, and 4 as "very close replications" and Study 5 as a
based on the criteria by LeBel et al. (2018) (see Figure S3 in the
Supplementary), with our classification analysis provided in Table 4.
Study 5: Method
First, in Study 5, participants answered questions regarding their greatest regrets.
Temporal distance (the past week vs entire life, in counterbalanced order) was the
independent variable. We asked participants to think about and describe their greatest lifetime
action-regret and greatest lifetime inaction-regret, as well as greatest past week action-regret
and greatest past week inaction-regret. We then asked them which they regretted more. In the
original study, participants were only required to think about but not describe their regrets.
However, in our replication we asked participants to briefly write about their regrets. By
having participants briefly describe their regrets, we felt they would be more likely to engage
in effortful reflections, and less likely to respond randomly, thereby ensuring better data
quality. We reminded participants that they do not need to disclose any information they feel
uncomfortable sharing. We also went to great lengths to align expectations about the task
we made it clear in our study recruitment and with a specific question in the consent screen
that the task involves brief writing and that the study is about life regrets
in the supplementary).
We randomized participants to either answer the replication questions or extension
questions. In the extension condition, we asked participants about the action decision that
they felt most personally responsible for and the inaction decision that they felt most
personally responsible for, in the past week and in their lifetime (in counterbalanced order).
Similarly, we asked participants to very briefly describe these events. They then answered
Gilovich and Medvec (1994) Replication and Extension 18
which of those they felt more responsible for. Full details are provided in Table S5 in the
supplementary.
Study 5: Results
Replication: Regret
For past week regrets, with the chi-square goodness-of-fit test, we failed to find support for a
deviation from a 50-50 split in participants experiencing stronger regret for the action
(52.34%) than inaction (47.66%), z = 1.04, χ2 (1, N = 535) = 1.17, p = .280, V = 0.05, 95%
CI [0.00, 0.13] (see Figure 1 top left plot).
For lifetime regrets, with the chi-square goodness-of-fit test, we failed to find support for a
deviation from 50-50 split in participants experiencing stronger regret for the action (52.15%)
than inaction (47.85%), z = 0.95, χ2 (1, N = 535) = 0.99, p = .320, V = 0.04, 95% CI [0.00,
0.13] (see Figure 1 top right plot).
Comparing the proportion of participants experiencing stronger action regret in the past
week and experiencing stronger inaction regret in lifetime (133/265, 265 is the total number
of participants who showed reversal, 50.19%), versus the proportion of participants choosing
inaction in the past week and choosing action in lifetime (132/265, 49.81%), we failed to find
support for a deviation from 50-50 distribution, z = 0.00, χ2 (1, N = 265) = 0.00, p = .951, V =
0.00, 95% CI [0.00, 0.14]. We also conducted a McNemar test, and failed to find support for
the association between temporal distance and inaction-action regret, OR = 0.99, 95% CI
[0.77, 1.27], p = 1.
Extension: Responsibility
For the responsibility over the week, with the chi-square goodness-of-fit test, we found
support for a deviation from 50-50 split in participants experiencing stronger responsibility
Gilovich and Medvec (1994) Replication and Extension 19
for the action (56.29%) versus participants experiencing stronger responsibility for the
inaction (43.71%), z = 2.63, χ2 (1, N = 453) = 7.17, p = .007, V = 0.13, 95% CI [0.03, 0.22].
More participants felt stronger responsibility for action compared to inaction(see Figure 1
bottom left plot).
For the responsibility over the lifetime, with the chi-square goodness-of-fit test, we found
support for a deviation from 50-50 split in participants experiencing stronger responsibility
for the action (56.51%) versus participants experiencing stronger responsibility for the
inaction (43.49%), z = 2.73, χ2 (1, N = 453) = 7.68, p = .006, V = 0.13, 95% CI [0.03, 0.22].
More participants felt stronger responsibility for action compared to inaction (see Figure 1
bottom right plot).
Comparing the proportion of participants experiencing stronger action responsibility in the
past week and experiencing stronger inaction responsibility in lifetime (100/201, 49.75%),
versus the proportion of participants choosing inaction in the past week and choosing action
in lifetime (101/201, 50.25%), we failed to find support for a deviation from 50-50, z = 0.00,
χ2 (1, N = 201) = 0.00, p = .944, V = 0.00, 95% CI [0.00, 0.16]. We also conducted a
McNemar test, and failed to find support for the association between temporal distance and
action-inaction responsibility, OR = 1.01, 95% CI [0.76, 1.35], p = 1.
Gilovich and Medvec (1994) Replication and Extension 20
Figure 1
Study 5: Action-inaction regret and responsibility - short-term (past week) and long-term
(lifetime)
Note. Error bars denote 95% confidence intervals. Short term = past week. Long term =
lifetime.
Study 1a: Method
After Study 5, we presented Study 1. We asked participants about their action and
inaction regretful experiences -
those things that you regret, what would you say you regret more, those things that you did
Medvec, 1994, p. 358). We asked another group of participants regarding their felt
responsibility for life's personally responsible actions and inactions - "When you look back
Gilovich and Medvec (1994) Replication and Extension 21
on your experiences in life and think of those things that you feel personally responsible for,
what would you say you feel personally responsible more, those things that you did but wish
you hadn't, or those things that you didn't do but wish you had?".
Study 1a: Results
Replication: Regret
We began by examining regret, we conducted a chi-square goodness of fit test against
a 50-50 action-inaction split , z = -N = 535) = 34.07, p < .001, V = 0.25, 95% CI
[0.17, 0.34], and found support for stronger regret for inactions; more participants reported
experiencing stronger regret over lifelong inactions (62.62%) than those reporting
experiencing stronger regret over lifelong actions (37.38%) (see Figure 2 left plot).
Extension: Responsibility
To examine our responsibility extension, we conducted a chi-square goodness of fit
test against a 50-50 action-inaction split, z N = 453) = 24.34, p < .001, V = 0.23,
95% CI [0.14, 0.32], and found that more participants reported stronger responsibility over
lifelong regrettable actions (61.59%) than those reporting stronger responsibility for lifelong
regrettable inactions (38.41%) (see Figure 2 right plot).
Gilovich and Medvec (1994) Replication and Extension 22
Figure 2
Study 1A: Action-Inaction Regret and Responsibility
Note. Error bars denote lower confidence intervals and upper confidence intervals (95%).
Studies 3 and 4: Method
After completing Studies 5 and 1a, we presented participants with scenario
experiments on college decisions: Inaction Dave stayed in the same college whereas Action
Jim switched to another college, and both were unsatisfied. We randomized participants into
either a within-subject design as in Study 3 (participants compared feelings of regret and
responsibility of Dave vs. Jim in both short-term and long-term), or a between-subject design
as in Study 4 (participants answered short-term or long-term questions). In Studies 3 and 4,
Gilovich and Medvec (1994) Replication and Extension 23
participants answered both replication and extension questions. We provide more details on
the designs of Studies 3 and 4 in the supplementary's Tables 7 and 8).
Study 4 (between-subject): Results
Replication: Regret
In the short-term condition of the between-subject design study, we conducted a chi-
square goodness of fit test and found support for more participants perceiving stronger regret
for action Jim (60.65%) than for inaction Dave (39.35%), z = 3.69, χ2 (1, N = 247) = 14.09, p
< .001, V = 0.24, 95% CI [0.12, 0.36] (see Figure 3 top left for the plot).
In the long-term condition, we conducted a chi-square goodness of fit test and found
that more participants perceiving stronger regret for inaction Dave (57.61%) than for action
Jim (42.39%), yet this did not meet our pre-defined alpha leading us to conclude no support, z
= -1.59, χ2 (1, N = 248) = 2.73, p = .099, V = 0.10, 95% CI [0.01, 0.23] (see Figure 3 top
right for the plot).
We conducted a chi-square test of independence and found support for the association
between temporal distance and action-inaction regret, χ2 (1, N = 495) = 14.68, p < .001, V =
0.17, 95% CI [0.08, 0.26]. We found that compared to short-term, long-term perspective was
associated with stronger perceived regret for inaction.
Extension: Responsibility
Examining responsibility in the short-term condition, we conducted a chi-square goodness of
fit test and found support for more participants perceiving stronger responsibility for action
Jim (61.94%) than for inaction Dave (38.06%), z = 7.76, χ2 (1, N = 247) = 61.25, p < .001, V
= 0.50, 95% CI [0.39, 0.60] (see Figure 3 bottom left for the plot).
Gilovich and Medvec (1994) Replication and Extension 24
Examining responsibility in the long-term condition, we conducted a chi-square
goodness of fit test and found support for more participants perceiving stronger responsibility
for action Jim (74.90%) than for inaction Dave (25.10%), z = 5.78, χ2 (1, N = 248) = 34.13, p
< .001, V = 0.37, 95% CI [0.25, 0.49] (see Figure 3 bottom right for the plot).
We conducted a chi-square test of independence and failed to find support for the
association between temporal distance and action-inaction responsibility, χ2 (1, N = 495) =
2.46, p = .117, V = 0.07, 95% CI [0.00, 0.16].
Figure 3
Study 4: Short-Term and Long-Term Action-Inaction Regrets and Responsibilities Plots
Note. Error bars denote lower confidence intervals and upper confidence intervals (95%).
Gilovich and Medvec (1994) Replication and Extension 25
Study 3 (within-subject): Results
Replication: Regret
In the short-term condition of the within-subject design study, we conducted a chi-square
goodness of fit test (meant to mirror the analyses for Study 4 to allow for a comparison) and
found support for more participants perceiving stronger regret for action Jim (61.26%) than
for inaction Dave (38.74%), z = 4.95, χ2 (1, N = 493) = 24.99, p < .001, V = 0.23, 95% CI
[0.14, 0.31] (see Figure 4 top left plot).
In the long-term condition, we conducted a chi-square goodness of fit test and found support
for more participants perceiving stronger regret for inaction Dave (57.61%) than for action
Jim (42.39%), z = -3.33, χ2 (1, N = 493) = 11.41, p < .001, V = 0.15, 95% CI [0.06, 0.24] (see
Figure 4 top right plot).
Comparing the proportion of participants choosing action Jim in the short term and choosing
inaction Dave in the long term (113/133, the total number of participants who showed
reversal in answers, 84.96%), versus the proportion of participants choosing inaction Dave in
the short term and choosing action Jim in the long term (20/133, 15.04%), we found support
for a deviation from 50-50, z = 7.98, χ2 (1, N = 133) = 65.03, p < .001, V = 0.70, 95% CI
[0.56, 0.80]. More participants chose action in the short term and inaction in the long term,
compared to inaction in the short term and action in the long term. We also conducted a
McNemar test, and found support for the association between temporal distance and action-
inaction regret, OR = 0.18, 95% CI [0.10, 0.29], p < .001.
Extension: Responsibility
Examining responsibility in the short-term condition, we conducted a chi-square goodness of
fit test and found support for more participants perceiving stronger responsibility for action
Gilovich and Medvec (1994) Replication and Extension 26
Jim (73.83%) than for inaction Dave (26.17%), z = 10.54, χ2 (1, N = 493) = 112.02, p < .001,
V = 0.48, 95% CI [0.40, 0.55] (see Figure 4 bottom left plot).
Examining responsibility in the long-term condition, we conducted a chi-square
goodness of fit test and found support for more participants perceiving stronger responsibility
for action Jim (60.65%) than for inaction Dave (39.35%) , z = 4.68, χ2 (1, N = 493) = 22.36,
p < .001, V = 0.21, 95% CI [0.13, 0.29] (see Figure 4 bottom right plot).
Comparing the proportion of participants choosing action Jim in the short term and choosing
inaction Dave in the long term (94/123, 76.42%), versus the proportion of participants
choosing inaction Dave in the short term and choosing action Jim in the long term (29/123,
23.58%), we found support for a deviation from 50-50, z = 5.77, χ2 (1, N = 123) = 34.35, p <
.001, V = 0.53, 95% CI [0.37, 0.67]. More participants chose stronger responsibility for
action in the short term and for inaction in the long term, compared to stronger responsibility
for inaction in the short term and for action in the long term. We also conducted a McNemar
test, and found support for the association between temporal distance and action-inaction
responsibility, OR = 0.31, 95% CI [0.20, 0.47], p < .001. Temporal distance had an impact on
the choice distribution. The difference in the proportion between action and inaction was
weaker in the long-term compared to that in the short-term. The effect was in the same
direction as that in regret, but did not lead to a complete reversal of perceptions.
Gilovich and Medvec (1994) Replication and Extension 27
Figure 4
Study 3: Short-Term and Long-Term Action-Inaction Regrets and Responsibilities Plots
Note. Error bars denote lower confidence intervals and upper confidence intervals (95%).
Overall summary of findings All Studies
We summarized descriptive statistics of all studies in Table 5. We summarized the
comparison of all effects of original studies versus our replications in Table 6, with an
interpretation of the results based on the LeBel et al. (2019) replication results evaluation
criteria.
Gilovich and Medvec (1994) Replication and Extension 28
Table 5
Summary of descriptive statistics for all studies for both regret and responsibility
Study
Action
Count
Action
Percentage
Inaction
Count
Inaction
Percentage
Study 1 General
Regret
200/535
37.38%, 95% CI
[33.39%, 41.56%]
335/535
62.62%, 95% CI
[58.44%, 66.61%]
Study 1 General
Responsibility
279/453
61.59%, 95% CI
[57.03%, 65.95%]
174/453
38.41%, 95% CI
[34.05%, 42.97%]
Study 3 Short Term
Regret
302/493
61.26%, 95% CI
[56.89%, 65.46%]
191/493
38.74%, 95% CI
[34.54%, 43.11%]
Study 3 Long Term
Regret
209/493
42.39%, 95% CI
[38.11%, 46.80%]
284/493
57.61%, 95% CI
[53.20%, 61.89%]
Study 3 Short Term
Responsibility
364/493
73.83%, 95% CI
[69.78%, 77.52%]
129/493
26.17%, 95% CI
[22.48%, 30.22%]
Study 3 Long Term
Responsibility
299/493
60.65%, 95% CI
[56.27%, 64.86%]
194/493
39.35%, 95% CI
[35.14%, 43.73%]
Study 4 Short Term
Regret
153/247
61.94%, 95% CI
[55.75%, 67.77%]
94/247
38.06%, 95% CI
[32.23%, 44.25%]
Study 4 Long Term
Regret
111/248
44.76%, 95% CI
[38.70%, 50.98%]
137/248
55.24%, 95% CI
[49.02%, 61.30%]
Study 4 Short Term
Responsibility
185/247
74.90%, 95% CI
[69.14%, 79.90%]
62/247
25.10%, 95% CI
[20.10%, 30.86%]
Study 4 Long Term
Responsibility
170/248
68.55%, 95% CI
[62.52%, 74.01%]
78/248
31.45%, 95% CI
[25.99%, 37.48%]
Study 5 Greatest Past
Week Regret
280/535
52.34%, 95% CI
[48.10%, 56.54%]
255/535
47.66%, 95% CI
[43.46%, 51.90%]
Study 5 Greatest
Lifetime Regret
279/535
52.15%, 95% CI
[47.92%, 56.35%]
256/535
47.85%, 95% CI
[43.65%, 52.08%]
Study 5 Greatest Past
Week Responsibility
255/453
56.29%, 95% CI
[51.69%, 60.79%]
198/453
43.71%, 95% CI
[39.21%, 48.31%]
Study 5 Greatest
Lifetime
Responsibility
256/453
56.51%, 95% CI
[51.91%, 61.00%]
197/453
43.49%, 95% CI
[39.00%, 48.09%]
Gilovich and Medvec (1994) Replication and Extension 29
Table 6
Summary of statistical tests and comparison with the original effect sizes
Chi-
square
p
Replication
Cramer V
and CI
Original Cramer V
and CI
Interpretation
Study 1 - Chi-Square Goodness of Fit Test
General Regret
χ2 (1, N
= 535) =
34.07
< .001
V = 0.25,
95% CI
[0.17, 0.34]
V = 0.50,
95% CI
[0.27, 0.70]
Signal, inconsistent, smaller (successful
replication)
Study 3 - Chi-Square Goodness of Fit Test
Short Term
Regret
χ2 (1, N
= 493) =
24.99
< .001
V = 0.23,
95% CI
[0.14, 0.31]
V = 0.53,
95% CI
[0.35, 0.70]
Signal, inconsistent, smaller (successful
replication)
Long Term
Regret
χ2 (1, N
= 493) =
11.41
< .001
V = 0.15,
95% CI
[0.06, 0.24]
V = 0.28,
95% CI
[0.05, 0.48]
Signal, inconsistent, smaller (successful
replication)
Action-inaction
vs. Temporal
Change
χ2 (1, N
= 133) =
65.03
< .001
V = 0.70,
95% CI
[0.56, 0.80]
Insufficient
information
Signal, the effect size cannot be directly
compared, but successful replication
Study 4 - Chi-Square Goodness of Fit Test and Test of Independence
Short Term
Regret
χ2 (1, N
= 247) =
14.09
< .001
V = 0.24,
95% CI
[0.12, 0.36]
V = 0.53,
95% CI [0.24,
0.76]
Signal, inconsistent, smaller (successful
replication)
Long Term
Regret
χ2 (1, N
= 248) =
2.73
= .099
V = 0.10,
95% CI
[0.01, 0.23]
V = 0.24,
95% CI
[0.00, 0.52]
Unclear (see notes) (likely successful
replication)
Action-inaction
vs. Temporal
Change
χ2 (1, N
= 495) =
14.68
< .001
V = 0.17,
95% CI
[0.08, 0.26]
V = 0.38,
95% CI
[0.16, 0.61]
Signal, inconsistent, smaller (successful
replication)
Study 5
Short Term
Regret
χ2 (1, N
= 535) =
1.17
= .280
V = 0.05,
95% CI
[0.00, 0.13]
V = 0.06,
95% CI
[0.00, 0.44]
No signal, consistent (successful
replication)
Long Term
Regret
χ2 (1, N
= 535) =
0.99
= .320
V = 0.04,
95% CI
[0.00, 0.13]
V = 0.56,
95% CI
[0.25, 0.81]
No-signal, inconsistent (failed
replication)
Temporal
Distance and
Action-
Inaction Regret
χ2 (1, N
= 265) =
0.00
= .951
OR = 0.99,
95% CI
[0.77, 1.27]
Insufficient
information
No signal, likely failed replication
Note. 1) We conducted Chi-Square goodness of fit tests for the above studies, except for
Study 4 association between Temporal Distance and Action and Inaction Regret, in which we
conducted a Chi-Square test of independence. The interpretation of outcome is based on
LeBel et al. (2019). 2) For Study 4 long-term regret part, LeBel et al. (2019) Criteria B does
not account that it is possible for a finding to not reach significance and for the effect size CIs
Gilovich and Medvec (1994) Replication and Extension 30
Comparing replication findings to original findings and extensions
We successfully replicated and found support for the original findings in Studies 1, 3,
and 4 (short-term regret and temporal effect) with smaller effect sizes. For our replication of
Study 4's long-term regret, the CIs of the replication did not cover the original effect size, and
we failed to find support for the effect (which was the case in both the original and the
replication).
We failed to successfully replicate Study 5. We failed to find support for an action-
effect in the short-term, and failed to find support for an inaction-effect in the long-term.
Regarding our responsibility extensions, we conclude stronger responsibility for action
over inaction across all studies, for both short-term and long-term.
Discussion
We conducted a pre-registered replication of the temporal pattern of action-effect by
Gilovich and Medvec (1994), with a more diverse (Buhrmester et al., 2011) and high-
powered sample. We successfully replicated Study 1, which focused on general regrets, as
well as Studies 3 and 4, which were scenario studies asking participants to compare regret for
action versus inaction. More participants reported stronger regret for action in the short-run,
but stronger regret for inaction in the long run.
In Study 5, we failed to find support for an action-effect in the short-term, failed to
replicate findings for the long-term, and the proposed association between temporal distance
and action-effect.
Gilovich and Medvec (1994) Replication and Extension 31
Possible reasons behind the discrepancy in findings of Studies 1, 3, and 4 versus Study 5
Why did the replication of Study 5 fail while the replication of the other studies
succeeded?
Previous studies have shown MTurk to be a reliable platform for the study of action
and inaction, and judgment and decision-making more broadly (e.g. Feldman, 2020; Feldman
& Albarracín, 2017). Our design and the other successful replications of Studies 1, 3, and 4
address concerns regarding sample characteristics or time, given that they were conducted
using the same sample. Therefore, we believe the more plausible explanations are the
differences in methods and the likelihood of a false positive. We note that the sample size in
the original Study 5 was 32, with a much higher likelihood of a false-positive. Our sample
was substantially larger and well-powered, yet we were unable to detect the inaction-effect in
the long run.
Why would the method used in Study 5 result in different findings? We believe this
might have to do with the ways action and inaction are conceptualized in the different studies.
In Study 5 action refers to versus inaction as
. In Study 3 and Study 4, action is conceptualized as a switching behavior, a change to the
status quo, versus inaction, which is sticking with the status quo. Unfortunately, these issues
seem to be widespread in this literature, with recent reviews alerting that action and inaction
are often ill-defined terms and in urgent need of clarifications (Feldman et al., 2021).
Our findings are consistent with a large body of literature showing support for an
action effect in high-risk recent situations that result in negative outcomes. The typical
action-effect scenarios refer to changing, switching, or deviating from a set reference point
(e.g., past behavior, status quo) (e.g. Feldman & Albarracín, 2017; Gleicher et al., 1990;
Landman, 1987), with norm theory (Kahneman & Miller, 1986) arguing that is likely due to
Gilovich and Medvec (1994) Replication and Extension 32
action being perceived as more abnormal than inaction (Feldman, 2020), and that
exceptionality tends to elicit higher regret than normality (Kutscher & Feldman, 2019). The
meaning of action and inaction in Studies 3 and 4 was closer to the typical action-effect and
norm theory scenarios (Kahneman & Miller, 1986). However, in Study 5, the meaning of
action was far broader with no clear reference to a norm or a reference point. Therefore, the
differences between action as in doing and inaction as in not doing seem less clear and with
no clear indication of what to compare to.
Beyond differences in meaning, another possible explanation is regarding the
differences between perceptions in Studies 3 and 4 and the evaluation of
personal actual experiences of emotions elicited in Study 5. Perceptions of emotions in others
tend to be less accurate and differ from actual personal experiences, especially if there is no
personal relevance to the situation evaluated in the presented scenario. The failed replication
of Study 5 seems consistent with most studies that directly ask participants about their
personal experiences (Bonnefon & Zhang, 2008; Feldman et al., 1999
3
; Towers et al., 2016),
but inconsistent with Zeelenberg et al. (1998) studies, in which they found support for
temporal pattern of regret in real life experiences for interpersonal regrets (they did not test
other kinds of regrets).
Another possible explanation for our different findings in Study 5 is a change we
made to the original studye required participants to describe their regrets,
whereas Gilovich and Medvec (1994) only required participants to recall their regrets without
writing those down. We, however, find this explanation unlikely. Tconcern was
that participants may be unwilling to describe their very embarrassing and shameful regrets,
3
Feldman et al. (1999) found support for higher frequencies of inaction regrets compared to action regrets, but
failed to find support for intensity differences between action and inaction regrets. Study 5 focuses on intensity
but not frequency.
Gilovich and Medvec (1994) Replication and Extension 33
yet taking a closer look at the responses (found in our dataset), we found that many
participants described highly personal, shameful, and somewhat tragic events, including
events involving death, major career failures, major educational failures, major relationship
failures, etc. However, we cannot completely rule out the possibility of such differences
having an impact on the findings, and future research can further test this possibility by
asking half of the participants to describe a major regret and asking other participants to
simply think about a major regret.
Another possible explanation raised in the peer-review was that Study 5 may not have
been suitable for our online MTurk target sample, given concerns of attentiveness and
seriousness. As we discussed earlier, MTurk with CloudResearch/TurkPrime provides high-
quality responses, in which participants are as if not more attentive than on other platforms
(Eyal et al., 2021). The successful replications of the other studies in our unified design
address concerns of attentiveness. We also addressed this concern by conducting additional
exploratory analyses (see the Exploratory Analyses of Study 5 section in the Supplementary)
excluding responses in which participants seem to have misunderstood or confused action
and inaction, or participants reporting lifetime mistakes in past week mistakes questions (or
vice versa), and non-regret/non-responsibility responses. We found that only a very low % of
participants (ranging from 1.68% for lifetime regret, to 7.28% for past week responsibility)
misclassified responses. Our results of the exploratory analyses with those participants
excluded were consistent with results we reported above.
Studies 3 and 4 were also better controlled with specific scenarios, whereas Study 5
did not restrict the range of elicited regrets, which included many different domains in life,
such as education, work, relationships, finance, etc.
Gilovich and Medvec (1994) Replication and Extension 34
What is puzzling about the above explanations is that although we failed to find
support for lifetime recalls in Study 5, we did find support for the effects in Study 1. Both
participants to evaluate those together rather than contrasting one most regrettable action
against one most regrettable inaction. Therefore, it is possible that specificity is a moderating
factor of the effect. The autobiographical memory framework by Davison and Feeney (2008)
suggested that regret is about remembering past events with different levels of specificity and
generality. They found that general regrets were more likely to be for inactions over action,
yet specific regrets were more likely to be for actions than inactions.
The above proposed reasons for the discrepancies between studies in the same article
are speculative, and we conclude that more work is needed to examine any of the proposed
moderators with direct testing.
Responsibility Extension
Regret and responsibility are often positively related (e.g. Zeelenberg et al., 2000,
2002), -yet are distinct constructs. We found consistent support for stronger responsibility for
action compared to inaction for the recent past, and the finding for responsibility in the recent
past generally aligned with that of regret. However, while there was some support for
differences between recent past and distant past for responsibility, the effect seems much
weaker than for regret, and we did not find a full reversal toward stronger responsibility for
inaction in the distant past. Instead, consistently across different designs, we also found
support for stronger responsibility for action for the distant past events. Responsibility seems
to be more strongly associated with morality, causality, and agency (Connolly et al., 1997).
Changes in perceived responsibility intensity perhaps fluctuate less over time compared to
Gilovich and Medvec (1994) Replication and Extension 35
regret, which is a counterfactual emotion that may fluctuate over time more as people may
feel more confident that they would have made it if they had taken actions, thereby regretting
inaction more in the long-run (Gilovich et al., 1993; Gilovich & Medvec, 1995). That said,
we note this explanation is speculative and more work comparing mechanisms of regret and
responsibility in action-inaction studies is needed.
These findings may hold important implications for the link between the action-effect
and omission-bias. Omission bias extends the action-effect to reflect action-inaction
asymmetries regarding responsibility and blame. More work is needed on the potential
moderating effect of time with regret and responsibility examined together for both action-
effect and omission-bias scenarios.
Our findings for responsibility in Study 3's within-subject design and Study 4's
between-subject design were slightly different, with stronger effects for the within-subject
design. There are quite a few judgment and decision-making effects that are stronger with
within-subject design compared to between-subject design (Charness et al., 2012), and more
work is needed to contrast the two regarding action-effect and omission-bias.
Possible limitations and future directions
We faced some challenges with the studies eliciting life events. A small number of
participants (1.68% to 7.28%) wrote inaction events in the action description box or vice
versa, with some participants reporting they did not experience any action or inaction regret
or responsibility in the past week. In an online study we cannot rule out the possibility that
some participants may have copy-pasted, and based on our experience there are indications
that MTurk participants typically dislike writing tasks. However, we tend to think that the
likelihood of this being an issue in our design is very low, as we adopted numerous quality
control methods, aligned expectations in advance about the task, and only asked for brief
Gilovich and Medvec (1994) Replication and Extension 36
descriptions in one or two sentences. We checked all responses, and found that most
responses were of high-quality. We also conducted exploratory non-pre-registered analyses
excluding possibly irrelevant and incorrect responses (which were only a very small
proportion of the entire sample) and the results (reported in the supplementary) were very
similar to the results after pre-registered exclusion or full results. Therefore, we believe it is
less likely that the null findings in Study 5 are due to this issue and find it more likely that
such null findings are due to differences in meanings of action-inaction. To resolve the
discrepancy in findings between Study 5 and Studies 1, 3, and 4, future studies can make
adaptations to Study 5 by manipulating the definitions of action and inaction (Feldman et al.,
2021). Also, we cannot completely rule out the possibility that our adjustments of adding
brief writing to the Study 5 recall task may have impacted the results, and so future studies
may also compare findings of recall tasks that involve versus do not involve writing the
recalled memory, as well as test this phenomenon with other non-MTurk/CloudResearch
samples to investigate if there are meaningful differences.
We note that a single replication of a single article is insufficient to answer all the
questions in the literature with high certainty, and we call for more well-powered pre-
registered replications of work in this domain, preferably by third-parties and in the form of
Registered Reports.
We reported aggregated tendencies regarding temporal effects related to action and
inaction yet there are individual differences factors that may play a role in moderating these
effects (e.g. action-state orientation, Diefendorff et al., 2000; regulatory focus, Itzkin et al.,
2016).
Many of the findings in this literature were conducted in mostly WEIRD settings
(Western, Educated, Industrialized, Rich and Democratic; Henrich et al., 2010), and more
Gilovich and Medvec (1994) Replication and Extension 37
research is needed to study these effects in less WEIRD regions, and/or include cultural
dimensions as potential moderators of these effects.
In our extension, we found differences in findings regarding regret and responsibility.
Studies in the action-inaction literature rarely measure regret and responsibility together, and
more work is needed to investigate the associations between the two constructs in the context
of action-inaction effects.
We believe that more replications with extensions are needed to better understand the
robustness of the findings in this literature and examine new directions, together with meta-
analyses of the action-inaction related literature (e.g., action-effect: Yeung & Feldman, 2022;
omission bias: Yeung et al., 2022), to examine possible moderating factors such as temporal
distance, scenarios versus experience, between-subject versus within-subject study design
comparison, and the used meanings of action versus inaction. We require a more
comprehensive systematic aggregation of findings and insights to identify boundary
conditions.
Conclusion
We conducted a replication and extension of Gilovich and Medvec (1994) revisiting the
temporal pattern of regret in action versus inaction and adding extensions examining
temporal pattern of responsibility. We found support for the original findings on regret with
different designs both examining lifelong experiences in Study 1 and hypothetical scenarios
in Studies 3 and 4, though with weaker effects. However, we failed to find support for such a
pattern in Study 5, and we discussed possible explanations. We called for better
conceptualizations of the terms action and inaction in the literature, with more replications
and extensions of classic studies of the action-inaction literature and follow-up meta-analyses
to help resolve inconsistencies in findings. We also reported the findings of an extension
Gilovich and Medvec (1994) Replication and Extension 38
examining temporal pattern regarding attributions of responsibility, and discussed the regret-
responsibility link.
Gilovich and Medvec (1994) Replication and Extension 39
Acknowledgements:
We would like to thank Qinyu Xiao, Adrien Fillon, Dr. Frances Jin. and Prof. Kin Fai Ellick
Wong for their helpful comments. We thank the original author Thomas Gilovich for
providing the original materials.
Declaration of Conflict of Interest:
The author(s) declared no potential conflicts of interest with respect to the authorship and/or
publication of this article.
Financial disclosure/funding:
The author(s) received no financial support for the research and/or authorship of this
manuscript.
Data accessibility statement:
Pre-registration, materials, data, and code are publicly available on: https://osf.io/7m3q2/
Gilovich and Medvec (1994) Replication and Extension 40
References
Adelina, N., & Feldman, G. (2021). Are past and future selves perceived differently from
present self? Replication and extension of Pronin and Ross (2006) temporal
differences in trait self-ascriptions. International Review of Social Psychology, 34(1):
29, 116. http://doi.org/10.5334/irsp.571
Anderson, C. A., Allen, J. J., Plante, C., Quigley-McBride, A., Lovett, A., & Rokkum, J. N.
(2019). The MTurkification of social and personality psychology. Personality and
Social Psychology Bulletin, 45(6), 842-850.
https://doi.org/10.1177/0146167218798821
Appelbaum, M., Cooper, H., Kline, R. B., Mayo-Wilson, E., Nezu, A. M., & Rao, S. M.
(2018). Journal article reporting standards for quantitative research in psychology: The
APA Publications and Communications Board task force report. American
Psychologist, 73(1), 3. https://psycnet.apa.org/doi/10.1037/amp0000389
Bonnefon, J. F., & Zhang, J. (2008). The intensity of recent and distant life regrets: An
integrated model and a large scale survey. Applied Cognitive Psychology: The Official
Journal of the Society for Applied Research in Memory and Cognition, 22(5), 653-
662. https://doi.org/10.1002/acp.1386
Brandt, M. J., IJzerman, H., Dijksterhuis, A., Farach, F. J., Geller, J., Giner-Sorolla, R.,
Grange, J. A., Perugini, M., Spies, J. R., & van't Veer, A. (2014). The replication
recipe: What makes for a convincing replication?. Journal of Experimental Social
Psychology, 50, 217-224. https://doi.org/10.1016/j.jesp.2013.10.005
Breugelmans, S. M., Zeelenberg, M., Gilovich, T., Huang, W. H., & Shani, Y. (2014).
Generality and cultural variation in the experience of regret. Emotion, 14(6), 1037.
https://doi.org/10.1037/a0038221
Broomhall, A. G., Phillips, W. J., Hine, D. W., & Loi, N. M. (2017). Upward counterfactual
thinking and depression: A meta-analysis. Clinical Psychology Review, 55, 56-73.
https://doi.org/10.1016/j.cpr.2017.04.010
source of inexpensive, yet high-quality, data? Perspectives on Psychological Science,
9(1), 35. https://doi.org/10.1177/1745691610393980
Byrne, R. M., & McEleney, A. (2000). Counterfactual thinking about actions and failures to
act. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(5),
1318. https://doi.org/10.1037/0278-7393.26.5.1318
Gilovich and Medvec (1994) Replication and Extension 41
Champely, S., Ekstrom, C., Dalgaard, P., Gill, J., Weibelzahl, S., Anandkumar, A., Ford, C.,
R package version, 1.
Chandrashekar, S. P., Yeung, S., Yau, K., Cheung, C., Agarwal, T. K., Wong, C., Pillai, T.,
Thirlwell, T. N., Leung, W., Li, Y., Tse, C., Cheng, B., Chan, H., & Feldman, G.
(2021). Agency and self-other asymmetries in perceived bias and shortcomings:
Replications of the Bias Blind Spot and extensions linking to free will
beliefs. Judgment and Decision Making, 16(6), 1392-1413.
https://sjdm.org/journal/20/201018/jdm201018.html
Charness, G., Gneezy, U., & Kuhn, M. A. (2012). Experimental methods: Between-subject
and within-subject design. Journal of Economic Behavior & Organization, 81(1), 1-8.
https://doi.org/10.1016/j.jebo.2011.08.009
Chen, J., Hui, L. S., Yu, T., Feldman, G., Zeng, S., Ching, T., Ng, C., Wu, K., Yuen, C., Lau,
T., Cheng, B., & Ng, K. (2021). Foregone opportunities and choosing not to act:
Replications of Inaction Inertia effect. Social Psychological and Personality Science,
12(3) 333-345. https://doi.org/10.1177/1948550619900570
Christley, R. M. (2010). Power and error: increased risk of false positive results in
underpowered studies. The Open Epidemiology Journal, 3(1).
https://doi.org/10.2174/1874297101003010016
Collaborative Open-science Research (2022). Replications and extensions of classic findings
in Social Psychology and Judgment and Decision Making. DOI
10.17605/OSF.IO/5Z4A8. Retrieved from http://osf.io/5z4a8 and https://mgto.org/pre-
registered-replications/
Connolly, T., Ordóñez, L. D., & Coughlan, R. (1997). Regret and responsibility in the
evaluation of decision outcomes. Organizational Behavior and Human Decision
Processes, 70(1), 73-85. https://doi.org/10.1006/obhd.1997.2695
Davison, I. M., & Feeney, A. (2008). Regret as autobiographical memory. Cognitive
Psychology, 57(4), 385-403. https://doi.org/10.1016/j.cogpsych.2008.03.001
Diefendorff, J. M., Hall, R. J., Lord, R. G., & Strean, M. L. (2000). Actionstate orientation:
Construct validity of a revised measure and its relationship to work-related variables.
Journal of Applied Psychology, 85(2), 250. https://doi.org/10.1037/0021-
9010.85.2.250
Eyal, P., David, R., Andrew, G., Zak, E., & Ekaterina, D. (2021). Data quality of platforms
and panels for online behavioral research. Behavior Research Methods, 1-20.
https://doi.org/10.3758/s13428-021-01694-3
Gilovich and Medvec (1994) Replication and Extension 42
Feeney, A., & Handley, S. J. (2006). Comparisons, mental models, and the action effect in
judgments of regret. Memory & Cognition, 34(7), 1422-1430.
https://doi.org/10.3758/bf03195907
Feldman, G. (2020). What is normal? Dimensions of action-inaction normality and their
impact on regret in the action-effect. Cognition and Emotion, 34(4), 728-742.
https://doi.org/10.1080/02699931.2019.1675598
Feldman, G., & Albarracín, D. (2017). Norm theory and the action-effect: The role of social
norms in regret following action and inaction. Journal of Experimental Social
Psychology, 69, 111-120. https://doi.org/10.1016/j.jesp.2016.07.009
Feldman, G., Chandrashekar, S. P., Yeung, S. K., Xiao, Q., Fillon, A., & Henne, P. (2021).
What is action, what is inaction? Clarifying action-inaction in judgment and decision
making and recommendations for term use and typology. [Manuscript in preparation].
Feldman, G., Kutscher, L., & Yay, T. (2020). Omission and commission in judgment and
decision making: Linking action-inaction effects using the concept of normality.
Social and Personality Psychology Compass, 14(8), e12557.
https://doi.org/10.1111/spc3.12557
Feldman, J., Miyamoto, J., & Loftus, E. F. (1999). Are actions regretted more than
inactions?. Organizational behavior and human decision processes, 78(3), 232-255.
https://doi.org/10.1006/obhd.1999.2833
Frijda, N. H., Kuipers, P., & Ter Schure, E. (1989). Relations among emotion, appraisal, and
emotional action readiness. Journal of Personality and Social Psychology, 57(2), 212.
https://doi.org/10.1037/0022-3514.57.2.212
Gilovich, T., Kerr, M., & Medvec, V. H. (1993). Effect of temporal perspective on subjective
confidence. Journal of Personality and Social Psychology, 64(4), 552.
https://doi.org/10.1037/0022-3514.64.4.552
Gilovich, T., & Medvec, V. H. (1994). The temporal pattern to the experience of regret.
Journal of Personality and Social Psychology, 67(3), 357.
https://doi.org/10.1037/0022-3514.67.3.357
Gilovich, T., & Medvec, V. H. (1995). The experience of regret: what, when, and why.
Psychological Review, 102(2), 379. https://doi.org/10.1037/0033-295x.102.2.379
Gilovich, T., Medvec, V. H., & Chen, S. (1995). Commission, omission, and dissonance
reduction: Coping with regret in the" Monty Hall" problem. Personality and Social
Psychology Bulletin, 21(2), 182-190. https://doi.org/10.1177/0146167295212008
Gilovich and Medvec (1994) Replication and Extension 43
Gilovich, T., Medvec, V. H., & Kahneman, D. (1998). Varieties of regret: A debate and
partial resolution. Psychological Review, 105(3), 602. https://doi.org/10.1037/0033-
295X.105.3.602
Gleicher, F., Kost, K. A., Baker, S. M., Strathman, A. J., Richman, S. A., & Sherman, S. J.
(1990). The role of counterfactual thinking in judgments of affect. Personality and
Social Psychology Bulletin, 16(2), 284-295.
https://doi.org/10.1177/0146167290162009
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world?.
Behavioral and brain sciences, 33(2-3), 61-83.
https://doi.org/10.1017/S0140525X0999152X
Huang, W. H., & Zeelenberg, M. (2012). Investor regret: The role of expectation in
comparing what is to what might have been. Judgment and Decision Making, 7(4),
441. http://sjdm.cybermango.org/journal/11/11303/jdm11303.pdf
Itzkin, A., Van Dijk, D., & Azar, O. H. (2016). At least I tried: The relationship between
regulatory focus and regret following action vs. inaction. Frontiers in Psychology, 7,
1684. https://doi.org/10.3389/fpsyg.2016.01684
Kahneman, D., & Miller, D. T. (1986). Norm theory: Comparing reality to its alternatives.
Psychological Review, 93(2), 136. https://doi.org/10.1037/0033-295X.93.2.136
Kahneman, D., & Tversky, A. (1982). The psychology of preferences. Scientific American,
246(1), 160-173. https://doi.org/10.1038/scientificamerican0182-160
Kordes-de Vaal, J. H. (1996). Intention and the omission bias: Omissions perceived as
nondecisions. Acta psychologica, 93(1-3), 161-172. https://doi.org/10.1016/0001-
6918(96)00027-3
Kutscher, L., & Feldman, G. (2019). The impact of past behaviour normality on regret:
replication and extension of three experiments of the exceptionality effect. Cognition
and Emotion, 33(5), 901-914. https://doi.org/10.1080/02699931.2018.1504747
Landman, J. (1987). Regret and elation following action and inaction: Affective responses to
positive versus negative outcomes. Personality and Social Psychology Bulletin, 13(4),
524-536. https://doi.org/10.1177/0146167287134009
Leach, F. R., & Plaks, J. E. (2009). Regret for errors of commission and omission in the
distant term versus near term: The role of level of abstraction. Personality and Social
Psychology Bulletin, 35(2), 221-229. https://doi.org/10.1177/0146167208327001
LeBel, E. P., McCarthy, R. J., Earp, B. D., Elson, M., & Vanpaemel, W. (2018). A unified
framework to quantify the credibility of scientific findings. Advances in Methods and
Gilovich and Medvec (1994) Replication and Extension 44
Practices in Psychological Science, 1, 389-402.
https://doi.org/10.1177/2515245918787489
LeBel, E. P., Vanpaemel, W., Cheung, I., & Campbell, L. (2019). A brief guide to evaluate
replications. Meta-Psychology, 3, 1-9. https://doi.org/10.15626/MP.2018.843
Litman, L., Robinson, J., & Abberbock, T. (2017). TurkPrime. com: A versatile
crowdsourcing data acquisition platform for the behavioral sciences. Behavior
research methods, 49(2), 433-442. https://doi.org/10.3758/s13428-016-0727-z
Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S. J., Buck,
S., Chambers, C. D., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E.,
Yarkoni, T. (2015). Promoting an open research culture. Science, 348(6242), 1422
1425. https://doi.org/10/gcpzwn
Ordónez, L. D., & Connolly, T. (2000). Regret and responsibility: A reply to Zeelenberg et
al. (1998). Organizational Behavior and Human Decision Processes, 81(1), 132-142.
https://doi.org/10.1006/obhd.1999.2834
Rajagopal, P., Raju, S., & Unnava, H. R. (2006). Differences in the cognitive accessibility of
action and inaction regrets. Journal of Experimental Social Psychology, 42(3), 302-
313. https://doi.org/10.1016/j.jesp.2005.05.003
Roese, N. J., Hur, T., & Pennington, G. L. (1999). Counterfactual thinking and regulatory
focus: Implications for action versus inaction and sufficiency versus necessity.
Journal of personality and social psychology, 77(6), 1109.
https://doi.org/10.1037/0022-3514.77.6.1109
RStudio Team. (2021). Integrated development for R. RStudio, IncBoston, MA
Savitsky, K., Medvec, V. H., & Gilovich, T. (1997). Remembering and regretting: The
Zeigarnik effect and the cognitive availability of regrettable actions and inactions.
Personality and Social Psychology Bulletin, 23(3), 248-257.
https://doi.org/10.1177/0146167297233004
Simmons, J. P., Nelson, L. D., & Simonsohn, U. (2012). A 21 word solution. Available at
SSRN 2160588.
Simonson, I. (1992). The influence of anticipating regret and responsibility on purchase
decisions. Journal of Consumer Research, 19(1), 105-118.
https://doi.org/10.1086/209290
Gilovich and Medvec (1994) Replication and Extension 45
Thomas, K. A., & Clifford, S. (2017). Validity and Mechanical Turk: An assessment of
exclusion methods and interactive experiments. Computers in Human Behavior, 77,
184197. https://doi.org/10.1016/j.chb.2017.08.038
Towers, A., Williams, M. N., Hill, S. R., Philipp, M. C., & Flett, R. (2016). What makes for
the most intense regrets? Comparing the effects of several theoretical predictors of
regret intensity. Frontiers in Psychology, 7, 1941.
https://doi.org/10.3389/fpsyg.2016.01941
Wickham, H. (2016). ggplot2: Elegant Graphics for Data Analysis. Springer-Verlag New
York. Retrieved from https://ggplot2.tidyverse.org
Xie, Y., Allaire, J. J., & Grolemund, G. (2018). R markdown: The definitive guide. CRC
Press.
Yeung, S. K., & Feldman, G. (2022). Action-Inaction Asymmetries in Emotions and
Counterfactual Thoughts: Meta-Analysis of the Action Effect [Registered Report
Stage 1]. https://doi.org/10.17605/OSF.IO/ACM24
Yeung, S. K., Yay, T., & Feldman, G. (2022). Action and inaction in moral judgments and
decisions:
Personality and Social Psychology
Bulletin. http://dx.doi.org/10.13140/RG.2.2.28468.58249
Zeelenberg, M., Van den Bos, K., Van Dijk, E., & Pieters, R. (2002). The inaction effect in
the psychology of regret. Journal of Personality and Social Psychology, 82(3), 314.
https://doi.org/10.1037/0022-3514.82.3.314
Zeelenberg, M., van der Pligt, J., & Manstead, A. S. (1998). Undoing regret on Dutch
television: Apologizing for interpersonal regrets involving actions or
inactions. Personality and Social Psychology Bulletin, 24(10), 1113-1119.
https://doi.org/10.1177/01461672982410008
Zeelenberg, M., Van Dijk, W. W., & Manstead, A. S. (2000). Regret and responsibility
resolved? Evaluating Ordóñez and Connolly's (2000) conclusions. Organizational
Behavior and Human Decision Processes, 81(1), 143-154.
https://doi.org/10.1006/obhd.1999.2865
Ziano, I., Kong, M., Kim, H., Liu, C., Wong, S., Cheng, B., & Feldman, G. (2021).
Revisiting Disjunction Effect: Replication of Tversky and Shafir (1992) and extension
comparing between and within subject designs. Journal of Economic Psychology, 83,
102350.
https://doi.org/10.1016/j.joep.2020.102350
Revisiting the Temporal Pattern of Regret:
Replication of Gilovich and Medvec (1994) with extensions
examining responsibility
Supplementary
Contents
Open Science disclosures ...................................................................................................... 4
Data and code .................................................................................................................... 4
Pre-registrations and Qualtrics study designs ..................................................................... 4
Procedure and data disclosures .......................................................................................... 4
Data collection .............................................................................................................. 4
Conditions reporting ...................................................................................................... 4
Participant exclusions .................................................................................................... 4
Variables reporting ........................................................................................................ 4
Data files ....................................................................................................................... 5
Table S1 ............................................................................................................ 5
Methods and Analyses of the original article ......................................................................... 6
Original Article Study 1A Methods ................................................................................... 6
Type of study ................................................................................................................ 6
One sample comparison study ....................................................................................... 6
Variable of Interest ........................................................................................................ 6
Sample size before and after exclusions ......................................................................... 6
Included sample description .......................................................................................... 6
Original Article Study 1B Methods ................................................................................... 6
Type of study ................................................................................................................ 6
One sample comparison study ....................................................................................... 6
Variable of Interest ........................................................................................................ 6
Sample size before and after exclusions ......................................................................... 7
Included sample description .......................................................................................... 7
Original Article Study 3 Methods ...................................................................................... 7
Type of study ................................................................................................................ 7
One sample comparison study ....................................................................................... 7
Independent variables (IV) ............................................................................................ 7
Dependent variables ...................................................................................................... 7
Sample size before and after exclusions ......................................................................... 8
Gilovich and Medvec (1994) replication and extension: Supplementary 2
Included sample description .......................................................................................... 8
Original Article Study 4 Methods ...................................................................................... 8
Type of study ................................................................................................................ 8
One sample comparison study ....................................................................................... 8
Independent variables (IV) ............................................................................................ 8
Dependent variables ...................................................................................................... 8
Sample size before and after exclusions ......................................................................... 8
Included sample description .......................................................................................... 9
Original Article Study 5 Methods ...................................................................................... 9
Type of study ................................................................................................................ 9
One sample comparison study ....................................................................................... 9
Independent variables (IV) ............................................................................................ 9
Dependent variables ...................................................................................................... 9
Sample size before and after exclusions ......................................................................... 9
Included sample description .......................................................................................... 9
Original article reported results ....................................................................................... 10
Table S2 .......................................................................................................... 10
Table S3 .......................................................................................................... 11
Effect Size and ............. 12
Original Article Study 1 .................................................................................................. 12
Original Article Study 3 .................................................................................................. 13
Original Article Study 4 .................................................................................................. 14
Original Article Study 5 .................................................................................................. 16
........ 18
Table S4 .......................................................................................................... 19
Evaluation criteria for replication findings ....................................................................... 21
Figure S1 ......................................................................................................... 21
Figure S2 ......................................................................................................... 22
Method of the replication + extension ................................................................................. 23
Method Tables of All Studies .......................................................................................... 23
Table S5 .......................................................................................................... 23
Table S6 .......................................................................................................... 25
Table S7 .......................................................................................................... 26
Table S8 .......................................................................................................... 27
Exclusion criteria ................................................................................................................ 28
Generalized exclusion criteria ......................................................................................... 28
Comparisons and deviations ................................................................................................ 29
Gilovich and Medvec (1994) replication and extension: Supplementary 3
Original versus replication ............................................................................................... 29
Table S9 .......................................................................................................... 29
Figure S3 ......................................................................................................... 30
Results Comparison Between Pre-exclusion and Post-exclusion .................................. 31
Table S10......................................................................................................... 31
Descriptive Statistics ................................................................................................... 33
Table S11......................................................................................................... 33
Exploratory Analyses of Study 5 ..................................................................................... 34
Table S12......................................................................................................... 35
Table S13......................................................................................................... 36
Table S14......................................................................................................... 37
Pre-registration plan versus final report ........................................................................... 38
Table S15......................................................................................................... 38
Other Limitations and Constraints of Generality .............................................................. 39
Future directions for broader action-inaction literature .................................................... 40
Future directions for replications evaluation .................................................................... 41
Data Collection Information ................................................................................................ 42
Study recruitment ............................................................................................................ 43
Writing task expectation alignment in consent ................................................................. 44
References .......................................................................................................................... 45
Gilovich and Medvec (1994) replication and extension: Supplementary 4
Open Science disclosures
Data and code
We shared data and code using the Open Science Framework. Review link for data and code
of the study: https://osf.io/7m3q2/
Pre-registrations and Qualtrics study designs
Link: https://osf.io/342td
Procedure and data disclosures
Data collection
We completed data collection before analyzing the actual data.
Conditions reporting
We reported all collected conditions.
Participant exclusions
We reported details in the Exclusion Criteria section of this document. We reported the
comparison between full results and post-exclusion results in the Pre-exclusions versus post-
exclusions section.
Variables reporting
We reported all variables and data collected for our studies.
Acknowledgement
We wrote some parts of the manuscript and the supplementary with reference to Feldman et
al. (2021b) template.
Gilovich and Medvec (1994) replication and extension: Supplementary 5
Data files
Table S1
Contents of datafiles
Data Filename
Content
Datasets -> Gilovich and Medvec 1994 Replication
Extension Dummy Data Analysis V2 K.Rmd
Pre-registered RMarkdown code file
Datasets -> Gilovich and Medvec 1994 Replication
Extension Pre Exclusion Full Data Analysis.Rmd
RMarkdown code file before any exclusion
Datasets -> Gilovich-and-Medvec-1994-Replication-
Extension-Pre-Exclusion-Full-Data-Analysis.html
Rmarkdown output file before any exclusion
Datasets -> Gilovich and Medvec 1994 Replication
Extension Post Exclusion Data Analysis.Rmd
RMarkdown code file after exclusion based on pre-
registered criteria and exploratory analyses of Study 5
after excluding irrelevant, incorrect, and no
regret/responsibility responses
Datasets -> Gilovich-and-Medvec-1994-Replication-
Extension-Post-Exclusion-Data-Analysis.html
RMarkdown output file after exclusion based on pre-
registered criteria and exploratory analyses of Study 5
after excluding irrelevant, incorrect, and no
regret/responsibility responses
Datasets ->
Gilovich+and+Medvec+(1994)+replication+and+exten
sion+V3-G_May+4,+2021_10.10.sav
Raw data in SAV format (used for final data analysis)
Datasets ->
Gilovich+and+Medvec+(1994)+replication+and+exten
sion+V3-G_May+4,+2021_10.09.csv
Raw data in CSV format
Datasets ->
Gilovich+and+Medvec+(1994)+replication+and+exten
sion+V3-G_May+4,+2021_10.10.xml
Raw data in XML format
Datasets ->
data_gilovichmedvec1994.sav
Post-exclusion data in SAV format
Gilovich and Medvec (1994) replication and extension: Supplementary 6
Methods and Analyses of the original article
Original Article Study 1A Methods
Type of study
One-sample comparison study
One sample comparison study
The participants compared whether they regret action or inaction more.
what would you say you regret more, those things that you did but wish you hadn't, or those
Variable of Interest
- experiencing stronger action regret versus stronger inaction regret
Sample size before and after exclusions
The original article did not report if there were exclusions of participants. 60 Participants
participated.
Included sample description
Age: M = 40.3, SD was not reported
Syracuse, New York, United States of America
General population, random sample of adults, recruited through a telephone directory
Original Article Study 1B Methods
Type of study
One-sample comparison study
One sample comparison study
The participants compared whether they regret their greatest action regret more or their
greatest inaction regret more.
Variable of Interest
- experiencing stronger action regret versus stronger inaction regret
Gilovich and Medvec (1994) replication and extension: Supplementary 7
Sample size before and after exclusions
The original article did not report if there were exclusions of participants. 30 Participants
participated.
Included sample description
Age: M = 40.1, SD was not reported
Chicago metropolitan area, United States of America
General population, random sample of adults, recruited through a telephone directory
Original Article Study 3 Methods
Type of study
One sample comparison study, within-subject
One sample comparison study
The participants compared whether Inaction Dave or Action Jim would experience stronger
regret, initially and in the long run.
University. Both are only moderately satisfied where they are and both are considering
transferring to another prestigious school. Each agonizes over the decision, going back and
forth between thinking he is going to stay and thinking he will leave. They ultimately make
different decisions: Dave opts to stay where he is and Jim decides to transfer. Suppose their
decisions turn out badly for both of them: Dave still doesn't like it where he is and wishes he
had transferred, and Jim doesn't like his new envi
They answered both of the questions below:
(a) Who do you think would regret his decision more on learning that it was a mistake? (b)
Who do you think would regret his decision more in the long run?
(p. 360)
Independent variables (IV)
Action vs Inaction (comparison), Temporal Distance: Short Term vs Long Term (within-
subject variable)
Dependent variables
Choice: The person who would experience stronger regret: Action Jim or Inaction Dave
Gilovich and Medvec (1994) replication and extension: Supplementary 8
Sample size before and after exclusions
The original article did not report if there were exclusions of participants. 80 Participants
participated.
Included sample description
Age: Not reported
Cornell University, United States of America
Undergraduate students sample
Original Article Study 4 Methods
Type of study
One sample comparison study, temporal distance as the between-subject factor
One sample comparison study
The participants compared whether Inaction Dave or Action Jim would experience stronger
regret, initially and in the long run.
University. Both are only moderately satisfied where they are and both are considering
transferring to another prestigious school. Each agonizes over the decision, going back and
forth between thinking he is going to stay and thinking he will leave. They ultimately make
different decisions: Dave opts to stay where he is and Jim decides to transfer. Suppose their
decisions turn out badly for both of them: Dave still doesn't like it where he is and wishes he
They were randomized and answered one of the questions below:
(a) Who do you think would regret his decision more on learning that it was a mistake? (b)
(p. 360)
Independent variables (IV)
Temporal Distance: Short Term vs Long Term (between-subject variable)
Dependent variables
Choice: The person who would experience stronger regret: Action Jim or Inaction Dave
Sample size before and after exclusions
The original article did not report if there were exclusions of participants. 34 participants in
the short-term condition and 42 participants in the long-term condition. 76 participants in
total
Gilovich and Medvec (1994) replication and extension: Supplementary 9
Included sample description
Age: Not reported
Cornell University, United States of America
Undergraduate students sample
Original Article Study 5 Methods
Type of study
One-sample comparison study, within-subject
One sample comparison study
The participants compared the intensity of regret between their greatest action regret and their
greatest inaction regret.
onnaire asked subjects to recall (but not write down)
their single most regrettable action and inaction from both the past week and from their entire
lives. Then, for each time period, the subjects were asked to indicate which they regretted
more, the acti
Independent variables (IV)
Action vs Inaction (comparison), temporal distance (short term: past week vs long term:
entire life)
Dependent variables
- experiencing stronger action regret versus stronger inaction regret
Sample size before and after exclusions
The original article did not report if there were exclusions of participants. 32 Participants
participated.
Included sample description
Age: Not reported
Ithaca, New York, United States of America
Adult samples from public places
Gilovich and Medvec (1994) replication and extension: Supplementary 10
Original article reported results
-life
experiences. Study 1A asked participants to compare if they regretted action or inaction
more. Study 1B asked participants to compare if they regretted their greatest action regret or
greatest inaction regret more. Study 3 and Study 4 are both scenario experiments with
temporal distance (short term vs long term) as the independent variable, in which Study 3
uses a within-subject design and Study 4 uses a between-subject design. Study 5 asked
participants about personal experiences of action-inaction regret, in the short run and in the
long run. For more information about the methods of these studies, please check Method of
the original article above.
Table S2
Descriptive Statistics and Reported Inferential Statistics of Original Article 1 Study 1
Study
N
Count of
participants
indicating
stronger
regret for
action
% of
participants
indicating
stronger regret
for action
Count of
participants
indicating
stronger regret
for inaction
% of
Inaction
indicating
stronger
regret for
inaction
Binomial
z
p
Study
1A
60
15
(Calculated)
25%
(Calculated)
45
75%
3.75
<.00
1
Study
1B
30
9
(Calculated)
30%
(Calculated)
21
70%
2.01
<.05
Gilovich and Medvec (1994) replication and extension: Supplementary 11
Table S3
Descriptive Statistics and Reported Inferential Statistics of Original Article 1 Study 3, Study
4, and Study 5
Study
N
Count of
participants
indicating stronger
regret for action
% of participants
indicating stronger
regret for action
Count of
participants
indicating stronger
regret for inaction
% of Inaction
indicating stronger
regret for inaction
Binomial z /
Chi square
statistics
p
Study 3
80
Short-Term:
61 (calculated)
Long-Term:
29 (calculated)
Short-Term: 76%
Long-Term: 36%
(calculated)
Short-Term:
19 (calculated)
Long-Term:
51 (calculated)
Short-Term: 24%
(calculated)
Long-Term: 64%
z = 2.35
<.02
Study 4
76
Short-Term:
26 (calculated)
Long-Term:
16 (calculated)
Short-Term: 76%
Long-Term: 38 %
(calculated)
Short-Term:
8 (calculated)
Long-Term:
26 (calculated)
Short-Term: 24%
(calculated)
Long-Term: 62%
x2 = 11.2
<.001
Study 5
32
Short-Term:
17 (calculated)
Long-Term:
7 (calculated)
Short-Term: 53%
Long-Term: 16%
(calculated)
Short-Term:
15 (calculated)
Long-Term:
25 (calculated)
Short-Term: 47%
(calculated)
Long-Term: 84%
z = 2.94
<.01
Gilovich and Medvec (1994) replication and extension: Supplementary 12
Effect Size and Confidence Interval Calculations of the original studies’ effects
Original Article Study 1
Based on the below code (Mangiafico, 2020), for Study 1A goodness of fit test, x2 (1) =
15.00, p < .001, V = 0.50 [0.27, 0.70].
library(rcompanion)
## Warning: package 'rcompanion' was built under R version 4.0.3
observed1a = c( 45, 15)
expected1a = c( 1/2, 1/2)
oneachisq <- chisq.test(x = observed1a, p = expected1a)
oneachisq
##
## Chi-squared test for given probabilities
##
## data: observed1a
## X-squared = 15, df = 1, p-value = 0.0001075
oneacramer <- cramerVFit(x = observed1a, p = expected1a, ci =
TRUE, conf = 0.95, type = "perc", R = 1000, reportIncomplete =
TRUE)
oneacramer
## Cramer.V lower.ci upper.ci
## 1 0.5 0.2667 0.7
Based on the below code, for Study 1B goodness of fit test, x2 (1) = 4.80, p = .028, V = 0.40
[0.07, 0.73].
library(rcompanion)
observed1b = c( 21, 9)
expected1b = c( 1/2, 1/2)
onebchisq <- chisq.test(x = observed1b, p = expected1b)
onebchisq
##
## Chi-squared test for given probabilities
##
## data: observed1b
## X-squared = 4.8, df = 1, p-value = 0.02846
onebcramer <- cramerVFit(x = observed1b, p = expected1b, ci = TRUE, conf
= 0.95, type = "perc", R = 1000, reportIncomplete = TRUE)
onebcramer
Gilovich and Medvec (1994) replication and extension: Supplementary 13
## Cramer.V lower.ci upper.ci
## 1 0.4 0.06667 0.7333
Original Article Study 3
Based on the below code, for Study 3 goodness of fit test in the short-term, x2 (1) = 22.05, p <
.001, V = 0.53 [0.35, 0.70].
library(rcompanion)
observed3s = c( 61, 19)
expected3s = c( 1/2, 1/2)
threeschisq <- chisq.test(x = observed3s, p = expected3s)
threeschisq
##
## Chi-squared test for given probabilities
##
## data: observed3s
## X-squared = 22.05, df = 1, p-value = 2.656e-06
threescramer <- cramerVFit(x = observed3s, p = expected3s, ci =
TRUE, conf = 0.95, type = "perc", R = 1000, reportIncomplete =
TRUE)
threescramer
## Cramer.V lower.ci upper.ci
## 1 0.525 0.325 0.7
Based on the below code, for Study 3 goodness of fit test in the long-term, x2 (1) = 6.05, p <
.001, V = 0.28 [0.05, 0.48].
library(rcompanion)
observed3l = c( 29, 51)
expected3l = c( 1/2, 1/2)
threelchisq <- chisq.test(x = observed3l, p = expected3l)
threelchisq
##
## Chi-squared test for given probabilities
##
Gilovich and Medvec (1994) replication and extension: Supplementary 14
## data: observed3l
## X-squared = 6.05, df = 1, p-value = 0.01391
threelcramer <- cramerVFit(x = observed3l, p = expected3l, ci =
TRUE, conf = 0.95, type = "perc", R = 1000, reportIncomplete =
TRUE)
threelcramer
## Cramer.V lower.ci upper.ci
## 1 0.275 0.075 0.475
For the association between temporal distance and action-inaction, we cannot calculate as it is
a within-subject study in which participants complete both questions so we cannot use the
Chi Square test of independence. The authors compared the proportion of participants
changing from short-term action with stronger regret to long-term inaction with stronger
regret for Study 5 to detect the association. However, the authors did not report z statistics or
other statistics that are necessary for effect size calculations in Study 3.
Original Article Study 4
Based on the below code, for Study 4 goodness of fit test in the short-term, x2 (1) = 9.53, p =
.002, V = 0.53 [0.24, 0.76].
library(rcompanion)
observed4s = c( 26, 8)
expected4s = c( 1/2, 1/2)
fourschisq <- chisq.test(x = observed4s, p = expected4s)
fourschisq
##
## Chi-squared test for given probabilities
##
## data: observed4s
## X-squared = 9.5294, df = 1, p-value = 0.002022
fourscramer <- cramerVFit(x = observed4s, p = expected4s, ci =
TRUE, conf = 0.95, type = "perc", R = 1000, reportIncomplete =
TRUE)
fourscramer
## Cramer.V lower.ci upper.ci
## 1 0.5294 0.2353 0.8235
Gilovich and Medvec (1994) replication and extension: Supplementary 15
Based on the below code, for Study 4 goodness of fit test in the long-term, x2 (1) = 2.38, p =
.123, V = 0.24 [0.00, 0.52].
library(rcompanion)
observed4l = c( 16, 26)
expected4l = c( 1/2, 1/2)
fourlchisq <- chisq.test(x = observed4l, p = expected4l)
fourlchisq
##
## Chi-squared test for given probabilities
##
## data: observed4l
## X-squared = 2.381, df = 1, p-value = 0.1228
fourlcramer <- cramerVFit(x = observed4l, p = expected4l, ci =
TRUE, conf = 0.95, type = "perc", R = 1000, reportIncomplete =
TRUE)
fourlcramer
## Cramer.V lower.ci upper.ci
## 1 0.2381 0 0.5238
Based on the below code (Signorell et al., 2020), for Study 4 test of association between
temporal distance and action-inaction, x2 (1) = 11.19, p < .001, V = 0.38 [0.16, 0.61].
library(DescTools)
## Warning: package 'DescTools' was built under R version 4.0.3
tab4 <- as.table(rbind(
c(26, 8),
c(16, 26)))
CramerV(tab4, conf.level = 0.95)
## Cramer V lwr.ci upr.ci
## 0.3837535 0.1589251 0.6085721
Fourasso <- chisq.test(tab4, correct = FALSE)
Fourasso
##
## Pearson's Chi-squared test
##
## data: tab4
## X-squared = 11.192, df = 1, p-value = 0.0008214
Gilovich and Medvec (1994) replication and extension: Supplementary 16
Original Article Study 5
Based on the below code, for Study 5 goodness of fit test in the short-term, x2 (1) = 0.13, p =
.724, V = 0.06 [0.00, 0.44].
library(rcompanion)
observed5s = c( 17, 15)
expected5s = c( 1/2, 1/2)
fiveschisq <- chisq.test(x = observed5s, p = expected5s)
fiveschisq
##
## Chi-squared test for given probabilities
##
## data: observed5s
## X-squared = 0.125, df = 1, p-value = 0.7237
fivescramer <- cramerVFit(x = observed5s, p = expected5s, ci =
TRUE, conf = 0.95, type = "perc", R = 1000, reportIncomplete =
TRUE)
fivescramer
## Cramer.V lower.ci upper.ci
## 1 0.0625 0 0.4359
Based on the below code, for Study 5 goodness of fit test in the long-term, x2 (1) = 10.13, p =
.001, V = 0.56 [0.25, 0.81].
library(rcompanion)
observed5l = c( 7, 25)
expected5l = c( 1/2, 1/2)
fivelchisq <- chisq.test(x = observed5l, p = expected5l)
fivelchisq
##
## Chi-squared test for given probabilities
##
## data: observed5l
## X-squared = 10.125, df = 1, p-value = 0.001463
fivelcramer <- cramerVFit(x = observed5l, p = expected5l, ci =
TRUE, conf = 0.95, type = "perc", R = 1000, reportIncomplete =
TRUE)
fivelcramer
## Cramer.V lower.ci upper.ci
Gilovich and Medvec (1994) replication and extension: Supplementary 17
## 1 0.5625 0.25 0.8125
Based on the below code, for Study 5 test of difference in the proportion of participants
changing from action with stronger regret in the short term to inaction in the long term to
participants changing from inaction with stronger regret in the short term to action with
stronger regret in the long term, x2 (1) = 8.64, p = .002. We are not able to find Cramer V as
the original authors did not report the number of participants with different responses to the
short term and long term questions.
#Study 5 Comparing Action to Inaction versus Inaction to Action, z
= 2.94, calculate chi square
chisqstudy5chan <- 2.94^2
chisqstudy5chan
## [1] 8.6436
pvalue <- pnorm(-abs(2.94))
pvalue
## [1] 0.001641061
Gilovich and Medvec (1994) replication and extension: Supplementary 18
Power analysis of original studies’ effects to assess required sample for replication
We conducted our power analysis based on meaningful effects that the original study found,
aiming for 95% power with .05 as the significance level. One tricky issue is that, in Study 4
comparison between action regret and inaction regret in the long term, there was an effect of
V
361), a-term, but inactions are regretted
(Abstract, p. 357). It was possible that significance was not reached due to a lack of statistical
power. Therefore, the minimum sample size, in the long-term condition, required to detect
such an effect would be 230. We randomly randomized participants into either Study 3 or
Study 4, meaning around half of all participants participated in Study 4 and half of all
participants participated in Study 3. Then within the half of participants participating in Study
4, half of those participants went through the long-term condition. Also, considering that we
may exclude participants due to various reasons, we aimed to recruit at least 1000
participants. We expected that we would exclude 80 participants, leading to 920 participants.
Half of 920 participants (460 participants) participated in Study 4, and half of 460
participants (230) participated in the long-term condition.
Gilovich and Medvec (1994) replication and extension: Supplementary 19
Table S4
Power Analysis and Minimum Sample Size Required
Study
Effect size
R Code and Outputs / G*Power Outputs (Faul et al.,
2007)
Sample
Required
Study 1A
V = 0.5
library(pwr)
pwr.chisq.test(w = 0.5, df = 1, power =
0.95, sig.level = 0.05)
Output: N = 51.97884
52
Study 3
Short Term
V = 0.525
library(pwr)
pwr.chisq.test(w = 0.525, df = 1, power =
0.95, sig.level = 0.05)
Output: N = 47.14635
48
Study 3
Long Term
V = 0.275
library(pwr)
pwr.chisq.test(w = 0.275, df = 1, power =
0.95, sig.level = 0.05)
Output: N = 171.8308
172
Study 3
Association
Insufficient
info
Insufficient information, not applicable
Study 4
Short Term
V = 0.5294
library(pwr)
pwr.chisq.test(w = 0.5294, df = 1, power =
0.95, sig.level = 0.05)
Output: N = 46.36591
47
Study 4
Long Term
V = 0.2381
library(pwr)
pwr.chisq.test(w = 0.2381, df = 1, power =
0.95, sig.level = 0.05)
Output: N = 229.2175
230
Study 4
Association
V =
0.3837535
library(pwr)
pwr.chisq.test(w = 0.3837535, df = 1,
power = 0.95, sig.level = 0.05)
Output: N = 88.23928
89
Study 5
Short Term
V =
0.0625*
library(pwr)
pwr.chisq.test(w = 0.0625, df = 1, power =
0.95, sig.level = 0.05)
Output: N = 3326.646
3327*
Study 5
Long Term
V = 0.5625
library(pwr)
pwr.chisq.test(w = 0.5625, df = 1, power =
0.95, sig.level = 0.05)
Output: N = 41.0697
42
Gilovich and Medvec (1994) replication and extension: Supplementary 20
Study
Effect size
R Code and Outputs / G*Power Outputs (Faul et al.,
2007)
Sample
Required
Study 5
Association
= 0.43
Exact - Proportion: Sign test (binomial test)
Analysis: A priori: Compute required sample size
Input: Tail(s) = One
Effect size g = 0.43
α err prob = 0.05
Power (1-β err prob) = 0.95
Output: Lower critical N = 9.0000000
Upper critical N = 9.0000000
Total sample size = 11
Actual power = 0.9630207
Actual α = 0.0327148
11
Note. 1) *As the original article found a minimal effect, we do not determine our sample size
based on the sample size calculation from Study 5 Short-Term action-inaction regret
comparison. 2) We reported confidence intervals in the Effect Size and Confidence Interval
Calculations of the original stud effects section. We used effect size calculated, without
rounding, to conduct our power analyses.
Gilovich and Medvec (1994) replication and extension: Supplementary 21
Evaluation criteria for replication findings
We aimed to compare the replication effects with the original effects in the target article
using the criteria set by LeBel et al. (2019) (see Figure S1 and Figure S2).
Figure S1
Interpretation criteria for evaluation of replications outcomes by LeBel et al. (2019)
Note. For comparison of replication effect size confidence intervals with original effect size.
Applicable if the original study detected a signal.
Gilovich and Medvec (1994) replication and extension: Supplementary 22
Figure S2
Interpretation criteria for evaluation of replications outcomes by (LeBel et al., 2019)
Note. Applicable if the original study did not detect a signal.
Gilovich and Medvec (1994) replication and extension: Supplementary 23
Method of the replication + extension
Please see the exported Qualtrics file for all materials: https://osf.io/342td/files
(Gilovich_and_Medvec_1994_replication_and_extension_V3-G.qsf)
Method Tables of All Studies
Table S5
Study 5: Greatest Lifetime and Past Week Regret and Responsibility Design
Greatest Regret Comparison Studies: The presentation of questions and choices are in
random order. Participants complete both questions below. We randomized participants
into the regret condition or the responsibility condition, to prevent participants from
describing the same event.
Temporal Distance as Within-Subject
Independent Variable: Lifetime
Condition
Replication on Regret:
Action:
Now we want you to think of your single
biggest regret of action of your entire life,
something you did, that in retrospect
you wish you had not done.
Take a moment to think of your biggest
regret of this type.
Inaction:
Now we want you to think of your single
biggest regret of inaction from your
entire life, something you did not do,
that in retrospect you wish you had
done.
Take a moment to think of your biggest
regret of this type.
Extension on Responsibility:
Action:
We would like you to think about an
action that resulted in a negative outcome
that you feel most personally responsible
for in your entire life.
Take a moment to think of a negative
outcome that resulted from you taking
action, something you did that led to an
Temporal Distance as Within-Subject
Independent Variable: Last Week
Condition
Replication on Regret:
Action:
Now we want you to think of your single
biggest regret of action of your past week,
something you did, that in retrospect you
wish you had not done.
Take a moment to think of your biggest
regret of this type.
Inaction:
Now we want you to think of your single
biggest regret of inaction from your past
week, something you did not do, that in
retrospect you wish you had done.
Take a moment to think of your biggest
regret of this type.
Extension on Responsibility:
Action:
We would like you to think about an action
that resulted in a negative outcome that you
feel most personally responsible for in the
past week.
Take a moment to think of a negative
outcome that resulted from you taking
action, something you did that led to an
Gilovich and Medvec (1994) replication and extension: Supplementary 24
undesired outcome you felt most
responsible for.
Inaction:
We would like you to think about an
inaction that resulted in a negative
outcome that you feel most personally
responsible for in your entire life.
Take a moment to think of a negative
outcome that resulted from you not taking
action, something you did not do that led
to an undesired outcome you felt most
responsible for.
undesired outcome you felt most responsible
for.
Inaction:
We would like you to think about an
inaction that resulted in a negative outcome
that you feel most personally responsible
for in the past week.
Take a moment to think of a negative
outcome that resulted from you not taking
action, something you did not do that led to
an undesired outcome you felt most
responsible for.
Dependent Variable: Stronger Action or Inaction Regret and Responsibility
Participants will be asked optionally to describe their regrets and responsibilities before
comparing their regrets/responsibilities below.
Two replication regret questions (one for the past week, one for lifetime): You've thought
about two biggest regrets of the past week, if you could change only one of them, which
one would it be?
· Change biggest regret of action
· Change biggest regret of inaction
Two extension responsibility questions (one for the past week, one for lifetime): You've
thought about those two decisions you feel very responsible for, which decision do you
feel most personally responsible for?
· The action that you feel most personally responsible for the negative
outcome
· The inaction that you feel most personally responsible for the negative
outcome
For each of the above questions, we also asked participants to explain why
Note. The above design is a replication of Study 5 of the original article, in which we added
questions on responsibility as the extension.
Gilovich and Medvec (1994) replication and extension: Supplementary 25
Table S6
Study 1A: General Regret and Responsibility Design
Study 1A General Regret and Responsibility Comparison Study: The presentation of
questions and choices are in random order. Participants complete both questions below.
Replication Question on Regret: This part is about your overall general regret
experiences in your life. When you look back on your experiences in life and think of
those things that you regret, what would you say you regret more, those things that you
did but wish you hadn't, or those things that you didn't do but wish you had?
Things that I did
Things that I did not do
Extension Question on Responsibility: This part is about your overall general
experiences that you feel personally responsible for in your life. When you look back on
your experiences in life and think of those things that you feel personally responsible
for, what would you say you feel personally responsible more, those things that you did
but wish you hadn't, or those things that you didn't do but wish you had?
Things that I did
Things that I did not do
Gilovich and Medvec (1994) replication and extension: Supplementary 26
Table S7
Study 3 Scenario Experiment Design
Scenario
University. Both are only moderately satisfied where they are and both are considering
transferring to another prestigious school. Each agonizes over the decision, going back and forth
between thinking he is going to stay and thinking he will leave. They ultimately make different
decisions: Dave opts to stay where he is and Jim decides to transfer. Suppose their decisions turn
out badly for both of them: Dave still doesn't like it where he is and wishes he had transferred, and
Medvec, 1994)
Temporal Distance as Within-Subject
Independent Variable: Short Term condition
regret his
decision more on learning that it was a
feel more responsible for learning that it was a
Temporal Distance as Within-Subject
Independent Variable: Long Term
condition
regret his
decision more in the long run
feel more responsible for in the long run
Dependent Variable
Comparison of intensity of regret (replication) and intensity of responsibility (extension) between
action Jim and inaction Dave. For both of the above conditions, participants chose between:
1) Jim (who transferred)
2) Dave (who stayed)
Gilovich and Medvec (1994) replication and extension: Supplementary 27
Table S8
Study 4 Scenario Experiment Design
Scenario
University. Both are only moderately satisfied where they are and both are considering
transferring to another prestigious school. Each agonizes over the decision, going back and forth
between thinking he is going to stay and thinking he will leave. They ultimately make different
decisions: Dave opts to stay where he is and Jim decides to transfer. Suppose their decisions turn
out badly for both of them: Dave still doesn't like it where he is and wishes he had transferred, and
Medvec, 1994)
Temporal Distance as Between-Subject
Independent Variable: Short Term condition
regret his decision more on learning that it was
feel more responsible for learning that it was a
Temporal Distance as Between-Subject
Independent Variable: Long Term
condition
would regret his decision more in the long
run
would
feel more responsible for in the long run
Dependent Variable
Comparison of intensity of regret (replication) and intensity of responsibility (extension) between
action Jim and inaction Dave. For one of the above conditions, participants chose between:
1) Jim (who transferred)
2) Dave (who stayed)
Gilovich and Medvec (1994) replication and extension: Supplementary 28
Exclusion criteria
Generalized exclusion criteria
We pre-registered the below general criteria for exclusion:
1. Participants indicating a low proficiency of English (self-report < 5, on a 1-7 scale)
2. Participants who self-reported not being serious about filling in the survey (self-report
< 4, on a 1-5 scale).
3. Participants who correctly guessed the hypotheses of this study in the funnelling
section. This means including 3 or more of these terms or terms with similar
4. Participants who had already seen or done the survey before.
5. Participants who failed to complete the survey.
6. Participants who are not from the United States.
We report post-exclusion results in the main manuscript and pre-exclusion results in the
supplementary.
Gilovich and Medvec (1994) replication and extension: Supplementary 29
Comparisons and deviations
Original versus replication
Table S9
Original vs Replication Method Comparison
Original
Replication
Reason for change
Stimuli
1) The wordings across
conditions (action and
inaction, one week versus
lifetime) in Study 5 are not
consistent. See photo
screenshots from the
original authors.
2) Study 5 regret
comparison question
1) We ensured the (non-
manipulated) wordings
across the conditions (e.g.
noting that participants do
not need to disclose
comfortable, asking
participants to take a
moment to think) are
consistent
f you
1) To prevent any
possible effect due to
differences in wordings
between conditions
seems only suitable for
action but not inaction,
as undoing inaction does
not make sense
Procedure
Study 1, Study 3, Study 4,
and Study 5 are separately
given to different groups
of participants
We combined them into a
single survey
This is more convenient
and efficient
Gilovich and Medvec (1994) replication and extension: Supplementary 30
Figure S3
Criteria for evaluation of replications by LeBel et al. (2018)
Target similarity
Highly similar
Highly dissimilar
Category
Direct replication
Conceptual replication
Design facet
Exact
replication
Very close
replication
Close
replication
Far
replication
Very far
replication
Effect/hypothesis
Same/similar
Same/similar
Same/similar
Same/similar
Same/similar
IV construct
Same/similar
Same/similar
Same/similar
Same/similar
Different
DV construct
Same/similar
Same/similar
Same/similar
Same/similar
Different
IV
operationalization
Same/similar
Same/similar
Same/similar
Different
DV
operationalization
Same/similar
Same/similar
Same/similar
Different
Population (e.g.
age)
Same/similar
Same/similar
Same/similar
Different
IV stimuli
Same/similar
Same/similar
Different
DV stimuli
Same/similar
Same/similar
Different
Procedural details
Same/similar
Different
Physical setting
Same/similar
Different
Contextual
variables
Different
Note. A classification of relative methodological similarity of a replication study to an
refers to design facets over which a researcher has control over. Procedural
details refer to minor experimental aspects (e.g. wording and font). We added the "Similar"
category to the LeBel et al. (2018) typology, referring to minor deviations aimed to adjust the
studies to our target sample that likely do not have substantial influences on the replication
results.
We classified Studies 1, 3, and 4 as very close replications and Study 5 as a close replication.
Gilovich and Medvec (1994) replication and extension: Supplementary 31
Results Comparison Between Pre-exclusion and Post-exclusion
Table S10
Summary Table of Pre-Exclusion (full results) and Post-Exclusion Results
Full Results
Post-Exclusion Results
Study
Chi-Square
Effect Size
p
Chi-Square
Effect Size
P
Study 1 Regret
χ2 (1, N =
546) =
33.88
V = 0.25,
95% CI
[0.17, 0.33]
< .001
χ2 (1, N =
535) = 34.07
V = 0.25,
95% CI
[0.17, 0.34]
< .001
Study 1
Responsibility
N =
471) = 25.23
V = 0.23,
95% CI
[0.14, 0.32]
<.001
N =
453) = 24.34
V = 0.23,
95% CI
[0.14, 0.32]
<.001
Study 3 Short-
Term Regret
χ2 (1, N =
498) =
27.41
V = 0.23,
95% CI
[0.14, 0.31]
< .001
χ2 (1, N =
493) = 24.99
V = 0.23,
95% CI
[0.14, 0.31]
< .001
Study 3
Long-Term Regret
χ2 (1, N =
498) =
13.24
V = 0.16,
95% CI
[0.08, 0.24]
< .001
χ2 (1, N =
493) = 11.41
V = 0.15,
95% CI
[0.06, 0.24]
< .001
Study 3 Temporal
Distance and
Action-Inaction
Regret
χ2 (1, N =
142) =
70.42
OR = 0.17,
95% CI
[0.10, 0.28]
< .001
χ2 (1, N =
133) = 65.03
OR = 0.18,
95% CI
[0.10, 0.29]
< .001
Study 3 Short-
Term
Responsibility
χ2 (1, N =
508) =
117.20
V = 0.48,
95% CI
[0.40, 0.56]
< .001
χ2 (1, N =
493) =
112.02
V = 0.48,
95% CI
[0.40, 0.55]
< .001
Study 3
Long-Term
Responsibility
χ2 (1, N =
508) =
22.96
V = 0.21,
95% CI
[0.13, 0.30]
< .001
χ2 (1, N =
493) = 22.36
V = 0.21,
95% CI
[0.13, 0.29]
< .001
Study 3
Temporal
Distance and
Action-Inaction
Responsibility
χ2 (1, N =
123) =
36.70
OR = 0.30,
95% CI
[0.19, 0.46]
< .001
χ2 (1, N =
123) = 34.35
OR = 0.31,
95% CI
[0.20, 0.47]
< .001
Study 4 Short-
Term Regret
χ2 (1, N =
254) =
16.13
V = 0.25,
95% CI
[0.12, 0.37]
< .001
χ2 (1, N =
247) = 14.09
V = 0.24,
95% CI
[0.12, 0.36]
< .001
Study 4
Long-Term Regret
χ2 (1, N =
255) = 3.30
V = 0.11,
95% CI
[0.01, 0.23]
.069
χ2 (1, N =
248) = 2.73
V = 0.10,
95% CI
[0.01, 0.23]
.099
Study 4 Regret
Temporal
Distance and
Action-Inaction
Test of
Independence
χ2 (1, N =
509) =
17.10
V = 0.18
95% CI
[0.10, 0.27]
< .001
χ2 (1, N =
495) = 14.68
V = 0.17,
95% CI
[0.08, 0.26]
< .001
Gilovich and Medvec (1994) replication and extension: Supplementary 32
Full Results
Post-Exclusion Results
Study
Chi-Square
Effect Size
p
Chi-Square
Effect Size
P
Study 4 Short-
Term
Responsibility
χ2 (1, N =
254) =
62.50
V = 0.50,
95% CI
[0.38, 0.60]
< .001
χ2 (1, N =
247) = 61.25
V = 0.50,
95% CI
[0.39, 0.60]
< .001
Study 4
Long-Term
Responsibility
χ2 (1, N =
255) =
36.90
V = 0.38,
95% CI
[0.26, 0.50]
< .001
χ2 (1, N =
248) = 34.13
V = 0.37,
95% CI
[0.25, 0.49]
< .001
Study 4
Responsibility
Temporal
Distance and
Action-Inaction
Test of
Independence
χ2 (1, N =
509) = 2.11
V = 0.06,
95% CI
[0.00, 0.15]
.147
χ2 (1, N =
495) = 2.46
V = 0.07,
95% CI
[0.00, 0.16]
.117
Study 5
Past Week Regret
χ2 (1, N =
546) = 0.89
V = 0.04,
95% CI
[0.00, 0.13]
.346
χ2 (1, N =
535) = 1.17
V = 0.05,
95% CI
[0.00, 0.13]
.280
Study 5 Lifetime
Regret
χ2 (1, N =
546) = 0.89
V = 0.04,
95% CI
[0.00, 0.13]
.346
χ2 (1, N =
535) = 0.99
V = 0.04,
95% CI
[0.00, 0.13]
.320
Study 5
Temporal
Distance and
Action-Inaction
Regret
χ2 (1, N =
270) = 0.00
OR = 1.00,
95% CI
[0.78, 1.28]
1
χ2 (1, N =
265) = 0.00
OR = 0.99,
95% CI
[0.77, 1.27]
.951
Study 5
Past Week
Responsibility
χ2 (1, N =
471) = 6.90
V = 0.12,
95% CI
[0.03, 0.21]
.009
χ2 (1, N =
453) = 7.17
V = 0.13,
95% CI
[0.03, 0.22]
.007
Study 5 Lifetime
Responsibility
χ2 (1, N =
471) = 8.43
V = 0.13,
95% CI
[0.04, 0.22]
.004
χ2 (1, N =
453) = 7.68
V = 0.13,
95% CI
[0.03, 0.22]
.006
Study 5
Temporal
Distance and
Action-Inaction
Responsibility
χ2 (1, N =
207) = 0.04
OR = 1.03,
95% CI
[0.78, 1.37]
.835
χ2 (1, N =
201) = 0.00
OR = 1.01,
95% CI
[0.76, 1.35]
.944
Gilovich and Medvec (1994) replication and extension: Supplementary 33
Descriptive Statistics
Table S11
Full sample: Descriptive statistics without exclusion
Study
Action
Count Stronger
Regret or
Responsibility
Action
Percentage Stronger
Regret or
Responsibility
Inaction
Count Stronger
Regret or
Responsibility
Inaction
Percentage Stronger
Regret or
Responsibility
Study 1 General
Regret
205/546
37.55%, 95% CI
[33.58%, 41.68%]
341/546
62.45%, 95% CI
[58.32%, 66.42%]
Study 1 General
Responsibility
290/471
61.57%, 95% CI
[57.10%, 65.85%]
181/471
38.43%, 95% CI
[34.15%, 42.90%]
Study 3 Short Term
Regret
313/508
61.61%, 95% CI
[57.31%, 65.74%]
195/508
38.39%, 95% CI
[48.69%, 60.54%]
Study 3 Long Term
Regret
213/508
41.93%, 95% CI
[37.71%, 46.26%]
295/508
58.07%, 95% CI
[53.74%, 62.29%]
Study 3 Short Term
Responsibility
376/508
74.02%, 95% CI
[70.03%, 77.64%]
132/508
25.98%, 95% CI
[22.36%, 29.97%]
Study 3 Long Term
Responsibility
308/508
60.63%, 95% CI
[56.32%, 64.78%]
200/508
39.37%, 95% CI
[35.22%, 43.68%]
Study 4 Short Term
Regret
159/254
62.60%, 95% CI
[56.50%, 68.32%]
95/254
37.40%, 95% CI
[31.68%, 43.50%]
Study 4 Long Term
Regret
113/255
44.31%, 95% CI
[38.35%, 50.45%]
142/255
55.69%, 95% CI
[49.55%, 61.65%]
Study 4 Short Term
Responsibility
190/254
74.80%, 95% CI
[69.12%, 79.75%]
64/254
25.20%, 95% CI
[20.25%, 30.88%]
Study 4 Long Term
Responsibility
176/255
69.02%, 95% CI
[63.10%, 74.38%]
79/255
30.98%, 95% CI
[25.62%, 36.90%]
Study 5 Greatest Past
Week Regret
284/546
52.01%, 95% CI
[47.82%, 56.18%]
262/546
47.99%, 95% CI
[43.82%, 52.18%]
Study 5 Greatest
Lifetime Regret
284/546
52.01%, 95% CI
[47.82%, 56.18%]
262/546
47.99%, 95% CI
[43.82%, 52.18%]
Study 5 Greatest Past
Week Responsibility
264/471
56.05%, 95% CI
[51.54%, 60.47%]
207/471
43.95%, 95% CI
[39.53%, 48.46%]
Study 5 Greatest
Lifetime
Responsibility
267/471
56.69%, 95% CI
[52.18%, 61.09%]
204/471
43.31%, 95% CI
[38.91%, 47.82%]
Gilovich and Medvec (1994) replication and extension: Supplementary 34
Exploratory Analyses of Study 5
We did not pre-register the following analyses. We
descriptions of past week and lifetime action and inaction regrets and responsibilities, some
participants entered irrelevant responses in which participants described past week
regrets/responsibilities in lifetime regrets/responsibilities box or participants described
lifetime regrets/responsibilities in past week regrets/responsibilities box, incorrect responses
in which participants described actions in the inaction description box or described inactions
in the action description box
1
, or responses not stating a personal responsibility or a personal
regret of a certain type (past week/lifetime action/inaction e.g. no regret / no responsibility
responses, the participant is not the actor/the non-actor, events with positive outcomes, or
completely not related to regret or responsibility). In the following, we first report the
proportion of participants excluded, and then we report the findings (Table S13 and Table
S14), which are very similar to findings after pre-registered exclusion and full results without
any exclusion reported above.
To summarize, for lifetime regret, 9 out of 535 (1.68%) responses were excluded due to
above reasons, whereas for past week regret, 20 out of 535 (3.74%) responses were excluded.
For lifetime responsibility, 15 out of 453 (3.31%) responses were excluded, whereas for past
week responsibility, 33 out of 453 (7.28%) responses were excluded. This demonstrates that
data quality is decent overall, with only a low % of participants not following the
instructions. For more specific information regarding proportions of responses excluded due
to different reasons mentioned above (past-week-lifetime reversal, action-inaction reversal,
non-regret/non-responsibility responses), please see Table S12. We note that both action and
inaction responses of a number of participant(s) were/was excluded, so the overall number of
participants excluded is smaller than the sum of responses excluded. For example, for
lifetime action regret, 7 responses were excluded whereas for lifetime inaction regret, 3
responses were excluded, but overall, only 9 participants were excluded as 1 participant did
not provide appropriate responses for both action and inaction regrets.
1
We understand that action and inaction are sometimes not clearly defined. Sometimes a response involves an
action and an inaction, and sometimes a response involves insufficient action which can be considered as action
or inaction. In this exploratory analysis, we did not exclude those responses, and we only excluded responses
that are clearly incorrect using any definition of action and inaction discussed in Feldman et al. (2021a).
Gilovich and Medvec (1994) replication and extension: Supplementary 35
Table S12
Proportions of Participants Excluded in the Exploratory Analyses due to
Irrelevant/Incorrect/Non-Regret/Non-Responsibility Responses
Study/Temporal
Distance/Action or
Inaction/Regret or
Responsibility
Past Week-Lifetime
Reversals
Action-Inaction
Reversals
Non-Regret / Non-
Responsibility
Responses
Excluded Proportion
Study 5
Lifetime Action
Regret
0/535 (0%)
7/535 (1.31%)
0/535 (0%)
7/535 (1.31%)
Study 5
Lifetime Inaction
Regret
0/535 (0%)
1/535 (0.19%)
2/535 (0.37%)
3/535 (0.56%)
Study 5
Past Week Action
Regret
1/535 (0.19%)
5/535 (0.93%)
10/535 (1.87%)
16/535 (2.99%)
Study 5
Past Week Inaction
Regret
0/535 (0%)
2/535 (0.37%)
6/535 (1.12%)
8/535 (1.50%)
Study 5
Lifetime Action
Responsibility
0/453 (0%)
8/453 (1.50%)
2/453 (0.44%)
10/453 (2.21%)
Study 5
Lifetime Inaction
Responsibility
0/453 (0%)
6/453 (1.32%)
3/453 (0.66%)
9/453 (1.99%)
Study 5
Past Week Action
Responsibility
0/453 (0%)
14/453 (3.09%)
14/453 (3.09%)
28/453 (6.18%)
Study 5
Past Week Inaction
Responsibility
0/453 (0%)
3/453 (0.66%)
16/453 (3.53%)
19/453 (4.19%)
Gilovich and Medvec (1994) replication and extension: Supplementary 36
Table S13
Inferential Statistics after Excluding Irrelevant/Incorrect/Non-Regret/Non-Responsibility
Responses
Study/Temporal
Distance/Regret or
Responsibility
Chi-Square
Effect Size
p
Study 5
Past Week Regret
χ2 (1, N = 515) = 1.63
V = 0.06, 95% CI [0.00, 0.15]
.201
Study 5
Lifetime Regret
χ2 (1, N = 526) = 1.29
V = 0.05, 95% CI [0.00, 0.14]
.257
Study 5
Temporal Distance and
Action-Inaction Regret
χ2 (1, N = 262) = 0.06
OR = 0.97, 95% CI [0.76, 1.25]
.805
Study 5
Past Week Responsibility
χ2 (1, N = 420) = 6.94
V = 0.13, 95% CI [0.04, 0.22]
.008
Study 5 Lifetime
Responsibility
χ2 (1, N = 438) = 7.68
V = 0.13, 95% CI [0.05, 0.22]
.006
Study 5
Temporal Distance and
Action-Inaction Responsibility
χ2 (1, N = 193) = 0.02
OR = 0.97, 95% CI [0.72 1.30]
.886
Gilovich and Medvec (1994) replication and extension: Supplementary 37
Table S14
Descriptive Statistics after Excluding Irrelevant/Incorrect/No Regret/Responsibility
Responses
Study/Temporal
Distance/Regret
or Responsibility
Action
Count Stronger
Regret or
Responsibility
Action
Percentage Stronger
Regret or
Responsibility
Inaction
Count Stronger
Regret or
Responsibility
Inaction
Percentage Stronger
Regret or
Responsibility
Study 5 Greatest
Past Week
Regret
272/515
52.82%, 95% CI
[48.50%, 57.09%]
243/515
47.18%, 95% CI
[42.91%, 51.50%]
Study 5 Greatest
Lifetime Regret
276/526
52.47%, 95% CI
[48.20%, 56.71%]
250/526
47.53%, 95% CI
[43.29%, 51.80%]
Study 5 Greatest
Past Week
Responsibility
237/420
56.43%, 95% CI
[51.65%, 61.09%]
183/420
43.57%, 95% CI
[38.91%, 48.35%]
Study 5 Greatest
Lifetime
Responsibility
251/447
56.15%, 95% CI
[51.52%, 60.68%]
196/446
43.85%, 95% CI
[39.32%, 48.48%]
Gilovich and Medvec (1994) replication and extension: Supplementary 38
Pre-registration plan versus final report
We completed Preregistration Planning and Deviation Documentation (PPDD) (V
al., 2019) below.
Table S15
Preregistration Planning and Deviation Documentation
Components in
your pre-
registration (e.g.,
stopping rule,
analyses,
hypotheses,
exclusion rules)
Location of 1) pre-
registered decision/plan and
2) rationale for
decision/plan
[Location / link]
Were there
deviations?
What type?
[no / minor
/ major]*
If yes - describe details of
deviation(s)
[brief description / location /
link]
Rationale for
deviation
[brief description /
location / link]
How might
the results
be different
if you
had/had not
deviated
[brief
description /
location /
link]
Date/time
of decision
for
deviation +
stage
Data analysis
Manuscript:
https://osf.io/342td/files/
Results section
1) It did not specify any
exploratory analysis
2) For plots of Study 3, Study
4, and Study 5, we separated
plots with regret and plots
with responsibility. The pre-
registered plots did not
include error bars, effect
sizes, middle line, and
proportions of action-
inaction.
Yes, minor.
1) For Study 5, we conducted
additional analyses excluding
responses that are irrelevant,
incorrect, or no regret/no
responsibility responses.
2) We combined plots of
regret and plots of
responsibility into the same
plot. We also included error
bars, effect sizes, middle line,
and proportions in the plots.
Location: See results section
of the final main manuscript.
1) We want to test if
there are any
differences in results
based on this
exclusion, compared
to full results or pre-
registered exclusion
results
2) It is much more
reader-
friendly and clearer
for readers,
reviewers, and
examiners.
1) No
meaningful
difference,
check
Exploratory
Analyses of
Study 5.
2) No
difference.
1) May
2021
Post data
collection
data
analysis
stage
2) June
2021
Post data
collection
data
analysis
stage
Gilovich and Medvec (1994) replication and extension: Supplementary 39
Other Limitations and Constraints of Generality
In terms of the temporal constraint of generality, we conducted our replications during
COVID-19, a pandemic in which many people stayed indoors more often than before. A
studies (Constandt et al., 2020). There may be minor effects on only short-term regrets. But
we doubt the impact is substantial, as we did not investigate the frequency of action and
inaction regrets, but whether people experience stronger regret for action or inaction.
Moreover, even though MTurk samples are more diverse and representative than the samples
in the original article, we recruited only American participants. Relying on WEIRD (Western,
Educated, Industrialized, Rich, and Democratic) samples is often criticized as a key problem
in psychology (Henrich et al., 2010). Chen et al. (2006) and Gilovich et al. (2003) found that
for lifetime regrets based on actual experiences, findings in non-WEIRD regions are
generally consistent with findings in the United States. That said, it is uncertain if other
findings, especially on short-term regret, temporal pattern of regret, responsibility, and
hypothetical scenario experiment findings, are generalizable to other non-WEIRD regions.
Gilovich and Medvec (1994) replication and extension: Supplementary 40
Future directions for broader action-inaction literature
somethin
may explain the mixed or contradictory findings. On a macro level of action-inaction effects,
eir studies
(Feldman et al., 2021a), and consider this factor as a plausible explanation for discrepancies
in findings when interpreting or discussing the findings. Also, when explaining to
participants, researchers
d elicit different or similar results.
regret intensity, and personal experiences may account for the discrepancy in our studies.
Future studies should carefully consider this factor when designing studies and explaining
their results. Both types of studies have their merits and limitations. Hypothetical scenario
studies ensure the consequences of both action and inaction are the same, but may not capture
actual feelings of regret of participants themselves. Real-life experience studies may be
higher in ecological validity, but do not control for the possible differences in consequences
between action and inaction.
Gilovich and Medvec (1994) replication and extension: Supplementary 41
Future directions for replications evaluation
We adopted LeBel et al. (2019) to assess the replicability of the findings. LeBel et al. (2019)
criteria are based on correlation coefficient, a bi-directional effect size. Since our replications
adopted binary choice questions, we adopted Cramer V, a one-directional effect size. This
means that the effect size never falls below 0. Future meta-science work can consider setting
criteria for other effect sizes to facilitate comparison of effect sizes between original studies
and replication studies.
A related issue we encountered is that Criteria B (for comparing null findings in the original
with the replication) of LeBel et al. (2019) does not take into account that it is possible for the
replication to have no signal and the confidence intervals of the replication to not cover the
original effect size, which is the case for Study 4 long-term regret part, meaning that LeBel et
-
category for replication-original results comparison. Perhaps future studies can consider this,
and future meta-science work can consider more possibilities for differences between original
and replication.
Medvec (1994), may consist of several studies with different methods. We successfully
replicated Study 1, Study 3, and Study 4, but failed to replicate most of the results in Study 5.
This can be considered as a mostly successful replication, depending on methods.
Gilovich and Medvec (1994) replication and extension: Supplementary 42
Data Collection Information
We conducted the studies on Amazon Mechanical Turk with mostly American participants.
We imposed the following settings in recruiting our participants to ensure high data quality
(adapted from Feldman et al., 2021b template):
1. The fixed participation reward was USD$0.8. We determined this amount by
multiplying the expected completion time (in minutes) with the minimum federal
wage in the U.S. ($0.125 per minute).
2. We set the expected completion time at 4 to 6 minutes in advance.
3. We collected data from 21/04/2021 to 04/05/2021.
4. The maximum time we allowed each worker to complete all studies was 30 minutes.
5. We ensured
6. We ensured number of HITs approved to be between 5,000 and 100,000.
7. We blocked Suspicious Geocode Locations and Universal Exclude List Workers.
8. We blocked duplicate IP addresses and duplicate geolocations.
9. We enabled HyperBatch so that all eligible workers were able to participate in our
HIT immediately after we launched the survey.
10.
Gilovich and Medvec (1994) replication and extension: Supplementary 43
Study recruitment
Gilovich and Medvec (1994) replication and extension: Supplementary 44
Writing task expectation alignment in consent
Gilovich and Medvec (1994) replication and extension: Supplementary 45
References
Chen, J., Chiu, C. Y., Roese, N. J., Tam, K. P., & Lau, I. Y. M. (2006). Culture and
counterfactuals: On the importance of life domains. Journal of Cross-Cultural
Psychology, 37(1), 75-84. https://doi.org/10.1177/0022022105282296
Constandt, B., Thibaut, E., De Bosscher, V., Scheerder, J., Ricour, M., & Willem, A. (2020).
Exercising in times of lockdown: an analysis of the impact of COVID-19 on levels
and patterns of exercise among adults in Belgium. International journal of
environmental research and public health, 17(11), 4144.
https://doi.org/10.3390/ijerph17114144
Faul, F., Erdfelder, E., Lang, A. G., & Buchner, A. (2007). G* Power 3: A flexible statistical
power analysis program for the social, behavioral, and biomedical sciences. Behavior
Research Methods, 39(2), 175-191. https://doi.org/10.3758/BF03193146
Feldman, G., Chandrashekar, S. P., Yeung, S. K., Xiao, Q., Fillon, A., & Henne, P. (2021a).
What is action, what is inaction? Clarifying action-inaction in judgment and decision
making and recommendations for term use and typology. [Manuscript in preparation].
Feldman, G., Xiao, Q., & Yeung, S. K. (2021b). HKU Registered Report template:
Supplementary. [Work in Preparation]
Gilovich, T., & Medvec, V. H. (1994). The temporal pattern to the experience of regret.
Journal of Personality and Social Psychology, 67(3), 357.
https://doi.org/10.1037/0022-3514.67.3.357
Gilovich, T., Wang, R. F., Regan, D., & Nishina, S. (2003). Regrets of action and inaction
across cultures. Journal of Cross-Cultural Psychology, 34(1), 61-71.
https://doi.org/10.1177/0022022102239155
Gilovich and Medvec (1994) replication and extension: Supplementary 46
Henrich, J., Heine, S. J., & Norenzayan, A. (2010). The weirdest people in the world?
Behavioral and Brain Sciences, 33(2-3), 61-83.
https://doi.org/10.1017/S0140525X0999152X
LeBel, E. P., McCarthy, R. J., Earp, B. D., Elson, M., & Vanpaemel, W. (2018). A unified
framework to quantify the credibility of scientific findings. Advances in Methods and
Practices in Psychological Science, 1, 389-402.
https://doi.org/10.1177/2515245918787489
LeBel, E. P., Vanpaemel, W., Cheung, I., & Campbell, L. (2019). A brief guide to evaluate
replications. Meta-Psychology, 3, 1-9. https://doi.org/10.15626/MP.2018.843
R package version 2.3.26
Signorell, A., Aho, K., Alfons, A., Anderegg, N., Aragon, T., & Arppe, A. (2020).
DescTools: Tools for descriptive statistics. R package version 0.99.39, 18.
July 1). Preregistration Planning and Deviation Documentation (PPDD). Retrieved
from osf.io/ywrqe
