Content uploaded by Nicolas Jacquemet
Author content
All content in this area was uploaded by Nicolas Jacquemet on Jan 10, 2024
Content may be subject to copyright.
Content uploaded by Nicolas Jacquemet
Author content
All content in this area was uploaded by Nicolas Jacquemet on Jun 03, 2019
Content may be subject to copyright.
Discrete Choice under Oaths∗
Nicolas Jacquemet†St´ephane Luchini‡Jason F. Shogren§Verity Watson¶
January 2024
Abstract
The Discrete Choice Experiment (DCE) remains by far the most popular mechanism used
to elicit preferences for non-market goods and services. Yet, the actual reliability of DCE
still is questionable. Using an induced value experimental design, we show that standard
benchmarks achieve no better than 56 to 60% of payoff-maximizing choices. In this paper,
we asses whether a truth-telling oath implemented before the DCE improves the reliability
of elicited preferences. Three key findings emerge. First, having respondents voluntarily sign
a a truth-telling oath achieves a 50% improvement in payoff-maximizing choices. According
to response times data, this is achieved thanks to increased cognitive effort. The induced-
value design allow us to directly measure attribute non-attendance. Using this measure, we
show this increased cognitive effort induces a significant decrease in attribute non-attendance
under oath. Second, based on structural utility models, we show the usual welfare measures
inferred from DCE responses are unbiased if and only if respondents were first exposed to the
truth-telling oath. Third, we show that the type of oath matters to improved DCE decision
making—the commitment to honesty via the truth-telling oath improves choices, whereas an
oath to task or an oath to duty did not improve choices.
Keywords: Discrete Choice Experiments, Stated Preferences, Oath, Truth-telling, External
validity, Welfare.
JEL codes: C9; H4; I3; Q5.
∗Revised version of CES WP n°2019.07. Financial support from both the Health Chair, a joint initiative by PSL,
Universit´e Paris-Dauphine, ENSAE and MGEN under the aegis of the Fondation du Risque (FDR), and from the
National Research Agency (program Investissements d’Avenir, ANR-10–LABX-93-0 and ANR-17-EURE-0001) are
gratefully acknowledged. J. F. Shogren thanks the Rasmuson Chair at the University of Alaska-Anchorage for the
support. The Health Economics Research Unit is funded by the Chief Scientist Office of the Scottish Government
Health and Social Care Directorates.
†Paris School of Economics and U. Paris 1 Panth´eon-Sorbonne. Centre d’Economie de la Sorbonne (CES),
Maison des Sciences Economiques, 106-112 boulevard de l’Hˆopital 75013 Paris. Nicolas.Jacquemet@univ-paris1.fr
‡Aix-Marseille U. (Aix-Marseille School of Economics) and CNRS, 5-9 Boulevard Maurice Bourdet, 13001 Mar-
seille, France. stephane.luchini@univ-amu.fr
§Department of Economics, U. of Wyoming, Laramie, WY 82071-3985, United States. JRamses@uwyo.edu
¶Health Economics Research Unit (HERU), U. of Aberdeen, Polwarth Building Foresterhill Aberdeen, AB25
2ZD. UK. v.watson@abdn.ac.uk
1
1 Introduction
Discrete choice experiments (DCE) have emerged as a go-to method to elicit preferences for new
goods, services, and public policy options. Examples abound. DCEs have been applied, for
example, to employment searches, health care and medical options, better environmental quality
policy, and new food products (see e.g., Mas and Pallais, 2017; Soekhai, de Bekker-Grob, Ellis,
and Vass, 2019; Adamowicz, Glenk, and Meyerhoff, 2014; Vossler, Doyon, and Rondeau, 2012;
Hensher, Rose, and Green, 2015; Schwarzinger, Watson, Arwidson, Alla, and Luchini, 2021).
In the UK, the HM Treasury Green Book recommends using DCEs to elicit values for cost-
benefit analysis (CBA) of new projects, such as infrastructure investments. Three main reasons
help explain why DCEs are so attractive: binary choices (between two goods differentiated by
alternative levels of common attributes and cost; see the discussion in Freeman III, Herriges, and
Kling, 2014) (i) are grounded in economic theory, although they are not necessarily incentive
compatible (see Manski and Lerman, 1977; McFadden, 2001; Carson and Groves, 2007); (ii) are
familiar and understandable to most people who make conscious and implicit multi-attribute
decisions everyday; and (iii) reveal the implicit marginal monetary values of attributes, which are
relatively straightforward to estimate empirically (see, e.g., Train, 2003; Adamowicz and Swait,
2011).
But as with any preference elicitation method, DCEs still face questions of accuracy (Bishop
and Boyle, 2019). Reliability is an issue because people misrepresent their preferences in DCEs
either by mistake or strategic behavior (see, for example, Meginnis, Burton, Chan, and Rigby,
2021). This paper explores whether a truth-telling oath can increase the reliability of the DCE
by inducing a real commitment to honesty. Such a truth-telling oath—similar to that taken by
witnesses before giving evidence in a court of law—has been shown to improve truthful revelation
of preferences in both induced and home-grown value auctions (Jacquemet, Joule, Luchini, and
Shogren, 2013) and home-grown value referenda (Jacquemet, Luchini, Shogren, and Zylbersztejn,
2017). The truth-telling oath works as a “preparatory act” that commits a person to truth-telling
in subsequent tasks—in this case stated preferences.1To the best of our knowledge, de-Magistris
and Pascucci (2014) is the only study extending theses results to a DCE format, and shows in
a home-grown DCE survey that a truth-telling oath can reduce the gap between hypothetical
choices and real economic choices. In the absence of an induced-value design, this study however
1The literature on the compliance without pressure paradigm in social psychology shows that, under certain
conditions, behavior is conditional on the sequence of past actions: decisions made at one point in time have strong
and long-lasting consequences on subsequent behavior. The literature has shown that volitional actions are much
more likely to produce the target behavior than mandatory preliminary behavior (see, e.g., Beauvois and Joule,
2002). In a seminal experiment in this field, an experimenter stands at the corner of a street and asks people who
pass-by for a token so as to take the bus. While the share of people who comply with this request is around 1/3,
it is twice as much when time has been asked before asking for a dime. Asking for time works as a preparatory act
which produces significant changes in subsequent behavior.
2
cannot contrast elicited choices with respondents’ true preferences.
We explore the preference revelation properties of a DCE in the context of an induced values
design that provides control over the true underlying preferences.2Our 2 ×2 benchmark design
combines hypothetical and paid choices with and without a calculator, which controls for both
hypothetical bias and computational mistakes. Payoff maximizing decisions in these benchmark
treatments account for at most 60% of observed decision. This implies that a DCE significantly
fails to accurately reveal preferences for a large share of the population even in a simple context
like an induced preferences design.3
The main treatment variable of interest is a truth-telling oath implemented before participants
are informed about the DCE preference elicitation task. Our results suggest that the truth-telling
oath drastically improves preference elicitation, reducing non-payoff maximizing choices by nearly
50%. We leverage the induced-value design of the experiment to develop a statistical classification
of subject’s decision rule that accounts for the number of attributes they ignore when making their
choice (a well-documented source of the failure of DCE to accurately reveal preferences, known
as ’Attribute Non-Attendance’; Hensher, Rose, and Greene, 2005). While this phenomenon is
widespread in all benchmarks DCE, the share of subjects who are best described by a decision
rule accounting for all attributes in the choice set raises to more than 70% under a truth-telling
oath.
Rather than predicting choices, the main challenge DCE try to address is to infer preferences,
and specifically willingness-to-pay, so as to support welfare analysis. We assess the performance of
DCE in the different treatments by comparing the estimated preference parameters to the induced
ones—and show the two are significantly different from one another except when respondents have
first been exposed to a truth-telling oath. Last, we consider additional robustness treatments
aimed at investigating why the oath fosters truthful reporting of preferences. To understand
better whether it is commitment to honesty specifically that leads to more reliability in the DCE,
or commitment itself or maybe the procedure implemented when DCE is combined with a truth-
telling oath, we explore two alternative oath frames (oath on task or on duty). We find that people
still use non-optimizing decision rules more frequently than under the truth-telling oath: the oath
itself is not what matters—it is rather the commitment to the truth that makes a significant
difference.
2For additional lab experiments that used an induced value approach to examine the reliability of DCEs without
an oath, see, e.g., Collins and Vossler (2009); Taylor, Morrison, and Boyle (2010); Carson, Groves, and List (2014);
Luchini and Watson (2014); Meginnis, Burton, Chan, and Rigby (2021).
3In contrast with homegrown values, an induced value design focuses on the elicitation of preferences that are
already clearly defined. Several pieces of evidence in the literature show that preference formation is itself affected
by the properties of the elicitation mechanism, and magnifies the failure to accurately reveal preferences (see, e.g.
Jacquemet, Joule, Luchini, and Shogren, 2011; Murphy, Stevens, and Yadav, 2010; Carlsson, 2010).
3
Table 1: Attributes’ induced values
Token Size Colour Shape Cost
Attributes Small Medium Large Red Yellow Blue Circle Triangle Square
£0.5 2.5 4.0 1.0 1.5 2.0 1.5 3.0 6.0 2.0 3.0 4.0
2 Design of the experiment
We rely on an experimental design that recreates the salient features of a stated choice study in an
experimental economics laboratory setting—with induced (i.e., exogenous and perfectly observed)
preferences. Given our focus, we define a set of benchmark treatments that allows us to capture
and control for hypothetical bias and mistakes.
2.1 Induced Values Discrete Choice Experiment
We follow the design of Luchini and Watson (2014, “wide hypothetical” treatment) and induce
preferences for a multi-attribute good called a “token”. Subjects may buy tokens during the
experiment at the announced cost, which they will then sell back to the experimenter at the
end of the experiment. Subject’s preferences for the tokens are induced by announcing that the
amount of money they will receive for a token depends on its attribute levels. A token is defined
by four attributes and each attribute can take three possible levels: colour (red, yellow, blue);
shape (circle, triangle, square); size (small, medium, large); and cost (low, medium and high).
Each level of each attribute is associated with a monetary value, as shown in Table 1. The sum
of the attribute levels determines the value of the token, i.e., the amount the monitor will pay to
buy it back from the subject. This replicates the linear additive utility typically assumed in the
estimation of preference parameters from DCE studies (Train, 2003).
Subjects are asked to complete nine choice tasks on a computer with one choice task per
screen. In each task, two tokens are presented and subjects have to choose either to buy one
of the two tokens (and subsequently sell it back to the experimenter), or to buy no token at
all. The tokens included in the choice tasks were chosen using a fractional factorial design (the
full list of choices is presented in the Appendix, Section A). The order in which choice tasks are
presented is randomized at the individual level: for each subject, before each screen is displayed,
we randomly draw without replacement which one of the remaining choice tasks will be displayed
on the upcoming screen. A table similar to Table 1 above is included in the written instructions
distributed to subjects and read aloud by the experimenter at the start of the experiment. Subjects
keep the written instructions with them during the entire duration of the experiment and can refer
to it at any time.4
4The written instructions are presented in the Appendix, Section B. Subjects who take part in our experiments
4
In this context, the payoff-maximizing choice amounts to buying the token with the highest
profit in each choice set, i.e., the token in which the difference between the value (sum of attribute
levels) and the cost is the greatest. The monetary value of attributes induces subject’s preferences
over the attributes in the choice sets—so that profits are the lab counterfactual of individual pref-
erences in the field. In contrast with a home-grown preferences DCE study, in which preferences
underlying elicited choices are respondent’s private information, this induced value design allows
us to assess whether subjects make the best choice for them, i.e., make choices that maximize
their profit (which is their experienced utility) by truthfully revealing the induced preference.
The main outcome variable from the experiment is the share of choices that coincides with this
payoff-maximizing choice—which measures the preference revelation performance of the DCE.
We additionally record (self-paced) response time—the time elapsed between the appearance
of the choice on the screen and the actual decision. Assuming that subjects who respond more
slowly engage in more cognitive effort, as shown by recent empirical evidence (Krajbich, Bartling,
Hare, and Fehr, 2015; Evans, Dillon, and Rand, 2015), this outcome provides a proxy of the
amount of cognitive effort devoted to the task.
2.2 Benchmark Treatments
The benchmark treatments result from a 2 ×2 combination of whether (i) discrete choices are
hypothetical or paid and (ii) subjects are provided with a calculator or not. All treatments
are implemented between subjects—the same person participates in only one treatment. Table 2
summarizes all experimental treatments carried out in this paper. We first focus on the preference
elicitation properties of the benchmark treatments, which define the main open challenge the oath
treatments (introduced in Sections 4 and 5 below) aim to address.
Treatment 1 (Hypothetical) mimics a typical DCE survey: subjects’ choices are hypotheti-
cal. Subjects are paid £12 for taking part in the experiment irrespective of the choices they make.
The experiment instructions use subjunctive language by asking individuals to “put yourself in
a situation where your account balance at the end of the experiment would depend on the choice
you made...” (Taylor, McKee, Laury, and Cummings, 2001). The data for this treatment comes
from Luchini and Watson (2014) “wide treatment”.
Several studies however suggest that the DCE tasks are too complicated for respondents,
and, as a consequence, respondents might not choose what is best for them. Choices are more
complicated when the multi-attribute goods included in the choice set are similar (Mazzotta and
Opaluch, 1995; Swait and Adamowicz, 2001a), when the goods are described by many attributes
are students at the University of Aberdeen, who are recruited using Exlab and ORSEE software (Greiner, 2015).
All subjects received a consent form, experiment instructions, and payment form before taking part in the experi-
ment. Before the experiment started, the subjects read and signed the consent form and this was collected by the
experimenter, then the experimenter read aloud the experiment instructions to the group and answered questions.
The experiment was programmed and conducted with z-Tree (Fischbacher, 2007).
5
Table 2: Summary of the experimental treatments
Design Hypothetical Calculator Oath
Benchmark treatments
Treatment 1.
(Hypothetical)
Each subject completes 9 choice tasks (1 choice be-
tween two tokens per screen), and chooses to (1) buy
one token to sell back, or (2) buy no token.
Yes No No
Treatment 2.
(Calculator)
Each subject has access to the Microsoft Windows™
calculator.
Yes Yes No
Treatment 3.
(Real)
We pay each subject based on his or her choices.
Payoffs are determined by randomly selecting 1
round out of 9.
No No No
Treatment 4.
(Calc&Real)Combines experiments 2 and 3.
No Yes No
Oaths treatments
Treatment 5.
(Oath-on-Truth)
“I, ..., the undersigned do solemnly swear that dur-
ing the whole experiment, I will tell the truth and
always provide honest answers”.
Yes No Yes
Treatment 6.
(Oath-on-Task)
“I, ..., the undersigned do solemnly swear that dur-
ing the entire experiment, I will faithfully and con-
scientiously fulfill the tasks that I am asked to com-
plete to the best of my skill and knowledge”.
Yes No Yes
Treatment 7.
(Oath-on-Duty)
“I, ..., the undersigned do solemnly swear that dur-
ing the whole experiment, I will faithfully and con-
scientiously fulfil my duties to the best of my skill
and knowledge”.
Yes No Yes
(Swait and Adamowicz, 2001a; DeShazo and Fermo, 2002), when the choice set include many
alternatives (DeShazo and Fermo, 2002), or when individuals are asked to answer many choice
tasks (DeShazo and Fermo, 2002). Although our induced value DCE tasks may seem to involve
basic mathematics (addition and subtraction) not all subjects may be able to complete this task
efficiently. Treatment 2 (Calculator) aims to account for this issue, by providing subjects with
a computerized calculator to help them make the calculations. This treatment is identical to
Hypothetical, but with a button added to the screen for each choice set. By clicking on this
button subjects can access the Microsoft windows™calculator. Subjects’ use of the calculator is
recorded throughout the experiment.
In Treatments 1 and 2, choices are hypothetical and do not affect how much subjects earn in
the experiment. Critics of stated preference methods, and survey methods in general, question
6
peoples’ motivation to choose the best for them when answers are hypothetical (see, e.g., Olof
and Henrik, 2008). Subjects’ intrinsic motivation alone may not be enough to engage them in
making the necessary cognitive effort to solve the task (see, e.g., Camerer and Hogarth, 1999, for
a discussion of this issue in economic experiments). In Treatment 3 (Real), we replicate Hypo-
thetical, except that subjects are paid based on the choices that they make in the experiment.
To that end, each subject receives a £2 show-up fee plus an extra £4 in an account, which changes
depending on earnings. We designed this payment such that the expected earnings is about £12,
which is the same in expectation as the flat payment in Treatments 1 and 2. To avoid a binding
budget constraint, all tokens cost less than £4. To avoid income effects across choice tasks, the
monitor randomly selects at the end of the experiment 1 of the 9 choice tasks for each subject.
This choice is binding and the balance of the subject’s account is updated with the cost of the
token and the values of the attributes. The experimental instructions are identical to that of
Hypothetical, with the only exception that instructions no longer use subjunctive language.
The data for this treatment come from the “wide-monetary” treatment in Luchini and Watson
(2014).
Finally, in Treatment 4 (Calc&Real), monetary incentives are combined with a calculator.
Our hypothesis is that, when choices affect earnings, cognitive effort could be fostered and this may
encourage subjects to use the calculator more. This would, in turn, lead to a higher proportion
of correct choices.
3 The open challenge: subjects performance in benchmark DCEs
Figure 1 provides a summary of the performance of subjects in benchmark treatments (along
with the number of participants per treatment). The left-hand side displays the proportion of
correct choices (i.e., tokens that are payoff-maximizing among each pair) by treatment, while the
right-hand side figure reports this same variable by round. Overall, people make non-optimizing
choices in nearly 35-45% of all decisions, regardless of whether the DCE is hypothetical or paid,
with or without a calculator. Non-profit maximizing choices are substantial and consistent across
treatments.
Two main results emerge at the aggregate level in Hypothetical. First, only a little over
half of the choices are payoff-maximizing (56.3%). Second, there is no significant difference across
rounds except for the first round, for which the proportion of payoff-maximizing choices is a
little lower than 50% (42.1%, see the Appendix, Section D, for detailed round-by-round results).
In Figure 2, we compute, for each subject, the percentage of payoff-maximizing choices made
in the 9 choice sets and present its cumulative distribution function (CDF). Each dot in the
figure corresponds to a subject. No subject achieved 100% (or 9) of correct choices. The highest
percentage of correct choices observed in this treatment is 77.7% (7 choices out of 9). Most of the
subjects (44.7%) made only 6 correct choices.
7
Figure 1: Proportion of correct choices by treatment and round
0
25
50%
75%
100%
Round
123456789
Treatment
1: Hypothetical
2: Calculator
3: Real
4: Calc&Real
N
47
47
54
47
56.3%
61.6%
59.9%
64.9%
The proportion of correct choices in Hypothetical is relatively low. One potential expla-
nation might be that subjects just make mistake, because they don’t do the maths required to
identify the payoff maximizing option inside each choice set. Calculator provides an empirical
test of this assumption. To ascertain that subjects do use the calculator provided in this treatment
to make their choices, we record if the calculator is used in a choice task, and how many keyboard
entries are made when the calculator is activated. One keyboard entry corresponds to a number,
an operator, a decimal mark or a delete key. For instance, a subject who would calculate the
value of a small yellow square token would type “.5+1.5+6=” and this would be counted
as 9 keyboard entries. Figure 3 presents the proportion of subjects who activated the calculator
and the mean number of keyboard entries across rounds and by choice set. The calculator was
activated in 24.6% of the choice tasks. 50% of subjects never activated the calculator, 19.5%
activated it only once and 13% activated it in every round; the remaining subjects are equally
distributed in between. Figure 3.a shows that the activation of the calculator is relatively stable
across rounds. There is no clear round effect, with only a small increase in activation in rounds
2 and 3 (28.2% and 32.6%, against 21.7% in round 1). The number of keyboard entries sharply
increases from round 1 to 2 (Figure 3.b), but remains stable afterwards.
As shown in Figure 1, this rather intensive use of the calculator only has a small effect on the
percentage of correct choices compared to Hypothetical (61.6% vs. 56.3%), but the change
is not statistically significant (p=.298, bootstrap proportion test).5At the subject level, we
observe no improvement in the percentage of correct choices a subject makes. In Figure 2.b the
5This two-sided bootstrap test of proportions consists of bootstrapping subjects rather than choice, which allows
to control for within-subject correlation across choices.
8
Figure 2: Cumulative distribution of payoff-maximizing choices by subject
(a) Baseline
0
25%
50%
75%
100%
0 25 50 75 100
Hypothetical
(b) Calculator
0 25 50 75 100
Hypothetical
Calculator
(c) Monetary incentives
0
25%
50%
75%
100%
Payoff maximizing choices (%)
0 25 50 75 100
Hypothetical
Real
(d) Incentives and calculator
Payoff maximizing choices (%)
0 25 50 75 100
Hypothetical
Calc&Real
CDF of the percentage of correct choices by subjects in Calculator is slightly to the right of
the one in Hypothetical, but the difference is not statistically significant (p=.192, bootstrap
Kolmogorov-Smirnov (KS) test).6Last, we examine whether subjects who use the calculator
6We use a bootstrap version of the Kolmogorov-Smirnov test, which allows for ties and small sample sizes (see
Abadie, 2002; Sekhon, 2011).
9
Figure 3: Use of the calculator across rounds in Calculator
(a) Proportion of subjects
who use the calculator
0
25%
50%
123456789
Round
(b) Mean number of keyboard
entries when calculator is used
0
25
50
123456789
Round
make better choices. At the choice level, the pairwise correlation between the activation of the
calculator and the choice being payoff-maximizing is very small (equal to .048). The correlation
with the number of entries is a little larger, but still small (.097). At the subject level, pairwise
correlation between the number of times the calculator is used and the total number of payoff-
maximizing choices is positive (.196) but not significantly different from zero (p=.193, Pearson’s
product moment correlation coefficient test). The same is true for the correlation between the
total number of entries and the number of correct choices by subject (with a correlation coefficient
equal to .207, p=.167; Pearson’s product moment correlation coefficient test). When choices are
hypothetical, providing a device to help subjects identifying the payoff maximizing option does
not improve choices in this induced values setting.
In Real, subjects are paid based on their decisions, to assess whether such behavior is due to
the lack of monetary incentives disciplining elicited choices. As shown in Figure 1, 59.9% of choices
are correct when money is at stake, which is similar to what has been observed in Hypothetical
(56.3%; p=.607, bootstrap proportion test). As in other treatments, no round effects are present
in the data. Figure 2.c provides a comparison of the CDF of the percentage of correct choices
by subjects between these two treatments. The comparison confirms aggregate findings at the
subject level: although the CDF in Real is slightly to the right of the one in Hypothetical
at the bottom of the distribution, the two are otherwise identical and statistically no different
(p=.480, KS bootstrap test). It does not mean, however, that subjects do not respond to
monetary incentives. Response times data suggests that subjects do pay more attention to their
decisions. The median total response time is 197s(econds) in Real, as compared to 157s in
Hypothetical (p=.050; median difference bootstrap test).7
7The Appendix, Section E, reports the CDF of total response times in each treatment. Monetary incentives
mainly reduce the number of subjects with very short response times. As a result, the CDF of response times in
Real first order dominates the one observed in Hypothetical (p=.025; KS bootstrap test).
10
Figure 4: Use of the calculator across rounds in Calc&Real
(a) Proportion of subjects
who use the calculator
0
25%
50%
123456789
Round
(b) Mean number of keyboard
entries when calculator is used
0
25
50
123456789
Round
The last benchmark treatment, Calc&Real, combines the two devices studied separately in
Real and Calculator: subjects are paid based on their choices and can use a calculator to help
them make choices. Figure 4.b reports the use of the calculator in this treatment across rounds.
Being paid based on choices does not lead to a more intensive us of the calculator as compared
to Calculator. The calculator was activated in 24.2% of the choice tasks in Calc&Real
compared to 24.6% of the choice tasks in Calculator. Again, nearly 50% of subjects (48.7%)
never use the calculator and 12.8% use it in every choice set (13% in Calculator). Among those
subjects who actually use the calculator, we do not find any difference in the number of keyboard
entries across the two treatments: the mean number of keyboard entries is 33.9 in Calc&Real
and 33.6 in Calculator.
The overall percentage of payoff-maximizing choices in Calc&Real, presented in Figure 1,
shows that being paid in combination with the calculator leads to a small, but significant, increase
in the percentage of correct choices: 64.9%, compared to 56.3% in Hypothetical (p=.037,
bootstrap proportion test). Again, no round effect seem to be present in the data. Figure 2.d shows
that the CDF of the percentage of correct choices by subject in this treatment is shifted to the
right compared to the one in Hypothetical: while the upper part of the CDF is similar in both
treatments, there are fewer subjects with a low number of payoff-maximizing choices (subjects
who often fail to make the correct choice). The CDF in Calc&Real first order dominates the
one in Hypothetical (p= 0.015, KS bootstrap test). The pairwise correlation between payoff-
maximizing decisions and both the activation of the calculator (equal to .165 and not significantly
different from 0 with p=.318, Pearson’s product moment correlation coefficient test) and the
number of entries in the calculator (.125 and not significantly different from 0 with p=.448,
Pearson’s product moment correlation coefficient test) are however small. These results suggest
that the effect of the calculator and of monetary incentives add up—they both improve the
proportion of payoff-maximizing choices. Even when these two devices are combined, however,
11
a large proportion of dominated choices remain. Although subjects do not face any uncertainty
regarding their preferences towards the two options thanks to the induced values design, the
preferences revealed by a DCE are inconsistent with true preferences in 35% of elicited choices.
Result 1 DCE choices with induced values are not profit maximizing for about 35-44% of the
choices, regardless of whether these choices are hypothetical or real, with or without a calculator.
The standard DCE method of hypothetical choices do as well as the typically proposed solutions,
e.g., real monetary payments or tools to reduce computation costs (in our case, a calculator).
4 Discrete Choices under a truth-telling oath
The proportion of payoff-maximizing decisions in the benchmark treatments are striking low.
Monetary incentives combined with the help of a calculator increase the proportion of payoff-
maximizing decisions, but only by a small amount. In Oath-on-Truth (Treatment 5 in Table 2)
we implement a non-monetary commitment device—a truth-telling oath—before Hypothetical.
The truth-telling oath procedure follows that of Jacquemet, Joule, Luchini, and Shogren (2013).
A monitor presents each subject with the oath at a private desk upon entry into the lab, after
completing the consent form. The form is entitled “Solemn oath” and contains an unique sentence
with a single prescription that reads “I, ..., the undersigned do solemnly swear that during the
whole experiment, I will tell the truth and always provide honest answers”.8Subjects are
told that signing the form is voluntary and that neither their participation in the experiment nor
their earnings depend on signing.9
The results, presented in Figure 5, are unambiguous. The truth-telling oath significantly
increases the percentage of correct choices by a large amount as compared to Hypothetical:
78.3% of elicited choices are payoff maximizing, compared to 56.3% (p<.001, bootstrap propor-
tion test). This corresponds to a 50% increase in the overall preference revelation performance
of the DCE mechanism. This increase is observed for all decision rounds: the truth-telling oath
induces an upward shift in each and every round, above the 75% mark. Figure 5.b presents the
cumulative distribution of the percentage of correct choices at the subject level. Under a truth-
telling oath, 40.9% of subjects make payoff-maximizing choices in all 9 choice sets (whereas only
one subject did so over all four benchmark treatments) and 54.5% of subjects make at most one
dominated choice. As a result, the CDF of correct choices in Oath-on-Truth significantly first
order dominates the CDF in Hypothetical (p<.001, bootstrap KS test).
8The oath procedure along with the form are presented in the Appendix, Section C.
9Since all 44 subjects who participated to this treatments except one did agree to voluntarily sign the oath, our
statistical analysis includes the choices of the subject who did not sign, i.e., we adopt an intention to treat strategy
to avoid selection effects.
12
Figure 5: Cumulative distribution of payoff-maximizing choices by subject
(a) Payoff-maximizing choices by round
0
25%
50%
75%
100%
Round
123456789
(b) CDF of Payoff-maximizing choices
Payoff maximizing choices (%)
0 25 50 75 100
Hypothetical
Oath-on-Truth
Result 2 The truth-telling oath significantly improves standard DCE choices—incorrect choices
are cut by half as compared to the baseline DCE and fall to around 20%.
Response time data indicate that this improvement in preference revelation is associated with
subjects paying more attention to their choices. The median total response time (spent to answer
all 9 choices), in Oath-on-Truth (208s) sharply increases by almost one minute as compared to
Hypothetical (157s; p=.008, median difference bootstrap test), to an extent that is comparable
to Real (197s; p=.400, median difference bootstrap test).10 In the next section, we explore how
the oath affects subjects’ reasoning in comparison to other treatments.
4.1 The influence of the oath on reasoning: An analysis of Attribute-Non
Attendance
A key requirement for truthful revelation of preferences in DCE is that respondents take into
account all attributes of the alternatives they face when making decisions. The possibility that
subjects rather use decision rules aimed at simplifying the decision problem by ignoring some
attributes, or focusing on a subset of them, is thus an important shortcoming of DCE tasks.11
10A comparison of the distribution of response times between the three treatments is provided in the Appendix,
Section E. The CDF of response time in Oath-on-Truth first order dominates that of Hypothetical (p=.004,
bootstrap KS test). Response times data from the Calculator- treatments cannot be compared to the others, as
the use of the calculator mechanically increases response times.
11In contrast with homegrown studies, subjects cannot ignore attributes just because they don’t value them in
an induced value design.
13
Table 3: Attribute non-attendance rule best fitting the sequence of individual decisions
Hypothetical Calculator Real Calc&Real Oath-on-Truth
(1) (2) (3) (4) (5)
AN-A191.5% 91.3% 77.8% 94.9% 27.2%
AN-A∗
183.0% 89.1% 74.1% 89.7% 15.9%
AN-A217.0% 10.9% 25.6% 10.3% 84.1%
AN-A38.5% 2.2% 12.9% 7.7% 72.7%
Note. For each treatment in column (the second line refers to the summary of treatments in Table 2), each cell reports the
proportion of subjects for whom the decision rule that best fits the sequence of observed choices relies on the cost and (i)
only one attribute (AN-A1), or exclusively so (AN-A∗
1); (ii) two attributes (AN-A2); or (iii) all three attributes, resulting in
payoff-maximizing choices (AN-A3). Since decision rules are not always mutually exclusive, some subjects might be assigned
to more than one of them based on our fit criterion.
Early empirical studies focused on extreme versions of such Attribute Non-Attendance (AN-A)
like for instance lexicographic preferences leading respondents to attend only one attribute and
to choose an alternative based on the ‘best’ level of that attribute across the choice task (Ryan
and Bate, 2001; Scott, 2002; Sælensminde, 2006; Swait and Adamowicz, 2001b). More recent
research has defined AN-A more broadly by assuming that respondents could possibly attend to,
or ignore, each of the kattributes in a DCE. Respondents in a DCE might accordingly display any
of 2kpossible AN-A patterns (Hensher, Rose, and Greene, 2005). In home-grown value studies,
AN-A patterns are analyzed either by directly asking respondents to self-report if they attend
to an attribute when making their choices (Puckett and Hensher, 2009; Scarpa, Zanoli, Bruschi,
and Naspetti, 2013), or thanks to two-steps econometric models in which preferences are inferred
conditional on the set of characteristics included in the decision process (Hole, 2011).
Thanks to our induced value design, we do not need to rely on self-reported patterns or
econometric models and statistical assumptions to infer AN-A from choices. For each subject, we
can directly compute the number of choices that are consistent with a decision rule that accounts
for only one attribute (being either the color, the size or the shape) and the cost (a rule of thumb
we label “AN-A1”); two attributes and the cost (“AN-A2”); or finally three attributes and the cost
(“AN-A3”, our payoff maximizing criterion). We then assign subjects to one of the three decision
rules based on the one that best fits the observed sequence of choices. Note that decision rules are
not always mutually exclusive in our design so that several decision rules can sometimes perform
equally well in (maximally) explaining observed choices for some subjects. We complement the
analysis with a more conservative AN-A1rule, denoted AN-A∗
1, which refers to subjects for whom
AN-A1is the only decision rule best fitting the observed sequence of choices.
The empirical distribution of subjects in each treatment is displayed in Table 3 (where the
proportions consequently do not sum to 100% within treatments). The table shows that a large
14
share of subject’s behavior is explained by a decision rule that focuses on only one attribute
in the non-oath treatments. Subjects who are (exclusively) assigned to such a rule are 83.0%
in Hypothetical, 89.1% in Calculator, 74.1% in Real and 89.7% in Calc&Real. The
distribution of subjects over the AN-A3rule confirms the analysis at the choice level: a very
small share of subjects (always lower than 15%) behave in a way that consistently accounts for
all attributes across all 9 choice sets in the four benchmark treatments. In sharp contrast with
these benchmark treatments, only 15.9% of subjects who were exposed to the Oath-on-Truth
make choices that are best predicted by a AN-A∗
1decision rule, while AN-A3is the best fitting
decision rule for more than 72% of them: the main behavioral effect of the truth-telling oath goes
through an improvement in attribute attendance.
The consequences of exposing respondents of a DCE preference elicitation task are three-fold:
first, the share of payoff maximizing decisions, which truthfully reveal respondents’ preferences,
sharply increases. Second, this is achieved thanks to subjects spending significantly more time
thinking about how to choose, which suggests the cognitive effort devoted to the preference report-
ing task increases. Third, respondents’ reasoning is far more likely to rely on the examination of
all attributes at stake. The open question we address next is whether such an increase in cognitive
effort, and the resulting improvement in attribute attendance, is a direct effect of the truth-telling
oath or an indirect effect of the commitment to the truth the oath aims to implement.
5 Does the truth-telling oath just foster cognitive effort?
This section aims to test whether it is the explicit commitment to honesty induced by the truth-
telling oath that improves payoff maximizing decisions, or if any alternative oath mechanism
would work just as well by simply fostering cognitive effort. To do so, we modify the content of
the oath and carry out two alternative oaths—an “oath on task” and an “oath on duty”.
5.1 Design of the alternative oaths
We carry out two alternative oaths treatments.12 In the first, Oath-on-Task (Treatment 6
in Table 2), we implement a modified oath that directly targets cognitive effort. The design
of this treatment replicates Oath-on-Truth, but with a modified oath form that explicitly
targets cognitive effort without referring to truth-telling behavior. The oath form now reads “I,
..., the undersigned do solemnly swear that during the entire experiment, I will faithfully and
conscientiously fulfill the tasks that I am asked to complete to the best of my skill
and knowledge”. Otherwise, the oath form and the oath procedure are identical to that of
Oath-on-Truth. All subjects agreed to voluntarily sign this oath.
12All oath procedures in Oath- treatments were carried out by the same person for all sub jects—who also ran
the benchmark treatments (1) to (4).
15
Figure 6: Proportion of correct choices by treatment and round, oaths treatments
0
25%
50%
75%
100%
Round
123456789
Treatment
1: Hypothetical
5: Oath-on-Truth
6: Oath-on-Task
7: Oath-on-Duty
N
47
44
37
37
56.3%
78.3%
63.7%
61.6%
The oath on task only targets cognitive effort while the truth-telling oath arguably contains
a moral connotation. Providing ethical standards to people has been shown to have a significant
effect on behavior (Mazar, Amir, and Ariely, 2008). In Oath-on-Duty (Treatment 7 in Table 2),
we implement a second modified oath that targets cognitive effort, but with a moral component.
To that end, we adapt a real world oath (that targets the effort to perform one’s assigned task)
with the moral reminders that one would encounter in the field if taking an oath before beginning
the duties of a public office. The oath form now reads “I, ..., the undersigned do solemnly swear
that during the whole experiment, I will faithfully and conscientiously fulfill my duties to
the best of my skill and knowledge”. All subjects except one agreed to voluntarily sign this
oath.13
5.2 Preference elicitation under oaths not targeting truth-telling
The results from the three oath treatments, along with the Hypothetical benchmark and the
number of participants per treatment, are presented in Figure 6. The Oath-on-Task has a small
but positive effect on choices as compared to Hypothetical, since 63.7% of observed choices
are now payoff-maximizing (as compared to 56.3%; p=.074, bootstrap proportion test). This
increase is, however, significantly lower than the improvement observed in Oath-on-Truth, both
on average (78.3% of choices are payoff maximizing under a truth-telling oath; p=.003, bootstrap
proportion test) and at all rounds of the experiment. These results are confirmed at the subject
level based on the comparison of the CDF of the share of payoff maximizing choices (Figure 7.a):
despite the slight improvement at the bottom of the distribution, the CDF in Oath-on-Task is
13As in Oath-on-Truth (see Footnote 9), statistical analysis is carried out without dropping this observation.
16
Figure 7: Cumulative distribution of payoff-maximizing choices by subject, oaths treatments
(a) Oath on task
0
25%
50%
75%
100%
Payoff maximizing choices (%)
0 25 50 75 100
Hypothetical
Oath-on-Truth
Oath-on-Task
(b) Oath on duty
Payoff maximizing choices (%)
0 25 50 75 100
Hypothetical
Oath-on-Truth
Oath-on-Duty
statistically similar to the one observed in Hypothetical (p=.536, bootstrap KS test), while
the CDF in Oath-on-Truth first-order dominates that observed in Oath-on-Task (p < .001,
bootstrap KS test).
Despite the small effect on behaviour, response times data confirm that subjects do take the
oath on task seriously. The median total response time is 237s in Oath-on-Task, which is sig-
nificantly higher than the one observed in Hypothetical (157s; p=.012, median difference
bootstrap test) and similar to the one observed in Oath-on-Truth (208s; p=.176, median
difference bootstrap test).14 This increase in response times shows that the oath on task fos-
ters cognitive effort; it has, however, little to no impact on choices as compared to benchmark
treatments. Table 4 presents the effect of the Oath-on-Task on attribute attendance, follow-
ing the methodology presented in Section 4.1, along with a reminder of the performance of the
Oath-on-Truth presented in Table 3. In line with the small effect of this treatment on payoff
maximizing decisions, a large share of subjects (more than 80%) are best characterized by using
a single attribute decision rule, while the share of subjects being best described by decisions rules
based on two or three attributes are low (and similar to the ones elicited in Hypothetical,
presented in Table 3). Contrary to the Oath-on-Truth, the Oath-on-Task has little effect on
14A comparison of the distribution of response times across the three Oath-on-Truth- treatments is provided in
the Appendix, Section F. The CDF of response time in Oath-on-Task first order dominates that of Hypothetical
(p < .001, KS bootstrap test), and is statistically similar as in Oath-on-Truth (p=.485, KS bootstrap test).
17
Table 4: Attribute non-attendance rule best fitting the sequence of individual decisions
Oath-on-Truth Oath-on-Task Oath-on-Duty
(5) (6) (7)
AN-A127.2% 89.2% 93.9%
AN-A∗
115.9% 81.1% 70.3%
AN-A284.1% 18.9% 29.7%
AN-A372.7% 8.1% 8.1%
Note. For each treatment in column (the second line refers to the summary of treatments in Table 2), each cell reports the
proportion of subjects for whom the decision rule that best fits the sequence of observed choices relies on the cost and (i)
only one attribute (AN-A1), or exclusively so (AN-A∗
1); (ii) two attributes (AN-A2); or (iii) all three attributes, resulting in
payoff-maximizing choices (AN-A3). Since decision rules are not always mutually exclusive, some subjects might be assigned
to more than one of them based on our fit criterion.
reasoning.
The oath on task might however appear too abstract and singular, whereas the truth-telling
oath is a real world institution with a moral content. The oath on duty aims to address this
issue. Figure 6 shows that this treatment do not manage either to improve choices to an extent
similar to what is achieved by Oath-on-Truth at the aggregate level. The overall share of
payoff-maximizing choices, equal to 61.6%, slightly increases as compared to the Hypothetical
(56.3%; p= 0.067, bootstrap proportion test). This improvement is however significantly lower
than the one observed in Oath-on-Truth (78.3%; p=.001, bootstrap proportion test). This
result is confirmed at the subject level in Figure 7.b: the CDF in Oath-on-Duty is similar to
that observed in Hypothetical (p=.224, KS bootstrap test) and is first-order dominated by
the CDF in Oath-on-Truth (p<.001, KS bootstrap test).
Again, response times data confirm that subjects nevertheless take the oath on duty as se-
riously as the other oath forms. The median total response time in Oath-on-Duty (213s) is
significantly longer than in Hypothetical (157s;p=.024, median difference bootstrap test) and
similar to the ones observed in both Oath-on-Truth (208s; p=.444, median difference boot-
strap test) and Oath-on-Task (237s; p=.827, median difference bootstrap test).15 The effect
of the Oath-on-Duty on reasoning, as measured by attribute non-attendance, are presented in
Table 4. The small improvement in payoff maximizing decisions induced by the Oath-on-Duty
translates into a lower share of subjects best characterized (solely) by a single attribute decision
rule. The share is however still at a high level (70.3%), which is much higher than the one elicited
by Oath-on-Truth (15.9%). The share of higher level decision rules remains low, and very
similar to the ones observed in Oath-on-Task.
15The CDF of total response time (presented in the Appendix, Section F) in Oath-on-Duty first order dominates
the CDF in Hypothetical (p=.009, bootstrap KS test) and is similar to the ones observed in both Oath-on-Task
(p=.315, bootstrap KS test) and Oath-on-Truth (p=.966, bootstrap KS test).
18
In sum, response time suggests that subjects do engage into more cognitive effort, to a level
comparable to those observed in Oath-on-Truth (and in Real), under the two alternative
oaths directly targeting cognitive effort. This, however, does not translate into significantly higher
proportions of payoff-maximizing choices and improved reasoning, in sharp contrast with the oath
on truth. This suggests that honesty, rather than cognitive effort, is a key aspect of the failure to
reveal preferences in DCE tasks.
Result 3 The notion of an oath in-and-of-itself is not enough. Neither the oath on duty nor the
oath on task significantly improve standard DCE choices.
We now turn to the implications of our results on the estimations of welfare estimates inferred
from observed choices.
6 Policy implications of the truth-telling oath: welfare estimates
Both the departure from payoff-maximization and the attribute non-attendance documented
above result in choices that do not match the induced preferences in the experiment. The oath
drastically improves demand revelation, leading to choices that are more consistent with induced
preferences. But the aim of DCE is to infer preferences from choices—rather than to predict
choices themselves—to obtain welfare estimates for public decision making. This section provides
an assessment of the effect of non payoff-maximizing choices on welfare estimates deduced from
the benchmark DCEs, and of the resulting improvement obtained thanks to the truth-telling oath.
To that end, we rely on the random utility framework used in standard DCE welfare analysis.
The induced value design implies that the econometrician knows the valuation of each attribute
a∈ {Colour; Size; Shape}in each token k,va
k, which equals the induced value. In this framework,
only the parameters of the utility derived by subject ifrom choosing token kremains unknown:
Uik =α+X
a
βava
k−γCostk+εik,∀i, k (1)
The intercept, α, measures the attractiveness of taking one of the tokens over the status quo
(whose utility is classically normalized to 0 for identification purposes, i.e., parameters associated
with not choosing a token are set to zero), irrespective of tokens’ attributes. Under the common
assumption that the noise, εik, is i.i.d. Gumbell, the probability of choosing token kover token
l,Pi[k≻l] = P[Uik > Uil] maps observed choices to utility parameters through a standard
multinomial Logit.
The estimates of the corresponding utility functions in each treatment are presented in Ta-
ble 5. Consistent with the AN-A analysis in the previous section, the intercept, that captures the
willingness to take one of the tokens irrespective of their characteristics, is significant in all treat-
ments but the Oath-on-Truth. In addition, the size attribute does not matter in all treatments
19
Table 5: Estimated utility parameters in each treatment
Hypothetical Calculator Real Calc&Real Oath-on-Truth
(1) (2) (3) (4) (5)
α2.0611 2.4191 2.1553 2.7946 0.0127
(0.0000) (0.0000) (0.0000) (0.0000) (0.9784)
βColour 0.3606 0.4421 0.2934 0.3619 0.5169
(0.0153) (0.0038) (0.0339) (0.0286) (0.0063)
βSize 0.0322 0.0707 0.1168 0.0927 0.3399
(0.4743) (0.1249) (0.0049) (0.0640) (0.0000)
βShape 0.0741 0.1130 0.0673 0.1257 0.3859
(0.0252) (0.0009) (0.0282) (0.0007) (0.0000)
γ-0.4269 -0.5317 -0.4079 -0.5602 -0.5796
(0.0000) (0.0000) (0.0000) (0.0000) (0.0000)
Note. Multinomial Logit Model Estimates of the utility model in (1). p-values appear in parenthesis below each coefficient,
which appears in bold when significant at the 5% level.
but Real and Oath-on-Truth. Finally, the weight of attributes, as compared to the preference
parameter associated to the cost, are much larger under Oath-on-Truth than in the benchmark
treatments.
Since the scale of preference parameters is arbitrary, welfare analysis relies on the Marginal
Rates of Substitution (MRS) that can be easily defined from preferences as MRSa/a′=−βa/βa′;
while MRS against the cost attribute provide measures of Willingness To Pay (WTP) for the
corresponding attribute: WTPa= MRSa/Cost =βa/γ. These quantities allow us to perform a
structural test of the effect of the Oath-on-Truth on welfare estimates. Because the values of
the attributes are all expressed in monetary terms, the MRS between attributes should all equal
−1 and the WTP for attributes should all equal 1 whenever choices are demand revealing. The
hypothesis that all restrictions jointly hold defines a specification test on the utility function that
rationalizes observed choices. The null hypothesis is a random utility model defined over the net
monetary value of token kup to a scale parameter δ:
(H0:Uik =δ(Pava
k−Costk) +εik
H1:Uik =α+Paβava
k−γCostk+εik
(2)
Table 6 displays the estimation of the MRS and the WTP in each treatment resulting from the
preferences in Table 5, along with the p-value of the corresponding LR test (with 5−1 = 4 degrees
of freedom). The estimated values of the MRS (top part of the Table) typically exhibit very large
confidence intervals (in particular for the Colour/Size and Colour/Shape comparisons), with point
estimates far off the −1 benchmark for all treatments but Oath-on-Truth. These estimates get
closer to the −1 benchmark in the Oath-on-Truth treatment, and the confidence intervals
20
Table 6: Marginal rates of substitution in each treatment
Hypothetical Calculator Real Calc&Real Oath-on-Truth
Colour vs. Size -11.205 -6.255 -2.511 -3.904 -1.521
[-43.858;21.447] [-15.458;-2.948] [-5.501;0.478] [-9.409;1.602] [-2.709;-0.332]
Colour vs. Shape -4.863 -3.913 -4.359 -2.879 -1.340
[-10.457;0.731;] [-7.297;0.530] [-9.759;1.041] [-5.845;0.087] [-2.300;-0.379]
Size vs. Shape -0.434 -0.626 -1.736 -0.737 -0.881
[-1.745;0.877] [-1.557;0.306] [-3.849;0.378] [-1.681;0.206] [-1.217;-0.545]
WTP for Colour 0.845 0.831 0.719 0.646 0.892
[0.126;1.564] [0.249;1.414] [0.034;1.405] [0.061;1.231] [0.240;1.544]
WTP for Size 0.075 0.133 0.286 0.165 0.586
[-0.137;-.288] [-0.046;0.312] [0.052;0.521] [-0.021;0.352] [0.298;0.875]
WTP for Shape 0.174 0.212 0.165 0.224 0.666
[0.014;0.334] [0.079;0.346] [0.011;0.319] [0.085;0.364] [0.407;0.924]
H0(LR test) p < .001 p < .001 p < .001 p < .001 p=.115
Note. MRSs (top part) and WTPs (bottom part) are computed based on the estimated parameters from Table 5 Confidence
in intervals provided in brackets below the estimation are computed using the Delta method (see, e.g., Bliemer and Rose,
2013). The last line provides the p-value from an LR test of the hypotheses in (2).
only contain negative values—consistent with the expected substitution between attributes. The
Oath-on-Truth treatment also provides the most consistent WTP estimates (bottom part of
the Table), which are closer to the 1 benchmark than in any other treatment (although the
confidence intervals do not cover the benchmark value). Given these estimates, we strongly reject
the constrained utility model resulting from perfectly demand revealing choices in all benchmark
treatments (p<.001) — preferences inferred from choices in these treatments are inconsistent with
the true, induced preferences. We cannot reject this same null in the Oath-on-Truth treatment
(p=.115), in which observed choices are jointly consistent with the induced preferences.16
7 Conclusion
Can a truth-telling oath increase the reliability of the Discrete Choice Experiment (DCE), a
popular elicitation mechanism used to reveal preferences? Using an induced value experimental
design, our results suggest the answer is yes. Relative to the standard benchmark treatments
(hypothetical or paid choices, with or without a calculator), we find that the truth-telling oath
reduces non-payoff maximizing choices by nearly 50%. Similar to previous works on second-price
16We replicate this exercise on the data from Oath-on-Task and Oath-on-Duty in the Appendix, Sections G
(estimation of the utility parameters replicating Table 5 above) and H (estimation of the WTP parameters repli-
cating Table 6 above), and confirm the two treatments lead to very small improvements in welfare estimates as
compared to what Oath-on-Truth achieves.
21
auctions and referenda under oath, we find that an explicit and voluntary commitment to honesty
can improve preference elicitation. Further, we show that is not the oath itself that matters, but
rather the commitment to the truth. Using two alternative oath frames, an oath on task and
an oath on duty, we still observe people using non-optimizing decision rules significantly more
frequently than under the truth-telling oath.
Given that DCEs are a popular tool to elicit preferences for multi-attribute goods like health
care, improved environmental quality, and new consumer products, our results suggest that the
truth-telling oath should play a more significant role outside the lab and into the field. Impor-
tantly, the oath does not rely on any assumption about the specific reason leading respondents
to move away from truthful reporting of their own preferences. The oath rather arises as a be-
havioral shortcut: asking respondent to swear to a solemn oath beforehand creates real economic
commitment towards the preference revelation exercise. The large share of decisions that signif-
icantly departs from true preferences show that truth-telling should be explicitly addressed and
not implicitly assumed in field applications. Researchers could ask respondents to take a truth-
telling oath prior to being interviewed (see, e.g. Bhanot, 2017; Koessler, Torgler, Feld, and Frey,
2019; Lawton, Mourato, Fujiwara, and Bakhshi, 2020; Heese, P´erez-Cavazos, and Peter, 2023,
for examples of field implementations of the oath). Alternatively, future research might explore
the effectiveness of alternative forms of non-market commitment devices (e.g., weaker promises,
pledges, vows) in a DCE setting relative to the solemn truth-telling oath.
References
Abadie, A. (2002): “Bootstrap Tests for Distributional Treatment Effects in Instrumental Variable
Model,” Journal of the American Statistical Association, 97(457), 284–292.
Adamowicz, W., K. Glenk, and J. Meyerhoff (2014): “Choice Modelling Research in Environmen-
tal and Resource Economics,” in Handbook of Choice Modelling, ed. by A. D. S. Hess. Edward Elgar
Publishing, Cheltenham, UK.
Adamowicz, W., and J. Swait (2011): “Discrete Choice Theory and Modeling,” in Oxford Handbook
of the Economics of Food Consumption and Policy, ed. by J. Lusk, J. Roosen, and J. Shogren. Oxford
University Press, NY.
Beauvois, J.-L., and R.-V. Joule (2002): Petit Trait´e de Manipulation `a l’usage Des Honnˆetes Gens.
Presse Universitaires de Grenoble.
Bhanot, S. P. (2017): “Cheap Promises: Evidence from Loan Repayment Pledges in an Online Experi-
ment,” Journal of Economic Behavior & Organization, 140, 246–266.
Bishop, R. C., and K. J. Boyle (2019): “Reliability and Validity in Nonmarket Valuation,” Environ-
mental and Resource Economics, 72(2), 559–582.
22
Bliemer, M., and J. Rose (2013): “Confidence intervals of willingness-to-pay for random coefficient
logit models,” Transportation Research Part B: Method, 58, 199–214.
Camerer, C. F., and R. M. Hogarth (1999): “The Effects of Financial Incentives in Experiments: A
Review and Capital-Labor-Production Framework,” Journal of Risk and Uncertainty, 19(1), 7–42.
Carlsson, F. (2010): “Design of Stated Preference Surveys: Is There More to Learn from Behavioral
Economics?,” Environmental & Resource Economics, 46(2), 167–177.
Carson, R., T. Groves, and J. List (2014): “Consequentiality: A Theoretical and Experimental
Exploration of a Single Binary Choice,” Journal of the Association of Environmental and Resource
Economists, 1, 171–207.
Carson, R. T., and T. Groves (2007): “Incentive and Informational Properties of Preference Ques-
tions,” Environmental & Resource Economics, 37(1), 181–210.
Collins, P., and A. Vossler (2009): “Incentive Compatibility Tests of Choice Experiment Value Elic-
itation Questions,” Journal of Environmental Economics and Management, 58(2), 226–235.
de-Magistris, T., and S. Pascucci (2014): “Does ”Solemn Oath” Mitigate the Hypothetical Bias in
Choice Experiment? A Pilot Study,” Economics Letters, 123(2), 252–255.
DeShazo, J., and G. Fermo (2002): “Designing Choice Sets for Stated Preference Methods: The Effects
of Complexity on Choice Consistency,” Journal of Environmental Economics and Management, 44(1),
123–143.
Evans, A. M., K. D. Dillon, and D. G. Rand (2015): “Fast but Not Intuitive, Slow but Not Reflective:
Decision Conflict Drives Reaction Times in Social Dilemmas,” Journal of Experimental Psychology:
General, 144(5), 951–966.
Fischbacher, U. (2007): “Z-Tree: Zurich Toolbox for Ready-Made Economic Experiments,” Experimen-
tal Economics, 10(2), 171–178.
Freeman III, A. M., J. Herriges, and C. Kling (2014): The Measurement of Environmental and
Resource Values. RFF Press and Routledge, Washington, D.C. and NY.
Greiner, B. (2015): “Subject Pool Recruitment Procedures: Organizing Experiments with ORSEE,”
Journal of the Economic Science Association, 1(1), 114–125.
Heese, J., G. P´
erez-Cavazos, and C. D. Peter (2023): “When Executives Pledge Integrity: The
Effect of the Accountant’s Oath on Firms’ Financial Reporting,” The Accounting Review, pp. 1–28.
Hensher, D., J. Rose, and W. Green (2015): Applied Choice Analysis. Cambridge University Press,
Cambridge (UK).
Hensher, D. A., J. Rose, and W. H. Greene (2005): “The Implications on Willingness to Pay of
Respondents Ignoring Specific Attributes,” Transportation, 32(3), 203–222.
23
Hole, A. R. (2011): “A Discrete Choice Model with Endogenous Attribute Attendance,” Economics
Letters, 110(3), 203–205.
Jacquemet, N., R.-V. Joule, S. Luchini, and J. F. Shogren (2011): “Do People Always Pay Less
than They Say? Testbed Laboratory Experiments with IV and HG Values,” Journal of Public Economic
Theory, 13(5), 857–882.
(2013): “Preference Elicitation under Oath,” Journal of Environmental Economics and Manage-
ment, 65(1), 110–132.
Jacquemet, N., S. Luchini, J. Shogren, and A. Zylbersztejn (2017): “Coordination with Com-
munication under Oath,” Experimental Economics, 21(3), 627–649.
Koessler, A.-K., B. Torgler, L. P. Feld, and B. S. Frey (2019): “Commitment to Pay Taxes:
Results from Field and Laboratory Experiments,” European Economic Review, 115, 78–98.
Krajbich, I., B. Bartling, T. Hare, and E. Fehr (2015): “Rethinking Fast and Slow Based on a
Critique of Reaction-Time Reverse Inference,” Nature Communications, 6.
Lawton, R. N., S. Mourato, D. Fujiwara, and H. Bakhshi (2020): “Comparing the Effect of
Oath Commitments and Cheap Talk Entreaties in Contingent Valuation Surveys: A Randomised Field
Experiment,” Journal of Environmental Economics and Policy, 9(3), 338–354.
Luchini, S., and V. Watson (2014): “Are Choice Experiments Reliable? Evidence from the Lab,”
Economics Letters, 124(1), 9–13.
Manski, C., and S. Lerman (1977): “The Estimation of Choice Probabilities from Choice Based Sam-
ples,” Econometrica, 45(8), 1977–88.
Mas, A., and A. Pallais (2017): “Valuing Alternative Work Arrangements,” American Economic Re-
view, 107(12), 3722–59.
Mazar, N., O. Amir, and D. Ariely (2008): “The Dishonesty of Honest People: A Theory of Self-
Concept Maintenance,” Journal of Marketing Research, 45(6), 633–644.
Mazzotta, M. J., and J. J. Opaluch (1995): “Decision Making When Choices Are Complex: A Test
of Heiner’s Hypothesis,” Land Economics, 71(4), 500–515.
McFadden, D. (2001): “Economic Choices,” American Economic Review, 91(3), 351–378.
Meginnis, K., M. Burton, R. Chan, and D. Rigby (2021): “Strategic Bias in Discrete Choice Ex-
periments,” Journal of Environmental Economics and Management, 109, 102163.
Murphy, J., T. Stevens, and L. Yadav (2010): “A Comparison of Induced Value and Home-Grown
Value Experiments to Test for Hypothetical Bias in Contingent Valuation,” Environmental & Resource
Economics, 47(1), 111–123.
Olof, J.-S., and S. Henrik (2008): “Measuring Hypothetical Bias in Choice Experiments: The Impor-
tance of Cognitive Consistency,” B.E. Journal of Economic Analysis & Policy, 8(1).
24
Puckett, S. M., and D. A. Hensher (2009): “Revealing the Extent of Process Heterogeneity in Choice
Analysis: An Empirical Assessment,” Transportation Research Part A: Policy and Practice, 43(2), 117–
126.
Ryan, M., and A. Bate (2001): “Testing the Assumptions of Rationality, Continuity and Symmetry
When Applying Discrete Choice Experiments in Health Care,” Applied Economics Letters, 8(1), 59–63.
Sælensminde, K. (2006): “Causes and Consequences of Lexicographic Choices in Stated Choice Studies,”
Ecological Economics, 59(3), 331–340.
Scarpa, R., R. Zanoli, V. Bruschi, and S. Naspetti (2013): “Inferred and Stated Attribute Non-
attendance in Food Choice Experiments,” American Journal of Agricultural Economics, 95(1), 165–180.
Schwarzinger, M., V. Watson, P. Arwidson, F. Alla, and S. Luchini (2021): “COVID-19 Vaccine
Hesitancy in a Representative Working-Age Population in France: A Survey Experiment Based on
Vaccine Characteristics,” The Lancet Public Health, 6(4), e210–e221.
Scott, A. (2002): “Identifying and Analysing Dominant Preferences in Discrete Choice Experiments: An
Application in Health Care,” Journal of Economic Psychology, 23(3), 383–398.
Sekhon, J. (2011): “Multivariate and Propensity Score Matching Software with Automated Balance
Optimization,” Journal of Statistical Software, 42(7), 1–52.
Soekhai, V., E. de Bekker-Grob, A. Ellis, and C. Vass (2019): “Discrete Choice Experiments in
Health Economics: Past, Present and Future,” Pharmacoeconomics, 37(2), 201–226.
Swait, J., and W. Adamowicz (2001a): “Choice Environment, Market Complexity, and Consumer
Behavior: A Theoretical and Empirical Approach for Incorporating Decision Complexity into Models of
Consumer Choice,” Organizational Behavior and Human Decision Processes, 86(2), 141–167.
(2001b): “The Influence of Task Complexity on Consumer Choice: A Latent Class Model of
Decision Strategy Switching,” Journal of Consumer Research, 28(1), 135–148.
Taylor, L., M. Morrison, and K. Boyle (2010): “Exchange Rules and the Incentive Compatibility
of Choice Experiments,” Environmental & Resource Economics, 47(2), 197–220.
Taylor, L. O., M. McKee, S. K. Laury, and R. G. Cummings (2001): “Induced-Value Tests of the
Referendum Voting Mechanism,” Economics Letters, 71(1), 61–65.
Train, K. (2003): Discrete Choice Methods with Simulation. Cambridge University Press, Cambridge
(U.S.).
Vossler, C., M. Doyon, and D. Rondeau (2012): “Truth in Consequentiality: Theory and Field
Evidence on Discrete Choice Experiments,” American Economic Journal-Microeconomics, 4(4), 145–
171.
25
Appendix
A Set of choices implemented in the experiment
All participants are exposed to the same 9 choices between token A and token B described in the table
below. The order of choices in the sequence is randomized at the individual subject level.
Token A Token B
Choice Size Shape Colour Cost Net Size Shape Colour Cost Net Best
value value token
LSB4MCR2 Large Square Blue 4 8.0 Medium Circle Red 2 3.0 A
LTR2MTY3 Large Triangle Red 2 6.0 Medium Triangle Yellow 3 4.0 A
LTY3MSB4 Large Triangle Yellow 3 5.5 Medium Square Blue 4 6.5 B
MTR4LSY2 Medium Triangle Red 4 2.5 Large Square Yellow 2 9.5 B
SCB3STR4 Small Circle Blue 3 3.0 Small Triangle Red 4 0.5 A
SCY4LTB2 Small Circle Yellow 4 -0.5 Large Triangle Blue 2 7.0 B
SSR3LCY4 Small Square Red 3 4.5 Large Circle Yellow 4 3.0 A
SSY2SCB3 Small Square Yellow 2 8.0 Small Circle Blue 3 1.0 A
STB2LSR3 Small Triangle Blue 2 3.5 Large Square Red 3 8.0 B
B Written experimental instructions
We provide below the instructions used in the Real treatment. Instructions specific to Hypothetical
appear in ‘()’; and those specific to Calculator appear in ‘[]’.
You are about to participate in an experimental study of how people make choices.
At the beginning of the experiment (put yourself in a situation where) you are given an account with
a balance of £4, and you (could)/can use the money in this account to buy tokens that are offered for sale
in this experiment.
The experiment has 9 rounds. In each round, you will be offered several tokens. You will be asked if
you want to buy one of the tokens and if so which token. The tokens have different prices and values. You
can buy at most one token in each round. The value of a token depends on the token’s characteristics.
In this experiment the tokens have three characteristics: their size (small, medium, large); their colour;
(red, yellow, blue); and their shape (circle, triangle, square). The table below presents the value of each
characteristic.
Size Small £0.50
Medium £2.50
Large £4.00
Colour Red £1.00
Yellow £1.50
Blue £2.00
Shape Circle £1.50
Triangle £3.00
Square £6.00
26
The total value of each token is calculated by adding up the value of each characteristic. For example:
a small, yellow, triangle token will be exchanged for £0.50 + £1.50 + £3.00 = £5.00.
In each choice you will be shown on screen the characteristics and the price of each token. [In each
choice the windows calculator is available for you to use. To use the calculator, click with your mouse
on the calculator icon at the bottom right-hand side of the screen. You can make calculations (addition,
subtraction, multiplication and division) with this calculator. To do so either use your mouse or the number
keys at the right-hand side of the keyboard. There is no constraint on the number of times you can use the
calculator, and your choice to use the calculator has no effect on your monetary payoff from the experiment.]
At the end of the experiment, one round will be chosen at random by the computer and displayed to
you on screen. (Put yourself in a situation where) Your account balance at the end of the experiment will
depend on the choice you made in this randomly chosen round.
•If you bought a token, the price of the token is deducted from your account and the total value of
the token is added to your account.
•If you did not buy a token, your account balance is unchanged at £4.
At the end of the experiment, you will be shown on screen the total value of the token you chose in
this round, the cost of the token you chose in this round, and your account balance.
Please enter your account balance, into the payment sheet you are provided with and add this to your
£2 participation fee.
C Oath procedure used in Oath-on-Truth
Subjects wait in front of the laboratory room. The experimenter’s assistant distributes consent forms and
pens to participants.
The experimenter announces: “You will enter the room one by one. Please wait until I call you to come
in. Once in the room, I will take the signed consent form from you and you will draw the name of your
computer. You will then enter the room and wait until everyone has completed this process.”
Inside the room, one participant enters and the experimenter says: “Hi, please give me the consent form;
thank you. Now please draw a paper with your computer name.”
Then, while giving the oath form in Figure A to the subject: “OK. We would also like you to sign this
document, but please notice that signing it is not mandatory and will not affect either your participation
in the experiment or your experimental earnings from the experiment. Please read this form carefully and
decide whether you want to sign it or not.”
Whatever the decision: get the oath form back and keep in mind the computer name drawn by the subject
in case he/she decided not to sign the oath (in this case, record the computer name once the participant
has left the room). “Thank you, you can now enter the next room and settle in front of the computer you
have just drawn. We will start in a few minutes.”
Hide the folder of signed/refused oaths so that they are not visible. Have the next subject enter the room
and repeat the process.
Throughout this process, another experimenter waits in the laboratory to help participants find their way
and tells them that they are not allowed to talk before the start of the experiment.
27
Figure A: Oath form given to subjects in Oath-on-Truth
Health Economics Research Unit
Polwarth Building
Aberdeen AB25 2ZD
Scotland
United Kingdom
Tel: +44 (0) 1224 553733
Fax: +44 (0) 1224 550926
Website: www.abdn.ac.uk/heru
Solemn Oath
Title of Study: An experimental study of choice experiments.
I, the undersigned _____________________________ do solemnly swear that, during the whole
experiment, I will:
Tell the truth and always provide honest answers
Signature of Participant__________________________________________ Date ___________
28
D Payoff-maximizing decisions by round and treatments
Round
Treatment 1 2 3 4 5 6 7 8 9
Hypothetical 42.5% 51.1% 57.4% 70.2% 61.7% 46.8% 55.3% 61.7% 61.7%
Calculator 60.9% 65.2% 58.7% 67.4% 60.8% 56.5% 56.5% 67.4% 60.8%
Real 53.7% 55.5% 68.5% 50.0% 61.1% 64.8% 59.3% 70.4% 55.5%
Calc&Real 74.4% 64.1% 56.4% 66.7% 69.2% 56.4% 69.2% 61.5% 66.7%
Oath-on-Truth 72.7% 77.3% 84.0% 75.0% 77.3% 79.5% 77.3% 77.3% 81.8%
Oath-on-Task 59.5% 67.6% 64.9% 48.6% 64.9% 64.9% 67.6% 67.6% 67.6%
Oath-on-Duty 35.1% 59.5% 56.8% 70.3% 73.0% 75.7% 59.5% 73.0% 51.4%
E Response times: benchmarks vs Oath-on-Truth
In the Figure below, we compute for each subject the time taken to answer all 9 choice sets (total response
time) and plot the CDF of total response time in Hypothetical and Real in panel (a); and in Oath-
on-Truth in panel (b). Note that we dropped one subject in Hypothetical with a response time of
765s to make the Figure easier to read. The KS bootstrap test presented in the text is carried out without
dropping this subject.
(a) Benchmark treatments (b) Truth-telling oath
Total response time
0.0
.25
.50
.75
1.0
0s 250s 500s
Hypothetical
Real
Total response time
0.0
.25
.50
.75
1.0
0s 250s 500s
Hypothetical
Real
Oath-on-Truth
29
F Response times: Oaths treatments
In the Figure below, we compute for each subject the time taken to answer all 9 choice sets (total response
time) and plot the CDF of total response time in Hypothetical and Oath-on-Truth along with:
Oath-on-Task in panel (a) and Oath-on-Duty in panel (b).
(a) Oath on task (b) Oath on duty
Total response time
0.0
.25
.50
.75
1.0
0s 250s 500s
Hypothetical
Oath-on-Truth
Oath-on-Task
Total response time
0.0
.25
.50
.75
1.0
0s 250s 500s
Hypothetical
Oath-on-Truth
Oath-on-Duty
G Estimated utility parameters for the Oath-on-Task and Oath-
on-Duty
Oath-on-Task Oath-on-Duty
(6) (7)
α2.6787 1.7429
(0.0000) (0.0001)
βColour 0.5038 0.5889
(0.0029) (0.0008)
βSize 0.1048 0.1424
(0.0381) (0.0065)
βShape 0.1018 0.1121
(0.0063) (0.0035)
γ-0.4826 -0.5540
(0.0000) (0.0000)
Note. Multinomial Logit Model Estimates of the utility model in (1). p-values appear in parenthesis below each coefficient,
which appears in bold when significant at the 5% level.
30
H Marginal rates of substitution in Oath-on-Task and Oath-on-
Duty
Oath-on-Task Oath-on-Duty
Colour vs. Size -4.809 -4.137
[-10.452;0.834] [-7.978;-0.296]
Colour vs. Shape -4.949 -5.255
[-9.585;-0.312;] [-9.733;-0.777]
Size vs. Shape -1.029 -1.270
[-2.330;0.272] [-2.601;0.060]
WTP for Colour 1.044 1.063
[0.307;1.780] [0.405;1.721]
WTP for Size 0.217 0.257
[-0.010;0.444] [0.049;0.465]
WTP for Shape 0.211 0.202
[0.049;0.373] [0.059;0.346]
H0(LR test) p < .001 p < .001
Note. MRSs (top part) and WTPs (bottom part) are computed based on the estimated parameters from Appendix G
Confidence in intervals provided in brackets below the estimation are computed using the Delta method (see,e.g., Bliemer
and Rose, 2013). The last line provides the p-value from an LR test of the hypotheses in (2).
31
