ArticlePDF Available

Abstract and Figures

Eighteen empirical studies from fourteen different researchers provide evidence that prepaid monetary incentives have a strong positive impact on the response rate in mail surveys. One of these studies is described here and an attempt is made to generalize from all eighteen about the relationship between size of incentives and reduction in nonresponse. These generalizations should be of value for the design of mail survey studies.
No caption available
… 
Content may be subject to copyright.
Published in Public Opinion Quarterly, 39, 1975, 111-116.
Monetary Incentives in Mail Surveys
J. Scott Armstrong
The Wharton School, University of Pennsylvania
Abstract
Eighteen empirical studies from fourteen different researchers provide evidence that prepaid monetary
incentives have a strong positive impact on the response rate in mail surveys. One of these studies is described here
and an attempt is made to generalize from all eighteen about the relationship between size of incentives and
reduction in nonresponse. These generalizations should be of value for the design of mail survey studies.
Previous Research
One of the mail surveys, conducted in 1969, consisted of I00 people randomly picked from the Philadelphia
metropolitan-area telephone books.1 The questionnaire, which was rather long, included a description of a new form
of transportation called the Minicar Transit System and 61 questions, some of them open-ended. Fifty of the
potential respondents received a dollar “honorarium” with the questionnaire. The other fifty received no money.
Seventy percent of the former group and only 22 percent of the latter group responded. While the sample size is
small, the difference between the two sub-samples was statistically significant (p < .05).
Table 1 [next page] summarizes results of this study and the seventeen previous ones.2 The conclusions are
rather clear: (1) prepaid monetary incentives yield large increases in response rates, and (2) the larger the monetary
incentive, the greater the increase in the response rate.
The only exception to an otherwise perfect ranking is the five-cent incentive in Kephart and Bressler's
study.3 This reversal is quite likely due to sampling error. Thus, the rankings were in the expected direction on 24 of
the 25 possible comparisons: an increase in monetary incentives led to an increase in response rate 96 percent of the
time, a statistically significant result (sign test). Unfortunately, only six of the studies tested multiple levels of
incentives, but these studies also support the hypothesis that the greater the incentive, the greater the response rate.
It is important to note that all of the studies in Table I refer to prepaid monetary incentives. The promise of
a reward does not have as strong an effect. Heads and Thrift, in a study on a British population, demonstrated that
there was some increase in response rate as the size of the promised reward was increased.4 However, as shown in
the studies in Table 2, the effect of a promised incentive is small in comparison to that of a prepaid incentive.
It is also interesting to note that the payment of an incentive only on follow-up mailings has an effect, but
one not quite as strong as when money is sent in the first mailing. Huck and Gleason obtained a response rate of 94
1 Scott Armstrong and Terry S. Overton (1971). “Brief vs. comprehensive descriptions in measuring intentions to
purchase,” Journal of Marketing Research, 8, 114-117.
2 Paul L. Erdos (1970), Professional Mail Surveys. New York: McGraw-Hill, provides a partial review of the
literature.
3 William M. Kephart, and Marvin Bressler (1958), “Increasing the response to mail questionnaires,” Public Opinion
Quarterly, 22, 123-132. However, there were only three comparisons covering the range from $0.25 to $1.00, a
range that would seem to be of particular interest.
4 J. Heads, and H. J. Thrift (1966), “Notes on a study in postal response rates,” Commentary, 8, 257-262.
2
Table 1
Mail Response Rates for Various Prepaid Monetary Incentives;
Entries Represent Percentage of Sample That Responded
Incentive
Study None $.01 $.05 $.10 $.25 $1.00
Armstrong and Overtona
Blumberg et al.b
Erdosc
Erdosd
Erdose
Erdosf
Heckler and Bourgetteg
Huck and Gleasonh
Hancocki
Kephart and Bresslerj
Kimballl
Maloneym
Newmann
Newmano
Shuttleworthp
Watsonq
Wisemanr
Wotrubas
22
20
40
35
34
23
39
65
10
52
30
58
29
19
30
39
18
55
54k
54
57
41
40
47
63
58
56
40
92
47
70
86
47
37
52
48
41
70
30
54
71
53
61
a Armstrong and Overton, op. cit., n. 1.
b Herbert H. Blumberg, Carolyn Fuller, and A. Paul Hare (1974), “Response rates in postal surveys;” Public
Opinion Quarterly, 38, 113-123.
c Erdos, op. cit., n. 2, p. 97.
d Ibid., pp. 97-98.
e Ibid., p. 98.
f Ibid.. p. 99.
g James C. Hackler and Patricia Bourgette (1973), “Dollars, dissonance and survey returns,” Public Opinion
Quarterly, 37, 276-281.
h Huck and Gleason, op cit.
I John Hancock (1940), “An experimental study of four methods of measuring unit costs an obtaining attitude
toward the retail store,” Journal of Applied Psychology, 24, 213-230.
j Kephart and Bressler, op. cit, n. 3.
k This figure is the only exception to otherwise perfect rankings.
l Andrew E. Kimball (1961), “Increasing the rate of return in mail surveys,” Journal of Marketing, 25, 63-64.
m Paul W. Maloney (1954), “Comparability of personal altitude scale administration with mail administration
with and without incentive,” Journal of Applied Psychology, 38, 238-239. The no-incentive form of Maloney had
two follow-ups while the group that got a quarter reward had only one follow-up.
n Sheldon W. Newman (1962), “Differences between early and late respondents to a mailed survey,” Journal of
Advertising Research, 2,. 27-39. (2-page questionnaire)
o Ibid. (4-page questionnaire)
p Frank K. Shuttleworth (1931), “A study of questionnaire technique;” Journal of Educational Psychology, 22,
652-658.
q John I. Watson (1965), “Improving the response rate in mail research,” Journal of Advertising Research, 5,
48-50.
r Frederick Wiseman (1973), “Factor interaction effects in mail survey response rates,” Journal of Marketing
Research, 10, 330-333.
s Thomas R. Wolruba (1966), “Monetary inducements and mail questionnaire response,” Journal of Marketing
Research. 3, 398-400.
3
percent when a quarter was sent on the first mailing, 92 percent when the same amount was sent on first follow-up,
78 percent when it was sent on the second follow-up, and 71 percent when no money was sent.5
Table 2
Mail Response Rates for Prepaid vs. Promised Incentives;
Entries Represent Percentage of Sample That Responded
Type of Incentive
Promised Prepaid
Studya No
Incentive $.25 $.50 $1.00 $.25 $1.00
Blumberg
Hancock
Wotruba
18
10
18
18
20
20
47
41
30
a See footnotes in Table I for study sources,
Estimating the Relationship Between Prepaid Incentives and Response
What generalizations can be drawn about the relationship between prepaid monetary incentives and
nonresponse? Consideration has been given only to those studies in which the money was sent with the initial
mailing. In order to utilize the previous studies, the question was formulated as follows: “What percentage reduction
in nonresponse might be expected from various levels of prepaid monetary incentives?” By stating the problem in
terms of the percentage reduction in nonresponse, one is able to control for the fact that some studies are inherently
more interesting to the respondents than others and thus draw larger response rates. (The response rates varied from
10 to 65 percent when no monetary incentives were provided.) This statement of the problem eliminated from the
sample one study, the four-page one by Newman, since it provided no data for the case with no monetary incentive.6
One further change was required. Since the studies were drawn from different time periods, ranging from
1931 to 1973, it was necessary to convert to constant dollars. The 1973 dollar was used as the standard here.7 In
current dollars, the incentives varied from one cent to one dollar, as shown in Table 1. In 1973 dollars, the incentives
varied from 15 cents to $1.56. This provided substantial variation.
Figure 1 provides a summary of the data. Twenty-four observations were drawn from the seventeen studies
since some of the studies provided data on different level of incentives.8 A crude relationship was obtained by fitting
the data to a curve that was consistent with the finding noted above, that each increment of incentive leads to an
increase in response rate. In addition, the following a priori assumptions were made:
1. The curve should go through the origin to reflect, by definition, that no reduction in non-response is
obtained with no incentive.
2. The curve should approach an asymptote. Obviously, the asymptote cannot exceed 100 percent. An
asymptote of 100 was selected under the assumption that "everyone has his price." However, it
should be noted that this decision was subject to much uncertainty.
5 Schuyler W. Huck and Edwin M. Gleason (1974), “Using monetary inducements to increase response rates from
mail surveys,” Journal of Applied Psychology, 59, 222-225 (2 waves).
6 Newman, op. cit.. Table 1.
7 The Consumer Price Index was obtained from the 1973 Economic Report of the President, and this was converted
to a January I, 1973, base. The year prior to the publication dale was used as the year of the study unless information
was available on the year in which the study was actually done. Exceptions were the Erdos study done in 1952, the
other Erdos studies done in 1956, and the Armstrong and Overton study done in 1969, finally, the index from 1933
was used for the 1931 publication by Shuttleworth, op. cit., Table 1.
8 One rationale for trying to assess the relationship by using different studies was provided by Wiseman, whose
study suggested that the various techniques used to reduce nonresponse bias operate independently of one another;
Wiseman, F. (1973), "Factor interaction in mail survey response rates," Journal of Marketing Research, 10, 330-
333.
4
3. There should be diminishing marginal returns: each additional amount of money should have less of
an impact.
4. The relationship should be simple. For this reason, a relationship was selected that required the
estimation of only one parameter.
Figure 1
Percentage Reduction in Nonresponse vs. Prepaid Monetary Incentive
(1973 Dollars)
One way to capture the above assumptions is to fit the data to the following functional form:
bM
e
a
aY = (1)
where
Y is the reduction in non-response rate
a is the asymptote
M is the monetary incentive in dollars, and
b is the parameter to be estimated.
This functional form may be transformed to:
In (a) - In (a - Y) = bM (2)
Assuming an asymptote of 100-percent,
In (100) - In (100 - Y) = bM (3)
The b was then estimated via regression analysis using the 24 observations to be 0.6 [missing footnote]
Thus, the reduction in the percentage of nonresponse, R, can be predicted:
M
e
R6.0
100
100 = (4)
5
The curve represented by Equation 4 is shown in Figure I. The adjusted R2 between the predicted and
actual figures for Y was only 0.13. The poor fit is due largely to the studies by Huck et al. and by Maloney. Also, the
predictions obtained from the curve are lower than the actual reductions in non-response bias on 18 of the 24
observations. On the average, the curve underestimates by about eight percent. This conservative bias is especially
true for “small” monetary incentives (those less than fifty cents).
While prepaid monetary incentives led to a reduction in the nonresponse rate in all 24 cases, and by almost
one third on the average, the attempt to relate this reduction to the monetary incentive was rather unsatisfactory. The
use of equation 4, along with an 8 percent compensation for bias, led to only a modest improvement over a model
that assumed that an incentive of any size would reduce nonresponsc by the average of 32 percent. The mean
absolute percentage error (MAPS) was 14.9 for the former method and 11.4 for the latter. This provided an index of
predictive efficiency of about 23 percent.9
The evidence for the a priori assumptions behind equation 4 was not, of course, strong even though the data
were consistent with the hypothesis that each increase in incentives leads to a reduction in nonresponse.
Furthermore, Hackler and Bourgette speculated that very large incentives would lead to lower response rates.10 By
dropping this a priori assumption, it is possible to develop a simple rule of thumb that provides a better fit to the
data: “There is a I percent decrease in the nonresponse rate for each one-cent increase in the prepaid incentive up to
a maximum of 40 percent.” This rule of thumb reduced the MAPE for the sample from 11.4 (with the curve) to 9.4.
Compared to a prediction based solely on the sample mean, the index of predictive efficiency was 37 percent.
When will monetary incentives be most effective? In view of the wide and unexplained variations among
the studies, it seems worthwhile to control for the design of the study and the target population. Certainly, more data
of the type examined here would be useful. More useful, however, would be large-scale surveys, each examining (1)
a large number of different incentives (e.g., 20 levels), and (2) a wide range of incentives (e.g., one cent to twenty
dollars).
Another promising route would be to conduct in-depth studies on how these monetary incentives affect
people. This might help determine which aspects of the survey design or the target population are important. For
example, on the design question, it may be more effective to call the incentive an “honorarium” than a “payment.”
Summary
What have we learned after four decades of research on monetary incentives in mail surveys? The only type
of monetary incentive that has an impact on the nonresponse rate is the prepaid monetary incentive (moderate
support). The reductions in nonresponse are substantial. In this sample, there was a reduction of about one-third
(very strong support). Also, we have learned that it is very difficult to quantify the relationship between size of
incentive and reduction in nonresponse. A rule of thumb – that there is a 1 percent decrease in nonresponse rate for
each one cent increase in the prepaid incentive up to 40 percent – seems to be of some help (moderate support).
Finally, progress over the next four decades is likely to be minimal unless large-scale surveys and/or in-depth
studies are directed at this issue.
9 This index of predictive efficiency was calculated by dividing the improvement (3.5 in this case), by the total error
using the sample mean only (14.9).
10 Hackler and Bourrette, op. cit., Table 1.
... Research questionnaires mailed to general populations have typically produced low return rates (B. K. Armstrong et al., 1992;Heberlein & Baumgartner, 1978;Myerson, 1993) unless the surveys incorporated intensive follow-up or monetary incentives (J. S. Armstrong, 1975;Dillman, 1978Dillman, , 1991Dillman, Christenson, Carpenter, & Brooks, 1974;Fox, Crask, & Kim, 1988;James & Bolstein, 1990;Yammarino, Skinner, & Childers, 1991). Surveys of physicians have also frequently yielded problematic response rates (Berk, 1985;Deehan, Templeton, Taylor, Drummond, & Strang, 1997;Tambor et al., 1993;Vogelzang et al., 1997) and these rates recently have been declining. ...
... This section evaluates the literature on recruitment of students, the general public, and farmers for surveys and experiments. Studies going back decades have claimed that small monetary incentives increase survey response rates (Armstrong 1975;Duncan 1979;Brennan, Seymour, and Gendall 1993;Deserranno 2019). Limited research suggests farmers are more likely to respond when promised monetary incentives. ...
Article
To develop evidence‐based agricultural policies, researchers increasingly use insights from economic field experiments. These insights are often limited by the challenges of recruiting large and representative samples of farmers. To improve the effectiveness and cost efficiency of farmer recruitment, researchers should apply the same experimental methods to the recruitment process that they apply to their main research questions. Here we experimentally evaluate ten recruiting strategies in two large‐scale, high stakes experiments. We find that monetary incentives and reminders are effective, but costly. Costless strategies, such as prominently citing a well‐known institution as the sponsor, had positive but small, effects on recruitment.
... Although a huge body of empirical literature 3 supports this practice (see e.g. Armstrong 1975, Church 1993, Goyder 1994, Warriner et al. 1996, Simmons & Wilmot 2004, Laurie & Lynn 2008, Jäckle & Lynn 2008, Gajic et al. 2012, Pforr et al. 2015, Luiten 2019, just to mention a few), there are conflicting evidences in Yu & Cooper's (1983) and Singer et al.'s (1999) meta-analyses that question the hypothesis that unconditional incentives would perform better than conditional ones (which signifies an economic exchange approach) in general. In theory, there are some serious limitations of this practice, as it can be guessed from the serious limitations of the Social Exchange Theory trust concept in general. ...
Conference Paper
Full-text available
Trust plays a crucial role in survey participation, among other factors. Acting upon Eleanor Singer’s calling for more theory, our proposal for workshop discussion is the applicability of recent advances in trust theory to survey incentive usage. Current approaches are mainly based on Social Exchange Theory, and its obscure trust notion emerging out of reciprocal relationships. Trust is assumed to be learnt in the course of recurring cost-benefit calculations in social exchange situations. However, the cognitive approach of trust conceptualisation to which this concept belongs to has been since proven to describe a calculative, strategic, interest-driven, shrewd, and/or manipulative frame of mind – the direct opposite of trust. Designing incentive strategies on such bases may well result in (unintended) manipulation. Any perception of manipulative intent can lead to reactance, alienation, and the feeling of exploitation, inducing subjective isolation from the perceived manipulator. Even the subliminal ‘nudges’ of behavioural economic choice architectures are manipulative, for they are to reduce subjective relevance of objective alternatives, thus also harming the autonomy of prospective respondents. Instead of counting on ill-assumed learnt trust mechanisms, incentive design could be based on the concept of ‘built-in’ basic trust described by the non-cognitive approaches. Just like doubt, distrust is learnt rather than trust. It may keep innate human trusting in latency, therefore incentive usage should be shifted from ‘trust-triggering/provoking’ design to engender the irrelevance of distrust regarding the response situation. The situation remains non-intrusive, even trust-assuming if the institute demonstrates trust, thus empowers the respondent and facilitates her relaxing into trust. In contrast with attempting to (gently) force out reciprocity, respondent autonomy is left unharmed. A properly tailored leverage-saliency design may be adequate for this purpose, but the appropriate form of incentives is yet to be found. Our early-stage research focuses on the development and testing of this approach.
Article
Purpose The purpose of this paper is to examine the impact of inter-group conflicts on the trust toward the acquirer and then further on cooperation intention after acquisitions in cross-border mergers and acquisitions (M&As), in the lens of the social classification theory. Two types of conflicts (realistic conflicts and symbolic conflicts) and a mediating mechanism (trust toward the acquirer) exhibit different effects on cooperation intention. The research further examines two moderating coping strategies (localizing management and assigning trustworthy expatriate managers) and tests their effectiveness in promoting trust toward the acquirer and increasing cooperation intention in cross-border M&As. Design/methodology/approach The research hypotheses were empirically tested in the context of post-acquisition of Chinese companies' cross-border M&As. In total, 600 questionnaires were provided to the research participants of 37 acquired firms/units from advanced economies of 12 Chinese companies, and 209 valid questionnaires were collected (the response rate is 34.83%). Confirmatory factor analysis was conducted to verify data validity and hierarchical multiple regression analyses were employed to test relational and moderating effects. Findings This research demonstrates that both realistic and symbolic conflicts can reduce the intention to cooperate, but the latter does not have a significant influence. The results also uncover that whether employees from acquired firms trust in their acquirers mediates the relationship between realistic conflicts and cooperation intention. Moreover, management localization (a measurement of whether local/original managers are retained with a high degree of freedom and autonomy after M&As) and trustworthiness of expatriate managers (a measurement of whether the assigned expatriate managers is trustworthy) positively moderate the relation between realistic conflicts and trust toward the acquirer. In addition, trust toward the acquirer mediates the interaction of realistic conflicts and management localization on the cooperation intention of the acquirers' employees. Originality/value This study examines how inter-organizational conflicts influence trust toward the acquirer and then cooperation intentions in the context of Chinese companies' M&A behavior of targets from developed countries and gain supportive evidence, which enriches the literature on the management of international M&As. By introducing two management localization and trustworthiness of expatriate managers into the model, the research deepens our knowledge of how to build trust toward the acquirer in cross-border M&As.
Article
Ictalurid catfish have a long history of popularity in Iowa fisheries, but recent changes highlighted the need to investigate catfish angling interest and potential for future growth. Ictalurid catfish were superseded by other taxa for the first time in recent statewide angler surveys, putting their importance into question. Ongoing urbanization in Iowa was associated with reduced interest in angling, and specifically in catfish angling. We sought to determine demographic characteristics, preferences, and constraints of potential, lapsed, active, and non‐catfish anglers, and to predict where catfish angling had the greatest potential for growth. A survey of urban areas was conducted in 2018; questions included interest and past experience fishing in Iowa, target species, characteristics of an ideal fishing trip, interest in various programs, and constraints to fishing participation. A total of 693 responses were obtained, yielding a 27.7% response rate. Catfish anglers were predominantly male, middle‐aged, and white, and interest in catfish angling was best explained by gender, age, and rurality (i.e., urban/rural background). Interest level (i.e., potential, lapsed, active, or non‐catfish angler) was related to ideal trip factors, program preferences, and constraints, with active catfish anglers being significantly different in numerous aspects from lapsed, potential, and non‐catfish anglers. Based on description of an ideal fishing trip, some potential anglers were predicted to be similar to active catfish anglers, while others would be similar to lapsed anglers. Spatial prediction of interest based on Tapestry™ segment composition revealed patterns in both Channel Catfish Ictalurus punctatus and Flathead Catfish Pylodictis olivaris interest, which were repeated across cities. Our findings suggested numerous differences between active catfish anglers and those who may be recruited or re‐activated; these differences translate into dissimilar management actions and interventions. We concluded that catfish angling can grow in Iowa’s urban centers, but recruitment and retention strategies should be specific to both audience and place.
Article
A total of 586 members of an online market research panel completed the International Personality Item Pool (IPIP) Big Five personality inventory and were then offered one of three incentives (lower than normal, normal and higher than normal) to evaluate a new consumer product. Consistent with predictions based on equity theory, participants who were offered lower than normal incentives rated the product less favourably than those who were offered normal incentives. Contrary to predictions, participants offered higher than normal incentives did not rate the product more favourably than those who were offered the normal incentive. Respondents scoring high on Agreeableness, Conscientiousness, Openness and Extraversion rated the product more favourably than those who scored low on these dimensions. Implications for market research are discussed.
Article
Sample attrition increases the risk of statistical bias and hinders the ability to plausibly estimate causal effects when patterns of nonresponse are correlated with key variables of interest. Drawing on leverage-salience theory and other work in the behavioral psychology field, we empirically capture the impact of distinct motivational appeals on the survey response rates of elementary education teachers in a large urban school district in the northeastern United States. During spring 2017, teachers were randomized to receive one of six motivational appeals and were rerandomized to receive a different appeal each subsequent week, conditional on not having completed the survey. We observe the results on four different margins that range in their time intensity (open email and click, start, and complete survey). We find that extrinsic rewards improved teacher response across all four margins, and the social norm of reciprocity substantially improved teacher response along margins of lower time intensity. As researchers continue to conduct multitreatment arm studies and large-scale evaluations that can suffer from serious issues of sample attrition, this work highlights the contribution of message framing in survey response.
Article
Full-text available
In this study, we evaluate short-and long-term effects of three different prepaid incentives: a ballpoint-pen (gift worth approximately 2 Swiss francs), a voucher (cash card worth 10 Swiss francs) and cash (a 10-Swiss-francs' banknote) on young panellists' cooperation and response rate in three waves of a mature panel study with a sequential multi-mode design (web-based online survey, CATI, and PAPI). The survey experiment involved an alternative procedure to analyse the effect of different types of prepaid incentives, taking selective attrition into account as well as considering problems related to causal inference. The subjects were students, from randomly-selected school classes, who had finished their compulsory school in 2013. The findings are clear: cash provides the strongest direct, positive effect on the overall response rate and also on the latency until response after first contact. The other incentives did not work as efficiently as did cash. Additionally, cash is the most likely to minimise social selectivity in response. Finally, cash provides the potential to convert refusals in previous waves into cooperation.
Article
Full-text available
In forecasting demand for expensive consumer goods, direct questioning of potential consumers about their future purchasing plans has had considerable predictive success [1, 2, 4]. Any attempt to apply such "intention to purchase" methods to forecast demand for proposed products or services must determine some way to convey product information to the potential consumer [3]. Indeed, all the prospective consumer knows about the product or service is what he may infer from the information given to him by the researcher. This paper presents a study of the effect upon intention to purchase of this seemingly crucial element—the extent and type of description of the new service. How extensive must the description of the new service be in order to measure intention to purchase?
Article
This paper reports on an experiment involving the use of ten different postage stamp and money inducements to stimulate the return of questionnaires mailed to a sample of the nursing profession. A follow-up letter was found just as effective as more expensive and complicated arrangements.