Content uploaded by Daniel Wilson
Author content
All content in this area was uploaded by Daniel Wilson on Oct 12, 2017
Content may be subject to copyright.
Measuring and Increasing Adoption Rates of Cookstoves in a
Humanitarian Crisis
Daniel L. Wilson,*
,†
Jeremy Coyle,
†
Angeli Kirk,
†
Javier Rosa,
†
Omnia Abbas,
‡
Mohammed Idris Adam,
§
and Ashok J. Gadgil
†,∥
†
University of California, Berkeley, California 94720, United States
‡
Potential Energy, Berkeley, California 94704, United States
§
Al-Fashir University, Al-Fashir, North Darfur, Sudan
∥
Lawrence Berkeley National Laboratory, Berkeley, California 94720, United States
*
SSupporting Information
ABSTRACT: Traditional smoky cooking fires are one of today’s greatest environ-
mental threats to human life. These fires, used by 40% of the global population, cause
3.9 million annual premature deaths. “Clean cookstoves”have potential to improve this
situation; however, most cookstove programs do not employ objective measurement of
adoption to inform design, marketing, subsidies, finance, or dissemination practices.
Lack of data prevents insights and may contribute to consistently low adoption rates.
In this study, we used sensors and surveys to measure objective versus self-reported
adoption of freely-distributed cookstoves in an internally displaced persons camp in
Darfur, Sudan. Our data insights demonstrate how to effectively measure and promote
adoption, especially in a humanitarian crisis. With sensors, we measured that 71% of participants were cookstove “users”
compared to 95% of respondents reporting the improved cookstove was their “primary cookstove.”No line of survey
questioning, whether direct or indirect, predicted sensor-measured usage. For participants who rarely or never used their
cookstoves after initial dissemination (“non-users”), we found significant increases in adoption after a simple followup survey (p
= 0.001). The followup converted 83% of prior “non-users”to “users”with average daily adoption of 1.7 cooking hours over 2.2
meals. This increased adoption, which we posit resulted from cookstove familiarization and social conformity, was sustained for a
2-week observation period post intervention.
■INTRODUCTION
Since the beginning of the modern Darfur conflict in 2003,
violence has forced Darfuri families from their homes. Many
displaced families have emigrated from their homelands to large
Internally Displaced Persons (IDP) camps; current UN figures
estimate 2.5 million IDPs in Darfur.
1
In 2005, The University of
California, Berkeley and Lawrence Berkeley National Labo-
ratory began a joint effort to design a fuel efficient cookstove
for use in Darfuri IDP camps. The impetus for the Berkeley-
Darfur Stove (BDS) was to reduce the burden and danger IDP
women face when acquiring fuel in and around the camps. The
BDS’s improved thermal efficiency allows customers to cook
food using less fuel than a traditional three-stone fire (TSF)
which is locally known as a “ladaya.”
2
As of June 2016, 42,000
BDSs have been distributed to households in Darfur. In this
and other studies, the average household size measured was
appoximately 7.
3
With the 44,284 BDS units disseminated as of
Potential Energy’s 2015 Impact Report,
4
the BDS is estimated
to have reached 310 000 individuals.
Objective monitoring and evaluation is a major barrier to
quantifying the impacts of “clean cookstoves”like the BDS. For
decades, clean cookstoves have promised to reduce the global
burden of disease and drudgery attributable to traditional
cooking. Air pollution from traditional cooking methods is the
world’s largest environmental health risk factor−traditional
biomass-fueled cooking is linked with 3.9 million annual
premature deaths.
5,6
Clean cookstoves’promise is to displace
traditional smoky biomass fires (used by almost half the world’s
population) with cleaner combustion.
7
However, positive
outcomes of clean cookstove interventions are rarely significant
or sustained.
8
This is because clean cookstoves have not been
widely adopted and they have not sufficiently displaced
traditional cookstoves (e.g., “stove stacking”).
9
Stove Use Monitor (SUM) sensors have the potential to
objectively inform implementation agencies, policy makers, and
analysts about field performance and adoption of cook-
stoves.
9−14
Many stakeholder agencies are eager to understand
which cookstoves, training programs, and marketing methods
are effective. However, most cookstove adoption studies use
unreliable survey data subject to three problematic sources of
error: social desirability or courtesy bias where respondents
over-report “ideal”behaviors, recall bias when respondents
cannot accurately recall past behavior, and observation bias (the
Received: June 10, 2016
Revised: July 7, 2016
Accepted: July 7, 2016
Published: July 20, 2016
Article
pubs.acs.org/est
© 2016 American Chemical Society 8393 DOI: 10.1021/acs.est.6b02899
Environ. Sci. Technol. 2016, 50, 8393−8399
“Hawthorne Effect”).
9,15−19
By contrast, SUMs provide
accurate information about adoption and may eventually
improve cooks’health and economic outcomes by creating
tighter and more accurate feedback loop between stakeholders
in the dissemination space and the customers who use clean
cookstoves. If used effectively, SUMs data can inform
dissemination stakeholders about which cookstoves are well-
adopted, and this information could lead to better cookstoves
reaching more customers.
Even when self-reported data is unbiased on average,
measurement error in dependent variables (say, expenditures
or frequency of symptoms) reduces the precision of estimates,
giving reduced statistical power. Unbiased measurement error
in independent variables (for example, usage rates) leads to
“attenuation bias”that pushes impact estimates toward zero.
20
This effect only worsens when users systematically overstate
adoption. Unlike surveys, sensors are unbiased, discrete, and
long-lasting. Objective sensor data lead to improvements in our
understanding of cookstove adoption and enables insights
about cookstove designs, training, or marketing techniques that
may increase utilization.
In this study, we add critical information to the small pool of
studies objectively measuring cookstove adoption with
sensors.
9,12−14,21,22
To our knowledge, other groups have not
analyzed adoption of cookstove technologies in an ongoing
humanitarian crisis. Evaluation of technologies and techniques
that improve living conditions and environmental conditions in
humanitarian crises is important but inadequately addressed in
the scientific literature. Crises are important case studies
because they are regrettably common, are a frequent target
market for cookstove dissemination programs, and represent
unique social and economic contexts that make non-crises
insights potentially nontransferable. In prior studies, sensor data
has been compared with survey data.
10,13
In Thomas et al.
2013, sensors measured 40% lower use than surveys, but in
Ruiz-Mercado et al. 2013, surveys were found to generally be
reliable predictors of sensor-measured behavior. This contra-
diction points to the need for additional data about what
circumstances result in reliable survey data. We extend the
literature by testing whether multiple surveying techniques can
be combined to better predict sensor-measured behavior.
Additionally, we present a novel framework for categorizing
stove recipients as “users”or “non-users”and demonstrate the
value of this delineation in revealing data insights. Unlike prior
work, this study rigorously evaluates causes of sensor damage
and loss, and we perform sensitivity analysis for lost data. We
also present the first study to our knowledge quantifying the
impacts of social pressure on cookstove adoption (“courtesy
use”), and we show an example of how sensors revealed user-
generated innovations. Lastly, and most importantly, this study
adds meaningful data to the literature by assessing the impact of
enumeration activitiesor the anticipation thereofon cook-
stove adoption behavior, and we reveal the potential for low-
cost strategies to dramatically increase cookstove adoption.
■DESIGN AND METHODS
A detailed description of the design and methods of this
experiment is discussed in prior work
14
and in the Supporting
Information (SI). For clarity, a brief overview follows.
This work took place in the Al-Salam IDP Camp outside Al-
Fashir, North Darfur, Sudan. Sustainable Action Group (SAG),
a Sudanese nonprofit that assembles and distributes the BDS in
IDP camps, selected participants for this study in the usual
procedure for BDS dissemination: 180 participants were
selected to receive free BDSs by chief camp administrators
(“Omdas”) who selected study participants from a compre-
hensive list of inhabitants. Selected participants were limited to
five of the camp’s“Administrative Units”representing the
geographical and cultural emigration origin of IDPs. In
compliance with the University of California, Berkeley’s
Institutional Review Board approval (CPHS #2013-03-5132),
participants were told they would be taking part in a study of
the BDS “to improve future BDSs”and that a temperature
sensor would be attached to the BDS. However, information
about the tracking of cooking behaviors was withheld. All five
Units participated in a baseline and followup survey, and three
units participated in an additional second followup.
The five Units, each with 36 participants, received their BDSs
and took a baseline survey between July 29th and August
second of 2013. Serious security concerns precluded enumer-
ation stafffrom travel and extended stay in the camps, so the
baseline survey and all subsequent interactions took place at
midday in a women’s center in Al-Salam Camp. A phased
rollout of the followup survey was conducted four to 12 weeks,
depending on Unit, after the baseline. All women were supplied
with the date, time, and location of their followup. Women
were instructed to bring their cookstoves to the Women’s
Center for the followup survey. On the appointed day, a survey
was conducted and SUMs were removed from cookstoves. Data
were discretely downloaded from SUMs using a laptop
computer. For three of the five Units, SUMs were reattached
and a second followup was conducted 2 weeks after the first
followup. The second followup also required bringing the
cookstove to the Women’s Center so SUMs could be removed,
but no additional surveys were conducted. In all cases, women
brought their BDSs home from the followup survey(s) and
owned the stoves indefinitely thereafter. Additional details
about scheduling can be found in the SI.
Building on the methods of others,
9,12,13,21−23
we utilized
Maxim’s DS1922E-series iButtons as temperature data loggers.
SI Figure S1 shows the mounting location of SUMs which was
chosen by laboratory cooking experiments to maximize signal
(temperature) while still preventing overheating of the sensor.
Surveys in this study were conducted by a team of two
redundant enumerators. One enumerator administered the
survey using paper and pen while the other used Open Data Kit
(ODK) (a smartphone-based survey tool). Data from these two
methods were tested against one another for quality control
purposes. Data from SUMs and surveys were processed in
Version 3 of the open-source statistical computing software R.
A further discussion of the algorithm used to label cooking
events can be found in Wilson et al., 2015.
■RESULTS
SUMs Loss and Bias. The SUMs in this experiment were
vulnerable to failure, which may produce SUMs data bias.
During the study we discovered that some women innovated by
flipping the BDS upside down, filling the bottom with charcoal,
and preparing drinks or small meals. The BDS was not
designed for this mode of use. The SUM was mounted at the
bottom of the BDS to avoid overheating from wood fires (see
SI Figure S1), but the bottom of the BDS is precisely where
charcoal fires would make the BDS hottest. Of 170 participants
with SUMs-equipped cookstoves, 29 participants had thermally
damaged SUMs. In followup surveys, participants who reported
using charcoal as a primary cooking fuel (for food or drink)
Environmental Science & Technology Article
DOI: 10.1021/acs.est.6b02899
Environ. Sci. Technol. 2016, 50, 8393−8399
8394
were 2.8 times more likely (p= 0.01, Fisher test) to thermally
damage their SUMs as participants who did not report using
charcoal as a primary fuel. A summary of SUMs failures by
Administrative Unit is presented in SI Table S2.
Because data were unrecoverable from thermally damaged
SUMs, this study has possible bias. We posit that adopters of
the BDS are more likely to damage SUMs and therefore
adopters are underrepresented in surviving SUMs data. Put
another way, it would be difficult to thermally damage a SUM
mounted on a BDS that was never used for cooking. Therefore,
data loss from thermal damage represents a nonrandom
sampling bias in SUMs data and a probable downward bias
on sensor-measured adoption rates. Accordingly, SUMs-derived
data presented in this study are a conservative estimate of
adoption throughout the entire experimental population. Other
less prevalent causes of SUMs loss were observed, namely
misplaced and faulty sensors (before distribution and baseline
survey), one stolen stove, and a small number of women not
returning for followup surveys. These data are summarized in
detail in SI Table S2. However, we assume nonthermal damage
loss factors do not meaningfully bias data. Unless otherwise
noted, quantitative SUMs data presented throughout this study
are derived from surviving SUMs.
Defining “User”and “Non-User”Groups. To perform
more meaningful analyses, we classified participants into two
groups based on their pre-followup BDS adoption: “users”and
“non-users.”First, using SUMs data for each participant, we
computed the proportion of cookstove ownership days where
at least one cooking event was observed. The “pre-followup
period”analyzed was defined from 1 day after the participant’s
baseline survey until 2 days before the participant’s follow up
survey (to avoid effects near the followup discussed later). This
Figure 1. Left: a cumulative distribution function of the proportion of days used as measured by SUMs. Dashed vertical line indicates the 10% of use
days delination between “non-users”(left of dashed line) and “users”. Right: the probability densities of hours of daily cooking pre and post followup.
Figure 2. Histograms demonstrate cooking events and hours per day in the pre-followup period. Stacked histograms are coded into user and non-
user subgroups.
Table 1. A Summary of SUMs Results
a
results by unit Korma Al-Fashir Zaghawa Jebel Si Tawila total
pre 1st followup
users 17 24 16 16 14 87
non-users 9 3 5 10 8 35
hours cooked per day 1.00 (0.99) 1.64 (1.01) 0.98 (0.91) 0.72 (0.84) 1.14 (1.38) 1.10 (1.07)
users 1.52 (0.85) 1.84 (0.87) 1.27 (0.84) 1.15 (0.82) 1.79 (1.37) 1.54 (0.97)
non-users 0.03 (0.07) 0.00 (0.00) 0.03 (0.06) 0.04 (0.09) 0.02 (0.02) 0.03 (0.06)
cooking events per day 1.58 (1.56) 2.10 (1.41) 1.36 (1.32) 0.90 (1.06) 1.33 (1.55) 1.47 (1.43)
users 2.39 (1.33) 2.36 (1.26) 1.77 (1.25) 1.44 (1.04) 2.08 (1.49) 2.04 (1.30)
non-users 0.05 (0.09) 0.00 (0.00) 0.04 (0.07) 0.04 (0.10) 0.03 (0.03) 0.04 (0.07)
post 1st followup
users 16 14 11 41
non-users 5 8 6 19
hours cooked per day 2.01 (1.10) 1.50 (1.08) 1.67 (1.62) 1.73 (1.26)
users 1.91 (1.15) 1.51 (1.28) 1.77 (1.44) 1.74 (1.26)
non-users 2.32 (0.94) 1.48 (0.69) 1.50 (2.05) 1.70 (1.30)
cooking events per day 2.65 (1.57) 1.97 (1.32) 1.62 (1.83) 2.11 (1.60)
users 2.51 (1.68) 1.84 (1.45) 1.70 (1.74) 2.06 (1.62)
non-users 3.11 (1.15) 2.21 (1.12) 1.47 (2.15) 2.21 (1.57)
a
Note that “user”and “non-user”are defined in the context of the pre-followup period. Data format is mean (standard deviation).
Environmental Science & Technology Article
DOI: 10.1021/acs.est.6b02899
Environ. Sci. Technol. 2016, 50, 8393−8399
8395
variable, termed “proportion of days used”is plotted as a
cumulative distribution function in Figure 1. An arbitrary
delineation was drawn at 10% of days used, and participants
utilizing the BDS more than 10% BDS ownership days in this
period were classified as “users.”Although the quantitative
delineation between users and non-users was arbitrary, it is
useful to characterize study participants in terms of women who
generally tried out the BDS before their follow up survey
(users) and those who very rarely or never used the BDS before
the follow up survey (non-users). It is important to note that
we do not intend to conflate the term “user”to mean
“successful adopter.”We would not be satisfied, and we suspect
that neither would most customers, if a cookstove was only
useful 3 days out of a month. In that way, 10% is not a
meaningful cutofffor a program’s success in any health,
environmental, or economic benefit sense. However, as will be
shown, the “user”and “non-user”delineation is useful in
comparing when and how cookstoves are adopted. Unless
otherwise specified, “user”and “non-user”classification refers
to pre-followup behavior.
Pre-Followup Adoption Measured by SUMs and
Surveys. In the pre-followup period, 87 of 122 (71%)
participants with surviving SUMs were classified as users of
the BDS. Remembering that SUMs thermal damage presents a
downward bias on this study, if all thermally damaged SUMs
were presumed to belong to “users,”the study-wide adoption
rate would be 77% users.
The proportion of users varied widely and significantly by
Administrative Unit (p= 0.004; Fisher’s exact test) with Al-
Fashir Rural having the highest rate of users at 89% and Jebel Si
having the lowest at 62%. Study-wide, a typical user utilized her
BDS 1.54 (SD = 0.97) hours per day over 2.04 (SD = 1.30)
cooking events; these data are displayed as histograms in Figure
2. Including non-users, the study-wide average adoption rates
were 1.10 (SD = 1.07) hours and 1.47 (SD = 1.43) events of
daily cooking. A summary of SUMs-measured adoption is
shown in Table 1. Although ownership periods were relatively
short compared with other longitudinal studies,
9,12
users
showed no significant linear trend in average hours cooked
per day over the pre-followup ownership period (estimated
increase of 0.0002 h/day; p= 0.84).
Participants reported high rates of BDS adoption regardless
of their SUMs-measured usage: 95% of participants reported
using the BDS as their “primary stove.”As shown in Figure 3,
nearly all users reported using the BDS three times a day on a
“normal day.”This is compared to a SUMs-measured daily
cooking events of 2.04 events for users and 0.04 (median of 0)
daily events for non-users. 77% of users and 86% of non-users
over-reported cooking events. Of the 62 participants who used
the BDS an average of less than once per day, only five actually
reported doing so. Both users and non-users over-reported
cooking hours with 85% of users and 86% of non-users over-
reporting. All told, over-reporting represents 1.2 h and 1.3
events of daily cooking overestimation.
Because of the disagreement between SUMs and surveys, we
explored whether calculation errors or ambiguity in the wording
of the question “On a ’normal day,’how many times/hours do
you use your BDS?”could explain the observed differences.
Because of the question’s design, answering in a way that would
perfectly correlate with SUMs would require respondents to
time-average usage and then report their average daily use,
recalling over long periods and aggregating many data points, a
task that has been shown in the Development Economics
literature to be error prone.
20,24
Aside from recall and averaging
errors, we posited that women may over-report if they interpret
the question to only consider usage days rather than all
ownership days. If this different interpretation was used, surveys
would overstate usage relative to the sensors. To test these
possibilities, we checked whether SUMs data correlated with
another way we asked about usage: for an exhaustive list of
meal types, we asked how many times in a normal week a meal
was prepared with the BDS and how long each meal took to
cook on the BDS. The sum of weekly occurrences is termed
“Computed Events”and the sum of the product of meal
durations and weekly occurrences is termed “Computed
Hours.”These data are shown in a correlation table with
SUMs data in Table 2. Although these variables do not rely as
heavily on cooks’recall, calculations, or question interpretation
as the “normal day”questions, they correlate even more weakly
with SUMs data.
Impacts of Enumeration Activities on Adoption.
Figure 4 illustrates the effect of enumeration activities on
adoption for the three Units tracked after the first followup.
Beginning roughly 2 days before the scheduled followup, non-
users begin to adopt their BDSs. After followup, enumeration
activities had a statistically significant positive impact on the
non-user group, increasing hours of daily cooking by 1.6 h (p<
0.001, paired ttest) and no meaningful impact on the user
group, increasing hours of daily cooking by only 10 min.
Compared with the pre-followup period, non-users also
increased adoption of the BDS in terms of events, with an
average increase of 2.1 events per day. As a reminder, “users”
and “non-users”are classified solely by their BDS adoption
before the followup survey. A summary of results both pre- and
post-followup is shown in Table 1.
Figure 3. Daily events (left) and hours of BDS use (right) are shown as scatter plots of SUMs-measured versus self-reported data. The scatter plots
include the 1:1 line that data would fall on if users’self-reports perfectly agreed with the SUMs algorithm. To avoid overplotting, plot points are
“jittered.”.
Environmental Science & Technology Article
DOI: 10.1021/acs.est.6b02899
Environ. Sci. Technol. 2016, 50, 8393−8399
8396
■DISCUSSION
Although the BDSs were distributed free of charge, it was
relatively well-adopted among recipients, with 71% of surviving
SUMs classifying cooks as “users.”The (downward) bias
caused by thermally damaged SUMs resulting from inverted
BDS charcoal fires means that up to 77% of participants were
potential “users”in the pre-followup period. Among “non-
users,”pre-followup behavior is characterized by little or no
BDS utilization whatsoever. There is little evidence that non-
users try and then abandon the BDS. Rather, non-users seem to
neglect using the BDS altogether until just days before the
followup survey. It should be noted that all modes of clean
cookstove adoption cannot automatically be considered
exposure-reducing and beneficial to health; improper cookstove
operation, movement of a cleaner cookstove to a less-ventilated
environment, or increased overall fuel consumption via stove
stacking could all contribute to increased exposure relative to
baseline.
Other studies have found that socioeconomic and educa-
tional factors are the most important predictors of cookstove
adoption,
25
and this trend likely holds in our study as well.
Namely, Al-Fashir Rural Unit, which exhibited the highest rates
of adoption, is comprised of inhabitants who have emigrated to
Al-Salam IDP Camp from periurban settlements near North
Darfur’s capital. Although socioeconomic and educational
factors were not measured explicitly in surveys, Al-Fashir
Rural residents were likely to have been exposed to better
educational and work opportunities than residents from other
Units representing poorer rural parts of Darfur.
As found in other studies of health-related technolo-
gies,
10,11,26,27
study participants tended to over-report cook-
stove adoption. In this study, average self-reported used was
roughly twice SUMs data in terms of hours and events of
cooking per day. However, because there is little correlation
between SUMs and survey data, it is somewhat misleading to
think of reporting as a 2-fold overestimation. For example,
almost all women surveyed (83%) report using the BDS three
times per day−every meal. In other words, it is incorrect to
think that women inflate their adoption by 2-fold; instead,
nearly uniformly, women report using the BDS for all daily
meals, but across the sample, women actually use their BDSs
for about half of all meals. Though we attempted to adjust for
users’inability to average over large time periods by calculating
adoption from other questions, no manner of questioning or
reinterpretation of reporting periods correlated well with
SUMs-measured behavior, leading us to believe that many
participants intentionally misrepresent cookstove adoption
when surveyed.
We were surprised by the strong influence of survey
enumeration activities on the “non-user”group. As easily
seen in Figure 4, non-users exhibit a strong up-tick in adoption
starting 2 days before their scheduled followup survey. This
effect is seen in all Units that were observed until a second
followup, and this effect spanned a range of socio-economic
levels and cultural or geographic origin (as indicated by
Administrative Unit) and pre-followup ownership duration.
We posit that non-users may feel social pressure to use their
BDS before the followup survey; perhaps it would be
embarrassing to bring a shiny, clean, unused, donated stove
Table 2. Correlation Coefficients Tabulated for SUMs and
Survey-Based Measures of Adoption
Figure 4. Left: average hours of daily cooking per day in the 10 days preceding and tailing the followup. Right: post followup versus pre followup
SUMs data of hours of cooking per day. Points falling along the 1:1 line represent participants whose behavior was unchanged by the followup.
Participants above the 1:1 line use their stove more after the followup, and participants below the 1:1 line use their stove less after the followup.
Environmental Science & Technology Article
DOI: 10.1021/acs.est.6b02899
Environ. Sci. Technol. 2016, 50, 8393−8399
8397
back to the women’s center. Or, perhaps, non-users felt the
need to educate themselves about the BDS before returning for
the followup survey. Regardless of motivation, in the days
leading up to the followup survey, the non-user group strongly
exhibits what we refer to as “courtesy use.”This spike in usage
in reaction to or anticipation of direct observation (sometimes
called “reactivity”by other researchers) is consistent with other
studies in the developing and developed world.
11,28−31
However, what was not expected was the non-user group’s
sustained adoption in the 2 weeks after followup survey
enumeration. Because enumeration activities were not
instructive or coercive, one would expect women who did
not adopt the BDS would continue to neglect it after the follow
up survey. Quite to the contrary, upon returning home, non-
users’cookstove utilization became indistinguishable (mean
1.77 h, standard error 0.31) from their “user”peers (mean 1.74
h, standard error 0.22). In fact, using the same definition of
“user”as the pre-followup period (BDS use on ≥10% of
ownership days), 83% of previous non-users transitioned to
“post-followup users.”Additionally, population wide, 86% of
participants would be classified as users in the post-followup
period compared with 71% in the pre-followup period.
We propose three hypotheses that may explain the
phenomenon of non-user conversion. First, non-users rarely
or never used their BDSs until just before the followup, so this
group may never have realized the benefits of the BDS.
Perhaps, after finally trialling the BDS as a courtesy immediately
before the followup, non-users realized they enjoyed the BDS
and subsequently continued use after the followup. Second, it is
possible that peer pressure at the followup survey influenced
non-users; one can imagine non-users walking and talking with
their user peers to the Women’s Center. At the Women’s
Center, although surveys were private, women may have seen
peers with well-used cookstoves. Non-users may have over-
heard others others talking, truthfully or not, about enjoying
the BDS and thus felt more comfortable trying the BDS.
Finally, SUMs vs surveys suggest that many non-users told
mistruths during the followup survey about how often they use
the BDS. This deceit, although untrue, could have built non-
users’self-efficacy as a BDS adopter and helped non-users
visualize themselves as BDS adopters, inducing adoption after
returning home.
32
Alternatively, deceit may generate unpleasant
cognitive dissonance, which participants may resolve through
adoption consistent with their self-reports.
33,34
These theories
were not tested in this study, but suggest potential contributors
to post-followup adoption.
In this study, we presented cookstove adoption and reporting
behaviors for recipients of the BDS, joining a small set of
studies that have been able to combine traditional self-reported
data with objective sensor-based measurements. Our analysis is
unusual in part because of its context: despite the distribution
of tens of thousands of BDS cookstoves in IDP camps to date,
the challenging operating environment that makes aid needed
also leads to data scarcity and other monitoring challenges.
The relevance of our results is not limited to the IDP context.
Free distribution of improved cookstoves is commonplace, and
may be accompanied by a desire among recipients to report
behaviors preferred by distributors. However, the impacts of
free distributions on survey bias have yet to be rigorously
studied. Indeed, this study contributes evidence of the
discrepancies between self-reported and sensor-detected
usage, and affirms the need for sensor-based inquiry when
impact must be accurately measured. More exploration is need
of factors that could predict or facilitate higher accuracy of self-
report data even when compliance with a normative behavior is
low.
Additionally, we were able to use sensors to show an example
of how monitoring activities can themselves alter the behaviors
being monitored: usage spiked just before a followup visit, and
for many previously non-users, the uptick was sustained for the
following 2 weeks. This discovery may have useful implications
for optimal followup after distribution and provide insight
about strategies for inexpensive “light-touch”interventions to
increase cookstove adoption.
Our findings do not suggest that sensors can or should
replace self-reported data more generally. Sensors can be costly
to implement and can only cover a small fraction of the types of
data that may be relevant for analysis. Still, it is important to
consider which types of data are most likely to be reliable and
whether objective data sources may complement survey data in
a given context. In this IDP context, it is apparent that surveys
are extremely unreliable means of measuring technology
adoption.
In summary, these data highlight the weaknesses of self-
reported adoption data, the importance of objective sensor-
based validation of adoption and impacts, and reengaging
technology recipients, especially those with low uptake. This
work has shown that effective monitoring and evaluation can
have dramatic positive impacts on adoption, potentially leading
to better health and economic outcomes for customers.
■ASSOCIATED CONTENT
*
SSupporting Information
The Supporting Information is available free of charge on the
ACS Publications website at DOI: 10.1021/acs.est.6b02899.
A full acknowledgment of participants in this study as
well as an expanded Background section that discuses the
Berkeley-Darfur Stove’s context and supporting figures
and tables referenced in the main text. These supporting
figures and tables are intended to give the reader
additional insight into study timeline and design
(including phased rollout), stove use monitor (SUM)
failures, and the internally displaced persons context
(PDF)
■AUTHOR INFORMATION
Corresponding Author
*E-mail dlwilson@berkeley.edu.
Notes
The authors declare no competing financial interest.
■ACKNOWLEDGMENTS
We thank our enumeration and implementation team, lead by
coauthor Dr. Adam, for their excellent work on this research
study. Additional thanks to Potential Energy, Sustainable
Action Group, and the United States Agency for International
Development (USAID) who generously funded this work
under AID-OAA-A-13-00002.
■REFERENCES
(1) UNOCHA. Sudan: Humanitarian Bulletin Issue 19, 4 - 10 May;
2015.
(2) Preble, C. V.; Hadley, O. L.; Gadgil, A. J.; Kirchstetter, T. W.
Emissions and Climate-Relevant Optical Properties of Pollutants
Emitted from a Three-Stone Fire and the Berkeley-Darfur Stove
Environmental Science & Technology Article
DOI: 10.1021/acs.est.6b02899
Environ. Sci. Technol. 2016, 50, 8393−8399
8398
Tested under Laboratory Conditions. Environ. Sci. Technol. 2014,48,
6484−6491.
(3) Galitsky, C.; Gadgil, A. J.; Jacobs, M.; Lee, Y.-M. Fuel Efficient
Stoves for Darfur camps Internally Displaced Persons Report of Field trip
to North and South Darfur, Nov. 16 - Dec.17, 2005 - LBNL-59540,
2006.
(4) Potential Energy. Testing and Developing a Sustainable Model for
Cookstoves in Darfur and Ethiopia - Final Impact Report for USAID,
Grant No. AID-OAA-G-12-00012; 2015.
(5) Lim, S. S.; Vos, T.; Flaxman, A. D.; Danei, G.; Shibuya, K. A
Comparative Risk Assessment of Burden of Disease and Injury
Attributable to 67 Risk Factors and Risk Factor Clusters in 21 Regions,
1990−2010: a Systematic Analysis for the Global Burden of Disease
Study 2010Lancet2013
(6) Smith, K. R.; Bruce, N.; Balakrishnan, K.; Adair-Rohani, H.;
Balmes, J.; Chafe, Z.; Dherani, M.; Hosgood, H. D.; Mehta, S.; Pope,
D.; Rehfuess, E. Millions Dead: How Do We Know and What Does It
Mean? Methods Used in the Comparative Risk Assessment of
Household Air Pollution. Annu. Rev. Public Health 2014,35, 185−206.
(7) Bank, T. W.Household Cookstoves, Environment, Health, and
Climate Change, 2011.
(8) Smith, K. R. Changing Paradigms in Clean Cooking. EcoHealth
2015, April.
(9) Pillarisetti, A.; Vaswani, M.; Jack, D.; Balakrishnan, K.; Bates, M.
N.; Arora, N. K.; Smith, K. R. Patterns of Stove Usage after
Introduction of an Advanced Cookstove: The Long-Term Application
of Household Sensors. Environ. Sci. Technol. 2014,48, 14525−14533.
(10) Thomas, E. A.; Barstow, C. K.; Rosa, G.; Majorin, F.; Clasen, T.
Use of remotely reporting electronic sensors for assessing use of water
filters and cookstoves in Rwanda. Environ. Sci. Technol. 2013,47,
13602−13610.
(11) Ram, P. K.; Halder, A. K.; Granger, S. P.; Jones, T.; Hall, P.;
Hitchcock, D.; Wright, R.; Nygren, B.; Islam, M. S.; Molyneaux, J. W.;
Luby, S. P. Is Structured Observation a Valid Technique to Measure
Handwashing Behavior? Use of Acceleration Sensors Embedded in
Soap to Assess Reactivity to Structured Observation. Am. J. Trop. Med.
Hyg. 2010,83, 1070−1076.
(12) Ruiz-Mercado, I.; Masera, O.; Zamora, H.; Smith, K. R.
Adoption and sustained use of improved cookstoves. Energy Policy
2011,39, 7557−7566.
(13) Ruiz-Mercado, I.; Canuz, E.; Walker, J. L.; Smith, K. R.
Quantitative metrics of stove adoption using Stove Use Monitors
(SUMs). Biomass Bioenergy 2013,57, 136−148.
(14) Wilson, D. L.; Adam, M. I.; Abbas, O.; Coyle, J. R.; Kirk, A. E.;
Rosa, J.; Gadgil, A. J. In Technologies for Development: What is
Essential?, 1st ed.; Hostettler, S., Hazbourn, E., Bolay, J.-C., Eds.;
Springer: New York, NY, 2015; pp 211−221.
(15) Lewis, J. J.; Pattanayak, S. K. Who adopts improved fuels and
cookstoves? A systematic review. Environ. Health Perspect. 2012,120,
637−645.
(16) Nunnally, J. C. Psychometric Theory, 2nd ed.; McGraw-Hill: New
York, 1978.
(17) Edwards, A. L. the Social Desirability Variable in Personality
Assessment and Research; Dryden Press, 1957.
(18) Burwen, J. From Technology to Impact: Understanding and
Measuring Behavior Change with Improved Biomass Cookstoves, 2011
(19) Parsons, H. M. What Happened at Hawthorne?: New evidence
suggests the Hawthorne effect resulted from operant reinforcement
contingencies. Science 1974,183, 922−932.
(20) Das, J.; Hammer, J.; Sanchez-Paramo, C. The impact of recall
periods on reported morbidity and health seeking behavior. Journal of
Development Economics 2012,98, 76−88.
(21) Berkeley Air Monitoring Group. Monitoring and Evaluation of
the Jiko Poa Cookstove in Kenya Report, 2012.
(22) Burwen, J.; Levine, D. I. A rapid assessment randomized-
controlled trial of improved cookstoves in rural Ghana. Energy
Sustainable Dev. 2012,16, 328−338.
(23) Mukhopadhyay, R.; Sambandam, S.; Pillarisetti, A.; Jack, D.;
Mukhopadhyay, K.; Balakrishnan, K.; Vaswani, M.; Bates, M. N.;
Kinney, P. L.; Arora, N.; Smith, K. R. Cooking practices, air quality,
and the acceptability of advanced cookstoves in Haryana, India: an
exploratory study to inform large-scale interventions. Global Health
Action 2012,5, 1347.
(24) Beaman, L.; Magruder, J.; Robinson, J. Minding small change
among small firms in Kenya. Journal of Development Economics 2014,
108, 69.
(25) Jessica J Lewis, S. K. P. Who Adopts Improved Fuels and
Cookstoves? A Systematic Review. Environ. Health Perspect. 2012,120,
637.
(26) Benjamin Arnold, D. M. A. H.; Byron, Arana; John, M.; Colford,
J. Evaluation of a pre-existing, 3-year household water treatment and
handwashing intervention in rural Guatemala. International Journal of
Epidemiology 2009,38, 1651−1661.
(27) Kremer, M.; Leino, J.; Miguel, E.; Zwane, A. P. Spring Cleaning:
Rural Water Impacts, Valuation, and Property Rights Institutions.
Quarterly Journal Economics 2011,126, 145−205.
(28) Darhl, M.; Pedersen, K. B. Sheila Keithly Effects of an observer
on conformity to handwashing norm. Perceptual and Motor Skills 1986,
62, 169−70.
(29) Alan Russell, D. M. Graeme Russell Observer Influences on
Mothers and Fathers: Self-Reported Influence During a Home
Observation. Merrill-Palmer Quarterly 1992,38, 263−283.
(30) Thomas, E. A.; Tellez-Sanchez, S.; Wick, C.; Kirby, M.;
Zambrano, L.; Abadie Rosa, G.; Clasen, T. F.; Nagel, C. Behavioral
Reactivity Associated With Electronic Monitoring of Environmental
Health Interventions−A Cluster Randomized Trial with Water Filters
and Cookstoves. Environ. Sci. Technol. 2016,50, 3773−3780.
(31) Clasen, T.; Fabini, D.; Boisson, S.; Taneja, J.; Song, J.; Aichinger,
E.; Bui, A.; Dadashi, S.; Schmidt, W.-P.; Burt, Z.; Nelson, K. L. Making
Sanitation Count: Developing and Testing a Device for Assessing
Latrine Use in Low-Income Settings - Environmental Science &
Technology (ACS Publications). Environ. Sci. Technol. 2012,46, 3295.
(32) Bandura, A. Self-efficacy: Toward a Unifying Theory of
Behavioral Change. Psychological Review 1977,84, 191−215.
(33) Aronson, E. Retrospection on Social Psychology; Festinger, L., Ed.;
Oxford University Press: Oxford, 1980; pp 3−21.
(34) Chris Ann Dickerson, E. A. D. M. Ruth Thibodeau Using
Cognitive Dissonance to Encourage Water Conservation. J. Appl.
Social Psychol. 1992,22, 841−854.
Environmental Science & Technology Article
DOI: 10.1021/acs.est.6b02899
Environ. Sci. Technol. 2016, 50, 8393−8399
8399
