Content uploaded by Christopher Tull

Author content

All content in this area was uploaded by Christopher Tull on Jan 05, 2018

Content may be subject to copyright.

How Much Water Does Turf Removal Save? Applying

Bayesian Structural Time-Series to California Residential

Water Demand

Christopher Tull

ctull17@gmail.com

Eric Schmitt

eric.schmitt@protix.eu Patrick Atwater

patrickatwater@gmail.com

California Data Collaborative

418 Bamboo Lane, Los Angeles, CA 90012

1. ABSTRACT

California water utilities have invested historic amounts

of money in turf rebates to incentivize customers to remove

their turf grass and replace it with more water eﬃcient land-

scaping. This study utilizes a data set of 545 unique single-

family residential turf rebates across 3 California water util-

ities, totaling 635,713 square feet of converted turf grass

to estimate the water savings from turf removal. Monthly

water savings are estimated at the household level as the

di↵erence between actual usage and a synthetic control and

then aggregated using a mixed-e↵ects regression model to

investigate the determinants of water savings. Analysis of

turf removal at the monthly level is found to be critical for

understanding the seasonal behavior inherent in outdoor wa-

ter use. Mean predicted savings for single-family residential

accounts are estimated at 24.6 gallons per square foot per

year for the households used in this study.

2. INTRODUCTION

With outdoor landscaping representing approximately half

of urban water usage, the water community has identiﬁed

outdoor water usage in general (Mayer, Lander, and Glenn

2015), and ornamental lawns speciﬁcally (CUWCC 2015)

as a key opportunity in the larger e↵ort to increase wa-

ter conservation. Between July 2014 and April 2016, the

Metropolitan Water District (MWD), the regional whole-

saler of Colorado and Bay Delta water for Southern Califor-

nia, paid out $270.7 million directly for turf rebates under its

regional program and another $15.1 million to supplement

member agency spending on turf replacement. Metropoli-

tan indirectly serves 6.1 million residential households across

Southern California (MWD 2016). In addition, millions in

local retailer turf rebate supplements have been paid out

(for example in Los Angeles, Long Beach, San Diego and

Moulton Niguel).

A small number of studies have investigated the impact of

turf removal conservation rebate programs on water usage.

Permission to make digital or hard copies of all or part of this work for personal or

classroom use is granted without fee provided that copies are not made or distributed

for proﬁt or commercial advantage and that copies bear this notice and the full cita-

tion on the ﬁrst page. Copyrights for components of this work owned by others than

ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or re-

publish, to post on servers or to redistribute to lists, requires prior speciﬁc permission

and/or a fee. Request permissions from permissions@acm.org.

KDD Workshop on Data Science for Food, Energy and Water ’16 San Fran-

cisco, California USA

c

2016 ACM. ISBN 978-1-4503-2138-9.

DOI: 10.1145/1235

In 2005, the Southern Nevada Water Authority (SNWA)

conducted a turf removal and Xeriscape planting study that

found these rebates led to 55 gallons in water savings per

year per sq. ft. of turf removed (Sovocool, Authority, and

Morgan 2005). These results may not be reﬂective of the im-

pact of such policies in the climate in Los Angeles and the

absence of a requirement for Xeriscape landscaping. Indeed,

P. Atwater, Schmitt, and Atwater (2015) found a more mod-

est residential reduction of approximately 18 gallons per year

per sq. ft. in the Moulton Niguel Water District in south Or-

ange County, California. Similarly, the Metropolitan water

district conducted a study in 2014 with robust hydrologi-

cal variation that found an average relative water reduction

of 18.2% from participating residential households and 24%

from participating commercial accounts (MWD 2014).

This study builds on previous research and develops a

novel methodology for assessing the impact of water con-

servation actions. Previous work in P. Atwater, Schmitt,

and Atwater (2015) utilized a multilevel quantile regression

model to control for the determinants of water use and iso-

late the average reduction in usage due to turf removal.

However, the authors were unsatisﬁed with the methodol-

ogy used to control for behavioral elements such as envi-

ronmental attitudes, and structural shifts in usage like Cal-

ifornia’s 2015 mandatory water conservation requirements.

The approach used here instead borrows from the market-

ing context to match residential households based on their

past water usage behavior instead of using static descriptive

attributes like household size and irrigable area. A syn-

thetic control is then created and a di↵erence-in-di↵erences

approach is applied to estimate monthly water savings at

the level of individual households. This enables analysis

of the full distribution of water use changes, including sea-

sonal ﬂuctuations in the amount of water saved that high-

light peak summer demand reduction. Finally the individual

estimates are aggregated using a meta-analytic mixed-e↵ects

model to control for moderator variables of interest.

3. METHODOLOGY

3.1 Data

The data used in this study was provided by 3 water utili-

ties: Moulton Niguel Water District (MNWD), Irvine Ranch

Water District (IRWD), and Eastern Municipal Water Dis-

trict (EMWD). Each utility provided two data sources. The

ﬁrst is a panel data set of monthly billed water usage and

customer characteristics identiﬁed by account and service

point (water meter) identiﬁers for single family households.

The second is a data set detailing participation in water

eﬃciency rebate programs, of which turf removals are the

primary interest for this study.

These two data sets are merged, and any turf rebate in-

stances tied to accounts that appear more than once are

dropped to prevent overcounting. These accounts are then

further restricted to those that have at least two years of

data (24 observations) in the pre-rebate period and one year

of data (12 observations) in the post-rebate period. The

pre- and post-rebate periods are determined relative to the

month that the post-rebate inspection was performed. Fi-

nally, the water districts make use of default values in cases

where the actual value is unknown. Some districts substi-

tute default values for irrigable area when actual values are

not known. Customers with default values were dropped

in cases where this was obvious due to bunching of many

customers at the same value of irrigable area.

The working dataset contains 545 observations of either

traditional or synthetic turf rebates after ﬁltering. The vari-

ables are deﬁned as follows:

•Customer ID: unique identiﬁer for each household.

•Month and Year: the month and year of the water bill.

•HH Size: number of permanent residents at the prop-

erty.

•Irr Area Sf and Rebate Quantity: the square feet of

irrigable area and square feet of turf removed during

rebate, respectively.

•Rebate Area Ratio: the proportion of turf area re-

moved, calculated as Rebate Quantity

Irr Area Sf .

•Evapotranspiration: The reference evapotranspira-

tion, ET0, in inches.

3.2 Time Series Matching and Rebate Impact

Estimation

The estimation of rebate water savings is implemented

using the Rprogramming language. It is done in three steps.

Given N= 545 treatment accounts which participated in a

turf removal rebate and are examined in this study:

1. Each treatment account tri,i21...N which has par-

ticipated in a turf rebate is matched with a set of con-

trol accounts Ci={cj

i},j 21...6 from the same zip

code which did not participate in a turf rebate. These

control accounts are chosen by how similar their his-

torical usage patterns are to the usage patterns of the

treatment account tri, based on a weighted combina-

tion of their Pearson correlation and their warping dis-

tance.

2. After the cihave been chosen, we ﬁt a Bayesian struc-

tural time series model and use it to estimate the

monthly impact of turf removal on water savings. The

structural time series (STS) model uses the water us-

age patterns of the control accounts to create a syn-

thetic control corresponding to the expected water us-

age of triif there had been no turf removal. The

predicted usage in the post-rebate period is then sub-

tracted from observed usage to obtain a monthly water

savings estimate for tri.

3. After a water savings estimate has been calculated for

each treatment account, the last step is to obtain an

overall summary estimate. This is done with a meta-

analytic approach that uses the estimates and vari-

ances from each treatment account as the inputs into

a random e↵ects model.

The ﬁrst two steps are implemented into a workﬂow by

the MarketMatching package. 1.

3.3 Choosing Control Accounts

The ﬁrst step in obtaining an estimate of the turf removal

impact for account triis to ﬁnd accounts that did not remove

their turf that show similar behavior to tri. Candidate ac-

counts were identiﬁed by choosing controls from within the

same zip code as tri. Within each zip code there may still

be thousands of possible controls. These remaining possibil-

ities are ranked by how similar their historical water usage

patterns are to the historical usage of tri.

Account matching is often based on variables like prop-

erty size, property value, or education levels. However, the

importance of environmental attitudes, for example aris-

ing from public awareness actions and social change has

been shown to inﬂuence water consumption (Hollis 2016).

The diﬃculty of incorporating these and other diﬃcult-to-

quantify factors driving household water usage, and the fairly

stable water consumption patterns observed by most house-

holds, make matching based on water consumption patterns

attractive. The premise of using historically predictive rela-

tionships between accounts to perform counterfactual anal-

ysis in this fashion has been advocated by cf. Abadie, Dia-

mond, and Hainmueller (2010) and Brodersen et al. (2015).

Let tr and cbe a treatment and control time series with m

observations each for which a similarity ranking is desired.

This similarity ranking is done as a weighted composite of

two other similarity measures. The ﬁrst is the Pearson cor-

relation:

⇢(tr, c)= Pp

t=1(trt¯

tr)(ct¯c)

pPp

t=1(trt¯

tr)2pPp

t=1(ct¯c)2(1)

The second ranks them according to their dynamic time

warping (DTW) distance from tri. To compute the warp-

ing distance between two time series, we must identify the

warping curve (t)=(tr(t),

c(t)) that has the minimum

warping distance,

D(tr, c)=

p

X

t=1

d(tr(t),

c(t))p(t),(2)

where d(tr(t),

c(t)) is the local of the points at time

tafter they have been remapped by the warping functions

tr(t) and c(t), and m(t) is per-step weight that control

the slope of the warping curve. The calculation of the DTW

distance is done using the dtw package. For details about

the package and about dynamic time warping see Giorgino

and others (2009).

Let the vector r

r

rdenote the similarity scores for Kcandi-

date control accounts ck,k =1,...,K with respect to tri,

1The code was modiﬁed and is available at

https://github.com/christophertull/MarketMatching/tree/usability-

improvements

where the kth element of r

r

ris given by:

rk=(1↵)⇢(tri,c

k)+↵D(tri,c

k),

with ↵2[0,1]. Then, the control households corresponding

to the ﬁrst mvalues of the sorted r

r

rare used as controls

for triin the structural time series model for that series

discussed in the next section.

3.4 Estimating Water Savings

A widely used approach for estimating the causal im-

pact of interventions, like rebate o↵erings, is di↵erences-in-

di↵erences. Taking this approach in the turf removal con-

text, the estimated causal impact of turf removal on water

savings is the di↵erence between water usage when turf was

removed, and the amount of water that would have been

used if no turf had been removed (Bamezai 1995).

To accurately estimate the reduction in water usage due

to turf removal, a model for the counterfactual case needs to

account for other variables determining water usage. Water

use is determined by a multitude of factors, such as weather,

user size, social perspectives on water usage, and turf re-

moval. Covariates like weather and user size are measured

by agencies and are straightforward to account for with a

model.

This leaves the matter of accounting for dynamic behav-

ioral patterns. Recognizing the need to address this aspect of

water use, Hollis (2016) took variables measuring media fac-

tors, like advertising volume, to explain water use patterns.

The inclusion of media presence explicitly in a usage model

is desirable, but two issues that arise with this approach are

properly quantifying media presence and accounting for the

di↵erent levels of exposure experienced by water users.

Another way to account for dynamic behavior is to ex-

plicitly model the counterfactual of a time series observed

both before and after the rebate and use the resulting model

to construct a synthetic control (cf. Abadie, Diamond, and

Hainmueller (2010)). The approach of Brodersen et al. (2015)

is to construct a synthetic control by combining three sources

of information using a state-space time-series model, where

one component of state is a linear regression on the contem-

poraneous predictors. The ﬁrst source of information is the

behavior of the response prior to the turf removal. A second

is to use other time series that were predictive of the target

series before the turf removal. In particular, a relationship

between a time series which removed turf and others that did

not can be used to estimate a synthetic control after the re-

bate. These series allow us to account for unmodeled causes

of variance such as a general decline in water usage due to

media campaigns or mandatory reductions due to drought

restrictions. Thirdly, in a Bayesian framework prior knowl-

edge about the model parameters, from prior studies, for

example, can be used to construct the counterfactual.

We will use static regression coeﬃcients in our Bayesian

structural time series model, which assumes that the linear

usage relationship between the controls and the counterfac-

tual expected usage for customers who did remove turf from

their lawn remains ﬁxed even after the turf is removed. Fur-

thermore, we will allow for a local linear trend. For a time

series y

y

y, this model has the form:

yt=µt

|{z}

level

+Zt

|{z}

regression

+"t,(3)

Zt=0x

x

x, (4)

µt+1 =µt+t+⌘µ,t

| {z }

random walk and trend

,(5)

t+1 =t+⌘,t

| {z }

random walk for trend

,(6)

where "t⇠N(0,

2

t), ⌘µ,t ⇠N(0,

2

µ,t) and ⌘,t ⇠N(0,

2

,t ).

The regression component, Ztcaptures the static linear rela-

tionship between the control series and the treatment series,

while the level component µtcaptures local linear trends,

enabling the model to react to unobserved sources of vari-

ability the control and treatment series are exposed to.

By placing a spike-and-slab prior on the set of regression

coeﬃcients, and by allowing the model to average over the

set of controls, it is possible to choose from many candi-

date controls (George and Mcculloch 1997). To combine

information about the target time series and the controls,

the posterior distribution of the counterfactual time series

is computed given the value of the target series in the pre-

intervention period, along with the values of the controls

in the post-intervention period. Given a predicted and ob-

served water use ˆytand yt, the di↵erence ˆytytyields a

semiparametric Bayesian posterior distribution for the wa-

ter savings attributable to the turf removal, which can be

used to obtain credible intervals. We take these estimates

and adjust them to gallons saved per square foot to obtain:

•µgpsf: monthly gallons saved per square foot of turf

removed, calculated as

748.052 ⇥(yit ˆyit)

rebate instance quantity

where ytand ˆytare the actual and estimated usage in

hundred cubic feet (CCF) of household triat month t.

The structural time series model was ﬁt using the CausalImpact

package provided by Google for estimating the e↵ectiveness

of marketing campaigns (Brodersen et al. 2015). A number

of di↵erences exist between the Google marketing context

described in Brodersen et al. (2015), for which this approach

was originally proposed, and the turf removal rebate context.

Firstly, Google is able to assess the impact of the marketing

campaign in terms of participation using this method, where

participation is measured in number of clicks, because they

have data on number of clicks prior to the campaign. It

is in their interest to distinguish how many clicks after the

start of the campaign were driven by the campaign, as op-

posed to organic. In contrast, prior to the rebate programs,

the water districts did not track turf removal. The number

of rebate claims before the start of the rebate programs is

zero, and the number of rebates claimed afterwards is best

summarized using simple statistics.

Another di↵erence is that in the marketing context, the

impact to estimate is the number of clicks generated as a

consequence a marketing campaign, where a marketing cam-

paign is either active or is not. The scale of the marketing

campaign is not addressed. We could stop at estimating an

average e↵ect of turf removal, but this neglects the impor-

tant relationship between how much water use is reduced

Parameter Values

WARPING LIMIT 0, 1

DTW EMPHASIS 0, 0.25, 0.5, 0.75, 1

NUMBER OF MATCHES 6, 12

Table 1: Parameter Values tested in sensitivity anal-

ysis.

and the amount of the turf removed. To account for this,

the estimated savings are divided by the square feet of turf

removed, as calculated by utility sta↵in a post-rebate in-

spection. This allows for a normalized measure of rebate

impact in terms of gallons per square foot of turf removed.

Additionally, variables to quantify the magnitude of the turf

removal are included in the meta-model in the ﬁnal step.

An added complexity in this study is that in place of a

single treatment cohort, or perhaps a few, hundreds of cus-

tomers participated in the rebate program. The approach

proposed in Brodersen et al. (2015) stops at providing im-

pact estimates on a single time series at a time. To obtain a

broad overview of the impact of turf removal, it is desirable

to aggregate estimates from all of the customers. In the sec-

tion that follows, this issue and the inclusion of the amount

of turf removed in our framework will be addressed using a

meta-analytic approach.

3.4.1 Example

Figure 1 below shows two examples of the process de-

scribed above. Speciﬁcally, the output of the matching pro-

cess is shown through charts of water usage over time for

the treatment household and its six closest matches. The

output of the STS model is given by showing the actual

and predicted consumption for the two examples. The ex-

ample households were chosen for their wildly di↵erent be-

havior patterns in the post-rebate period. One household

appears to cease outdoor watering completely after their

turf removal, causing their usage to stabilize at winter levels

and achieving an estimated 66% reduction in overall water

use. The other example household shows a decrease in usage

relative to its own past behavior, but shows no signiﬁcant

reduction compared to its similarly-behaving peers. This

e↵ect may be due to increased awareness of the California

drought and the mandatory restrictions put in place in April

2015. Thus the water savings would be attributed to behav-

ioral change among households in the region but not directly

to the removal of turf.

3.5 Parameter selection for the matching and

STS steps

A number of parameters must be chosen when applying

the matching procedure and STS model. We assessed these

in terms of their impact on the mean water savings estimates

obtained from the STS models.

A sensitivity analysis was performed to determine the ef-

fect of parameter choices at the matching stage on ﬁnal es-

timates of water savings. Speciﬁcally, a random sample of

150 accounts that made it through the ﬁltering were rerun

under all combinations of the di↵erent parameter conﬁgu-

rations visible in Table 3. While these are not the only

parameters in the model, they are three of the ones most

likely to impact the water savings estimates because they

directly impact the choice of control accounts.

In the STS model, the value for 2

µ,t in the local linear

trend must also be selected. This is the local level stan-

dard deviation which controls the prior standard deviation

of the local linear trend submodel. The local level term

modiﬁes how adaptable the model is to short term changes,

and its standard deviation is important because it e↵ects the

breadth of the posterior intervals. Brodersen et al. (2015)

recommend that the value of 0.01 can be used when the re-

lationship between the controls and the treatment is strong

enough to obtain an informative model. The authors indi-

cate that this is more likely when many control candidates

are available. The water usage data set contains a large

pool of control candidates, and matching results are typi-

cally strong. The choice of 0.01 results

After calculating savings estimates under each parameter

set, the mean of estimated savings for the sample under each

parameter set was calculated. This gives an idea of how sen-

sitive the matching process is to changes in the parameters.

These estimates are visible below in Figure 2.

Figure 2: The charts display the sensitivity of the

meta-estimate results under various values of the

DTW EMPHASIS parameter. Each chart in turn

uses a di↵erent warping limit or number of control

account matches.

Table 2 shows the values of the matching procedure pa-

rameters based on the results from the sensitivity analysis,

as well as required minimum observation period lengths and

matching pool sizes.

3.6 Combining the Estimates

Monthly estimated water savings attributable to turf re-

moval are obtained from each of the Bayesian STS models,

yielding a total of 10759 impact estimates for 545 house-

holds. Furthermore, a credible interval can be calculated for

each of these estimates. The aggregation of these estimates

can be seen as a meta-analysis. Before a meta-analysis is

conducted, a robust regression is performed using the same

continuous moderator variables as the meta-model to re-

move large outliers. The robust method, Least Trimmed

Squares is used with default settings as implemented by the

ltsReg function in the robustbase package. After remov-

ing outliers, a random e↵ects model to determine an overall

meta-estimate for water savings is ﬁtted. The meta analysis

is done using the metafor package. Details on the technique

Figure 1: The ﬁrst row shows the expected and observed usage patterns for two participating rebate accounts,

where the di↵erence between expected and observed after removal (dashed) is the estimated savings. The

account on the left shows a visible reduction in usage compared to the counterfactual, while the right side

has more ambiguous results. The bottom row shows the raw time series of water usage for the treatment

and corresponding matched controls.

Table 2: Key Parameter choices in the modeling

process.

Parameter Value Description

Min. Months Post-

Period

12 Require at least 12 months

since the rebate took place.

Min. Months Pre-

Period

24 Require at least 24 months

before the rebate for accu-

rate matching.

Zip Sample Size 500 Randomly sample a maxi-

mum of 500 control accounts

within the zip code as possi-

ble matches.

Min. Matching Se-

ries

100 Require a pool of at least 100

possible matches within the

zip code.

Warping Limit 1 The size of the Sakoe-Chiba

band limiting how much the

time series are allowed to

warp.

DTW Emphasis 0.7 Controls the trade-o↵be-

tween the DTW distance

and Pearson correlation.

Number of Matches 6 The number of control ac-

counts to match with and

pass into the STS model.

and the software are available in Viechtbauer and others

(2010).

4. RESULTS

The meta-analysis we conduct to aggregate the results

from the Bayesian STS model estimates of the water savings

from the ith turf-removing household at time tis a mixed

e↵ects model with the following ﬁxed e↵ects structure:

µgpsfi,t =↵i+0+1⇥HH Sizei,t

+2⇥Rebate Area Ratioi,t

+3⇥sin(2⇡/12Monthi,t)

+4⇥cos(2⇡/12Monthi,t)

+5⇥sin(4⇡/12Monthi,t)

+6⇥cos(4⇡/12Monthi,t)

+7⇥ln(Rebate Quantityi,t)

+8⇥ln(Irr Area Sfi,t)

+9⇥Evapotranspirationi,t +"i,t.

(7)

where µgpsf is the monthly savings in gallons per square

foot. The trigonometric terms in the model account for sea-

sonality. Month, in this model, is a unique number for each

of the months in the study and runs from 1,...,51.

Table 3 contains the ﬁxed e↵ects estimates of the ﬁtted

model. Examining the e↵ect of household size, we see that

the more people there are in a household, the lower the im-

pact of turf removal per square foot. This is likely because

indoor water usage is larger in larger households, diminish-

ing the potential savings from outdoor water usage relative

to a similarly sized house with fewer inhabitants. Rebate

area ratio has a large negative coeﬃcient, meaning that the

larger the share of the household’s irrigable area that is re-

moved, the greater the savings. Three of the trigonometric

e↵ects are signiﬁcant, and are used by the model to capture

general seasonal trends in water savings. The positive co-

eﬃcient of ln(Rebate Instance Quantity) means that per

foot savings are smaller as the amount of turf removed in-

creases, possibly because watering eﬃciency increases with

larger gardens and lawns. In contrast, the greater the irri-

gable area in total, the larger the savings. This e↵ect can be

similar to the household e↵ect. The larger the irrigable area

of a household, the larger outdoor watering’s share of wa-

ter use, and thus the greater the impact of turf removal on

household water use per square foot of property. Lastly the

evapotranspiration (ET) coeﬃcient is negative, indicating

greater savings with increased ET.

Table 3: Fixed e↵ect estimates for the meta-model

of turf removal water savings.

Variable Estimate SE t-stat p-value

Intercept -0.57 0.67 -0.84 0.40

HH Size 0.12 0.03 3.48 0.00

Rebate Area Ratio -3.66 0.57 -6.45 0.00

Month Sin 2 0.43 0.03 15.49 0.00

Month Cos 2 0.24 0.05 4.52 0.00

Month Sin 4 0.13 0.02 5.57 0.00

Month Cos 4 0.03 0.03 1.10 0.27

ln(Rebate Instance Quantity) 2.08 0.22 9.32 0.00

ln(Irr Area Sf) -1.77 0.22 -7.89 0.00

Evapotranspiration -0.08 0.02 -3.61 0.00

The ﬁrst analysis we conduct is a comparison of average

household savings by year. We do this for the sample in this

study by using the model to predict the household savings

given their moderator variables. Predicted savings are then

grouped by household and year and averaged. The result-

ing savings estimates give an impression of the distributions

of savings outcomes that would be expected by an analyst

or policy-maker on this population. We see that annual

savings were about 20 gallons per square foot. However, by

aggregating the monthly savings to an annual level, we loose

important details about the savings patterns.

A more nuanced approach is to use the model to predict

the monthly savings. Overlaying the predictions are quan-

tiles ranging from 5% to 95%. The savings pattern illus-

trated in Figure 4 is highly intuitive. The highest savings

are in the months of July, August and September, reaching

a monthly average of -2.7 gallons per square square foot less

water use. During the months of January, February, and

March, the reduction is much smaller but still valuable at

-1.5 gallons per square foot.

4.1 Time-Series vs. Traditional Matching

One remaining question of interest is whether time series

matching on historical usage produces comparable results

to traditional matching on static attributes. In order to ad-

dress this question, the mean distance from each treatment

account to its matched control accounts was compared to the

mean distance from each treatment to its potential controls

that were not matched.

Distance was calculated by standardizing the covariates

for household size and irrigable area within each zip code

and customer class. The mean euclidean distance was then

calculated between the treatment and each of the matched

and unmatched groups. The results of this calculation are

Figure 4: Predicted monthly savings for each household in the data set. The dark green line corresponds

to median savings. Seasonal variation leads to swings in average savings from -1.5 to -2.7 gallons per square

foot.

visible in Figure 5. One can see that matching on usage pat-

terns tends to result, on average, in matches that are also

similar in their household size and irrigable area. However,

this was not universally true and manual inspection revealed

a large variation even among the matched control accounts.

This aligns with the intuition that static covariates do not

capture all aspects of water usage, and that dissimilar ac-

counts may have very similar water usage patterns.

5. CONCLUSIONS

This methodology enables estimation of the water sav-

ings associated with turf removal using very minimal data

by requiring only observational water use over time and a

bare minimum of contextual customer attribute data. Many

other approaches rely on extensive lists of covariates that are

at best proxies for water use behavior. This work matches

on observed behavior directly and thereby attempts to incor-

porate the complexities of individual customer conservation

behavior, resulting in estimates of 24.6 gallons saved per year

per square foot of turf removed. Boostrapped standard er-

rors of those predicted water savings are .11 gallons per year

per square foot of turf removed. Those water savings are sta-

ble across district and vary sinusoidally over time highlight-

ing the structural water savings of turf market transforma-

tion for regional and statewide water reliability initiatives.

At $2 paid per square foot turf removed and assuming a hy-

perbolic discount rate of ﬁve percent over a landscape con-

version lifespan of thirty years, that translates into a present

value of $1422 plus or minus seven dollars per acre foot of

water saved.

Still, these results should be considered an early data point

measuring the e↵ects of the generational shift away from wa-

ter intensive lawns as the default landscaping. Landscape

conversions can involve up to a two year period for new

drought tolerant plants to establish and thus these results

may need to be reassessed in the future. In addition, the

turf rebate program has some uncertainty regarding the ex-

act timing of turf removal introducing additional variation

into these estimates. Furthermore, this study lacks data on

whether artiﬁcial turf or California native or other non-turf

landscaping were implemented after the rebate. Fortunately

the simple data requirements of this method make it easy

to redeploy on regularly updated customer use, rebate, cus-

tomer survey and other creative data sources such as aerial

remote sensing. This is the approach being pioneered by

the California Data Collaborative utilities in this study and

others as they centralize water use data. It enables water

managers to measure the water savings with turf removal

over time and adaptively manage this historic investment in

turf removal.

Measuring savings at the household level also allows wa-

ter managers to target educational materials on eﬃcient wa-

tering practices to customers that have seen dis-savings in

the post rebate period compared to their expected coun-

terfactual water use. Finally, the approach can be used to

evaluate the water savings associated with other conserva-

tion rebates, other customer-level demand management in-

terventions, and potentially other natural resource conser-

vation programs in energy or natural gas. As the old adage

Figure 3: Average yearly savings for each household

over 5 years.

goes, “you cannot manage what you cannot measure” and

such rigorous impact evaluations can help California’s public

managers navigate the uncertain future we face with climate

change.

6. ACKNOWLEDGEMENTS

This research was funded through the California Data Col-

laborative. The authors would like to thank the Moulton

Niguel, Irvine Ranch, Eastern Municipal, Las virgenes Mu-

nicipal, Santa Margarita, and Monte Vista water districts,

along with the Inland Empire Utilities Agency, the East Bay

Municipal Utility District, and the Metropolitan Water Dis-

trict of Southern California for all of their support. The

authors also thank Michael Hollis for his thoughtful com-

ments and peer review.

References

Abadie, Alberto, Alexis Diamond, and Jens Hainmueller.

2010. “Synthetic Control Methods for Comparative Case

Studies: Estimating the E↵ect of California’s Tobacco Con-

trol Program.” Journal of the American Statistical Associa-

tion 105 (490): 493–505.

Atwater, Patrick, Eric Schmitt, and Drew Atwater. 2015.

“Towards California Water Conservation Impact Evaluations

by Default: Lessons of a Turf Removal Rebate Study in

South Orange County.”

Bamezai, Anil. 1995. “Application of Di↵erence-in-Di↵erence

Techniques to the Evaluation of Drought-Tainted Water Con-

servation Programs.” Evaluation Review 19 (5). Sage Pub-

lications: 559–82.

Brodersen, Kay H, Fabian Gallusser, Jim Koehler, Nicolas

Remy, Steven L Scott, and others. 2015. “Inferring Causal

Impact Using Bayesian Structural Time-Series Models.” The

Annals of Applied Statistics 9 (1). Institute of Mathematical

Statistics: 247–74.

CUWCC. 2015. Sustainable Landscaping: Market Trans-

formation Framework. 716 10th Street, Suite 200, Sacra-

Figure 5: The top histogram shows the distribu-

tion of mean distances between treatments and their

matched controls with similar historical usage. The

bottom shows the distances between treatments and

the unmatched accounts with more dissimilar usage

patterns. On average, accounts with similar usage

tend to be more similar in household size and irri-

gable area than those with very di↵erent usage pat-

terns.

mento, CA 95814: California Urban Water Conservation

Council.

George, Edward I., and Robert E. Mcculloch. 1997. “Ap-

proaches for Bayesian Variable Selection.” Statistica Sinica,

339–74.

Giorgino, Toni, and others. 2009. “Computing and Visu-

alizing Dynamic Time Warping Alignments in R: The Dtw

Package.” Journal of Statistical Software 31 (7): 1–24.

Hollis, M. 2016. “The E↵ects of Conservation Messaging

on Retail Water Deliveries.” Water Practice and Technology

11 (1): 26–34.

Mayer, Peter, Paul Lander, and Diana Glenn. 2015. Out-

door Water Savings Research Initiative, Phase 1 - Analysis

of Published Research. 300 W. Adams Street, Suite 601,

Chicago, Illinois 60606-5109: Alliance for Water Eﬃciency.

MWD. 2014. California Friendly Turf Replacement In-

centive Program Southern California, Appendix E Water

Savings from Turf Replacement, Resource Analyst Unit. The

Metropolitan Water District of Southern California.

MWD. 2016. 2015 Urban Water Management Plan, Draft

March 2016. The Metropolitan Water District of Southern

California.

Sovocool, Kent A, Southern Nevada Water Authority, and

Mitchell Morgan. 2005. “Xeriscape Conversion Study.” Fi-

nal Report. Sout.

Viechtbauer, Wolfgang, and others. 2010. “Conducting

Meta-Analyses in R with the Metafor Package.” Journal of

Statistical Software 36 (3): 1–48.