Content uploaded by Javier Saez-Gallego
Author content
All content in this area was uploaded by Javier Saez-Gallego on Nov 05, 2017
Content may be subject to copyright.
© 2016 IEEE. Personal use of this material is permitted. Permission from IEEE must be obtained for
all other uses, in any current or future media, including reprinting/republishing this material for
advertising or promotional purposes, creating new collective works, for resale or redistribution to
servers or lists, or reuse of any copyrighted component of this work in other works.
1
A Data-driven Bidding Model for a Cluster of
Price-responsive Consumers of Electricity
Javier Saez-Gallego, Juan M. Morales, Member, IEEE, Marco Zugno, Member, IEEE, Henrik Madsen
Abstract—This paper deals with the market-bidding problem
of a cluster of price-responsive consumers of electricity. We
develop an inverse optimization scheme that, recast as a bilevel
programming problem, uses price-consumption data to estimate
the complex market bid that best captures the price-response
of the cluster. The complex market bid is defined as a series of
marginal utility functions plus some constraints on demand, such
as maximum pick-up and drop-off rates. The proposed modeling
approach also leverages information on exogenous factors that
may influence the consumption behavior of the cluster, e.g.,
weather conditions and calendar effects. We test the proposed
methodology for a particular application: forecasting the power
consumption of a small aggregation of households that took part
in the Olympic Peninsula project. Results show that the price-
sensitive consumption of the cluster of flexible loads can be largely
captured in the form of a complex market bid, so that this could
be ultimately used for the cluster to participate in the wholesale
electricity market.
Index Terms—Smart grid, demand response, electricity mar-
kets, inverse optimization, bilevel programming
I. NOTATIO N
The main notation used throughout the paper is stated below
for quick reference. Other symbols are defined as required.
A. Indexes
tIndex for time periods t∈ {1. . . 24}.
bIndex for bidding blocks b∈ B.
iIndex for features i∈ I.
B. Lower-level variables
xb,t Estimated load at block band time t.
λu
tDual variable of pick-up limit constraint at time t.
λd
tDual variable of drop-off limit constraint at time t.
ψb,t Dual variable of maximum power constraint for
block bat time t.
ψb,t Dual variable of maximum power constraint for
block bat time t.
C. Upper-level variables
e+
t, e−
tAuxiliary variables to model the absolute value of
the estimation error.
ab,t Marginal utility corresponding to bid block band
time t.
J. Saez-Gallego, J. M. Morales (corresponding author), and H. Madsen are
with the Technical University of Denmark, DK-2800 Kgs. Lyngby, Denmark
(email addresses: {jsga, jmmgo, hmad}@dtu.dk), and their work is partly
funded by DSF (Det Strategiske Forskningsr˚
ad) through the CITIES research
center (no. 1035-00027B) and the iPower platform project (no. 10-095378).
PtMinimum power consumption at time t.
PtMaximum power consumption at time t.
ru
tMaximum load pick-up rate at time t.
rd
tMaximum load drop-off rate at time t.
a0
bIntercept relative to the estimation of the marginal
utility corresponding to bid block b.
P0Intercept relative to the estimation of the minimum
power consumption.
P0Intercept relative to the estimation of the maximum
power consumption.
ru0Intercept relative to the estimation of the maximum
load pick-up rate.
rd0Intercept relative to the estimation of the maximum
load drop-off rate.
αa
iAffine coefficient relative to the marginal utility for
feature i.
αu
iAffine coefficient relative to the maximum load pick-
up rate for feature i.
αd
iAffine coefficient relative to the maximum load drop-
off rate for feature i.
αP
iAffine coefficient relative to the minimum power
consumption for feature i.
αP
iAffine coefficient relative to the maximum power
consumption for feature i.
D. Parameters
ptPrice of electricity at time t.
wtWeight of observation at time t.
xmeas Measured load at time t.
xmeas0
b,t Split of the measured load at block band time t.
Zi,t Feature iat time t.
L Penalization factor of the complementarity con-
straints.
E Forgetting factor.
II. INTRODUCTION
We consider the case of a cluster of flexible power con-
sumers, where flexibility is understood as the possibility for
each consumer in the cluster to change her consumption
depending on the electricity price and on her personal prefer-
ences. There are many examples of methods to schedule the
consumption of individual price-responsive loads (see, e.g.,
[1]–[3]). The portfolio of flexible consumers is managed by
a retailer or aggregator, which bids in a wholesale electricity
market on behalf of her customers. We consider the case where
such a market accepts complex bids, consisting of a series of
2
price-energy bidding curves, consumption limits, and maxi-
mum pick-up and drop-off rates. In this paper, we present a
data-driven methodology for determining the complex bid that
best represents the reaction of the pool of flexible consumers
to the market price.
The contributions of this paper are fourfold. The first
contribution corresponds to the methodology itself: we propose
a novel approach to capture the price-response of a pool of
flexible consumers in the form of a market bid using price-
consumption data. In this work, the price is given as the result
of a competitive market-clearing process, and we have access
to it only from historical records. This is in contrast to some
previous works, where the price is treated as a control variable
to be decided by the aggregator or retailer. In [4], for example,
the relationship between price and consumption is first mod-
eled by a Finite Impulse Response (FIR) function as in [5]
and peak load reduction is achieved by modifying the price.
Similar considerations apply to the works of [6]–[9], where a
bilevel representation of the problem is used: the lower-level
problem optimizes the household consumption based on the
broadcast electricity price, which is determined by the upper-
level problem to maximize the aggregator’s/retailer’s profit.
Another series of studies concentrate on estimating price-
energy bids for the participation of different types of flexible
loads in the wholesale electricity markets, for example, time-
shiftable loads [10], electric vehicles [11] and thermostatically-
controlled loads [12]. Contrary to these studies, our approach
is data-driven and does not require any assumption about the
nature of the price-responsive loads in the aggregation. In that
sense, our work is more similar to [13], where the satisfaction
or the utility of users is estimated through historical data.
The main differences are that we aim to minimize prediction
errors instead of just estimating the utility, and that our
estimated utility and further technical parameters defining a
complex market bid may depend on time and external factors.
Furthermore, we test our methodology on actual data obtained
from a real-life experiment.
The second contribution lays in the estimation procedure:
we develop an inverse optimization framework that results in a
bilevel optimization problem. Methodologically, our approach
builds on the inverse optimization scheme introduced in [14],
but with several key differences. First, we let the measured
solution be potentially non-optimal, or even non-feasible, for
the targeted optimization problem as in [15]–[18]. Second, we
seek to minimize the out-of-sample prediction error through
the use of a penalty factor L. Moreover, we extend the concept
of inverse optimization to a problem where the estimated
parameters may depend on a set of features and are also
allowed to be in the constraints, and not only in the objective
function. Lastly, the estimation procedure is done using a two-
step algorithm to deal with non-convexities.
Third, we study heuristic solution methods to reduce the
computing times resulting from the consideration of large
datasets of features for the model estimation. We do not solve
the resulting bilevel programming problem to optimality but
instead we obtain an approximate solution by penalizing the
violation of complementarity constraints following a procedure
inspired by the work of [19].
Finally, we test the proposed methodology using data from
a real-world experiment that was conducted as a part of the
Olympic Peninsula Project [20].
It should be stressed that the proposed methodology aims
to capture the price-response behavior of the pool of flexible
consumers, and not to modify it. For this reason, we treat
the electricity price as an exogenous variable to our model
and not as a control signal to be determined. By means of
our methodology, the consumers are directly exposed to the
wholesale market price, without the need for the aggregator to
artificially alter this price or to develop any trading strategy.
Notwithstanding this, the price-response model for the pool of
flexible consumers that we estimate in the form of a complex
market bid could also be used by the aggregator to determine a
series of prices such that the consumption of the pool pursues
a certain objective.
III. METHODOLOGY
In this section, we describe the methodology to deter-
mine the optimal market bid for a pool of price-responsive
consumers. The estimation procedure is cast as a bilevel
programming problem. The upper level is the parameter-
estimation problem and represents the aggregator, who aims
at determining the parameters of the complex market bid
such that the estimated absolute value of the prediction error
is minimized. This bid can be directly processed by the
market-clearing algorithms currently in place in most elec-
tricity markets worldwide. A detailed explanation is given in
Section III-B. The estimated bid, given by the upper-level
problem, is relative to the aggregated pool of consumers. The
lower-level problem, explained in Section III-A, represents
the price-response of the whole pool of consumers under the
estimated bid parameters.
A. Lower-Level Problem: Price-response of the Pool of Con-
sumers
The lower-level problem models the price-response of the
pool of consumers in the form of a market bid, whose
parameters are determined by the upper-level problem. The
bid is given by θt={ab,t, ru
t, rd
t, P t, P t}, which consists of
the declared marginal utility corresponding to each bid block
b, the maximum load pick-up and drop-off rates (analogues
to the ramp-up and -down limits of a power generating unit),
the minimum power consumption, and the maximum power
consumption, at time t∈ T ≡ {t:t= 1 . . . T }, in that
order. If the whole aggregation of consumers behaves indeed
as a utility-maximizer individual, the declared utility function
represents the benefit that the pool of flexible consumers
obtains from consuming a certain amount of electricity. In the
more general case, the declared marginal utility, or simply the
bidding curve, reflects the elasticity of the pool of consumers
to changes in the electricity price. The declared marginal utility
function together with the rest of parameters in (1) define
a complex market bid that can be processed by the market-
clearing algorithms used by most wholesale electricity markets
worldwide, while representing, as much as possible, the price-
response behavior of the aggregation of consumers.
3
The declared marginal utility ab,t at time tis formed by
b∈ B ≡ {b:b= 1 . . . B}blocks, where all blocks have
equal size, spanning from the minimum to the maximum
allowed consumption. In other words, the size of each block
is Pt−Pt
B. Furthermore, we assume that the marginal utility
is monotonically decreasing as consumption increases, i.e.,
ab,t ≥ab+1,t for all times t. Finally, the total consumption
at time tis given by the sum of the minimum power de-
mand plus the consumption linked to each bid block, namely,
xtot
t=Pt+Pb∈B xb,t.
Typically, the parameters of the bid may change across
the hours of the day, the days of the week, the month, the
season, or any other indicator variables related to the time.
Moreover, the bid can potentially depend on some external
variables such as temperature, solar radiation, wind speed,
etc. Indicator variables and external variables, often referred
to as features, can be used to explain more accurately the
parameters of the market bid that best represents the price-
response of the pool of consumers. This approach is potentially
useful in practical applications, as numerous sources of data
can help better explain the consumers’ price-response. We
consider the Iexternal variables or features, named Zi,t for
i∈ I ≡ {i:i= 1, . . . , I}, to be affinely related to
the parameters defining the market bid by a coefficient αi.
This affine dependence can be enforced in the model by
letting ab,t =a0
b+Pi∈I αa
iZi,t,ru
t=ru0+Pi∈I αu
iZi,t,
rd
t=rd0+Pi∈I αd
iZi,t,Pt=P0+Pi∈I αP
iZi,t, and
Pt=P0+Pi∈I αP
iZi,t. The affine coefficients αa
i,αu
i,αd
i,
αP
iand αP
i, and the intercepts a0
b, ru0, rd0, P 0, P 0enter the
model of the pool of consumers (the lower-level problem) as
parameters, together with the electricity price.
The objective is to maximize consumers’ welfare, namely,
the difference between the total utility and the total payment:
Maximize
xb,t X
t∈T X
b∈B
ab,txb,t −ptX
b∈B
xb,t!(1a)
where xb,t is the consumption assigned to the utility block b
during the time t,ab,t is the marginal utility obtained by the
consumer in block band time t, and ptis the price of the
electricity during time t. For notational purposes, let T−1=
{t:t= 2, . . . , T }. The problem is constrained by
Pt+X
b∈B
xb,t −Pt−1−X
b∈B
xb,t−1≤ru
tt∈ T−1(1b)
Pt−1+X
b∈B
xb,t−1−Pt−X
b∈B
xb,t ≤rd
tt∈ T−1(1c)
xb,t ≤Pt−Pt
Bb∈ B, t ∈ T (1d)
xb,t ≥0b∈ B, t ∈ T .(1e)
Equations (1b) and (1c) impose a limit on the load pick-up and
drop-off rates, respectively. The set of equations (1d) defines
the size of each utility block to be equally distributed between
the maximum and minimum power consumptions. Constraint
(1e) enforces the consumption pertaining to each utility block
to be positive. Note that, by definition, the marginal utility is
decreasing in xt(ab,t ≥ab+1,t), so one can be sure that the
first blocks will be filled first. We denote the dual variables
associated with each set of primal constraints as λu
t, λd
t, ψb,t
and ψb,t.
Problem (1) is linear, hence it can be equivalently recast as
the following set of KKT conditions [21], where (2a)–(2c) are
the stationary conditions and (2d)–(2g) enforce complemen-
tarity slackness:
−λu
2+λd
2−ψb,1+ψb,1=ab,1−p1b∈ B (2a)
λu
t−λu
t+1 −λd
t+λd
t+1 −ψb,t +ψb,t =ab,t −pt
∀b∈ B, t ∈ T−1(2b)
λu
T−λd
T−ψb,T +ψb,T =ab,T −pTb∈ B (2c)
Pt+X
b∈B
xb,t −Pt−1−X
b∈B
xb,t−1≤ru
t⊥λu
t≥0
t∈ T−1(2d)
Pt−1+X
b∈B
xb,t−1−Pt−X
b∈B
xb,t ≤rd
t⊥λd
t≥0
t∈ T−1(2e)
xb,t ≤Pt−Pt
B⊥ψb,t ≥0b∈ B, t ∈ T (2f)
0≤xb,t ⊥ψb,t ≥0b∈ B, t ∈ T .(2g)
B. Upper-Level Problem: Market-Bid Estimation Via Inverse
Optimization
Given a time series of price-consumption pairs (pt, xmeas
t),
the inverse problem consists in estimating the value of the
parameters θtdefining the objective function and the con-
straints of the lower-level problem (1) such that the optimal
consumption xtresulting from this problem is as close as
possible to the measured consumption xmeas
tin terms of a
certain norm. The parameters of the lower-level problem θt
form, in turn, the market bid that best represents the price-
response of the pool.
In mathematical terms, the inverse problem can be described
as a minimization problem:
Minimize
x,θ X
t∈T
wtPt+X
b∈B
xb,t −xmeas
t(3a)
subject to
ab,t ≥ab+1,t b∈ B, t ∈ T (3b)
(2).(3c)
Constraints (3b) are the upper-level constraints, ensuring
that the estimated marginal utility must be monotonically
decreasing. Constraints (3c) correspond to the KKT conditions
of the lower-level problem (1).
Notice that the upper-level variables θt, which are parame-
ters in the lower-level problem, are also implicitly constrained
by the optimality conditions (2) of this problem, i.e., by the
fact that xb,t must be optimal for (1). This guarantees, for
example, that the minimum power consumption be positive
and equal to or smaller than the maximum power consumption
(0≤Pt≤Pt). Furthermore, the maximum pick-up rate
4
is naturally constrained to be equal to or greater than the
negative maximum drop-off rate (−rd
t≤ru
t). Having said
that, in practice, we need to ensure that these constraints are
fulfilled for all possible realizations of the external variables
and not only for the ones observed in the past. We achieved
this by enforcing the robust counterparts of these constraints
[22]. An example is provided in the appendix.
Parameter wtrepresents the weight of the estimation error at
time tin the objective function. These weights have a threefold
purpose. Firstly, if the inverse optimization problem is applied
to estimate the bid for the day-ahead market, the weights could
represent the cost of balancing power at time t. In such a
case, consumption at hours with a higher balancing cost would
weigh more and consequently, would be fit better than that
occurring at hours with a lower balancing cost. Secondly, the
weights can include a forgetting factor to give exponentially
decaying weights to past observations. Finally, a zero weight
can be given to missing or wrongly measured observations.
The absolute value of the residuals can be linearized by
adding two extra nonnegative variables, and by replacing the
objective equation (3a) with the following linear objective
function plus two more constraints, namely, (4b) and (4c):
Minimize
xt,θt,e+
t,e−
t
T
X
t=1
wt(e+
t+e−
t)(4a)
subject to
Pt+X
b∈B
xb,t −xmeas
t=e+
t−e−
tt∈ T (4b)
e+
t, e−
t≥0t∈ T (4c)
ab,t ≥ab+1,t t∈ T (4d)
(2).(4e)
In the optimum, and when wt>0, (4b) and (4c) imply
that e+
t=xt−xmeas
tif xt≥xmeas
t, else e−
t=xmeas
t−xt.
By using this reformulation of the absolute value, the weights
could also reflect whether the balancing costs are symmetric
or skewed. In the latter case, there would be different weights
for e+
tand e−
t.
To sum up, we have, on the one hand, problem (1), which
represents the postulated price-response model for the pool
of flexible consumers. This optimization problem, in turn,
takes the form of a complex market bid that can be directly
submitted to the electricity market. On the other hand, we have
problem (4), which is an estimation problem in a statistical
sense: it seeks to estimate the parameters of problem (1), that
is, the parameters defining the complex market bid, by using
the sum of the weighted absolute values of residuals as the
loss function to be minimized. This problem takes the form
of a bilevel programming problem.
Finally, it is worth pointing out that there is no obstacle
to reformulating (3) as a least-squares estimation problem
by the use of the L2-norm to quantify the prediction error.
However, the minimization of the absolute value of residuals
(i.e., the minimization of the L1-norm of the estimation error)
allows interpreting the objective function of (3) as an energy
mismatch (to be traded on a market closer to real time), while
keeping the estimation problem linear.
IV. SOLUTION MET HO D
The estimation problem (4) is non-linear due to the com-
plementarity constraints of the KKT conditions of the lower-
level problem (2). There are several ways of dealing with these
constraints, for example, by using a non-linear solver [23], by
recasting them in the form of disjunctive constraints [24], or by
using SOS1 variables [25]. In any case, problem (4) is NP-hard
to solve and the computational time grows exponentially with
the number of complementarity constraints. Our numerical
experiments showed that, for realistic applications involving
multiple time periods and/or numerous features, none of these
solution methods were able to provide a good solution to
problem (4) in a reasonable amount of time. To tackle this
problem in an effective manner, we propose the following two-
step solution strategy, inspired by [19]:
Step 1: Solve a linear relaxation of the mathematical
program with equilibrium constraints (4) by penalizing viola-
tions of the complementarity constraints.
Step 2: Fix the parameters defining the constraints of
the lower-level problem (1), i.e., ru, rd, P , P , αa
i, αd
i, αP
iand
αP
i, at the values estimated in Step 1. Then, recompute the
parameters defining the utility function, ab,t and αa
d. To this
end, we make use of the primal-dual reformulation of the
price-response model (1) [15].
Both steps are further described in the subsections below.
Note that the proposed solution method is a heuristic in the
sense that it does not solve the bilevel programming problem
(4) to optimality. However, data can be used to calibrate it
(through the penalty parameter L) to minimize the out-of-
sample prediction error. For the problem at hand, this is clearly
more important than finding the optimal solution to (4), see
Section IV-C for further details.
A. Penalty Method
The so-called penalty method is a convex (linear) relaxation
of a mathematical programming problem with equilibrium
constraints, whereby the complementarity conditions of the
lower-level problem, that is, problem (1), are moved to the
objective function (4a) of the upper-level problem. Thus,
we penalize the sum of the dual variables of the inequality
constraints of problem (1) and their slacks, where the slack
of a “≤”-constraint is defined as the difference between
its right-hand and left-hand sides, in such a way that the
slack is always nonnegative. For example, the slack of the
constraint relative to the maximum pick-up rate (1b) is defined
as st=ru
t−Pt−Pb∈B xb,t +Pt−1+Pb∈B xb,t−1, and
analogously for the rest of the constraints of the lower-level
problem.
The penalization can neither ensure that the complemen-
tarity constraints are satisfied, nor that the optimal solution
of the inverse problem is achieved. Instead, with the penalty
method, we obtain an approximate solution. In the case study
of Section V, nonetheless, we show that this solution performs
notably well.
5
After relaxing the complementarity constraints (2d)–(2g),
the objective function of the estimation problem writes as:
Minimize
ΩX
t∈T
wt(e+
t+e−
t)+
LX
b∈B
t∈T
wtψP
b,t +ψP
b,t +Pt−Pt
B+
X
t∈T−1
wtλu
t+λd
t+ru
t+rd
t(5a)
with the variables being Ω={xt, θt, e+
t, e−
t, ψP
t, ψP
t, λu
t,
λd
t,}, subject to the following constraints:
(4b) −(4d),(1b) −(1e),(2a) −(2c) (5b)
λu
t, λd
t≥0t∈ T−1(5c)
ψb,t, ψb,t ≥0b∈ B, t ∈ T .(5d)
The objective function (5a) of the relaxed estimation prob-
lem is composed of two terms. The first term represents the
weighted sum of the absolute values of the deviations of the
estimated consumption from the measured one. The second
term, which is multiplied by the penalty term L, is the sum of
the dual variables of the constraints of the consumers’ price-
response problem plus their slacks. Note that summing up
the slacks of the constraints of the consumers’ price-response
problem eventually boils down to summing up the right-hand
sides of such constraints. The weights of the estimation errors
(wt) also multiply the penalization terms. Thus, the model
weights violations of the complementarity constraints in the
same way as the estimations errors are weighted.
Objective function (5a) is subject to the auxiliary constraints
modeling the absolute value of estimation errors (4b)–(4c); the
upper-level-problem constraints imposing monotonically de-
creasing utility blocks (4d); and the primal and dual feasibility
constraints of the lower-level problem, (1b)–(1e), (2a)–(2c),
and (5c)–(5d).
The penalty parameter Lshould be tuned carefully. We use
cross-validation to this aim, as described in the case study; we
refer to Section V for further details.
Finding the optimal solution to problem (5) is computation-
ally cheap, because it is a linear programming problem. On
the other hand, the optimal solution to this problem might be
significantly different from the one that we are actually looking
for, which is the optimal solution to the original estimation
problem (4). Furthermore, the solution to (5) depends on the
user-tuned penalization parameter L, which is given as an
input and needs to be decided beforehand.
B. Refining the Utility Function
In this subsection, we elaborate on the second step of
the strategy we employ to estimate the parameters of the
market bid that best captures the price-response of the cluster
of loads. Recall that this strategy has been briefly outlined
in the introduction of Section IV. The ultimate purpose of
this additional step is to re-estimate or refine the parameters
characterizing the utility function of the consumers’ price-
response model (1), namely, a0
band the coefficients αa
i. In
plain words, we want to improve the estimation of these
parameters with respect to the values that are directly obtained
from the relaxed estimation problem (5). With this aim in
mind, we fix the parameters defining the constraints of the
cluster’s price-response problem (1) to the values estimated in
Step 1, that is, to the values obtained by solving the relaxed
estimation problem (5). Therefore, the bounds Pt, P tand the
maximum pick-up and drop-off rates ru
t, rd
tare now treated as
given parameters in this step. Consequently, the only upper-
level variables that enter the lower-level problem (1), namely,
the intersects a0
bof the various blocks defining the utility
function and the linear coefficients αa
i, appear in the objective
function of problem (1). This will allow us to formulate the
utility-refining problem as a linear programming problem.
Indeed, consider the primal-dual optimality conditions of
the consumers’ price-response model (1), that is, the primal
and dual feasibility constraints and the strong duality condi-
tion. These conditions are also necessary and sufficient for
optimality due to the linear nature of this model.
We determine the (possibly approximate) block-wise repre-
sentation of the measured consumption at time t,xmeas
t, which
we denote by Pb∈B xm0
b,t and is given as a sum of Bblocks
of size Pt−Pt
Beach. In particular, we define Pb∈B xmeas0
b,t as
follows:
X
b∈B
xm0
b,t =
Ptif xmeas
t> P t, t ∈ T
xmeas
tif Pt≤xmeas
t≤Pt, t ∈ T
Ptif xmeas
t< P t, t ∈ T .
where each xm0
b,t is determined such that the blocks with higher
utility are filled first. Now we replace xtin the primal-dual
reformulation of (1) with Pb∈B xm0
b,t . Consequently, the primal
feasibility constraints are ineffective and can be dropped.
Once xthas been replaced with Pb∈B xm0
b,t in the primal-
dual reformulation of (1) and the primal feasibility constraints
have been dropped, we solve an optimization problem (with
the utility parameters aband αa
ias decision variables) that
aims to minimize the weighted duality gap, as in [15]. For
every time period tin the training data set, we obtain a
contribution (t) to the total duality gap (Pt∈T t), defined
as the difference between the dual objective function value
at time tminus the primal objective function value at time
t. This allows us to find close-to-optimal solutions for the
consumers’ price-response model (1). Thus, in the case when
the duality gap is equal to zero, the measured consumption,
if feasible, would be optimal in (1). In the case when the
duality gap is greater than zero, the measured consumption
would not be optimal. Intuitively, we attempt to find values
for the parameters defining the block-wise utility function such
that the measured consumption is as optimal as possible for
problem (1).
Hence, the utility-refining problem consists in minimizing
the sum of weighted duality gaps
Minimize
ab,t,λu
t,λd
t,
ψP
t,ψP
t,ψb,t,ψ b,t,tX
t∈T
wtt.(6a)
6
Note that we assign different weights to the duality gaps
accrued in different time periods, in a way analogous to what
we do with the absolute value of residuals in (3). Objective
function (6a) is subject to
X
b∈B
ab,1xm0
b,1−p1X
b∈B
xm0
b,1+1=X
b∈B P1−P1
Bψb,1(6b)
X
b∈B
ab,txm0
b,t −ptX
b∈B
xm0
b,t +t=X
b∈B Pt−Pt
Bψb,t +
ru
t−Pt+Pt−1λu
t+rd
t+Pt−Pt−1λd
tt∈ T−1(6c)
(2a) −(2c) (6d)
ab,t ≥ab+1,t t∈ T (6e)
λu
t, λd
t≥0t∈ T−1(6f)
ψP
t, ψP
t, ψb,t , ψb,t ≥0t∈ T (6g)
The set of constraints (6c) constitutes the relaxed strong
duality conditions, which express that the objective function
of the original problem at time t, previously formulated in
Equation (1), plus the duality gap at time t, denoted by t,
must be equal to the objective function of its dual problem
also at time t. Equation (6b) works similarly, but for t= 1.
The constraints relative to the dual of the original problem are
grouped in (6d). Constraint (6e) requires that the estimated
utility be monotonically decreasing. Finally, constraints (6f)
and (6g) impose the non-negative character of dual variables.
C. Statistical Learning Interpretation
The proposed method to solve the bilevel programming
problem (4) is a heuristic in the sense that it is not guaranteed
to provide the optimal solution to (4), that is, it may not deliver
the parameters of the market bid that minimize the sum of the
weighted absolute values of residuals (4a). However, objective
function (4a) measures the in-sample prediction error and it is
well known, from the theory of statistical learning [26], that
minimizing the prediction error in-sample is not equivalent
to minimizing it out-of-sample. Consequently, the market bid
that is the optimal solution to the bilevel program (4), i.e., that
minimizes the in-sample prediction error, as given in (4a), is
not necessarily the one performing best in practice. In fact,
one could arbitrarily decrease the in-sample prediction error
down to zero, for example, by enlarging the parameter space
defining the market bid in order to overfit the data, while the
out-of-sample prediction error would dramatically increase as
a result. Our aim must be, therefore, to reduce the out-of-
sample error as much as possible. In this vein, the solution
method that we propose shows two major advantages over
solving the bilevel program (4) to optimality, namely:
1) It runs swiftly as we indicate later on in Section V and
can even be used for real-time trading and very short-
term forecasting. In contrast, finding the optimal solution
to (4) becomes rapidly computationally intractable for
sizes of the data sample acceptable to guarantee a proper
estimation of the market bid, that is, to avoid overfitting.
2) Besides its good computational properties, the relaxed
problem (5) is parameterized on the penalty factor L,
which is to be tuned by the user. Statistically speaking,
this provides our solution approach with a degree of
freedom that directly solving (4) does not have. Indeed,
we can let the data decide which value of the penalty Lis
the best, that is, which value of Lminimizes the out-of-
sample prediction error. To compute a proxy of the out-of-
sample prediction error, we conduct a thorough validation
analysis [26], which essentially consists in recreating the
use of our approach in practice for several values of L,
from among which we pick up the one that returns the
highest prediction performance. Furthermore, note that
both the tuning of the penalty Land the consequent re-
estimation of the market-bid parameters can be conducted
offline, as new information becomes available and as soon
as we perceive a statistically significant deterioration of
the prediction performance of our approach.
In the case study of Section V, we demonstrate the ef-
fectiveness of the proposed solution method by evaluating
its performance out-of-sample and comparing it against other
solution approaches over the same data set from a real-life
experiment.
V. CA SE ST UDY
The proposed methodology to estimate the market bid
that best captures the price-response of a pool of flexible
consumers is tested using data from a real-life case study.
The data relates to the Olympic Peninsula experiment, which
took place in Washington and Oregon states between May
2006 and March 2007 [20]. The electricity price was sent
out every fifteen minutes to 27 households that participated in
the experiment. The price-sensitive controllers and thermostats
installed in each house decided when to turn on and off
the appliances, based on the price and on the house owner’s
preferences.
For the case study, we use hourly measurements of load
consumption, broadcast price, and observed weather variables,
specifically, outside temperature, solar irradiance, wind speed,
humidity, dew point and wind direction. Moreover, we include
0/1 feature variables to indicate the hour of the day, with one
binary variable per hour (from 0 to 23), and one per day of
the week (from 0 to 6). A sample of the dataset is shown in
Figure 1, where the load is plotted in the upper plot, the price
in the middle plot, and the load versus the outside temperature
and the dew point in the bottom plots. The lines depicted in
the bottom plots represent the linear relationship between the
pairs of variables, and these are negative in both cases. The
high variability in the price is also noteworthy: from the 1st to
the 8th of December, the standard deviation of the price is 5.6
times higher than during the rest of the month ($67.9/MWh
versus $12.03/MWh).
On average, when using hourly data from the previous 3
months, i.e., 2016 samples, and a total of 37 features per
sample, the time for the whole estimation process takes 287
seconds to solve, with a standard deviation of 22 seconds,
on a personal Linux-based machine with 4 cores clocking at
2.90GHz and 6 GB of RAM. R and CPLEX 12.3 under GAMS
are used to process the data and solve the optimization models.
7
50 100 150 200
Price, $
Load in kW
50 100 150 200 250
Price in USD per MWh
Dec 04 Dec 06 Dec 08 Dec 10 Dec 12 Dec 14 Dec 16
●
●●●
●
●
●
●
●
●●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●
●●●●
●●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●●●●
●●●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●
●●
●●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●●●●●●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●●●●●
●
●
●
●
●
●
−2 0 2 4 6 8 10
50 150
Temperature, °C
Load in kW
Temperature Outside in °C
●●●
●
●
●●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●●●●●
●
●
●
●
●
●
●
●
●
●●
●●
●
●
●
●
●
●
●
●●●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●●●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●●
●
●
●
●●
●
●●
●
●
●●●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●
●●
●
●●
●
●
●
●
●
●
−6 −4 −2 0 2 4 6
50 150
Dew point, °C
Load in kW
Dew Point in °C
Fig. 1. The upper and the middle plot show the load and the price,
respectively. The bottom plots represent the load in the vertical axis versus
the outside temperature and the dew point, on the left and on the right,
respectively. The data shown spans from the 4th to the 18th of December.
These running times depend on the number of data points and
on the data itself. We conclude that the running time makes this
methodology attractive for bidding on short-term electricity
markets.
Furthermore, in practice, we have parallelized a great deal
of the code using the High-Performance-Computing facility at
the Technical University of Denmark [27], achieving solution
times in the order of seconds, so that the proposed solution al-
gorithm can even be used to bid in current real-time/balancing
markets.
Also, parallel computing proves to be specially useful to
tune the penalty parameter Lthrough cross-validation. In this
regard, it is important to stress that both the value of Land the
bid parameters as a function of the features can be periodically
recomputed offline (for example, every day, or every week,
or every month) to capture potential changes in the pool
of consumers that may eventually deteriorate the prediction
performance of our method.
A. Benchmark Models
To test the quality of the market bid estimated by the
proposed methodology, we quantify and assess the extent
to which such a bid is able to predict the consumption of
the cluster of price-responsive loads. For the evaluation, we
compare two versions of the inverse optimization scheme
proposed in this paper with the Auto-Regressive model with
eXogenous inputs (ARX) described in [5]. Note that this time
series model was also applied by [5] to the same data set of
the Olympic Peninsula project. All in all, we benchmark three
different models:
ARX, which stands for Auto-Regressive model with eX-
ogenous inputs [28]. This is the type of prediction model
used in [4] and [5]. The consumption xtis modeled as a
linear combination of past values of consumption up to lag
n,Xt−n={xt, . . . , xt−n}, and other explanatory variables
Zt={Zt, . . . , Zt−n}. In mathematical terms, an ARX model
can be expressed as xt=ϑxXt−n+ϑzZt+t,with
t∼N(0,σ2)and σ2is the variance.
Simple Inv This benchmark model consists in the utility-
refining problem presented in Section IV-B, where the param-
eters of maximum pick-up and drop-off rates and consumption
limits are computed from past observed values of consumption
in a simple manner: we set the maximum pick-up and drop-
off rates to the maximum values taken on by these parameters
during the last seven days of observed data. All the features
are used to explain the variability in the block-wise marginal
utility function of the pool of price-responsive consumers:
outside temperature, solar radiation, wind speed, humidity,
dew point, pressure, and hour and week-day indicators. For
this model, we use B=12 blocks of utility. This benchmark
is inspired from the more simplified inverse optimization
scheme presented in [16] and [15] (note, however, that neither
[16], nor [15] consider the possibility of leveraging auxiliary
information, i.e., features, to better explain the data, unlike we
do for the problem at hand).
Inv This corresponds to the inverse optimization scheme
with features that we propose, which runs following the two-
step estimation procedure described in Section IV with B=12
blocks of utility. Here we only use the outside temperature
and hourly indicator variables as features. We re-parametrize
weights wtwith respect to a single parameter, called forgetting
factor, and denoted as E≥0, in the following manner:
wt=gaptt
TEfor t∈ T and Tbeing the total number
of periods. The variable gap indicates whether the observation
was correctly measured (gap = 1) or not (gap = 0). Parameter
Eindicates how rapidly the weight drops (how rapidly the
model forgets). When E= 0, the weight of the observations
is either 1 or 0 depending on the variable gap. As Eincreases,
the recent observations weight comparatively more than the
old ones.
B. Validation of the Model and Performance in December
In this subsection we validate the benchmarked models and
assess their performance during the test month of December
2006. As previously mentioned, the evaluation and comparison
of the different benchmarked models is conducted in terms
of prediction errors, and not in monetary values (e.g., in
the form of market revenues). This relieves us of having to
arbitrarily assume a particular market organization behind the
Olympic Peninsula experiment and a particular strategy for
bidding into the market based on the ARX model that we use
for benchmarking. Furthermore, a well-functioning electricity
market should not reward prediction errors, that is, energy
imbalances. In fact, a number of electricity markets throughout
the world explicitly penalize prediction errors through the use
of a two-price balancing settlement (see, for instance, [29] for
further information on this).
8
To predict the aggregated consumption of the pool of
flexible loads, we need good forecasts of the electricity price
and the features, as these work as explanatory variables in
all the considered models. For the sake of simplicity, we
use the actual values of the electricity price and the features
that were historically recorded as such good forecasts. Since
this simplification applies to all the benchmarked models, the
analysis and comparison that follow is fair.
It is worth noticing, though, that the proposed methodology
need not a prediction of the electricity price when used for
bidding in the market and not for predicting the aggregated
consumption of a cluster of loads. This is so because the
market bid expresses the desired consumption of the pool of
loads for any price that clears the market. The same cannot
be said, however, for prediction models of the type of ARX,
which would need to be used in combination with extra
tools, no matter how simple they could be, for predicting the
electricity price and for optimizing under uncertainty in order
to generate a market bid.
There are two parameters that need to be chosen before
testing the models: the penalty parameter Land the forgetting
factor E. We seek a combination of parameters such that the
prediction error is minimized. We achieve this by validating
the models with past data, in a rolling-horizon manner, and
with different combinations of the parameters Land E. The
results are shown in Figure 2. The MAPE is shown on the
y-axis against the penalty Lin the x-axis, with the different
lines corresponding to different values of the forgetting factor
E. From this plot, it can be seen that a forgetting factor of
E= 1 or E= 2 yields a better performance than when there
is no forgetting factor at all (E= 0), or when this is too high
(E≥3). We conclude that selecting L= 0.1and E= 1
results in the best performance of the model, in terms of the
MAPE.
0.18 0.22 0.26
L
Average MAPE
1e−04 0.01 0.02 0.05 0.1 0.15 0.2 0.3 0.5
●●●●●●
●
●
●
●
E
0 1 2 3 5
Fig. 2. Results from the validation of the input parameters Land E, to be
used during December.
Once the different models have been validated, we proceed
to test them. For this purpose, we first set the cross-validated
input parameters to L= 0.1and E= 1, and then, predict
the load for the next day of operation in a rolling-horizon
manner. In order to mimic a real-life usage of these models,
we estimate the parameters of the bid on every day of the
test period at 12:00 using historical values from three months
in the past. Then, as if the market were cleared, we input
the price of the day-ahead market (13 to 36 hours ahead) in
the consumers’ price-response model, obtaining a forecast of
the consumption. Finally, we compare the predicted versus
the actual realized consumption and move the rolling-horizon
window to the next day repeating the process for the rest of
the test period. Similarly, the parameters of the ARX model
are re-estimated every day at 12:00, and predictions are made
for 13 to 36 hours ahead.
Results for a sample of consecutive days, from the 10th
to the 13th of December, are shown in Figure 3. The actual
load is displayed in a continuous solid line, while the load
predictions from the various benchmarked models are shown
with different types of markers. First, note that the Simple
Inv model is clearly under-performing compared to the other
methodologies, in terms of prediction accuracy. Recall that, in
this model, the maximum and minimum load consumptions,
together with the maximum pick-up and drop-off rates, are
estimated from historical values and assumed to remain con-
stant along the day, independently of the external variables (the
features). This basically leaves the utility alone to model the
price-response of the pool of houses, which, judging from the
results, is not enough. The ARX model is able to follow the
load pattern to a certain extent. Nevertheless, it is not able to
capture the sudden decreases in the load during the night time
or during the peak hours in the morning. The proposed model
(Inv) features a considerably much better performance. It is
able to follow the consumption pattern with good accuracy.
40 60 80 100 140
kW
●●●●●●
●
●●
●
●●●●●●●●
●●●●●●●●●●
●
●
●
●●
●
●●●●●●●●
●●●●●●
●●●●
●
●
●
●●
●
●●●●●
●●●
●●●●●●
●
00:00 10th Dec
12:00 10th Dec
00:00 11th Dec
12:00 11th Dec
00:00 12th Dec
12:00 12th Dec
00:00 13th Dec
●Actual load
ARX
Simple Inv
Inv
Fig. 3. Load forecasts issued by the benchmark models, and actual load, for
the period between the 10th and the 13th of December.
TABLE I
PERFORMANCE MEASURES FOR THE THREE BENCHMARKED MODELS
DURING DECEMBER.
MAE RMSE MAPE
ARX 22.176 27.501 0.2750
Simple Inv 44.437 54.576 0.58581
Inv 17.318 23.026 0.1899
The performance of each of the benchmarked models during
the whole month of December is summarized in Table I.
The first column shows the Mean Absolute Error (MAE),
the second column provides the Root Mean Square Error
(RMSE), and the third column collects the Mean Absolute
Percentage Error (MAPE). The three performance metrics
lead to the same conclusions: that the price-response models
we propose, i.e., Inv, perform better than the ARX model
and the Simple Inv model. The results collated in Table I
also yield an interesting conclusion: that the electricity price
is not the only driver of the consumption of the pool of
9
houses and, therefore, is not explanatory enough to predict the
latter alone. We conclude this after seeing the performance
of the Simp Inv, which is not able to follow the load just
by modeling the price-consumption relationship by means of
a marginal utility function. The performance is remarkably
enhanced when proper estimations of the maximum pick-up
and drop-off rates and the consumptions bounds as functions
of the features are employed.
The estimated block-wise marginal utility function, aver-
aged for the 24 hours of the day, is shown in the left plot
of Figure 4 for the Inv model. The solid line corresponds
to the 4th of December, when the price was relatively high
(middle plot), as was the aggregated consumption of the pool
of houses (right plot). The dashed line corresponds to the 11th
of December and shows that the estimated marginal utility is
lower, as is the price on that day.
60 80 100 120
0.05 0.10 0.15 0.20
Total load in kW
Estimated mean marginal utility in $/kWh
0 5 10 15 20
0.05 0.10 0.15 0.20
Hour of the day
Price in $/kWh
0 5 10 15 20
40 60 80 100 120 140 160 180
Hour of the day
Total load in kW
Day
2006−12−04 2006−12−11
Fig. 4. Averaged estimated block-wise marginal utility function for the Inv
model (left panel), price in $/kWh (middle panel), and load in kW (right
panel). The solid lines represent data relative to the 4th of December. Dashed
lines represent data relative to the 11th of December.
C. Performance During September and March
In this section, we summarize the performance of the
benchmarked models during September 2006 and March 2007.
In Table II, summary statistics for the predictions are
provided for September (left side) and March (right side). The
conclusions remain similar as the ones drawn for the month
of December. The Inv methodology consistently achieves the
best performance during these two months as well.
TABLE II
PERFORMANCE MEASURES FOR THE THREE BENCHMARKED MODELS
September March
MAE RMSE MAPE MAE RMSE MAPE
ARX 7.649 9.829 0.2350 17.439 23.395 0.2509
Simple Inv 14.263 17.8 0.4945 44.687 54.616 0.8365
Inv 5.719 8.582 0.1462 12.652 16.776 0.1952
By means of cross-validation [26], we find that the user-
tuned parameters yielding the best performance vary over the
year. For September, the best combination is L= 0.3,E= 0,
while for March it is L= 0.3,E= 1.
The optimized penalization parameter Lturns out to be
higher in September and March than in December. This
penalization parameter is highly related to the actual flexibility
featured by the pool of houses. Indeed, for a high enough value
of the penalty (say L≥0.4for this case study), violating the
complementarity conditions associated with the consumers’
price-response model (1) is relatively highly penalized. Hence,
at the optimum, the slacks of the complementarity constraints
in the relaxed estimation problem (5) will be zero or close
to zero. When this happens, it holds at the optimum that
ru
t=−rd
tand Pt=Pt. The resulting model is, there-
fore, equivalent to a linear model of the features, fit by
least weighted absolute errors. When the best performance
is obtained for a high value of L, it means that the pool of
houses does not respond so much to changes in the price.
On the other hand, as the best value for the penalization
parameter Ldecreases towards zero, the pool becomes more
price-responsive: the maximum pick-up and drop-off rates and
the consumption limits leave more room for the aggregated
load to change depending on the price.
Because the penalization parameter is the lowest during
December, we conclude that more flexibility is observed
during this month than during September or March. The reason
could be that December is the coldest of the months studied,
with a recorded temperature that is on average 9.4◦C lower,
and it is at times of cold whether when the electric water
heater is used the most.
VI. SUMMARY AND CONCLUSIONS
We consider the market-bidding problem of a pool of
price-responsive consumers. These consumers are, therefore,
able to react to the electricity price, e.g., by shifting their
consumption from high-price hours to lower-price hours. The
total amount of electricity consumed by the aggregation has to
be purchased in the electricity market, for which the aggregator
or the retailer is required to place a bid into such a market.
Traditionally, this bid would simply be a forecast of the load,
since the load has commonly behaved inelastically. However,
in this paper, we propose to capture the price-response of
the pool of flexible loads through a more complex, but still
quite common market bid that consists of a stepwise marginal
utility function, maximum load pick-up and drop-off rates,
and maximum and minimum power consumption, in a manner
analogous to the energy offers made by power producers.
We propose an original approach to estimate the parameters
of the bid based on inverse optimization and bi-level program-
ming. Furthermore, we use auxiliary variables to better explain
the parameters of the bid. The resulting non-linear problem
is relaxed to a linear one, the solution of which depends on
a penalization parameter. This parameter is chosen by cross-
validation, proving to be adequate from a practical point of
view.
For the case study, we used data from the Olympic Peninsula
project to asses the performance of the proposed methodology.
We have shown that the estimated bid successfully models
the price-response of the pool of houses, in such a way that
the mean absolute percentage error incurred when using the
estimated market bid for predicting the consumption of the
pool of houses is kept in between 14% and 22% for all the
months of the test period.
10
We envision two possible avenues for improving the pro-
posed methodology. The first one is to better exploit the
information contained in a large dataset by allowing for
non-linear dependencies between the market-bid parameters
and the features. This could be achieved, for example, by
the use of B-splines. The second one has to do with the
development of efficient solution algorithms capable of solving
the exact estimation problem within a reasonable amount of
time, instead of the relaxed one. This could potentially be
accomplished by decomposition and parallel computation.
ACKNOWLEDGMENT
The authors would like to thank Pierre Julien Trombe for
support regarding time-series models.
REFERENCES
[1] R. Halvgaard, N. Poulsen, H. Madsen, and J. Jorgensen, “Economic
model predictive control for building climate control in a smart grid,”
Innovative Smart Grid Technologies (ISGT), 2012 IEEE PES, pp. 1–6,
2012.
[2] A. J. Conejo, J. M. Morales, and L. Baringo, “Real-time demand
response model,” IEEE Transactions on Smart Grid, vol. 1, no. 3, pp.
236–242, 2010.
[3] A.-H. Mohsenian-Rad and A. Leon-Garcia, “Optimal residential load
control with price prediction in real-time electricity pricing environ-
ments,” IEEE Transactions on Smart Grid, vol. 1, no. 2, pp. 120–133,
2010.
[4] G. Dorini, P. Pinson, and H. Madsen, “Chance-constrained optimization
of demand response to price signals,” IEEE Transactions on Smart Grid,
no. 99, pp. 1–1, 2013.
[5] O. Corradi, H. Ochsenfeld, H. Madsen, and P. Pinson, “Controlling
electricity consumption by forecasting its response to varying prices,”
IEEE Transactions on Power Systems, vol. 28, no. 1, pp. 421–429, 2013.
[6] M. Zugno, J. M. Morales, P. Pinson, and H. Madsen, “A bilevel model
for electricity retailers’ participation in a demand response market
environment,” Energy Economics, vol. 36, pp. 182 – 197, Mar. 2013.
[7] C. Chen, S. Kishore, and L. Snyder, “An innovative RTP-based res-
idential power scheduling scheme for smart grids,” in 2011 IEEE
International Conference on Acoustics, Speech and Signal Processing
(ICASSP), 2011, pp. 5956–5959.
[8] F.-L. Meng and X.-J. Zeng, “A Stackelberg game-theoretic approach to
optimal real-time pricing for the smart grid,” Soft Computing, pp. 1–16,
2013.
[9] G. E. Asimakopoulou, A. G. Vlachos and N. D. Hatziargyriou, “Hierar-
chical decision making for aggregated energy management of distributed
resources,” Power Systems, IEEE Transactions on, vol. 30, no. 6, pp.
3255–3264, Nov. 2015.
[10] H. Mohsenian-Rad, “Optimal demand bidding for time-shiftable loads,”
Power Systems, IEEE Transactions on, vol. 30, no. 2, pp. 939–951,
March 2015.
[11] M. Gonzalez Vaya and G. Andersson, “Optimal bidding strategy of a
plug-in electric vehicle aggregator in day-ahead electricity markets under
uncertainty,” Power Systems, IEEE Transactions on, vol. 30, no. 5, pp.
2375–2385, Sep. 2015.
[12] S. Li, W. Zhang, J. Lian, and K. Kalsi, “Market-based coordination of
thermostatically controlled loads—Part I: A mechanism design formu-
lation,” Power Systems, IEEE Transactions on, in press, 2015.
[13] L. Ratliff, R. Dong, H. Ohlsson and S. S. Sastry, “Incentive Design and
Utility Learning via Energy Disaggregation,” arXiv:1312.1394, 2015.
[14] R. K. Ahuja and J. B. Orlin, “Inverse optimization,” Operations Re-
search, vol. 49, pp. 771–783, 2001.
[15] T. C. Chan, T. Craig, T. Lee, and M. B. Sharpe, “Generalized inverse
multiobjective optimization with application to cancer therapy,” Opera-
tions Research, 2014.
[16] A. Keshavarz, Y. Wang, and S. Boyd, “Imputing a convex objective func-
tion,” in Intelligent Control (ISIC), 2011 IEEE International Symposium
on, Sept 2011, pp. 613–619.
[17] D. Bertsimas, V. Gupta and I. Paschalidis “Data-Driven Estimation in
Equilibrium using Inverse Optimization”, Mathematical Programming,
2015.
[18] A. Aswani, Z.-J. M. Shen and A. Siddiq, “Inverse Optimization with
Noisy Data”, arXiv:1507.03266, 2015.
[19] S. Siddiqui and S. Gabriel, “An SOS1-based approach for solving mpecs
with a natural gas market application,” Networks and Spatial Economics,
vol. 13, no. 2, pp. 205–227, 2013.
[20] E. D. J. Hammerstrom, “Pacific Northwest gridwise testbed demon-
stration projects. Part I. Olympic Peninsula project,” Pacific Northwest
National Laboratory, October 2007.
[21] D. Luenberger and Y. Ye, Linear and Nonlinear Programming, ser.
International Series in Operations Research & Management Science.
Springer, 2008.
[22] A. Ben-Tal, L. El Ghaoui, and A. Nemirovski, Robust optimization.
Princeton University Press, 2009.
[23] M. C. Ferris and T. S. Munson, “Complementarity problems in gams and
the path solver,” Journal of Economic Dynamics and Control, vol. 24,
no. 2, pp. 165–188, 2000.
[24] J. Fortuny-Amat and B. McCarl, “A representation and economic
interpretation of a two-level programming problem,” Journal of the
operational Research Society, pp. 783–792, 1981.
[25] E. M. L. Beale and J. A. Tomlin, “Special facilities in a general math-
ematical programming system for non-convex problems using ordered
sets of variables,” Operations Research, vol. 69, pp. 447–454, 1970.
[26] T. Hastie, R. Tibshirani, J. Friedman and J. Franklin, The elements
of statistical learning: data mining, inference and prediction, The
Mathematical Intelligencer, 27(2), 83-85, 2005.
[27] DTU Computing Center http://www.hpc.dtu.dk/.
[28] H. Madsen, Time Series Analysis. Chapman & Hall, 2007.
[29] A. Skajaa, K. Edlund and J. M. Morales, “Intraday trading of wind
energy,” Power Systems, IEEE Transactions on, vol. 30, no. 6, pp. 3181–
3189, Nov. 2015.
APPENDIX
Next we show how to formulate robust constraints to ensure
that the estimated minimum consumption be always equal to or
lower than the estimated maximum consumption. At all times,
and for all plausible realizations of the external variables, we
want to make sure that:
P0+X
i∈I
αP
iZi,t ≤P0+X
i∈I
αP
iZi,t, t ∈ T ,∀Zi,t .(7)
If (7) is not fulfilled, problem (1) is infeasible (and the
market bid does not make sense). Assuming we know the
range of possible values of the features, i.e., Zi,t ∈[Zi, Zi],
(7) can be rewritten as:
P0−P0+ Maximize
Z0
i,t
s.t.Zi≤Z0
i,t≤Zi
i∈I
(X
i∈I
(αP
i−αP
i)Z0
i,t)≤0, t ∈ T .
(8)
Denote the dual variables of the upper and lower bounds
of Z0
i,t by φi,t and φi,t respectively. Then, the robust con-
straint (8) can be equivalently reformulated as:
P0−P0+X
i∈I
(φi,tZi−φi,t Zi)≤0t∈ T (9a)
φi,t −φi,t =αP
i−αP
ii∈ I, t ∈ T (9b)
φi,t, φi,t ≥0i∈ I, t ∈ T .(9c)
Following the same reasoning, one can obtain the set of
constraints that guarantees the non-negativity of the lower
bound and consistent maximum pick-up and drop-off rates.
We leave the exposition and explanation of these constraints
out of this paper for brevity.
