Content uploaded by Motab Almousa
Author content
All content in this area was uploaded by Motab Almousa on Feb 26, 2017
Content may be subject to copyright.
1
Abstract—This paper proposes a novel multi-agent collaborative
algorithm to unveil economic potentials of local collaborations
between generation companies’ (GenCos). We investigate the
potential of local collaborations to further increase GenCos’
profit after market clearing by the system operator. A multi-
agent based analytical target cascading (M-ATC), which is a
distributed optimization algorithm, is developed to accommodate
the need to model GenCos as self-interested agents. The main
novelty of M-ATC is twofold. It enables the agents to
simultaneously determine the amount of local energy exchange as
well as the optimal cost for energy transactions. In addition, a
novel dynamic cutting plane procedure is presented to convert
the distributed model to an agent-based model. The proposed
algorithm guarantees economic feasibility and information
privacy of the self-interested agents in a complete regulation-free
environment. The practicality and efficiency of the proposed
model is tested on a unit commitment problem.
Index Terms—Cooperative optimization, distributed solution
algorithm, multi-agent, electricity market.
INTRODUCTION
I.
N deregulated electricity markets, private generation
companies (GenCos) are considered as self-interested
agents in a competitive environment. Their role is to supply
the electricity for consumers. Each agent pursues a single
objective to maximize its own profit. The agents (in this paper
we interchangeability use the terms “GenCo” and “agent”)
send their bids to the independent system operation (ISO).
From the system perspective, ISO clears the market to
maximize social welfare. Locational marginal prices as well as
the power output of each agent are determined. Consider that
multiple GenCos share the same bus. This raises a possibility
for mutual collaborations between the agents to further
increase their benefits. Fig. 1 shows a possible system
structure in which multiple agents are located on the same bus.
After market clearing, the agents on a bus can collaborate to
locally reschedule power outputs of their units. Local
collaborations and any bus must not affect locational marginal
prices and the overall social welfare from the system vantage
point. This is, the collaborations between the agents do not
conflict with higher system’ merits.
On the other hand, GenCos are in a competitive
environment, where information exchange is crucial and
brings a barrier against any potential profit-based
collaboration. Thus, information privacy is a critical part of
the local collaboration of the participated agent. Revealing
characteristics of an agent can be utilized by others to develop
dominate biding strategies.
The authors are with the Division of Electrical and Computer Engineering,
Louisiana State University, Baton Rouge, LA 70803 USA, email:
malmou2@lsu.edu, kargarian@lsu.edu.
Fig. 1. A possible structure of multi-agents GenCos.
As a result, novel coordination strategies are required to
coordinate potential collaborations between GenCos.
In the past few years, researchers were successfully able to
develop powerful distributed algorithms to coordinate multi-
agents and distributed systems. In [1] and [2], the authors
proposed an algorithm to coordinate the multi-area unit
commitment problem in a distributed manner. The algorithm
is computationally efficient and provides the optimal solution
to the centralized problem while preserving information of the
agents. However, economic benefits of each agent have not
been addressed. To satisfy self-interest agents’ requirements, a
further step is needed to allocate the benefits that are obtained
through the agents’ collaborations. Authors in [3] tried to
resolve this problem by presenting propose an independent
system operator, called mini-ISO. However, the approach
contradicts the movement to deregulate the market and the
needs for information privacy. Another solution to this
problem is to use the concept of multi-agent systems (MAS) in
the electricity market as a complex system. Evolutionary and
artificial intelligent based methods have been used for
GenCos’ decision-making [4]. In [5] and [6], Q-Learning and
reinforcement learning approaches have been proposed to
solve the schedule the GenCos. However, learning-based
approaches are not robust enough to be relied on at practical
markets.
This paper proposes a novel distributed optimization
algorithm for multi-agent based decision-making for
scheduling the GenCos after clearing the market for additional
economic gain. The distributed algorithm is based on the
concept of analytical target cascading (however, our algorithm
is general and does not dependent on the choice of the
distributed optimization algorithm; one can use another
algorithm such as auxiliary problem principle). To the best of
our knowledge, this paper is the first to provide a
mathematical-based framework to consider economic
feasibility of self-interested GenCos after the market clearing
Multi-Agent Collaborative Model in Electrical
Energy Market: A Distributed-Based Approach
Motab Almousa, Student Member, IEEE, Amin Kargarian, Member, IEEE
I
2
by the operator. The contributions of the paper are
summarized as follow:
• A distributed-based economic feasibility cutting plane
algorithm is developed to convert a distributed
optimization algorithm to a distributed self-interested
agent-based algorithm.
• Hybrid transaction-electricity flow is considered for more
potential of collaboration and improvement of
convergence properties.
• No information is needed to exchange between agents
which is aligned with the information privacy in MAS.
A
GENT
-B
ASED
C
OLLABORATIVE
M
ODEL
U
SING A
II.
DISTRIBUTED
A
PPROACH
A. General Description and Mathematical Derivation of the
Proposed M-ATC
GenCos are considered as self-interested agents. Thus,
distributed optimization algorithms, such as ATC, need to take
into account the cost of energy transaction between the agents.
Otherwise, one agent might gain benefit while another losing.
In order to avoid this, we need to modify the traditional ATC
algorithm to adopt it with the concept of multi-agent systems.
We develop a multi-agent based ATC (M-ATC) in which each
GenCo is allowed to provide an upper-bound for its objective
function. We set the upper-bound to the GenCo’s power
scheduled by the ISO multiply by the locational marginal price
at the bus that the GenCo is located. To guarantee the
feasibility of the model as well as economic benefits for all
participated agents, transaction flows are considered. The
transaction flows enable GenCos to reach a cost allocation
scheme that is feasible to all agents. A positive transaction
means that a GenCo sends money to others, whereas a
negative transaction implies that the GenCo receives money
from others. Although multi-agent systems (MAS) problems
are classified as uncertain game-based problems, they need
statistical and learning-based approaches. The potential of
modeling them as distributed mathematical-based problems is
provided in this paper.
Claim: The proposed M-ATC algorithm is economically
feasible.
Proof: Consider the following optimization problem for
two agents and
(
) (1)
Given that
=(
,
,):≤,
,
,∈
, where M is a non-convex feasible set. The local variables
of agents A and B are indicated by
and
, and the shared
variables between them is denoted by, i.e.,
,
⊆ , and
∩
= . Optimization problem (1) can be rewritten as:
,
,
(
,
,)
(2)
..
∈ M1,
∈ M2, ∈ (M1 ⋂ M2).
Assuming agents A and B solve their problem independently,
the optimization problem will be as follow:
,
(
,
) +
,
(
,
)
..
+
=0 (3)
It is trivial that objective function (2) is always lower than or
equal to (3) for all feasible solutions
+
=0. Yet not
optimal . Thus
,
,
(
,
,) (4)
≤
,
(
,
) +
,
(
,
)
without any loss of generality, a slack variable, 1, is added
such that:
,
,
(
,
,)+1 (5)
=
,
(
,
) +
,
(
,
)
1 is always greater than or equal to zero. S1 represents the
potential savings of the agents because of their collaboration.
However, it is not decomposable to be modeled as an
independent saving of the agents. In a similar manner, but
with assuming a case of feasible collaboration and no
collaboration, we have:
,
(
,
) +
,
(
,
)
≤
(
) +
(
) (6)
Let us transform inequality constraint (6) to the following
equality constraints:
,
(
,
) +
,
(
,
)+2
=
(
) +
(
) (7)
2 is always greater than or equal to zero. By moving 2 to
the right-hand side, we have:
,
(
,
) +
,
(
,
)
=
(
) +
(
) −2 (8)
Since S2 is proven to be greater than or equal to zero after
collaboration and the right-hand side is separable, this proves
the overall economic feasibility of the collaboration. The slack
variable can be rewritten as 2=2+2. Since there is
no restriction on the sign for 2 and 2, we have:
(
) +
(
) −2
=
(
)−2 +
(
)
−2
(9)
We now derive the economic feasibility of the collaboration
between agents. The economic feasibility for agent A is
defined as
(
)−2
and for Agent B, it is
(10)
3
(
) −2
Two approaches can guarantee the potential of the
collaboration. First:
(
)−
,
(
,
)=1
and second:
(
)−
,
(
,
)=2
.. 1≥0, 2≥0, and
+
=0
(11)
It is straightforward to formulate 1≥0.
,
..
(
,
)=
≤
(
)
(12)
and in a similar manner for 2≥0. However, this approach
does not guarantee feasibility. There could not be any
+
=0 over the feasible region of the collaboration ∈
(M1 ⋂ M2). In addition, it is known that minimizing overall
cost is equivalent to maximizing the social welfare (slack
variable). We introduce two cuts, 1≥0 and 2≥0, to
cut the exiting feasible space of the optimization problem. To
overcome this cutting issue, we propose new variables that
represent contracted transactions
and
. The
proposed formulation becomes:
(
)−
,
(
,
)
=1+
and
(
)−
,
(
,
)
=2+
.. 1+
≥0, 2+
≥0,
and
+
=0
+
=0
(13)
Since for any feasible solution of multi-agents systems the loss
of one system is always less than or equal to the gain of the
other system,
and
guarantee feasibility of the
model. The ATC model includes two levels, subsystem
(agent) level and coordination level. After proving economic
feasibility of the proposed model, subsystems in ATC is
developed to be self-interested agents in the proposed M-ATC
formulation. Each objective function at the agent level is
modeled as:
,
+
−
+
−
..
(
,
)+
≤
≤
(
)
(14)
where
and
are the targets from the coordinator (or
master) problem, and
and
are penalty functions
for the distrusted algorithm convergence.
B. Derivation of M-ATC Model for Unit Commitment in
Electrical Energy Market
We apply the proposed algorithm to solve a multi-agent based
unit commitment problem.
Agents Level: Assume that an agent operates independently.
The optimization problem is to minimize cost of the agent to
its local constraints:
,
(
)
+
(15)
.. (
,
)≤0
(
,
)=0
Where
(⋅) is the generation cost function of unit . SUD
represents start up/down cost of the units. and are
respectively power output and unit status variables. and
denote the scheduling horizon and number of units. and
are compact representative of the UC inequality and
equality constraints, e.g., ramping, and minimum on/off time.
Now, let us incorporate energy and transaction flows
between the agents. The new optimization problem after
considering the flow interchange is rewritten as:
,
,
,℘
(
)
+
+
+℘
(16)
.. (
,
,
,℘
)≤0
(
,
,
,℘
)=0
where
and ℘
are the energy and contract amounts that
exchange between agent j and its neighboring agents at time t,
and ℘
is the amount of the transaction flow. After
considering the exchange with other agents, the local
optimization problem includes both local and public (or
shared) variables. The local variables are variables that
exclusively belong to only one agent, such as unit status and
energy dispatch . Any variable that depends on at least two
agents is classified as a public variable. Thus, energy flow
and contract ℘
are classified as public variables. The
contracts and energy flow are not restricted in sign, and thus
they can be received or provided. However, since these two
variables are dependent on other agents, they need to be
considered in all connected agents.
,
(
)
+
+
(
−
) +
(℘
−Φ
)
(17)
.. (
,
,
,℘
)≤0
4
(
,
,
,℘
)=0
In (17), penalty functions are considered to penalize the
deviation between agents. The concept is widely known in the
field of distributed optimization [2, 6, 8]. The terms
and
are the penalty functions. This formulation ideally
reaches the global optimal without considering the agents’
economic benefits. However, this is not the goal for the self-
interested agents. As agents are self-interested, the objective is
to minimize total cost locally. To address this issue, upper
bound cuts for the optimization problem is introduced to
modify the transaction-based unit commitment problem such
that:
1. A dynamic local cut given in (18) to minimize the actual
operating cost without considering the penalty functions.
2. In (19), a static upper bound is applied to the dynamic cut
presented to ensure the economic feasibility for the
agents.
(
)
+
+
+℘
≤
(18)
and,
≤
,
(
)
+
..
(
,
)≤0
(
,
)=0
.()
(19)
The static upper bound could be interpreted as the maximum
contract amount that the agent is willing to agree on as a result
of the model clearing. For more flexibility of the agents, a
dynamic function () can be considered. This function
represents the bidding strategy for GenCo. The agent’s local
optimization problem of is expressed as follow:
,
,
,℘
+
(
−
) +
(℘
−Φ
)
(20)
.. (
,
,
,℘
)≤0
(
,
,
,℘
)=0
(
)
+
+
+℘
≤
≤
,
(
)
+
..
(
,
)≤0
(
,
)=0
.()
Now, each agent formulation is completed. However,
and
Φ
are needed to be obtained from the coordinator level.
Coordinator Level: At this stage, each agent should provide
the coordinator with the solution of the public variables and
the contracted amount willing to pay/receive from/to other
agents. The process is implemented as follow. All agents’
responses are aggregated to the master coordinator. The
master coordinator considers the responses as constants in (21)
and (22).
.
..
....
.
=
.
.
....
.
(21)
℘
℘
.
℘
= ℘
℘
.
℘
(22)
Then, the master coordinator solves for new targets to achieve
convergence for all participated agents. The objective function
at coordinator level is:
,
−
+
Φ
−℘
..
=
∀,
Φ
=Φ
∀,
(23)
where
and Φ
are vector of the shared targets with all
neighboring agents. The penalty terms in (23) are:
=
−
+ ∘
−
(24)
= Φ
−℘
+ ∘Φ
−℘
(25)
where ∘ is Hadamard product. Parameters , , , and are
penalty multipliers that need to be updated after every iteration
as:
=
+(
)
(′
−′
) (26)
=
(27)
=
+(
)
(℘′
−℘′
) (28)
=
(29)
The updated values of
and Φ
will be sent to all agents for
the next iteration. The agents use the updated target and solve
their problems again. We follow this iterative process until the
algorithm converges. The solution algorithm is shown in
Algorithm 1.
C
ASE
S
TUDIES
III.
A unit Commitment problem for three GenCos is considered
to illustrate the effectiveness of the proposed model. The
motivation is to achieve an additional profit for each energy
provider after the market is cleared and power quantities are
awarded to GenCos. More profit is achieved by maximizing
the difference between the payment received from ISO and the
operation cost. Under the assumption that GenCos are
obligated to quantities disaggregated from ISO, solving the
UC problem is equivalent to the profit-based solution. Figure
5
2 shows the quantities disaggregated to each agent for 24 hour.
Characteristics of the units are provided in Table I. Min
up/down times and ramping constraints are considered. The
units’ cost functions are assumed to be linear.
Algorith
m
1 Contract-based UC for multi-agent system
1: Start all agents solve (15) in parallel and determine local upper
bound in (19)≤
2: Initialization
• Initialize penalty multipliers, , , and
• Initialize
,
, and ℘
3: While
−
AND ℘
−℘
> , n = n+1 do
4: Each agent solves (20) and determines the values of
,
,
, and℘
5: Send
and ℘
to the master coordinator
6: Aggregate the shared variables in the master coordinator
7: Solve (23) and propagate
and Φ
to all agents
8: update , , , and
9: end while
TABLE I
UNITS CHARACHTERISTIC
Sys./Unit 1 2 3 4 5 6 7 8 9 10
1
a 50 50 100 100 100 200 100 100 188 105
b 8 12 10 6 7 11 10 30 8 7
c 0.05 0.3 0.16 0.008 0.25
0.1 0.006 0.04 0.035
0.066
MD|MU 2|2 2|4 3|5 2|7 1|2 1|4 3|4 3|5 2|2 5|8
RD|RU 60 |50
80|
50
100|
150
100|
120
100|
120
80|
80
150|
120
100|
100
50|
150
120|
210
2
a 50 50 100 150 160 250 110 100 120 115
b 8 6 10 25 50 39 11 9 10 7
c 0.25 0.32 0.6 0.1 0.06
0.05 0.04 0.03 0.05 0.04
MD|MU 2|4 2|4 3|5 4|7 1|2 1|1 1|6 3|6 3|4 2|8
RD|RU
50|
50
50|
50
150|
150
200|
200
100|
100
80|
80
150|
150
150|
150
150|
150
300|
300
3
a 50 50 100 100 100 200 100 100 100 105
b 6 8 10 6 7 10 10 20 6 7
c 0.02 0.07 0.09 0.04 0.02
0.1 0.08 0.06 0.05 0.08
MD|MU 2|2 2|4 3|5 2|7 1|2 1|4 3|4 3|5 2|2 5|8
RD|RU
60|
50
80|
50
120|
150
120|
120
120|
120
80|
80
140|
120
120|
100
50|
150
80|
210
Each GenCo determines its upper limit and the algorithm is
initialized.
and ℘
are initialized at 10000 and
at 0,and
=10
. and are fixed at 1. The algorithm
converged after 22 iterations. The three GenCos reduce their
costs (or increase their benefits). Energy exchange responses
between agents 1 and 2 in, for instance, hour 8 is shown in
Fig. 3. A transaction flow of $1103.8 is to be paid to GenCo 2
by GenCo 1. GenCo 2 needs to pay $982.4 to GenCo 3.
GenCos 1, 2, and 3 have savings of $8237.7, $11076.3, and
$5762.1, respectively. Table II provides energy scheduling
cost for each GenCo.
C
ONCLUSION
IV.
In this paper, a multi-agent collaborative algorithm is
proposed to locally coordinate GenCos to gain additional
profits after market clearing. As shown in the results,
additional savings has been achieved for all participated
GenCos. Further savings can be achieved with more iteration.
However, the objective of this paper is to show the feasibility
of reaching savings for all participated agents using a
distributed optimization algorithm.
Fig. 2. 24-hr power quantities scheduled for GenCos.
Fig. 3. Flow responses of GenCo 1-2 and inverse of GenCo 2-1.
TABLE II
COST COMPARISON
System Cost ($)
Without Interchange Our Approach
1 455486 447248.3
2 658842 647765.7
3 404956 399193.9
R
EFERENCES
[1] A. Kargarian, Y. Fu, Z. Li. “Distributed security-constrained unit
commitment for large-scale power systems,” IEEE Transactions on
Power Systems, vol. 30, no.4 pp. 1925-19-36, 2015.
[2] A. Kargarian, Y. Fu and H. Wu, "Chance-Constrained System of
Systems Based Operation of Power Systems," IEEE Transactions on
Power Systems, vol. 31, no. 5, pp. 3404-3413, Sept. 2016.
[3] J. Li, y. liu and L. Wu, “Optimal Operation for Community Based Multi-
Party Microgrid in Grid-Connected and Islanded Modes,” IEEE
Transactions on Smart Grid, 2016.
[4] K. Dehghanpour, M. Nehrir, J. Sheppard and N. Kelly, “Agent-Based
Modeling in Electrical Energy Markets Using Dynamic Bayesian
Networks,” IEEE Transactions on Power Systems, vol. 31, no. 6, pp.
4744-4754, 2016.
[5] N.P. Yu, C.C. Liu, and J. Price, “Evaluation of Market Rules Using a
Multi-Agent System Method,” IEEE Transactions on Power Systems,
vol. 25, no. 1, pp. 470-479, 2010.
[6] V. Nanduri and T. K. Das, “A reinforcement learning model to assess
market power under auction-based energy pricing,” IEEE Transactions
on Power Systems, vol. 22, no. 1, pp. 85–95, Feb. 2007.
[7] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein “Distributed
Optimization and Statistical Learning via the Alternating Direction
Method of Multipliers,” Foundations and Trends® in Machine
Learning, vol. 3, no. 1, pp. 1-122, 2010.
[8] A. Kargarian, J. Mohammadi, J. Guo; S. Chakrabarti, M. Barati, G. Hug,
S. Kar, and R. Baldick, “Toward Distributed/Decentralized DC Optimal
Power Flow Implementation in Future Electric Power Systems,” IEEE
Transactions on Smart Grid, 2016.
