Content uploaded by Motab Almousa

Author content

All content in this area was uploaded by Motab Almousa on Feb 26, 2017

Content may be subject to copyright.

1

Abstract—This paper proposes a novel multi-agent collaborative

algorithm to unveil economic potentials of local collaborations

between generation companies’ (GenCos). We investigate the

potential of local collaborations to further increase GenCos’

profit after market clearing by the system operator. A multi-

agent based analytical target cascading (M-ATC), which is a

distributed optimization algorithm, is developed to accommodate

the need to model GenCos as self-interested agents. The main

novelty of M-ATC is twofold. It enables the agents to

simultaneously determine the amount of local energy exchange as

well as the optimal cost for energy transactions. In addition, a

novel dynamic cutting plane procedure is presented to convert

the distributed model to an agent-based model. The proposed

algorithm guarantees economic feasibility and information

privacy of the self-interested agents in a complete regulation-free

environment. The practicality and efficiency of the proposed

model is tested on a unit commitment problem.

Index Terms—Cooperative optimization, distributed solution

algorithm, multi-agent, electricity market.

INTRODUCTION

I.

N deregulated electricity markets, private generation

companies (GenCos) are considered as self-interested

agents in a competitive environment. Their role is to supply

the electricity for consumers. Each agent pursues a single

objective to maximize its own profit. The agents (in this paper

we interchangeability use the terms “GenCo” and “agent”)

send their bids to the independent system operation (ISO).

From the system perspective, ISO clears the market to

maximize social welfare. Locational marginal prices as well as

the power output of each agent are determined. Consider that

multiple GenCos share the same bus. This raises a possibility

for mutual collaborations between the agents to further

increase their benefits. Fig. 1 shows a possible system

structure in which multiple agents are located on the same bus.

After market clearing, the agents on a bus can collaborate to

locally reschedule power outputs of their units. Local

collaborations and any bus must not affect locational marginal

prices and the overall social welfare from the system vantage

point. This is, the collaborations between the agents do not

conflict with higher system’ merits.

On the other hand, GenCos are in a competitive

environment, where information exchange is crucial and

brings a barrier against any potential profit-based

collaboration. Thus, information privacy is a critical part of

the local collaboration of the participated agent. Revealing

characteristics of an agent can be utilized by others to develop

dominate biding strategies.

The authors are with the Division of Electrical and Computer Engineering,

Louisiana State University, Baton Rouge, LA 70803 USA, email:

malmou2@lsu.edu, kargarian@lsu.edu.

Fig. 1. A possible structure of multi-agents GenCos.

As a result, novel coordination strategies are required to

coordinate potential collaborations between GenCos.

In the past few years, researchers were successfully able to

develop powerful distributed algorithms to coordinate multi-

agents and distributed systems. In [1] and [2], the authors

proposed an algorithm to coordinate the multi-area unit

commitment problem in a distributed manner. The algorithm

is computationally efficient and provides the optimal solution

to the centralized problem while preserving information of the

agents. However, economic benefits of each agent have not

been addressed. To satisfy self-interest agents’ requirements, a

further step is needed to allocate the benefits that are obtained

through the agents’ collaborations. Authors in [3] tried to

resolve this problem by presenting propose an independent

system operator, called mini-ISO. However, the approach

contradicts the movement to deregulate the market and the

needs for information privacy. Another solution to this

problem is to use the concept of multi-agent systems (MAS) in

the electricity market as a complex system. Evolutionary and

artificial intelligent based methods have been used for

GenCos’ decision-making [4]. In [5] and [6], Q-Learning and

reinforcement learning approaches have been proposed to

solve the schedule the GenCos. However, learning-based

approaches are not robust enough to be relied on at practical

markets.

This paper proposes a novel distributed optimization

algorithm for multi-agent based decision-making for

scheduling the GenCos after clearing the market for additional

economic gain. The distributed algorithm is based on the

concept of analytical target cascading (however, our algorithm

is general and does not dependent on the choice of the

distributed optimization algorithm; one can use another

algorithm such as auxiliary problem principle). To the best of

our knowledge, this paper is the first to provide a

mathematical-based framework to consider economic

feasibility of self-interested GenCos after the market clearing

Multi-Agent Collaborative Model in Electrical

Energy Market: A Distributed-Based Approach

Motab Almousa, Student Member, IEEE, Amin Kargarian, Member, IEEE

I

2

by the operator. The contributions of the paper are

summarized as follow:

• A distributed-based economic feasibility cutting plane

algorithm is developed to convert a distributed

optimization algorithm to a distributed self-interested

agent-based algorithm.

• Hybrid transaction-electricity flow is considered for more

potential of collaboration and improvement of

convergence properties.

• No information is needed to exchange between agents

which is aligned with the information privacy in MAS.

A

GENT

-B

ASED

C

OLLABORATIVE

M

ODEL

U

SING A

II.

DISTRIBUTED

A

PPROACH

A. General Description and Mathematical Derivation of the

Proposed M-ATC

GenCos are considered as self-interested agents. Thus,

distributed optimization algorithms, such as ATC, need to take

into account the cost of energy transaction between the agents.

Otherwise, one agent might gain benefit while another losing.

In order to avoid this, we need to modify the traditional ATC

algorithm to adopt it with the concept of multi-agent systems.

We develop a multi-agent based ATC (M-ATC) in which each

GenCo is allowed to provide an upper-bound for its objective

function. We set the upper-bound to the GenCo’s power

scheduled by the ISO multiply by the locational marginal price

at the bus that the GenCo is located. To guarantee the

feasibility of the model as well as economic benefits for all

participated agents, transaction flows are considered. The

transaction flows enable GenCos to reach a cost allocation

scheme that is feasible to all agents. A positive transaction

means that a GenCo sends money to others, whereas a

negative transaction implies that the GenCo receives money

from others. Although multi-agent systems (MAS) problems

are classified as uncertain game-based problems, they need

statistical and learning-based approaches. The potential of

modeling them as distributed mathematical-based problems is

provided in this paper.

Claim: The proposed M-ATC algorithm is economically

feasible.

Proof: Consider the following optimization problem for

two agents and

(

) (1)

Given that

=(

,

,):≤,

,

,∈

, where M is a non-convex feasible set. The local variables

of agents A and B are indicated by

and

, and the shared

variables between them is denoted by, i.e.,

,

⊆ , and

∩

= . Optimization problem (1) can be rewritten as:

,

,

(

,

,)

(2)

..

∈ M1,

∈ M2, ∈ (M1 ⋂ M2).

Assuming agents A and B solve their problem independently,

the optimization problem will be as follow:

,

(

,

) +

,

(

,

)

..

+

=0 (3)

It is trivial that objective function (2) is always lower than or

equal to (3) for all feasible solutions

+

=0. Yet not

optimal . Thus

,

,

(

,

,) (4)

≤

,

(

,

) +

,

(

,

)

without any loss of generality, a slack variable, 1, is added

such that:

,

,

(

,

,)+1 (5)

=

,

(

,

) +

,

(

,

)

1 is always greater than or equal to zero. S1 represents the

potential savings of the agents because of their collaboration.

However, it is not decomposable to be modeled as an

independent saving of the agents. In a similar manner, but

with assuming a case of feasible collaboration and no

collaboration, we have:

,

(

,

) +

,

(

,

)

≤

(

) +

(

) (6)

Let us transform inequality constraint (6) to the following

equality constraints:

,

(

,

) +

,

(

,

)+2

=

(

) +

(

) (7)

2 is always greater than or equal to zero. By moving 2 to

the right-hand side, we have:

,

(

,

) +

,

(

,

)

=

(

) +

(

) −2 (8)

Since S2 is proven to be greater than or equal to zero after

collaboration and the right-hand side is separable, this proves

the overall economic feasibility of the collaboration. The slack

variable can be rewritten as 2=2+2. Since there is

no restriction on the sign for 2 and 2, we have:

(

) +

(

) −2

=

(

)−2 +

(

)

−2

(9)

We now derive the economic feasibility of the collaboration

between agents. The economic feasibility for agent A is

defined as

(

)−2

and for Agent B, it is

(10)

3

(

) −2

Two approaches can guarantee the potential of the

collaboration. First:

(

)−

,

(

,

)=1

and second:

(

)−

,

(

,

)=2

.. 1≥0, 2≥0, and

+

=0

(11)

It is straightforward to formulate 1≥0.

,

..

(

,

)=

≤

(

)

(12)

and in a similar manner for 2≥0. However, this approach

does not guarantee feasibility. There could not be any

+

=0 over the feasible region of the collaboration ∈

(M1 ⋂ M2). In addition, it is known that minimizing overall

cost is equivalent to maximizing the social welfare (slack

variable). We introduce two cuts, 1≥0 and 2≥0, to

cut the exiting feasible space of the optimization problem. To

overcome this cutting issue, we propose new variables that

represent contracted transactions

and

. The

proposed formulation becomes:

(

)−

,

(

,

)

=1+

and

(

)−

,

(

,

)

=2+

.. 1+

≥0, 2+

≥0,

and

+

=0

+

=0

(13)

Since for any feasible solution of multi-agents systems the loss

of one system is always less than or equal to the gain of the

other system,

and

guarantee feasibility of the

model. The ATC model includes two levels, subsystem

(agent) level and coordination level. After proving economic

feasibility of the proposed model, subsystems in ATC is

developed to be self-interested agents in the proposed M-ATC

formulation. Each objective function at the agent level is

modeled as:

,

+

−

+

−

..

(

,

)+

≤

≤

(

)

(14)

where

and

are the targets from the coordinator (or

master) problem, and

and

are penalty functions

for the distrusted algorithm convergence.

B. Derivation of M-ATC Model for Unit Commitment in

Electrical Energy Market

We apply the proposed algorithm to solve a multi-agent based

unit commitment problem.

Agents Level: Assume that an agent operates independently.

The optimization problem is to minimize cost of the agent to

its local constraints:

,

(

)

+

(15)

.. (

,

)≤0

(

,

)=0

Where

(⋅) is the generation cost function of unit . SUD

represents start up/down cost of the units. and are

respectively power output and unit status variables. and

denote the scheduling horizon and number of units. and

are compact representative of the UC inequality and

equality constraints, e.g., ramping, and minimum on/off time.

Now, let us incorporate energy and transaction flows

between the agents. The new optimization problem after

considering the flow interchange is rewritten as:

,

,

,℘

(

)

+

+

+℘

(16)

.. (

,

,

,℘

)≤0

(

,

,

,℘

)=0

where

and ℘

are the energy and contract amounts that

exchange between agent j and its neighboring agents at time t,

and ℘

is the amount of the transaction flow. After

considering the exchange with other agents, the local

optimization problem includes both local and public (or

shared) variables. The local variables are variables that

exclusively belong to only one agent, such as unit status and

energy dispatch . Any variable that depends on at least two

agents is classified as a public variable. Thus, energy flow

and contract ℘

are classified as public variables. The

contracts and energy flow are not restricted in sign, and thus

they can be received or provided. However, since these two

variables are dependent on other agents, they need to be

considered in all connected agents.

,

(

)

+

+

(

−

) +

(℘

−Φ

)

(17)

.. (

,

,

,℘

)≤0

4

(

,

,

,℘

)=0

In (17), penalty functions are considered to penalize the

deviation between agents. The concept is widely known in the

field of distributed optimization [2, 6, 8]. The terms

and

are the penalty functions. This formulation ideally

reaches the global optimal without considering the agents’

economic benefits. However, this is not the goal for the self-

interested agents. As agents are self-interested, the objective is

to minimize total cost locally. To address this issue, upper

bound cuts for the optimization problem is introduced to

modify the transaction-based unit commitment problem such

that:

1. A dynamic local cut given in (18) to minimize the actual

operating cost without considering the penalty functions.

2. In (19), a static upper bound is applied to the dynamic cut

presented to ensure the economic feasibility for the

agents.

(

)

+

+

+℘

≤

(18)

and,

≤

,

(

)

+

..

(

,

)≤0

(

,

)=0

.()

(19)

The static upper bound could be interpreted as the maximum

contract amount that the agent is willing to agree on as a result

of the model clearing. For more flexibility of the agents, a

dynamic function () can be considered. This function

represents the bidding strategy for GenCo. The agent’s local

optimization problem of is expressed as follow:

,

,

,℘

+

(

−

) +

(℘

−Φ

)

(20)

.. (

,

,

,℘

)≤0

(

,

,

,℘

)=0

(

)

+

+

+℘

≤

≤

,

(

)

+

..

(

,

)≤0

(

,

)=0

.()

Now, each agent formulation is completed. However,

and

Φ

are needed to be obtained from the coordinator level.

Coordinator Level: At this stage, each agent should provide

the coordinator with the solution of the public variables and

the contracted amount willing to pay/receive from/to other

agents. The process is implemented as follow. All agents’

responses are aggregated to the master coordinator. The

master coordinator considers the responses as constants in (21)

and (22).

.

..

....

.

=

.

.

....

.

(21)

℘

℘

.

℘

= ℘

℘

.

℘

(22)

Then, the master coordinator solves for new targets to achieve

convergence for all participated agents. The objective function

at coordinator level is:

,

−

+

Φ

−℘

..

=

∀,

Φ

=Φ

∀,

(23)

where

and Φ

are vector of the shared targets with all

neighboring agents. The penalty terms in (23) are:

=

−

+ ∘

−

(24)

= Φ

−℘

+ ∘Φ

−℘

(25)

where ∘ is Hadamard product. Parameters , , , and are

penalty multipliers that need to be updated after every iteration

as:

=

+(

)

(′

−′

) (26)

=

(27)

=

+(

)

(℘′

−℘′

) (28)

=

(29)

The updated values of

and Φ

will be sent to all agents for

the next iteration. The agents use the updated target and solve

their problems again. We follow this iterative process until the

algorithm converges. The solution algorithm is shown in

Algorithm 1.

C

ASE

S

TUDIES

III.

A unit Commitment problem for three GenCos is considered

to illustrate the effectiveness of the proposed model. The

motivation is to achieve an additional profit for each energy

provider after the market is cleared and power quantities are

awarded to GenCos. More profit is achieved by maximizing

the difference between the payment received from ISO and the

operation cost. Under the assumption that GenCos are

obligated to quantities disaggregated from ISO, solving the

UC problem is equivalent to the profit-based solution. Figure

5

2 shows the quantities disaggregated to each agent for 24 hour.

Characteristics of the units are provided in Table I. Min

up/down times and ramping constraints are considered. The

units’ cost functions are assumed to be linear.

Algorith

m

1 Contract-based UC for multi-agent system

1: Start all agents solve (15) in parallel and determine local upper

bound in (19)≤

2: Initialization

• Initialize penalty multipliers, , , and

• Initialize

,

, and ℘

3: While

−

AND ℘

−℘

> , n = n+1 do

4: Each agent solves (20) and determines the values of

,

,

, and℘

5: Send

and ℘

to the master coordinator

6: Aggregate the shared variables in the master coordinator

7: Solve (23) and propagate

and Φ

to all agents

8: update , , , and

9: end while

TABLE I

UNITS CHARACHTERISTIC

Sys./Unit 1 2 3 4 5 6 7 8 9 10

1

a 50 50 100 100 100 200 100 100 188 105

b 8 12 10 6 7 11 10 30 8 7

c 0.05 0.3 0.16 0.008 0.25

0.1 0.006 0.04 0.035

0.066

MD|MU 2|2 2|4 3|5 2|7 1|2 1|4 3|4 3|5 2|2 5|8

RD|RU 60 |50

80|

50

100|

150

100|

120

100|

120

80|

80

150|

120

100|

100

50|

150

120|

210

2

a 50 50 100 150 160 250 110 100 120 115

b 8 6 10 25 50 39 11 9 10 7

c 0.25 0.32 0.6 0.1 0.06

0.05 0.04 0.03 0.05 0.04

MD|MU 2|4 2|4 3|5 4|7 1|2 1|1 1|6 3|6 3|4 2|8

RD|RU

50|

50

50|

50

150|

150

200|

200

100|

100

80|

80

150|

150

150|

150

150|

150

300|

300

3

a 50 50 100 100 100 200 100 100 100 105

b 6 8 10 6 7 10 10 20 6 7

c 0.02 0.07 0.09 0.04 0.02

0.1 0.08 0.06 0.05 0.08

MD|MU 2|2 2|4 3|5 2|7 1|2 1|4 3|4 3|5 2|2 5|8

RD|RU

60|

50

80|

50

120|

150

120|

120

120|

120

80|

80

140|

120

120|

100

50|

150

80|

210

Each GenCo determines its upper limit and the algorithm is

initialized.

and ℘

are initialized at 10000 and

at 0,and

=10

. and are fixed at 1. The algorithm

converged after 22 iterations. The three GenCos reduce their

costs (or increase their benefits). Energy exchange responses

between agents 1 and 2 in, for instance, hour 8 is shown in

Fig. 3. A transaction flow of $1103.8 is to be paid to GenCo 2

by GenCo 1. GenCo 2 needs to pay $982.4 to GenCo 3.

GenCos 1, 2, and 3 have savings of $8237.7, $11076.3, and

$5762.1, respectively. Table II provides energy scheduling

cost for each GenCo.

C

ONCLUSION

IV.

In this paper, a multi-agent collaborative algorithm is

proposed to locally coordinate GenCos to gain additional

profits after market clearing. As shown in the results,

additional savings has been achieved for all participated

GenCos. Further savings can be achieved with more iteration.

However, the objective of this paper is to show the feasibility

of reaching savings for all participated agents using a

distributed optimization algorithm.

Fig. 2. 24-hr power quantities scheduled for GenCos.

Fig. 3. Flow responses of GenCo 1-2 and inverse of GenCo 2-1.

TABLE II

COST COMPARISON

System Cost ($)

Without Interchange Our Approach

1 455486 447248.3

2 658842 647765.7

3 404956 399193.9

R

EFERENCES

[1] A. Kargarian, Y. Fu, Z. Li. “Distributed security-constrained unit

commitment for large-scale power systems,” IEEE Transactions on

Power Systems, vol. 30, no.4 pp. 1925-19-36, 2015.

[2] A. Kargarian, Y. Fu and H. Wu, "Chance-Constrained System of

Systems Based Operation of Power Systems," IEEE Transactions on

Power Systems, vol. 31, no. 5, pp. 3404-3413, Sept. 2016.

[3] J. Li, y. liu and L. Wu, “Optimal Operation for Community Based Multi-

Party Microgrid in Grid-Connected and Islanded Modes,” IEEE

Transactions on Smart Grid, 2016.

[4] K. Dehghanpour, M. Nehrir, J. Sheppard and N. Kelly, “Agent-Based

Modeling in Electrical Energy Markets Using Dynamic Bayesian

Networks,” IEEE Transactions on Power Systems, vol. 31, no. 6, pp.

4744-4754, 2016.

[5] N.P. Yu, C.C. Liu, and J. Price, “Evaluation of Market Rules Using a

Multi-Agent System Method,” IEEE Transactions on Power Systems,

vol. 25, no. 1, pp. 470-479, 2010.

[6] V. Nanduri and T. K. Das, “A reinforcement learning model to assess

market power under auction-based energy pricing,” IEEE Transactions

on Power Systems, vol. 22, no. 1, pp. 85–95, Feb. 2007.

[7] S. Boyd, N. Parikh, E. Chu, B. Peleato, and J. Eckstein “Distributed

Optimization and Statistical Learning via the Alternating Direction

Method of Multipliers,” Foundations and Trends® in Machine

Learning, vol. 3, no. 1, pp. 1-122, 2010.

[8] A. Kargarian, J. Mohammadi, J. Guo; S. Chakrabarti, M. Barati, G. Hug,

S. Kar, and R. Baldick, “Toward Distributed/Decentralized DC Optimal

Power Flow Implementation in Future Electric Power Systems,” IEEE

Transactions on Smart Grid, 2016.