Formal handling of threats and rewards in a negotiation dialogue.

Full text

Page 1

Formal Handling of Threats and Rewards in a Negotiation
Dialogue
Leila Amgoud
IRIT - CNRS
118, route de Narbonne
Toulouse, France
amgoud@irit.fr
Henri Prade
IRIT - CNRS
118, route de Narbonne
Toulouse, France
prade@irit.fr
ABSTRACT
Argumentation plays a key role in finding a compromise dur-
ing a negotiation dialogue. It may lead an agent to change
its goals/preferences and force it to respond in a particu-
lar way. Two types of arguments are mainly used for that
purpose: threats and rewards. For example, if an agent re-
ceives a threat, this agent may accept the offer even if it is
not fully “acceptable” for it (because otherwise really im-
portant goals would be threatened).
The contribution of this paper is twofold. On the one hand,
a logical setting that handles these two types of arguments
is provided. More precisely, logical definitions of threats and
rewards are proposed together with their weighting systems.
These definitions take into account that negotiation dia-
logues involve not only agents’ beliefs (of various strengths),
but also their goals (having maybe different priorities), as
well as the beliefs about the goals of other agents.
On the other hand, a “simple” protocol for handling such
arguments in a negotiation dialogue is given. This protocol
shows when such arguments can be presented, how they are
handled, and how they lead agents to change their goals and
behaviors.
Categories and Subject Descriptors
I.2.3 [Deduction and Theorem Proving]: Nonmono-
tonic reasoning and belief revision
; I.2.11 [Distributed Artificial Intelligence]: Intelligent
agents
General Terms
Human Factors, Theory
Keywords
Argumentation, Negotiation
Permission to make digital or hard copies of all or part of this work for
personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that copies
bear this notice and the full citation on the first page. To copy otherwise, to
republish, to post on servers or to redistribute to lists, requires prior specific
permission and/or a fee.
AAMAS’05, July 25-29, 2005, Utrecht, Netherlands.
Copyright 2005 ACM 1-59593-094-9/05/0007 ...$5.00.
1.INTRODUCTION
Negotiation is the predominant interaction mechanism be-
tween autonomous agents looking for a compromise. Indeed,
agents make offers that they find acceptable and respond to
offers made to them.
Recent works on negotiation [2, 3, 4, 6, 7, 9, 10, 11] have
argued that argumentation can play a key role in finding
the compromise. Indeed, an offer supported by a ‘good ar-
gument’ has a better chance to be accepted, because the
argument brings new information possibly ignored by the
receiver. If this information conflicts with previous beliefs
of the receiver, this agent may even revise its beliefs if it has
no strong counter-argument for challenging the information.
Moreover, argumentation may constrain the future behavior
of the agent, especially if it takes the form of a threat or of
a reward. Such arguments complement more classical argu-
ments, called here explanatory arguments, which especially
aim at providing reasons for believing in a statement. Even
if the interest of using threats and rewards in a negotiation
dialogue [7, 12] has been emphasized, there has been almost
no attempt at modeling and incorporating them in a formal
dialogue.
This paper aims at providing a logical setting which handles
these two types of arguments, together with explanatory ar-
guments. More precisely, logical definitions of threats and
rewards are proposed together with their weighting systems.
These definitions take into account that negotiation dia-
logues involve not only agents’ beliefs (of various strengths),
but also their goals (having maybe different priorities), as
well as the beliefs about the goals of other agents. This
paper provides also a “simple” protocol for handling such
arguments in a negotiation dialogue. This protocol shows
when such arguments can be presented, how they are han-
dled, and how they lead agents to change their goals and
behaviors.
The paper is organized as follows: Section 2 introduces
the logical language for describing the mental states of the
agents. Sections 3, 4 and 5 introduce resp. the explana-
tory arguments, the threats and rewards. For each type of
argument, logical definitions are given together with their
weighting systems. Note that the given definitions enable
us to distinguish between what the agent finds rewarding
(resp. threatening) for it and what it finds rewarding (resp.
threatening) for the other agent.
argumentation system which handles the three types of ar-
guments is presented. Section 7 introduces a negotiation
protocol which is based on the notions of threats and re-
In section 6, a general
529

Page 2

wards, and which show when such arguments can be pre-
sented, how they are handled by their receivers, and how
they lead agents to change their behaviors. The approach
is illustrated in section 8 on the example of a negotiation
between a boss and a worker. In section 9, we conclude by
comparing our proposal with existing works and by present-
ing some perspectives.
2.THEMENTALSTATESOFTHEAGENTS
In what follows, L denotes a propositional language, ?
classical inference, and ≡ logical equivalence. We suppose
that we have two negotiating agents: P (for proponent) and
O (for opponent).
Each agent has got a set G of goals to pursue, a knowledge
base K, gathering the information it has about the environ-
ment, and a base GO, containing what the agent believes
the goals of the other agent are. K may be pervaded with
uncertainty (the beliefs are more or less certain), and the
goals in G and GO may not have equal priority.
Thus, each base is supposed to be equipped with a complete
preordering ≥. Relation a ≥ b holds iff a is at least as cer-
tain (resp. as preferred) as b. For encoding it, we use the set
of integers {0,1,...,n} as a linearly ordered scale, where n
stands for the highest level of certainty or importance and 0
corresponds to the complete lack of certainty or importance.
This means that the base K is partitioned and stratified into
K1,...,Kn(K = K1∪ ... ∪ Kn)
such that all beliefs in Ki have the same certainty level and
are more certain than beliefs in Kjwhere j < i. Moreover, K0
is not considered since it gathers formulas which are totally
uncertain, and which are not at all beliefs of the agent.
Similarly,
GO = GO1∪ ... ∪ GOnandG = G1∪ ... ∪ Gn
such that goals in GOi (resp. in Gi) have the same prior-
ity and are more important than goals in GOj(resp. in Gj
where j < i).
Note that some Ki’s (resp. Gi, GOi) may be empty if there
is no piece of knowledge (resp. goal) corresponding to the
level i of certainty (resp. importance).
For the sake of simplicity, in all our examples, we only spec-
ify the strata that are not empty. Both beliefs and goals
are represented by propositional formulas of the language
L. Thus a goal is viewed as a piece of information describ-
ing a set of desirable states (corresponding to the models of
the associated proposition) one of which should be reached.
3.EXPLANATORY ARGUMENTS
Explanations constitute the most common category of ar-
guments. In classical argumentation-based frameworks that
can handle inconsistency in knowledge bases, each conclu-
sion is justified by arguments. They represent the reasons
to believe in a fact.
3.1Logical definition
Such arguments have a deductive form.
premises, a fact or a goal is entailed. Formally:
Indeed, from
Definition 1
tory argument is a pair <H, h> such that:
(Explanatory argument). An explana-
1. H ⊆ K,
2. H ? h,
3. H is consistent and minimal (for ⊆) among the sets
satisfying 1) and 2).
Ae will denote the set of all the explanatory arguments that
can be constructed from K.
Note that the bases of goals are not considered when con-
structing such arguments (only based on agent’s beliefs) in
order to avoid wishful thinking.
3.2Strength of explanatory arguments
In [1], it has been argued that arguments may have forces
of various strengths. These forces will play two roles:
1. they allow an agent to compare different arguments in
order to select the ‘best’ ones,
2. the forces are useful for determining the acceptable
arguments among the conflicting ones.
Different definitions of the force of an argument have been
proposed in [1]. Generally, this force of an argument can rely
on the beliefs from which it is constructed. Explicit priori-
ties between beliefs, or implicit priorities such as specificity,
can be the basis for defining the force of an argument. How-
ever, different other aspects can be taken into account when
defining the force of explanatory arguments. In particular,
the length of the argument (in terms of the number of pieces
of knowledge involved) may be considered since the shorter
is the explanation, the better it is and the more difficult it
is to challenge it (provided that it is based on propositions
that are sufficiently certain).
When explicit priorities are given between the beliefs, such
as certainty levels, the arguments using more certain beliefs
are found stronger than arguments using less certain beliefs.
The force of an explanatory argument corresponds to the
certainty level of the less entrenched belief involved in the
argument. In what follows, we consider this view of the
force. In the case of stratified bases, the force of an argu-
ment corresponds to the smallest number of a stratum met
by the support of that argument. Formally:
Definition 2
∪ Kn be a stratified base, and H ⊆ K.
The certainty level of H, denoted Level(H) = min{j | 1 ≤ j
≤ n such that Hj ?= ∅}, where Hj denotes H ∩ Kj.
Note that <H, h> is all the stronger as Level(H) has a large
value.
(Certainty level). Let K = K1 ∪ ...
Definition 3
<H, h> ∈ Ae. The force of A is Force(A) = Level(H).
This definition agrees with the definition of an argument
as a minimal set of beliefs supporting a conclusion. Indeed,
when any member of this minimal set is seriously challenged,
the whole argument collapses. This makes clear that the
strength of the least entrenched argument fully mirrors the
force of the argument whatever are the strengths of the other
components in the minimal set. The forces of arguments
make it possible to compare any pair of arguments. Indeed,
arguments with a higher force are preferred.
(Force of an explanation). Let A =
Definition 4
∈ Ae. A is preferred to B (A ?eB) iff Force(A) > Force(B).
(Comparing explanations). Let A, B
530

Page 3

4.THREATS
Threats have a negative flavor and are applied to intend
to force an agent to behave in a certain way. Two forms of
threats can be distinguished:
i) You should do ‘a’ otherwise I will do ‘b’,
ii) You should not do ‘a’ otherwise I will do ‘b’.
The first case occurs when an agent P needs an agent O to
do ‘a’ and O refuses. Then, P threatens O to do ‘b’ which,
according to its beliefs, will have bad consequences for O.
Let us consider an example.
Example 1. Let’s consider a mother and her child.
Mother: You should carry out your school work (‘a’).
Child: No, I don’t want to.
Mother: You should otherwise I will not let you go to the
party organized by your friend next week-end (‘b’).
The second kind of threats occurs when an agent O wants
to do some action ‘a’, which is not acceptable for P. In this
case, P threatens that if O insists to do ‘a’ then it will do ‘b’
which, according to P’s beliefs, will have bad consequences
for O. The following example from [7] illustrates this kind
of threat.
Example 2.
Labor union: We want a wage increase (‘a’).
Manager: I cannot afford that. If I grant this increase,
I will have to lay off some employees (‘b’).
compensate for the higher cost entailed by the increase.
4.1 Logical definition
In all what follows, we suppose that P presents an argu-
ment to O. In a dialogue, each agent plays these two roles
in turn. For a threat to be effective, it should be painful
for its receiver and conflict with at least one of its goals. A
threat is then made up of three parts: the conclusion that
the agent who makes the threat wants, the threat itself and
finally the threatened goal. Moreover, it has an abductive
form. Formally:
It will
Definition 5
such that:
(Threat). A threat is a triple <H,h,φ>
1. h is a propositional formula,
2. H ⊆ K,
3. H ∪ {¬h} ? ¬φ such that φ ∈ GO,
4. H ∪ {¬h} is consistent and H is minimal (for set in-
clusion) among the sets satisfying the above conditions.
When GO is replaced by G in the above definition, one ob-
tains the definition of an “own-threat”. At will denote the
set of all threats and own-threats that may be constructed
from the bases <K, G, GO>.
With definition 5, the notion of own-threat covers both the
own evaluation of P for the threats it receives, and the
threats it may construct or imagine against itself from its
own knowledge. Note that h may be a proposition whose
truth can be controlled by the agent (e.g the result of an
action), as well as a proposition which is out of its control.
In a negotiation, an agent P may propose an offer x refused
by O. In this case, the offer x is seen as an own-threat by
O. P then entices O in order to accept the offer otherwise
it will do an action which may be more painful for O. Here
h is Accept(x).
Definition 5 captures the two forms of threats. Indeed, in
the first case (You should do ‘a’ otherwise I will do ‘b’), h =
‘a’, and in the second case (You should not do ‘a’ otherwise
I will do ‘b’), h = ¬a. ‘b?refers to an action which may be
inferred from H. The formal definition of threats is then
slightly more general.
Example 3. As said in example 1, the mother threatens
her child not to let him go to the party organized by his friend
if he does’t finish his school work. The mother is supposed
to have the following bases:
KMo = {¬Work → ¬Party},
GMo = {Work},
GOMo= {Party}.
The threat addressed by the mother to her child is formalized
as follows: <{¬Work → ¬Party}, Work, Party>.
Let’s now consider another dialogue between a boss and his
employee.
Example 4.
Boss: You should finish your work today.
Employee: No, I will finish it another day.
Boss: If you don’t finish it you’ll come this week-end to
make overtime.
In this example, the boss has the three following bases:
KBo = {¬ FinishWork → Overtime},
GBo = {FinishWork} and
GOBo= {¬Overtime}.
The threat enacted by the boss is: < {¬ FinishWork →
Overtime}, FinishWork, ¬Overtime>.
4.2Strength of threats
Threats involve goals and beliefs. Thus, the force of a
threat depends on two criteria: the certainty level of the be-
liefs used in that threat, and the importance of the threat-
ened goal.
Definition 6
φ> ∈ At.
The force of a threat A is a pair Force(A) = <α, β> s.t:
α = Level(H); β = j such that φ ∈ GOj.
However, when a threat is evaluated by its receiver (oppo-
nent), the threatened goal is in G. In fact, the threatened
goal may or may not be a goal of the opponent.
(Force of a threat). Let A = <H, h,
Definition 7
<H, h, φ> ∈ At.
The force of an own-threat A is a pair <α, β> s.t. α =
Level(H); β = j if φ ∈ Gj otherwise β = 0.
Intuitively, a threat is strong if, according to the most cer-
tain beliefs, it invalidates an important goal. A threat is
weaker if it involves beliefs with a low certainty, or if it only
invalidates a goal with low importance. In other terms, the
force of a threat represents to what extent the agent sending
(Force of an own-threat). Let A =
531

Page 4

it (resp. receiving it) is certain that it will violate the most
important goals of the other agent (resp. its own important
goals). This suggests the use of a conjunctive combination
of the certainty of H and the priority of the most important
threatened goal. Indeed, a fully certain threat against a very
low priority goal is not a very serious threat.
Definition 8
∈ At with Force(A) = <α, β> and Force(B) = <α’, β’>.
A is stronger than B, denoted by A ?t B, iff min(α, β) >
min(α’, β’).
(Conjunctive combination). Let A, B
Example 5. Assume the following scale {0,1,2,3,4,5}.
Let us consider two threats A and B whose forces are respec-
tively (α,β) = (3, 2) and (α’, β’) = (1, 5). In this case the
threat A is stronger than B since min(3, 2) = 2, whereas
min(1, 5) = 1.
However, a simple conjunctive combination is open to dis-
cussion, since it gives an equal weight to the importance of
the goal threatened and to the certainty of the set of beliefs
that establishes that the threat takes place. Indeed, one
may feel less threatened by a threat that is certain but has
‘small’ consequences, than by a threat which has a rather
small plausibility, but which concerns a very important goal.
This suggests to use a weighted minimum aggregation as fol-
lows:
Definition 9
Let A, B ∈ At with Force(A) = <α, β>, Force(B) = <α’,
β’>.
A is stronger than B, A ?t B, iff min(max(λ, α), β) >
min(max(λ, α?), β?), where λ is the weight that discounts
the certainty level component.
(Weighted conjunctive combination).
The larger λ is, the smaller the role of α in the evaluation.
The conjunctive combination is recovered when the value of
λ is minimal.
Example 6. Assume the following scale {0,1,2,3,4,5}.
Let us consider two threats A and B whose forces are re-
spectively (α,β) = (5, 2) and (α’, β’) = (2, 5). Using a
simple conjunctive combination, they both get the same eval-
uation 2. Taking λ = 3, we have min(max(3,5),2) = 2 and
min(max(3,2),5) = 3. Thus B is stronger than A.
The above approach assumes the commensurateness of three
scales, namely the certainty scale, the importance scale, and
the weighting scale.This requirement is questionable in
principle. If this hypothesis is not made, one can still define
a relation between threats.
Definition 10. Let A, B ∈ Atwith Force(A) = <α, β>
and Force(B) = <α’, β’>.
A is stronger than B iff:
1. β > β’ or,
2. β = β’ and α > α’.
This definition also gives priority to the importance of the
threatened goal, but is less discriminating than the previous
one.
5.REWARDS
During a negotiation an agent P can entice agent O in
order that it does ‘a’ by offering to do an action ‘b’ as a
reward. Of course, agent P believes that ‘b’ will contribute
to the goals of O. Thus, a reward has generally, at least
from the point of view of its sender, a positive character. As
for threats, two forms of rewards can be distinguished:
i) If you do ‘a’ then I will do ‘b’.
ii) If you do not do ‘a’ then I will do ‘b’.
The following example illustrates this idea.
Example 7. A seller proposes to offer a set of blank CDs
to a customer if this last accepts to buy a computer.
5.1Logical definitions
Formally, rewards have an abductive form and are defined
as follows:
Definition 11
such that:
(Reward). A reward is a triple <H,h,φ>
1. h is a propositional formula,
2. H ⊆ K,
3. H ∪ {h} ? φ such that φ ∈ GO,
4. H ∪ {h} is consistent and H is minimal (for set inclu-
sion) among the sets satisfying the above conditions.
When GO is replaced by G in the above definition, one gets
the definition of an own-reward.
Ar will denote the set of all the rewards that can be con-
structed from <K, G, GO>.
Note that the above definition captures the two forms of
rewards. Indeed, in the first case (If you do ‘a’ then I will
do ‘b’), h = ‘a’, and in the second case (If you do not do ‘a’
then I will do ‘b’), h = ¬a.
Example 8. Let’s consider the example of a boss who
promises one of his employee to increase his salary.
Boss: You should finish this work (‘a’).
Employee: No I can’t.
Boss: If you finish the work I promise to increase your
salary (‘b’).
The boss has the following bases:
Kn = {FinishWork → IncreasedBenefit},
Kn−1 = {IncreasedBenefit → HigherSalary},
Gn = {FinishWork} and
GOn= {HigherSalary}.
The boss presents the following reward in favor of its request
‘FinishWork’: <{FinishWork → HighBenefit, HighBenefit
→ HighSalary}, FinishWork, HighSalary>.
Threats are sometimes thought as negative rewards. This is
reflected by the parallel between the two definitions which
basically differ in the third condition.
Remark 1. Let K, G, GO be the three bases of agent P.
If h ∈ G ∪ GO, <∅, h, h> is both a reward and a threat.
The above property says that if h is a common goal of the
two agents P and O, then <∅, h, h> can be both a reward
and a threat, since the common goals jointly succeed or fail.
This is either both a reward and a own-reward, or a threat
or a own-threat for P.
532

Page 5

5.2Strength of rewards
As for threats, rewards involve beliefs and goals. Thus, the
force of a reward depends also on two criteria: the certainty
level of its support and the importance of the rewarded goal.
Definition 12
h, φ> ∈ Ar.
The force of a reward A is a pair Force(A) = < α,β > s.t:
α = Level(H); β = j such that φ ∈ GOj.
However, when a reward is evaluated by its receiver (oppo-
nent), the rewarded goal is in G. In fact, if the proponent
does not misrepresent the opponent’s goals, the rewarded
goal is a goal of the opponent.
(Force of a reward). Let A = <H,
Definition 13
<H, h, φ> ∈ At.
The force of an own-reward A is a pair <α, β> s.t. α =
Level(H); β = j if φ ∈ Gj, otherwise β = 0.
Example 9. In example 8, the force of the reward
<{FinishWork → HighBenefit,HighBenefit →
HighSalary}, FinishWork, HighSalary> is <n-1, n>.
A reward is strong when for sure it will contribute to the
achievement of an important goal. It is weak if it is not sure
that it will help to the achievement of an important goal, or
if it is certain that it will only enable the achievement of a
non very important goal. Formally:
(Force of an own-reward). Let A =
Definition 14
B be two rewards in Ar with Force(A) = <α, β> and
Force(B) = <α’, β’>.
A is preferred to B, denoted by A ?r B, iff min(α, β) >
min(α’, β’).
(Conjunctive combination). Let A,
However, as for threats, a simple ‘min’ combination is debat-
able, since it gives an equal weight to the importance of the
rewarded goal and to the certainty of the set of beliefs that
establishes that the reward takes place. Indeed, one may
feel less rewarded by a reward that is certain but has ‘small’
consequences, than by a reward which has a rather small
plausibility, but which concerns a very important goal. This
suggests to use a weighted minimum aggregation as follows:
Definition 15
A, B ∈ Ar with Force(A) = <α, β> and Force(B) = <α’,
β’>.
A ?r B iff min(max(λ,α),β) > min(max(λ,α?),β?), where
λ is the weight that discounts the certainty level component.
(Weighted conj. combination). Let
The larger λ is, the smaller the role of α in the evaluation.
The ’min’ combination is recovered when the value of λ is
minimal. In some situations, an agent may prefer a reward
which is sure, even if the rewarded goal is not very impor-
tant for it, than an uncertain reward with very ‘valuable’
consequences. This suggests to use a weighted minimum
aggregation giving priority to the certainty component of
the force, as follows:
Definition 16. Let A, B ∈ Ar with Force(A) = <α,
β> and Force(B) = <α’, β’>.
A ?r B iff min(α,max(λ,β)) > min(α?,max(λ,β?)), where
λ is the weight that discounts the importance of the goal.
Finally, as for threats, if there is no commensurateness of the
three scales, we can still be able to compare two rewards as
follows, in the spirit of definition 15:
Definition 17. Let A, B ∈ Ar with Force(A) = <α,
β> and Force(B) = <α’, β’>.
A ?r B iff:
1. β > β’ or,
2. β = β’ and α > α’.
This definition also gives priority to the importance of the
rewarded goal. In the case of an agent which prefers rewards
that are certain even if the rewarded goals are not very im-
portant, one can use the following preference relation.
Definition 18. Let A, B ∈ Ar with Force(A) = <α,
β> and Force(B) = <α’, β’>.
A ?r B iff:
1. α > α’ or,
2. α = α’ and β > β’.
6. ARGUMENTATION SYSTEM
Due to the presence of potential inconsistency in knowl-
edge bases, arguments may be conflicting. The most com-
mon conflict which may appear between explanatory argu-
ments is the relation of undercut where the conclusion of an
explanatory argument contradicts an element of the support
of another explanatory argument. Formally:
Definition 19. Let <H,h>, <H?,h?> ∈ Ae. <H,h>
defeatse <H?,h?> iff
1. <H,h> undercuts <H?,h?> and
2. not (<H?,h?> ?e <H,h>)
Two threats may be conflicting for one of the three following
reasons:
• the support of an argument infers the negation of the
conclusion of the other argument. It occurs when, for
example, an agent P threatens O to do ‘b’ if O refuses
to do ‘a’, and at his turn, O threatens P to do ‘c’ if P
does ‘b’.
• the threats support contradictory conclusions. It oc-
curs, for example, when two agents P and O have con-
tradictory purposes.
• the threatened goals are contradictory. Since a rational
agent should have consistent goals, GO should be as
well consistent, and thus this arises when two threats
are given by different agents.
As for threats, rewards may also be conflicting for one of the
three following reasons:
• the support of an argument infers the negation of the
conclusion of the other argument. It occurs when an
agent P promises to O to do ‘b’ if O refuses to do ‘a’.
C, at his turn, promises to P to do ‘c’ if P does not
pursue ‘b’.
• the rewards support contradictory conclusions. This
kind of conflict has no sense if the two rewards are
constructed by the same agent. Because this means
that the agent will contribute to the achievement of
a goal of the other agent regardless what the value
of h is. However, when the two rewards are given by
different agents, this means that one of them wants h
and the other ¬h and each of them tries to persuade
the other to change its mind by offering a reward.
533