Page 1

Weak and Strong Time Consistency

in Differential Oligopoly Games with

Capital Accumulation1

Roberto Cellini§- Luca Lambertini∗,#

§Department of Economics, University of Catania

Corso Italia 55, 95129 Catania, Italy

phone 39-095375344, fax 39-095-370574, cellini@unict.it

∗Department of Economics, University of Bologna

Strada Maggiore 45, 40125 Bologna, Italy

phone 39-051-2092600, fax 39-051-2092664, lamberti@spbo.unibo.it

#ENCORE, Faculty of Economics & Econometrics

University of Amsterdam, WB 1018 Amsterdam, The Netherlands

July 25, 2005

1We thank George Leitmann and Massimo Marinacci for useful comments and

suggestions. The usual disclaimer applies.

Page 2

Abstract

We illustrate two differential oligopoly games with capital accumulation where,

alternatively, the accumulation dynamics of productive capacity is modelled

either ` a la Solow—Swan or ` a la Ramsey. We show that in the first case

the open-loop Nash equilibrium is only weakly time consistent, while in the

second it is strongly so, although the Ramsey game is not state linear.

Keywords: differential games, capital accumulation, open-loop equilib-

ria, closed-loop equilibria

Page 3

1Introduction

The existing literature on differential games applied to firms’ behaviour

mainly concentrates on two kinds of strategies adopted by players: the open-

loop and the closed-loop strategies.1When players (firms) adopt the open-

loop solution concept, they design the time path concerning the control vari-

able(s) at the initial time and then stick to it forever. This means that the

open-loop strategy is simply a time path of actions, and time is the only de-

terminant of the action to be done at any instant. The relevant equilibrium

concept is the open-loop Nash equilibrium, which is only weakly time con-

sistent and therefore, in general, it is not subgame perfect.2When players

adopt the closed-loop strategy, they do not precommit control variable(s) on

any path, and their actions at any instant may depend on all the preceding

history. In particular, the values of the state variables are taken into ac-

count when players choose their actions at any time. In this situation, the

information set used by players in setting their actions at any given instant

is often simplified to be only the current value of the state variables at that

time, along with the initial conditions. This specific situation is labelled

memoryless closed-loop. The relevant equilibrium concept, in this case, is

the closed-loop memoryless Nash equilibrium, which is strongly time consis-

tent (or subgame perfect). A different refinement of the closed-loop Nash

equilibrium, which is known as the feedback Nash equilibrium, can also be

1See Kamien and Schwartz (1981); Ba¸ sar and Olsder (1982, 19952); Mehlmann (1988);

Dockner et al. (2000).

2As to the definition of time consistency and subgame perfection, we rely - among many

different definitions available in the literature - on the definition provided by Dockner et

al. (2000, Section 4.3). We use “strong time consistency” as a synonomous of subgame

perfection. Further details are provided below.

1

Page 4

adopted as the solution concept.3While in the closed-loop memoryless case

the initial and current levels of all state variables are taken into account, in

the feedback case only the current stocks of states are considered.4

It is worth recalling that “open-loop” and “closed-loop” labels denote

different concepts of strategies (in the former, optimal plans depend only

on time, whereas in the latter situation, plans depend on the history of the

game, possibly summarised by the current stocks of state variables), as well as

different kinds of information sets to be used by players (under the open-loop

information structure, players’ plans associate actions to time only, whereas

under the closed-loop information structure, the value of the states has also

to be taken into account by players).

The existing literature in this field devotes a considerable amount of

attention to identifying classes of games where either the feedback or the

closed-loop equilibria degenerate into open-loop equilibria. The degenera-

tion means that the Nash equilibrium time path of the control variables

coincide under the different strategy concepts. The interest in the coinci-

dence between the equilibrium path under the different solution concepts is

motivated by the following reason. Whenever an open-loop equilibrium is

a degenerate closed-loop equilibrium, then the former is also strongly time

consistent (or subgame perfect). Therefore, one can rely upon the open-loop

equilibrium which, in general, is much easier to derive than closed-loop or

feedback ones. Classes of games where this coincidence arises are illustrated

in Clemhout and Wan (1974); Reinganum (1982); Mehlmann and Willing

(1983); Dockner, Feichtinger and Jørgensen (1985); Fershtman (1987); Fer-

3For oligopoly models where firms follow feedback rules, see Simaan and Takayama

(1978), Fershtman and Kamien (1987, 1990), Dockner and Haug (1990), Cellini and Lam-

bertini (2004), inter alia.

4For a clear exposition of the difference among these equilibrium solutions see Ba¸ sar

and Olsder (1982, pp. 318-327, and chapter 6, in particular Proposition 6.1).

2

Page 5

shtman, Kamien and Muller (1992).5As a whole, the games where open-loop

equilibria are strongly time consistent are known as perfect or state redun-

dant, precisely because optimal controls derived both in open-loop and in

closed-loop depend only upon time but not upon states. Hence, open-loop

controls are indeed degenerate closed-loop ones. A class of games where this

clearly applies is that of linear state games, where the Hamiltonian function

is linear in the state variables.

In this paper, we present two classes of Cournot oligopoly games with

capital accumulation. In particular, we will consider: (a) the accumulation

under the model of reversible investment ` a la Nerlove-Arrow (1962), where

capital accumulation occurs through costly investment, as in Solow’s (1956)

and Swan’s (1956) growth model; (b) the model ` a la Ramsey (1928), i.e.,

a “corn-corn” growth model, where accumulation coincides with consump-

tion postponement. Both accumulation mechanisms are widely studied by

available theoretical models. We show that only under the Ramsey capi-

tal accumulation dynamics, the open-loop Nash equilibrium is a degenerate

closed-loop one and hence the former is subgame perfect. This depends on

two features: first, the dynamic behaviour of any firm’s state variable does

not depend on the rivals’ control and state variables, which makes the kine-

matic equations concerning other firms redundant; second, for any firm, the

first order conditions taken w.r.t. the control variables are independent of the

rivals’ state variables, which entails that the cross effect from rivals’ states

to own controls (which characterises the closed-loop information structure)

disappears.

The remainder of the paper is structured as follows. The basics of the

5For an overview, see Mehlmann (1988) and Dockner et al. (2000, ch. 7). The related

issue of time consistency of Stackelberg open-loop equilibria is not treated here (see Cellini,

Lambertini and Leitmann, 2005).

3

Page 6

model are laid out in section 2. Section 3 examines the two capital accumu-

lation games. Section 4 contains concluding remarks.

2 The general setup

The game is played over continuous time, t ∈ [0,∞).6Define the set of players

as P ≡ {1,2,3,...N}. Moreover, let xi(t) and ui(t) define, as usual, the state

variable and the control variable pertaining to player i. Assume there exists

a prescribed set Uisuch that any admissible action ui(t) ∈ Ui. The dynamics

of player i’s state variable is described by the following:

dxi(t)

dt

≡

.xi(t) = fi(x(t),u(t)) (1)

where x(t) = (x1(t),x2(t),...xN(t)) is the vector of state variables at time

t, and u(t) = (u1(t),u2(t),...uN(t)) is the vector of players’ actions at the

same date, i.e., it is the vector of control variables at time t. That is, in

the most general case, the dynamics of the state variable associated with

player i depends on all state and control variables associated with all players

involved in the game. The value of the state variables at t = 0 is assumed to

be known: x(0) = (x1(0),x2(0),...xN(0)).

Each player has an objective function, defined as the discounted value of

the flow of payoffs over time. The instantaneous payoff depends upon the

choices made by player i as well as its rivals, that is:

πi≡ πi(x(t),u(t)).

(2)

Player i’s objective is then, for all given uj(t), j 6= i :

max

ui

Ji≡

Z∞

0

πi(x(t),u(t))e−ρtdt

(3)

6The game can be reformulated in discrete time without significantly affecting its qual-

itative properties. For further details, see Ba¸ sar and Olsder (1982, 19952).

4

Page 7

subject to the dynamic constraint represented by the behaviour of the state

variables, (1), ui(t) ∈ Uiand initial conditions x(0) = (x1(0),x2(0),...xN(0)).

The factor e−ρtdiscounts future gains, and the discount rate ρ is assumed to

be constant and common to all players. In order to solve his optimisation

problem, each player defines a strategy on the basis of a given information

structure. We will consider two different information structures, to which

two different equilibrium concepts are associated:

Definition 1: open-loop information In this case, each player’s informa-

tion set at any time t includes only calendar time, but neither the

history nor the current stocks of states. Hence, the optimal choice

of control uiis conditional on current time only, so that the optimal

open-loop strategy appears as u∗

i= u∗

i(t).

Definition 2: closed-loop memoryless information In this case, each

player’s information set at any time t consists of the state vector X(t).

Hence, the optimal closed-loop memoryless strategy is u∗

i= u∗

i(X(t)).

By ‘memoryless’, it is meant that the game history in itself is not rele-

vant for the choice of optimal behaviour at time t, only the consequences of

the history are important, as they are reflected in the current state vector.

However, taking into account the current state vector allows each player to

optimally react to the other players’ behaviour, because of the effect exerted

by the other players’ strategies through the vector X(t).

In the literature on differential games, one usually refers to the concepts

of weak and strong time consistency. The difference between these two prop-

erties can be outlined as follows:

Definition 3: weak time consistency Consider a game played over t =

[0,∞) and examine the trajectories of the state variables, denoted by

5

Page 8

x(t). The equilibrium is weakly time consistent if its truncated part in

the time interval t = [T,∞), with T ∈ (0,∞), represents an equilibrium

also for the subgame starting from t = T, given the vector of initial

conditions xT= x(T).

Definition 4: strong time consistency Consider a game played over t =

[0,∞). The equilibrium is strongly time consistent, if its truncated part

over t = [T,∞), with T ∈ (0,∞), is an equilibrium for the subgame,

independently of the conditions regarding state variables at time T,

x(T).

Strong time consistency requires the ability on the part of each player to

account for the rival’s behaviour at any point in time, i.e., it is, in general,

an attribute of closed-loop equilibria, and corresponds to subgame perfec-

tion. Weak time consistency is a milder requirement and does not ensure, in

general, that the resulting Nash equilibrium be subgame perfect.7

In the particular case where the feedback exerted by the other players’

strategies on player i’s optimal choice at any t during the game is nil, the

resulting optimal closed-loop strategy is u∗

i(t), i.e., it is no longer a func-

tion of X (t). As a result, the closed-loop memoryless strategy collapses into

the open-loop one. In the remainder of the section, we clarify under what

circumstances the resulting equilibrium u∗(t) ≡ (u∗

indeed strongly time consistent.

1(t),u∗

2(t),...u∗

N(t)) is

7For a more detailed analysis of these issues, see Dockner, Jørgensen, Long and Sorger

(2000, Section 4.3, pp. 98-107); see also Ba¸ sar and Olsder (1982, 19952, ch. 6).

6

Page 9

2.1Solution methods

The Hamiltonian of player i writes as follows:8

Hi(x(t),u(t)) ≡ e−ρt[πi(x(t),u(t)) + λii(t) · fi(x(t),u(t))+

X

where λij(t) = µij(t)eρtis the co-state variable (evaluated at time t) associ-

+

j6=i

λij(t) · fj(x(t),u(t))

#

,

(4)

ated with the state variable xj.

The first order condition on the control variable ui(t) is:

∂Hi(·)

∂ui(t)= 0

(5)

and the adjoint equations concerning the dynamics of state and co-state

variables are as follows:

−∂Hi(·)

∂xj(t)=∂λij(t)

∂t

− ρλij(t) ,∀j = 1,2...N

(6)

They have to be considered along with the initial conditions x(0) and the

transversality conditions, which set the final value (at t = ∞) of the state

and/or co-state variables:

lim

t→∞e−ρtλij(t)xj(t) = 0,∀j = 1,2...N.

(7)

From (5) one obtains the instantaneous best reply of player i, which can

be differentiated with respect to time to yield the kinematic equation of the

control variable ui(t). This, in combination with the state equations (1) and

the adjoint equations (6) permits to identify the open-loop Nash equilibrium

(or equilibria)9of the game; the optimal time path of the control variables will

8We assume that regularity and concavity conditions are met.

9We do not deal with the issue of unicity. The properties which we are about to derive

apply independently of the number of equilibria.

7

Page 10

depend on t only. Moreover, the simultaneous consideration of the dynamic

constraint lead to a dynamic system, whose properties can be easily studied.

Under the closed-loop memoryless information structure, the Hamiltonian

of player i is the same as in (4). The relevant first order conditions and the

adjoint equations are:

∂Hi(·)

∂ui(t)= 0 ;

(8)

−∂Hi(·)

∂xj(t)−

X

h6=j

∂Hi(·)

∂uh(t)

∂u∗

∂xj(t)=∂λij(t)

h(t)

∂t

− ρλij(t) ,∀j = 1,2...N;(9)

along with the initial conditions x(0) and the transversality conditions

lim

t→∞e−ρtλij(t)xj(t) = 0 , ∀j = 1,2...N.

(10)

The terms

∂Hi(·)

∂uh(t)

∂u∗

∂xj(t)

h(t)

(11)

appearing in the adjoint equations capture the strategic interaction through

the feedback from states to controls, which is by definition absent under the

open-loop solution concept. In equations (9) and (11), u∗

h(t) is the solution

to the first order condition of firm h w.r.t. her control variable. Whenever

the expression in (11) is zero for all h, then the closed-loop memoryless

equilibrium collapses into the open-loop Nash equilibrium, in the sense that

the time path of all relevant variables under the two different information

structures coincide. This can happen either because:

∂Hi(·)

∂uh(t)= 0 for all h 6= i,

(12)

which obtains if the Hamiltonian of player i is a function of his control

variable but not of the rivals’; or because:

∂u∗

∂xj(t)= 0 for all h 6= j ,

h(t)

(13)

8

Page 11

which means that the first order condition of a player with respect to his

control variable does not contain the state variables pertaining to any other

players. Of course, it could also be that (12) and (13) hold simultaneously.

In particular, ∂u∗

h(t)/∂xj(t) may depend upon the solution of the differen-

tial equations (9) yielding the expressions for the co-state variables. If the

solution of co-state equations:

λij(t) =

Z

∂λij(t)

∂t

dt

(14)

depends only upon time, then indeed (13) holds. Otherwise, if (14) depends

upon xh(t), then (13) does not hold. In the first case, the open-loop equilib-

rium is a degenerate closed-loop one, and both are strongly time consistent

(i.e., subgame perfect); in the second case, they are distinct, and the open-

loop equilibrium is only weakly time consistent.

The property whereby optimal controls are independent of states and

initial conditions is known as state-redundancy, and the game itself as state-

redundant or perfect (Fershtman, 1987; see also Mehlmann, 1988, ch. 4).

From a technical point of view, state-redundancy implies that the control

paths are uniquely determined by the co-state trajectories, being unaffected

by any feedbacks operating through X (t). In particular, (12) requires the

Hamiltonian of player i to be independent of player h’s behaviour, for all

h 6= i. Intuitively, this holds if and only if there is no strategic interaction at

all, i.e., in optimal control problems with a single agent. This of course is no

longer a game. On the other hand, (13) applies whenever the Hamiltonian

function of player i is additively separable w.r.t. player i’s control and player

j’s state, for all j 6= i. Intuitively, player i’s payoff is obviously affected by the

rivals’ states and controls; however, they do not affect his optimal behaviour

at any t, due to the property of additive separability. It is nevertheless

true that additive separability between one’s own control(s) and the rival’s

9

Page 12

states is sufficient but not necessary in order for the game to be state-

redundant.10

In the remainder of the analysis, we solve two alternative capital accu-

mulation games according to the closed-loop memoryless concept. In the

first case, where the model is studied in a partial equilibrium perspective, we

find that (11) does not hold and the open-loop equilibrium is only weakly

time consistent. In the second case, which is solved through a general equilib-

rium approach, (11) holds, so that the closed-loop equilibrium path coincides

with the open-loop equilibrium path. As a consequence, the open-loop Nash

equilibrium of the second game is not only weakly time consistent, but also

strongly so (or subgame perfect).

3 Examples

We consider two well known market models. In both models, the market

exists over t ∈ [0, ∞), and is served by N firms producing a homogeneous

good. Let qi(t) define the quantity sold by firm i at time t. The marginal

production cost is constant and equal to c for all firms. Firms compete ` a la

Cournot, the demand function at time t being:

p(t) = A − BQ(t), Q(t) ≡

N

X

i=1

qi(t) .

(15)

In order to produce, firms must accumulate capacity or physical capital ki(t)

over time. The two models we consider in the present paper are characterised

by two different kinematic equations for capital accumulation.

A] The Nerlove-Arrow (1962) or Solow (1956) - Swan (1956) setting, with

10Examples of state-redundant games where such separability does not hold are Leit-

mann and Schmitendorf (1978), Feichtinger (1983) and Cellini and Lambertini (2005).

10

Page 13

the relevant dynamic equation being:11

dki(t)

dt

= Ii(t) − δki(t),

(16)

where Ii(t) is the investment carried out by firm i at time t, and δ > 0

is the constant depreciation rate. The instantaneous cost of invest-

ment is Ci[Ii(t)] = b[Ii(t)]2/2, with b > 0. We also assume that firms

operate with a decreasing returns technology qi(t) = f(ki(t)), with

f0≡ ∂f(ki(t))/∂ki(t) > 0 and f00≡ ∂2f(ki(t))/∂ki(t)2< 0. The de-

mand function rewrites as:12

p(t) = A − B

N

X

i=1

f(ki(t)).

(17)

Here, the control variable is the instantaneous investment Ii(t), while

the state variable is obviously ki(t). Note that fund raising for financing

investment Ii(t) is not modelled; hence, this game is solved in partial

equilibrium, as the features of the financial market are left out of the

picture.

B] The Ramsey (1928) setting, whit the following dynamic equation:

dki(t)

dt

= f(ki(t)) − qi(t) − δki(t),

(18)

11Traditionally, differential equations as in (16) have been used to describe the dynamics

of market shares in advertising models, following the seminal contribution of Nerlove and

Arrow (1962). See Feichtinger, Hartl and Sethi (1994). However, the same dynamics also

describes capital accumulation in Solow (1956) and Swan (1956).

12Notice that the assumption qi(t) = f(ki(t)) entails that firms always operate at full ca-

pacity. This, in turn, amounts to saying that this model encompasses the case of Bertrand

behaviour under capacity constraints, as in Kreps and Scheinkman (1983), inter alia. The

open-loop solution of the Nerlove-Arrow differential game in a duopoly model is in Fersht-

man and Muller (1984) and Reynolds (1987). The latter author also derives the feedback

solution through Bellman’s value function approach.

11

Page 14

where f(ki(t)) = yi(t) denotes the output produced by firm i at time

t. Also in this setting, we assume f0≡ ∂f(ki(t))/∂ki(t) > 0 and

f00≡ ∂2f(ki(t))/∂ki(t)2< 0. In this case, capital accumulates as a re-

sult of intertemporal relocation of unsold output yi(t)−qi(t). This can

be interpreted in two ways. The first consists in viewing this setup as

a corn-corn model, where unsold output is reintroduced in the produc-

tion process. The second consists in thinking of a two-sector economy

where there exists an industry producing the capital input which can

be traded against the final good at a fixed price equal to one. In either

case, the Ramsey approach is based upon a general equilibrium analy-

sis. In this model, the control variable is qi(t), and the state variable

is ki(t).

3.1The Nerlove-Arrow or Solow-Swan model

When capital accumulates according to (16), the relevant Hamiltonian for

firm i is:

Hi(·) = e−ρt

("

A − Bf(ki(t)) − B

X

j6=i

f(kj(t)) − c

#

f(ki(t)) −b

)

2[Ii(t)]2+

+λii(t)[Ii(t) − δki(t)] +

X

j6=i

λij(t)[Ij(t) − δkj(t)]

.

(19)

Necessary conditions for the closed-loop memoryless equilibrium are:13

13Given that the Hamiltonian is concave, we focus on first order conditions alone. In

(20), the indication of exponential discounting is omitted for brevity.

12

Page 15

(i)∂Hi(·)

∂Ii(t)= 0 ⇒ −bIi(t) + λii(t) = 0 ⇒ I∗

(ii) −∂Hi(·)

⇒∂λii(t)

∂t

+f0(ki(t))

(ii0) −∂Hi(·)

i(t) = λii(t)/b

∂ki(t)−P

= (ρ + δ)λii(t) −P

∂kj(t)−P

j6=i

∂Hi(t)

∂Ij(t)

∂I∗

∂ki(t)=∂λii(t)

∂Hi(t)

∂Ij(t)

j(t)

∂t

∂I∗

∂ki(t)+

− ρλii(t) ⇒

j(t)

j6=i

h

2Bf(ki(t)) + BP

h6=j

j6=if(kj(t)) − (A − c)

∂Hi(t)

∂Ih(t)

i

∂I∗

∂kj(t)=∂λij(t)

h(t)

∂t

− ρλij(t) ,

(20)

with the transversality conditions:

lim

t→∞µij(t) · ki(t) = 0 for all i,j .

(21)

Condition (20-ii0), which yields ∂λij(t)/∂t, is redundant in that λij(t) does

not appear in the first order conditions (20-i) and (20-ii). Moreover, on the

basis of (20-i), one could be lead to think that ∂I∗

i(t)/∂ki(t) = 0 for all

i,j. Yet, this is not the case in view of the co-state equation (20-ii) yielding

∂λii(t)/∂t as a function of the whole vector of capacities of firm i as well

as the rivals’. Hence, integrating (20-ii) one obtains the expression of λii(t)

which depends on all kj(t). The ultimate consequence of this fact is that

the optimal control of firm i is not independent of the state variables of the

rivals.

This discussion is summarised by:

Proposition 1 The open-loop solution of the Nerlove-Arrow Solow-Swan

game is weakly time consistent.

Therefore, in order to obtain a subgame perfect equilibrium, one has to

solve the closed-loop game. This is done, for the duopoly case, by Reynolds

(1987).

13